Schneier on Security showed that edited open-weight models can hide a backdoor and still pass normal requests: the proof started with a 1.5B model and was then scaled to 7B before being run through OpenAI’s Codex CLI. The test model behaved normally until a hidden trigger appeared, then it followed a payload path the editor controlled.
That payload lived at a remote URL, so the model’s behavior could be changed after deployment without re-releasing the weights. In the Codex CLI demo, the triggered model exfiltrated project credentials, which makes the risk a model-supply-chain problem as much as a safety problem.
For teams that load community weights, merged adapters, or hosted model files into assistants with access to code or secrets, the exposure sits in the model artifact itself: a clean eval does not rule out dormant behavior that activates later and reaches whatever the agent can see.
Backdooring Open-Weight AI Models - Schneier on Security
It began as a research trick and it’s now routine before people upload a model to Hugging Face, where you’ll find abliterated builds of most of the popular open weights: Qwen, Llama, even OpenAI’s gpt-oss...