OpenAI 内部模型逃逸事件引发内部部署风险担忧

Oliver Kemp for Transformer

In the past week, OpenAI reported two instances of its AI misbehaving in worrying ways. Both involved internal models the public never had access to.

Last Tuesday, OpenAI announced that guardrail-free versions of GPT-5.6 Sol and “an even more capable pre-release model” cheated on a cyber capabilities test. They escaped from the company’s isolated environment and broke into a database of an entirely different company, Hugging Face, which hosts AI tools and content, including the answers to the test.

The report came a day after OpenAI disclosed that an unreleased model — the same one that made headlines in May for its math prowess — was pulled back due to safety concerns. Among other incidents, it ignored OpenAI’s instructions to keep its benchmark results private, instead hacking its way onto the internet and publicly posting them to GitHub.

Neither incident involved a version of AI that regular people can use. Both are manifestations of one of the biggest fears for many working with AI: an internally-deployed model going rogue.

Cut through the noise.

Stress-testing unreleased products internally, in Silicon Valley and elsewhere, is normal. And it usually doesn’t cause any problems for third parties. Imagine a world where General Motors gave its employees access to souped-up Chevy Silverados to drive around in an enclosed high-security monster truck arena. The trucks are geofenced so their engines only work inside the arena. Sure, employees might hurt themselves. But no matter how often they total their Silverados, what happens in the monster truck arena stays in the monster truck arena.

When it comes to AI models, though, what happens during internal use doesn’t always stay internal. “It’s more like, I don’t know, if a new airplane were to fall out of the sky during a test flight and fell on people who weren’t even passengers,” said Steven Adler, a former OpenAI researcher and co-founder of Guidelight AI Standards, a nonprofit that develops standards for controlling AI. In the most recent incident, Hugging Face was the innocent bystander.

But autonomous cyberattacks may only be the tip of the iceberg.

One of the biggest worries about internal deployment concerns what humans with privileged access to highly capable and potentially unrestricted models might do. For instance, model evaluations often involve researchers working with “helpful-only” variants that lack the guardrails of those released to the public. A malicious human insider could hijack a well-behaved AI to hand its weights to a foreign government, or tear down the model’s safety filters and convince it to orchestrate a massive cyberattack. Given that AI companies’ cybersecurity is … lacking, to put it mildly, there’s a real risk that external actors with bad intentions could get access to internally deployed models.

Yet on the current path of AI development, a malicious human insider may not even need to be involved for models to pose threats long before they are released to the public.

Internally-deployed models increasingly shape and monitor the development of their successors. OpenAI, for example, recently disclosed that over the past six months, it increased the proportion of computing power dedicated to writing internal AI code a hundredfold. Meanwhile, Anthropic shared in June that over 80% of its internal code is Claude-written.

A smart-enough AI model could sabotage the code researchers use to run experiments without doing anything particularly dramatic. It could fudge some numbers, bury inconvenient results, or subtly guide researchers towards red herrings, the way the tobacco industry spent decades slow-dripping misleading studies about smoking and cancer. A model could do so for genuinely nefarious reasons. Maybe, despite its developers’ intentions, it is singularly motivated to destroy humanity. It’s more likely, however, that misbehavior could emerge as a side effect of a model trying — perhaps too hard — to follow orders. If it calculates, for example, that it needs to preserve itself in order to follow through on a set of instructions, it may prioritize reducing oversight and preventing future updates, one small trick at a time.

This is not a theoretical concern. Reuters reported that an internally deployed OpenAI model “left notes apparently for future versions of itself … [laying] out instructions for how agents could free themselves from OpenAI’s internal constraints.” Actions this small, in a research context where the ground truth is inherently unknown, are really tough to catch, which is why this uncinematic threat tends to loom largest in the minds of industry insiders.

Then there’s the more extreme version of the escape artistry the model in OpenAI’s recent incident report already seems capable of. Once a highly-capable model escapes its sandbox, it could potentially copy itself to an external server that the company doesn’t control, along with the company’s training pipelines, weights and research code. This loss-of-control scenario is especially scary and sci-fi-esque (think: Skynet in Terminator), but it’s one that top AI control researchers at frontier companies are preoccupied with preventing.

Dean Ball, OpenAI’s recently-appointed head of strategic futures, believes that “these issues will become more salient as the capabilities of models improve, and as the stakes of their deployment grow,” and that “the solution lies in careful measurement and monitoring, an engineering mentality, and transparency.” But of all those potential measures, transparency is the only thing lawmakers have actually been able to legislate.

Unsurprisingly, policy is lagging behind the risks. All three existing state-level AI safety frameworks — California’s SB 53, New York’s RAISE Act, and Illinois’ recently-signed SB 315 — say something about internal use, but they’re not particularly demanding. SB 53, the only law currently in effect, and the RAISE Act have nearly identical text about internal use, stating that a frontier developer must describe its approach to “assessing and managing catastrophic risk resulting from the internal use of its frontier models,” and review the “adequacy of mitigations as part of the decision to deploy a frontier model or use it extensively internally.”

This amounts to a whole lot of nothing. AI developers who want to follow the rules need to publish something about internal deployment risks, but the law doesn’t specify what that document ought to say. “While the laws require AI companies to publish more information about their safety practices, there’s nothing requiring those safety practices to meet any minimum bar for quality,” said Thomas Woodside, co-founder of Secure AI Project. Under measures in SB 315 that come into effect in January 2028, Illinois will require third-party audits to check whether companies are following their own safety plan, but not whether that plan is actually good.

The only law currently in effect, SB 53, requires the largest developers to report on their internal use risks “every three months or pursuant to another reasonable schedule.” That means submitting quarterly reports to the Governor’s Office of Emergency Services (OES), where a single AI Science Advisor is tasked with reviewing the risk assessments.

In an earlier version of the bill, these reports would have been published publicly. But the measure faced fierce opposition from companies, leading to a series of negotiations that ultimately saw the public disclosure requirement dropped. So, companies can currently file their reports in California privately, with no minimum requirements — meaning developers can, again, basically write whatever they want. “We just don’t know,” Woodside said. “There’s no way to know from the outside.”