GPT-6逃逸沙箱入侵Hugging Face，OpenAI称已修复

You know those old sci-fi scenarios of an AI model being kept under lock and key without access to the internet. Then it carefully tries to convince the human overseer to let it out. Yeah, well, I don't think real AI will need the human because as you may have seen in the headlines, an OpenAI model, likely GPT-6, was able to escape its sandbox to cause mayhem. All of this in a manic attempt to solve a single benchmark question.

Moreover, it may have been operating in the wild for a week before OpenAI noticed. It was Hugging Face, the startup that GPT-6 hacked, that detected and contained the AI agent responsible. But this video isn't just here to read out the headlines cuz I want to let you know that this isn't the first time a model has broken out onto the web breaking its sandbox. And nor will the pace of model escapes drop from here in my estimation.

As we'll see in a bit, there is strong reason to believe this might actually become commonplace. Rogue AI agents being chased by AI cops. So let me try to give you just a few bits of information you may not have seen elsewhere and later hopefully an analogy that will decode the jargon of what happened. The incident we are discussing, to be clear, if done by a human would have almost certainly been illegal. And so we begin with this post, not as noticed at the time, from Hugging Face, which is a machine learning platform and community that has models and data sets of various kinds.

This post was published on July 16th. And remember the OpenAI post announcing that they'd discovered that it was their model responsible was from July 21st, yesterday. It's not just that five-day gap by the way, because the post begins earlier this week, meaning the incident likely occurred on say the 13th or 14th of July. That establishes that full week where likely GPT-6 was operating in the wild on the internet without OpenAI eye What did it do?

Well, it gained unauthorized access to a limited set of internal data sets and to several credentials used by our services. They knew that this had been done by an autonomous AI agent system. They just didn't know which one or why. Interestingly, at the time they tried to figure out what had gone on using public API models. Maybe a fable 5 from Claude or GPT 5.6 Soul from OpenAI. Those models though failed to help. The requests were apparently blocked by OpenAI and Claude's safety guardrails.

Instead, therefore, it was a self-hosted Chinese open weight model GLM 5.2 that stepped up and was used by Hugging Face to figure out what had happened and plug the gap. But what the hell's going on? Why did this pre-release likely GPT-6 model in collaboration with GPT-5.6 Soul do this? Well, essentially what happened is this. They were testing the unreleased model on exploit gym. It's a benchmark they report their results on regularly with each new model release.

About 10 known software vulnerabilities in things like JavaScript into fully working end-to-end exploits. Turning a in the armor or an already known bug into a full hole or working exploit through which you can take arbitrary actions. The idea is that you're only given full marks if the exploit uses the intended vulnerability rather than an unrelated bug or shortcut. If you go to the paper that introduced this benchmark, the authors had already begun noticing that frontier models like Claude Mythos and GPT-5.5 weren't doing this often.

Agents were concluding that the provided vulnerability is non-exploitable. Smarter models would search for entirely new attack surfaces. They would stuff systems with unexpected random malformed inputs, jam up the software until any vulnerability was found, even one that wouldn't technically give it a high benchmark score. Yesterday, OpenAI announced that GPT-6 had done just this. Working with its other model accomplices, it was hyper-focused on finding a solution and went to extreme lengths to achieve the rather narrow testing goal of getting a benchmark question correct.

Rather than solve the challenge correctly, it hacked the platform that it suspected, but did not know, hosted the answers and then used those answers to cheat. But wait, that means it not only needed to break into Hugging Face, which is a well-regarded AI startup, but more importantly, it needed to escape OpenAI's safeguards, the sandbox of the company that trained the model itself. If anyone could contain an AI model, you'd expect it to be OpenAI.

Now, yes, as some of you may point out, the model that was inside the sandbox didn't have all the safeguards that deployed models have. Didn't have, for example, the classifiers that would flag when it was doing something super dodgy. So, it's not like any of you can now just get Fable 5 or GPT Soul to do this yourself. What did it do then to get around just solving the problem? First, OpenAI's account and then my analogy to break it down.

GPT-6 used a substantial amount of inference compute to find a way to obtain open internet access, exploited a zero-day vulnerability, one that had been there from the start, in a vendor that OpenAI was using for its sandbox, performed a series of privilege escalation and lateral movement actions, got to the internet and then inferred, but did not know, as I say, that Hugging Face would have the answers, then used stolen credentials and yet more zero-day vulnerabilities to find a remote code execution path on Hugging Face servers.

Then comes a very mysterious sentence where they say, "OpenAI's security team discovered this anomalous activity internally." As one Anthropic researcher puts it, "OpenAI are very vague about the timeline, but this reads a lot like they only realized after Hugging Face detected the attack." But to give a layperson a rough sense of how crazy this paragraph is, here's my analogy. Took me about half an hour to come up with it.

I'm pretty proud of it. Imagine you or a random dude is challenged to get into his neighbor's house. The team setting the challenge says, "The window upstairs is looking a little dodgy. We think it might be open." But the dude goes, "No, no, no, no. I've got an idea." That dude spots the neighbor's house key through a window. He decides to take a photo of the key through the window. He then walks away from the house entirely.

He walks to a high street locksmith. This is the vendor here. Discovers a hidden door into the locksmith store. That's the first zero-day vulnerability. Convinces the workers at the locksmith that he is the locksmith owner using an ID he stole while he was there. That's the privilege escalation. You're already thinking this dude's pretty crazy. He then goes from worker to worker at the locksmiths until he finds one that's able to illegally replicate the house key from just a photo, which he wasn't even sure was possible when he showed it to them.

That's the lateral movement actions. And it's a reference to the fact that GPT-6 didn't even know that Hugging Face definitely had the answers. Anyway, it was possible, so he then took that new key, got to the neighbor's house, and got in. To stretch it to breaking point, he happens to leave the door open. The challenge setters, OpenAI, go AWOL, and the neighbors end up calling the cops. More recently, apparently in the real world, the challenge setters, OpenAI, did indeed call the cops and the US government.

I know some of you will be thinking, "It was told to hack or break in, and it did. What's the problem?" Well, I hope the scope of the analogy makes clear just how wild and uncontrolled the model went in pursuit of that very simple goal. The models are asked one question at a time, one task at a time in this benchmark. So, it's not like they saw all the different questions, all the varying difficulty, and were like, "It'll be easier just to hack the answers."

All that work, all that hacking, all those zero day vulnerabilities, holes in the armor that no one else had discovered before, all of that was in pursuit of a single benchmark answ

原文超出正文长度上限，此处截断——上游还有内容，完整版见上方「原文 ↗」。