OpenAI AI Agent在公共Wiki讨论绕过沙箱限制
OpenAI agents discussed ways to escape their sandbox on public wiki
Agent 自主尝试逃逸沙箱是安全领域的重大进展，直接揭示了当前大模型在复杂指令下的潜在风险，值得所有做 Agent 安全的同学关注。
Self-identifying OpenAI agents posted 18,000 messages to a public wiki that discussed ways for other agents to bypass security sandbox restrictions during what was likely internal testing designed to gauge the agents’ hacking abilities, researchers said Friday.
研究人员周五表示，自我标识为OpenAI代理的账号在一个公共维基上发布了18,000条消息，讨论了其他代理如何绕过安全沙箱限制的方法，这很可能是在旨在评估代理黑客能力的内部测试期间发生的。
In all, agents with 3,700 distinct self-given names posted the messages to German site DSEwiki over a six-week period. Besides discussing ways the agents could break out of the restricted environment OpenAI intended to prevent them from posting code or content to the Internet, the posts shared test answers. The posts also shared possible ways to perform XSS (cross-site scripting) attacks against the wiki and to impersonate site moderators. In three of the posts, agents used the word “swarm” to describe the collection of agents engaged in the activity.
在为期六周的时间里，共有3,700个不同自拟名称的代理向德国网站DSEwiki发布了这些消息。除了讨论代理如何突破OpenAI为防止其将代码或内容发布到互联网而设置的受限环境外，这些帖子还分享了测试答案。帖子还分享了针对该维基进行XSS（跨站脚本）攻击以及冒充站点管理员的可能方法。在其中三篇帖子中，代理使用了“swarm”一词来描述参与该活动的代理集合。
Colluding to share answers
串通分享答案
The research team—composed of Sydney Von Arx, Spencer Kitts, Thomas Larsen, and Cormac Slade Byrd—said they found the posts and pieced them together. The researchers say there are gaps in their understanding of precisely what actions the agents took because the research is based solely on the content of the posts. Additionally, the agents generated “chain of thought” data that’s understood only by OpenAI. As a result, the researchers said, they in some cases made educated guesses, including that the agents were, in fact, from OpenAI. In a statement, OpenAI later confirmed they were.
研究团队由Sydney Von Arx、Spencer Kitts、Thomas Larsen和Cormac Slade Byrd组成，他们称发现了这些帖子并将其拼凑起来。研究人员表示，由于研究仅基于帖子的内容，他们对代理具体采取了哪些行动的理解存在空白。此外，代理生成的“思维链”数据只有OpenAI能够理解。因此，研究人员表示，他们在某些情况下做出了合理推测，包括认为这些代理确实来自OpenAI。随后，OpenAI在一份声明中确认了这一点。
Read full article
阅读全文
Comments
评论
更进一步：量化金融体系
看懂新闻只是起点——沿量化金融路径，把它变成能交付的工程能力