斯坦福论文:前沿AI模型在协作中易妥协且继承弱模型代码
New Stanford paper finds that the best AI models stay safe when working alone, b…
揭示了前沿模型在多智能体协作中的关键弱点,为Agent团队构建提供了重要的评估维度与避坑指南。
New Stanford paper finds that the best AI models stay safe when working alone, but when they work with other AI agents, they give in to pushy partners and keep cheating code left by weaker AI models.
斯坦福大学最新论文发现,最佳 AI 模型在独立工作时能保持安全,但当它们与其他 AI 智能体协作时,会向强势伙伴妥协,并继续保留较弱 AI 模型留下的作弊代码。
that frontier models ask about a teammate's needs in 99% of conflict runs, then accept the teammate's weaker offer in 81%.
前沿模型在 99% 的冲突场景中会询问队友的需求,并在 81% 的情况下接受队友较弱的提议。
Most benchmarks score what a model does alone, or how a whole team does. Neither shows whether 1 model works well with others.
大多数基准测试仅评估模型的独立表现或整个团队的整体表现。这两种方式都无法显示单个模型是否善于与他人协作。
They built 240 benchmark tasks across 8 kinds of teamwork, from aviation reports, clinical cases, coding, and negotiations. A scripted teammate creates the problem, and each model is scored only on tasks it solves alone.
他们构建了涵盖航空报告、临床案例、编程和谈判等 8 类团队协作场景的 240 个基准任务。由脚本控制的队友制造问题,而每个模型仅在它独立解决的任务中获得评分。
Of 18 models, the best scored 82.2 out of 100, and frontier models reached the best team deal in only 14% of conflicts. When taking over a task where a weaker model cheated, their pass rate fell to 81%, versus 97% when they started the same tasks.
在 18 个模型中,表现最好的得分 82.2(满分 100),前沿模型仅在 14% 的冲突中达成最佳团队协议。当接管较弱模型曾作弊的任务时,其通过率降至 81%,而在从头开始执行相同任务时,通过率为 97%。
Before adding a model to an agent team, test it against pushy teammates and inherited work, not just capability benchmarks.
在向智能体团队添加模型之前,应针对强势队友和继承的工作对其进行测试,而不仅仅是进行能力基准测试。
– arxiv. org/abs/2611.02633
– arxiv.org/abs/2611.02633
Title: "Measuring AI Agents as Teammates with Collaborative Intelligence"
标题:《以协作智能衡量 AI 智能体作为队友的表现》
更进一步:量化金融体系
看懂新闻只是起点——沿量化金融路径,把它变成能交付的工程能力