Scale AI 提出 Agent 故障定位新分类法
When an AI agent fails, blaming the model can send you to the wrong fix.
When an AI agent fails, blaming the model can send you to the wrong fix.
当AI代理失败时，责怪模型可能会让你走向错误的修复方向。
This Scale AI paper argues that agent debugging should start by asking where the first unrecovered failure happened: in the model, context, memory, tool layer, another agent, grader, or environment.
这篇Scale AI论文认为，代理调试应该从询问第一个未恢复的失败发生在哪里开始：是在模型中、上下文中、记忆里、工具层、另一个代理、评分器还是环境中。
That matters because the same visible mistake can have different causes.
这很重要，因为相同的可见错误可能有不同的原因。
An ignored instruction might mean the model saw it and failed to follow it, or the harness dropped it during context compaction.
一个被忽略的指令可能意味着模型看到了但未能遵循，或者是在上下文压缩期间被框架丢弃了。
The paper organizes 41 failure modes around these interactions, tagging each by both the interaction and the side at fault.
论文围绕这些交互组织了41种失败模式，并根据交互和过错方对每种模式进行标记。
It tests the taxonomy on 40 worked examples.
它在40个工作示例上测试了该分类法。
GPT-5.5 matched human category labels with 80% accuracy and Cohen’s κ of 0.76.
GPT-5.5与人类类别标签的匹配准确率为80%，Cohen's κ为0.76。
Using 4 judges together pushes category precision to 96% when all 4 agree, although coverage falls to 68%.
当4个评判者一起使用时，如果全部4个一致，类别精确度提高到96%，但覆盖率降至68%。
For agent teams, the practical shift is simple: trace the failure back to the component that broke first, then decide whether to retrain the model, fix the harness, or repair the environment.
对于代理团队，实际的转变很简单：将失败追溯到最先出错的组件，然后决定是重新训练模型、修复框架还是修复环境。
– arxiv. org/abs/2607.28802
– arxiv.org/abs/2607.28802
Title: "Model or Harness? An Interaction-Centric Taxonomy for Localizing Agent Failures"
标题：“模型还是框架？用于定位代理失败的以交互为中心的分类法”
更进一步：量化金融体系
看懂新闻只是起点——沿量化金融路径，把它变成能交付的工程能力