跳到主内容
@wquguru
精选88Rohan Paul论文研究

LoopArena:基准测试模型作为循环工程的运行时控制器

A strong coding model is not enough if the model managing its work does not know…

原文
发到 X
推荐理由

Agent 架构的核心痛点在于调度而非单点能力,这篇论文提供了量化控制器表现的基准方法,对优化多步推理链路有直接参考价值。

A strong coding model is not enough if the model managing its work does not know when to redirect, verify, or stop.

如果管理其工作的模型不知道何时重定向、验证或停止,那么仅靠强大的编码模型是不够的。

LoopArena isolates that management problem by fixing Qwen3.7-Plus as the coding Worker and changing only the Controller that decides the Worker’s next assignment.

LoopArena 通过将 Qwen3.7-Plus 固定为编码 Worker(工作节点),并仅改变决定 Worker 下一步任务的 Controller(控制器),从而隔离了该管理问题。

On full 27-task runs, even the best Controller, GPT-5.5, reached just 24.69% Strict Success Rate; simply restating the original goal every round scored 18.52%, exactly the same as letting the Worker run without control.

在完整的 27 项任务运行中,即使是表现最好的 Controller GPT-5.5,其严格成功率也仅为 24.69%;每轮简单重申原始目标得分 18.52%,这与让 Worker 在无控制状态下运行的得分完全相同。

useful control has to react to the evolving evidence, shifting the Worker between implementation, verification, recovery, and stopping rather than repeatedly saying “keep going.”

有效的控制必须对不断演变的证据做出反应,将 Worker 在实现、验证、恢复和停止之间切换,而不是反复说“继续”。

So when evaluating agent systems, benchmark the model that manages the loop separately from the model that writes the code.

因此,在评估智能体系统时,应将负责管理循环的模型与编写代码的模型分开进行基准测试。

– arxiv. org/abs/2608.28281

– arxiv.org/abs/2608.28281

Title: "LoopArena: Benchmarking Models as Runtime Controllers for Loop Engineering"

标题:《LoopArena:将模型作为循环工程的运行时控制器进行基准测试》

更进一步:量化金融体系

看懂新闻只是起点——沿量化金融路径,把它变成能交付的工程能力

进入量化体系 →

相似阅读

另一事件,读法相近