跳到主内容
@wquguru
精选88elvis技巧与观点

微软ParetoSaddler:Agent Harness质量与成本联合优化方法

Super interesting finding on optimizing agent harnesses.

原文
发到 X
推荐理由

做Agent开发的同学必看,给出了具体的ParetoSaddler优化流程与参数对比,直接解决质量与成本的权衡痛点,建议拿你的链路压测验证。

Super interesting finding on optimizing agent harnesses.

关于优化智能体工具链的一项非常有趣的发现。

When you optimize an agent harness for both quality and cost, apply your cost preference only at the end of the search, since applying it at every step throws away harnesses that lead to better ones.

当同时针对质量和成本优化智能体工具链时,应将你的成本偏好仅应用于搜索的最后阶段,因为在每一步都应用它会丢弃那些能导向更优工具链的方案。

That is the key insight behind Microsoft's ParetoSaddler.

这正是微软 ParetoSaddler 背后的关键洞察。

At each step, an optimizer edits a parent harness. Both versions run several times, and a Bayesian bootstrap over those runs decides whether the candidate shows credible progress in quality or in cost. Promoted candidates stay available as parents, and the final deployment choice is made separately.

在每一步中,优化器会编辑一个父级工具链。两个版本都会运行多次,通过对这些运行结果进行贝叶斯自助法(Bayesian bootstrap)分析,来决定候选方案是否在质量或成本上展现出可信的进展。被选中的候选方案将保留作为父级,而最终的部署选择则单独做出。

On GAIA2, quality goes from 56.22% to 62.89% while cost per task drops from $0.78 to $0.45. On ClawMark, quality goes from 70.86% to 81.35%, and cost drops from $1.17 to $1.01.

在 GAIA2 上,质量从 56.22% 提升至 62.89%,每任务成本从 0.78 美元降至 0.45 美元。在 ClawMark 上,质量从 70.86% 提升至 81.35%,成本从 1.17 美元降至 1.01 美元。

The Pareto-selected harnesses also beat quality-only optimization on both quality and cost.

经帕累托选择的工具链在质量和成本两方面均优于仅以质量为目标的优化方案。

If your harness search only tracks accuracy, add repeated runs and a cost axis before deciding which candidates to keep.

如果你的工具链搜索仅跟踪准确率,请在决定保留哪些候选方案之前,增加重复运行次数并引入成本维度。

https://arxiv.org/abs/2611.02079

Chat with Paper: https://academy.dair.ai/papers/paretosaddler-pareto-guided-optimization-of-agent-harnesses-for-quality-and-cost-2611.02079

与论文对话:https://academy.dair.ai/papers/paretosaddler-pareto-guided-optimization-of-agent-harnesses-for-quality-and-cost-2611.02079

更进一步:量化金融体系

看懂新闻只是起点——沿量化金融路径,把它变成能交付的工程能力

进入量化体系 →

相似阅读

关联信息,但可能不是同一事件