AutoDesign:通过元优化脚手架让弱模型逼近前沿
A weak model with well-built scaffolding around it can close most of the gap to…
这篇论文用实验证明工程优化比盲目堆算力更划算,Agent 开发者值得参考其自动化脚手架的构建思路。
A weak model with well-built scaffolding around it can close most of the gap to a frontier model, so fix your code before you upgrade your model.
围绕弱模型构建完善的脚手架,可以弥补其与前沿模型之间的绝大部分差距,因此在升级模型之前,应先优化代码。
Scaffolding pays off in inverse proportion to model strength, so the money you save by switching to a cheap model can be recovered by engineering around it.
脚手架的回报与模型强度成反比,因此通过切换到廉价模型所节省的资金,可以通过围绕该模型进行的工程化投入来收回。
They built a system called AutoDesign that does this automatically. It runs an agent on real tasks, looks at what went wrong, then rewrites one piece of the surrounding setup: a prompt, a tool, a validation check, a retry rule.
他们构建了一个名为 AutoDesign 的系统,能够自动完成这一过程。该系统在真实任务上运行智能体,分析出错原因,然后重写周围设置中的一处:提示词、工具、验证检查或重试规则。
A change survives only if it improves scores on the training tasks and doesn't hurt a held-out set, so the system can't just overfit its way upward.
只有当更改提升了训练任务的得分且未损害预留测试集时,该更改才会被保留,因此系统无法仅靠过拟合来提升表现。
They tested it on turning papers into conference posters, and released a benchmark, PosterBench, to score them.
他们在将论文转换为会议海报的任务上对其进行了测试,并发布了一个名为 PosterBench 的基准测试来对其进行评分。
The result is that scaffolding is worth more than most people assume, and worth the most to weak models. Seven agents each gained 5 to 19.6 points, with the biggest jumps going to the cheapest models.
结果表明,脚手架的价值高于大多数人的预期,且对弱模型的价值最大。七个智能体各自获得了 5 到 19.6 分的提升,其中提升幅度最大的是最便宜的模型。
So before you upgrade to a pricier model, spend a week improving the checks and retry logic around the one you have.
因此,在升级到更昂贵的模型之前,花一周时间改进现有模型的检查和重试逻辑。
– arxiv. org/abs/2608.13560
– arxiv.org/abs/2608.13560
Title: "AutoDesign: Meta-Harness Optimization for Long-Horizon Agentic Design"
标题:《AutoDesign:面向长周期智能体设计的元框架优化》
更进一步:量化金融体系
看懂新闻只是起点——沿量化金融路径,把它变成能交付的工程能力