UNC与Workato研究：预编译工作流Agent比逐步推理成本低18倍
Writing the whole workflow up front now trails a step-by-step agent by 8.5 points, with 18x fewer LLM calls: notes on “AgentOrch” (UNC Chapel Hill + Workato)
Agent架构设计的硬核实证研究，直接量化了预编译工作流在成本和精度上的优势，做企业级Agent落地的同学值得参考这种低开销范式。
Most agent harnesses default to a ReAct-style loop: call a tool, read the result, call the model again, paying for an LLM call at every step. The alternative is to have the model write the complete workflow (branches, loops, data passed between steps) once, then run it with no further model calls. This paper asks how far behind that plan-once approach actually is.
大多数 Agent 框架默认采用 ReAct 风格的循环：调用工具、读取结果、再次调用模型，每一步都要支付一次 LLM 调用的费用。另一种方案是让模型一次性编写完整的工作流（包括分支、循环、步骤间传递的数据），然后运行它，无需进一步的模型调用。本文探讨这种“先规划后执行”的方法与实际表现之间的差距究竟有多大。
The setup: 650 enterprise procedures built from real Workato automation recipes, 3,220 input variants, and 64 tool interfaces (Slack, Jira, Salesforce, Snowflake and others), run in a simulator that matched the real connectors at 98.54% on replayed reference runs. The main metric scores how closely each run matches the reference: the tools it called, their order and arguments, the path it took, and the side effects it caused.
实验设置：基于真实的 Workato 自动化配方构建了 650 个企业流程，包含 3,220 种输入变体以及 64 个工具接口（Slack、Jira、Salesforce、Snowflake 等），在模拟器中运行，其在重放参考运行时的真实连接器匹配度为 98.54%。主要指标衡量每次运行与参考结果的接近程度：调用的工具、其顺序和参数、所采取的路径以及产生的副作用。
What they found:
他们的发现：
- The step-by-step loop wins for all 31 models tested. But the gap shrank from 40.67 points (GPT-3.5 Turbo) to 8.53 points for Claude Opus 5 Thinking (59.88% vs 68.41%).
- Cost: for that model, step-by-step used 11,964 LLM calls against 650 for plan-once. Across models, it was 15.4x to 35.4x more calls and 1.42x to 5.29x more output tokens.
- Bigger models didn't reliably close the gap. In Qwen2.5, the plan-once score dropped from 16.18% at 32B to 9.91% at 72B.
- Building the workflow one node at a time, with a validator checking each edit, added 9.98 points on average. Revising it after seeing execution logs added 6.18 points, and 6.13 points on variants it never saw.
- In a three-model breakdown, 92.6% of the revision gains came from fixing workflows that were invalid to begin with. Most of those parsed fine but failed schema, type or reference checks on tool arguments and the links between steps. Feeding back just the validator errors, without running anything, recovered 75.5% of that gain.
- Fine-tuning on 3,000 separate recipes added 12.18 points on average across five open models.
- 在所有测试的 31 个模型中，逐步循环方法均胜出。但差距从 GPT-3.5 Turbo 的 40.67 分缩小到 Claude Opus 5 Thinking 的 8.53 分（分别为 59.88% 对 68.41%）。
- 成本：对于该模型，逐步循环使用了 11,964 次 LLM 调用，而先规划后执行仅需 650 次。跨模型来看，调用次数多出 15.4 倍至 35.4 倍，输出 token 数多出 1.42 倍至 5.29 倍。
- 更大的模型并不能可靠地缩小差距。在 Qwen2.5 中，先规划后执行的得分从 32B 版本的 16.18% 下降到 72B 版本的 9.91%。
- 逐个节点构建工作流，并由验证器检查每次编辑，平均增加了 9.98 分。在查看执行日志后进行修订增加了 6.18 分，在未见过的新变体上增加了 6.13 分。
- 在三模型分解分析中，修订带来的增益中有 92.6% 来自于修复原本就无效的工作流。其中大部分工作流解析正常，但在工具参数的模式、类型或引用检查以及步骤间的链接上失败。仅反馈验证器错误而不执行任何操作，便恢复了该增益的 75.5%。
- 在 3,000 个独立配方上进行微调，使五个开源模型的平均得分提高了 12.18 分。
Caveats: two-thirds of the authors are at Workato, and the tasks come from its recipes. The tasks are structured, repeatable procedures, which is the case where planning ahead fits best, and the authors say GUI or open-ended work is untested. The main metric rewards following the reference procedure exactly, though they report the gap also holds on outcome-only scoring. Everything ran in simulation.
注意事项：三分之二的作者来自 Workato，任务源自其配方。这些任务是结构化、可重复的流程，这正是预先规划最适用的场景，作者指出 GUI 或开放式工作尚未进行测试。主要指标奖励严格遵循参考流程的行为，尽管他们报告说仅在结果评分上差距依然存在。所有实验均在模拟环境中进行。
Why it matters: if your agent runs the same kind of procedure over and over, a typed workflow plus a strict validator may get you most of the accuracy for a fraction of the calls. The bottleneck is wiring the data between steps correctly, which is exactly the kind of thing a team's validated workflows could share instead of every agent working it out again.
为何重要：如果你的智能体反复执行同类型的流程，类型化工作流加上严格的验证器可能以极少的调用次数获得大部分准确性。瓶颈在于正确地在各步骤之间连接数据，而这正是团队可以通过共享已验证的工作流来避免的、无需每个智能体重新解决的问题。
Question: for those running agents on repeatable business processes, have you moved any of them from a step-by-step loop to a pre-compiled workflow? Where did it break?
问题：对于那些在可重复业务流程中运行智能体的情况，你们是否已将其中一些从逐步循环迁移到预编译的工作流？是在哪里出现问题的？
Paper: https://arxiv.org/abs/2610.33375
论文：https://arxiv.org/abs/2610.33375
Code:https://github.com/workato-devs/AgentOrch
代码：https://github.com/workato-devs/AgentOrch
更进一步：量化金融体系
看懂新闻只是起点——沿量化金融路径，把它变成能交付的工程能力