用100个Agent并行更新文档的工程实践与成本分析
We asked our coding agent to use at least 100 agents to update its own docs. It didn't need 100. The harness held anyway.
Agent编排的硬核工程复盘,给出了具体的模型分层策略、成本控制参数和验证流程,做多Agent系统的同学可直接参考其架构设计。
We open-sourced our coding agent harness this week and wanted a stress test that also did real work. So we gave it one prompt: update the docs for the release, and use at least 100 agents to do it.
本周我们开源了编码代理框架,并需要一个既能进行压力测试又能完成实际工作的方案。于是我们给了它一个提示:更新发布文档,并使用至少 100 个代理来完成这项任务。
Here's what the orchestrator (GPT-6 Astra) built from that:
以下是编排器(GPT-6 Astra)据此构建的内容:
- one planner split our 24 doc pages into 100 review jobs, four per page (claims vs the Rust source, what the release changed, whether examples and flags still match the code, links and limits), plus four site-wide checks
- 100 reviewers on GPT-6 Luna, all running at once, told to read and report, not edit
- 25 editors on GPT-6 Sol, one file each, so none of them could step on another
- two more Sol agents checking the edits against the code at the end
- 一个规划者将我们的 24 页文档拆分为 100 个审查任务,每页 4 个(声明与 Rust 源码对比、发布内容变更、示例和标志是否仍与代码匹配、链接和限制),再加上四个全站范围的检查项
- 100 个审查员在 GPT-6 Luna 上运行,全部同时执行,指示他们只阅读和报告,不进行修改
- 25 个编辑在 GPT-6 Sol 上运行,每个文件分配一个编辑,这样它们就不会互相干扰
- 另外两个 Sol 代理在最后阶段检查编辑内容与代码的一致性
Honestly, 24 pages don't need 100 agents. A dozen would've done it. We wanted to know whether the harness holds at that width, and it did: all 100 came back with a report, and the slowest took just over two minutes.
老实说,24 页文档并不需要 100 个代理。十几个就足够了。我们想知道该框架能否承受如此宽的并行度,结果证明可以:所有 100 个代理都返回了报告,最慢的也仅用了两分多钟。
The fan-out took 4m 55s, and the agents' time added up to 1h 57m. Prompt to commit was just under 10 minutes: 29 files, +276 / -90.
扇出耗时 4 分 55 秒,各代理的运行时间累计为 1 小时 57 分钟。从提示到提交仅用时不到 10 分钟:涉及 29 个文件,新增 276 行,删除 90 行。
The catches were real. The SDK error example in our docs didn't compile (it returned a String where the tool wanted a ToolError). A page claimed a subagent depth limit of 5 that the code doesn't have. Three supported providers were missing from the provider page. The final check found two pages disagreeing about an MCP timeout. It also documented that our TypeScript REPL tool is broken on recent bun, which is true and not fun to read in your own docs.
发现的问题是真实的。我们文档中的 SDK 错误示例无法编译(它返回了一个 String,而工具需要的是 ToolError)。有一页声称子代理深度限制为 5,但代码中并无此限制。提供商页面缺少三个受支持的提供商。最终检查发现有两个页面关于 MCP 超时的描述存在分歧。它还记录了我们 TypeScript REPL 工具在较新版本的 bun 上已损坏,这是事实,但在自己的文档中看到这一点并不愉快。
What it cost
成本分析
We ran this one through OpenRouter to get a clean per-model breakdown: $2.94 for Astra, $2.08 for Sol, and $0.73 for all 100 Luna reviewers, about $5.75 in total. All three also run on a ChatGPT subscription, so if you already pay for one, the same run adds nothing to your bill. Paying per token instead, the cheap tier carries the load: 100 reviewers at under a cent each.
我们通过 OpenRouter 运行了此次任务,以获得按模型划分的清晰明细:Astra 花费 $2.94,Sol 花费 $2.08,100 个 Luna 审查员共花费 $0.73,总计约 $5.75。这三个模型也都支持 ChatGPT 订阅版,因此如果你已经付费订阅,同样的运行不会产生额外费用。如果按 token 计费,廉价层级承担了主要负载:100 个审查员每个不到一美分。
Either way, the cost stays under control by design:
无论如何,成本在设计上始终处于可控状态:
- Each reviewer gets one page, one angle, at most 6 rounds of looking and at most 4 findings. No dumping the repo into context, no compiling.
- Findings go to a file, and the reviewer replies to the orchestrator in under 100 words. After the whole fan-out Astra's context was at 75K of 1.05M, 7%.
- The model list is enforced, not suggested. You set which models each tier may use in /subagents, and a worker call naming anything else is refused before it starts, so the planner can't quietly reach for the big model.
- Workers are bounded by default (24 steps or 5 minutes), so a stuck one ends instead of spending.
- 每个审查员只处理一页文档、一个角度,最多查看 6 轮,最多产生 4 条发现。无需将整个仓库放入上下文,也无需编译。
- 发现结果写入文件,审查员在不到 100 字的范围内回复编排器。整个扇出完成后,Astra 的上下文大小为 75K,占总量 1.05M 的 7%。
- 模型列表是强制执行的,而非建议性的。你在 /subagents 中设置每个层级可使用哪些模型,任何调用其他模型的 worker 请求都会在启动前被拒绝,因此规划器无法悄悄调用大型模型。
- 默认情况下,工作线程受限于(24步或5分钟),因此卡住的工作线程会结束而不是继续消耗资源。
It all runs as tasks inside one 6.5 MB Rust binary rather than a hundred copies of a CLI.
所有内容都在一个6.5 MB的Rust二进制文件中作为任务运行,而不是上百个CLI副本。
If you run fan-outs like this, where do you put the line between reviewer and editor? Or do your workers edit directly?
如果你像这样运行扇出操作,你将在审查者和编辑者之间划清界限吗?或者让你的工作线程直接进行编辑?
更进一步:量化金融体系
看懂新闻只是起点——沿量化金融路径,把它变成能交付的工程能力