Google Research开源RRSI：防过拟合的AI Agent自改进框架
Google Research Open-Sources RRSI: AI Agents That Improve Their Own Harness Without Overfitting
Agent Harness自进化是前沿方向，RRSI提供了一套完整的防过拟合正则化方案与可复现代码，对做Agent工程的同学极具参考价值。
Google Cloud AI Research, with UNC-Chapel Hill, Stanford and Washington University in St. Louis, has released RRSI (Regularized Recursive Self-Improvement). It lets an LLM agent rewrite its own harness: prompts, tools, memory, control flow and sub-agents. Model weights never change. RRSI constrains the improvement loop itself, so gains hold on benchmarks the agent never optimized against.
Google Cloud AI Research 与北卡罗来纳大学教堂山分校、斯坦福大学和圣路易斯华盛顿大学合作，发布了 RRSI（正则化递归自我改进）。它允许 LLM 代理重写自身的工具包：包括提示词、工具、记忆、控制流和子代理。模型权重永不改变。RRSI 对改进循环本身施加约束，因此其在基准测试上的提升能够泛化到代理从未针对其优化的任务上。
Deployable? Yes, as a research framework. The code is Apache 2.0, needs Python 3.10+, and accepts any LiteLLM model string. Defaults assume Claude Opus 4.8 on Vertex AI.
可部署吗？是的，作为研究框架。代码采用 Apache 2.0 许可证，需要 Python 3.10+，并接受任何 LiteLLM 模型字符串。默认配置假设在 Vertex AI 上使用 Claude Opus 4.8。
Why Self-Improving Harnesses Overfit
为什么自我改进的工具包会过拟合
Harness evolution loops propose edits, score them on a fixed evolve set and keep the winner. The same tasks are reused every round, so the loop can memorize them. The RRSI research names 3 failure modes: benchmark-specific fitting, noise chasing and complexity accumulation. Each one widens the gap between evolve-set scores and real transfer.
工具包演化循环提出编辑建议，在固定的演化集上对其进行评分并保留优胜者。每轮都重复使用相同的任务，因此循环可能会记住这些任务。RRSI 研究指出了三种失败模式：特定于基准的拟合、追逐噪声和复杂性累积。每一种都会扩大演化集得分与实际迁移能力之间的差距。
How RRSI Works
RRSI 的工作原理
RRSI keeps every harness component editable. It regularizes how the search moves instead.
RRSI 保持所有工具包组件可编辑。相反，它对搜索的移动方式施加正则化。
Proposal side
提议侧
- Annealed edit budget: a cosine schedule lets early rounds bundle several edits. Late rounds allow a single attributable change.
- Evidence-aware credit: each candidate is logged with its component, hypothesis, diff, score change and cost change. The proposer reads this ledger, so falsified ideas are not retried.
- Structured exploration: when progress stalls inside the noise band, budget shifts to components the run never touched.
- 退火编辑预算：余弦调度允许早期轮次捆绑多次编辑。后期轮次只允许单个可归因的更改。
- 基于证据的信用分配：每个候选方案都与其组件、假设、差异、分数变化和成本变化一起记录。提议者读取此分类账，因此被证伪的想法不会被重试。
- 结构化探索：当进展停滞在噪声带内时，预算转移到运行中从未触及的组件。
Selection side
选择侧
- Leakage critic: rejects task names, entities, answers or benchmark-specific logic before any scoring.
- Noise-adjusted floor: gains must clear the variance measured on the unchanged base harness.
- Cost rule: extra inference tokens must be paid for by measured gain.
- Pruning: components that stop producing gains become deletion targets.
- 泄漏批评家：在任何评分之前拒绝任务名称、实体、答案或特定于基准的逻辑。
- 噪声调整下限：增益必须超过在未更改的基础工具包上测量的方差。
- 成本规则：额外的推理令牌必须由测量的增益来支付。
- 剪枝：停止产生增益的组件成为删除目标。
The research team frame these as analogies to classic regularizers. The edit budget maps to L0, pruning to Lasso (L1) and the cost rule to Ridge (L2).
研究团队将这些类比为经典正则化器。编辑预算映射到 L0，剪枝映射到 Lasso (L1)，成本规则映射到 Ridge (L2)。
Results Across 8 Benchmarks
跨 8 个基准的结果
- Terminal-Bench 2.1 (evolve split): 74.2% to 80.2%.
- SWE-bench Verified (never used for selection): 82.0% to 83.8%.
- Out of distribution: JobBench +4.7, GDPval +3.5 and APEX-Agents +3.7 points.
- EngDesign (evolve) +4.9; Frontier-Eng +4.3 Medal points.
- Harvey LAB: +1.1 on the evolve split, +2.3 on its held-out split.
- Terminal-Bench 2.1（演化分割）：从 74.2% 提升至 80.2%。
- SWE-bench Verified（从未用于选择）：从 82.0% 提升至 83.8%。
- 分布外：JobBench +4.7，GDPval +3.5 和 APEX-Agents +3.7 分。
- EngDesign（演化）+4.9；Frontier-Eng +4.3 奖牌分。
- Harvey LAB：在演化分割上 +1.1，在其保留分割上 +2.3。
All 6 held-out splits improved. With Gemini 3.5 Flash as the policy, Terminal-Bench 2.1 rose from 64.6 to 78.7. SWE-bench Verified rose from 76.8 to 79.0.
所有 6 个保留分割均有所改善。以 Gemini 3.5 Flash 作为策略时，Terminal-Bench 2.1 从 64.6 升至 78.7。SWE-bench Verified 从 76.8 升至 79.0。
The harness is also lighter. On the agentic workspace instance, RRSI uses 2.42M policy tokens per trial. Unregularized evolution uses 3.80M. The abstract reports this as 30% fewer; the project page says 36%.
线束也更轻。在代理工作区实例上，RRSI 每次试验使用 2.42M 策略令牌。无正则化进化使用 3.80M。摘要报告为减少了 30%；项目页面显示为减少了 36%。
RRSI vs Closest Competitors
RRSI 与最接近的竞争对手
Scores come from Table 1 of the RRSI research paper. All methods share the same starting harness, policy, evolve split and candidate budget.
分数来自 RRSI 研究论文中的表 1。所有方法共享相同的起始线束、策略、进化拆分和候选预算。
| Feature | RRSI | Meta-Harness | AHE | TTHE | HarnessX |
|---|---|---|---|---|---|
| Core idea | Regularized proposal and selection | Agentic proposer over code, scores and traces of all prior candidates | Observability-driven loop; edits paired with verified predictions | Evolves harness during test time, no gold labels | Modular typed primitives, trace-driven adaptation |
| Model weights | Frozen | Frozen | Frozen | Frozen | Frozen |
| Cost rule and pruning | Yes | No* | No* | No* | No* |
| Harvey LAB evolve score | 90.5 | 93.0 | 90.7 | 91.1 | 91.8 |
| OOD average (H0 = 39.7) | 43.6 | 40.6 | 39.2 | 38.0 | 39.7 |
| 特性 | RRSI | Meta-Harness | AHE | TTHE | HarnessX |
|---|---|---|---|---|---|
| 核心理念 | 正则化的提议与选择 | 基于代码、评分及所有先前候选项轨迹的代理提议者 | 可观测性驱动的循环；编辑与经过验证的预测配对 | 在测试时进化线束，无需黄金标签 | 模块化类型原语，轨迹驱动的适应 |
| 模型权重 | 冻结 | 冻结 | 冻结 | 冻结 | 冻结 |
| 成本规则与剪枝 | 是 | 否* | 否* | 否* | 否* |
| Harvey LAB evolve 得分 | 90.5 | 93.0 | 90.7 | 91.1 | 91.8 |
| OOD 平均值 (H0 = 39.7) | 43.6 | 40.6 | 39.2 | 38.0 | 39.7 |
*Per the RRSI research team. OOD average is the mean of JobBench, GDPval and APEX-Agents, computed from Table 1.
*根据 RRSI 研究团队说明。OOD 平均值是 JobBench、GDPval 和 APEX-Agents 的平均值，数据源自表 1。
Meta-Harness leads on the Harvey LAB evolve split. RRSI has the smallest evolve gain but the only OOD average more than 1 point above H0.
Meta-Harness 在 Harvey LAB evolve 拆分上领先。RRSI 的进化增益最小，但它是唯一一个 OOD 平均值比 H0 高出超过 1 点的方法。
Interactive Explainer
交互式解释器
How RRSI Regularizes Agent Self-Improvement
RRSI 如何正则化代理的自我改进
The model stays frozen. The harness (prompts, tools, memory, control flow) evolves, but every edit must pass the regularizers.
模型保持冻结。线束（提示词、工具、记忆、控制流）发生进化，但每次编辑都必须通过正则化器的检查。
1. Run a round 2. Edit budget 3. Acceptance gate 4. Results
1. 运行一轮 2. 编辑预算 3. 接受门控 4. 结果
Pick a candidate edit, then press Run. Watch where RRSI stops it.
选择一个候选编辑，然后按下运行。观察 RRSI 在哪里阻止它。
Generic tool fix Hardcoded task name Gain inside noise Token-hungry gain Stale component ▶ Run
通用工具修复 硬编码任务名称 内部噪声中的收益 消耗大量令牌的收益 陈旧组件 ▶ 运行
ProposerReads full edit ledger, shrinking edit budget
提议者读取完整的编辑账本，从而缩减编辑预算
Leakage criticScreens diff before scoring
泄漏批评者在评分前筛选差异
EvaluateRun on evolve set
评估在进化集上运行
GateNoise floor + cost rule
门控噪声下限 + 成本规则
Harness Ht+1Accepted, then pruning check
线束 Ht+1 已接受，随后进行剪枝检查
Candidates are illustrative. The rules they hit are the ones described in the RRSI paper.
候选项仅为示例。它们触发的规则即为 RRSI 论文中描述的那些规则。
// edit ledger: component | hypothesis | Δscore | Δcost | verdict
// 编辑账本：组件 | 假设 | Δ得分 | Δ成本 | 裁决
bt = ⌈ bmin + (bmax − bmin) · ½(1 + cos(πt / T)) ⌉
bmax (edits per candidate, early) 5 bmin (edits per candidate, late) 1 T (rounds) 16 Early rounds may bundle coordinated edits to find a mechanism. Late rounds get single, attributable changes.
bmax（每个候选项的编辑次数，早期）5 bmin（每个候选项的编辑次数，晚期）1 T（轮数）16 早期轮次可能会捆绑协调的编辑以寻找机制。晚期轮次则获得单一、可归因的变更。
round 0round T−1
第 0 轮到第 T−1 轮
Cosine schedule from the paper (Eq. 4). Slider values are for exploration; the paper lists its own settings in Appendix D.
论文中的余弦调度公式（公式 4）。滑块数值用于探索；论文附录 D 列出了其自身的设置。
ΔS, score gain vs incumbent (pts) 2.0 ΔC, token cost change (%) 20 δ, noise band from repeated base runs (pts) 1.5 β0 + β1ΔS cost allowance: β1 (% per pt) 8
ΔS，相对于当前最优的得分增益（分）2.0 ΔC，令牌成本变化（%）20 δ，来自重复基线运行的噪声带（分）1.5 β0 + β1ΔS 成本许可：β1（每分百分比）8
?Noise-adjusted floor: score must not drop more than δ below the best so far
?噪声调整基准：得分不得比当前最高分低超过 δ
?Clear gain: ΔS > δ, otherwise the paper’s within-band rule applies
?明确增益：ΔS > δ，否则适用论文中的带内规则
?Cost rule: ΔC ≤ β0 + β1ΔS
?成本规则：ΔC ≤ β0 + β1ΔS
…
Illustrative thresholds (β0 fixed at 5% here). In RRSI, δ is estimated on the unchanged base harness and β values are tuned on the evolve set, then frozen.
示例阈值（此处 β0 固定为 5%）。在 RRSI 中，δ 在未更改的基础 harness 上估计，β 值在演化集上调优，然后冻结。
All 8 benchmarks vs prior methods Token cost
所有 8 个基准测试与先前方法的对比 Token 成本
Data: RRSI paper · GitHub · project pageBuilt by Marktechpost
数据：RRSI 论文 · GitHub · 项目页面由 Marktechpost 构建
Getting Started
入门指南
git clone https://github.com/google-research/rrsi.git && cd rrsi
pip install -e ".[dev]"
python3 rrsi.py --domain coding baseline
python3 rrsi.py --domain coding runEach round drafts 2 candidates in separate git worktrees, screens them, evaluates both and fast-forwards the branch to the winner. The coding instance also needs Docker and harbor. New domains plug in through a single adapter module.
每轮在独立的 git worktree 中起草 2 个候选方案，筛选它们，评估两者并将分支快进到获胜者。编码实例还需要 Docker 和 harbor。新领域通过单个适配器模块接入。
Key Takeaways
关键要点
- RRSI evolves prompts, tools, memory and workflows while model weights stay frozen.
- A leakage critic, noise floor, cost rule and pruning decide which edits stick.
- Terminal-Bench 2.1 rose from 74.2% to 80.2% with Claude Opus 4.8.
- SWE-bench Verified, never used for selection, rose from 82.0% to 83.8%.
- Apache 2.0 code on GitHub; research-grade, not an official Google product.
- RRSI 演化提示词、工具、记忆和工作流，而模型权重保持冻结。
- 泄漏批评器、噪声基准、成本规则和剪枝决定哪些编辑得以保留。
- 使用 Claude Opus 4.8，Terminal-Bench 2.1 从 74.2% 提升至 80.2%。
- SWE-bench Verified（从未用于选择）从 82.0% 提升至 83.8%。
- GitHub 上的 Apache 2.0 代码；研究级产品，非 Google 官方产品。
Check out the Paper, Codes and Technical details. All credit goes to the researcher of this project. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.
查看论文、代码和技术细节。所有功劳归于该项目的研究人员。也欢迎在 Twitter 上关注我们，别忘了加入我们拥有 15 万+成员的 ML SubReddit 并订阅我们的新闻通讯。等等！你在 Telegram 上吗？现在你也可以在 Telegram 上加入我们。
更进一步：量化金融体系
看懂新闻只是起点——沿量化金融路径，把它变成能交付的工程能力