AI Agent自动优化：GEPA等三种方法对比

The next step after Karpathy's autoresearch idea:

Tuning an agent is mostly manual work, done by editing prompts, tools, and control flow by hand and rerunning the evals to see what moved.

Researchers have been building several automated optimizers to do that outer loop instead.

The underlying process is the same:

• An LLM proposes a change - An evaluator scores it - And the proposer reads that result before proposing the next one.

They differ in what they edit and what feedback they get to read.

1) Berkeley built GEPA that optimizes the text of a system, like prompts, tool descriptions, or the agent's own code.

Instead of collapsing a run into one scalar reward the way RL does, it reads the full execution trace (errors, reasoning, tool output), diagnoses why the run failed, and proposes a targeted fix.

It also keeps every candidate that's best at some part of the task, not just the one with the highest average score. So a strong specialist survives even when a more balanced candidate beats it overall.

That trace-level feedback lets it converge in hundreds of rollouts instead of the thousands that GRPO needs.

2) AutoResearch, inspired by Karpathy, runs a narrower version of the loop, where a coding agent iterates on a program(.)md file, scores the outputs against fixed evals, and keeps whatever improves.

3) Meta-Harness points the loop at the harness itself, the scaffolding code that decides what to retrieve, how to format it, and what state to keep between calls.

The natural question is which of the three to use. And the answer is none of them alone.

On the Frontier-CS benchmark, holding the model, thinking effort, and budget fixed, no optimizer wins everywhere. Across 10 tasks, GEPA led on 3, AutoResearch on 3, and Meta-Harness on 4.

Each optimizer hill-climbs fast and then stalls, making most of its progress in a few iterations before flattening out.

But the researchers found that handing the stalled candidate to a different optimizer breaks the plateau, since each one attacks the problem differently.

To use this in practice, omni (open-source) already automates all of that.

It runs every optimizer on a fraction of the budget, takes the best candidate, and hands it to a fresh optimizer to keep going.

It scores 7.8 percentage points above the best standalone optimizer at the same budget, and finishes faster.

The whole meta-optimizer is around ten lines in the optimize_anything API, the same interface these optimizers already run through.

You can also point an agent at the gepa-ai/gepa repo and have it use the gepa-optimize-anything skill.

Here's the repo: http://github.com/gepa-ai/gepa

For a first-principles guide to GEPA and the engine under most of this, my co-founder wrote a full breakdown of why reflective evolution beats GRPO by around 10 points with 35x fewer rollouts and no GPU training, purely from natural-language reflection on trajectories rather than backprop on weights.

Read it below.