Agent技能并非总是值得使用

Are agent skills always worth using?

The answer is no?

This paper provides some important insights to understand this more.

(bookmark it)

Paper summary:

Adding procedural skills to an agent is usually scored by average task success. That number nets gains against damage and hides half of what happened.

Setup: nearly 6,000 paired runs across two office automation benchmarks and three model harness stacks, comparing the same agent with and without skills.

A regression is a task the agent solved without skills and then failed once skills were added. Regressions are large enough that the best performing skills separate themselves mainly through fewer regressions. Larger gains contribute much less.

Three mechanisms drive it.

Skill description osmosis, where a skill changes agent behavior just by sitting in context even when it is never invoked. Grounding displacement, where a prescribed procedure overrides how the agent reads its inputs. Verification displacement, where the procedure suppresses checks the agent would otherwise run on its own output.

Trace analysis surfaces that procedural guidance is the stage least often responsible for failure, while grounding and verification dominate the errors that remain. Existing skills are almost entirely procedure.

Paper: https://arxiv.org/abs/2607.22520

Learn to build effective AI agents in our academy: https://academy.dair.ai/