给编码会话每个提示附加技能文件通常是净损失
Attaching a skill file to every prompt in a coding session is usually a net loss…
做 Agent 工程的同学必看,这项研究用严格对照实验证明技能文件注入在多数场景是负收益,还给出了按后端路由、早期任务暂缓注入的实操建议,值得拿你的链路验证一遍。
Attaching a skill file to every prompt in a coding session is usually a net loss.
在编码会话中,将技能文件附加到每个提示通常是一种净损失。
A skill file hurts most on the easy early tasks your agent already handles correctly.
技能文件在您的代理已经能正确处理的简单早期任务上伤害最大。
Whatever gain exists belongs to a particular skill, project, and model combination, not to the skill itself.
任何存在的收益都属于特定的技能、项目和模型组合,而非技能本身。
So treat injection as a routing decision you measure per backend, and hold the file back on early tasks until error rates rise.
因此,将注入视为按后端测量的路由决策,并在错误率上升前,在早期任务中保留文件。
Skills can help, but this paper finds the helpful cases are a minority that a single published ranking cannot pick out in advance, since per-pair effects barely transfer across models.
技能可能有帮助,但本文发现,有帮助的情况是少数,单一发布的排名无法提前识别,因为每对效果在模型间几乎不转移。
WebDev-Skills-Bench holds the harness, project, and decoding fixed and varies only the injected file: no skill, the stack-matched skill, an equally long irrelevant one, and slice-removed variants.
WebDev-Skills-Bench 固定了测试框架、项目和解码,仅变化注入的文件:无技能、堆栈匹配技能、同等长度的无关技能,以及移除片段变体。
Across all 4 models, injection lowers mean Pass@2 on stack-matched pairs by 1.3 to 4.2 points while raising token cost by at least 72%.
在所有4个模型中,注入将堆栈匹配对的平均 Pass@2 降低了1.3至4.2个百分点,同时将令牌成本提高了至少72%。
The losses concentrate on easy early tasks, running from 4.0 to 10.7 points there, while harder tasks show no consistent effect.
损失集中在简单的早期任务上,在那里下降了4.0至10.7个百分点,而较难的任务没有一致的效果。
That looks like retry lock-in: the skill fixes structural choices the model would otherwise vary on a second attempt, so a recoverable first mistake becomes a chain-terminating failure.
这看起来像重试锁定:技能固定了模型在第二次尝试中可能变化的结构选择,因此可恢复的首次错误变成了终止链的失败。
– arxiv. org/abs/2608.23067
– arxiv.org/abs/2608.23067
Title: "Signal or Noise? A Benchmark Study of Agent Skills in Web Development"
标题:“信号还是噪音?Web开发中代理技能的基准研究”
更进一步:量化金融体系
看懂新闻只是起点——沿量化金融路径,把它变成能交付的工程能力