RAG经验检索：外部结果优于模型自判，相似度排序收益有限
Similarity ranking beat unranked same-repo lessons by 0.019 (95% CI -0.031 to +0.069) on 160 forks
Agent记忆检索是落地痛点，这篇用扎实数据证明了“外部结果”比“模型自判”有效，且否定了盲目做相似度排序的做法，给做RAG和Agent的同学提供了明确的避坑指南。
Setup first, because it decides how far this goes. The random baseline was not random over everything: each random lesson came from the same repository and carried the same surface cues as the lesson similarity search picked. We did not test a learned ranker. And every lesson carried the fork's true outcome, which no deployed agent gets.
先进行设置，因为它决定了最终效果的上限。随机基线并非完全随机：每个随机课程都来自同一个代码仓库，并带有与课程相似度搜索所选取的课程相同的表面线索。我们并未测试学习到的排序器。而且每门课程都包含了该分支的真实结果，这是任何已部署的智能体都无法获得的。
What we ran: 160 engineering forks from Taste-Bench (Pan et al.). Each fork is a real coding decision with two options and one that led to the better outcome. Claude Opus 5.5 sees both options in both orders and only scores when both answers are right. A lesson is one line from another task: "choosing X over Y led to the better outcome".
我们运行了：来自 Taste-Bench (Pan et al.) 的 160 个工程分支（fork）。每个分支都是一个真实的编码决策，包含两个选项和一个导致更好结果的选项。Claude Opus 5.5 会以两种顺序查看这两个选项，并且仅在两个答案都正确时才进行评分。一门课程来自另一个任务的一行内容：“选择 X 而非 Y 导致了更好的结果”。
- No lessons: 0.594
- Random same-repo lessons: 0.750
- Top 3 by embedding similarity: 0.769
- 无课程：0.594
- 随机同仓库课程：0.750
- 按嵌入相似度排名的前 3 名：0.769
So the lessons helped a lot, and picking them by similarity added 0.019, which this run cannot tell from zero.
因此，课程起到了很大的帮助作用，而通过相似度选择它们增加了 0.019，这一增量在本次运行中无法与零区分开来。
What did matter was where the outcome came from. In a separate Qwen run, lessons written after it was told the outcome added 0.087 (+0.036 to +0.138). When it judged its own earlier choices, its lessons scored 0.044 below no lessons (-0.087 to 0.000).
真正重要的是结果的来源。在单独的 Qwen 运行中，在告知其结果后编写的课程增加了 0.087（从 +0.036 到 +0.138）。当它评估自己之前的选择时，其课程得分比无课程情况低 0.044（从 -0.087 到 0.000）。
How are you scoring the memories your agent learns from: by an outside result, or by the agent itself?
你是如何对你智能体所学的记忆进行评分的：是通过外部结果，还是由智能体自身？
Write-up, with a probe that prints every number:
撰写报告，附带一个打印所有数值的探针：
https://dancenitra.github.io/agora/public/posts/agent-memory-outside-outcomes-not-ranking.html
更进一步：量化金融体系
看懂新闻只是起点——沿量化金融路径，把它变成能交付的工程能力