Reddit开发者实测RAG检索优化：源路由与重排器效果显著
A few of you asked in my last post how I know the "9 techniques" actually help. Fair question, so I measured them.
这是一篇极具参考价值的RAG工程实战复盘，用真实数据证明了“源路由”等简单策略在特定场景下的巨大威力，避开了常见的技术陷阱，开发者可直接复用其评测框架。
Setup: FinanceBench, 150 analyst-written questions over 84 real 10-Ks/10-Qs, about 55k chunks in one index. Everything local: nomic-embed-text + Llama 3.1 8B on Ollama, on a laptop. The metric is whether the exact evidence page lands in the top 5 passages given to the LLM.
设置：FinanceBench，包含84份真实10-K/10-Q文件上的150个分析师撰写的问题，一个索引中约有5.5万个块。全部本地运行：在笔记本电脑上使用Ollama运行nomic-embed-text和Llama 3.1 8B。指标是给定LLM的前5个段落中是否包含确切的证据页面。
| Retrieval setup | Evidence page in top 5 | Right filing in top 5 |
|---|---|---|
| Plain vector search | 25% | 78% |
| Hybrid (BM25 + vector) | 25% | 71% |
| + entity graph | 25% | 71% |
| + cross-encoder reranker | 29% | 77% |
| + source routing | 29% | 87% |
| Routing + reranker | 39% | 93% |
| 检索设置 | 证据页面在前5名中 | 正确文件在前5名中 |
|---|---|---|
| 纯向量搜索 | 25% | 78% |
| 混合（BM25 + 向量） | 25% | 71% |
| + 实体图 | 25% | 71% |
| + 交叉编码器重排序器 | 29% | 77% |
| + 源路由 | 29% | 87% |
| 路由 + 重排序器 | 39% | 93% |
"Source routing" is simple: if the question names a company that matches a file name, search only that file. With 84 filings full of the same finance vocabulary, plain search keeps landing in the wrong company's 10-K. Routing plus a reranker gets the evidence page about 1.6× as often, with no extra LLM calls.
"源路由"很简单：如果问题命名的公司与文件名匹配，则仅搜索该文件。由于84份文件中充斥着相同的财务词汇，普通搜索总是落入其他公司的10-K文件中。路由加上重排序器使证据页面的命中率提高了约1.6倍，且没有额外的LLM调用。
Things that surprised me:
令我惊讶的事情：
- Hybrid search didn't help by itself. Long analyst questions match generic finance wording in every filing, so BM25 pulled in look-alike passages from other companies.
- The entity graph I was proud of changed nothing.
- The benchmark caught real bugs: 39 of the 84 PDFs were encrypted and being silently skipped, and my RAG-Fusion was literally searching for the model's "Here are three alternative queries:" line.
- 混合搜索本身没有帮助。长长的分析师问题与每份文件中通用的财务措辞相匹配，因此BM25从其他公司拉入了外观相似的段落。
- 我一直引以为豪的实体图毫无作用。
- 基准测试发现了真正的bug：84个PDF中有39个被加密并被静默跳过，而且我的RAG-Fusion实际上是在搜索模型的“以下是三个替代查询：”这一行。
Still running: HyDE, RAG-Fusion, CRAG and the "everything on" config, plus end-to-end answer accuracy compared with a naive baseline. I also found something odd about embeddings differing between two Ollama versions on the same machine, which I want to verify before I say more. I'll post the full results, including the configs that made things worse.
仍在运行：HyDE、RAG-Fusion、CRAG以及“全开”配置，以及与朴素基线相比的最终答案准确性。我还发现同一台机器上两个不同Ollama版本之间的嵌入存在差异，这有点奇怪，我想在进一步说明之前验证一下。我将发布完整结果，包括那些让情况变得更糟的配置。
The harness is in the repo if you want to run it on your own documents: https://github.com/SaiAkhil066/CORTEX-AI-SUPER-RAG/tree/main/eval
如果您想在自己的文档上运行它，工具包位于仓库中：https://github.com/SaiAkhil066/CORTEX-AI-SUPER-RAG/tree/main/eval
What else should I test? A different embedding model and chunk size are next on my list.
我还应该测试什么？下一个在我的列表上是不同的嵌入模型和块大小。
更进一步：量化金融体系
看懂新闻只是起点——沿量化金融路径，把它变成能交付的工程能力