RAG检索优化:让Jev模型直接选页面而非分块,错误率降至3%
Our RAG kept retrieving "How to Find a Reliable Plumber" for plumbing complaints, so we tested letting Jev pick pages instead
RAG落地中的经典痛点——检索噪声大。这篇分享给出了具体的工程取舍:放弃盲目切块,用轻量Router做页面级路由,错误率从33%降到3%,数据扎实且逻辑清晰,值得RAG开发者参考其架构思路。
Small-business websites are already split into pages that are each about one thing, so chunking them always felt backwards. We kept the pages whole, gave each a short description, and asked Jev to pick the page that answers the caller, or "none".
小型企业网站已经按页面划分,每个页面只关注一件事,因此对它们进行分块处理总让人觉得本末倒置。我们保持页面的完整性,为每个页面提供简短描述,并让 Jev 选择能回答来电者问题的页面,或者选择“无”(none)。
Over 168 questions on four sites it handed the LLM a wrong page 3% of the time against 33% for our chunk embeddings. GPT-6 Luna picking from the same list tied it on ranking, but was more than twice as slow and less keen to say none.
在四个网站的 168 多个问题中,它向 LLM 提供错误页面的比例仅为 3%,而我们的分块嵌入方法这一比例为 33%。GPT-6 Luna 从同一列表中挑选时,排名与之持平,但速度慢了兩倍以上,且不太愿意选择“无”。
The surprise was the "none". Tuning our embedding cut-off still left 15% wrong pages, because a similarity score always finds a closest match.
令人意外的是“无”这个选项。调整我们的嵌入阈值后,仍有 15% 的错误页面,因为相似度评分总会找到最接近的匹配项。
Has anyone tried the open Jev-style models (Kev, Jeff, Clef) for retrieval like this? That's what I'd test next.
有人尝试过使用开源的 Jev 风格模型(如 Kev、Jeff、Clef)进行此类检索吗?这正是我下一步要测试的内容。
Write-up with the full method and caveats at https://sayora.ai/many-hands/jev-vs-embeddings/ (I wrote it; we build an AI phone receptionist and have no tie to TypeSafe).
完整方法及注意事项的撰写见 https://sayora.ai/many-hands/jev-vs-embeddings/(文章由我撰写;我们开发了一款 AI 电话接待员,与 TypeSafe 无任何关联)。
更进一步:量化金融体系
看懂新闻只是起点——沿量化金融路径,把它变成能交付的工程能力