跳到主内容
@wquguru
精选88Avi Chawla技巧与观点

Jev+RAG:用可验证证据过滤替代传统重排序提升RAG可靠性

RAG vs. Jev + RAG, clearly explained!

原文
发到 X
推荐理由

给做 RAG 的同学看,这个思路把重排序从“找相关”变成了“找能回答的证据”,配合阈值拦截无效生成,能有效降低幻觉,值得参考其工程实现。

RAG vs. Jev + RAG, clearly explained!

RAG 与 Jev + RAG,清晰解析!

In a standard RAG setup, documents are split into chunks, converted into embeddings, and stored in a vector DB.

在标准的 RAG 设置中,文档被拆分为块(chunks),转换为嵌入向量,并存储在向量数据库中。

When a query arrives, the system embeds it and retrieves the top-k chunks with the closest vectors.

当查询到达时,系统将其嵌入,并检索出向量最接近的前 k 个块。

Many production setups add a reranker after retrieval.

许多生产环境在检索后添加了重排序器(reranker)。

The reranker compares the query with each retrieved passage, improves their ordering, and keeps the highest-scoring results.

重排序器将查询与每个检索到的段落进行比较,优化它们的排序顺序,并保留得分最高的结果。

Those passages are then placed in the context window, and the LLM generates an answer from them.

这些段落随后被放入上下文窗口,大语言模型(LLM)从中生成答案。

The problem is that ranking and answerability are different questions.

问题在于,相关性排序和可回答性是两种不同的问题。

A passage can be more relevant than the other candidates while still containing weak, incomplete, or merely adjacent evidence.

一个段落可能比其他候选者更相关,但仍包含微弱、不完整或仅是相邻的证据。

The LLM receives it anyway and may produce a plausible answer from context that never supported one.

LLM 无论如何都会收到它,并可能基于从未支持过答案的上下文产生一个看似合理的答案。

Jev + RAG keeps the retrieval stage as is but changes what happens before generation.

Jev + RAG 保持检索阶段不变,但改变了生成之前发生的事情。

Instead of using a conventional reranker at this stage, Jev can judge every retrieved candidate against a typed question such as:

在此阶段不使用传统的重排序器,而是让 Jev 针对特定类型的查询对每个检索到的候选者进行判断,例如:

"Does this passage help answer the query?"

“这个段落有助于回答查询吗?”

The query becomes the shared state, while the retrieved passages become individual candidates. Jev evaluates them together and returns a probability for each one.

查询成为共享状态,而检索到的段落成为独立的候选者。Jev 将它们一起评估,并为每个候选者返回一个概率。

Application code then applies a threshold.

应用程序代码随后应用阈值。

↳ Candidates above the threshold continue to the LLM. ↳ Candidates below it are removed from the context window.

↳ 高于阈值的候选者继续传递给 LLM。 ↳ 低于阈值的候选者从上下文窗口中移除。

The same Jev request can also judge whether the remaining evidence is sufficient to answer the query.

同一个 Jev 请求还可以判断剩余证据是否足以回答查询。

If answerability falls below the threshold, the application can skip the LLM and return "not in the documents."

如果可回答性低于阈值,应用程序可以跳过 LLM 并返回“文档中未找到”。

Jev does not replace the embedding model, vector database, or generation model. Retrieval still sets the cap because Jev cannot recover a passage that never entered the candidate set.

Jev 并不替换嵌入模型、向量数据库或生成模型。检索仍然设定上限,因为 Jev 无法恢复从未进入候选集的段落。

The diagram below depicts the complete flow.

下图描绘了完整的流程。

  • RAG retrieves a broad candidate set. - Jev reranks and gates those candidates. - The LLM generates an answer.
  • RAG 检索出一个广泛的候选集。 - Jev 对这些候选者进行重排序和筛选。 - LLM 生成答案。

If you want to dive deeper, I have also written a hands-on guide showing how to build a Jev-style model with open models, entirely locally.

如果你想深入了解,我还写了一篇实操指南,展示了如何使用开源模型完全在本地构建 Jev 风格的模型。

Read it below.

请在下方阅读。

更进一步:量化金融体系

看懂新闻只是起点——沿量化金融路径,把它变成能交付的工程能力

进入量化体系 →

相似阅读

关联信息,但可能不是同一事件