Routing-Table RAG实测：700题准确率99.86%及成本分析

Follow-up: measured the routing-table RAG on 700 questions. It beat retrieval everywhere except the one place a human wrote a false sentence

Follow-up to my routing-table RAG post, where I asked how to evaluate it. I measured it: 700 questions, one frozen corpus (1,126 docs, 5 areas), four arms (rag, rag+rerank, routing, routing+overlay), full count, not a sample. Walker is claude-sonnet-5, reranker gpt-5, embeddings text-embedding-3-large.

What I tested, in order:

• Find where plain RAG breaks. Then look at what the routing table does on exactly those questions.
• Bolt a reranker onto RAG at those breaking points. Find where that breaks. Then look at what the routing table does there.
• Put each tool's cost next to its result — calls, tokens, dollars, latency, per question.

Short version: routing hit 699/700, and the one miss wasn't a routing failure — it was one false sentence a human wrote in the map, and both routing arms obeyed it identically. That's the part worth reading. The chapter where routing just wins on accuracy comes first only because it's the shortest — reporting only the winning side is advertising, so the cost section is where the real numbers are.

(Terms: the map is the human-written routing table. A walk is one agent run on one question — several tool calls. routing+overlay is routing plus an explicit working set of docs opened so far.)

1. Where plain RAG breaks — and what routing does there

I split the 700 questions by how the answer has to be reached, and ran every arm on all of them.

(≈ values are read off my chart, not exact. routing+overlay is omitted — it matches routing to three decimals on every row and misses the same single question.)

Plain RAG breaks on two nameable classes: prose questions against code-indexed docs (0.028) and questions about a superseded version (0.133). On those it doesn't degrade — it just fails. There's no map to be wrong, but also no map to be right; nothing tells it the doc it wants is filed under a code it never saw.

Routing on exactly those questions: 1.000 and 1.000. The map row for that area says in plain English when to go there, and the walk goes there.

Worth saying plainly: 320 of the 700 are questions plain RAG already gets on the first try (0.991). Nothing in this data says to walk those.

2. Add a reranker — where does that break?

Reranking fixes what it can see. It takes direct from 0.991 to 1.000, and that's about it: indirect moves 0.028 → ≈0.07, two versions back 0.133 → ≈0.20. The reason is mechanical: the reranker only fired on 390 of 700 questions (0.56×), because on the other 310 the fusion candidates didn't contain the answer, so there was nothing to reorder. A reranker can't promote a doc that retrieval never surfaced. RAG's breaking points and rerank's breaking points are the same points.

Routing on those: still 1.000. Which brings us to the one row where routing loses.

The one miss, and why it's the real finding

The single routing failure is a current rule question — a family plain RAG gets 1.000 on. It wasn't a bad walk. Two docs in the corpus are relocation notices — stubs left behind when a topic moves to another area, telling the walker where it went. Both said:

These pages remain correct for dates before 2026-01-01, and incorrect after.

False. Those pages were already superseded on 2024-07-01, 18 months before the relocation the notice describes. The notice wasn't wrong about the move; it was silent about the revision before it, and silence reads as "nothing more to know." A walk read it, did exactly what it said, and answered a 2025 claim with the 2023 rule.

Evidence it was the map and not the model: routing and routing+overlay — no shared context, one of them carrying a record of its own decisions — failed the same question through the same two docs onto the same wrong era. The working set couldn't help. Nothing had been forgotten; the notice was read, understood, and obeyed.

I fixed the sentence (full version table, explicit "dates after 2024-07-01 are answered elsewhere") and reran the 10 current rule questions:

Same notice, different destination. The agent didn't get more careful. It was handed a true sentence.

Two things this taught me about where the approach is exposed:

• Docs compete; map rows are obeyed. A bad doc is one of ten candidates weighed against nine others. A bad sentence in the map isn't weighed — it's an instruction.
• The failure hides. RAG's wrong answers look wrong (thin, off-topic, empty). Routing's wrong answers look right — specific numbers, cited source, a clean-looking walk log. A 50-question sample I ran first happened to miss the bad path and reported 1.000. That's why it took all 700.

3. Cost next to result, per question

Within routing, cost tracks how many docs the answer stands on (n=5, one walk per family):

A two-doc question costs ~2× and takes ~5× a one-row question. That's the work the question demanded, not overhead to engineer away.

The number that will get misquoted: raw usage shows 200k–500k input tokens per walk. Real, but every tool call resends the conversation so far and nearly all of it is served from cache:

Quote 500k and you overstate cost 10×. Quote the ~20 new tokens and you understate it 1000×. $0.12–$0.28 is the measured number.

And that price is the price when the map works. A full traversal of this corpus would be ~1,340 calls and ~2.7M tokens — it fits in no context. The runner caps at 40 turns (~82k tokens), which is about one session's capacity. So one walk can see at most ~3% of the tree; the median walk sees 0.52%.

The map's job isn't to save calls — it's to make the 0.5% a walk can afford the right 0.5%.