Qwen3.8-Flash-Next Engram表免权重注入100条事实的工

Qwen3.8-Flash-Next ships with a giant n-gram lookup table bolted onto the side of the transformer, and when I saw it I had what I can only call an idiot's insight:

hey, that's a table. You can write stuff in a table.

Three weeks later I can confirm that yes, it is a table, and yes, you can write stuff in it. I can also confirm that "it's just a table" is the kind of sentence that costs you three weeks.

Here's the mechanism in one paragraph, because nothing below makes sense otherwise. At every position the model takes the last 2 and 3 tokens, hashes them into 16 rows of a 320-million-row table, reads those rows, and adds them into the residual stream early on. That's it. A key-value memory addressed by exact token n-grams, read before basically all of the model's actual thinking. DeepSeek calls the design Engram. llama.cpp calls Qwen's version the PLE table.

What I built writes new facts into that table. No weights change. None. Only table rows get trained, and they come out as one 9 MB overlay file the engine applies while it reads the table. The GGUF on disk is never touched. Delete the overlay and the model is exactly its old self again. I'm calling it ENGRAFT, mostly because all the good acronyms were taken.

One thing up front, because you'd find out anyway: I built this in Claude Code sessions, one human (me) driving. I set the goals, approved every design step, ran the hardware and read every result; design, implementation, adversarial review and verification were done by separate model instances, and I made the calls. I'd rather you know that before you read the numbers than after. I'm not here for stars. I'm here to have this broken by someone who knows what they're doing.

The numbers

The test world is Quail: 100 facts about a place that doesn't exist, written as seven Italian documents, which become 3,209 training sentences and 841 held-out test sentences. 88 facts I made up from scratch; the other 12 follow Philip K. Dick's We Can Remember It for You Wholesale — the one that became Total Recall — as I remember it, which is to say badly. It felt thematically correct to implant memories into something about a guy getting memories implanted.

Exact answer, greedy decoding, on the real llama.cpp engine, 841 sentences it had never seen:

• with the overlay: 0.841
• base model, same sentences: 0.005 (4 of 841, and I'm fairly sure it guessed)
• whole thing re-run with a second seed: 0.873
• by question shape: chat 0.883, paraphrase 0.849, statement 0.839, question 0.813, cloze 0.794
• 14,032 rows trained, out of 320,000,000. Overlay: 9 MB.
• 2.4 hours on an integrated GPU — an AMD Ryzen AI MAX+ 395, 128 GB unified memory. A mini PC.

Two things I'd point at in that table. The first is that nothing falls off a cliff by question shape: the spread from chat to cloze is nine points, so the overlay isn't just memorizing one sentence pattern. The second is the second seed. 0.873 against 0.841 means the result isn't one lucky descent, and the gap between the two is roughly the size of the noise I should expect.

Separately, on a different corpus of short made-up facts, with a lighter metric (first answer token at rank 1, expert routing pinned to the base model's picks): 24 facts 0.804, 100 facts 0.792, 300 facts 0.821. Flat. No ceiling in sight at 300. Different corpus, different metric — I'm keeping them apart on purpose, because gluing them together would make this look better than it is.

The same world in three languages

I rebuilt Quail fact by fact in Chinese and in English and grafted each one. Exact answer on the real engine: Italian 0.841, English 0.797, Chinese 0.676; base models 0.005, 0.004, 0.003. Collateral damage on neutral text of each language: 0.0131, 0.0160, 0.0166.

The ordering looks like a language effect. It probably isn't. Control for how much training text each fact actually received and the effect disappears — the language coefficient's 95% interval is [−0.124, +0.020], which contains zero. The boring explanation: my row budget was tuned on Italian, Chinese burns almost twice as many new rows per sentence, so the same budget buys half the training sentences per fact. The test that settles it — a Chinese cell given equal mass per fact — hasn't been run.

But here's the one that surprised me. A graft does not cross languages at all. I took the Italian test set and asked it against the Chinese overlay:

• exact answer 0.0048, against the base model's 0.0048
• first answer token at rank 1 0.0951, against the base model's 0.0951
• 764 of 841 sentences identical to the base model down to the probability of the first token
• and the engine did read 922 overlay rows along the way

The overlay fires and changes nothing. Even probes that mix the scripts — the subject's name in one language, the sentence in the other — get 0 of 20 and 1 of 20. A fact has to be grafted in the language it will be asked in, because the rows are keyed by the tokens of its own script. One pair of languages, one cell, but the numbers are not subtle.

What doesn't work

Three things, and this is the part I actually want you to read.

1. Two facts in one question. Both answers right in 10 of 83 probes; exactly one in 20; neither in 53. Each fact on its own is answered well, so something breaks when they have to combine. When I went back and read all 83 outputs, most of the failure turned out to be my measurement rather than the method — the generation was capped too short, the prompt had no chat template so the model wandered off into free continuation, and I had never measured single-fact recall in free generation to compare against. A seven-arm measurement that separates those from real composition failure is running now. I'd rather fix the cause than the symptom.

2. The rest of the model moves. Mean KL to the base model on neutral text: 0.0037 at 24 facts, 0.0059 at 100, 0.0067 at 300, against a yardstick of 0.0033 — an overlay that only re-quantizes the same rows, i.e. the table's own noise floor. Damage grows slower than the number of facts, but it grows, and on Quail at 100 facts it's 0.0131, about 4× the yardstick, which is past the point where I get to call it noise. The fix I want to try is narrower addressing: key the overlay to the exact token window instead of the hashed rows, so each fact steps on fewer shared rows.

3. Facts are stuck in their language. The section above. Useful if you want per-language behaviour, a real limit if you expected one graft to serve every user.

One negative result worth having, since it saved me from fixing the wrong thing: when a graft fails, it's not hash collisions. I looked. There are none. Failures track the direction of the update and how much text a fact got — when two facts share rows, the heavier one wins about 60% of the time. Budget problem, not addressing problem.

And the limits I can't argue away: rephrasing only works as far as the corpus reaches (the table fires on exact n-grams, by construction); one model measured, with DeepSeek V4.1 Flash next and nothing tried on it yet; and no head-to-head against LoRA or ROME/MEMIT, which is the comparison I'd demand first if I were you.

What's public

Not everything. The part that turns arbitrary documents into a usage corpus — including the row budget that decides which rows get trained — is still mine while I work on it. I'd rather ship it when I'm not embarrassed by it.

Everything else is up: the method, the descent and capture code, the measurement code, the engine fork, the compiled Quail corpus with its row set, the overlays, the config of the measured run, and every result file behind every number above. There's a script that recomputes each number in the README from its file, so if one ever drifts, you get to catch me.

github.com/fulvian/engraft-ngram

Two ways in:

• No model needed: descent, capture and measurement all run against fakes. uv run pytest tests/... on any laptop and you can watch the mechanism work.