Strata优化DeepSeek V4.1：KV拼接与常驻路由加速本地推理

Sam Altman recently said that "from a global allocation perspective, actually, it's quite terrible to have everybody have a bunch of the world's limited supply of DRAM sitting in their own computer, and being used maybe 5% of the time."

Challenge accepted. Here's my "quite terrible" home setup: an RX 9070 XT 16 GB, a 5800X3D, 64 GB of DDR4 that's now worth more than the rest of the PC, and a PCIe 3.0 NVMe. It runs DeepSeek V4.1 Flash at 1M context (IST-DAS 1.75 bpw; the lab reports 98.7% of the full model's task average) through Strata, fully offline, with the experts spread across VRAM, RAM and the SSD.

Generation at ~7 tok/s was fine. The problem was prompt reading: every new OpenCode session re-read the full ~37K-token system prompt plus tools, about 100 s before the first token. That's datacenter-grade waiting without the datacenter.

So I patched the DeepSeek engine. Two changes:

1. KV splice: stop re-reading what you already read

Agents put per-session text (working directory, date, project CLAUDE.md, skill paths with random IDs) inside the system prompt, so the prefix cache misses on every new session. The engine now aligns a new prompt against every saved conversation, cuts the stale lines out of the cached state (moving the cached rows down, rotating the window rings, recording a RoPE gap), and reads only the changed lines plus your message. Changed lines move as whole lines, so the model never sees fragments.

2. Resident-only routing: make the RAM earn its keep

Even a short prompt touches nearly every expert of every layer, so prompt reads streamed most of the model off the SSD: 58 GB for a 1.2K-token prompt. During prompt reading only, tokens now pick from experts already sitting in VRAM or RAM. Generation routes normally. Turns out home DRAM is pretty useful when you actually let it do things.

Results (same machine) as seen above on the body image

Caveats:

• HIP (gfx1201) only. I haven't built or tested CUDA, SYCL or Metal.
• The splice is approximate: tokens after a cut were computed with the removed text in context. I haven't seen it matter for short guidance lines, but there's no formal eval.
• Resident routing changes answers slightly (substitute experts during prompt reads). My hard coding suite didn't move (12/13 both ways), but that's one run each.
• Both are on by default in the branch; STRATA_DS4_REORDER=0 and STRATA_DS4_PROMPT_RESIDENT=0 turn them off.
• It also includes a one-line fix for a GPU page fault I hit on a 37K-token prompt (#4912).

Branch: https://github.com/PhialsBasement/Strata/tree/dsv41-kv-splice

Build (HIP example):

If you use OpenCode, also add "interleaved": { "field": "reasoning_content" } to the model entry, otherwise OpenCode drops earlier reasoning and the cache misses more often.

Upstream issues with details: #4921 (splice), #4936 (resident routing). If you try it on other hardware, especially CUDA, numbers in those threads would really help get it merged.

Utilization report for Sam pissman, VRAM holding experts, RAM holding experts, SSD streaming the rest at full bandwidth. The only idle component in this system is my wallet, thanks to DRAM prices.