FreeToken：8GB 游戏本跑 35B 模型，速度超 Codex
An 8 GB gaming laptop can run a 35B model at 39.3 tokens a second, faster than C…
做本地推理优化的同学必看，FreeToken 把专家缓存命中率大幅提升，8GB 显存跑 35B 模型的速度直接对标 Codex，赶紧去复现一下这套路由感知的缓存策略。
An 8 GB gaming laptop can run a 35B model at 39.3 tokens a second, faster than Codex in production, if the software stops guessing which parts of the model to keep on the GPU.
一台8GB显存的游戏笔记本能以每秒39.3个token的速度运行35B模型，比生产环境中的Codex还快，前提是软件不再猜测模型哪些部分应保留在GPU上。
That's FreeToken, New research from Berkeley and University of Texas, a new engine for running big open models on ordinary machines.
这就是FreeToken，来自伯克利和得克萨斯大学的新研究，一个在普通机器上运行大型开放模型的新引擎。
Every other local engine picks which experts sit on the GPU at load time and leaves them there, so when the model routes elsewhere, that expert gets copied over PCIe while the CPU sits idle.
其他所有本地引擎在加载时决定哪些专家驻留在GPU上并保持不变，因此当模型路由到别处时，该专家会通过PCIe复制过去，而CPU则闲置。
FreeToken lets the GPU cache follow whatever the router just asked for, cutting missed experts from llama.cpp's 62% to 16%.
FreeToken让GPU缓存跟随路由器的最新请求，将未命中专家率从llama.cpp的62%降至16%。
The rest it splits two ways: some copied to the GPU, the others run in place in system RAM, in a ratio computed from your machine's real PCIe and memory speeds, so neither side waits on the other.
其余部分则分两路处理：一部分复制到GPU，另一部分在系统内存中原位运行，比例根据你机器的实际PCIe和内存速度计算，确保双方互不等待。
更进一步：量化金融体系
看懂新闻只是起点——沿量化金融路径，把它变成能交付的工程能力