Google DeepMind 新论文：推理时回灌深层激活可恢复模型丢失的上下
New Google DeepMind paper shows, a transformer's shallow layers never see what i…
New Google DeepMind paper shows, a transformer's shallow layers never see what its deeper layers have already worked out about the context. Leaking a little of it back down during inference recovers part of what was lost.
谷歌DeepMind的新论文显示，Transformer的浅层网络从未看到其深层网络已经对上下文得出的结论。在推理过程中将其中一小部分信息回传，可以恢复部分丢失的信息。
So if your model keeps losing state over a long input, the fix may not need retraining, though the cost moves to prefill, which can no longer run in parallel.
因此，如果你的模型在处理长输入时不断丢失状态，修复可能无需重新训练，尽管成本会转移到预填充阶段，该阶段无法再并行运行。
The paper shows this working on frozen weights, but the size of the benefit does not transfer across model families.
论文表明这在冻结权重上有效，但收益的大小在不同模型家族间并不通用。
Recirculation mixes a small fraction of a deep layer's activations into a shallow layer at the next input step, weights untouched, and sets that against the off-the-shelf model.
循环混合将深层激活的一小部分在下一个输入步骤时混入浅层，权重保持不变，并与现成模型进行对比。
On Gemma3, perplexity drops on 9 of 10 language-modeling datasets, by as much as 35% for the 12B model, though multiple-choice gains are modest.
在Gemma3上，10个语言建模数据集中有9个的困惑度下降，12B模型下降高达35%，尽管多项选择的提升较为有限。
– arxiv. org/abs/2608.17981
– arxiv.org/abs/2608.17981
Title: "Recirculation"
标题：“循环混合”
更进一步：量化金融体系
看懂新闻只是起点——沿量化金融路径，把它变成能交付的工程能力