NVIDIA提出TwoTower架构：并行生成提速2.4倍且保留98.7%质量
NVIDIA might just have solved the biggest tradeoff in LLMs.
TwoTower用双塔结构巧妙解耦了自回归与扩散生成的优劣，在几乎不损失质量的前提下大幅提升推理速度，对延迟敏感场景极具参考价值，建议关注其开源实现。
NVIDIA might just have solved the biggest tradeoff in LLMs.
英伟达可能刚刚解决了大语言模型（LLM）中最大的权衡问题。
Every LLM makes you pick between speed and quality. Autoregressive models like GPT give you quality, but they're slow because they generate one token at a time.
每个 LLM 都让你必须在速度和质量之间做出选择。像 GPT 这样的自回归模型能提供高质量，但它们很慢，因为它们是逐个 token 生成的。
Diffusion models do the opposite. They generate whole blocks in parallel, so they're fast, but that speed has always come at the cost of quality.
扩散模型则相反。它们并行生成整个块，因此速度很快，但这种速度一直以来都是以牺牲质量为代价的。
Before we get to NVIDIA's solution, let's understand why diffusion isn't up to the mark in the first place.
在了解英伟达的解决方案之前，让我们先弄清楚为什么扩散模型一开始就不够出色。
A diffusion LLM starts with a block of masked tokens and unmasks them over a few passes. On every pass, it reads all the text written so far and predicts the masked tokens in that block together.
一个扩散式 LLM 从一个被掩码的 token 块开始，并在几次迭代中逐步解除掩码。在每次迭代中，它会读取到目前为止写好的所有文本，并共同预测该块中的掩码 token。
So the same network is doing two very different jobs at once. It has to understand the context that's already there while also guessing at tokens that don't exist yet.
因此，同一个网络同时在做两件截然不同的工作。它既要理解已有的上下文，又要猜测尚不存在的 token。
That's one set of weights pulled in two directions, so it never gets good at either.
这意味着同一组权重被向两个方向拉扯，导致它在两方面都无法做到精通。
TwoTower fixes this by not forcing one network to do both. It clones a pretrained autoregressive model into two towers.
TwoTower 通过不让一个网络同时承担两项任务来解决这个问题。它将一个预训练的自回归模型克隆为两个塔。
The first is a frozen context tower that reads everything written so far, exactly like the original did. It's never trained, so the model's intelligence stays perfectly intact.
第一个是冻结的上下文塔，它读取到目前为止写好的所有内容，与原始模型完全一样。它从不进行训练，因此模型的智能得以完美保留。
The second is a trainable denoiser tower that fills in each new block of tokens in parallel. To stay on track, it constantly cross-attends back to the context tower.
第二个是可训练的降噪塔，它并行填充每个新的 token 块。为了保持正轨，它会不断地对上下文塔进行交叉注意力计算。
The two connect layer by layer, so denoiser layer 5 talks to context layer 5, and so on.
这两个塔逐层连接，因此降噪塔的第五层与上下文塔的第五层通信，依此类推。
That gives the denoiser the backbone's full multi-scale understanding, not just a final summary of it.
这使得降噪塔获得了主干网络完整的尺度级理解能力，而不仅仅是其最终摘要。
This results in:
这带来了以下结果：
↳ 2.42x higher generation throughput ↳ Keeps 98.7% of the original model's quality ↳ Built on a 30B hybrid Mamba-Transformer MoE backbone ↳ Trained on just ~2.1T tokens, a fraction of the 25T used to pretrain the original
↳ 生成吞吐量提高 2.42 倍 ↳ 保留了原始模型 98.7% 的质量 ↳ 基于 30B 混合 Mamba-Transformer MoE 主干网络构建 ↳ 仅在约 2.1T 个 token 上训练，仅为预训练原始模型所用 25T token 的一小部分
Nothing here was trained from scratch.
这里没有任何东西是从头开始训练的。
It's an adaptation you build on top of an autoregressive model you already have, and it turns slow one-token-at-a-time decoding into fast parallel block generation, cheaply.
这是一种在你已有的自回归模型之上构建的适配方案，它以低成本将缓慢的逐个 token 解码转变为快速的并行块生成。
So if you're shipping anything latency-sensitive, check this blueprint. You no longer have to choose between the model you trust and the speed you need.
因此，如果你正在交付任何对延迟敏感的产品，请查看此蓝图。你不再需要在信任的模型和所需的速度之间做出选择。
Weights and code are all open-source.
权重和代码均已开源。
Link to the paper: https://arxiv.org/pdf/2606.26493
论文链接：https://arxiv.org/pdf/2606.26493
That said, if you want to go deeper, I wrote a detailed article that explains LLM inference from first principles, step by step.
话虽如此，如果你想深入了解，我写了一篇详细文章，从第一性原理出发，一步步解释 LLM 推理过程。
The article is quoted below.
文章引用如下。
更进一步：量化金融体系
看懂新闻只是起点——沿量化金融路径，把它变成能交付的工程能力