DLoop：循环式投机解码，提升推理速度5-41%
[Paper] DLoop: Looped Speculative Decoding
投机解码是本地部署大模型提效的核心技术，DLoop通过改变验证时机显著提升了加速比且无损精度，对追求极致推理速度的开发者极具参考价值。
Speculative decoding accelerates autoregressive generation in large language models. In each drafting stage, a lightweight draft model proposes tokens that the target model subsequently verifies. With increasingly capable draft models, we find that the target model frequently accepts all tokens produced in a drafting stage. A verification nevertheless follows each drafting stage, resulting in unnecessary target-model forward passes even when drafting could have continued. Adaptive draft length methods decide during decoding how many draft tokens precede a verification, but they raise the speedup only for autoregressive draft models. For a parallel draft model, drafting further requires target-model hidden states for draft tokens that have not been verified. We propose DLoop, a looped form of speculative decoding that adaptively performs multiple drafting stages before verification. DLoop continues drafting while the draft model remains confident and verifies all accumulated draft tokens together. Loop-aware training keeps the draft model reliable in the additional drafting stages by exposing it to its own hidden states for unverified draft tokens. By spending additional draft-model forward passes, DLoop reduces the number of target-model forward passes required for verification. Across diverse speculative decoding methods including EAGLE-3, DFlash, Domino, DSpark, and multi-token prediction modules, DLoop improves the wall-clock speedup by 5 to 41 percent while preserving lossless decoding. Code will be available at this https URL.
推测解码（Speculative decoding）加速了大语言模型中的自回归生成。在每一个草稿阶段，轻量级草稿模型会提出令牌（tokens），随后由目标模型进行验证。随着草稿模型能力的不断增强，我们发现目标模型经常接受在一个草稿阶段中产生的所有令牌。然而，每个草稿阶段之后仍会进行验证，导致即使草稿过程本可继续，也会产生不必要的目标模型前向传播。自适应草稿长度方法在解码过程中决定在验证之前有多少个草稿令牌，但它们仅针对自回归草稿模型提升加速比。对于并行草稿模型，进一步的草稿生成需要未验证草稿令牌的目标模型隐藏状态。我们提出了 DLoop，这是一种循环形式的推测解码，能够在验证之前自适应地执行多个草稿阶段。DLoop 在草稿模型保持自信时继续生成草稿，并一起验证所有累积的草稿令牌。感知循环的训练通过在额外的草稿阶段中将草稿模型暴露于其自身对未验证草稿令牌的隐藏状态，从而保持草稿模型的可靠性。通过消耗额外的草稿模型前向传播，DLoop 减少了验证所需的目标模型前向传播次数。在包括 EAGLE-3、DFlash、Domino、DSpark 以及多令牌预测模块在内的多种推测解码方法中，DLoop 在保持无损解码的同时，将实际运行时间加速比提高了 5% 到 41%。代码将在以下网址提供：this https URL。
- Paper : https://arxiv.org/abs/2610.07659
- PDF : https://arxiv.org/pdf/2610.07659
- GitHub : https://github.com/naver-ai/DLoop
- 论文：https://arxiv.org/abs/2610.07659
- PDF：https://arxiv.org/pdf/2610.07659
- GitHub：https://github.com/naver-ai/DLoop
The code is currently under internal review and will be released soon. Stay tuned!
代码目前正处于内部审查阶段，即将发布。敬请期待！
更进一步：量化金融体系
看懂新闻只是起点——沿量化金融路径，把它变成能交付的工程能力