斯坦福新方法Prefix Sliding:长推理提速3倍且不降性能
Banger paper from Stanford on efficient test-time scaling.
做长推理Agent的同学必看,这个Prefix Sliding方法直接砍掉中间token,3倍提速还不掉点,赶紧拿你的长链路试试。
Banger paper from Stanford on efficient test-time scaling.
斯坦福大学关于高效测试时扩展的精彩论文。
If you run agents that think for a long time, this one is worth your time.
如果你运行需要长时间思考的智能体,这篇论文值得你花时间阅读。
(bookmark it)
(收藏起来)
Long reasoning keeps the entire trace in memory through full attention.
长推理过程将整个轨迹保留在内存中,并通过全注意力机制处理。
This means that the hardest problems, the ones that need the most thinking, are also the ones that cost the most to run.
这意味着最困难的问题,即那些需要最多思考的问题,也是运行成本最高的。
The authors measured what the middle of a reasoning trace is actually worth.
作者测量了推理轨迹中间部分实际的价值。
Intermediate tokens steadily lose importance as the model keeps going.
随着模型继续运行,中间令牌的重要性逐渐降低。
Their new approach, Prefix Sliding, drops those tokens. It keeps the prefix, which holds the instructions and the available tools, plus a window of the last few thousand tokens. Everything in between gets discarded during generation.
他们的新方法“前缀滑动”会丢弃这些令牌。它保留前缀,其中包含指令和可用工具,以及最后几千个令牌的窗口。生成过程中,中间的所有内容都会被丢弃。
Total memory stays capped no matter how long the model reasons.
无论模型推理多长时间,总内存都保持上限。
Without any training, this runs existing models 3x faster while matching full-attention performance, and it enables RL rollouts past 100,000 tokens.
无需任何训练,这种方法就能让现有模型运行速度提升3倍,同时匹配全注意力性能,并支持超过10万令牌的强化学习回滚。
Paper: https://arxiv.org/abs/2608.26070
论文:https://arxiv.org/abs/2608.26070
Chat with Paper: https://academy.dair.ai/papers/prefix-sliding-for-efficient-test-time-scaling-2608.26070
与论文对话:https://academy.dair.ai/papers/prefix-sliding-for-efficient-test-time-scaling-2608.26070
更进一步:量化金融体系
看懂新闻只是起点——沿量化金融路径,把它变成能交付的工程能力