亚马逊论文：微调策略匹配KV缓存可解决长上下文失效
New Amazon paper shows KV-cache policy is not just an inference optimization; bu…
长上下文失效是落地痛点，这篇论文给出了具体的微调方案而非仅做分析，有直接复现价值。
New Amazon paper shows KV-cache policy is not just an inference optimization; but it can change the training regime the model needs.
一篇新的亚马逊论文表明，KV缓存策略不仅仅是一种推理优化手段；它还能改变模型所需的训练范式。
If your LLM will forget parts of its context at inference, train it to forget that way too: this paper shows matching fine-tuning to the KV-cache policy can prevent long-context failures.
如果你的大语言模型在推理时会遗忘部分上下文信息，那么在训练时也让它以同样的方式遗忘：这篇论文表明，将微调过程与KV缓存策略相匹配，可以防止长上下文场景下的失效问题。
Sparse attention lets long-context inference use a fixed-size KV cache by keeping only part of the model’s past context. But models are often fine-tuned with full attention, then asked to work with missing memory at inference.
稀疏注意力机制通过仅保留模型部分历史上下文，使长上下文推理能够使用固定大小的KV缓存。但模型通常使用全注意力机制进行微调，随后却在推理时被要求处理缺失的记忆信息。
That mismatch can break behavior. In 128k-token tests, models trained with full attention often produced long, nonsensical answers under sparse inference, while models fine-tuned with the same cache policy learned to answer and stop normally.
这种不匹配可能导致行为异常。在128k token的测试中，使用全注意力机制训练的模型在稀疏推理下往往会产生冗长且无意义的回答，而采用相同缓存策略进行微调的模型则学会了正常回答并适时停止。
The method makes this policy-matched training practical for arbitrary cache policies, including computing gradients for a 4B model on a 40 GB A100.
该方法使得针对任意缓存策略的策略匹配训练变得切实可行，包括在配备40GB显存的A100 GPU上为4B参数模型计算梯度。
– arxiv. org/abs/2608.19920
– arxiv.org/abs/2608.19920
Title: "Learning how to Forget: Fine-tuning for Long-Context Sparse Attention"
标题：《学习如何遗忘：面向长上下文稀疏注意力的微调方法》
更进一步：量化金融体系
看懂新闻只是起点——沿量化金融路径，把它变成能交付的工程能力