EngramEdit：通过条件记忆解耦LLM知识更新
[Paper] EngramEdit: Decoupled Knowledge Updates in LLMs through Conditional Memory
针对大模型知识更新痛点提出的全新范式，实现事实与计算解耦且保留通用能力，对 Agent 知识库维护极具参考价值。
Conditional memory architectures such as DeepSeek Engram use input n-grams to look up learned embeddings, expanding the capacity of large language models (LLMs) with limited additional computation. Beyond model scaling, this architecture has demonstrated the potential to decouple factual knowledge storage from general-purpose computation, offering a promising route to updating factual knowledge while keeping the Transformer backbone fixed. Realizing this potential is challenging because different expressions of a fact may activate different n-gram embeddings, while updating shared embeddings can unintentionally change the model's predictions about other facts. We propose EngramEdit for decoupled knowledge updates through conditional memory. EngramEdit first computes target memory representations that make the model predict the updated fact across multiple expressions. It then jointly updates the shared n-gram embeddings to match these targets across expressions and edits, penalizing updates to frequently reused embeddings more strongly to preserve unrelated knowledge. Experiments show that EngramEdit enables independent factual knowledge updates through conditional memory, achieving near-perfect editing success. Revised knowledge is usable across unseen expressions and in multi-hop reasoning, with nearly three times the strongest baseline's accuracy under chain-of-thought (CoT) prompting. Unrelated knowledge and general capabilities are largely preserved even as factual updates accumulate. These findings show that EngramEdit turns conditional memory into an editable knowledge interface, extending its role beyond model scaling to support decoupled knowledge updates.
条件记忆架构（如 DeepSeek Engram）使用输入 n-gram 来检索已学习的嵌入，从而在增加有限额外计算量的情况下扩展大语言模型 (LLM) 的容量。除了模型规模扩展之外，该架构还展示了将事实知识存储与通用计算解耦的潜力，为在保持 Transformer 主干固定的同时更新事实知识提供了一条有前景的路径。实现这一潜力具有挑战性，因为同一事实的不同表述可能激活不同的 n-gram 嵌入，而更新共享嵌入可能会无意中改变模型对其他事实的预测。我们提出了 EngramEdit，通过条件记忆实现解耦的知识更新。EngramEdit 首先计算目标内存表示，使模型能够针对多种表述预测更新后的事实。随后，它联合更新共享的 n-gram 嵌入以跨表述和编辑匹配这些目标，并对频繁重用的嵌入施加更强的更新惩罚，以保留无关知识。实验表明，EngramEdit 通过条件记忆实现了独立的事实知识更新，达到了近乎完美的编辑成功率。修订后的知识可用于未见过的表述和多跳推理中，在思维链 (CoT) 提示下，其准确率是最强基线的近三倍。即使事实更新不断累积，无关知识和通用能力也得到了 largely 保留。这些发现表明，EngramEdit 将条件记忆转变为可编辑的知识接口，将其作用从模型规模扩展延伸至支持解耦的知识更新。
- Paper : https://arxiv.org/abs/2610.10533
- PDF : https://arxiv.org/pdf/2610.10533
- GitHub : https://github.com/ModalityDance/EngramEdit
- HuggingFace : https://huggingface.co/collections/ModalityDance/engramedit
- Project : https://modalitydance.github.io/EngramEdit/
- 论文：https://arxiv.org/abs/2610.10533
- PDF：https://arxiv.org/pdf/2610.10533
- GitHub：https://github.com/ModalityDance/EngramEdit
- HuggingFace：https://huggingface.co/collections/ModalityDance/engramedit
- 项目主页：https://modalitydance.github.io/EngramEdit/
更进一步：量化金融体系
看懂新闻只是起点——沿量化金融路径，把它变成能交付的工程能力