AKBASCORE MAM：在DeepSeek-V2-Lite等模型实现数值记忆
AKBASCORE MAM: Breaking the Text-Context Paradigm Third Model, DeepSeek-V2-Lite-Chat (MLA)
这是一份极具参考价值的工程实践，详细拆解了如何在DeepSeek MLA架构上落地数值记忆技术，代码与日志齐全，适合做长上下文优化的研究者直接复现。
AKBASCORE MAM has now been implemented on three different Transformer model families. The principle is simple: read information once, write it into numerical memory, consolidate it, and answer questions later without feeding the original source text back into the model (source-free numerical memory, append-only consolidation).
AKBASCORE MAM 现已在三个不同的 Transformer 模型系列中实现。其原理很简单：读取一次信息，将其写入数值记忆，进行巩固，然后在后续回答问题时不再将原始源文本重新输入模型（无源数值记忆，仅追加式巩固）。
This is the third experiment. I recommend following the first two in order, because understanding how we got here makes the DeepSeek implementation much easier to follow.
这是第三次实验。我建议按顺序先阅读前两篇，因为理解我们是如何走到这一步的，会让 DeepSeek 的实现过程更容易跟上。
- Mistral — mistralai/Mistral-7B-Instruct-v0.3 (DC6):
- https://www.reddit.com/r/LLMDevs/s/tg9qb80QLm
- Qwen — Qwen/Qwen2.5-7B-Instruct (DC3):
- https://www.reddit.com/r/LLMDevs/s/MQKoxzF6ng
- DeepSeek — deepseek-ai/DeepSeek-V2-Lite-Chat (MLA, DC3): this post.
- Mistral — mistralai/Mistral-7B-Instruct-v0.3 (DC6)：
- https://www.reddit.com/r/LLMDevs/s/tg9qb80QLm
- Qwen — Qwen/Qwen2.5-7B-Instruct (DC3)：
- https://www.reddit.com/r/LLMDevs/s/MQKoxzF6ng
- DeepSeek — deepseek-ai/DeepSeek-V2-Lite-Chat (MLA, DC3)：本文。
Each experiment includes its own numerical memory heatmap. You can compare all three by opening the posts in sequence. I'll also add a Markdown comparison table in the comments.
每次实验都包含其专属的数值记忆热力图。你可以通过依次打开这些帖子来比较三者。我还会在评论中添加一个 Markdown 对比表格。
Here is what changed between the three models:
以下是这三个模型之间的变化：
Mistral: 32 layers, 93 memory tokens, 11.62 MiB, 72/72 correct. Numerical intervention at layer 6 (H6 injection), with consolidation through layers 7–31.
Mistral：32 层，93 个记忆 token，11.62 MiB，正确率 72/72。在第 6 层进行数值干预（H6 注入），并通过第 7–31 层进行巩固。
Qwen: 28 layers, 82 memory tokens, 4.48 MiB, 72/72 correct. Intervention at layer 3 (H3 injection), with consolidation through layers 4–27.
Qwen：28 层，82 个记忆 token，4.48 MiB，正确率 72/72。在第 3 层进行干预（H3 注入），并通过第 4–27 层进行巩固。
DeepSeek: 27 layers, 87 memory tokens, 2.58 MiB, 69/72 correct. Intervention at layer 3 (H3 injection), with consolidation through layers 4–26.
DeepSeek：27 层，87 个记忆 token，2.58 MiB，正确率 69/72。在第 3 层进行干预（H3 注入），并通过第 4–26 层进行巩固。
The biggest architectural difference is DeepSeek's Multi-head Latent Attention (MLA).
最大的架构差异在于 DeepSeek 的多头潜在注意力机制（Multi-head Latent Attention, MLA）。
Mistral and Qwen use conventional key/value attention caches (K/V). DeepSeek stores a compressed 512-dimensional latent component (c_KV) and a separate 64-dimensional rotary key (k_R). Its heatmap therefore looks different because we're measuring different internal representations.
Mistral 和 Qwen 使用传统的键/值注意力缓存（K/V）。DeepSeek 存储一个压缩的 512 维潜在分量（c_KV）和一个独立的 64 维旋转键（k_R）。因此，它的热力图看起来不同，因为我们测量的是不同的内部表示。
The dark blue vertical bands in DeepSeek's heatmap appear inside the shared 28-token instruction prefix, not at cartridge boundaries. They represent relatively low numerical norms, not evidence of broken memory.
DeepSeek 热图中出现的深蓝色垂直带位于共享的 28 个 token 指令前缀内部，而非出现在卡槽边界处。它们代表相对较低的数值范数，并非记忆失效的证据。
DeepSeek answered 69 of 72 questions correctly. The three failures are documented and reproduced in TEST597. We haven't established a verified fix, and the heatmap alone doesn't explain those failures.
DeepSeek 正确回答了 72 个问题中的 69 个。那三个失败案例已在 TEST597 中记录并复现。我们尚未建立经过验证的修复方案，且仅凭热力图无法解释这些失败。
The 2.58 MiB footprint is also worth noting, but it reflects DeepSeek's compressed MLA cache architecture, not a directly measured compression gain produced by MAM.
2.58 MiB 的内存占用也值得注意，但它反映的是 DeepSeek 压缩后的 MLA 缓存架构，而非由 MAM 直接产生的压缩增益。
Three models, three architecture-specific implementations, and the same underlying mechanism. Model weights remain frozen throughout.
三个模型，三种针对特定架构的实现，以及相同的底层机制。模型权重在整个过程中保持冻结。
The results don't establish universal compatibility. They demonstrate that the numerical-memory consolidation principle can operate across three different Transformer implementations under the tested conditions.
这些结果并未确立通用兼容性。它们表明，在测试条件下，数值记忆巩固原则可以在三种不同的 Transformer 实现中运行。
Full executable DeepSeek code:
完整的可执行 DeepSeek 代码：
https://github.com/ceceli33/titan-cognitive-core-v2/blob/main/Deepseek\_full\_code\_demo.py
https://github.com/ceceli33/titan-cognitive-core-v2/blob/main/Deepseek_full_code_demo.py
Actual DeepSeek execution log:
实际的 DeepSeek 执行日志：
https://github.com/ceceli33/titan-cognitive-core-v2/blob/main/Deepseek\_demo.log
https://github.com/ceceli33/titan-cognitive-core-v2/blob/main/Deepseek_demo.log
DOI https://zenodo.org/records/23287752
更进一步：量化金融体系
看懂新闻只是起点——沿量化金融路径，把它变成能交付的工程能力