Metis：将记忆内化进大模型骨干的持久状态

This is such a wild idea.

What if memory were a capability of the LLM itself, rather than a retrieval system bolted around it?

Can an LLM remember something from an earlier interaction without having to feed that old interaction back into its prompt?

Metis proposes a foundation model with a persistent memory state inside the backbone, updated during ordinary forward passes while the learned model weights stay frozen.

Instead of storing memories as text and retrieving them later, Metis compresses past interactions directly into the model's internal state. through a separate memory-attention path.

Normal LLMs don’t keep a persistent memory of past interactions. If you tell one “Alice lives in Beijing,” an external memory system usually has to save and retrieve that fact later. Metis instead compresses the interaction into an internal numerical memory state that stays inside the model and can be used in future calls.

The model is mid-trained to learn memory procedures such as remembering, updating, forgetting, and reflecting, rather than relying on hand-written retrieval rules.

Under the no-context setting, Metis-27B scores 26.74 on LoCoMo (Gold), versus 0.07 for the vanilla Qwen3.5-27B backbone and 4.24 for Temp-LoRA-27B.

It is still well below Qwen3.5-27B with full context at 65.03, so native memory has not replaced replaying history.

The failure mode is also revealing: performance drops as more information is compressed into the fixed-size state, and irrelevant stored memory can interfere with ordinary tasks.

The bigger direction here is making memory part of model computation itself, but Metis also exposes the hard part: that internal state has to stay selective, stable, and non-interfering.