新论文提出无需重写模型参数的知识注入方法

New paper from @NaceAI shows a possible way to add large bodies of knowledge without rewriting the language model’s core parameters.

The study advances a key continual-learning goal: teaching a model new facts without directly disturbing its existing parameters.

i.e. the method may reduce the need to repeatedly alter billions of base-model parameters whenever knowledge changes.

The big idea is that one model can learn how to manufacture knowledge updates for another model.

The base model’s weights never move, but extra low-rank matrices are added alongside them. Those matrices carry the injected information and redirect the model’s internal computation toward answers supported by the facts.

Knowledge injection here becomes a reusable process rather than a separate fine-tuning job for every factual corpus.

The method separates storing new facts from changing the language model that must reason with them.

A frozen language model may absorb new knowledge through generated adapters while preserving its original parameters.

The hypernetwork acts like a compiler that converts facts into temporary changes inside another model.

The strongest result is that these generated updates generalized better as the target language model became larger.