DeepMind新论文：SkillSmith推理时组合技能

New Google DeepMind Paper.

Model weights do not have to remain static artifacts that can only be fine-tuned or averaged.

A new capability can now be initialized from both what an agent wrote down and what it previously learned into weights.

SkillSmith treats prefix key-value caches as another input modality.

For a new task, it interleaves existing prefix weights with task descriptions, examples, and a rationale explaining how the source capabilities relate to the target.

A forward pass then produces a fresh prefix cache for a frozen Gemma 3 4B model, so the initial composition happens at inference time rather than through a new target-specific training run.

The cleanest evidence is the ablation: on Composite-SNI, Elo is 1,455 with only K-V caches, 1,622 with only text, and 1,714 with both.

The gain cannot be explained by richer prompting or a learned weight merger alone.

The generated prefixes also give downstream fine-tuning a stronger starting point, especially on difficult, data-sparse MMLU-ProX tasks.

The boundary matters: on simpler SNI tasks with roughly 1,000 examples each, fine-tuned methods converge.

SkillSmith does not remove training, but it changes where adaptation starts: previous text and previous weights can now be composed into the initialization for a new capability.

– arxiv. org/abs/2607.27497

Title: "SkillSmith: Learning to Compose Parametric Skills and Textual Knowledge"