冻结主干只训投影层,可为语言模型添加新模态且不损伤已有能力
You can teach a language model a new modality without ever touching the backbone…
You can teach a language model a new modality without ever touching the backbone's weights.
你可以教语言模型一种新模态,而无需触碰骨干网络的权重。
Train only the projector between encoder and backbone, and the capabilities you already shipped cannot regress.
仅训练编码器与骨干网络之间的投影器,你已发布的能力就不会退化。
To add a new modality to a language model, you only need to train the small projector that maps the encoder's output into the model's embedding space — fine-tuning the language model itself adds nothing reliable and wrecks its existing skills.
要为语言模型添加新模态,你只需训练那个将编码器输出映射到模型嵌入空间的小型投影器——微调语言模型本身既不可靠,还会破坏其现有技能。
In their 3D tests, frozen-backbone models matched or beat jointly fine-tuned ones while training twice as fast, and the jointly fine-tuned Llama backbone collapsed from 86.96% to 0.61% on GSM8K.
在他们的3D测试中,冻结骨干网络的模型与联合微调的模型表现相当或更优,同时训练速度快两倍,而联合微调的Llama骨干网络在GSM8K上的准确率从86.96%暴跌至0.61%。
– arxiv. org/abs/2608.19726
– arxiv.org/abs/2608.19726
Title: "Projector Is All You Train"
标题:“投影器即全部训练”
更进一步:量化金融体系
看懂新闻只是起点——沿量化金融路径,把它变成能交付的工程能力