多任务贝叶斯上下文学习：模型从示例中学习先验

A transformer based model can learn a task’s starting assumptions from example datasets and adjust predictions without changing its weights.

Most prior-fitted models hide one prior inside their trained parameters, so unfamiliar settings can produce brittle predictions.

The paper found one trained model can adapt to different environments by reading examples that show what usually happens there.

The new method , proposed in this paper, places several related datasets before the target examples, using them as the prior, or starting belief.

A decoder-only transformer learns how tasks share hidden structure, then uses that structure to predict the target task.

Across regression tests, heavy-tailed shifts, complex priors, and climate data, it often matched Bayesian reference predictions.

It also ran orders of magnitude faster than sampling-based inference, though severe seasonal shifts still exposed important limits.

– arxiv. org/abs/2606.20538

Title: "Multi-Task Bayesian In-Context Learning"