LittleLearner：控制预训练数据边界，研究模型能力获取
当 LLM 从未接触五年级以上材料会怎样？
Talk to LittleLearner
与LittleLearner对话
The hosted 5B model, live in your browser. Open in a new tab ↗ if the chat doesn’t load below.
托管的5B模型，在您的浏览器中实时运行。如果下方聊天未加载，请在新标签页中打开 ↗。
A controlled sandbox for studying how models acquire knowledge
一个受控沙盒，用于研究模型如何获取知识
Modern LMs are trained on everything at once, so it is hard to tell whether a new skill was learned or merely elicited. We constrain the training distribution itself: an 88B-token corpus filtered to the U.S. elementary-school curriculum, with models trained from scratch on it and matched unfiltered controls.
现代语言模型一次性在所有数据上训练，因此很难判断新技能是习得的还是仅仅被激发出来的。我们限制了训练分布本身：一个880亿词元的语料库，过滤至美国小学课程，模型从头开始训练，并匹配未过滤的对照组。
Dataset
数据集
LittleCurriculum
LittleCurriculum
An 88B-token corpus distilled from FineWeb-Edu through a five-stage filtering pipeline aligned with Common Core standards (K–5). Concepts, facts, and vocabulary taught above Grade 5 are explicitly excluded.
一个880亿词元的语料库，从FineWeb-Edu中通过五阶段过滤流程提炼而成，与共同核心标准（K-5）对齐。明确排除了五年级以上教授的概念、事实和词汇。
Models
模型
LittleLearner
LittleLearner
Three scales (0.6B / 1.3B / 5B) trained from scratch on LittleCurriculum: chattable models with an interpretable knowledge boundary. Each ships with a matched Unfiltered control for clean comparison.
三种规模（0.6B / 1.3B / 5B）在LittleCurriculum上从头训练：具有可解释知识边界的可聊天模型。每个模型都配有匹配的未过滤对照组，以便进行清晰比较。
Findings
研究发现
Elicitation, not acquisition
激发而非习得
In our experiments, scaling, SFT+GRPO post-training, and in-context learning amplify what the curriculum taught, but none meaningfully improves out-of-scope performance, indicating that the pretraining filter sets the effective capability ceiling.
在我们的实验中，扩展、SFT+GRPO后训练和上下文学习放大了课程所教授的内容，但均未显著改善范围外的表现，表明预训练过滤器设定了有效能力上限。
Model checkpoints
模型检查点
LittleLearner at three scales (0.6B / 1.3B / 5B), each with a matched Unfiltered control sharing its architecture, tokens, and recipe.
三种规模（0.6B / 1.3B / 5B）的LittleLearner，每个都配有匹配的未过滤对照组，共享架构、词元和配方。
Base: the pretrained model.
基础：预训练模型。
GRPO: math specialists post-trained on MathCAMPS; responses may exhibit a tendency toward math-oriented output.
GRPO：在MathCAMPS上后训练的数学专家；回答可能倾向于数学导向的输出。
Chatty: variants tuned for general chat behavior.
Chatty：针对通用聊天行为调整的变体。
Scale | LittleLearner · K–5 | chatty | Matched control · unfiltered
规模 | LittleLearner · K-5 | chatty | 匹配对照 · 未过滤
Capability stays inside the curriculum
能力保持在课程范围内
Can standard interventions push a model past what its pretraining data taught it? With the boundary under experimental control, we can ask cleanly. In our experiments, each intervention amplifies in-scope ability; none of them meaningfully improves out-of-scope performance.
标准干预能否将模型推过其预训练数据所教的内容？在边界受实验控制的情况下，我们可以清晰地提问。在我们的实验中，每种干预都放大了范围内的能力；但均未显著改善范围外的表现。
Scaling Post-training In-context learning
扩展 后训练 上下文学习
Scaling
扩展
Scaling model size improves performance within the model’s controlled knowledge exposure and extends modestly to problems along the same learning trajectory, but yields little improvement on problems requiring more advanced capabilities outside the exposure.
扩展模型大小可提高模型在受控知识暴露范围内的性能，并适度延伸至同一学习轨迹上的问题，但对于需要超出暴露范围的更高级能力的问题，几乎没有改进。
MathCAMPS accuracy by grade, across model size
MathCAMPS 按年级、模型规模的准确率
Post-training
后训练
Post-training through GRPO significantly boosts in-scope K–5 capabilities, but fails to recover out-of-scope beyond-K–5 capabilities, even when training with out-of-scope data.
通过 GRPO 进行的后训练显著提升了 K–5 范围内的能力，但即使使用范围外的数据进行训练，也无法恢复超出 K–5 范围的能力。
Post-training amplifies K–5, not the beyond-K–5 gap
后训练放大了 K–5 能力，而非超出 K–5 的差距
In-context learning
上下文学习
In-context learning with the prompts we test does not unlock new reasoning capabilities in beyond-K–5 for our trained 5B LittleLearner.
在我们测试的提示条件下，上下文学习并未为我们训练的 5B LittleLearner 解锁超出 K–5 的新推理能力。
Accuracy by prompting condition
按提示条件的准确率
What will you teach it?
你会教它什么？
Because LittleLearner’s training exposure is explicitly specified, behavioral and representational changes can be related directly to the concepts you introduce. Three directions we’re excited about:
由于 LittleLearner 的训练暴露是明确指定的，行为和表征的变化可以直接与您引入的概念相关联。我们感兴趣的三个方向是：
01
01
RL & discovery
强化学习与发现
Can RL create capability?
强化学习能创造能力吗？
The prior is restricted to K–5, so capabilities that emerge under RL can be attributed to the RL process itself. A tractable proxy for reward-driven discovery.
先验被限制在 K–5 范围内，因此强化学习中出现的能力可以归因于强化学习过程本身。这是奖励驱动发现的可行代理。
02
02
Continual learning
持续学习
Watch a concept being learned
观察概念的学习过程
Introduce negative numbers and measure sample efficiency, retention, and interference. Or probe behavior near the boundary: does it answer, abstain, or hallucinate?
引入负数并测量样本效率、保留和干扰。或者探测边界附近的行为：它会回答、弃权还是产生幻觉？
03
03
Educational science
教育科学
Machine vs. child learners
机器与儿童学习者
Specified exposure enables controlled human-model comparison. Do models and children need similar exposure to learn fractions, or make similar errors on word problems?
指定的暴露使得受控的人机比较成为可能。模型和儿童是否需要类似的暴露来学习分数，或者在应用题上犯类似的错误？
＋
＋
Your turn
轮到你了
Bring your own question
带上你自己的问题
A known boundary turns your idea into a clean experiment!
已知的边界将您的想法变成清晰的实验！
If you find this work useful
如果您觉得这项工作有用
Please cite our paper:
请引用我们的论文：
Copy@misc{littlelearner2026,
title={LittleLearner: Language Models Under Pedagogically-Controlled Knowledge Exposure},
author={Fanfei Li and Jana Zeller and Manuel Prada-Corral and Thaddäus Wiedemer and Prasanna Mayilvahanan and Ryan Cotterell and Wieland Brendel},
year={2026},
eprint={2608.13545},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2608.13545}
}Copy@misc{littlelearner2026,
title={LittleLearner: Language Models Under Pedagogically-Controlled Knowledge Exposure},
author={Fanfei Li and Jana Zeller and Manuel Prada-Corral and Thaddäus Wiedemer and Prasanna Mayilvahanan and Ryan Cotterell and Wieland Brendel},
year={2026},
eprint={2608.13545},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2608.13545}
}更进一步：量化金融体系
看懂新闻只是起点——沿量化金融路径，把它变成能交付的工程能力