将NES游戏机编译进Qwen3.5模型权重中运行
Show HN: A working Nintendo in ~1GB of Qwen weights
极具创意的工程实践，详细拆解了如何将通用大模型转化为特定硬件模拟器，包含完整的编译原理与配置细节，值得AI工程师研究模型能力的边界。
nesformer
An NES CPU compiled into the weights of a stock Qwen3.5 model.
将 NES CPU 编译为预训练 Qwen3.5 模型的权重。
The NES's 6502 CPU (the core of its 2A03 chip), its 2 KB of work RAM, and a cartridge's program ROM run inside a standard Qwen3_5ForCausalLM checkpoint. Nobody trained these weights; a compiler computed every one. When you play a game, the model predicts the CPU's execution log one token at a time. A small host program plays the rest of the console around it: it draws the picture, reads the controller, and fires the vblank interrupt.
NES 的 6502 CPU（其 2A03 芯片的核心）、2 KB 的工作 RAM，以及卡带中的程序 ROM，运行在一个标准的 Qwen3_5ForCausalLM 检查点内。这些权重并非经过训练得到；每一个权重均由编译器计算得出。当你玩游戏时，模型会一次预测一个 token 的 CPU 执行日志。一个小型宿主程序负责模拟控制台的其他部分：它绘制画面、读取控制器输入并触发 vblank 中断。
nesformer builds on Qwenputer by Divyansh Agrawal, which compiled a CHIP-8 and then a RISC-V CPU into Qwen3.5-4B. It reuses Qwenputer's compiler and its central trick: the model's Gated DeltaNet state is the machine's memory.
nesformer 建立在 Divyansh Agrawal 的 Qwenputer 之上，后者将 CHIP-8 和随后的 RISC-V CPU 编译到了 Qwen3.5-4B 中。它复用了 Qwenputer 的编译器及其核心技巧：模型的 Gated DeltaNet 状态即为机器的内存。
Super Mario Bros. (World 1-1) | Full Game by Doug Fraker (title screen and first level)
超级马里奥兄弟（世界 1-1）| 完整游戏由 Doug Fraker 制作（标题画面和第一关）
Every CPU token in these runs came from the model. For the bundled test games, every frame also matches a conventional 6502 emulator's output exactly (see How it was checked).
这些运行中的所有 CPU token 均来自模型。对于捆绑的测试游戏，每一帧的输出也与传统的 6502 模拟器完全一致（参见“如何验证”）。
Quick start
快速开始
1. Folder layout
1. 文件夹布局
Unzip nesformer.zip and nesformer-weights.zip into the same folder. The weights go next to the nesformer/ package, not inside it:
将 nesformer.zip 和 nesformer-weights.zip 解压到同一文件夹中。权重文件应位于 nesformer/ 包旁边，而不是在其内部：
nesformer/ <- run commands from here
├── nesformer/ <- the Python package
├── chipformer/ <- Qwenputer's compiler
├── roms/ <- test ROMs (put your own here too)
├── examples/ <- sample GIFs and frame sheets
├── tests/
├── LICENSE
├── LICENSE.chipformer
├── README.md
├── pyproject.toml
└── weights/ <- from nesformer-weights.zip
├── config.json
├── generation_config.json
├── model.safetensors
├── nesformer.json
└── SHA256SUMSThe CLI looks for weights/ beside the nesformer/ package. To keep the weights elsewhere, pass --weights /path/to/weights to run, check or trace, or give the path to info.
命令行界面（CLI）会在 nesformer/ 包旁边查找 weights/ 目录。若要将权重存放在其他位置，请向 run、check 或 trace 命令传递 --weights /path/to/weights，或向 info 命令提供该路径。
To confirm the weights unzipped correctly, run shasum -a 256 -c weights/SHA256SUMS from the top folder.
要确认权重已正确解压，请在顶层文件夹中运行 shasum -a 256 -c weights/SHA256SUMS。
2. Install
2. 安装
You need Python 3.11 or newer.
你需要 Python 3.11 或更高版本。
python3 -m venv .venv
source .venv/bin/activate # Windows: .venv\Scripts\activate
pip install numpy scipy numba pillow3. Play a game
3. 玩游戏
python -m nesformer run roms/thwaite.nes --frames 600 --show \
--buttons "start@90-95+200-205,a@230-233+250-253+270-273+290-293+310-313+330-333+350-353+370-373+390-393+410-413+430-433+450-453+470-473+490-493+510-513+530-533+550-553+570-573+590-593"--show draws each frame in your terminal as the model generates it. When the run ends, the CLI saves an animated GIF named after the ROM (here thwaite.gif) in the current folder.
--show 参数会在终端中逐帧显示模型生成的画面。当运行结束时，CLI 会将动画 GIF 保存至当前文件夹，文件名与 ROM 同名（此处为 thwaite.gif）。
Expect it to be slow. The first run spends about 20 seconds compiling the engine with numba. After that, a frame takes 1–6 seconds, so 600 frames (10 seconds of NES time) take around 25 minutes.
预计速度会很慢。首次运行需要约 20 秒来使用 numba 编译引擎。此后，渲染一帧需要 1–6 秒，因此 600 帧（相当于 NES 时间的 10 秒）大约需要 25 分钟。
Playing your own ROMs
玩你自己的 ROM
Point run at any .nes file that uses the NROM board (iNES mapper 0), such as Super Mario Bros., Donkey Kong or Excitebike. You supply the ROM; nesformer only ships the three freely licensed test ROMs in roms/.
将 run 指向任何使用 NROM 板（iNES mapper 0）的 .nes 文件，例如《超级马里奥兄弟》、《大金刚》或《越野摩托》。你需提供 ROM 文件；nesformer 仅随附三个拥有自由许可的测试 ROM，位于 roms/ 目录下。
python -m nesformer run roms/smb.nes --frames 600 --show \
--buttons "start@60-65,right@200-600,a@250-265+300-315+360-375+420-435+480-495+540-555"That script starts the game, then walks Mario right and jumps about once a second. The CLI loads the ROM into the model's state at startup, so the same weights run any NROM game without a rebuild. Games on other boards stop with a message: see Limitations.
该脚本启动游戏后，控制马里奥向右移动并每秒跳跃一次。CLI 在启动时将 ROM 加载到模型的状态中，因此相同的权重无需重新编译即可运行任何 NROM 游戏。其他板子的游戏会停止并显示消息：详见“局限性”。
Controller scripts
控制器脚本
--buttons takes a comma-separated list of entries. Each entry names one or more buttons joined by +, then @, then one or more frame ranges joined by +:
--buttons 接受一个由逗号分隔的条目列表。每个条目命名一个或多个通过 + 连接的按钮，然后是 @，接着是一个或多个通过 + 连接的视频帧范围：
start@60-65 press Start during frames 60 to 65
right@200-600 hold Right from frame 200 to frame 600
a@250-265+300-315 press A twice
right+b@200-600 hold Right and B together (run, in Super Mario Bros.)The buttons are a, b, select, start, up, down, left and right. Frames count from power-on at 60 per second, so frame 600 is 10 seconds in. Games ignore input on screens where you can't move, so holding a button a little early does no harm. In Super Mario Bros., a longer A press makes a higher jump.
按钮包括 a、b、select（选择）、start（开始）、up（上）、down（下）、left（左）和 right（右）。帧数从开机时开始计算，每秒 60 帧，因此第 600 帧表示过去了 10 秒。游戏会忽略在无法移动的画面上的输入，因此稍微提前按住按钮不会造成危害。在《超级马里奥兄弟》中，长按 A 键可以跳得更高。
Commands
命令
| Command | What it does |
|---|---|
| python -m nesformer run ROM | Play a ROM on the weights and save a GIF |
| python -m nesformer check ROM | Run the weights and the reference CPU side by side; stop at the first difference |
| python -m nesformer trace ROM | Print the model's token stream (default 40 tokens) |
| python -m nesformer build | Compile the circuit and write weights/ (about 10 seconds) |
| python -m nesformer info | Describe the checkpoint |
| 命令 | 功能 |
|---|---|
| python -m nesformer run ROM | 使用权重播放 ROM 并保存 GIF |
| python -m nesformer check ROM | 并行运行权重和参考 CPU；在出现第一个差异时停止 |
| python -m nesformer trace ROM | 打印模型的 token 流（默认 40 个 token） |
| python -m nesformer build | 编译电路并写入 weights/（约 10 秒） |
| python -m nesformer info | 描述检查点 |
Options for run:
run 的选项：
| Option | Default | Meaning |
|---|---|---|
| --frames N | 120 | Stop after N frames |
| --buttons SCRIPT | none | Controller input (see above) |
| --show | off | Draw frames in the terminal (needs 24-bit color, e.g. iTerm2 or Windows Terminal) |
| --gif PATH | ROM name | Where to save the animated GIF |
| --png DIR | off | Save each kept frame as a PNG in DIR. With --png alone, no GIF is saved. Add --gif to get both. |
| --every N | 1 | Keep every N-th frame |
| --scale N | 2 | Enlarge saved frames N times |
| --engine | fast | fast (numba), numpy, or hf (stock transformers; needs torch, ~10 tokens/s) |
| --no-skip | off | Run every iteration of idle wait loops (slower; identical frames) |
| --max-tokens N | none | Stop after N tokens |
| --trace FILE | none | Save the token ids |
| 选项 | 默认值 | 含义 |
|---|---|---|
| --frames N | 120 | 在 N 帧后停止 |
| --buttons SCRIPT | none | 控制器输入（见上文） |
| --show | off | 在终端中绘制帧（需要 24 位颜色，例如 iTerm2 或 Windows Terminal） |
| --gif PATH | ROM name | 保存动画 GIF 的位置 |
| --png DIR | off | 将每个保留的帧另存为 DIR 中的 PNG。仅使用 --png 时，不保存 GIF。添加 --gif 可同时获取两者。 |
| --every N | 1 | 每第 N 帧保留一帧 |
| --scale N | 2 | 将保存的帧放大 N 倍 |
| --engine | fast | fast (numba)、numpy 或 hf (stock transformers；需要 torch，约 10 个 token/s) |
| --no-skip | off | 运行空闲等待循环的每次迭代（较慢；帧相同） |
| --max-tokens N | none | 在 N 个 token 后停止 |
| --trace FILE | none | 保存 token id |
run, check and trace all accept --weights, --engine, --buttons and --no-skip. check --tokens N sets how far to compare (default 50,000). trace --tokens N sets how many tokens to print (default 40). build --out DIR writes the checkpoint somewhere other than weights/.
run、check 和 trace 均接受 --weights、--engine、--buttons 和 --no-skip。check --tokens N 设置比较的深度（默认 50,000）。trace --tokens N 设置要打印的 token 数量（默认 40）。build --out DIR 将检查点写入 weights/ 以外的位置。
How it works
工作原理
What's in the weights, and what isn't
权重中包含什么，不包含什么
| In the model (weights + DeltaNet state) | In the host (nesformer/host.py) |
|---|---|
| The 6502 core: all 151 official opcodes, every addressing mode, flags, stack, BRK and NMI entry | The PPU: a scanline renderer and the palette |
| 2 KB of work RAM | Controllers; APU registers (written, otherwise ignored) |
| 32 KB of PRG-ROM, stored twice: one copy for instruction fetch, one for data reads | NMI timing at vblank, injected as an NMI token |
| OAM DMA reads (the 256 bytes appear as DM tokens) | Answers to I/O reads, injected as IN tokens |
| 在模型中（权重 + DeltaNet 状态） | 在主机中（nesformer/host.py） |
|---|---|
| 6502 核心：所有 151 条官方操作码、每种寻址模式、标志位、堆栈、BRK 和 NMI 入口 | PPU：扫描线渲染器和调色板 |
| 2 KB 的工作 RAM | 控制器；APU 寄存器（写入，否则被忽略） |
| 32 KB 的 PRG-ROM，存储两次：一份用于指令获取，一份用于数据读取 | vblank 期间的 NMI 定时，作为 NMI token 注入 |
| OAM DMA 读取（256 字节显示为 DM token） | I/O 读取的答案，作为 IN token 注入 |
This split follows Qwenputer's CHIP-8 build, whose host injects timer ticks and draws the display from the trace. The host never sees the CPU's registers or RAM except through the trace.
此拆分遵循 Qwenputer 的 CHIP-8 构建，其主机从追踪记录中注入计时器滴答声并绘制显示。除了通过追踪记录外，主机从未看到 CPU 的寄存器或 RAM。
The memory trick works like this. Each DeltaNet head keeps a 128 × 128 state matrix that the delta rule updates at every token. With the gate held open and a one-hot key, an update overwrites exactly one row of that matrix, and a one-hot query reads it back. So every row is a memory cell, and the cartridge, RAM and registers all live in rows.
内存技巧的工作原理如下。每个 DeltaNet 头都维护一个 128 × 128 的状态矩阵，delta 规则在每个 token 处更新该矩阵。当门保持开启且使用 one-hot 键时，更新会覆盖该矩阵的恰好一行，而 one-hot 查询则将其读回。因此，每一行都是一个存储单元，卡带、RAM 和寄存器都存在于这些行中。
The checkpoint
检查点
weights/ is an ordinary Hugging Face folder. It loads with Qwen3_5ForCausalLM.from_pretrained and needs no custom code.
weights/ 是一个普通的 Hugging Face 文件夹。它可以通过 Qwen3_5ForCausalLM.from_pretrained 加载，无需自定义代码。
| Architecture | Qwen3_5ForCausalLM, with Qwen3.5-4B's DeltaNet head shapes (16 key heads, 32 value heads, dim 128) and MLP width (9,216) |
|---|---|
| Layers | 7 Gated DeltaNet layers (the circuit), plus 1 all-zero attention layer that Hugging Face's cache needs |
| Hidden size | 768 |
| Vocabulary | 263,808 tokens |
| Parameters | 466.6M in bf16 (933 MB), of which 9.3M are nonzero, so the weights zip to 3.3 MB |
| 架构 | Qwen3_5ForCausalLM，采用 Qwen3.5-4B 的 DeltaNet 头形状（16 个 key 头，32 个 value 头，维度 128）和 MLP 宽度（9,216） |
|---|---|
| 层数 | 7 个 Gated DeltaNet 层（电路），加上 Hugging Face 缓存所需的 1 个全零注意力层 |
| 隐藏层大小 | 768 |
| 词汇表 | 263,808 个 token |
| 参数量 | bf16 格式下为 466.6M（933 MB），其中非零参数为 9.3M，因此权重压缩后为 3.3 MB |
nesformer.json records which layer and head holds each part of the machine. A run writes the cartridge, registers and RAM into those DeltaNet states, then decodes greedily.
nesformer.json 记录了机器的各个部分位于哪一层和哪个头。运行过程将卡带、寄存器和 RAM 写入这些 DeltaNet 状态，然后进行贪婪解码。
Layer plan
层级规划
更进一步：量化金融体系
看懂新闻只是起点——沿量化金融路径，把它变成能交付的工程能力