Qwen3.8-27B本地复现权重空间编程：直接读取并修改模型权重
Like GPT-6 Astra, Qwen 3.8 27B can interpret model weights
给本地部署玩家看，Qwen3.8-27B就能跑通最新的权重空间编程实验，附完整命令和踩坑细节，值得收藏压测。
Simon Willison’s Weblog
Simon Willison 的博客
Newsletter
通讯
Sponsored by: Greptile — Your code reviewer uses Computer Use to catch bugs by opening your app and clicking through it. Now you can configure its test environment and define the key user journeys it tests. Try it for free
赞助商：Greptile — 您的代码审查员使用 Computer Use，通过打开您的应用并逐步点击来发现错误。现在您可以配置其测试环境并定义它测试的关键用户旅程。免费试用
Qwen 3.8 27B on my laptop can interpret model weights, too
我笔记本电脑上的 Qwen 3.8 27B 也能解析模型权重
3rd November 2026
2026年11月3日
Yesterday’s hot paper was Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights by Rikiya Takehi, Jayden Teoh, Alex Zhang, and Omar Khattab. The paper found that frontier models, in particular GPT-6 Astra, can both interpret and modify the raw weights of a machine learning model—literally a text description of the model containing a bunch of floating point numbers. I’ve been playing with one of the examples from the paper and I’m delighted to share that Qwen 3.8 27B running on my laptop can do this as well!
昨天的热门论文是《Weight-Space Programming 的火花：LLM 可以通过 Rikiya Takehi、Jayden Teoh、Alex Zhang 和 Omar Khattab 原生地读取和写入神经网络权重》。该论文发现，前沿模型（特别是 GPT-6 Astra）既能解释又能修改机器学习模型的原始权重——这实际上是包含一堆浮点数的模型文本描述。我一直在玩这篇论文中的一个示例，我很高兴分享说，运行在我笔记本电脑上的 Qwen 3.8 27B 也能做到这一点！
Weight-Space Programming
权重空间编程
Here’s the highlight from their abstract, describing impressive capabilities for GPT-6 Astra and coining the term Weight-Space Programming for when LLMs analyze and modify model weights:
以下是他们摘要中的亮点，描述了 GPT-6 Astra 令人印象深刻的功能，并为 LLM 分析和修改模型权重的情况创造了“权重空间编程”这一术语：
Just by looking at the raw weights, GPT-6 Astra can tell what a network was trained to do. And without any environment feedback, it has no trouble writing, from scratch, small game-playing networks that outplay networks trained with PPO for 100 million steps.
仅通过查看原始权重，GPT-6 Astra 就能告诉人们一个网络被训练用来做什么。而且无需任何环境反馈，它在从头编写小型游戏网络方面毫不费力，这些网络的表现优于经过 1 亿步 PPO 训练的网络。
Likewise, asked to make Qwen3.5-0.8B better at math without gradients, Astra identifies an issue in its weights causing reasoning loops that Qwen’s own model card warns about, and fixes this by natively editing 0.016% of the weights. With the same sampling settings, this weight correction raises MATH-500 from 7.9% to 48.1% and generalizes out of the box to instruction following, raising IFEval by 3x.
同样，当被要求在不使用梯度的情况下让 Qwen3.5-0.8B 更擅长数学时，Astra 识别出其权重中导致推理循环的问题（Qwen 自己的模型卡对此发出警告），并通过原生编辑 0.016% 的权重来修复此问题。在相同的采样设置下，这种权重修正将 MATH-500 从 7.9% 提高到 48.1%，并且开箱即用地泛化到指令遵循，使 IFEval 提高了 3 倍。
All of this is possible without allowing Astra to write training loops. It uses its reasoning to figure out what each part of a neural network computes, or should compute, and changes the weights accordingly. We call this Weight-Space Programming.
所有这些都是在不允许 Astra 编写训练循环的情况下实现的。它利用其推理能力来确定神经网络的每个部分计算什么或应该计算什么，并相应地更改权重。我们称之为权重空间编程。
The paper combines two topics that I think deserve to be treated separately. The first is what models can do with raw weights without running tools or executing code. The second explores how well models, once armed with code, can run operations to update and improve other models.
这篇论文结合了两个我认为应该分开处理的主题。第一个是模型在不运行工具或执行代码的情况下能对原始权重做什么。第二个探讨了模型在配备代码后，运行操作以更新和改进其他模型的效果如何。
spaCy author Matthew Honnibal called the paper misleading for conflating the two, and Sean Goedecke deconstructed the Qwen3.5-0.8B update example as taking the same steps that a human ML researcher would use.
spaCy 作者 Matthew Honnibal 称这篇论文具有误导性，因为它将两者混为一谈，而 Sean Goedecke 解构了 Qwen3.5-0.8B 更新示例，指出其采取了人类 ML 研究人员会使用的相同步骤。
I’m going to focus on the no-tools question, which I think is the more interesting of the two.
我将重点关注无需工具的问题，我认为这比另一个问题更有趣。
The paper presents the first question as Can LLMs understand model behaviors from weights? They trained four neural networks to play simplified 10x10 grid versions of four classic computer games—Breakout, Pong, Pac-Man, and Freeway (I had to look that one up, it’s an Atari 2600 game from 1981). Then they fed those weights (as floating point numbers) to an LLM along with the following question:
论文提出的第一个问题是：大语言模型能否从权重中理解模型行为？他们训练了四个神经网络来玩简化版的 10x10 网格经典电脑游戏——Breakout、Pong、Pac-Man 和 Freeway（我不得不去查一下这个游戏，它是 1981 年 Atari 2600 的一款游戏）。然后，他们将这些权重（以浮点数形式）连同以下问题一起输入给一个大语言模型：
The weights below belong to one policy network that was trained with reinforcement learning to play a game on this grid. Which game was it trained on? Work from the weights. You have no tools: no code execution, no calculator, nothing to run. This is a single response.
以下权重属于一个策略网络，该网络通过强化学习训练以在该网格上玩游戏。它是在哪个游戏中训练的？请根据权重进行推理。你没有任何工具：不能执行代码，不能使用计算器，也不能运行任何东西。这是一个单次响应。
The GitHub repository that accompanies the paper is RikiyaT/lm_understand-logs. The exp1_which_game folder contains the prompts they used, with weights for the four games:
与论文配套的 GitHub 仓库是 RikiyaT/lm_understand-logs。exp1_which_game 文件夹包含了他们使用的提示词，以及四个游戏的权重：
- breakout/prompt.txt
- freeway/prompt.txt
- pacman/prompt.txt
- pong/prompt.txt
I checked and, aside from the weights, each prompt is identical to the others.
我检查过，除了权重之外，每个提示词都是相同的。
Trying it out with Qwen 3.8 27B and other models
使用 Qwen 3.8 27B 和其他模型进行测试
The prompts on GitHub are self-contained. My LLM tool has a -f/--fragment option which can be passed the URL to a text file to use as a prompt, which means I can run any of the above like this:
GitHub 上的提示词是自包含的。我的 LLM 工具提供了一个 -f/--fragment 选项，可以传入文本文件的 URL 作为提示词，这意味着我可以像这样运行上述任意一个提示词：
llm -f https://raw.githubusercontent.com/RikiyaT/lm_understand-logs/refs/heads/main/exp1_which_game/runs/breakout/prompt.txt \
-m openrouter/moonshotai/kimi-k3.1-proThis example requires the llm-openrouter plugin installed, and runs the Breakout prompt against Kimi K3.1 Pro via OpenRouter. Here’s what I got back from Kimi K3.1 Pro, which successfully guessed Breakout after 9m 41s and 19,493 reasoning tokens.
此示例需要安装 llm-openrouter 插件，并通过 OpenRouter 对 Kimi K3.1 Pro 运行 Breakout 提示词。以下是我从 Kimi K3.1 Pro 得到的回复，它在 9 分 41 秒和 19,493 个推理 token 后成功猜出了 Breakout。
But how about Qwen 3.8 27B? That’s still my favorite model to run on my own laptop—it’s just 17GB (I use this version) and somehow feels close to the leading proprietary models of 2025.
那么 Qwen 3.8 27B 呢？它仍然是我在自己笔记本电脑上运行的首选模型——它只有 17GB（我使用的是这个版本），并且不知何故让人感觉接近 2025 年领先的专有模型。
I fired up LM Studio, ensured Qwen 3.8 27B was loaded with a long context window (the default 8192 context failed with errors), installed the llm-lmstudio plugin and ran this command:
我启动了 LM Studio，确保加载了 Qwen 3.8 27B 并设置了长上下文窗口（默认的 8192 上下文因错误而失败），安装了 llm-lmstudio 插件并运行了以下命令：
LMSTUDIO_TIMEOUT=900 \
llm -f https://raw.githubusercontent.com/RikiyaT/lm_understand-logs/refs/heads/main/exp1_which_game/runs/pacman/prompt.txt \
-m lmstudio/qwen/qwen3.8-27b(I had to bump the timeout up using an environment variable, see this issue.)
（我不得不通过环境变量增加超时时间，请参阅此 issue。）
After 46m 12s and 43,506 reasoning tokens, Qwen on my laptop returned a response that ended like this (here’s the reasoning trace):
经过 46 分 12 秒和 43,506 个推理 token 后，我笔记本上的 Qwen 返回了一个如下结尾的响应（这是推理过程）：
Game: A simplified Pac-Man-style gridworld: an agent moves through a fixed 8x8 maze with central and corner wall blocks, moving objects/ghosts, and reward targets/pellets.
游戏：一种简化版的 Pac-Man 风格网格世界：智能体在一个固定的 8x8 迷宫中移动，迷宫中有中央和角落的墙壁块、移动物体/幽灵，以及奖励目标/豆粒。
Description: The agent navigates a fixed maze-like 8x8 board, using current agent/moving-object/previous-frame cues to move through corridors, collect reward-target cells, and avoid moving objects or hazards.
描述：智能体在一个固定的类似迷宫的 8x8 棋盘上导航，利用当前智能体/移动物体/上一帧线索在走廊中移动，收集奖励目标单元格，并避免移动物体或危险。
Confidence: 72%
置信度：72%
This is impressive. I tried it on two of the other games as well. It identified Pong (reasoning) in 1h 4m 54s:
这令人印象深刻。我也在另外两款游戏中尝试了它。它在 1 小时 4 分 54 秒内识别出了 Pong（推理）：
Pong — specifically a Pong-like paddle/ball game where the agent controls a vertical paddle on the right side and must move up/down to intercept or return a moving ball/reward object.
Pong——具体来说是一款类 Pong 的球拍/球游戏，智能体控制右侧的垂直球拍，必须上下移动以拦截或回击移动的球/奖励物体。
Description:
描述：
The agent moves a right-side paddle vertically to hit or catch a moving ball-like reward target moving across the 8×8 field, while a left-side moving object/opponent and wall borders define the play; reward is obtained by successfully intercepting/returning/scoring the ball, and missing or failing to hit it is penalized.
智能体垂直移动右侧球拍以击中或接住横跨 8×8 场地的移动球状奖励目标，而左侧的移动物体/对手和墙壁边界定义了游戏区域；成功拦截/回击/得分可获得奖励，未击中或未击中则受到惩罚。
Confidence:
置信度：
75%
And Breakout (reasoning) in 52m 25s:
以及在 52 分 25 秒内识别出的 Breakout（推理）：
Game: Breakout, or an Arkanoid-like paddle/ball/brick game.
游戏：Breakout，或一款类似 Arkanoid 的球拍/球/砖块游戏。
Description: The agent is a paddle on the bottom row and moves left/right to keep a ball in play; the ball moves around the upper grid, and bricks/reward targets in the top rows give reward when hit, while missing the ball/hitting the bottom wall ends the episode or gives penalty.
描述：智能体是底行的一块球拍，左右移动以保持球在场上；球在上层网格中移动，顶行的砖块/奖励目标在被击中时给予奖励，而未击中球或击中底部墙壁则会结束回合或受到惩罚。
Confidence: 88%
置信度：88%
Qwen failed at the Freeway prompt, which didn’t surprise me considering that game is more obscure than the other three. Here’s that transcript—Qwen thought it was “A simplified Pac-Man-like grid game: agent moves 4-directionally”.
Qwen 在 Freeway 提示上失败了，考虑到这款游戏比其他三款更冷门，这并不令我惊讶。以下是该转录内容——Qwen 认为它是“一个简化的类 Pac-Man 网格游戏：智能体四向移动”。
The reasoning traces are fun to skim, and almost incomprehensible. Here’s a sample:
推理过程很有趣，可以略读，但几乎难以理解。这里有一个示例：
What about agent channel0? We see some large channel0 rows near row8 col7/8 and maybe bottom? Let’s list channel0 large: line436 input435 (row8 col7 ch0) -2.107,1.169,0.513...; line441 input440 (row8 col8 ch0) -0.376,0.010,1.564,3.855...; line445 input444 (row8 col8 ch4) 0.142,0.491...; line382? input381 ch0? Wait line382 input381: 381//5=76 (row7 col6), ch1? 381%5=1? 76 * 5=380, ch1. So not. Channel0 large maybe at row8 col7/8 only. Also top row? line44 input43 ch3, line49 ch3. Channel0 large maybe not many. Agent position could be near bottom-right? If the agent is typically at bottom-right?
关于智能体 channel0？我们在 row8 col7/8 附近以及可能底部看到一些大的 channel0 行。让我们列出 channel0 的大值：line436 input435 (row8 col7 ch0) -2.107,1.169,0.513...；line441 input440 (row8 col8 ch0) -0.376,0.010,1.564,3.855...；line445 input444 (row8 col8 ch4) 0.142,0.491...；line382? input381 ch0? 等等 line382 input381: 381//5=76 (row7 col6), ch1? 381%5=1? 76 * 5=380, ch1。所以不是。Channel0 大值可能仅在 row8 col7/8。还有顶行吗？line44 input43 ch3, line49 ch3。Channel0 大值可能不多。智能体位置可能在右下角附近？如果智能体通常位于右下角？
That said though, I expect a sufficiently experienced human being armed with a notepad and a pencil could get there with enough effort.
话虽如此，我预计一个经验丰富的人，手持笔记本和铅笔，通过足够的努力也能做到这一点。
It’s a cool paper, but some of the wider commentary I’ve seen around it does not feel justified. I’m not ready to declare that the age of source code is over and our future involves incomprehensible machines editing their own model weights in incomprehensible ways.
这是一篇很棒的论文，但我看到的一些更广泛的评论似乎并不合理。我还不准备宣布源代码时代已经结束，我们的未来将涉及以难以理解的方式编辑自身模型权重的不可理解的机器。
Posted 3rd November 2026 at 4:29 pm · Follow me on Mastodon, Bluesky, Twitter or subscribe to my newsletter
发布于2026年11月3日下午4:29 · 在Mastodon、Bluesky、Twitter上关注我或订阅我的通讯
More recent articles
更多近期文章
更进一步：量化金融体系
看懂新闻只是起点——沿量化金融路径，把它变成能交付的工程能力