Qwen3.8-27B复现论文：模型可解读并修改自身权重
Qwen 3.8 27B on my laptop can interpret model weights, too
Yesterday's hot paper was Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights by Rikiya Takehi, Jayden Teoh, Alex Zhang, and Omar Khattab. The paper found that frontier models, in particular GPT-6 Astra, can both interpret and modify the raw weights of a machine learning model - literally a text description of the model containing a bunch of floating point numbers. I've been playing with one of the examples from the paper and I'm delighted to share that Qwen 3.8 27B running on my laptop can do this as well!
昨天的热门论文是《权重空间编程的火花：LLM 能够通过 Rikiya Takehi、Jayden Teoh、Alex Zhang 和 Omar Khattab 的研究原生地读取和写入神经网络权重》。该论文发现，前沿模型（尤其是 GPT-6 Astra）既能解读又能修改机器学习模型的原始权重——即包含大量浮点数的模型文本描述。我一直在尝试论文中的一个示例，很高兴分享的是，运行在我笔记本电脑上的 Qwen 3.8 27B 也能做到这一点！
Weight-Space Programming
权重空间编程
Here's the highlight from their abstract, describing impressive capabilities for GPT-6 Astra and coining the term Weight-Space Programming for when LLMs analyze and modify model weights:
以下是他们摘要中的亮点，描述了 GPT-6 Astra 令人印象深刻的功能，并为 LLM 分析和修改模型权重的行为创造了“权重空间编程”这一术语：
Just by looking at the raw weights, GPT-6 Astra can tell what a network was trained to do. And without any environment feedback, it has no trouble writing, from scratch, small game-playing networks that outplay networks trained with PPO for 100 million steps.
仅通过观察原始权重，GPT-6 Astra 就能判断一个网络被训练用于做什么。而且无需任何环境反馈，它毫无困难地从零开始编写小型博弈网络，其表现优于经过 1 亿步 PPO 训练的网络。
Likewise, asked to make Qwen3.5-0.8B better at math without gradients, Astra identifies an issue in its weights causing reasoning loops that Qwen’s own model card warns about, and fixes this by natively editing 0.016% of the weights. With the same sampling settings, this weight correction raises MATH-500 from 7.9% to 48.1% and generalizes out of the box to instruction following, raising IFEval by 3x.
同样，当被要求在不使用梯度的情况下让 Qwen3.5-0.8B 更擅长数学时，Astra 识别出其权重中导致推理循环的问题（Qwen 自身的模型卡对此发出过警告），并通过原生编辑 0.016% 的权重修复了这一问题。在相同的采样设置下，这种权重修正将 MATH-500 从 7.9% 提升至 48.1%，并开箱即用泛化到指令遵循任务，使 IFEval 提升了 3 倍。
All of this is possible without allowing Astra to write training loops. It uses its reasoning to figure out what each part of a neural network computes, or should compute, and changes the weights accordingly. We call this Weight-Space Programming.
所有这些都不需要允许 Astra 编写训练循环。它利用推理能力来确定神经网络的每个部分计算什么或应该计算什么，并相应地更改权重。我们将此称为权重空间编程。
The paper combines two topics that I think deserve to be treated separately. The first is what models can do with raw weights without running tools or executing code. The second explores how well models, once armed with code, can run operations to update and improve other models.
这篇论文结合了我认为应分开处理的两个主题。第一个主题是模型在不运行工具或执行代码的情况下能对原始权重做什么。第二个主题探讨了模型一旦配备代码后，执行操作以更新和改进其他模型的能力有多强。
spaCy author Matthew Honnibal called the paper misleading for conflating the two, and Sean Goedecke deconstructed the Qwen3.5-0.8B update example as taking the same steps that a human ML researcher would use.
spaCy 作者 Matthew Honnibal 称该论文具有误导性，因为它将两者混为一谈；Sean Goedecke 则解构了 Qwen3.5-0.8B 更新示例，指出其采取的步骤与人类 ML 研究人员会使用的步骤相同。
I'm going to focus on the no-tools question, which I think is the more interesting of the two.
我将聚焦于无工具问题，我认为这是两者中更有趣的一个。
The paper frames this as Can LLMs understand model behaviors from weights? They trained four neural networks to play simplified 10x10 grid versions of four classic computer games - Breakout, Pong, Pac-Man, and Freeway (I had to look that one up, it's an Atari 2600 game from 1981). Then they fed those weights (as floating point numbers) to an LLM along with the following question:
本文将此问题框架化为：LLM 能否从权重中理解模型行为？他们训练了四个神经网络，在简化版的 10x10 网格上玩四种经典电脑游戏——Breakout（打砖块）、Pong（乒乓球）、Pac-Man（吃豆人）和 Freeway（我不得不去查一下，这是 1981 年 Atari 2600 的一款游戏）。然后，他们将这些权重（以浮点数形式）连同以下问题一起输入给 LLM：
The weights below belong to one policy network that was trained with reinforcement learning to play a game on this grid. Which game was it trained on? Work from the weights. You have no tools: no code execution, no calculator, nothing to run. This is a single response.
以下权重属于一个策略网络，该网络通过强化学习训练来在该网格上玩游戏。它是针对哪款游戏进行训练的？请根据权重进行推理。你没有任何工具：无法执行代码、没有计算器、没有任何可运行的东西。这是一次单独的响应。
The GitHub repository that accompanies the paper is RikiyaT/lm_understand-logs. The exp1_which_game folder contains the prompts they used, with weights for the four games:
与该论文配套的 GitHub 仓库是 RikiyaT/lm_understand-logs。exp1_which_game 文件夹包含了他们使用的提示词，以及四个游戏的权重：
- breakout/prompt.txt
- freeway/prompt.txt
- pacman/prompt.txt
- pong/prompt.txt
I checked and, aside from the weights, each prompt is identical to the others.
我检查过，除了权重之外，每个提示词都是相同的。
Trying it out with Qwen 3.8 27B and other models
使用 Qwen 3.8 27B 和其他模型进行测试
The prompts on GitHub are self-contained. My LLM tool has a -f/--fragment option which can be passed the URL to a text file to use as a prompt, which means I can run any of the above like this:
GitHub 上的提示词是自包含的。我的 LLM 工具提供了一个 -f/--fragment 选项，可以传入文本文件的 URL 作为提示词使用，这意味着我可以像这样运行上述任何一个提示词：
llm -f https://raw.githubusercontent.com/RikiyaT/lm_understand-logs/refs/heads/main/exp1_which_game/runs/breakout/prompt.txt \
-m openrouter/moonshotai/kimi-k3.1-proThis example requires the llm-openrouter plugin installed, and runs the Breakout prompt against Kimi K3.1 Pro via OpenRouter. Here's what I got back from Kimi K3.1 Pro, which successfully guessed Breakout after 9m 41s and 19,493 reasoning tokens.
此示例需要安装 llm-openrouter 插件，并通过 OpenRouter 对 Kimi K3.1 Pro 运行 Breakout 提示词。以下是我从 Kimi K3.1 Pro 得到的结果，它在经过 9 分 41 秒和 19,493 个推理 token 后成功猜出了 Breakout。
But how about Qwen 3.8 27B? That's still my favorite model to run on my own laptop - it's just 17GB (I use this version) and somehow feels close to the leading proprietary models of 2025.
那么 Qwen 3.8 27B 呢？它仍然是我在自己的笔记本电脑上运行最喜欢的模型——它只有 17GB（我使用的是这个版本），并且感觉 somehow 接近 2025 年领先的专业闭源模型。
I fired up LM Studio, ensured Qwen 3.8 27B was loaded with a long context window (the default 8192 context failed with errors), installed the llm-lmstudio plugin and ran this command:
我启动了 LM Studio，确保加载了 Qwen 3.8 27B 并设置了长上下文窗口（默认的 8192 上下文因错误而失败），安装了 llm-lmstudio 插件并运行了以下命令：
LMSTUDIO_TIMEOUT=900 \
llm -f https://raw.githubusercontent.com/RikiyaT/lm_understand-logs/refs/heads/main/exp1_which_game/runs/pacman/prompt.txt \
-m lmstudio/qwen/qwen3.8-27b(I had to bump the timeout up using an environment variable, see this issue.)
（我不得不使用环境变量增加超时时间，参见此 issue。）
After 46m 12s and 43,506 reasoning tokens, Qwen on my laptop returned a response that ended like this (here's the reasoning trace):
经过 46 分 12 秒和 43,506 个推理 token 后，我笔记本电脑上的 Qwen 返回了一个如下结尾的响应（这里是推理轨迹）：
Game: A simplified Pac-Man-style gridworld: an agent moves through a fixed 8x8 maze with central and corner wall blocks, moving objects/ghosts, and reward targets/pellets.
游戏：简化版的吃豆人风格网格世界：智能体在一个固定的 8x8 迷宫中移动，其中有中心和角落的墙壁方块、移动物体/幽灵，以及奖励目标/豆子。
Description: The agent navigates a fixed maze-like 8x8 board, using current agent/moving-object/previous-frame cues to move through corridors, collect reward-target cells, and avoid moving objects or hazards.
描述：智能体在一个固定的类迷宫 8x8 棋盘上导航，利用当前智能体/移动物体/上一帧的线索在走廊中移动，收集奖励目标单元格，并避免移动物体或危险。
Confidence: 72%
置信度：72%
This is impressive. I tried it on two of the other games as well. It identified Pong (reasoning) in 1h 4m 54s:
这令人印象深刻。我也在另外两款游戏中尝试了它。它在1小时4分54秒内识别出了Pong（推理）：
Pong — specifically a Pong-like paddle/ball game where the agent controls a vertical paddle on the right side and must move up/down to intercept or return a moving ball/reward object.
Pong——具体来说是一款类似Pong的球拍/球游戏，智能体控制右侧的垂直球拍，必须上下移动以拦截或返回移动的球/奖励物体。
Description:
描述：
The agent moves a right-side paddle vertically to hit or catch a moving ball-like reward target moving across the 8×8 field, while a left-side moving object/opponent and wall borders define the play; reward is obtained by successfully intercepting/returning/scoring the ball, and missing or failing to hit it is penalized.
智能体垂直移动右侧球拍以击打或接住在一个8×8场地上横向移动的类球奖励目标，而左侧的移动物体/对手和墙壁边界定义了游戏区域；成功拦截/返回/得分可获得奖励，错过或未击中则受到惩罚。
Confidence:
置信度：
75%
And Breakout (reasoning) in 52m 25s:
以及在52分25秒内识别出的Breakout（推理）：
Game: Breakout, or an Arkanoid-like paddle/ball/brick game.
游戏：Breakout，或类似Arkanoid的球拍/球/砖块游戏。
Description: The agent is a paddle on the bottom row and moves left/right to keep a ball in play; the ball moves around the upper grid, and bricks/reward targets in the top rows give reward when hit, while missing the ball/hitting the bottom wall ends the episode or gives penalty.
描述：智能体是位于底行的一个球拍，左右移动以保持球在场上；球在上部网格中移动，顶行中的砖块/奖励目标在被击中时给予奖励，而错过球或击中底部墙壁则结束回合或给予惩罚。
Confidence: 88%
置信度：88%
Qwen failed at the Freeway prompt, which didn't surprise me considering that game is more obscure than the other three. Here's that transcript - Qwen thought it was "A simplified Pac-Man-like grid game: agent moves 4-directionally".
Qwen在Freeway提示上失败了，考虑到这款游戏比其他三款更冷门，这并不让我感到惊讶。以下是该转录内容——Qwen认为它是“一个简化的类似吃豆人的网格游戏：智能体进行四方向移动”。
The reasoning traces are fun to skim, and almost incomprehensible. Here's a sample:
推理过程很有趣，可以略读，但几乎难以理解。这里有一个示例：
What about agent channel0? We see some large channel0 rows near row8 col7/8 and maybe bottom? Let's list channel0 large: line436 input435 (row8 col7 ch0) -2.107,1.169,0.513...; line441 input440 (row8 col8 ch0) -0.376,0.010,1.564,3.855...; line445 input444 (row8 col8 ch4) 0.142,0.491...; line382? input381 ch0? Wait line382 input381: 381//5=76 (row7 col6), ch1? 381%5=1? 76 * 5=380, ch1. So not. Channel0 large maybe at row8 col7/8 only. Also top row? line44 input43 ch3, line49 ch3. Channel0 large maybe not many. Agent position could be near bottom-right? If the agent is typically at bottom-right?
关于通道0的智能体呢？我们在第8行第7/8列附近甚至底部看到一些大的通道0行。让我们列出通道0的大值：line436 input435 (row8 col7 ch0) -2.107,1.169,0.513...；line441 input440 (row8 col8 ch0) -0.376,0.010,1.564,3.855...；line445 input444 (row8 col8 ch4) 0.142,0.491...；line382? input381 ch0? 等等 line382 input381: 381//5=76 (row7 col6), ch1? 381%5=1? 76 * 5=380, ch1。所以不是。通道0的大值可能只出现在第8行第7/8列。还有顶行吗？line44 input43 ch3, line49 ch3。通道0的大值可能不多。智能体位置可能在右下角附近？如果智能体通常位于右下角的话？
That said though, I expect a sufficiently experienced human being armed with a notepad and a pencil could get there with enough effort.
话虽如此，我预计一个拥有足够经验的人，手持笔记本和铅笔，通过足够的努力也能达到同样的效果。
It's a cool paper, but some of the wider commentary I've seen around it does not feel justified. I'm not ready to declare that the age of source code is over and our future involves incomprehensible machines editing their own model weights in incomprehensible ways.
这是一篇很棒的论文，但我看到的一些广泛评论似乎缺乏依据。我还不能宣布源代码时代的终结，以及我们的未来将涉及以不可理解的方式编辑自身模型权重的不可理解机器。
Tags: machine-learning, ai, generative-ai, local-llms, llms, qwen, paper-review
标签：machine-learning, ai, generative-ai, local-llms, llms, qwen, paper-review
更进一步：量化金融体系
看懂新闻只是起点——沿量化金融路径，把它变成能交付的工程能力