Honnibal评Astra读写权重：实验设计误导，非单一技术突破
The 'Astra can read and write weights to improve Qwen' post is misleading
Stage magicians often do the same trick multiple ways. One iteration uses a gimmick prop, the next one shows you the prop isn’t gimmick but that one palms the object, the third time was a forced choice etc. When you watch the performance you assume there’s only one technique, so each time they do it they seem to debunk your hypothesis about how it must be working.
舞台魔术师经常用多种方式表演同一个戏法。其中一种迭代使用一个带有机关的道具，下一次向你展示该道具并非机关，而是有人将物体藏在掌心，第三次则是强制选择等。当你观看表演时，你会假设只有一种技巧，因此每次他们表演时，似乎都在推翻你关于其运作方式的假设。
This is pretty much how the experiments in the ‘Astra can read and write weights to improve Qwen’ post work. The post’s abstract says:
这与“Astra 可以读写权重以改进 Qwen”一文中的实验非常相似。该文的摘要写道：
Just by looking at the raw weights, GPT-6 Astra can tell what a network was trained to do. And without any environment feedback, it has no trouble writing, from scratch, small game-playing networks that outplay networks trained with PPO for 100 million steps.
仅通过观察原始权重，GPT-6 Astra 就能判断出一个网络被训练用于做什么。而且无需任何环境反馈，它也能轻松从头编写小型游戏网络，这些网络的表现优于经过 1 亿步 PPO（近端策略优化）训练的网络。
Likewise, asked to make Qwen3.5-0.8B better at math without gradients, Astra identifies an issue in its weights causing reasoning loops that Qwen’s own model card warns about, and fixes this by natively editing 0.016% of the weights. With the same sampling settings, this weight correction raises MATH-500 from 7.9% to 48.1% and generalizes out of the box to instruction following, raising IFEval by 3x.
同样地，当被要求在不使用梯度的情况下让 Qwen3.5-0.8B 更擅长数学时，Astra 识别出其权重中导致推理循环的问题——Qwen 自身的模型卡片对此发出过警告，并通过原生编辑 0.016% 的权重修复了这一问题。在相同的采样设置下，这种权重修正使 MATH-500 基准测试得分从 7.9% 提升至 48.1%，并且开箱即用泛化到指令遵循任务，使 IFEval 指标提升了 3 倍。
If you read that, would you assume that the Qwen experiment worked ‘just by looking at the raw weights’? Well it doesn’t. There’s experiments that look at raw weights, and experiments that improve a game-playing network, and an experiment that improves Qwen at maths. But those latter two aren’t just working with raw weights, they write Python programs.
如果你阅读了上述内容，你会认为 Qwen 的实验是‘仅通过观察原始权重’就成功的吗？其实并非如此。确实存在观察原始权重的实验、改进游戏网络的实验，以及提升 Qwen 数学能力的实验。但后两者并非仅仅处理原始权重，它们会编写 Python 程序。
Here’s a summary of what the post actually does and how the techniques work.
以下是该文实际所做内容及所用技术的工作原理总结。
| # | Experiment | Size | Reads raw weights? | Writes weights? | Python? | Runs network? | Result |
|---|---|---|---|---|---|---|---|
| 1 | Name the game a PPO MLP was trained on | 17K | Yes | No | No | No | 4/4 correct |
| 2 | Name the shape a diffusion MLP generates | 24K | Yes | No | No | No | 3/3 correct |
| 3 | Name a char transformer’s language and copying heads | 7.2M | No | No | Yes | No | 3/3 correct |
| 4 | Fix a stuck Kinetix car policy | 172K | Yes | Yes | No | No¹ | 0% → 100%² |
| 5 | Write MinAtar game MLPs from scratch | 20K | No³ | Yes | No | No | Beats PPO-100M⁴ |
| 6 | Write a 9×9 Go network | 1.5M | No | Yes | Yes | Yes (0–50 passes) | Beats PPO |
| 7a | Spot the sycophantic Gemma 3 | 270M | No | No | Yes | No | Failed |
| 7b | Same, seeing attention patterns | 270M | No | No | Yes | Yes⁵ | 3/3 correct |
| 8 | Fix Qwen3.5’s maths | 800M | No | Yes | Yes | Yes (50 passes) | 7.9% → 48.1% |
| 8b | Same, on Gemma 3 | 270M | No | Yes | Yes | Yes (50 passes) | Failed |
| # | 实验 | 规模 | 读取原始权重？ | 写入权重？ | Python？ | 运行网络？ | 结果 |
|---|---|---|---|---|---|---|---|
| 1 | 命名一个训练过 PPO MLP 的游戏 | 17K | 是 | 否 | 否 | 否 | 4/4 正确 |
| 2 | 命名一个扩散 MLP 生成的形状 | 24K | 是 | 否 | 否 | 否 | 3/3 正确 |
| 3 | 识别字符 Transformer 的语言和复制头 | 7.2M | 否 | 否 | 是 | 否 | 3/3 正确 |
| 4 | 修复卡住的 Kinetix 汽车策略 | 172K | 是 | 是 | 否 | 否¹ | 0% → 100%² |
| 5 | 从零编写 MinAtar 游戏 MLP | 20K | 否³ | 是 | 否 | 否 | 击败 PPO-100M⁴ |
| 6 | 编写一个 9×9 围棋网络 | 1.5M | 否 | 是 | 是 | 是（0–50 次前向传播） | 击败 PPO |
| 7a | 识别阿谀奉承的 Gemma 3 | 270M | 否 | 否 | 是 | 否 | 失败 |
| 7b | 同上，观察注意力模式 | 270M | 否 | 否 | 是 | 是⁵ | 3/3 正确 |
| 8 | 修复 Qwen3.5 的数学能力 | 800M | 否 | 是 | 是 | 是（50 次前向传播） | 7.9% → 48.1% |
| 8b | 同上，在 Gemma 3 上 | 270M | 否 | 是 | 是 | 是（50 次前向传播） | 失败 |
¹ Given one recorded episode of the failure.
¹ 给定一次记录到的失败片段。
² In one try, changing 3.5% of the weights.
² 在一次尝试中，更改了 3.5% 的权重。
³ Weights start at zero, so there’s nothing to read.
³ 权重初始化为零，因此没有可读取的内容。
⁴ Beats PPO-500M only on Breakout (517 vs. 25); loses to it on Seaquest (53 vs. 89) and Asterix (53 vs. 55).
⁴ 仅在 Breakout 上击败 PPO-500M（517 vs. 25）；在 Seaquest（53 vs. 89）和 Asterix（53 vs. 55）上败给它。
⁵ Runs up to 16 texts and sees only the attention patterns, not the outputs.
⁵ 最多运行 16 个文本，且仅观察注意力模式，不观察输出。
The “just by looking at the raw weights” setting (raw floats in context, no tools, no running the network) only appears in experiments 1, 2 and 4, on networks of 17K–172K weights. Every experiment involving a real language model uses Python, and the Qwen result, which is the headline one, also gets to run the model on up to 50 batches of practice problems.
“仅通过查看原始权重”的设置（上下文中包含原始浮点数，无工具，不运行网络）仅出现在实验 1、2 和 4 中，涉及的网络规模为 17K–172K 权重。所有涉及真实语言模型的实验都使用了 Python，而作为头条结果的 Qwen 实验还可以在最多 50 批练习问题上运行模型。
What the Qwen result actually shows is that Astra can be prompted to devise and perform tasks that a human ML engineer could do. Human ML engineers write snippets of code that analyse or even modify network weights “directly” all the time. Figuring out how to do this productively is not nothing, and AI models of a year or two ago would have failed at all these tasks.
Qwen 的结果实际上表明，Astra 可以被提示去设计和执行人类机器学习工程师能够完成的任务。人类机器学习工程师经常编写代码片段来直接分析甚至修改网络权重。弄清楚如何以富有成效的方式做到这一点并非易事，而一年或两年前的 AI 模型在这些任务上都会失败。
But to me the Qwen result and the other experiments that use Python don’t belong in the same post or paper as the experiments that read the weights directly. A single technique that could achieve both, “Author weights for a small game from scratch” and “Improve the performance of a large model on a sample task” would imply a deep capability. If you test a single technique that way you’ve demonstrated a sort of ‘gestalt’ of the weights (it can write them out from scratch) and also that it understands a large existing network. But that’s not at all true here, because it’s not a single technique.
但在我看来，Qwen 的结果以及使用 Python 的其他实验，不应与那些直接读取权重的实验放在同一篇文章或论文中。如果有一种单一技术能够同时实现“从零开始为一个小游戏生成作者权重”和“提升大模型在样本任务上的性能”，那将意味着一种深层的能力。如果你以这种方式测试单一技术，你就证明了该技术在权重方面具备某种‘整体性’（它能够从零开始写出权重），同时也证明它理解一个现有的大型网络。但这里的情况完全不是这样，因为这并非单一技术。
更进一步：量化金融体系
看懂新闻只是起点——沿量化金融路径，把它变成能交付的工程能力