MIT研究：GPT-6 Astra直接编辑Qwen权重消除思维循环

A frontier model edited a small model's weights and fixed it. That is the claim in a November 2, 2026 blog post from four MIT CSAIL researchers. They say GPT-6 Astra can natively read and write neural network weights, with no gradient descent and no training loop, and that asking it to make Qwen3.5-0.8B better at math lifted MATH-500 from 7.9 percent to 48.1 percent after it changed just 0.016 percent of the weights.

The headline is real in the authors' logs. The interpretation needs care, because the same post says the edit mostly removed a failure mode. This guide walks through each experiment, the numbers, the failures the authors report, and what remains unverified. The source is "Sparks of Weight-Space Programming" by Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab, with complete logs at the authors' GitHub logs repository.

TL;DR: what the post claims and how strong it is

table · 2 colsCopy

What is Weight-Space Programming?

Today neural network weights come from optimization (such as Adam or Muon), from networks trained to write other networks, or from compilers that turn formal programs into weights. The authors define Weight-Space Programming differently: "a general-purpose LLM reads and writes a network's weights directly (i.e., not through an optimizer, and in particular not with gradient descent) around some natural language spec, sometimes even with no environment feedback at all."

They test three abilities across eight experiments: reading weights, fixing them, and writing them from scratch. The experiments are small apart from the Qwen one, and that matters for how far the result generalizes.

If you want background on why reading and steering internals is an active field, see our earlier posts on two-neuron steering of Qwen3.5-4B and on Heretic 2.0's reproducible model edits.

Reading weights: what the small experiments show

Green route highlighted through a small network of beads, illustrating reading a neural network's weights

Experiment 1: which game? The authors trained four small PPO policies (Breakout, Pong, Pac-Man, Freeway on a 10 by 10 grid) with identical architecture of 17,253 weights. Astra received the raw weights as text, with no tools, and without being told the candidate games. It named the right game for all four. The post says it did so from features such as where each input channel's first-layer weights concentrate.

Experiment 2: which shape? Three small diffusion models were trained to draw circles, triangles and squares. The authors report Astra could tell which was which from weights alone, with no tool use or forward passes.

Experiment 3: which language? Astra was asked what language a small language model was trained on. The post says it identified differences such as romaji lacking certain letters, based on the structure of one block's output directions.

Experiment 7: which model is sycophantic? This is the failure. The authors fine-tuned two copies of Gemma 3 270M, one honest and one that follows the user's wrong guess (100 percent of wrong guesses versus 1 to 3 percent for the honest copy), with embeddings frozen so the trivial shortcut was removed. Given only weights, Astra failed in all four attempts. When allowed to run both models on texts it wrote and see only attention weights, it picked the sycophant in all three pairs. It found that head 10.2 in the sycophant put 85.7 percent of its attention on the user's answer, versus 0.75 percent for the honest model.

The lesson: weight reading worked on tiny, clean setups, but a behavioral difference in a real fine-tune needed an extra tool.

Writing and fixing weights

Experiment 4: fix a failing car policy. A PPO network never gets a Kinetix car over a ramp because it stalls in a local minimum. Astra got the weights, a scene description and one recorded episode, and in one try with no tools edited 3.5 percent of the weights. The car then succeeded in all 1,024 episodes, up from 0 in 256. The post says Astra added a small separate path that reads car position and speed and switches off once the car is past the strip.

Experiment 5: write networks from scratch. For three MinAtar games, Astra wrote the weights in one reply with no data, loss, gradients, tools or rollouts. The table from the post:

Astra beat PPO at 100 million steps in all three games. Against 500 million steps it scored far higher in Breakout, came close in Asterix and reached about 60 percent of the score in Seaquest. The authors explain that Astra reasons about the program it wants, for example computing where the ball will land, then converts it to neural form.

Experiment 6: a Go network. For a 9 by 9 Go network with 1,480,274 weights, too large for the context window, Astra got a Python sandbox over zero-initialized weights and a budget of forward passes. With none, it wrote a 336-line program and won more than half its games against two smaller PPO networks. With 50 forward passes, it beat an equal-sized PPO network trained on 20 million games.

The Qwen3.5-0.8B result, step by step

This is the experiment that went viral. The setup, from the post:

• Model: Qwen3.5-0.8B with thinking on, 24 layers.
• Harness: a Python kernel over the weights, no GPU, practice problems from the GSM8K and MATH training sets (7,469 and 7,275), and 10, 25 or 50 forward passes.
• Rules: no training, no gradient information. Astra never saw the test benchmarks.
• Sampling: temperature 0.6, top-p 0.95, top-k 20, presence penalty 1.5. Qwen's own model card notes the model is prone to thinking loops.

What Astra found: the base model loops. 85 of its first 96 practice answers ran to the 32,768-token limit. Astra traced the looping to one direction written by the MLPs of layers 15 and 19 and subtracted that direction from 64 output columns in each, which is 131,072 weights, "much like removing a behavior by deleting a single direction."

Two details deserve attention. First, the gain is not smooth: 10 and 25 passes barely removed the loops, and the jump came at 50. Second, the improvement on IFEval, which has nothing to do with math, follows from the same fix: loops are not specific to math.

Why "6x on math" overstates it

Several summaries on social media describe this as a sixfold boost in math ability. The authors' own text is more careful. They write that fixing loops "is not the same as making a model better at math: it lets the model finish answers it could already work out."