Researchers test direct editing of neural-network weights

Weight-Space Programming asks an LLM to inspect or alter a network’s weights directly, rather than using gradient descent to update them.[1] The researchers reported eight experiments covering weight interpretation, network creation and model improvement, using OpenAI’s GPT-6 Astra.[2] The work therefore tested more than one task, but the results cited here span small game networks, diffusion models and a language-model math edit.[1][2]

For a game-identification test, Astra received raw network weights as text and was not told which games were under consideration.[1] It named the right game for all four networks.[1] The networks used the same MLP structure—500 → 32 → 32 → 5—with 17,253 weights, and the games were grid versions of Breakout, Pong, Pac-Man and Freeway on a 10×10 grid with five actions.[1] Separately, Astra identified the shapes generated by small diffusion models by examining their weights, without running the models or performing forward passes.[2]

The researchers asked Astra to improve Qwen3.5-0.8B at math without gradients.[1] The reported weight correction raised MATH-500 from 7.9% to 48.1%, with a native change to 0.016% of the weights.[1] Analytics India Magazine reported increases from 13.8% to 67.1% on GSM8K and from 21% to 62.3% on IFEval.[2] VentureBeat reported that the strongest attempt took 158 conversational turns, 80 Python runs and 20.3 hours, including time waiting through an outage.[3]

The researchers attributed Qwen’s improvement to correcting repetitive reasoning loops.[3] The blog author described the issue as loops after 32,000 tokens that caused longer math tasks to fail, and said Astra wrote four pairs of texts, measured weights contributing to looping and began cutting them down.[4] The blog author called direct editing an impressive trick, but not a radical new capability, and argued conventional training remains the best way to update weights.[4] VentureBeat noted that the research did not establish consistent advantages over conventional training in speed, cost or effectiveness.[3]