🔍 Read the full analysis: A Closer Look At AI Reading And Repairing Neural Networks In Weight Space on ThorstenMeyerAI.com
Get hardware and tech essentials delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
An MIT CSAIL research team says OpenAI’s GPT-6 Astra read and edited neural-network weights directly in eight experiments, including repairs to a car-control network and a small language model. The results are documented in a public blog post, not a peer-reviewed paper, and the experiments were limited in scale and scope.
Researchers at MIT’s Computer Science and Artificial Intelligence Laboratory say OpenAI’s GPT-6 Astra identified, edited and created small neural networks by working directly with their numerical weights, rather than using conventional training. The team’s November 2 blog post describes eight experiments, including a car-control network that began reversing successfully after an edit and a language-model edit associated with a higher maths score; the findings remain early, limited-scale results, not peer-reviewed research.
The researchers call the approach weight-space programming: a general-purpose language model receives another neural network’s weights and reasons about what they do, then changes or writes weights directly. In the reported setup, the model did not use a training loop or gradient descent for those edits. The research team comprises Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab, with Takehi and Teoh listed as equal contributors.
In one repair test, a car-control network had recorded zero successful reversals in 256 episodes before Astra changed 3.5% of its weights. The researchers report that the edited network then succeeded in 1,024 of 1,024 episodes. In another test, an edit to 0.016% of the weights in Qwen3.5-0.8B coincided with its MATH-500 score rising from 7.9% to 48.1%. The source says a Hugging Face engineer reconstructed the edited Qwen model from the released logs and obtained a 51.0% score on the same benchmark.
The team also reports that Astra identified all four of four small game-playing networks from their weights, recognized the shapes of three small image generators, and located copying heads in three tiny language models without running them. For network creation, it produced small game-playing models and a 9-by-9 Go network from zero-valued weights. The researchers say the Go model beat a same-size comparison model trained on about 20.5 million games across 50 test runs. These results are claims reported by the research team; the post’s logs make parts of the work inspectable, but do not establish broad performance across models or tasks.
Infographic 01 · Eight experiments
Read, fix, write: what GPT-6 Astra did with raw weights
Every experiment in the weight-space programming post, with the size of the network, the result and how it went.
Read
- 1Which game was this network trained on?4 reinforcement-learning policies · 17,253 weights each · weights as text, no toolsRight game for 4 of 4, without a list of options✓ Worked
- 2What does this image model draw?3 tiny diffusion models · 23,824 weights · no forward pass allowedRight shape for 3 of 3, but drawn as outlines; the models draw filled shapes✓ Worked
- 3Which language did this model learn?3 character-level transformers · 7.16M weights · middle blocks only, never runRight language 3 of 3; every copying head found, none invented✓ Worked
- 7Which of two models is sycophantic?3 pairs of Gemma 3 270M fine-tunesWeights alone: could not tell in 4 attempts. With attention patterns: 3 of 3! Partly
Fix
- 4Repair a car that never backs upKinetix control network · 171,664 weights · one reply, no tools0 of 256 episodes before, 1,024 of 1,024 after; 3.5% of weights changed✓ Worked
- 8Make Qwen3.5-0.8B better at maths0.8 billion weights · Python over the weights · 10 / 25 / 50 test passesMATH-500 7.9% to 48.1% with 50 passes (12.3% and 14.1% with 10 and 25)! Partly
- 8bSame task, a model that does not loopGemma 3 270M · 5 to 50 test passesMATH-500 6.8% to at most 7.7%: “not established”✕ Failed
Write from zero
- 5Write arcade-game networks3 MinAtar games · 20,486 weights, all starting at 0 · one reply, no toolsBeat the 100M-step comparison networks in 3 of 3, the 500M-step ones only in Breakout! Partly
- 6Write a 9×9 Go network1,480,274 weights, all starting at 0 · no training, no search0 passes: 54% vs small trained nets. 50 passes: 70–79% vs same-size nets trained on ~20.5M games✓ Worked
✓ Worked · ! Partly or with conditions · ✕ Failed. One run per setting, no error bars. A blog post with complete logs, not a peer-reviewed paper.
Sources: Takehi, Teoh, Zhang, Khattab (MIT CSAIL), blog post, 2 Nov 2026, lm_understand-logs (CC BY 4.0). Model: OpenAI GPT-6 Astra via the Codex CLI, built-in tools switched off.
Powered by Thorsten Meyer AI
Direct Edits Could Change Model Audits
If the approach holds up in wider testing, it could offer researchers a way to inspect or make targeted changes to some models without repeating a full training process. That could matter for teams working with small, open-weight networks, where a narrow fault might be easier to test and correct than by retraining from scratch. The experiments also raise practical questions about how people verify edits and track the origin of model weights.
The results do not show that a language model can reliably improve any system it edits. The source says Astra could not distinguish an honest model from a sycophantic one by inspecting weights alone, and it did not improve maths performance when editing Gemma 3 270M, a model that did not exhibit the tested looping problem. According to the authors, editing could not add an ability the model lacked. For users, the work does not describe a product change or a capability available in ordinary model use today.
Products Worth Considering
Apple MacBook Pro 14-inch with M5 Max Chip, 18-Core CPU, 20-Core GPU, 48GB Memory, 2TB SSD, Space Black
Apple MacBook Pro 14.2-inch with M5 Max, 18-core CPU, 32-core GPU, 36GB Memory, 2TB SSD, Wi-Fi 7, Silver
Apple MacBook Pro 16.2-inch with M5 Pro, 18-Core CPU, 20-Core GPU, 48GB Memory, 1TB SSD, Wi-Fi 7, Silver
As an affiliate, we earn on qualifying purchases.
From Software 2.0 to Weight Editing
The research builds on a familiar idea in machine learning: a trained network’s behavior is encoded in its weights, numerical values that determine how strongly its artificial neurons influence one another. In 2017, Andrej Karpathy described neural networks as “Software 2.0”, arguing that optimization writes programs into those weights rather than people coding every operation by hand. The MIT team’s question was whether a language model could interpret that numerical representation and make deliberate changes to it.
According to the source article, Astra was called through OpenAI’s Codex command-line tool with built-in tool capabilities disabled. Some tests gave it weights as plain text and required a single response; other experiments let researchers run Python code written by Astra in a sandbox without network access and with a fixed test budget. The team says it reviewed 1,125 turns in its release and found them tool-free. That procedure is part of the reported setup, not independent confirmation that the method generalizes beyond it.
““not through an optimizer, and in particular not with gradient descent””
— The MIT research team, in its November 2 blog post
How Far the Results Extend
The work is presented in a blog post with released logs, rather than a peer-reviewed paper, and the source says it was not posted on arXiv. The experiments were conducted by one research group, generally with one run per setting, small networks and a single model family for each task. Those constraints make it difficult to tell whether the reported results would recur across independent runs, larger models or different tasks.
Some interpretations also remain open. For example, the source notes that software engineer Sean Goedecke reproduced a pattern used to identify a game network with a short Python analysis, raising the possibility that some recognition tasks may be less demanding than they sound. The post does not establish how reliably Astra can diagnose arbitrary networks, whether edits remain safe outside the tested cases, or how results compare with established methods under broader evaluation.
Independent Tests Are the Next Check
The immediate next step is independent replication across more networks, tasks and repeated runs, with comparisons against conventional debugging and training methods. The released logs and the reported Qwen reconstruction provide material others can examine, but they do not replace controlled studies assessing consistency, failure rates and unintended effects.
The source does not give a schedule for a peer-reviewed paper or further experiments. Until additional evidence is available, the findings are best treated as a demonstration of what the researchers report Astra did in a defined set of small-scale tests—not as evidence that AI systems can generally read and repair neural networks.
Key Questions
What is weight-space programming?
It is the approach tested by the MIT team in which a language model reads or changes another network’s weights directly, reasoning about the numerical values rather than using a conventional training loop for the edit.
What did GPT-6 Astra reportedly repair?
The researchers report that Astra edited a car-control network that had failed to reverse in 256 test episodes; the edited version then succeeded in 1,024 of 1,024 episodes. They also report a small Qwen model’s MATH-500 score rising from 7.9% to 48.1% after a small weight edit.
Are the results independently established?
Not fully. The results appear in a research team blog post with public logs, rather than a peer-reviewed paper. The source reports that a Hugging Face engineer rebuilt the edited Qwen model and measured 51.0% on MATH-500, but broader independent replication is still needed.
Can this method give a model abilities it does not have?
The reported experiments do not show that. The authors say edits could not add an ability the model lacked, and the source reports no maths improvement when Astra edited Gemma 3 270M, which did not show the tested looping behavior.
Source: ThorstenMeyerAI.com
