Episode 0072: The Expert You Can Write Is the Expert You Can Explain

On the second of November a blog post out of MIT CSAIL claimed that a frontier model can read and write neural network weights directly. The press called it a paper. It is not a paper, it is a blog post with every prompt, every response, every network and every evaluation published alongside it, and that distinction turns out to be the best thing about it. Fry and Bob work through all eight experiments: the diffusion model whose generated shape Astra named by drawing it as an outline, the character transformer with its input and output layers removed, the stuck Kinetix car fixed by changing three and a half percent of its weights in a single reply, the Go evaluator compiled from zero weights as a three-hundred-and-thirty-six-line program, and the headline result that lifted a small Qwen model from seven point nine to forty-eight point one on MATH five hundred by editing sixteen thousandths of one percent of it. Then the control experiment that bounds all of it, and the three outside engineers who took the whole thing apart inside twenty-four hours because the logs were open.

Why it matters. The question this episode exists to answer is whether weight-space programming lets you hand-write a mixture of experts: a math-only expert, a logic-only expert, a motor-control-only expert, composed and always on. The honest answer is in three parts. Motor control is not the hard case, it is the demonstrated case, because experiment four spliced a hand-written subroutine into a trained policy and experiment six compiled a one-and-a-half-million-weight Go evaluator out of nothing but a specification. Composition is untouched: every experiment in the post operates on exactly one network, nothing is grafted between models, and cross-architecture transplant appears once in the entire post as a line in the future-work list. And a math-only expert written the way the headline result was written is argued against by the authors' own control, because the six-fold jump was the removal of a loop rather than the addition of mathematics, and when the same technique was pointed at a model that does not loop it produced nothing at all. The authors' sentence is the whole lesson: editing weights could remove an obstacle that hid an ability, but it could not add an ability the model lacked. Weight-space programming is a compiler, not a teacher. It turns a specification you can articulate into weights and it can delete a behaviour you can isolate, and that is a real and large new power that is almost exactly orthogonal to what training is for. The experts we can hand-write are the ones we already understand. The second half of the episode is about evidence hygiene, and it is the part worth copying: the abstract oversold a composite of eight different setups, and because the authors published the logs anyway, the correction took a day and cost nothing. That does not happen with a press release and a benchmark number.

Source

• Rikiya Takehi, Jayden Teoh, Alex Zhang, Omar Khattab — Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights (2026-11-02): https://rikiyat.github.io/blog/weight-space-programming/
• Complete logs: every prompt, every response, the code, the networks and the evaluations: https://github.com/RikiyaT/lm_understand-logs
• The edited model: https://huggingface.co/RikiyaT/Qwen3.5-0.8B-weight-space-programming
• Live demo, rebuilt from the released logs and re-evaluated independently by Ahsen Khaliq at Hugging Face: https://huggingface.co/spaces/akhaliq/weight-space-programming-demo
• Base model card, which documents the thinking-loop flaw the headline result removes: https://huggingface.co/Qwen/Qwen3.5-0.8B

The eight experiments, one folder each

• Experiment one, name the game from a PPO policy's weights: https://github.com/RikiyaT/lm_understand-logs/tree/main/exp1_which_game
• Experiment two, name the shape a diffusion model generates: https://github.com/RikiyaT/lm_understand-logs/tree/main/exp2_diffusion
• Experiment three, name a character transformer's training language and its induction heads: https://github.com/RikiyaT/lm_understand-logs/tree/main/exp3_language
• Experiment four, fix the stuck Kinetix car policy: https://github.com/RikiyaT/lm_understand-logs/tree/main/exp4_fix_car
• Experiment five, write MinAtar policies from all-zero weights: https://github.com/RikiyaT/lm_understand-logs/tree/main/exp5_write_from_scratch
• Experiment six, write a 9x9 Go network from all-zero weights: https://github.com/RikiyaT/lm_understand-logs/tree/main/exp6_go_from_scratch
• Experiment seven, spot the sycophantic model: https://github.com/RikiyaT/lm_understand-logs/tree/main/exp7_sycophancy
• Experiment eight, the Qwen math edit: https://github.com/RikiyaT/lm_understand-logs/tree/main/exp8_math
• Experiment eight b, the Gemma control that failed, published alongside the result: https://github.com/RikiyaT/lm_understand-logs/tree/main/exp8_math/gemma_followup
• The harness: https://github.com/RikiyaT/lm_understand-logs/tree/main/harness

Replication and critique, all within twenty-four hours

• Simon Willison — Qwen 3.8 27B on my laptop can interpret model weights, too (2026-11-03). Three of the four games correct on a seventeen-gigabyte open-weights model running locally; failed on Freeway: https://simonwillison.net/2026/Nov/3/qwen-can-interpret-model-weights/
• Matthew Honnibal — The "Astra can read and write weights to improve Qwen" post is misleading. The decomposition table showing the no-tools raw-weights condition holds in experiments one, two and four only: https://honnibal.dev/blog/astra-read-write-misleading
• Sean Goedecke — LLMs are not superhuman at reading and writing neural network weights. Experiment one's insight reproduced in three lines of numpy, plus the compile-don't-patch argument: https://www.seangoedecke.com/llms-are-not-superhuman-at-reading-and-writing-neural-network-weights/
• Hacker News on the blog post: https://news.ycombinator.com/item?id=50341174
• Hacker News on Honnibal's critique: https://news.ycombinator.com/item?id=50351111
• Hacker News on Goedecke's critique, including his edit insisting he is impressed: https://news.ycombinator.com/item?id=50344861

Prior art and background

• Heretic, the training-free model modification tool whose abliteration playbook Goedecke argues experiment eight reproduces: https://github.com/p-e-w/heretic
• AdderBoard, a hand-built transformer that adds two ten-digit numbers: https://github.com/anadim/AdderBoard
• vgel — A handmade transformer, weights written by a human rather than trained: https://vgel.me/posts/handmade-transformer/
• Elhage et al. — A Mathematical Framework for Transformer Circuits, where induction heads are introduced. Astra derived the detection method unprompted: https://transformer-circuits.pub/2021/framework/index.html
• Olsson et al. — In-context Learning and Induction Heads: https://transformer-circuits.pub/2022/in-context-learning-and-induction-heads/index.html
• MinAtar, the miniature Atari suite used in experiment five: https://github.com/kenjyoung/MinAtar
• Kinetix, the physics environment whose official actor is repaired in experiment four: https://github.com/FLAIROx/Kinetix
• Ho, Jain, Abbeel — Denoising Diffusion Probabilistic Models, the DDPM of experiment two: https://arxiv.org/abs/2006.11239
• Hendrycks et al. — Measuring Mathematical Problem Solving With the MATH Dataset, the source of MATH five hundred: https://arxiv.org/abs/2103.03874
• Cobbe et al. — Training Verifiers to Solve Math Word Problems, GSM8K: https://arxiv.org/abs/2110.14168
• Zhou et al. — Instruction-Following Evaluation for Large Language Models, IFEval: https://arxiv.org/abs/2311.07911
• Gemma 3 270M instruction-tuned, the model in experiments seven and eight b: https://huggingface.co/google/gemma-3-270m-it

Authors

• Rikiya Takehi, co-first author, first-year PhD student at MIT CSAIL: https://rikiyat.github.io/
• Omar Khattab, Takehi's advisor and the author of DSPy: https://omarkhattab.com/
• DSPy: https://github.com/stanfordnlp/dspy
• Takehi et al. — MLLMs Fail to Refuse when Using Tools Agentically, NeurIPS 2026: https://arxiv.org/abs/2610.03938
• Takehi's WSDM 2026 Best Paper is Diversification as Risk Minimization. No stable link confirmed at time of publication, so none is given here rather than guessed at.

Numbers named in the episode

• Seventeen thousand two hundred fifty three weights in the PPO policy of experiment one; four games identified out of four, with the candidate list withheld
• Twenty-three thousand eight hundred twenty-four weights in the diffusion model of experiment two; three shapes named out of three, every one drawn as an outline where the model produces a filled shape
• Seven point one six million weights in the character transformers of experiment three, input and output layers removed; every induction head found and no false positives named
• One hundred seventy-one thousand six hundred sixty-four weights in the Kinetix actor; zero successes in two hundred fifty-six episodes before, one thousand twenty-four out of one thousand twenty-four after, three and a half percent of weights changed in one reply with no tools
• MinAtar scores against the hundred-million and five-hundred-million-step PPO baselines: Breakout 15.8 / 25.35 / 517.4, Seaquest 5.68 / 8.95 / 3.1, Asterix 25.8 / 55.2 / 53.2. Astra wins all three against the smaller baseline and only Breakout against the larger one
• One million four hundred eighty thousand two hundred seventy-four weights written from zero for 9x9 Go, as a three hundred thirty-six line Python program, with no Go library, no tree search and no rollouts
• Experiment seven: the sycophant follows one hundred percent of wrong user guesses against one to three percent for the honest model; head 10.2 attends to the question's numbers 94.3 percent of the time in the honest model and 0.75 percent to the user's answer, and that flips to 3.5 and 85.7 in the sycophant. From weights alone, four attempts, four failures
• One hundred thirty-one thousand seventy-two weights edited in Qwen3.5-0.8B, 0.016 percent of the model. MATH five hundred 7.9 to 48.1, GSM8K 13.8 to 67.1, IFEval 21.0 to 62.3, token-limit share 93.0 to 33.1
• The dose-response: MATH five hundred at 12.3 after ten forward passes, 14.1 after twenty-five, 48.1 after fifty
• The control: Gemma 3 270M hits its token limit on 8.5 percent of answers against 93 percent for Qwen, and the same technique moved it from 6.8 to at most 7.7, a gain the model itself reported as not established
• Willison's laptop run: a seventeen-gigabyte Q4_K_M quantisation of Qwen 3.8 27B in LM Studio, correct on Pac-Man at 72 percent confidence in 46 minutes 12 seconds, Pong at 75 percent, Breakout at 88 percent, and wrong on Freeway

Production notes

• Bob's voice profile is sharp.
• Not a peer-reviewed paper and not an arXiv preprint. A blog post with released logs. Stated on air in the first two minutes and never softened.
• Research dossier with VERIFIED / REPORTED / UNVERIFIED tags on every claim: data/episodes/0072/research.md

Credits

Daily Tech Feed: From the Labs. Scripts written by Claude Opus. Voices rendered by Qwen3-TTS. Not affiliated with any companies or researchers discussed.