Labs & Research4 min read
AI Research
Muon groks modular addition faster, then its solutions collapse
Muon-trained transformers grok modular addition faster than AdamW, then lose the solution in every configuration tested. The paper pins the collapse to the representation-readout interface, where freezing either parameter group prevents it. Fourier analysis shows the task circuit survives and simply gets outvoted.
2026-08-19