The MNIST-to-CIFAR flip that exposed a blind spot in brain-like AI research

For years, the quest to train neural networks the way brains might actually work has run into the same wall: it works on MNIST, the handwritten-digit dataset that has been the field's comfort zone since the 1990s, and falls apart on anything harder. A team at Sakana AI has now pushed one such method, Error Diffusion, onto convolutional networks, CIFAR-10 images, and continuous-control reinforcement learning. But the headline number is not the point. The point is what happened when the researchers changed the benchmark. The company's focus on routing problems to the right method feels relevant here, as their broader work on model routing suggests they think hard about benchmark specificity.

Why obeying the brain is expensive

Biological neurons follow Dale's principle: each neuron is either excitatory or inhibitory across all of its synapses, never a mix. Standard artificial networks ignore this entirely, any connection can be positive or negative, and backpropagation exploits that freedom by sending exact transposed copies of the forward weights backward. Neuroscientists have long objected that this "weight transport" has no biological basis.

The alternatives that avoid it, Feedback Alignment, Direct Feedback Alignment (DFA), predictive coding, all still permit arbitrary positive and negative weights. Sakana AI's dual-stream Error Diffusion goes further: it splits every layer into a positive and a negative population, keeps all four weight matrices per layer non-negative, and hardcodes the inhibitory sign into the architecture rather than learning it. That structural honesty carries a bill: roughly 4x the parameters of an unconstrained network of the same width (about 32 million versus 8 million for the DFA baseline).

The scores, in context

With a new "modulo error routing" trick that assigns each hidden unit a fixed output channel, the method reaches 96.7% on MNIST and establishes a 61.7% baseline on CIFAR-10, the first time Error Diffusion has been made to work on convolutional networks at all. Read against the competition, though, the results are a study in trade-offs rather than a victory lap. This kind of nuanced leaderboard reading echoes Mistral's own argument that benchmark numbers don't tell the real story.

Method	MNIST	CIFAR-10	Dale-compliant?
Proposed ED (dual-stream)	96.7%	61.7%	Yes
DFA (random feedback)	97.6%	69.1%	No (~2.84M negative weights)
Seed ED (no innovations)	50.4%	11.6%	Yes

The gap to DFA widens from 0.9 points on MNIST to 7.4 points on CIFAR-10. That spread is arguably the paper's cleanest contribution: a concrete price tag on the cost of enforcing Dale's principle, and evidence that the price grows with task difficulty.

The reversal that should worry benchmark-watchers

To make the architecture competitive, the team added three classification-specific tricks: layer-specific sigmoid widths (to fight vanishing gradients), batch-centered class error (to counter the 9-to-1 imbalance of one-vs-all targets), and asymmetric excitatory/inhibitory initialization. The instructive result is what an ablation study revealed when each was removed.

Component removed	MNIST impact	CIFAR-10 impact
Layer-specific sigmoid widths	−71.4 pp	−15.1 pp
Batch-centered class error	−0.3 pp	−47.9 pp
Asymmetric E/I init	+0.0 pp	−5.5 pp

On MNIST, removing the sigmoid-width fix is catastrophic while batch-centering barely registers. On CIFAR-10 the hierarchy flips: batch-centering becomes the difference between learning and collapse, and the sigmoid fix drops to secondary. A component that looks essential on one dataset looks negligible on the other. Had the researchers stopped at MNIST, as most work in this area effectively does, they would have concluded that error-signal centering does not matter. It matters enormously. This mirrors a broader observation that agents trained under narrow conditions flinch in the real world, and the same training trick logic applies here.

This is the analytical core of the work: the bottleneck in credit assignment is not a fixed property of the method, it is a property of the task. Evaluate a biologically plausible learning rule on a single benchmark and you may certify the wrong design decisions as universal truths.

The same lesson, from the reinforcement-learning side

The team also bolted Error Diffusion onto Proximal Policy Optimization and ran it on Google's Brax locomotion tasks and the open-ended Craftax benchmark. Here the roles switch again. On HalfCheetah, ED-PPO (5,494 average return) beats standard backprop-PPO (3,520) and matches DFA. But on Craftax, DFA, the classification champion, is the weakest method, falling below both backprop and Error Diffusion. Random feedback pathways that suffice for supervised learning fail on complex, exploratory RL.

The mirror image is deliberate: DFA is strong where ED struggles and weak where ED holds up. The message across both domains is identical. There is no single leaderboard that tells you which brain-inspired method is "best," because each method's weaknesses surface only under specific task demands. This is the same difficulty agent evaluation faces when benchmarks don't generalize; the bottleneck isn't just exploration but how we judge success.

What emerges on its own

One finding hints at why these constrained networks behave the way they do. The 3:1 excitatory-to-inhibitory initialization drifts, through training alone, toward roughly 1:1 balance, with deeper layers becoming progressively more inhibitory. That echoes how cortical circuits are thought to settle into balanced excitation and inhibition during maturation, and it arrives without any explicit balancing mechanism. Meanwhile the non-negative floor prunes 37.3% of weights to near-zero, hitting inhibitory connections hardest (up to 68.8%), a form of "free" sparsity, and a possible hardware advantage for analog, photonic or neuromorphic chips where physical synapses naturally encode non-negative quantities.

The honest bottom line

62% on CIFAR-10 is nowhere near what a conventional gradient-trained network achieves, and Sakana AI says so plainly. ED-PPO also carries markedly higher variance than backprop and stumbles on tasks needing fine-grained temporal credit assignment. The value here is not a state-of-the-art model; it is a well-instrumented map of where biologically constrained learning breaks and why. And the sharpest line on that map is a methodological one: if the field keeps grading brain-like AI on toy datasets, it will keep drawing confident conclusions that the next dataset quietly overturns. The pattern is not new across AI research, and this study adds a fresh, concrete example to the pile.