math reasoning
2 published articles
Benchmarks & TestsFeatured2 min read
Grade-School Math Benchmark Quality Audit
GSM8K Was Supposed to Test Grade-School Math. Up to 42% of Its Questions Don't
GSM8K became a standard test of arithmetic reasoning in LLMs, but audits have found error rates in its question set as high as 42%, undermining what its scores actually measure.
2026-07-29
AI5 min read
Mathematical Reasoning
The M3 team found a way to stop AI math verifiers from cheating, and it's a blueprint for every lab
M3's approach to mathematical reasoning combines three expert models, Proof, Verifier, and Fixed, with an evolutionary search framework. The account offers rare detail on reward hacking, verifier alignment, and the engineering required to make generative verifiers reliable in high-stakes reasoning tasks.
2026-07-11