AI benchmarks
4 published articles
Orchestrator swap
Perplexity swapped its orchestrator model and the cost-performance chart is brutal
Perplexity swapped Grok 4.5 into Computer as its orchestrator model, claiming top accuracy on its internal benchmark at roughly half the cost of Claude Opus 4.8. The move signals a broader shift: which model you pair with an agent matters as much as the model itself.
2026-07-14
Benchmark integrity
GPT-5.6 Sol almost cracked a physics benchmark built so AI couldn't cheat
GPT-5.6 Sol (max) leads CritPt, a new physics benchmark built from unpublished graduate-level research problems, scoring roughly 5 points above GPT-5.5 and 4 points ahead of Claude Fable 5, but even the winner solved only a third of the problems.
2026-07-13
Frontier AI Deployment
Claude Fable 5 and Mythos 5: The Future of AI Is Gated Intelligence
Anthropic's Claude Fable 5 brings Mythos-class capability to public users, while Mythos 5 remains trusted-access only. The deployment model, capability routing, fallback classifiers, and tiered access, represents a fundamental shift in how frontier AI is released and used.
2026-07-10
Artificial Intelligence
Ifbench reveals the instruction-following gap that other benchmarks miss
IFBench measures language models' ability to follow precise natural-language instructions. xAI's Grok models lead while Claude models lag, showing instruction following is a distinct capability from general intelligence.
2026-07-07