multi-token prediction
2 published articles
LLMs & ModelsFeatured6 min read
LLMs & Inference
Speculative decoding is the nearest free lunch in LLM inference
Speculative decoding accelerates autoregressive generation by proposing candidate tokens with a cheap drafting mechanism and verifying them in a single target-model forward pass, without changing the output. This analysis breaks down the math, the main methods (draft models, EAGLE-3, DFLASH, Multi-Token Prediction, n-gram), and what acceptance rates actually mean for real-world latency.
2026-07-21
Benchmarks & TestsFeatured3 min read
Performance
Gemma 4 runs 90% faster in Ollama 0.31 with a trick that needs no config
Ollama 0.31 introduces multi-token prediction for Gemma 4 on Apple Silicon, achieving near 90% faster token generation on coding benchmarks. The speedup comes from an auto-tuned draft model and a custom MLX kernel that eliminates redundant weight reads.
2026-06-29