LLMs & ModelsFeatured6 min read
LLMs & Inference
Speculative decoding is the nearest free lunch in LLM inference
Speculative decoding accelerates autoregressive generation by proposing candidate tokens with a cheap drafting mechanism and verifying them in a single target-model forward pass, without changing the output. This analysis breaks down the math, the main methods (draft models, EAGLE-3, DFLASH, Multi-Token Prediction, n-gram), and what acceptance rates actually mean for real-world latency.
2026-07-21