speculative decoding
3 published articles
Systems optimization
DSpark shows why fast AI inference is a scheduling problem, not a model trick
DeepSeek's DSpark paper reveals that naive speculative decoding degrades throughput under high concurrency. Its solution, confidence-scheduled verification, adapts block length per request and shifts the Pareto frontier of serving performance.
2026-07-12
DeepSeek's DSpark framework rearms speculative decoding for high-concurrency serving
Semi-autoregressive decoding just broke the 85% speed barrier in production AI inference
A new speculative decoding framework from DeepSeek tackles the two bottlenecks that have limited parallel drafters: suffix decay and wasteful verification. DSpark achieves 60, 85% faster generation speeds in production by coupling a semi-autoregressive architecture with a confidence-scheduled scheduler that prunes low-value tokens before the target model verifies them.
2026-07-08
Performance
Gemma 4 runs 90% faster in Ollama 0.31 with a trick that needs no config
Ollama 0.31 introduces multi-token prediction for Gemma 4 on Apple Silicon, achieving near 90% faster token generation on coding benchmarks. The speedup comes from an auto-tuned draft model and a custom MLX kernel that eliminates redundant weight reads.
2026-06-29