llama.cpp
2 published articles
LLMs & ModelsFeatured6 min read
LLMs & Inference
Speculative decoding is the nearest free lunch in LLM inference
Speculative decoding accelerates autoregressive generation by proposing candidate tokens with a cheap drafting mechanism and verifying them in a single target-model forward pass, without changing the output. This analysis breaks down the math, the main methods (draft models, EAGLE-3, DFLASH, Multi-Token Prediction, n-gram), and what acceptance rates actually mean for real-world latency.
2026-07-21
Tools & FrameworksFeatured3 min read
Local AI
Ollama 0.30 just made local AI cheaper than cloud inference for more people
Ollama 0.30 boosts NVIDIA inference by up to 20%, enables Vulkan GPU support by default for AMD and Intel devices, and expands GGUF model compatibility, including fine-tuned models from Hugging Face and support for tool-calling with coding agents.
2026-06-05