KV cache
2 published articles
AI5 min read
Deep Tech
One routing pass to rule them all: 7x faster LLM decoding without the quality trade-off
CLSA uses one routing pass per sequence instead of one per layer, cutting KV-cache overhead while keeping token-level selectivity. Benchmarks show 17.1x throughput improvement at 128K context.
2026-07-26
DeepSeek4 min read
Open-source AI research
Million-token context on a tenth of the KV cache: DeepSeek-V4's efficiency bet
DeepSeek's V4 preview pairs a 1.6T-parameter Pro model with a 284B Flash variant, both at one million tokens of context. The paper claims 27% of the inference FLOPs and 10% of the KV cache of V3.2, the numbers that make the context cost-effective to serve.
2026-06-22