LLM inference
4 published articles
Test-Time Scaling
CoBa routing matches best-of-16 voting with 58.9% fewer tokens
CoBa, a compute-balanced routing policy from a new arXiv paper, matches best-of-16 majority voting within 0.01 points while cutting parameter-weighted tokens by 58.9%. On 3,129 evaluations across MATH-500, AIME, and AMC, it also beats single-sample decoding outright, though a small best-of-16 edge remains when budget is no object.
2026-08-19
Open Source Tools
30 GitHub repos that power the AI stack, from agents to local inference
A practical rundown of 30 essential GitHub repositories for AI developers, grouped by use case and ranked by community adoption.
2026-07-30
Systems optimization
DeepSeek just proved most AI inference speed tricks are working against each other
DeepSeek's DSpark paper reveals that naive speculative decoding degrades throughput under high concurrency. Its solution, confidence-scheduled verification, adapts block length per request and shifts the Pareto frontier of serving performance.
2026-07-12
LLM Inference Optimization
Aleph Alpha builds a theoretical inference model to decode DeepSeek V3 performance from hardware primitives
Aleph Alpha's theoretical model predicts DeepSeek V3 inference performance from hardware parameters alone, revealing how GPU count and interconnect bandwidth shift the bottleneck between compute, memory, and communication.
2026-07-04