MoE
4 published articles
AI Models
The 33B model that just beat 137B models on coding benchmarks without changing hardware
Poolside's Laguna XS 2.1 improves SWE-bench Multilingual by 5.4 points to 63.1% while keeping the same 33B-total-3B-activated MoE architecture. The release includes quantized checkpoints, speculative decoding draft models, and an OpenMDW-1.1 license, making local AI coding more practical.
2026-07-21
Infrastructure deep-dive
Nous Research's MoE field notes: what actually happens when 1 trillion parameters hit 1024 GPUs
Nous Research publishes field notes on scaling MoE expert parallelism with DeepEP. The report details throughput, configuration trade-offs, and bottlenecks from pretraining a 1T-parameter MoE model, offering practical deployment insights for large-scale distributed training.
2026-07-19
Artificial Intelligence
Inkling is open-source AI's $53 billion reality check
Inkling is the first openly available near-1T parameter model with native audio, image, and text input alongside a 1M-token context window. The raw benchmark scores are strong. The real story is how the open-source ecosystem has moved from playing catch-up to competing at the frontier, and where Inkling fits on that new map.
2026-07-15
Google DeepMind
Gemma 4 just made every other open-weight model look 10x too big
Google DeepMind's Gemma 4 natively multimodal open-weight family introduces thinking mode, encoder-free architecture, and MoE options. The 2.3B model matches Gemma 3's 27B performance. The 31B model tops open-weight leaderboards.
2026-07-13