mixture of experts
13 published articles
Open-weight AI
Xiaohongshu's dots3-note: a 280B open MoE that only activates 16B
Xiaohongshu's dots studio has released dots3-note preview, its first open-weight model: a multimodal MoE with 280B parameters, 16B active, and a 512K context window. The sparse design targets low serving cost on one 8-GPU node, but the card has not published benchmark numbers yet.
2026-08-20
Open Source AI: Alibaba Opens the Max Tier
Qwen 3.8-Max: Alibaba's most powerful model is now free to download
Alibaba is open-sourcing Qwen 3.8-Max, its most capable model ever: a 2.4T-parameter MoE that beats GPT-5.6 Sol on SWE-bench Pro, PaperBench, and IFBench. We break down the benchmark caveats and what a 95B-active open flagship means for developers.
2026-08-16
AI Models | NVIDIA Nemotron 3.5 Lightning
Why Nemotron 3.5 Lightning bets most agent steps don't need a big model
Nvidia's new open model runs locally with 3B active parameters and a 1M-token context, built for agents that stay running. Nvidia claims up to 4x throughput and 30% faster task completion than comparable open models.
2026-08-14
Open Source AI
Alibaba's biggest model coded alone for 16 days. Next week, its weights go public.
Qwen3.8-Max activates just 95 billion of its 2.4 trillion parameters, ranks second in Vision Arena, and built an agent framework in a 16-day autonomous run. Alibaba publishes the weights next week in its first open-weight flagship release.
2026-08-10
Open Source
A relay race of rented GPUs trained NanoColibri's 2.7B MoE for $200
NanoColibri-Instruct went from blank weights to a working 2.7B MoE for about $200. Volunteers passed a training baton on the Hugging Face Hub, one rented GPU at a time, with a compare-and-swap lease so no two people ever trained the same leg.
2026-08-04
Grok 4.5
Cursor's Grok 4.5 was built by AI agents, not humans. That's the real story.
Cursor's Grok 4.5 is a Mixture-of-Experts model built using reinforcement learning in environments created by earlier AI agents, not humans. It handles complex, long-duration tasks across software engineering, data science, finance, and law, and it's available now.
2026-07-10
Portfolio strategy
Qwen isn't chasing a single champion model, it's building a fleet
Alibaba Cloud's Qwen team quietly released a 35B MoE agent world model, three new ASR models, and an image-generation RL report, revealing a strategic bet on breadth over spectacle in the AI model race.
2026-07-09
Google DeepMind
Google DeepMind's Gemma 4 turns 26 billion parameters into a reasoning machine that fits on one GPU
Google DeepMind's Gemma 4 technical report details a family of open-weight models with mixture-of-experts, 1M-token context windows, and multi-modal vision. The release signals a strategic play to bring frontier-level reasoning to developers without the cost of proprietary APIs.
2026-07-09
Artificial intelligence
Aleph alpha's new megakernel library cuts moe inference latency by 200%
Alpha-MoE fuses multiple operations into a single persistent kernel to achieve up to 200% inference speed gains over Triton-based kernels in vLLM and SGLang, targeting FP8-precision MoE models.
2026-07-09
Artificial Intelligence
Ai2's EMO: the MoE model where modularity emerges from data, not human priors
Ai2's new MoE model, EMO, uses a novel training method that lets expert modules emerge naturally from data, enabling selective expert use with minimal performance loss. The model matches standard MoE performance on benchmarks while offering vastly improved modularity.
2026-07-07
Deep Dive
Aleph Alpha builds theoretical inference model for DeepSeek: Deriving performance from hardware primitives
Aleph Alpha created a theoretical inference model for DeepSeek v3 to estimate throughput from hardware parameters, analyzing trade-offs across GPU setups to help practitioners optimize performance and cost for large MoE models.
2026-07-05
LLM Inference Optimization
Aleph Alpha builds a theoretical inference model to decode DeepSeek V3 performance from hardware primitives
Aleph Alpha's theoretical model predicts DeepSeek V3 inference performance from hardware parameters alone, revealing how GPU count and interconnect bandwidth shift the bottleneck between compute, memory, and communication.
2026-07-04