Tools & Frameworks2 min read
Infrastructure deep-dive
Nous Research's MoE field notes: what actually happens when 1 trillion parameters hit 1024 GPUs
Nous Research publishes field notes on scaling MoE expert parallelism with DeepEP. The report details throughput, configuration trade-offs, and bottlenecks from pretraining a 1T-parameter MoE model, offering practical deployment insights for large-scale distributed training.
2026-07-19