LLM training
5 published articles
Open Source AI
Boris-2 feeds its 125M model 90B tokens, its 250M just 60B
Boris-2 pre-announces three small models with lopsided token budgets: the 125M gets 90B tokens, the 250M just 60B. Training runs from August 12 to September 10, with no benchmarks shared, only the stated ambition to reach SmolLM2-135M strength.
2026-08-16
Open Source
A relay race of rented GPUs trained NanoColibri's 2.7B MoE for $200
NanoColibri-Instruct went from blank weights to a working 2.7B MoE for about $200. Volunteers passed a training baton on the Hugging Face Hub, one rented GPU at a time, with a compare-and-swap lease so no two people ever trained the same leg.
2026-08-04
LLM distillation
LLM distillation stalls when context goes static. Flux-OPD makes it evolve
Contexts can carry task preferences into LLM training, but once distilled into a student they add little supervision. Peking University's Flux-OPD keeps the context moving with the student and uses a conflict term to weight teacher corrections. The abstract reports gains over existing OPD paradigms without naming numbers.
2026-07-31
Training optimization
Unsloth's new kernels just made LLM fine-tuning 5x faster, no VRAM overpay
Unsloth's latest update introduces fused QK RoPE Triton kernels for 2.3x faster rotary embeddings, int64 indexing for long context, and auto padding-free packing. Benchmarks show 1.7-3x faster throughput on Qwen3-32B with no accuracy loss.
2026-07-16
Synthetic Data Strategy
Nvidia's data atlas shows why synthetic data matters more than model weights
Nvidia's Nemotron Post-Training v3 Prompt Atlas provides an interactive map of billions of synthetic data samples, highlighting how open synthetic data is the missing layer for building reliable AI agents. The company argues that agent behavior must be inspectable and that synthetic data, released openly, is the only way to preserve proprietary signals without exposing trade secrets.
2026-07-12