LLM training
3 published articles
LLM distillation
LLM distillation stalls when context goes static. Flux-OPD makes it evolve
Contexts can carry task preferences into LLM training, but once distilled into a student they add little supervision. Peking University's Flux-OPD keeps the context moving with the student and uses a conflict term to weight teacher corrections. The abstract reports gains over existing OPD paradigms without naming numbers.
2026-07-31
Training optimization
Unsloth's new kernels just made LLM fine-tuning 5x faster, no VRAM overpay
Unsloth's latest update introduces fused QK RoPE Triton kernels for 2.3x faster rotary embeddings, int64 indexing for long context, and auto padding-free packing. Benchmarks show 1.7-3x faster throughput on Qwen3-32B with no accuracy loss.
2026-07-16
Synthetic Data Strategy
Nvidia's data atlas shows why synthetic data matters more than model weights
Nvidia's Nemotron Post-Training v3 Prompt Atlas provides an interactive map of billions of synthetic data samples, highlighting how open synthetic data is the missing layer for building reliable AI agents. The company argues that agent behavior must be inspectable and that synthetic data, released openly, is the only way to preserve proprietary signals without exposing trade secrets.
2026-07-12