LLMs & Models3 min read
Long-context LLMs
Jet-Long's bifocal attention just killed the fixed-scaling trade-off for long-context LLMs
Jet-Long adapts RoPE rescaling dynamically per sequence length, using a local window and a long-range window merged via inclusion-exclusion. On Qwen3 models up to 128K context, it beats existing zero-shot baselines by over 2 percentage points on RULER and achieves the lowest perplexity on PG-19, all while adding less than 4% generation overhead.
2026-07-12