Video generation gets 120x faster on one GPU, inside Nvidia's hybrid attention breakthrough

The cost of generating video has always been attention. Not the kind you pay for clicks, but the self-attention mechanism inside diffusion transformers. Every frame adds more tokens, and full softmax attention makes every token look at every other token. That quadratic scaling turns a short clip into a long wait.

Nvidia Research's SANA-Video 2.0, released as a preprint in July 2026, tries to unbundle that cost. It mixes linear attention with periodic softmax steps at a 3:1 ratio, and the paper claims the result runs 120 times faster than Wan 2.2-A14B on a single H100 for 5-second 720p clips. That number is eye-catching, but unpacking it shows how much engineering went into closing the gap between theory and practice. Other work has attacked the same quadratic problem from different angles, such as CLSA's shared-routing approach.

The hybrid attention mechanism

The architecture avoids the quadratic penalty by replacing most softmax operations with a gated linear alternative. Every fourth attention step retains softmax, acting as a full-rank anchor. The idea is that pure linear attention loses expressiveness because tokens interact indirectly. The periodic softmax steps restore direct interaction. The paper calls this Hybrid Linear-Softmax Attention, and it combines a gated linear path for O(N) mixing with periodic softmax anchors at a 3:1 ratio.

To push those refreshed representations through the network, Block Attention Residuals (AttnRes) route completed block summaries into later layers. The paper reports a boost in deep-layer effective rank of about 12%. The model was trained from scratch to learn the hybrid pattern directly, avoiding the degradation that can happen when linearizing a pretrained transformer.

The optimization stack

On top of the backbone, Nvidia applied its Sol-Engine optimization stack, which includes kernel fusion, caching, and sparse attention. The paper says this additional layer accelerates the hardware-friendly backbone by a further 3.58x. Stacking both improvements puts the 5-billion-parameter pipeline at 13.06 seconds for a 720p 5-second clip on one H100. This kind of efficiency on a single GPU echoes other compact designs like Microsoft's Mage-Flow 4B model aimed at image generation. At 720p and 60 seconds, the compiled DiT forward pass is 3.2 times faster than a matched full-softmax baseline, and the gap widens with longer videos.

Benchmarks and comparisons

Quality is measured on VBench, where SANA-Video 2.0 scores 84.30 at 480p using 40 sampling steps, completing in 13.2 seconds on a single H100. The authors say this is competitive with much larger full-softmax video DiTs. But video quality benchmarks remain a contested ground, Kling's recent MultiRef-Compass and KeyFrame-Compass suggest current metrics still miss key failure modes. At 720p and 5 seconds, the full pipeline is about 120 times faster than Wan 2.2-A14B. The gap grows with video duration, which is important for long-form generation.

Open questions and trade-offs

The paper tests at 480p and 720p up to 60 seconds. It does not explore 1080p or longer sequences. The 25% softmax ratio was chosen from proxy studies at lower resolution, and it is not clear if the same ratio holds for higher resolutions or longer context. The speedup numbers bundle the attention change with Sol-Engine, making it hard to isolate the contribution of each component. And the comparison to Wan 2.2, a different model with different architecture and training, leaves room for debate about how much of the 120x factor is generalizable beyond Nvidia hardware.

These are not fatal flaws, but they are real caveats for anyone evaluating whether this approach is the right fit for their own pipeline. The engineering is impressive. The claims are large. But preprints are not press releases for a reason.