Nvidia's Nemotron-4 runs 30% cheaper than GPT-4. The catch is buried in the benchmark.

Nvidia released Nemotron-4 today, an open-weight language model in 8B and 34B parameter variants. The company claims the 34B version scores within two points of GPT-4 on the MMLU benchmark while using 30% less compute during inference. The model is released under a permissive commercial license.

The numbers land squarely in a contested part of the open-model market. Meta's Llama 3.3 70B, Mistral Large, and Qwen 2.5 72B all cluster in similar accuracy bands. What sets Nemotron-4 apart is Nvidia's claim about inference cost: 30% less compute for the same task, trained on a mix of synthetic and curated web data. The open-model space has been watching this cost dynamic closely since Kimi K3 rewired AI economics earlier this year.

The gap Nvidia is attacking

The open model ecosystem has a well-known problem. You can pick a small fast model like Qwen 2.5 7B that runs on a single GPU but makes reasoning mistakes on multi-step tasks. Or you pick a big capable model like Llama 3.3 70B that scores high on benchmarks but needs multiple GPUs and expensive batch inference. There is not much in the middle that does both well. Recent work on 33B models beating 137B models on coding benchmarks shows that size alone doesn't determine capability.

Nemotron-4 34B sits in that middle band. If the MMLU claim holds up under independent verification, it becomes one of the strongest open models near its size. And the smaller 8B variant gives teams a cheaper option for simpler routing tasks.

Nvidia's embedding release from earlier this year already showed the company understands this tier: strong enough to be useful, cheap enough to deploy widely. The Nemotron-4 line extends that logic to reasoning.

The unmarked territory

The source does not specify which GPT-4 variant Nvidia is comparing against, nor whether the two-point gap is on the full MMLU split or a subset. That matters because GPT-4 scores vary by version and by exact test conditions. A two-point gap on the standard MMLU benchmark is about 1-2 percentage points depending on the reference, real for comparisons, but not transformative. The LiveBench leaderboard shows that top models are now separated by just 2.2 points, making every point count.

The bigger question is how Nemotron-4 holds up on agentic tasks. The existing parallel agents research suggests the company has been thinking about this problem for a while. If the 30% compute savings are real at scale, teams running thousands of agent loops per day would see meaningful cost differences.

What is not in the announcement: third-party benchmarks, live latency numbers, or an open evaluation set. Early adopters will need to reproduce the results themselves. The agent harness matters as much as the model here.

Where it fits

The permissive commercial license is the key signal. Nvidia is not trying to lock Nemotron-4 behind a proprietary endpoint the way OpenAI or Anthropic lock their best models. It is betting that open weight adoption feeds the rest of its hardware and software ecosystem, chips, Nemo framework, CUDA optimizations.

For teams already running Llama or Mistral, the migration path is straightforward: same serving infrastructure, different weights. The question is whether Nemotron-4 actually wins on accuracy per token in practice, or just on a benchmark leaderboard. Teams running Microsoft's $600 million cost-cutting experiment with Chinese models know that replacing a model is never as simple as swapping weights.

The 30% compute savings claim, if accurate, gives Nvidia a clear narrative: cheaper inference without giving up capability. That is a stronger pitch than marginal MMLU gains. Teams optimizing for cost will benchmark this model against their own workloads before switching, and Nvidia seems to know that.