Ifbench reveals the instruction-following gap that other benchmarks miss

An AI model's job is not just to produce plausible text but to follow instructions exactly. That sounds deceptively simple until you ask for a three-sentence summary in a casual tone that uses the word "orange" but not "apple." A model can grasp the topic perfectly and still flub the ask.

Accepted to NeurIPS 2025, IFBench tests that precise kind of instruction following. Last year, Artificial Analysis, an independent benchmarking organization, added it to its Intelligence Index, a composite that mixes several evaluations into one overall score. The addition reshuffled rankings in ways that caught attention.

"We saw that a model's ability to follow user instructions was something developers cared about a lot, so we wanted to assess it explicitly," says Declan Jackson, a member of the technical staff at Artificial Analysis. "IFBench was designed to fill that gap and was challenging even for the frontier models."

Measuring adherence to a prompt

IFBench forces a model to juggle several rules at once. Some rules are straightforward, like minimum word counts or required keywords. Others are trickier: sentences must match in length, consecutive words cannot start with the same letter, or a keyword has to land in a precise spot.

"That is different from many other benchmarks, where instruction following is captured only indirectly through output templates or requested answer structures," says Jackson.

Each constraint might seem arbitrary on its own. But together they mirror a common user scenario: ask a model for multiple things at once, and missing one detail can ruin the response. To ground IFBench in real use, its prompts come from actual user conversations, not researcher-written templates. This makes it harder to game than many other evaluation suites, as noted in a broader analysis of sandbox benchmarks.

"IFBench measures instruction following in a way that feels closer to real-world use than earlier instruction following evals," says Jackson. "The prompts use casual, user-like language, cover a wide range of tones and lengths rather than following a fixed template, and focus on common tasks such as factual question answering, content review and summarization, and creative support. IFBench's wider coverage also makes it a stronger overall signal of instruction-following ability."

What IFBench shows that other benchmarks miss

AI benchmarks usually have a short shelf life. Once models start scoring near the top, the evaluations stop telling systems apart. Most evaluations added to Artificial Analysis's Intelligence Index saturate within about six months, says Jackson.

But IFBench has not saturated yet.

"While IFBench scores have improved over time, that progress has not been uniform across models, and new frontier models still do not always perform well on it," says Jackson.

Two factors explain that. First, complex instruction following does not overlap much with what most labs actively train for, says Jackson. Coding and tool use get heavy post-training investment because gains there tend to generalize across tasks and benchmarks. Instruction following is narrower and rarely improves as a side effect of progress in those areas.

Second, the sheer breadth of what IFBench measures makes progress slower than on targeted domain evaluations, which labs can chip away at with focused post-training recipes.

The numbers bear out that divergence. IFBench scores cluster sharply by model family, and the rankings do not align with the broader Intelligence Index. xAI leads: Grok 4.20 (0309, Reasoning) takes the top spot at 82.9%, with Grok 4.3 close behind at 81.3%. Recent Google models also score well: Gemini 3 Flash Preview (Reasoning) reaches 78.0%, while Gemini 3.1 Flash-Lite Preview and Gemini 3.1 Pro Preview land at 77.2% and 77.1%. OpenAI's GPT-5.5 (xhigh) and GPT-5.4 (xhigh) follow at 75.9% and 73.9%. The leading Claude models cluster lower on IFBench, with Claude Opus 4.7, Claude Sonnet 4.6, and Claude 4.5 Haiku scoring between 54.3% and 58.6%, even though Claude Opus 4.7 ranks near the top of the Intelligence Index at 57 points, behind GPT-5.5 (xhigh) at 60 and effectively tied with Gemini 3.1 Pro Preview and GPT-5.4 (xhigh). These results show how instruction following is a distinct capability, much like the price-performance gap that Sonnet 4.6 exposed.

A truly open approach to evals

IFBench is useful to Artificial Analysis for two reasons: what it measures, and the fact that it is released openly. Openness lets Jackson's team implement the evaluation faithfully and run it across many models, feeding the leaderboards their users rely on. It also makes the benchmark easier to understand, since anyone can see what is being measured and why.

For Artificial Analysis, IFBench tests something that comes up in nearly every interaction: whether a model can keep track of what a user is asking, especially when the request has a lot going on. It is now a regular part of Artificial Analysis's evaluations. That kind of open, transparent benchmark design is part of a broader shift in evaluation methodology, as seen in Ai2's fully open Olmo 3 release.

"Beyond evaluations, Ai2 is an important leader in open source," says Jackson. "Their work not only helps advance the industry through open research, but also gives users access to research artifacts with transparency around data and methodology."