The 2.8 trillion parameter model that beats the frontier on the benchmarks that matter

The benchmark table that matters

Moonshot AI posted a 50-row benchmark table with its Kimi K3 announcement. The company's own summary, that overall performance still trails the most powerful proprietary models, is correct but incomplete. Sort the rows by which model leads, and a different picture shows up: Kimi K3 takes first on at least six major evaluations across coding, agentic, and vision tasks.

The table below lists every benchmark where Kimi K3 (max) finishes first or ties for first, along with the closest competitor's score. See also Ai2's open evaluation workbench for how benchmark methodology affects results.

Kimi K3 leads outright on SWE Marathon (42.0 vs 40.0), BrowseComp (91.2 vs 90.4), DeepSearchQA (95.0 vs 94.2), Automation Bench (30.8 vs 29.7), and OmniDocBench (91.1 vs 89.8). On Terminal-Bench 2.1 it trails GPT 5.6 Sol by 0.5 points, close enough to call a tie. On GPQA-Diamond and PerceptionBench it trails by less than 1.2 points. That's a tighter race than the overall-trailing line implies, and similar to how NousCoder-14B's open RL pipeline challenges closed models on programming tasks.

Coding: where open models win

The clearest pattern shows up on agentic coding benchmarks. SWE Marathon, measuring sustained software engineering over long sessions, puts Kimi K3 five points ahead of GPT 5.6 Sol and seven points ahead of Claude Fable 5. Moonshot AI credits K3's architecture: Kimi Delta Attention (KDA) and Attention Residuals (AttnRes), which improve information flow across long sequences. The model's 1-million-token context window and 16-of-896 expert activation density support long-horizon tasks without thrashing.

On Program Bench, Kimi K3 (77.8) edges Claude Fable 5 (76.8) and GPT 5.6 Sol (77.6), a narrow but consistent lead. On MLS Bench Lite (48.3 vs 49.9) and DeepSWE (67.5 vs 70.0) it loses, but by margins small enough that the open model is clearly in the same tier as proprietary frontier systems on coding. This pattern echoes how MiniMax's M2.7 matched Claude Opus on SWE-Pro, and how the broader verification problem for coding agents gets harder as they improve, as covered in the verification horizon analysis.

The internal Kimi Code Bench 2.0, which Moonshot says uses the KimiCode harness for K3 and Claude Code for competitors, shows K3 at 72.9, ahead of GPT 5.6 Sol (64.8) and Claude Opus 4.8 (71.7), behind only Claude Fable 5 (76.9). Harness choices matter here: Moonshot notes that GLM-5.2's scores come from a different harness, and that Claude Fable 5's scores include fallback to Opus 4.8 when usage policy blocks requests. Still, the pattern holds: on several coding benchmarks, K3 competes with or beats proprietary models that cost orders of magnitude more per token.

Agentic benchmarks: three clear wins

BrowseComp (91.2), DeepSearchQA (95.0 f1), and Automation Bench (30.8) are agentic evaluations that test a model's ability to navigate information, synthesize data, and execute multi-step workflows. On all three, Kimi K3 leads. Moonshot notes that BrowseComp used a context-compaction strategy at 300K tokens; without it, K3 scored 90.4, still ahead of every other publicly reported model.

On MCP Atlas (84.2) and Toolathlon-Verified (73.2), K3 trails Claude Fable 5 by 0.5 and 4.7 points respectively, solid scores, but not the top spot. On GDPval-AA v2 (1668 Elo vs 1760) and AA-Briefcase (1548 vs 1583), the gap is 92 and 35 Elo points, a visible gap that matches Moonshot's own admission that K3 exhibits a noticeable gap in user experience compared with Claude Fable 5 and GPT 5.6 Sol. The importance of such agentic capabilities is also highlighted by Alibaba's experiments with multi-agent group chats, which show that agentic coordination is becoming a key frontier.

Vision: OmniDocBench and PerceptionBench stand out

Kimi K3 leads on OmniDocBench (91.1 vs 89.8), a document understanding benchmark, and PerceptionBench (58.5 vs 57.2), which Moonshot describes as an internal atomic visual perception evaluation. On MathVision (94.3), MMMU-Pro (81.6), and CharXiv (84.8), K3 scores within 1-3 points of the leaders. The gap grows on harder multi-modal tasks: ZeroBench w/ python (41.0 vs 46.0) and BabyVision w/ python (85.7 vs 90.5) show that Claude Fable 5 and GPT 5.6 Sol still hold advantages on complex visual reasoning with tool use.

Moonshot reports that all visual evaluations used the same image-ordering and prompt format, and that K3 scored highest on document understanding without python tool augmentation, a meaningful edge given how much enterprise work involves document analysis. For context on how benchmark numbers can mislead, see Mistral's own audit of its OCR benchmarks.

Where the gap is real: reasoning and post-training

On HLE-Full, the hardest reasoning benchmark, Kimi K3 scores 43.5, 9.8 points below Claude Fable 5 (53.3) and 1 point below GPT 5.6 Sol (44.5). With tools, the gap narrows to 7 points behind Fable 5. On PostTrain Bench, measuring post-training quality, K3 scores 36.6, 4.8 points behind Fable 5 (41.4). These are the raw reasoning and instruction-following gaps that explain why Moonshot still admits it trails proprietary leaders.

Yet even here, the gap is narrower than the parameter difference would suggest. Claude Fable 5 likely uses significantly more than 2.8T parameters (Anthropic hasn't disclosed the exact count), and its training compute budget is unknown but almost certainly larger. That a 2.8T partially open model competes within single-digit points on GPQA-Diamond (93.5 vs 93.5), Terminal-Bench 2.1 (88.3 vs 88.8), and SpreadsheetBench 2 (34.8 vs 34.7) signals a structural shift in the landscape.

The architecture behind the numbers

Moonshot attributes these gains to three architectural innovations. First, Kimi Delta Attention (KDA), a new attention mechanism that Moonshot says improves scaling efficiency by roughly 2.5x compared to Kimi K2. Second, Attention Residuals (AttnRes), which selectively retrieve representations across depth rather than accumulating them uniformly. Third, Stable LatentMoE with 896 experts and 16 active per token, using a Quantile Balancing method to replace heuristic expert allocation. The combined effect, Moonshot claims, is a model that converts compute into intelligence more effectively than its predecessor. This approach to efficient scaling mirrors the findings in Jet-Long's bifocal attention paper, which rethinks long-context efficiency trade-offs.

Independent verification of these claims will have to wait for the technical report, which Moonshot promises alongside the weight release by July 27, 2026. The company also notes that KDA poses new challenges for conventional prefix caching and has contributed a corresponding implementation to the vLLM community, a clue that the architecture introduces real engineering tradeoffs.

What the table says about the open frontier

Kimi K3 doesn't beat the best proprietary models on every metric. But the benchmarks it leads, SWE Marathon, BrowseComp, DeepSearchQA, OmniDocBench, PerceptionBench, aren't obscure lab tests. They measure sustained coding over long sessions, multi-step information retrieval, document understanding, and visual perception: exactly the tasks that enterprise customers care about. On those tasks, the open model matches or exceeds proprietary systems that cost $15/MTok for output (K3's own API price) or more.

The 2.8 trillion parameter number is impressive, but the real story is narrower: open-weight models have become competitive on specific, high-value slices of the frontier. The question for the rest of 2026 is whether that gap widens, or whether proprietary labs widen the disparity on the remaining benchmarks where they still lead.