Inkling is open-source AI's $53 billion reality check

Thinking Machines dropped Inkling on Hugging Face this week without a paywall or a keynote. The release page reads more like a technical brief than a press rollout: a decoder-only MoE transformer with relative attention, hybrid sliding-window and global attention, a short 1D convolution over hidden states, and an MLP patchifier for vision instead of a separate encoder stack. The model is multimodal at the architectural level, not bolted on.

At 975 billion total parameters with 41 billion active per token, backed by 45 trillion training tokens spanning text, images, audio, and video, Inkling enters a market that has shifted dramatically since the days when Llama 2 defined the open-source landscape. Open-source AI no longer just chases proprietary scores, it sets some of them. Inkling's arrival sharpens a question that matters more than where it lands on a specific benchmark: what exactly does 'open' mean now? That question has real stakes, see Alibaba's $53 billion bet on AI infrastructure.

A technical singularity, strategically framed

The most notable architectural choice in Inkling is the absence of separate per-modality encoders. Visual input goes through a hierarchical MLP that merges pixels into patch embeddings; audio is discretized into mel spectrogram bins embedded by a dedicated tower. This contrasts with models like Gemini or Claude, which rely on dedicated vision encoders (SigLIP or region proposal networks) that are mostly frozen at inference time. By folding everything into a single decoder with a 'short convolution' (a 1D conv over hidden states with a window of W), Inkling forces the attention mechanism to learn cross-modal reasoning rather than rely on alignment heads trained separately. This encoder-free approach echoes the design described in Gemma 4's architecture.

The relative attention mechanism is another departure. Most modern LLMs use rotary position embeddings (RoPE). Inkling uses a learned relative feature tensor R that is distance-adjusted per-head and injected directly into attention logits. The result is a model whose positional understanding is trained end-to-end rather than hardcoded, which may explain why it handles the full 1M context window without the degradation that often plagues RoPE-based models beyond 128k tokens. For more on dynamic RoPE scaling, see Jet-Long's bifocal attention breakthrough.

The speculative MTP (Multi-Token Prediction) layers add a drafter that predicts multiple future tokens at once. During inference the drafter acts as a speculative decoder, yielding 2-3x throughput gains on the same outputs. The approach is similar to Medusa but is integrated natively rather than as an inference-time add-on that required structural changes. Crucially, the drafter comes as a separate weight file, so anyone can serve it alongside the base model without structural changes. For more on speculative decoding limits, see DeepSeek's DSpark analysis.

Where the numbers fall, and what they hide

Inkling's benchmark suite is unusually thorough for an open release: 24 different tests covering reasoning, coding, agentic tasks, factuality, vision, audio, and safety. Selected results include:

The headline numbers tell a story of a model that competes with mid-tier proprietary systems on reasoning and agentic coding, but trails top-closed models like Claude Fable 5 and Gemini 3.1 Pro by 10-20 points on HLE and SWEBench. On AIME 2026, Inkling (97.1%) nearly matches DeepSeek V4 Pro (96.7%) and the unnamed GPT 5.6 Sol (99.9%). That parity in math reasoning would have seemed implausible two years ago for any open release. For context on coding agent cost-performance, see Cognition's SWE-1.7 results.

The audio benchmarks are where Inkling has no direct comparison. No comparable open model publishes Audio MC, MMAU, or VoiceBench scores, because few open models handle audio input at all. Thinking Machines is essentially defining the baseline here. VoiceBench at 91.4% is strong, though it lacks a closed-model competitor for calibration. The audio section of the benchmark table is mostly dashes, an acknowledgment of the field's gap and a convenient framing: Inkling cannot lose on a metric only it reports.

The safety numbers are genuinely interesting. FORTRESS Adversarial at 78% is above Kimi K2.6 (65.6%) and Nemotron (77.6%), but notably behind Claude (96.0%) and GPT-5.6 (82.4%). The benign FORTRESS score (95.9%) is competitive with everyone, and the StrongREJECT at 98.6% is near perfect. This suggests Inkling has been carefully aligned against standard jailbreak attempts but may be more vulnerable to adaptive attacks than top proprietary labs. For a model intended for fine-tuning and domain adaptation, the adversarial safety margin matters more than the benign one. 78% leaves room for misuse in a downstream deployment that does not add its own guardrails. On reward verification and agent safety, see the verification horizon analysis.

The hardware reality check

The BF16 checkpoint requires 2 TB of VRAM. The NVFP4 variant cuts that to 600 GB on Blackwell GPUs. This is not a model that runs on a single workstation. Deployment means a cluster of H100s or B200s, a Slurm script, and a willingness to manage tensor parallelism across nodes. The Llama.cpp GGUF quantizations from Unsloth bring memory down to about 30 GB, enabling inference on a single high-end GPU, but at 1-bit precision, with the inevitable quality trade-off that the 74.2% accuracy retention figure implies. On Unsloth's quantization technique, see their detailed breakdown.

Day-0 support in Transformers, SGLang, and vLLM is real and well-documented. Code snippets are detailed, with both high-level pipeline usage and lower-level AutoModel patterns. The Hugging Face Inference Providers route, where Thinking Machines covers inference costs for two hours, lowers the barrier to experimentation. But the practical entry point for most developers will be the quantized Llama.cpp path, which sidesteps the cluster requirement at the cost of the model's full capability.

What 'open' means here

Thinking Machines has not published Inkling's license on the release page in the provided source material. The term 'open weight' appears, but the weight license, data license, and any restrictions on distilled fine-tunes are absent. For a model of this scale, the license choice is arguably more consequential than a percentage point on HLE. An open-weight release under a permissive license (MIT, Apache 2.0, or even Llama 2-style) would be genuinely disruptive. A research-only or non-commercial license would make Inkling a technical artifact rather than an ecosystem player. For a contrasting approach, see Gemma 4's infrastructure-first philosophy.

The source material also does not detail the training data composition of the 45 trillion tokens, beyond listing modalities. The ECHO algorithm used for RL post-training (which trains the model to predict environment outcomes without a verifier) is described but unvalidated by external replication. These gaps do not undermine the release. They are standard for a model at this stage, but they mean the community evaluation of Inkling is still waiting for the community to run it.

The market map has shifted

Two years ago, an open model at Inkling's performance level would have dominated headlines for a week. Now it lands into a landscape where:

• Mistral AI has demonstrated that small open models (8B-120B) can compete with 10x-larger closed ones on niche reasoning tasks
• DeepSeek has shown that 600B+ open MoE models trained on carefully curated synthetic data can match frontier labs on math and coding
• Llama 4 has blurred the line between open-weight and proprietary, with Meta controlling the downstream license tightly
• Per-modality evaluation (vision, audio, video) has become a separate competitive axis, not an afterthought

Inkling sits in an uncomfortable middle: too large to run affordably without quantization, competitive but not dominant on reasoning, and unique on audio but uncalibrated against closed alternatives. Its strongest value may be as a research platform for multimodal MoE architectures, the attention mechanisms, the shared-expert sink with 256 experts, the short convolution design, rather than as a deployable product out of the box.

The release also reveals something about the state of open model development. The gap between open and proprietary in the 500B-1T parameter tier has narrowed to the point where a single percentage point separates them on several benchmarks. That does not mean open models have caught up. Proprietary labs still lead on HLE by 15-24% and on adversarial safety by significant margins. But it means the conversation is no longer about whether open models can compete. It is about which axis they choose to compete on, and whether that axis is the one the market actually values.