Make your AI agents think in a whisper, not a shout, and save 75% on tokens

Researchers from UIUC, Stanford, NVIDIA, and MIT built RecursiveMAS, a multi-agent framework that treats the whole system as a single recursive computation in latent space. The preprint asks a simple question: if recursive language models work for single models, why not for agents? The parallel to other agent coordination efforts is worth noting, as described in work on parallel agent architectures.

Why latent-space recursion?

The insight is almost too obvious once you hear it. In most multi-agent systems, every thought gets decoded into words before passing to the next agent. That is expensive and creates a semantic bottleneck. RecursiveMAS keeps all intermediate reasoning in continuous latent space, only decoding to text in the final round.

The framework connects heterogeneous agents through RecursiveLink, a lightweight two-layer residual network. Inner links map an agent's last hidden state back into its input embedding space. Outer links transfer latent thoughts between agents, even when those agents have different hidden sizes. The residual design means the module only needs to learn the distributional shift, not the whole semantics. This approach aligns with research on how evolution-like search can structure agent reasoning.

Architecture and training

RecursiveMAS chains all agents into a single recursive loop. Each agent generates latent thoughts via the inner link, then passes them to the next agent via the outer link. After the last agent finishes, its outputs feed back to the first, closing the loop. Training uses a two-stage inner-outer loop algorithm. The inner loop warm-starts each RecursiveLink with a regression objective aligned to the ground-truth answer's embedding distribution. The outer loop unrolls the full system for multiple recursion rounds and optimizes all links via backpropagation through the entire trace. Only about 13 million parameters, or 0.31% of the full system, are trainable. The base LLMs stay frozen.

Theoretical guarantees

Two results stand out. First, text-based recursive multi-agent systems pay a per-step decoding cost proportional to vocabulary size. RecursiveMAS replaces that with a much cheaper latent-space transformation, removing the vocabulary projection bottleneck. Second, under realistic assumptions with confident token predictions, text-based recursive fine-tuning suffers from vanishing gradients. RecursiveMAS maintains stable, near-constant gradients across looped backpropagation. This is a known subtlety in agent production deployments, as discussed in the subtle trap waiting for AI agents in production.

Performance and efficiency

Across nine benchmarks covering math, science, medicine, code, and search, RecursiveMAS consistently beats strong baselines. On MATH500, AIME 2025, AIME 2026, GPQA-D, LiveCodeBench, and MedQA, the framework delivers an average accuracy improvement of 8.3% over the best baseline under matched training budgets. On the logic-dense AIME 2025 competition math tasks, it improves by 18.1%. On held-out code generation benchmark MBPP+, it gains 13%.

The efficiency numbers are even starker at deeper recursion rounds. At round 3, RecursiveMAS achieves up to 2.4x end-to-end inference speedup over text-based recursive multi-agent systems, while reducing token usage by 75.6%. These advantages compound as recursion deepens, because most rounds happen entirely in latent space. The trend mirrors broader cost-efficiency findings in open-source coding model benchmarks.

Generalization across collaboration patterns

RecursiveMAS is structure-agnostic. The team instantiated it under four collaboration patterns: sequential (planner-critic-solver), mixture (domain experts with summarizer), distillation (expert-learner), and deliberation (reflector and tool-caller). Across all patterns, the framework improves over the strongest standalone agent: 6.2% in mixture style, 4.8% in deliberation style, and 8.0% in distillation style, where it lifts a smaller learner while preserving a 1.5x speed advantage over a larger expert.

What this means for multi-agent systems

The work points to a new scaling axis for multi-agent systems. Instead of adding more agents or larger models, recursion depth in latent space yields significant performance and efficiency gains. That opens the door to more flexible agent coordination without the overhead of decoding to text at every step, potentially making complex multi-agent architectures practical for resource-constrained deployments. This reinforces a broader shift toward smaller, specialized models redefining AI's future.