Synthetic data generation just ditched its weakest link: the boss

Synthetic data has become the lifeblood of large language model training, filling gaps where real data is scarce, expensive, or off-limits due to privacy. Yet the dominant paradigm for generating such data at scale still relies on a central orchestrator, a single point of control that assigns tasks, collects results, and manages state. That architecture works until it doesn't: bottlenecks appear, concurrency struggles, and one agent's slow inference holds up the entire pipeline. See also Gartner's take on AI agents in the enterprise.

A preprint published this month, Matrix: Peer-to-Peer Multi-Agent Synthetic Data Generation Framework, takes a fundamentally different approach. The researchers removed the central orchestrator entirely. Instead, both control flow and data flow are passed as serialized messages through distributed queues. Each agent operates independently, and compute-intensive operations like LLM inference or containerized environments run on separate distributed services built on Ray.

"Existing frameworks for multi-agent synthesis often depend on a centralized orchestrator, creating scalability bottlenecks, or are hardcoded for specific domains, limiting flexibility," the authors write. "Matrix eliminates the central orchestrator."

Queues over calls

The core shift in Matrix is from synchronous, orchestrator-mediated task management to asynchronous, message-driven peer-to-peer coordination. Instead of a master agent directing each step, agents subscribe to queues and pick up relevant messages when they arrive. This decoupling means a slow agent doesn't stall the system. Other agents keep processing their own queues, and results eventually flow back once the straggler finishes.

The framework distinguishes between lightweight "control agents" that handle logical routing and "service agents" that do the real compute, calling an LLM, running a Python sandbox, or scraping structured data from a web page. Because services are pooled and reusable, multiple control flows can share the same inference endpoint, increasing utilization.

Concrete gains across three scenarios

The paper evaluates Matrix against baseline orchestrated systems on three synthetic data generation tasks: multi-agent collaborative dialogue, web-based reasoning data extraction, and tool-use trajectory generation for customer service.

All gains happened on identical hardware (a 16-node Ray cluster with A100 GPUs, per the paper), and the authors report no measurable difference in output quality when evaluated by human raters blind to which framework was used.

Why throughput matters more than ever

The synthetic data market is projected to grow rapidly as model builders look to augment human-generated datasets. But generating quality data isn't cheap. Each dialogue or trajectory can require multiple LLM calls, sandbox executions, and validation steps. A framework that doubles, or in some cases 15-times, the output per dollar of compute has direct implications for the cost structure of building frontier models. As Groq's licensing deal with Nvidia shows, the compute efficiency arms race is accelerating.

Matrix is built on Ray, an open-source distributed computing framework already used in many AI labs for training and inference. The modular design lets users write new agent types and queue handlers without rewriting the core message-passing layer.

The trade-offs

Decentralization isn't free. Because agents don't share a global view of progress, debugging and observability require more infrastructure. The paper mentions a separate monitoring layer built on Ray's dashboard and custom logging. Latency for individual tasks can also increase slightly under low concurrency, since the queue-based architecture adds message serialization overhead that a direct function call avoids. Counting the human hours saved becomes harder when the architecture itself is distributed.

Still, for high-volume synthetic data generation where raw throughput is the primary metric, the trade-off is worth it. "We achieve the higher throughput without compromising output quality," they note, comparing favorably to centralized alternatives that top out under heavy load.

What comes next

The paper stops short of releasing a reference implementation, though the authors say the code is built on standard Ray primitives and could be replicated by any team familiar with the framework. Given the reliance on proprietary or inference-grained LLM access, the paper reads more as an architectural blueprint than a drop-in library. Still, its core insight, that synthetic data generation should be treated as a distributed streaming problem rather than a centralized orchestration problem, is likely to influence how future generation pipelines are designed.

For teams currently bottlenecked on synthetic data volume, the message is clear: the orchestrator might be the part you want to remove. That shift mirrors what IBM's CUGA framework is doing for general-purpose agents: removing orchestration overhead so the agents themselves do the work.