OpenForgeRL trains AI agents without touching their inference harnesses

Modern AI agents don't rely on a single model call. They lean on inference harnesses like Claude Code, Codex, and OpenClaw to manage multi-turn reasoning, tool use, and external systems. Those harnesses make agents capable, but they also make training a headache. Standard supervised fine-tuning and RL stacks weren't built for stateful, multi-process harness inference. Researchers often had to pick: use a powerful harness or train at all. That's the gap OpenForgeRL aims to bridge.

OpenForgeRL, from a team of researchers, sidesteps the tradeoff. It introduces a lightweight proxy between the harness and the model. The proxy intercepts model calls during inference, logs inputs and outputs, and pipes them into a standard RL codebase like veRL. A Kubernetes orchestrator runs each rollout in its own container, scaling across environments without touching the harness. Unlike fully open training stacks that demand custom infrastructure, OpenForgeRL works with existing harnesses.

The team validated the approach on two agent types: tool-based code agents (OpenForgeClaw) and multimodal GUI agents (OpenForgeGUI). With just hundreds to a few thousand training tasks, OpenForgeClaw scored 31.7 pass^3 and 55.9 pass@3 on ClawEval and 33.7 on QwenClawBench. OpenForgeGUI hit 37.7 on OSWorld-Verified, 63.0 on Online-Mind2Web, and 72.3 on WebVoyager. On almost every benchmark, both agents outperformed open baselines of similar size. In the GUI setting, they matched or surpassed models several times larger, similar to results from other open web agents.

Beyond the numbers, the paper digs into how different harnesses respond to RL training. Not all are equally trainable. Some are substantially harder to learn from. The researchers found that RL consistently improved agentic reliability factors: self-verification, tool coverage, and multi-step planning. One weakness remained: error recovery stayed poor even after training, a gap for future work. This echoes the broader pattern of open-weight models driving the new normal in AI development.

The real contribution is practical. OpenForgeRL decouples training from inference, letting researchers train agents directly in the harnesses and environments they'll deploy in. The code is open source, so teams can reproduce and extend the work without proprietary infrastructure, much like xAI's open release of Grok Build.