Harness-of-Harness claims a 52.25% lift by running other coding agents

The Harness-of-Harness preprint went up on arXiv on 1 September 2026 and argues for a split most agent papers avoid. Instead of shipping a coding agent, it wraps the ones that already exist and takes charge of what they do next.

HoH organises each harness's runs into planning, coding and testing loops, then repeats those loops so every pass starts from what the last one left behind. The abstract's claim is that this outer layer lifts performance without touching the model underneath: an average relative gain of 52.25 percent, a top figure of 82.86 percent, measured after three iterations.

The claim: a harness that improves other harnesses

Much of the abstract is about what HoH refuses to do. It constrains what counts as a verifiable output instead of prescribing how an agent should work, and it cuts development into small increments that can be checked. Testing done while code is being written stays separate from an evaluation step that runs on its own.

Then there is the balance the authors keep returning to: repair against capability growth. A loop that only fixes defects keeps a project alive without moving it forward. A loop that only adds features piles up breakage. HoH is described as holding both in tension, exposing deliverables, tools and skills a few at a time rather than all at once, and keeping a versioned project history so the record survives the run. That last promise runs against a documented failure mode, since stacking procedural skills onto an agent can hurt it as often as it helps, a pattern recent work on the regression tax has started to quantify.

52.25 percent of what?

The gains were measured on GameCraft-Bench, FrontierSWE and ProgramBench against the same harness running standalone. Three iterations produced the average. One of the pairings produced the maximum. That is where the abstract stops.

It never names the metric that improved. A 52.25 percent relative gain could be a pass rate, a count of finished tasks, or a composite the authors defined for the occasion, and a reader cannot tell which from the text. Nor is the result split by harness, model or benchmark, which leaves the average as the only number on offer. "Relative" is the load-bearing word: a gain measured against a low baseline is a smaller absolute change than it sounds.

Claim in the abstract	What the text actually specifies
Average relative gain of 52.25 percent	Across three harness-model pairs, after three iterations
Maximum gain of 82.86 percent	Best of those same three pairings, after three iterations
Consistent improvement over standalone harnesses	Stated for all three pairs; no per-pair figures given
Multi-day autonomous development	One project, more than 70 iterations, a first-person shooter
Cost, token budget, wall-clock time	Not reported

Seventy iterations, one first-person shooter

The deployment is the part that will travel furthest. Across more than 70 iterations, HoH is said to have produced a first-person shooter with a coherent storyline, implemented core mechanics, a human-playable build, and "polished visuals and integrated audio."

A shooter tests things a command-line utility does not: real-time state, input handling, collision, an asset pipeline that survives an agent rewriting files it wrote an hour earlier. Whether HoH's build clears that bar is asserted, not measured. The abstract offers no frame rate, no playtest count and no comparison against a human-made project of similar scope. Multi-day is also not a number of days, and the text gives no compute figure or human-intervention count for the run.

Model versus harness

The three pairs are Codex with GPT-5.5, OpenCode with DeepSeek-V4-Pro, and Pi with MiniMax-M3. Three harnesses on three model families is the right shape for a paper about scaffolding. Had the loop only helped the harness it was built around, the whole idea would read like a rebranded prompt recipe. The harness side of that split has been moving fast on its own, and OpenCode's own bet on token efficiency shows how quickly a backend swap can change what a harness delivers.

The abstract says HoH beat the corresponding standalone harness every time, which is the strongest claim it could make without publishing the split. Effort is the thing it cannot rule out from the text. An iterative loop that runs longer, tries more variants and discards more failures will tend to score higher on any benchmark, and no token budget, wall-clock figure or cost accounting appears in the abstract. How much of the 52.25 percent comes from better scheduling and how much from spending more of everything is left open. That question has a track record of going unanswered: teams routinely report fewer mistakes without publishing what the extra coordination cost them, as an analysis of Qoder's multi-agent mode found, and the same scores can carry very different bills, as Cursor's SQLite rebuild showed.

What the paper does not show

This is a preprint submitted on 1 September 2026, and every number belongs to its authors. No independent replication is mentioned. Naming three public benchmarks raises the usual contamination question, since suites that circulate widely can end up in training data, and the text does not address it. Resource accounting is missing for the longest-running demonstration, the one that reads as most impressive.

The result may well hold up. For now it is unverified, which is the ordinary condition of a fresh preprint and a reason to hold the 52.25 percent loosely until someone reproduces it. Verifying it will not be easy either, because enterprises running coding agents today often cannot say what those agents actually did, a visibility gap reporting on agent observability keeps running into.

HoH's more interesting argument sits below its headline number. If the loop around a coding agent is itself a place to find gains, then the competition between harnesses matters as much as the competition between models. That claim is what replication would settle, and there is not much of it to check yet.