SafeEvolve cuts agent attack success 3x by evolving harness and model together

Most agent-safety work picks a side. You harden the scaffolding the agent runs inside, or you fine-tune the model until its behavior improves. A paper submitted to arXiv on 2 September argues that the split is the flaw: runtime control and intrinsic safety never reinforce each other, so progress on one side gets quietly spent on the other.

Titled "SafeEvolve: Harness-Policy Co-Evolution from Agent Experience for Safety Alignment," the paper sits in arXiv's Artificial Intelligence section. Its premise is structural. An agent's behavior is shaped jointly by the base model and by the harness that mediates its interaction with the environment, so risk has two places to hide: a harmful final response, or a multi-step execution trajectory that never produces one obviously wrong sentence.

The harness is now part of the threat model

Most alignment pipelines address one of those surfaces at a time. External harness updates patch the runtime. Policy optimization changes the model. Applied in isolation, the paper says, neither one connects what the runtime can enforce to what the model has actually learned, and agentic failures accumulate in that gap. Runtime patches tend to catch the destructive command rather than the run that produced it, the same gap that pushes teams toward sandboxes instead of allow-prompts.

SafeEvolve runs a continual loop instead. It draws safety experience from completed on-policy trajectories, meaning the agent's own finished runs, and uses that evidence to update both sides at once.

How the co-evolution loop works

On the harness side, the system turns trajectory-level safety evidence into bounded, component-level updates. Those updates land in the safety prompt and in a set of hierarchical skills. The paper calls the resulting harness artifacts auditable and reversible. Anyone who has had to explain a deployed agent's behavior after the fact knows why that matters. Skills are also where self-improving agents tend to overcorrect, accumulating fixes that misfire elsewhere, as SkillProx's pruning work found.

On the policy side, SafeEvolve uses a two-stage SFT-RL pipeline. Harness-use supervised fine-tuning bootstraps the policy to actively use whatever harness artifacts have evolved so far. Then reinforcement learning with verifier-decomposed rewards shapes autonomous safety behavior during multi-step exploration, so the agent is not leaning on the harness alone.

What the AgentDojo numbers show, and what they leave out

The authors report experiments on agentic safety benchmarks and claim a stronger safety-utility tradeoff than existing baselines. The one concrete result in the abstract concerns Qwen3.5-4B, a small model from the same line that produces agents like Alibaba's Qwen-UI-Agent.

That table is lopsided in a way the abstract does not resolve. The paper publishes a ratio for attack success, a threefold reduction, but not the raw rates it was computed from. The utility figures appear absolutely, and the movement is 2.07 percentage points.

Why "reversible" carries more weight than the ratio

A safety mechanism you can inspect at the component level and roll back is a different object from a model retrain. Teams running agents in regulated settings can log what changed and when, then reverse a change that breaks legitimate work. Nothing in the abstract suggests SafeEvolve has been tested against that kind of governance pressure, but the artifact design points at the constraint operators actually hit. Being able to say what an agent did is already a known weak spot in enterprise deployments, per reporting on agent visibility gaps.

The scope here is modest. One mechanism, benchmark results on one small model, and a question the abstract does not answer: whether a loop that learns from an agent's own trajectories keeps improving with volume, or eventually learns from its own mistakes. The results come from agentic safety benchmarks, not from production traffic. Verification-first designs like BAAI's AREX agent exist precisely because raw iteration is not enough on its own.