The noisy trick that stops LLM agents from collapsing in production

The gap between a lab demo and a production agent comes down to one thing: the world is messy. Benchmarks test clean instructions and perfect tool responses. Real users type vaguely. APIs return errors. Tools drop connections mid-call. The agent trained on one kind of reality flinches at the other. Your AI agent passed by accident. SkillCoach grades the…

A preprint from the Meituan LongCat group, Learning to Act under Noise: Enhancing Agent Robustness via Noisy Environments, takes on that mismatch. The paper, posted on Hugging Face in late May 2025, introduces NoisyAgent, a framework that contaminates training environments with two specific noise sources, user noise and tool noise, and gradually intensifies both as the agent learns to cope.

The two faces of interaction noise

The authors identify a core problem: the training environment is a curated bubble. Task instructions are spelled out without ambiguity. Tools always return expected data. The agent learns to read clean signals and execute perfect chains. Give it a real user who types "fix the table" without context or a weather API that times out, and the chain breaks. Ifbench reveals the instruction-following gap that…

They define two noise categories:

• User noise captures ambiguity, typos, missing parameters, and variation in phrasing. The paper simulates this by injecting perturbations into user interaction patterns during training.
• Tool noise models execution failures: timeouts, malformed responses, unexpected error codes, inconsistent output schemas. The team simulates tool anomalies in the environment so the agent sees failures before deployment.

The insight is not just that noise helps. How noise is administered determines whether the agent learns resilience or confusion. The verification horizon: why verifying coding agents…

Progressive hardening, not drowning

NoisyAgent uses a two-part stabilisation mechanism. First, noise is applied to only a subset of training rollouts, not all of them. The agent still sees enough clean trajectories to anchor its learning. Second, noise difficulty escalates incrementally: the agent starts with small perturbations and, as it adapts, faces harder ones. The authors call it a curriculum-like schedule for environmental uncertainty.

In experiments detailed in the paper, agents trained under NoisyAgent outperformed standard agents on both noisy and clean benchmarks. That last part matters: exposure to imperfections does not trade off against ideal-condition performance. It appears to teach agents to reason more robustly in general, not just to cope with noise. OPID feeds agents dense rewards from their own past, no…

Why this paper matters now

The LLM agent field has seen an avalanche of benchmarks, AgentQ, AgentBench, WebArena, SWE-Bench, each testing more complex multi-step tool use. But practitioners keep complaining that agents acing these benchmarks collapse under production load. The paper reframes the problem: the benchmark environments are too clean, and the training loop has overfit to that cleanliness. AI agents can't tell when a Java migration is actually done

NoisyAgent is not the first attempt at robustification. Adversarial training and domain randomisation are established techniques in reinforcement learning and computer vision. But applying them to language agents with explicit user and tool noise taxonomies is novel. The paper positions itself alongside concurrent work from the same research community, including EnvFactory (synthesizing executable environments) and CoEvolve (agent-data co-evolution), suggesting a deeper movement toward training agents in adversarially realistic conditions.

Open questions and caveats

The preprint does not release model weights or training code at this stage. Experiments are conducted on simulated environments, not live production systems. The realism of the noise model, how well synthetic user ambiguity matches true human fuzziness, remains an open question. The paper also does not address the compute cost of training with progressive noise, which may be nontrivial when each rollout must simulate tool failures or user variability.

Still, the core argument is hard to dismiss. The agent community has been chasing benchmarks because benchmarks are clear. This paper suggests clearing the benchmark is not the terminal goal. Surviving deployment is. NoisyAgent offers a concrete way to train for that survival. AI agents are rewriting the rules of cybersecurity

The full paper is available on Hugging Face under the ID 2605.27209.