Catastrophic remembering: why CLAUDE.md files grow 226% and never shrink

Every developer who has worked with a coding agent knows the feeling. The project's instruction file, CLAUDE.md for Claude Code setups or one of its equivalents, starts as a tidy list of rules. Months later it reads like an accumulation of decisions nobody can fully explain, and nobody dares prune it, because removing the wrong line might quietly break the agent's behavior. It is the same opacity that leaves enterprises unable to say what their agents actually did.

That is not a human habit problem. It is structural, and a paper posted to arXiv on August 11, 2026 gives it a name. Written by Kushal Chakrabarti, "Why Does CLAUDE.md Keep Growing?" calls the pattern catastrophic remembering: the mirror image of the catastrophic forgetting that continual learning research has circled for years.

The numbers behind the bloat

The paper tracks 247,694 instruction lifetimes across 1,867 repositories. The pattern holds everywhere: agentic prompts grow without bound, more than tripling over their lifetime, a +226% increase. Every commit adds a net 4.9 instructions. And the older an instruction gets, the less likely it is to go: the hazard of deletion drops 0.032 per commit on a log scale. The file stops growing only when the repository retires or someone rewrites it from scratch.

The mechanics explain the data. Appending an instruction costs nothing: one more line, done. Deleting one is a different game. Once the reasoning behind an instruction is lost, removing it without risking a correctness regression costs O(2^|D|) in a prompt of |D| instructions. Nobody pays that bill. The file only ever grows, and because every call re-reads the file, bloat is an input-token problem, the same one Alibaba claims Qoder 1.0 cut by 40%.

Why deletion is the expensive move

The bill is hard to price, and not because of the prompt itself. Related work from Qwen's research team, "The Verification Horizon," argues that the bottleneck for today's coding agents has flipped: generating candidate solutions is no longer the hard part, reliably verifying them is. Every verifier a team can build is only a proxy for human intent, never the intent itself. If you cannot cheaply prove that deleting an instruction changes nothing, keeping it looks rational even when it is dead weight. Operators who have pushed agents into production describe the same pattern, in which the hardest failures are not in reasoning but in subtle, unglamorous details.

That is where the paper's fix lands. The authors invert IFEval, a standard instruction-following benchmark, to build "verifiable worlds" where the optimal prompt is known in advance.

The same logic powered Cursor's swarm rebuilding SQLite from scratch and passing its full verification suite: with a trustworthy verifier, radical edits stop being dangerous.

In those controlled settings, comments that encode the latent reasoning behind instructions remove 99.3% of excess instructions, cutting growth from +211.3% to +1.4%. Applying the same inversion to WildIFEval, a noisier real-world benchmark, prompt comments improve instruction-following by up to 23.1%.

Comments, but for prompts

The paper ends with a question that deserves to travel: "If English is the new code, why don't we have comments yet?"

The point is not documentation for humans. Comments that record why an instruction exists give the next editor, human or model, the information needed to delete it safely. The rationale is what makes deletion cheap again, because the cost problem only appears once that rationale is gone. A prompt with reasons attached is a prompt that can shrink. Without them, every instruction becomes a permanent commitment nobody can audit. The practical read is almost boring: treat the instruction file as code, review additions the way you would review any commit, and demand that every rule carry its why. The instinct to review additions has evidence behind it: loading LLM agents with procedural skills can backfire.

The paper is a preprint, and its controlled experiments do not prove comments will tame every production CLAUDE.md. But the underlying finding is hard to argue with: a file that only accumulates, where nothing can be safely removed, is a file on a one-way road to bloat. Naming the failure is the first step, and the name will stick, because anyone who has watched an instruction file triple in size knows exactly what catastrophic remembering feels like.