Over half your AI's reasoning is froth, and nobody noticed until now

Chain-of-thought (CoT) prompting has become the standard way to get multi-step reasoning out of large language models. Break a problem into intermediate steps, and the model's accuracy climbs. But a research paper from KT Corporation and Pohang University of Science and Technology (POSTECH) argues that the field has been optimizing for the wrong signal. The same pattern of chasing the wrong metric shows up elsewhere, verifying coding agents has become harder than building them.

The paper, Valid ≠ Necessary: Diagnosing Latent Inefficiency in Chain-of-Thought, identifies a blind spot in current reasoning evaluation: evaluators that check for correctness and logical validity often give high scores to steps that are factually true but functionally useless. The authors call this informational froth, and their work suggests it makes up a significant portion of standard CoT traces.

The researchers introduce RIV-GSM8K, a diagnostic benchmark derived from the GSM8K math dataset. RIV-GSM8K injects five distinct types of inefficiency into reasoning chains: Simple Duplication (repeating a step), Paraphrase (rewording the same idea), Decompose (splitting a single step into many micro-steps), Circular Reasoning (restating without progress), and Irrelevant (tangents that are mathematically valid but unrelated to the solution).

When the paper tested state-of-the-art evaluators including ReasonEval, ThinkPRM, and Qwen2.5-Math-PRM against RIV-GSM8K, the results were stark. Existing models retained 70 to 97 percent of injected inefficient steps, meaning they failed to flag steps that were clearly redundant or circular. ReasonEval, despite having an explicit redundancy score, preserved about 70 percent of simple duplications. Even the 72-billion-parameter Qwen2.5-Math-PRM kept 83 percent of them.

To close this gap, the paper proposes CAID (Context-Aware Information Density), a training-free metric that combines four signals: local similarity (redundancy), global goal alignment (relevance), information density (verbosity), and semantic delta (logical progress). CAID uses only 146 million parameters in total, a GPT-2 Small model for density estimation and a MiniLM encoder for embeddings, making it orders of magnitude smaller than the supervised evaluators it outperforms. The contrast between tiny, efficient metrics and bloated evaluators mirrors broader efficiency trends, Leanstral 1.5 has proved the old rules of AI pricing don't apply to math.

On RIV-GSM8K, CAID achieved near-perfect detection of duplication (0.0 percent preservation) and paraphrase (1.7 percent preservation). It also caught circular reasoning (1.9 percent preservation) and decomposition (20 percent preservation), areas where baselines struggled. More provocatively, CAID flagged roughly half of the original human-written GSM8K steps as compressible, not as removable, but as expressible more compactly. Only 1.5 percent of those flagged steps were marked for outright deletion; the remaining 98.5 percent were assigned to a merge action, indicating that the reasoning itself was sound but the language was inefficient.

The researchers validated CAID's diagnostic power by building a compression pipeline called PACE (Pruning And Compression for Efficiency). PACE applies the CAID signals to compress reasoning chains after generation. Across GSM8K, StrategyQA, and ARC-Challenge, PACE reduced token consumption by 31 to 53 percent while maintaining or improving accuracy. On ARC-Challenge, accuracy actually rose by 0.94 points alongside a 43.6 percent token reduction. This kind of pruning-after-the-fact approach resembles the systems-level thinking behind DSpark's demonstration that faster LLM inference is a scheduling problem, not a model trick.

Control experiments separated PACE's effect from trivial pruning. Randomly removing 50 percent of steps from GSM8K chains caused accuracy to collapse from 82 percent to 66.7 percent. Even removing only the last step hurt performance. By contrast, PACE's selective compression preserved accuracy at 81.12 percent while cutting 31 percent of tokens. When the researchers replaced CAID with strong PRM evaluators inside the same PACE pipeline, those validity-focused models could maintain accuracy but achieved only 6 to 7 percent token compression, far below CAID's 31 percent.

One of the paper's more surprising findings is that the notion of gold steps, human-written reasoning steps assumed to be optimal, may itself be flawed. CAID's merge assignments suggest that standard datasets contain latent inefficiency: verbose restatements, over-explained intermediate calculations, and fragmented logic that could be expressed more concisely. The researchers argue that this compressible content does not contradict the reasoning's validity, but it does raise questions about training data quality and the efficiency assumptions baked into current evaluation practices. The gap between human-written data and what is actually optimal parallels issues flagged in the quiet finding on chain-of-thought monitoring that worries frontier labs.

The authors acknowledge limitations. PACE operates in a generate-then-refine pipeline, which means it does not reduce the latency of the initial inference pass. It is better suited to offline dataset construction or context compression than real-time acceleration. The merge step relies on an LLM rewriter with safety constraints; smaller models may oversimplify complex reasoning. And the paper's domain focus on arithmetic, commonsense, and scientific reasoning means the notion of efficiency may differ in open-ended tasks like creative writing, where verbosity serves a purpose.

Still, the work makes a strong case that reasoning evaluation has been fighting the wrong war. Models can produce logically valid chains that are padded with factually correct but unnecessary steps. Current evaluators, optimized for correctness, reward that padding. CAID and RIV-GSM8K offer a path toward metrics that measure not just whether a step is true, but whether it is needed.

The paper was led by Daeyeop Lee of KT Corporation and Hwanjo Yu of POSTECH, and was supported by an IITP grant from the Korean Ministry of Science and ICT.