The simple topology trick that beats prompt magic in LLM induction tasks

For months, a growing number of papers have warned that LLM self-refinement is brittle. Asking a model to critique and revise its own output within a single context can degrade performance rather than improve it, as the safety implications of uncontrolled chain-of-thought also suggest. A new preprint from a researcher at Peking University offers a structural fix: replace monolithic context windows with role-isolated stages connected only by a compressed symbolic state.

The method, called Hourglass reasoning, treats the frozen LLM as a meta-constructor that builds a symbolic encoder-decoder for each task. An Induction module compresses few-shot examples into a schema φ and a transient scaffold z. A Deduction module derives a transformation rule T and discards z. An Implementer then compiles (φ, T) into executable artifacts. If errors occur, a Refiner revises (φ, T), not the output, and regenerates from scratch. Only (φ, T) crosses stage boundaries.

Across three benchmarks, visual abstraction, hardware synthesis, and textual rule induction, the same domain-agnostic pipeline yields consistent gains. On ARC-AGI-2, Hourglass raises pass@5 accuracy by 14 points with GPT-5.5 (from 62.8% to 76.8%) and by nearly 10 points with Gemini 3.1 Pro (from 76.9% to 86.7%). On the ChipBench Verilog synthesis subtask, it nearly doubles GPT-5.5's accuracy from 31% to 58%. The pattern echoes findings in other domains where structured isolation beats raw scaling, as seen in DSpark's scheduling-based inference speedups.

Why naive self-refinement fails

Standard self-refinement operates within a single, dense context: raw examples, current code, error feedback, and repair instructions all coexist without partition. The model tends to anchor on low-level perceptual details rather than abstracting the latent rule. When execution feedback arrives, it patches the code directly, hardcoding if-else branches for example-specific coordinates, rather than revising an explicit rule.

The paper's ablation study on ARC-AGI-2 isolates the decisive factor. When the same structured intermediates are produced inside a monolithic context window (Structured Self-Refine), accuracy collapses below even the unconstrained Self-Refine baseline, from 76.9% to 59.9% on Gemini 3.1 Pro. Physical context isolation between stages is necessary.

Stripping all auxiliary prompts (Hourglass-Plain) or removing all formatting constraints on φ and T (Hourglass-Unstructured) leaves performance virtually unchanged. Removing φ and T from the Refiner's input (Code-Only) causes only a marginal drop. The gain persists so long as the role-isolated topology is preserved. This starkly contrasts with the conventional wisdom that prompt phrasing matters most, a finding that aligns with the growing difficulty of verifying agent behavior without structural guards.

The result that surprises

The method's most provocative result comes from BBEH-Linguini, a Linguistics Olympiad benchmark where prior work showed explicit rule verbalization can hurt performance. Self-Refine severely degrades accuracy on both models, from 67.4% to 27.8% on GPT-5.5, and from 68.1% to 33.3% on Gemini 3.1 Pro. Hourglass roughly recovers the Raw Prompt baseline on GPT-5.5 (63.1%) and surpasses it by nearly 12 points at pass@5 on Gemini 3.1 Pro (79.9%).

The author attributes the collapse of Self-Refine to two intertwined mechanisms: lossy compression, each iteration of verbalizing an induced rule compounds information loss, and context entanglement, where rule generation, execution, and feedback coexist in a single context, making the model prone to attention distraction and inconsistent rule application. Hourglass mitigates both by enforcing that the symbolic state (φ, T) is the only persistent channel of information.

The cost of isolation

Role isolation comes with a token premium. On GPT-5.5, a single Hourglass run averages 61,911 tokens across 3.7 API calls, compared to 19,594 tokens over 1.4 calls for the monolithic Self-Refine baseline, roughly a 3 times increase. On Gemini, the gap is narrower: 122,007 tokens (4.4 calls) vs. 61,426 tokens (1.7 calls), approximately a 2 times increase. The paper notes its main comparisons are not compute-matched, though the magnitude of the gains suggests the trade-off may be acceptable for high-stakes reasoning tasks. The token cost tension mirrors a broader pattern in AI infrastructure: the hidden overhead of resource hopping shows that efficiency often depends on architectural choices beyond the model itself.

What's next: meta-strategies

The current Hourglass implementation is ephemeral: the symbolic state is discarded once a task is solved and re-derived from scratch for the next. The paper speculates about a Meta-Hourglass that accumulates reusable meta-strategies across tasks, analogous to how a human meteorologist draws on the same mental routine, look for periodicities, segment into regimes, identify anomalies, when faced with a novel time series, regardless of domain.

These meta-strategies do not prescribe exact operations, nor do they guarantee correctness, yet they systematically elevate analysis above the arbitrary.

Whether this meta-reasoning can be learned remains open. But the paper's core finding, that the topology of information flow, not the language used to express it, drives inductive reasoning in frozen LLMs, suggests a path that is about structural design rather than larger models or fancier prompts. This mirrors the insight that engineering benchmarks can be cracked without brute-force scaling.