Qwen, Llama and Gemma split on when a model stops routing and starts answering

What the preprint actually did

The question is mechanical. As a language model answers, how much does it lean on information that routes the query, and how much on the knowledge that supports the answer? From Parameters to Answers, submitted to arXiv on 10 September 2026, goes at it from the inside: it intervenes on the hidden state at the end of the question, layer by layer, rather than reading the output and reverse-engineering a story from it. That layer-localized style is spreading. Penelope, for one, confines recurrent computation to a single decoder layer to cut inference cost.

The team compares country-continent questions with noun, adjective and code answers, and keeps several fitted measurements apart that are usually lumped together. Two request directions get names. A pair-conditioned request direction describes which country is queried in natural single-country questions. A global request direction describes first- versus second-country requests in paired questions. A separate set of selection candidates tests control among the contents already sitting in the hidden state.

The three models appear as families only. The material names Qwen, Llama and Gemma and nothing finer, no variant, size or checkpoint, so nothing here should be read as a claim about a specific release.

Qwen, Llama and Gemma do not switch on the same schedule

The paired three-model trajectories are not uniform, and that is the first result worth carrying out of the paper. Gemma shows a partially overlapping mid-layer routing-content profile, reported in the abstract without further breakdown. Separate work has already found Gemma's internal representations of materials-science mechanisms to be causal, which makes the family a reasonable place to look for structure of this kind. Llama has no sustained routing-effect window under the same gates.

This is a split in kind, not degree. Anyone who assumed the three families share a layer-wise schedule for routing and content is wrong about at least one of them, and the preprint never says which training or architectural difference produces the divergence.

Qwen arrives by a different route. Its evidence comes from a diagnostic reanalysis of frozen natural-question states, not the paired protocol, and it carries the paper's sharpest distinction. The family has also become a target for blunter interventions: an abliterated Qwen3.8-27B build drops refusals to zero while its general benchmarks barely move, one more case where what shifts inside a model and what shifts on the scoreboard are not the same thing.

The handoff: request information fades, content stays

In the paired protocol, dependence on the global request direction shrinks as the analysis moves from fixed earlier layer sets to later ones. Dependence on fitted content does not. Routing weakens with depth; the material behind the answer holds.

The Qwen reanalysis gives that shift a shape. Its pair-conditioned request direction grows stronger before interventions on it start to alter later fitted knowledge, and the causal window opens while answer-supporting content is still forming. Reading a direction at a layer and changing the model through it are two different properties, and here they come apart at different depths. The same disconnect shows up in monitoring: token log-probability fails as a decoder monitor precisely when a reasoning model goes wrong.

The boundary the authors draw around their own result

The matched Qwen comparison keeps the finding from overreaching. The pair-conditioned direction retains a late effect, so the operational handoff concerns the global fitted direction specifically, not request information in general. Steering a model through a country cue and steering it through a paired-question cue behave differently at the same depths.

None of it is quantified. The abstract reports no effect sizes, no layer counts, and no direct comparison of the late pair-conditioned effect against the earlier one. That silence leaves a real question open, and the paper does not answer it: is the handoff a strong signal, or a direction that quietly stops mattering?

Why four measurements beat one

The paper's methodological claim is that four properties usually reported together should be held apart: early readability, natural strength, causal steering, and later content dependence. A direction can be legible in a hidden state without being the thing that moves the output, and it can steer an output at one set of layers while something else carries the answer at another. Work on distributional RL heads makes the point from another angle: the risk trade-offs a network appears to encode at a given state can be exactly the ones it never acts on.

That separation is what makes the Llama and Gemma results comparable to the Qwen one at all, and it is also where the preprint's limits sit. With the protocol available only in abstract form, its most portable claim is a negative one: a direction that reads clearly at a layer is not automatically the direction that decides the answer there.