{"id":"90308081-cd59-478a-b770-78c05abe81a9","arxiv_id":"2608.05660","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"Combining residual-stream motion with a coarse region code and a fine direction readout improves zero-shot selection of correct LLM answers across reasoning and factual benchmarks.","lead":"This paper adds two compact location readers, a quantized region code and normalized layer directions, to a trajectory-based detector that watches how an LLM's internal state changes. On benchmarks the detector never trained on, it selects correct answers more often than prior displacement-only or single-layer probing methods, and it also works on factual tasks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"A matched static-state baseline is missing: the paper's own direction-only reader trails the full motion+state detector by only ~2 points on reasoning and beats it on FACTOR-expert, so 'state-conditioned motion beats static states' is not yet established.","rationale":"I read the paper as trying to show that restoring restricted state context around displacement is what makes residual-stream readouts transfer. The strongest evidence is the motion-plus-location ablations, the three-seed stability, and the cross-model replication on Qwen dense and MoE models. The most load-bearing vulnerability is not the exact depth window, although Appendix O's +3.0 point middle-third result is a related robustness worry; it is the absence of a matched static-state baseline. The paper's own Appendix J contains a direction-only reader, and that reader nearly matches the fused model on reasoning and beats it on FACTOR-expert. If that result survives a standalone training run, the central dichotomy collapses: normalized multi-layer static states may be sufficient, and the motion stream may add little or nothing. The window concern is secondary because Table 19 shows several windows work and the fixed-window choice is disclosed, whereas the static-baseline gap directly undermines the headline comparison. I agree with the reader's CONDITIONAL verdict, and the missing direction-only control should be a required condition rather than only a re-windowed ablation.","tokens_in":26174,"tokens_out":6471,"duration_ms":73064,"concrete_test":"Train the direction reader alone, with no motion stream and no region code, under the exact Appendix C recipe, same projection, MLP, head, seeds, and evaluation grids as the full detector, and compare on all eight reasoning targets and all four factual targets. If direction-only is within about 2 points of the full model on reasoning and matches it on FACTOR-expert, the state-conditioned-motion interpretation fails; if the full model wins by more than 5 points on the reasoning mean and more than 5 on the factual mean across three seeds, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that state-conditioned motion (motion+region+direction) beats both decontextualized motion and static states. Against motion the comparison is fair: Table 3 and Table 11 show that adding the location streams to the same motion LSTM helps. Against static states, the only main-text baseline is a single-layer linear probe (Table 1), yet the paper's own direction reader is a much stronger static baseline: it reads normalized states at six layers and the last 64 answer tokens through a shared projection and an MLP. Appendix J's decomposition reports direction-only scores of 73.2 vs 75.1 on ARC-Challenge, 94.8 vs 96.1 on Story Cloze, 53.1 vs 65.0 on VitaminC, and 69.5 vs 57.2 on FACTOR-expert. Thus on two reasoning sets the static direction reader trails the full detector by only 1.3-1.9 points, and on FACTOR-expert it beats it by 12.3 points; the main model's FACTOR-expert mean (69.5, Table 2) matches the direction-only score. Because a direction-only detector trained standalone was never reported, the headline 10-21 point gain over 'static states' may come from replacing a weak single-layer probe with a stronger multi-layer normalized static reader rather than from conditioning motion on state. The late-fusion numbers are single-seed and FACTOR-expert has only 236 items, so the concern is falsifiable rather than established, but it targets the least secure support for the central claim. The fixed layer window in Section 4.2 is a secondary worry: Appendix O shows the middle third reads 3.0 points higher than the chosen window, but the window sweep is at least reported transparently.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a three-stream validity detector for LLM candidate-answer selection. A motion reader consumes layerwise residual-stream displacements (prior work); a region reader adds a coarse vector-quantized code of the answer's final-token state; a direction reader adds fine normalized multi-layer state directions over the answer tokens. All streams are fused into a small MLP head and trained on a single reasoning benchmark, then evaluated zero-shot on held-out reasoning and factual benchmarks. The authors claim that this state-conditioned motion reading outperforms both decontextualized displacement-only methods and static single-layer probes, and that the signal transfers to factual errors, placing it on correctness rather than on a kind of reasoning.","tokens_in":26356,"tokens_out":6295,"duration_ms":64473,"significance":"If the central claim were fully supported, the method would be a practical, low-overhead detector of reasoning validity with transferable signal, and the articulated state-restoration trade-off is a useful conceptual contribution. The paper is transparent and unusually thorough: the main reasoning table uses three-seed means, the layer-window sweep and codebook-health diagnostics are reported, wording-leakage measurements are given, and cross-model checks on two additional model families are included. However, the central comparison against static states is not established, because the only static-state baseline in the main text is a single-layer linear probe; the paper's own direction reader, a much stronger static-state reader, nearly matches the full model on reasoning and beats it on one factual target. This gap is load-bearing for the abstract's central claim.","major_comments":[{"comment":"The abstract and Section 6.1 claim that state-conditioned motion beats static states by 10 to 21 points, but the only static-state baseline in Table 1 is a single-layer linear probe. Appendix J's late-fusion decomposition (Table 10) shows that the direction reader alone—a multi-layer, normalized, answer-span static reader—achieves 73.2 vs 75.1 on ARC-Challenge, 94.8 vs 96.1 on Story Cloze, and 69.5 vs 57.2 on FACTOR-expert against the full three-stream model. The evidence therefore supports 'motion plus state beats motion alone' and 'motion plus state beats a single-layer probe,' but not the general claim that state-conditioned motion beats static states. Moreover, the direction-only column in Table 10 comes from a jointly trained late-fusion variant, not from a standalone detector. To support the headline claim, the authors should train a direction-only detector with the same architecture and training recipe (motion stream removed) and compare it against the full model on all reported reasoning and factual targets.","section":"Section 6.1, Table 1; Appendix J, Table 10"},{"comment":"The factual-transfer result, which the abstract uses to argue that the signal is about correctness rather than a kind of reasoning, rests on single-seed runs and includes FACTOR-expert with only 236 items. Appendix I acknowledges that the per-donor cells for FACTOR-expert are noisy. Separately, Appendix J Table 10 shows that the direction-only reader outperforms the fused model on FACTOR-expert by 12.3 points (69.5 vs 57.2), so the factual claim is also sensitive to the missing static-state baseline. The authors should report multi-seed means and standard deviations for Table 2 and for the per-donor cells, or explicitly mark the factual-transfer claim as preliminary.","section":"Section 6.2, Table 2; Appendix I"}],"minor_comments":[{"comment":"The column header 'Target Motion+Direction+Region Ours' is misaligned with the four numeric columns; the columns should be labeled 'Motion', '+Direction', '+Region', and 'Ours' to match the additive ablation described in the text.","section":"Table 3"},{"comment":"The paper states that the chosen window was fixed in advance and that the middle third reads 3.0 points higher on the reasoning mean, but the sentence 'The gain rests on reading the middle of the network, and not on a window selected to produce it' does not address the fact that the reported main results use a suboptimal window. Given that the middle-third arm is better on 11 of 12 targets, the authors should report the main-table results under the middle-third window as a robustness check, or explicitly state that the headline numbers are conservative relative to the depth sweep.","section":"Appendix O, Table 19"},{"comment":"The citation 'van den Oord, Vinyals, and kavukcuoglu 2017' contains a lowercase author name; it should be 'Kavukcuoglu' in both the citation and the reference list.","section":"Equation (4) and references"},{"comment":"The important caveat that the FACTOR-expert in-domain ceiling is based on a held-out split of about 47 items and is optimistic appears only in the appendix; a one-sentence version of this caveat should accompany the FACTOR-expert row in Table 2.","section":"Appendix D and Section 6.2"},{"comment":"The statement that the full model is best on 47 of 48 cells refers to Table 1, but it is not explicit whether the in-distribution cells are counted; the authors should state this explicitly to avoid ambiguity.","section":"Section 6.1"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is well organized and unusually transparent about hyperparameter sweeps and potential confounds. The core issue is a missing matched static-state baseline, which is fixable by training a standalone direction-only detector. The depth-window sweep is handled honestly, but its implications for the headline numbers should be stated more directly."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Plainly: the main reasoning-transfer result is real and worth taking seriously. The three-stream detector beats motion-only and single-layer linear probe on almost all of 48 donor-target cells, with three-seed stability and confirmation on two other model families. But the abstract's claim to beat \"static states\" is oversold. The main tables compare only against a single-layer linear probe, and the paper's own Appendix J shows a direction-only reader—which is itself a stronger static reader—trails the full detector by only about 1.3 to 1.9 points on two reasoning sets and beats it by 12.3 points on FACTOR-expert. So \"state-conditioned motion beats static states\" is not established; the gain may largely come from replacing a weak static baseline with a stronger normalized multi-layer one. That should be fixable: train a direction-only detector standalone, put it in the main table, and reframe the claim.\n\nWhat is genuinely new and good: the region and direction readers, the demonstrated zero-shot transfer to unseen reasoning and factual benchmarks, the per-cell transfer maps, the honest limitation notes, the exclusion of TriviaQA and HaluEval with a documented surface-level heuristic, and the transparent if not fully satisfying window sweep in Appendix O. The window itself is a real fragility—the middle third scores 3 points higher than the chosen fixed window—but the paper reports it rather than hiding it.\n\nSoft spots in proportion: the main transfer table is solid, but the factual-transfer results are single-seed, FACTOR-expert has only 236 items with an in-domain ceiling estimated on about 47, and the likelihood baseline beats the detector on three of four factual sets—something the abstract omits. No code or checkpoints are released, and several ablations are single-seed with no confidence intervals. None of this kills the primary result.\n\nThis paper is for interpretability and reliability researchers. It deserves a serious referee, not desk rejection, but the authors should be pushed to add the missing static baseline, report seeds and intervals on secondary claims, and open the code. The central reasoning-transfer result likely survives those revisions; the conceptual claim will need to be reworded.","headline":"Solid transfer results and a clever state-restoration design, but the headline claim against static states rests on a weak comparison; it should be revised rather than rejected.","tokens_in":27087,"tokens_out":2352,"would_cite":true,"duration_ms":25471,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A detector that reads where residual-stream states sit as well as how they move selects correct answers on unseen reasoning benchmarks up to 12 points better than displacement-only methods, and carries the same signal to unseen factual…","keywords":["residual stream","LLM interpretability","trajectory analysis","linear probing","vector quantization","zero-shot transfer","reasoning validity","state-conditioned motion"],"falsifier":"Construct a held-out benchmark whose candidates are surface-matched (same token length, topic, and wording template) and whose correct answer is determined only by reasoning stated in the question; if the answer-token-only, fixed-window detector scores at chance while a probe on question-token states succeeds, the claim that the restored answer-state location is the correctness signal fails.","tokens_in":25790,"feed_emoji":"🧠","tokens_out":6981,"duration_ms":68228,"temperature":0.7,"pith_summary":"Trajectory-based detectors read the residual stream through layerwise displacement, which suppresses stable token-surface information but discards the state from which each update begins. This paper argues that restoring a restricted view of that state, a coarse region code and a fine per-layer direction, is enough to interpret the motion without reintroducing the shortcut-prone signals that full-state probing picks up. The claim is that reasoning validity is read better from state-conditioned motion than from static states or decontextualized trajectories alone. The three-stream detector, trained on one reasoning benchmark, selects the correct answer on unseen reasoning benchmarks up to 12 points better than the displacement-only state of the art and up to 21 points better than a single-layer linear probe, and also reads unseen factual completions and fact-verification claims.","feed_headline":"State plus motion beats motion alone in spotting bad reasoning","feed_subtitle":"Adding a coarse region code and fine direction to layerwise displacement reads correctness on unseen reasoning and factual benchmarks.","key_machinery":"The load-bearing object is the three-stream validity detector. A motion reader takes the displacement $d_{t,\\ell}=h_{t,\\ell+1}-h_{t,\\ell}$ across all tokens and layers and passes it through a bidirectional LSTM. A region reader projects normalized states from a fixed window of six middle-to-late layers, then replaces the projected state at the answer's final token with the nearest of $K=128$ learned codebook entries, so the decision head receives a coarse location label instead of a continuous vector. A direction reader keeps the full normalized multi-layer projection $u_{t,\\ell}=h_{t,\\ell}/\\|h_{t,\\ell}\\|$ for answer tokens, combining the final token's projection with the mean over answer tokens through a shared MLP. The three embeddings are concatenated and scored by one MLP head; the design work is that the two location streams restore context that differencing removes at coarser and finer resolutions, measured at only 6.7% of the motion reader's parameter count.","core_discovery":"Reasoning errors leave a trace not only in how residual-stream states move from layer to layer but in where those states sit when they move. The paper's central discovery is that a detector can exploit both by reading three complementary streams: the layerwise displacement, a vector-quantized coarse region label, and normalized per-layer directions. Together these streams restore enough state context to interpret cross-layer updates while staying narrow enough to suppress stable, label-correlated surface information. Trained on a single reasoning benchmark, the full model is the best of the three trained readers on 47 of 48 reasoning-transfer cells and beats every trained detector on unseen factual benchmarks, which the authors take as evidence that the signal is about correctness rather than a particular kind of reasoning.","pith_inferences":["If the region code is truly a shared coordinate of correctness, then a detector trained on reasoning should also transfer to other correctness-labeled domains such as code execution or translation adequacy; that is a testable prediction the paper does not run.","The fixed six-layer window is load-bearing: Appendix O shows the middle third of layers scores about 3 points higher on reasoning than the chosen window, so a learned layer-selection mechanism could raise the ceiling the paper reports.","Because correctness appears to be carried by answer-state location and motion, interventions that nudge a state toward the correct-answer region or that alter displacement directions might steer generation, not just detect errors.","The paper reads only the answer tokens; if question-token states encode the reasoning state, combining them with the current streams could close part of the gap between the trained fusion and the per-item oracle."],"forward_implications":["Training on any of the six reasoning donors yields a detector that reads the other reasoning benchmarks it never saw more accurately than the motion-only baseline and the linear probe, with the largest 12-point margin over prior work on the ARC-Challenge donor.","Because the same reasoning-trained detector outscores every trained baseline on FACTOR and VitaminC without ever seeing factual training data, the paper positions the restored signal as correctness-specific rather than reasoning-task-specific.","The two added streams add only 0.31M parameters (6.7% of the motion LSTM), so the gain is attributed to the design, reading location alongside motion, rather than to added capacity.","Component ablations show motion, region, and direction each recover items the other two miss, and a per-item oracle stands 3 to 21 points above the trained fusion, implying a learned per-item gate could push the method further."],"supporting_citations":[{"why":"Supplies the displacement-based motion reader and the 'Truth as a Trajectory' baseline that the three-stream model extends by 7 to 12 points.","marker":"Damirchi et al. 2026"},{"why":"Documents the cross-task orthogonality of truth directions, used to motivate why single-layer linear probes fail to transfer and as the probe baseline being compared.","marker":"Azizian et al. 2025"},{"why":"Provides ARC-Challenge and ARC-Easy, the main donor and evaluation benchmarks where the state-restoration trade-off is measured.","marker":"Clark et al. 2018"},{"why":"Supplies the vector-quantization formalism and straight-through estimator used by the region reader.","marker":"van den Oord, Vinyals, and kavukcuoglu 2017"},{"why":"Provides FACTOR-wiki/news/expert, the minimal-edit factual transfer benchmarks that test whether the detector reads correctness beyond reasoning.","marker":"Muhlgay et al. 2024"},{"why":"Provides VitaminC, the claim-verification benchmark used to test factual transfer against evidence.","marker":"Schuster, Fisch, and Barzilay 2021"}],"fun_headline_variants":["Motion alone misses reasoning errors; add state location to find them","Three-stream detector reads reasoning errors from motion and state","State-conditioned motion outperforms motion alone for detecting reasoning flaws","Spot bad reasoning by tracking where states are, not just how they move","Reasoning errors show up in state plus motion, not just motion"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that the information needed to interpret motion for correctness is present in the last 64 answer tokens and in a fixed window of six middle-to-late layers; if the signal lives mainly in the question tokens, early layers, or output-adjacent layers, the restored state is partial and the claimed transfer gains could shrink.","fun_headline_variants_meta":{"raw":{"variants":["Motion alone misses reasoning errors; add state location to find them","Three-stream detector reads reasoning errors from motion and state","State-conditioned motion outperforms motion alone for detecting reasoning flaws","Spot bad reasoning by tracking where states are, not just how they move","Reasoning errors show up in state plus motion, not just motion"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00019,"raw_usage":{"total_tokens":1328,"prompt_tokens":924,"completion_tokens":404,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":540,"completion_tokens_details":{"reasoning_tokens":317}},"tokens_in":540,"tokens_out":404,"duration_ms":4182,"temperature":1.0,"reasoning_tokens":317,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T04:36:38.715618+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Construct a held-out benchmark whose candidates are surface-matched (same token length, topic, and wording template) and whose correct answer is determined only by reasoning stated in the question; if the answer-token-only, fixed-window detector scores at chance while a probe on question-token states succeeds, the claim that the restored answer-state location is the correctness signal fails.","supporting_citations":[{"cited_title":"Truth as a Trajectory: What Internal Representations Reveal About Large Language Model Reasoning","cited_arxiv_id":null,"evidence_quote":"Supplies the displacement-based motion reader and the 'Truth as a Trajectory' baseline that the three-stream model extends by 7 to 12 points."}],"review_version":1}