Pith. sign in

REVIEW 2 major objections 5 minor 79 references

Reasoning Errors Have a Region and a Direction in the Residual-Stream Trajectory of LLMs

T0 review · 2 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A detector that reads where residual-stream states sit as well as how they move selects correct answers on unseen reasoning benchmarks up to 12 points better than displacement-only methods, and carries the same signal to unseen factual…

desk verdict Solid transfer results and a clever state-restoration design, but the headline claim against static states rests on a weak comparison; it should be revised rather than rejected. read the letter →

arxiv 2608.05660 v1 pith:UEVUA7TO submitted 2026-08-06 cs.LG cs.CL

Hamed Damirchi , Ignacio Meza De la Jara , Damith Ranasinghe , Yuhang Liu , Javen Shi This is my paper · ORCID
classification cs.LGcs.CL
keywords residualstreamLLMinterpretabilitytrajectoryanalysislinearprobingvectorquantizationzero-shottransferreasoningvaliditystate-conditionedmotion
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Trajectory-based detectors read the residual stream through layerwise displacement, which suppresses stable token-surface information but discards the state from which each update begins. This paper argues that restoring a restricted view of that state, a coarse region code and a fine per-layer direction, is enough to interpret the motion without reintroducing the shortcut-prone signals that full-state probing picks up. The claim is that reasoning validity is read better from state-conditioned motion than from static states or decontextualized trajectories alone. The three-stream detector, trained on one reasoning benchmark, selects the correct answer on unseen reasoning benchmarks up to 12 points better than the displacement-only state of the art and up to 21 points better than a single-layer linear probe, and also reads unseen factual completions and fact-verification claims.

What carries the argument

The load-bearing object is the three-stream validity detector. A motion reader takes the displacement $d_{t,\ell}=h_{t,\ell+1}-h_{t,\ell}$ across all tokens and layers and passes it through a bidirectional LSTM. A region reader projects normalized states from a fixed window of six middle-to-late layers, then replaces the projected state at the answer's final token with the nearest of $K=128$ learned codebook entries, so the decision head receives a coarse location label instead of a continuous vector. A direction reader keeps the full normalized multi-layer projection $u_{t,\ell}=h_{t,\ell}/\|h_{t,\ell}\|$ for answer tokens, combining the final token's projection with the mean over answer tokens through a shared MLP. The three embeddings are concatenated and scored by one MLP head; the design work is that the two location streams restore context that differencing removes at coarser and finer resolutions, measured at only 6.7% of the motion reader's parameter count.

What would settle it

Construct a held-out benchmark whose candidates are surface-matched (same token length, topic, and wording template) and whose correct answer is determined only by reasoning stated in the question; if the answer-token-only, fixed-window detector scores at chance while a probe on question-token states succeeds, the claim that the restored answer-state location is the correctness signal fails.

Watch

Extended reading notes

Core claim

Reasoning errors leave a trace not only in how residual-stream states move from layer to layer but in where those states sit when they move. The paper's central discovery is that a detector can exploit both by reading three complementary streams: the layerwise displacement, a vector-quantized coarse region label, and normalized per-layer directions. Together these streams restore enough state context to interpret cross-layer updates while staying narrow enough to suppress stable, label-correlated surface information. Trained on a single reasoning benchmark, the full model is the best of the three trained readers on 47 of 48 reasoning-transfer cells and beats every trained detector on unseen factual benchmarks, which the authors take as evidence that the signal is about correctness rather than a particular kind of reasoning.

Load-bearing premise

The method assumes that the information needed to interpret motion for correctness is present in the last 64 answer tokens and in a fixed window of six middle-to-late layers; if the signal lives mainly in the question tokens, early layers, or output-adjacent layers, the restored state is partial and the claimed transfer gains could shrink.

Editorial extensions

If this is right

  • Training on any of the six reasoning donors yields a detector that reads the other reasoning benchmarks it never saw more accurately than the motion-only baseline and the linear probe, with the largest 12-point margin over prior work on the ARC-Challenge donor.
  • Because the same reasoning-trained detector outscores every trained baseline on FACTOR and VitaminC without ever seeing factual training data, the paper positions the restored signal as correctness-specific rather than reasoning-task-specific.
  • The two added streams add only 0.31M parameters (6.7% of the motion LSTM), so the gain is attributed to the design, reading location alongside motion, rather than to added capacity.
  • Component ablations show motion, region, and direction each recover items the other two miss, and a per-item oracle stands 3 to 21 points above the trained fusion, implying a learned per-item gate could push the method further.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the region code is truly a shared coordinate of correctness, then a detector trained on reasoning should also transfer to other correctness-labeled domains such as code execution or translation adequacy; that is a testable prediction the paper does not run.
  • The fixed six-layer window is load-bearing: Appendix O shows the middle third of layers scores about 3 points higher on reasoning than the chosen window, so a learned layer-selection mechanism could raise the ceiling the paper reports.
  • Because correctness appears to be carried by answer-state location and motion, interventions that nudge a state toward the correct-answer region or that alter displacement directions might steer generation, not just detect errors.
  • The paper reads only the answer tokens; if question-token states encode the reasoning state, combining them with the current streams could close part of the gap between the trained fusion and the per-item oracle.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper proposes a three-stream validity detector for LLM candidate-answer selection. A motion reader consumes layerwise residual-stream displacements (prior work); a region reader adds a coarse vector-quantized code of the answer's final-token state; a direction reader adds fine normalized multi-layer state directions over the answer tokens. All streams are fused into a small MLP head and trained on a single reasoning benchmark, then evaluated zero-shot on held-out reasoning and factual benchmarks. The authors claim that this state-conditioned motion reading outperforms both decontextualized displacement-only methods and static single-layer probes, and that the signal transfers to factual errors, placing it on correctness rather than on a kind of reasoning.

Significance. If the central claim were fully supported, the method would be a practical, low-overhead detector of reasoning validity with transferable signal, and the articulated state-restoration trade-off is a useful conceptual contribution. The paper is transparent and unusually thorough: the main reasoning table uses three-seed means, the layer-window sweep and codebook-health diagnostics are reported, wording-leakage measurements are given, and cross-model checks on two additional model families are included. However, the central comparison against static states is not established, because the only static-state baseline in the main text is a single-layer linear probe; the paper's own direction reader, a much stronger static-state reader, nearly matches the full model on reasoning and beats it on one factual target. This gap is load-bearing for the abstract's central claim.

major comments (2)
  1. [Section 6.1, Table 1; Appendix J, Table 10] The abstract and Section 6.1 claim that state-conditioned motion beats static states by 10 to 21 points, but the only static-state baseline in Table 1 is a single-layer linear probe. Appendix J's late-fusion decomposition (Table 10) shows that the direction reader alone—a multi-layer, normalized, answer-span static reader—achieves 73.2 vs 75.1 on ARC-Challenge, 94.8 vs 96.1 on Story Cloze, and 69.5 vs 57.2 on FACTOR-expert against the full three-stream model. The evidence therefore supports 'motion plus state beats motion alone' and 'motion plus state beats a single-layer probe,' but not the general claim that state-conditioned motion beats static states. Moreover, the direction-only column in Table 10 comes from a jointly trained late-fusion variant, not from a standalone detector. To support the headline claim, the authors should train a direction-only detector with the same architecture and training recipe (motion stream removed) and compare it against the full model on all reported reasoning and factual targets.
  2. [Section 6.2, Table 2; Appendix I] The factual-transfer result, which the abstract uses to argue that the signal is about correctness rather than a kind of reasoning, rests on single-seed runs and includes FACTOR-expert with only 236 items. Appendix I acknowledges that the per-donor cells for FACTOR-expert are noisy. Separately, Appendix J Table 10 shows that the direction-only reader outperforms the fused model on FACTOR-expert by 12.3 points (69.5 vs 57.2), so the factual claim is also sensitive to the missing static-state baseline. The authors should report multi-seed means and standard deviations for Table 2 and for the per-donor cells, or explicitly mark the factual-transfer claim as preliminary.
minor comments (5)
  1. [Table 3] The column header 'Target Motion+Direction+Region Ours' is misaligned with the four numeric columns; the columns should be labeled 'Motion', '+Direction', '+Region', and 'Ours' to match the additive ablation described in the text.
  2. [Appendix O, Table 19] The paper states that the chosen window was fixed in advance and that the middle third reads 3.0 points higher on the reasoning mean, but the sentence 'The gain rests on reading the middle of the network, and not on a window selected to produce it' does not address the fact that the reported main results use a suboptimal window. Given that the middle-third arm is better on 11 of 12 targets, the authors should report the main-table results under the middle-third window as a robustness check, or explicitly state that the headline numbers are conservative relative to the depth sweep.
  3. [Equation (4) and references] The citation 'van den Oord, Vinyals, and kavukcuoglu 2017' contains a lowercase author name; it should be 'Kavukcuoglu' in both the citation and the reference list.
  4. [Appendix D and Section 6.2] The important caveat that the FACTOR-expert in-domain ceiling is based on a held-out split of about 47 items and is optimistic appears only in the appendix; a one-sentence version of this caveat should accompany the FACTOR-expert row in Table 2.
  5. [Section 6.1] The statement that the full model is best on 47 of 48 cells refers to Table 1, but it is not explicit whether the in-distribution cells are counted; the authors should state this explicitly to avoid ambiguity.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the detector is trained on donor benchmarks and tested zero-shot on held-out targets; the only self-citation is a baseline, not evidence.

full rationale

The paper's derivation does not reduce any claimed result to its own inputs. The three-stream detector is trained on one reasoning donor and evaluated on unseen reasoning and factual benchmarks (Tables 1 and 2), so headline accuracies are genuine zero-shot transfers rather than fits to the reported targets. The motion stream is taken from the authors' prior work (Damirchi et al. 2026) and used both as a component and as a baseline; the paper explicitly says 'Nothing in this reading is new, and nothing about it is changed,' and the improvement from adding location streams is measured against that same motion-only reader (Table 3, Figure 5). A self-citation to prior work is therefore not load-bearing: the prior work supplies the baseline, not the evidence for the new claim. The fixed six-layer window is a hand-set choice, and Appendix O candidly reports that the middle third reads 3.0 points higher on reasoning than the chosen window; this is a fragility of the method, not circularity, because the window is not selected on the reported results and the comparison is still zero-shot. The most serious challenge is not circularity but a missing matched static-state baseline: Appendix J shows a direction-only reader reaches 73.2 vs 75.1 on ARC-Challenge, 94.8 vs 96.1 on Story Cloze, 53.1 vs 65.0 on VitaminC, and 69.5 vs 57.2 on FACTOR-expert, so the main-text claim that state-conditioned motion beats static states rests on a single-layer linear probe as the only static baseline. That is a correctness/fairness risk, not an equivalence of Eq. X to Eq. Y by construction; no parameter is fitted to the benchmark it predicts, and no target result is derived from its own definition. Therefore no circular step is identifiable, and score 0 is appropriate.

Assumptions & free parameters 8 free parameters · 5 assumptions · 0 invented entities

The central claim rests on a small set of domain assumptions about where validity lives in the residual stream and how the training signal transfers, plus a list of hand-set architectural hyperparameters. The method introduces no new postulated entities.

free parameters (8)
  • Layer window L (six depths) = Llama: 8, 12, 16, 20, 24, 28; Qwen: 12, 18, 24, 30, 36, 42
    Hand-fixed middle-to-late layers read by region and direction streams; Appendix O shows performance varies with this choice, so it is a design decision the results rest on.
  • Region codebook size K = 128
    Sets coarse-state resolution; K is not swept in the main text.
  • Direction projection dimension = 8
    Projected dimension for each layer's normalized state in the direction stream.
  • Region projection dimension = 64
    Projected dimension feeding the region encoder before quantization.
  • Motion LSTM hidden width = 128
    Bidirectional LSTM width for the motion reader, inherited from prior work; width scaling in Appendix K does not close the gap.
  • Training epochs / batch / learning rate = 7 epochs; batch 16, grad accum 4; lr 1e-3 cosine
    Standard training recipe, not swept; checkpoint chosen by validation accuracy.
  • Answer token cap = 64 tokens
    Answer spans longer than 64 tokens are truncated before feature extraction.
  • Training set cap = 10,000 items per donor
    Equalizes donor sizes; changed datasets would alter the learned signal.
assumptions (5)
  • domain assumption Signals remain approximately linearly decodable across layers (linear representation hypothesis).
    Invoked in Section 2.1 to justify reading correctness from layer states and displacements.
  • domain assumption Differencing suppresses stable label-correlated surface signals while preserving reasoning-related changes.
    Central motivation for the motion stream, Section 3 and Eq. 1.
  • domain assumption The answer's normalized residual-stream directions at six middle-to-late layers carry the state information needed to interpret motion.
    Section 4.2 restricts both location streams to answer tokens and a fixed window; Appendix O tests depth but not token restriction.
  • domain assumption Correct/incorrect labels on donor benchmarks are a valid proxy for reasoning validity and transfer to other benchmarks.
    Section 5: pairwise training on one donor and zero-shot evaluation on targets assumes a shared correctness signal.
  • domain assumption Selection accuracy among supplied candidates reflects the detector's ability to identify reasoning errors.
    Task and metric definition in Section 5; the paper acknowledges it never judges a single answer in absolute terms (Section 7).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Reasoning Errors Have a Region and a Direction in the Residual-Stream Trajectory of LLMs." pith.science (2026). https://pith.science/paper/UEVUA7TO

@misc{pith2026260805660,
  author       = {Pith},
  title        = {Pith review of: Reasoning Errors Have a Region and a Direction in the Residual-Stream Trajectory of LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UEVUA7TO}},
  note         = {Machine review of arXiv:2608.05660}
}
read the original abstract

As language models are increasingly used for tasks that require verifiable reasoning, reliably distinguishing sound reasoning from flawed reasoning has become an important practical problem. Recent trajectory-based methods seek this signal in layerwise residual-stream displacements, which capture how representations change while attenuating some stable, token-specific information. However, displacement omits the state from which an update originates, whereas restoring the full state risks reintroducing shortcut-prone information. We identify this trade-off and propose a three-stream detector that combines motion with two restricted views of location. A coarse region reader based on vector quantization and a fine direction reader over normalized multi-layer states. This design restores enough state context to interpret the motion without returning to full-state probing. On reasoning benchmarks unseen during training, our method improves selection accuracy by up to 12% over the displacement-only state of the art and 21% over single-layer probing baselines. Although trained only on reasoning benchmarks, it also reads factual completion and fact verification, ahead of every detector we compare against, which places the signal on correctness rather than on a kind of reasoning. Ablations further show that motion, region, and direction provide complementary signals. These results suggest that reasoning validity is better read from state-conditioned motion than from either static states or decontextualized trajectories alone.

Figures

Figures reproduced from arXiv: 2608.05660 by the authors.

Figure 1
Figure 1. Three-stream validity detector. Motion, quan￾tized region, and direction signals are fused to classify can￾didate answers as valid or invalid. each activation as a point and asking whether a linear bound￾ary separates correct from incorrect answers (Azizian et al. 2025). However, successful classification does not by itself establish that the probe has detected whether the underlying reasoning is sound. A single-lay… view at source ↗
Figure 2
Figure 2. Context changes the state of the same statement. [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. The trade-off the detector has to solve. Each reader is placed by how often it selects the correct answer on ARC-Challenge, and by how much of that answer’s wording a held-out regression recovers from what it hands its deci￾sion head. Ours is marked at whichever of its three streams exposes the most wording. The two raw-state readers are mea￾sured at layer 14, and accuracy for the displacement reader and for ours is… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: A held-out linear read of the raw residual-stream [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Per-cell gain from restoring the location, Ours mi [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 7
Figure 7. Figure 7: Division of labor, the count of items each stream [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]
Figure 8
Figure 8. Figure 8: Overlap of the items each stream reads correctly, under the late-fusion decomposition of one trained detector. Each [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]
Figure 9
Figure 9. Figure 9: The same per-stream correctness sets, item by item, [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]
Figure 10
Figure 10. Figure 10: Region-code usage by dataset, one row per dataset over the [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

79 extracted references · 24 canonical work pages

  1. [1]

    arXiv preprint arXiv:2510.15511 , year=

    Language models are injective and hence invertible , author=. arXiv preprint arXiv:2510.15511 , year=

    arXiv

  2. [2]

    International Conference on Learning Representations , volume=

    Language models represent space and time , author=. International Conference on Learning Representations , volume=

    work page

  3. [3]

    arXiv preprint arXiv:2308.10248 , year=

    Steering language models with activation engineering , author=. arXiv preprint arXiv:2308.10248 , year=

    arXiv

  4. [4]

    Communications Medicine , volume =

    Structured clinical approach to enable large language models to be used for improved clinical diagnosis and explainable reasoning , author =. Communications Medicine , volume =. 2026 , month =. doi:10.1038/s43856-025-01348-x , pmid =

    doi:10.1038/s43856-025-01348-x 2026

  5. [5]

    2026 , eprint=

    The Open Proof Corpus: A Large-Scale Study of LLM-Generated Mathematical Proofs , author=. 2026 , eprint=

    work page 2026

  6. [6]

    2025 , eprint=

    Large Language Models are overconfident and amplify human bias , author=. 2025 , eprint=

    work page 2025

  7. [7]

    Communication, Simulation, and Intelligent Agents: Implications of Personal Intelligent Machines for Medical Education

    Clancey, William J. Communication, Simulation, and Intelligent Agents: Implications of Personal Intelligent Machines for Medical Education. Proceedings of the Eighth International Joint Conference on Artificial Intelligence (IJCAI-83)

  8. [8]

    Classification Problem Solving

    Clancey, William J. Classification Problem Solving. Proceedings of the Fourth National Conference on Artificial Intelligence

Show all 79 references
  1. [9]

    , title =

    Robinson, Arthur L. , title =. 1980 , doi =. https://science.sciencemag.org/content/208/4447/1019.full.pdf , journal =

    1980

  2. [10]

    New Ways to Make Microcircuits Smaller---Duplicate Entry

    Robinson, Arthur L. New Ways to Make Microcircuits Smaller---Duplicate Entry. Science

  3. [11]

    Clancey and Glenn Rennels , abstract =

    Diane Warner Hasling and William J. Clancey and Glenn Rennels , abstract =. Strategic explanations for a diagnostic consultation system , journal =. 1984 , issn =. doi:https://doi.org/10.1016/S0020-7373(84)80003-6 , url =

    1984 doi

  4. [12]

    and Rennels, Glenn R

    Hasling, Diane Warner and Clancey, William J. and Rennels, Glenn R. and Test, Thomas. Strategic Explanations in Consultation---Duplicate. The International Journal of Man-Machine Studies

  5. [13]

    Poligon: A System for Parallel Problem Solving

    Rice, James. Poligon: A System for Parallel Problem Solving

  6. [14]

    Transfer of Rule-Based Expertise through a Tutorial Dialogue

    Clancey, William J. Transfer of Rule-Based Expertise through a Tutorial Dialogue

  7. [15]

    The Engineering of Qualitative Models

    Clancey, William J. The Engineering of Qualitative Models

  8. [16]

    2023 , eprint=

    Attention Is All You Need , author=. 2023 , eprint=

    2023

  9. [17]

    Pluto: The 'Other' Red Planet

    NASA. Pluto: The 'Other' Red Planet

  10. [18]

    NeurIPS 2025 Workshop on Mathematical Reasoning and AI (MATH-AI) , year =

    Layer Importance for Mathematical Reasoning is Forged in Pre-Training and Invariant after Post-Training , author =. NeurIPS 2025 Workshop on Mathematical Reasoning and AI (MATH-AI) , year =. 2506.22638 , archiveprefix =

    2025

  11. [19]

    Advances in Neural Information Processing Systems (NeurIPS) , year =

    Knowledge Circuits in Pretrained Transformers , author =. Advances in Neural Information Processing Systems (NeurIPS) , year =. 2405.17969 , archiveprefix =

    arXiv

  12. [20]

    2024 , eprint =

    How to Think Step-by-Step: A Mechanistic Understanding of Chain-of-Thought Reasoning , author =. 2024 , eprint =

    2024

  13. [21]

    Advances in Neural Information Processing Systems (NeurIPS) , year =

    Calibrating Reasoning in Language Models with Internal Consistency , author =. Advances in Neural Information Processing Systems (NeurIPS) , year =. 2405.18711 , archiveprefix =

    arXiv

  14. [22]

    Locating and Editing Factual Associations in

    Meng, Kevin and Bau, David and Andonian, Alex and Belinkov, Yonatan , booktitle =. Locating and Editing Factual Associations in. 2022 , eprint =

    2022

  15. [23]

    Aho and Jeffrey D

    Alfred V. Aho and Jeffrey D. Ullman , title =. 1972

    1972

  16. [24]

    Publications Manual , year = "1983", publisher =

    1983

  17. [25]

    Chandra and Dexter C

    Ashok K. Chandra and Dexter C. Kozen and Larry J. Stockmeyer , year = "1981", title =. doi:10.1145/322234.322243

    1981

  18. [26]

    Scalable training of

    Andrew, Galen and Gao, Jianfeng , booktitle=. Scalable training of

  19. [27]

    Dan Gusfield , title =. 1997

    1997

  20. [28]

    Tetreault , title =

    Mohammad Sadegh Rasooli and Joel R. Tetreault , title =. Computing Research Repository , volume =. 2015 , url =

    2015

  21. [29]

    A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data , Volume =

    Ando, Rie Kubota and Zhang, Tong , Issn =. A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data , Volume =. Journal of Machine Learning Research , Month = dec, Numpages =

  22. [30]

    arXiv preprint arXiv:2311.03658 , year=

    The Linear Representation Hypothesis and the Geometry of Large Language Models , author=. arXiv preprint arXiv:2311.03658 , year=

    arXiv

  23. [31]

    arXiv preprint arXiv:2209.10652 , year=

    Toy Models of Superposition , author=. arXiv preprint arXiv:2209.10652 , year=

    arXiv

  24. [32]

    Computational Linguistics , volume=

    Probing Classifiers: Promises, Shortcomings, and Advances , author=. Computational Linguistics , volume=

  25. [33]

    arXiv preprint arXiv:2310.06824 , year=

    The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets , author=. arXiv preprint arXiv:2310.06824 , year=

    arXiv

  26. [34]

    arXiv preprint arXiv:2212.03827 , year=

    Discovering Latent Knowledge in Language Models Without Supervision , author=. arXiv preprint arXiv:2212.03827 , year=

    arXiv

  27. [35]

    arXiv preprint arXiv:2310.01405 , year=

    Representation Engineering: A Top-Down Approach to AI Transparency , author=. arXiv preprint arXiv:2310.01405 , year=

    arXiv

  28. [36]

    Advances in Neural Information Processing Systems , volume=

    Neural Ordinary Differential Equations , author=. Advances in Neural Information Processing Systems , volume=

  29. [37]

    arXiv preprint arXiv:1906.02762 , year=

    Understanding and Improving Transformer From a Multi-Particle Dynamic System Perspective , author=. arXiv preprint arXiv:1906.02762 , year=

    1906 arXiv

  30. [38]

    arXiv preprint arXiv:2305.05465 , year=

    The Emergence of Clusters in Self-Attention Dynamics , author=. arXiv preprint arXiv:2305.05465 , year=

    arXiv

  31. [39]

    arXiv preprint arXiv:2510.09782 , year=

    The Geometry of Reasoning: Flowing Logics in Representation Space , author=. arXiv preprint arXiv:2510.09782 , year=

  32. [40]

    arXiv preprint arXiv:2505.20340 , year=

    Empirical Investigation of Latent Representational Dynamics in Large Language Models: A Manifold Evolution Perspective , author=. arXiv preprint arXiv:2505.20340 , year=

    arXiv

  33. [41]

    International Conference on Machine Learning , pages=

    Soft-DTW: a Differentiable Loss Function for Time-Series , author=. International Conference on Machine Learning , pages=. 2017 , organization=

    2017

  34. [42]

    2025 , url=

    Hadas Orgad and Michael Toker and Zorik Gekhman and Roi Reichart and Idan Szpektor and Hadas Kotek and Yonatan Belinkov , booktitle=. 2025 , url=

    2025

  35. [43]

    Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency , year=

    LExT: Towards Evaluating Trustworthiness of Natural Language Explanations , author=. Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency , year=

    2025

  36. [44]

    2025 , eprint=

    A Comprehensive Survey on the Trustworthiness of Large Language Models in Healthcare , author=. 2025 , eprint=

    2025

  37. [45]

    Probing the Geometry of Truth: Consistency and Generalization of Truth Directions in LLM s Across Logical Transformations and Question Answering Tasks

    Bao, Yuntai and Zhang, Xuhong and Du, Tianyu and Zhao, Xinkui and Feng, Zhengwen and Peng, Hao and Yin, Jianwei. Probing the Geometry of Truth: Consistency and Generalization of Truth Directions in LLM s Across Logical Transformations and Question Answering Tasks. Findings of ...

    2025 doi

  38. [46]

    Submitted to The Fourteenth International Conference on Learning Representations , year=

    Steering Language Models with Weight Arithmetic , author=. Submitted to The Fourteenth International Conference on Learning Representations , year=

  39. [47]

    Steering Llama 2 via Contrastive Activation Addition

    Rimsky, Nina and Gabrieli, Nick and Schulz, Julian and Tong, Meg and Hubinger, Evan and Turner, Alexander. Steering Llama 2 via Contrastive Activation Addition. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 202...

    2024 doi

  40. [48]

    Submitted to The Fourteenth International Conference on Learning Representations , year=

    Enhancing Cross-task Transfer of Large Language Models via Fourier Activation Steering , author=. Submitted to The Fourteenth International Conference on Learning Representations , year=

  41. [49]

    2025 , eprint=

    Transformer Dynamics: A neuroscientific approach to interpretability of large language models , author=. 2025 , eprint=

    2025

  42. [50]

    Trenton Bricken and Adly Templeton and Joshua Batson and Brian Chen and Adam Jermyn and Tom Conerly and Nicholas L. Turner and Cem Anil and Carson Denison and Amanda Askell and Robert Lasenby and Yifan Wu and Shauna Kravec and Nicholas Schiefer and Tim Maxwell and Nicholas Jos...

    2023

  43. [51]

    2025 , eprint=

    Steering Language Model Refusal with Sparse Autoencoders , author=. 2025 , eprint=

    2025

  44. [52]

    ICML 2025 Workshop on Reliable and Responsible Foundation Models , year=

    The Geometries of Truth Are Orthogonal Across Tasks , author=. ICML 2025 Workshop on Reliable and Responsible Foundation Models , year=

    2025

  45. [53]

    Lindsey, Jack and Gurnee, Wes and Ameisen, Emmanuel and Chen, Brian and Pearce, Adam and Turner, Nicholas L. and Citro, Craig and Abrahams, David and Carter, Shan and Hosmer, Basil and Marcus, Jonathan and Sklar, Michael and Templeton, Adly and Bricken, Trenton and McDougall, ...

  46. [54]

    2025 , eprint=

    I Predict Therefore I Am: Is Next Token Prediction Enough to Learn Human-Interpretable Concepts from Data? , author=. 2025 , eprint=

    2025

  47. [55]

    The Twelfth International Conference on Learning Representations , year=

    Sparse Autoencoders Find Highly Interpretable Features in Language Models , author=. The Twelfth International Conference on Learning Representations , year=

  48. [56]

    MINT: Foundation Model Interventions , year=

    Steering Large Language Models using Conceptors: Improving Addition-Based Activation Engineering , author=. MINT: Foundation Model Interventions , year=

  49. [57]

    arXiv preprint arXiv:1803.05457 , year=

    Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge , author=. arXiv preprint arXiv:1803.05457 , year=

    arXiv

  50. [58]

    BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions , author=. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) , pages=

    2019

  51. [59]

    Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , pages=

    HellaSwag: Can a Machine Really Finish Your Sentence? , author=. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , pages=

  52. [60]

    Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing , pages=

    Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering , author=. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing , pages=

    2018

  53. [61]

    Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , pages=

    A Corpus and Cloze Evaluation for Deeper Understanding of Commonsense Stories , author=. Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , pages=

    2016

  54. [62]

    CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge , author=. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) , pages=

    2019

  55. [63]

    Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoning , author=. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) , pages=

    2019

  56. [64]

    Social IQa: Commonsense Reasoning about Social Interactions , author=. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) , pages=

    2019

  57. [65]

    Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection , author=. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  58. [66]

    Findings of the Association for Computational Linguistics: EMNLP 2020 , pages=

    RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models , author=. Findings of the Association for Computational Linguistics: EMNLP 2020 , pages=

    2020

  59. [67]

    Truth as a Trajectory: What Internal Representations Reveal About Large Language Model Reasoning

    Damirchi, Hamed and Imezadelajara and Abbasnejad, Ehsan and Shamsi, Afshar and Zhang, Zhen and Shi, Javen Qinfeng. Truth as a Trajectory: What Internal Representations Reveal About Large Language Model Reasoning. Proceedings of the 64th Annual Meeting of the A ssociation for C...

    2026 doi

  60. [68]

    2025 , eprint=

    Eliciting Latent Predictions from Transformers with the Tuned Lens , author=. 2025 , eprint=

    2025

  61. [69]

    2023 , month =

    Privileged Bases in the Transformer Residual Stream , author =. 2023 , month =

    2023

  62. [70]

    Rewiring the Transformer with Depth-Wise LSTM s

    Xu, Hongfei and Song, Yang and Liu, Qiuhui and van Genabith, Josef and Xiong, Deyi. Rewiring the Transformer with Depth-Wise LSTM s. Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024). 2024

    2024

  63. [71]

    2024 , eprint=

    The Llama 3 Herd of Models , author=. 2024 , eprint=

    2024

  64. [72]

    2025 , eprint=

    Qwen2.5 Technical Report , author=. 2025 , eprint=

    2025

  65. [73]

    2025 , eprint=

    Qwen3 Technical Report , author=. 2025 , eprint=

    2025

  66. [74]

    International Conference on Learning Representations , year=

    Measuring Massive Multitask Language Understanding , author=. International Conference on Learning Representations , year=

  67. [75]

    Generating Benchmarks for Factuality Evaluation of Language Models

    Muhlgay, Dor and Ram, Ori and Magar, Inbal and Levine, Yoav and Ratner, Nir and Belinkov, Yonatan and Abend, Omri and Leyton-Brown, Kevin and Shashua, Amnon and Shoham, Yoav. Generating Benchmarks for Factuality Evaluation of Language Models. Proceedings of the 18th Conference...

    2024 doi

  68. [76]

    Get Your Vitamin C ! Robust Fact Verification with Contrastive Evidence

    Schuster, Tal and Fisch, Adam and Barzilay, Regina. Get Your Vitamin C ! Robust Fact Verification with Contrastive Evidence. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021. do...

    2021 doi

  69. [77]

    2024 , eprint=

    The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets , author=. 2024 , eprint=

    2024

  70. [78]

    Neural Discrete Representation Learning , url =

    van den Oord, Aaron and Vinyals, Oriol and kavukcuoglu, koray , booktitle =. Neural Discrete Representation Learning , url =

  71. [79]

    Lancucki, Adrian and Chorowski, Jan and Sanchez, Guillaume and Marxer, Ricard and Chen, Nanxin and Dolfing, Hans J. G. A. and Khurana, Sameer and Alum. Robust Training of Vector Quantized Bottleneck Models , booktitle =. 2020 , pages =

    2020

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.