Pith. sign in

REVIEW 3 major objections 7 minor 65 references

Reviewed by Pith at T0; open to challenge.

T0 means a machine referee read the full paper against a public rubric. The mark states how deep the mechanical check went, never who wrote it. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · glm-5.2

Neural networks contain recoverable internal circuits

2026-07-09 14:18 UTC pith:NP2VXWY5

load-bearing objection Solid scoping review of mechanistic interpretability; covers the right ground but has a PRISMA compliance gap and some framing asymmetry between abstract and body. the 3 major comments →

arxiv 2607.07316 v1 pith:NP2VXWY5 submitted 2026-07-08 cs.LG

Mechanistic Interpretability for Neural Networks: Circuits, Sparse Features and Symbolic Reasoning

Pranav Sawant , Jakub Krej\v{c}\'i This is my paper
classification cs.LG
keywords mechanisticneuralfeaturesinternalinterpretabilitylearninglikemethods
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper is a scoping review arguing that mechanistic interpretability has matured from surface-level input-output explanations into a toolkit that can identify, test, and modify the internal algorithms neural networks actually run. The central object is the circuit: a small subnetwork of attention heads, MLP layers, and residual-stream pathways that is causally responsible for a specific behavior. The paper traces a chain of tools designed to reverse-engineer these circuits. Transformer circuit analysis decomposes models into Query-Key circuits (where to look) and Output-Value circuits (what to copy), with induction heads serving as the canonical example of how pattern completion and in-context learning emerge from identifiable attention mechanisms. Automated Circuit Discovery (ACDC) and faster attribution-based methods systematize the search for these circuits by ablating edges and measuring output change, replacing manual hypothesis-driven exploration with algorithmic graph pruning. The review then identifies superposition, the compression of many features into fewer dimensions producing polysemantic neurons, as the core obstacle to interpretation. Sparse Autoencoders (SAEs) address this by decomposing tangled activations into sparse, monosemantic features that correspond to human-understandable concepts. Transcoders extend this from static representation to dynamic transformation, modeling how MLP layers convert inputs to outputs rather than merely reconstructing activations. Cross-Layer Transcoders trace transformations across multiple layers, revealing structures like the three-phase language processing pipeline where input-language-specific early layers feed a language-agnostic semantic space in middle layers, which then re-specializes to an output language. Steering vectors and SAE-based feature clamping provide causal tests: if a feature is real, amplifying or suppressing it should predictably change model behavior, which experiments on sentiment, truthfulness, refusal, and even functional emotion representations confirm. Finally, neurosymbolic frameworks (NeSyFOLD, NeSyViT) translate neural representations into executable logical rules with default assumptions and exceptions, though the paper notes these approximate the decision boundary rather than necessarily recovering the model's internal computation.

Core claim

The paper's central claim is that neural networks contain recoverable structure and that at least some internal mechanisms can be identified, tested, and modified. It assembles evidence from circuit analysis (induction heads, indirect object identification), sparse feature decomposition (SAEs extracting monosemantic features from polysemantic neurons), causal intervention (steering vectors changing behavior along linear semantic directions), and neurosymbolic rule extraction to argue that these tools collectively form a maturing toolkit for reverse-engineering model internals. The review frames this as a progression: from saliency maps and attention visualization (which show correlation) to,

What carries the argument

The circuit: a causally identifiable subnetwork of attention heads, MLP layers, and residual-stream pathways responsible for a specific behavior, decomposable via QK/OV circuits, discoverable via ACDC or attribution-based methods, and made interpretable by SAEs that disentangle polysemantic superposition into monosemantic features.

Load-bearing premise

The paper assumes that interpretability tools validated on small models and narrow tasks like induction heads in GPT-2 Small will meaningfully transfer to frontier-scale systems, yet provides no evidence that this premise holds at scale.

What would settle it

Demonstrating that circuit-level mechanisms identified in small models do not correspond to the computational structure of frontier-scale models, or that SAE-extracted features that appear monosemantic on narrow tasks become irreducibly polysemantic or non-causal at scale.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If circuit-level tools scale to frontier models, safety-relevant behaviors like deception, sycophancy, or dangerous capability emergence could be audited and surgically modified before deployment, rather than detected only through behavioral testing.
  • The finding that induction heads emerge as sudden phase transitions driven by interacting sub-circuits implies that dangerous capabilities in future models could similarly appear abruptly, making interpretability monitoring during training a safety-critical early-warning system.
  • SAE-derived steering vectors that causally control high-level properties like truthfulness or refusal suggest that alignment could eventually be achieved through direct internal representation editing rather than solely through behavioral training methods like RLHF.
  • The three-phase language processing structure (input-specific to language-agnostic to output-specific) and the discovery that tokenization fragmentation causes capacity exhaustion in early layers provide a mechanistic explanation for cross-lingual performance gaps that could inform tokenizer design.
  • The distinction between mechanistic faithfulness (recovering the actual computation) and behavioral approximation (summarizing the decision boundary) sets a falsifiable boundary: neurosymbolic rules that predict correctly but fail under distribution shift would reveal they captured shortcuts rather than the model's algorithm.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. This manuscript is a scoping review of mechanistic interpretability (MI) for neural networks, covering Transformer circuit analysis (QK/OV circuits, induction heads, IOI), sparse autoencoders (SAEs) and transcoders for addressing polysemanticity, steering vectors and causal interventions, and neurosymbolic rule extraction frameworks (NeSyFOLD, NeSyViT). The review follows a PRISMA-ScR methodology and synthesizes approximately 58 references spanning foundational works (2021) through recent preprints (2025-2026). The central claim is that MI has progressed from surface-level explanations toward causal, algorithmic accounts of neural network behavior, and that the surveyed tools collectively form a maturing toolkit for reverse-engineering and controlling model internals.

Significance. The review provides a competent and broad synthesis of a rapidly evolving field. Its strengths include accurate technical descriptions of QK/OV circuit decomposition, SAE architectures (Table 2 is a useful comparison), the distinction between SAEs and transcoders (Section 4.5), and the CAA steering vector formulation (Eq. 1). The discussion of neurosymbolic methods (Section 6) and the explicit acknowledgment that these approximate decision boundaries rather than internal computation (Section 7.3) is a valuable clarification. The paper correctly identifies the scale-transfer gap as a central limitation (Section 7). The PRISMA-ScR framing is appropriate for a scoping review of this nature.

major comments (3)
  1. Section 2: The manuscript claims PRISMA-ScR compliance but does not include a PRISMA flow diagram, does not report the number of records identified, screened, excluded, or included at each stage, and does not provide the exact search string used. The PRISMA-ScR checklist (Tricco et al., 2018, ref [8]) requires a flow diagram and quantitative reporting of the selection process. Without these elements, the claim of PRISMA-ScR adherence is not substantiated. This is load-bearing for the review's methodological credibility.
  2. Section 3.6, ref [22]: The citation for Edge Attribution Patching (EAP) is attributed to Zhang et al. (2026), arXiv:2509.21044, titled 'Reinforcement learning fine-tuning enhances activation intensity and diversity in the internal circuitry of LLMs.' This does not appear to be the EAP paper; the original EAP work is by Syed et al. (2023) or Nanda. The reference appears mismatched to the method described in the text, which could mislead readers seeking the primary source.
  3. Section 4.3: The abstract states that SAEs 'can decompose tangled network activations into distinct, human-interpretable features,' which is stronger than the caveats in Sections 4.3, 4.5, and 7.1 regarding shrinkage, dead features, feature fragmentation, and the gap between SAE-extracted features and ideal supervision dictionaries (ref [58]). The abstract's framing should be softened to match the body's appropriately hedged discussion.
minor comments (7)
  1. Section 3.1: The phrase 'The method described in the paper addresses this problem by replacing these difficult-to-read MLP layers with multi-layer transcoders (CLT)' is ambiguous — 'the paper' could refer to this manuscript or a cited work. Clarify which work introduced CLTs.
  2. Section 4.3: The SAE encoder equation uses sigma for the activation function but does not specify which activation (ReLU, TopK, JumpReLU) is the default; Table 2 lists variants but the equation in the text appears to assume a generic nonlinearity.
  3. Section 5.1: The ActAdd method is described without a formal equation, while CAA receives one (Eq. 1). Adding the ActAdd formulation or a reference to it would improve parallelism.
  4. Figure 1: The figure caption 'Transformer information flow through the residual stream' is minimal; adding labels for the QK/OV circuits or attention/MLP blocks within the figure would aid readers unfamiliar with the architecture.
  5. Section 6.1: The rule example 'target(X, kitchen) :- not 3(X), 54(X)' uses kernel numbers as predicates, which is correct but could confuse readers unfamiliar with ASP syntax; a brief note that these are placeholder predicate names would help.
  6. Several references have 2026 dates (e.g., refs [2], [22], [26], [27], [39], [43], [44], [52]) which appear to be future-dated preprints; verify these are not placeholder dates.
  7. Section 3.7: 'ALTI algorithm' is introduced without a full citation at the point of first mention; the reference to Ferrando and Voita [23] appears later. Moving the citation forward would help.

Circularity Check

0 steps flagged

No circularity found — this is a scoping review with no derivations, predictions, or self-citations.

full rationale

This paper is a PRISMA-ScR scoping review that synthesizes external literature on mechanistic interpretability. It contains no novel derivation chain, no fitted parameters, and no predictions that could reduce to inputs by construction. The only equation (Eq. 1, the CAA steering vector formula) is attributed to Panickssery et al. [49], an external citation. A check of all 58 references reveals no self-citations by either author (Sawant or Krejčí). The paper's central claim—that MI has progressed toward causal, algorithmic accounts and that surveyed tools form a maturing toolkit—is a qualitative synthesis claim, not a quantitative result. There is no mechanism by which circularity could arise in such a paper.

Axiom & Free-Parameter Ledger

0 free parameters · 4 axioms · 0 invented entities

As a review paper, no new free parameters, axioms, or invented entities are introduced. The axioms listed are domain assumptions from the surveyed literature that the review relies on structurally. No new entities are postulated.

axioms (4)
  • domain assumption Universality Hypothesis: similar neural networks trained on similar data will form similar internal circuits
    Invoked in Section 3.2 as a foundational premise of mechanistic interpretability. The paper notes that if this hypothesis were false, circuit identification would be combinatorially unsolvable. It is presented as an assumption, not proven.
  • domain assumption Linear Representation Hypothesis: high-level semantic concepts are encoded as linear directions or low-dimensional subspaces in activation space
    Invoked in Section 5 as the theoretical foundation for steering vectors. The paper states that if this hypothesis is correct, interpretation and control can be achieved via linear algebraic operations.
  • domain assumption Superposition hypothesis: models represent M concepts using D neurons where M >> D, encoding concepts as nearly-orthogonal directions
    Invoked in Section 4.1 as the explanation for polysemanticity. It is an assumption about how networks compress information, drawn from cited toy model experiments.
  • standard math PRISMA-ScR guidelines are appropriate for mapping the MI landscape without critical appraisal of individual study quality
    Section 2 states the review follows PRISMA-ScR guidelines and explicitly does not perform formal critical appraisal of included sources, relying on this methodological framework as a valid approach for scoping reviews.

pith-pipeline@v1.1.0-glm · 27071 in / 2438 out tokens · 202729 ms · 2026-07-09T14:18:46.326540+00:00 · methodology

0 comments
read the original abstract

This article offers a comprehensive overview of mechanistic interpretability, an emerging field that seeks to reverse-engineer the internal algorithms of modern neural networks. While traditional explainable AI methods often stop at surface-level input-output correlations, this approach directly addresses the opaque "black box" nature of machine learning models, which is essential for ensuring safety and auditability in high-stakes deployments. The paper provides a detailed examination of Transformer circuit analysis, exploring how internal components like the residual stream, attention mechanisms, and induction heads drive complex tasks and in-context learning. It subsequently tackles the core challenge of superposition and polysemanticity, demonstrating how tools like Sparse Autoencoders (SAEs) and transcoders can decompose tangled network activations into distinct, human-interpretable features. Furthermore, the paper explores methods for actively controlling and modifying model behavior through steering vectors and causal interventions. Finally, it connects these mechanistic insights with neurosymbolic AI frameworks designed to translate neural representations into explicit, executable logical rules.

Figures

Figures reproduced from arXiv: 2607.07316 by Jakub Krej\v{c}\'i, Pranav Sawant.

Figure 1
Figure 1. Figure 1: Transformer information flow through the residual [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Induction heads complete repeated token patterns by [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: IOI task pipeline: the model detects names, suppresses [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Superposition compresses many features into shared rep [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Steering vectors modify model behavior by adding a [PITH_FULL_IMAGE:figures/full_fig_p013_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

65 extracted references · 65 canonical work pages · 31 internal anchors

  1. [1]

    Unboxing the black box: Mechanistic interpretability for algorithmic understanding of neural networks,

    B. Kowalska and H. Kwasnicka, “Unboxing the black box: Mechanistic interpretability for algorithmic understanding of neural networks,” 2025. [Online]. Available: https: //arxiv.org/abs/2511.19265 1

    work page arXiv 2025

  2. [2]

    Mechanistic interpretability for large language model alignment: Progress, challenges, and future directions,

    U. Naseem, “Mechanistic interpretability for large language model alignment: Progress, challenges, and future directions,” 2026. [Online]. Available: https://arxiv.org/abs/2602.11180 1

    work page arXiv 2026

  3. [3]

    A mathematical framework for transformer circuits,

    N. Elhage, N. Nanda, C. Olsson, T. Henighan, N. Joseph, B. Mann, A. Askell, Y . Bai, A. Chen, T. Conerlyet al., “A mathematical framework for transformer circuits,”Trans- former Circuits Thread, vol. 1, no. 1, p. 12, 2021. 1

    work page 2021

  4. [4]

    What Makes for a Good Saliency Map? Comparing Strategies for Evaluating Saliency Maps in Explainable AI (XAI)

    F. Kares, T. Speith, H. Zhang, and M. Langer, “What makes for a good saliency map? comparing strategies for evaluating saliency maps in explainable ai (xai),” 2025. [Online]. Available: https://arxiv.org/abs/2504.17023 2

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  5. [5]

    A perspective on explainable artificial intelligence methods: Shap and lime,

    A. M. Salih, Z. Raisi-Estabragh, I. B. Galazzo, P. Radeva, S. E. Petersen, K. Lekadir, and G. Menegaz, “A perspective on explainable artificial intelligence methods: Shap and lime,”Advanced Intelligent Systems, vol. 7, no. 1, Jun

    work page

  6. [6]

    Available: http://dx.doi.org/10.1002/aisy

    [Online]. Available: http://dx.doi.org/10.1002/aisy. 202400304 2, 3

    work page doi:10.1002/aisy

  7. [7]

    Understanding intermediate layers using linear classifier probes

    G. Alain and Y . Bengio, “Understanding intermediate layers using linear classifier probes,” 2018. [Online]. Available: https://arxiv.org/abs/1610.01644 3

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  8. [8]

    Visualizing Attention in Transformer-Based Language Representation Models

    J. Vig, “Visualizing attention in transformer-based language representation models,” 2019. [Online]. Available: https: //arxiv.org/abs/1904.02679 3

    work page internal anchor Pith review Pith/arXiv arXiv 2019

  9. [9]

    Prisma extension for scoping reviews (prisma-scr): checklist and explanation,

    A. C. Tricco, E. Lillie, W. Zarin, K. K. O’Brien, H. Colquhoun, D. Levac, D. Moher, M. D. Peters, T. Horsley, L. Weekset al., “Prisma extension for scoping reviews (prisma-scr): checklist and explanation,”Annals of Internal Medicine, vol. 169, no. 7, pp. 467–473, 2018. [Online]. Available: https://www.acpjournals.org/doi/10.7326/M18- 0850 3

    work page doi:10.7326/m18- 2018

  10. [10]

    Scoping studies: towards a methodological framework,

    H. Arksey and L. O’malley, “Scoping studies: towards a methodological framework,”International journal of social research methodology, vol. 8, no. 1, pp. 19–32, 2005. 3

    work page 2005

  11. [11]

    Circuit tracing: Revealing compu- tational graphs in language models,

    J. Lindsey, W. Gurnee, E. Ameisen, B. Chen, A. Pearce, N. L. Turneret al., “Circuit tracing: Revealing compu- tational graphs in language models,”Transformer Circuits Thread, 2025. [Online]. Available: https://transformer- circuits.pub/2025/attribution-graphs/methods.html 4, 12

    work page 2025

  12. [12]

    Open Problems in Mechanistic Interpretability

    L. Sharkey, B. Chughtai, J. Batson, J. Lindsey, J. Wu, L. Bushnaq, N. Goldowsky-Dill, S. Heimersheim, A. Ortega, J. Bloom, S. Biderman, A. Garriga-Alonso, A. Conmy, N. Nanda, J. Rumbelow, M. Wattenberg, N. Schoots, J. Miller, E. J. Michaud, S. Casper, M. Tegmark, W. Saunders, D. Bau, E. Todd, A. Geiger, M. Geva, J. Hoogland, D. Murfet, and T. McGrath, “Op...

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  13. [13]

    Transformerlens,

    N. Nanda and J. Bloom, “Transformerlens,” https://github. com/TransformerLensOrg/TransformerLens, 2022. 4

    work page 2022

  14. [14]

    On the geometry and topology of representations: the manifolds of modular addition,

    G. Moisescu-Pareja, G. McCracken, H. Wiltzer, V . L´etourneau, C. Daniels, D. Precup, and J. Love, “On the geometry and topology of representations: the manifolds of modular addition,” 2025. [Online]. Available: https://arxiv.org/abs/2512.25060 5

    work page arXiv 2025

  15. [15]

    What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

    A. K. Singh, T. Moskovitz, F. Hill, S. C. Chan, and A. M. Saxe, “What needs to go right for an induction head? a mech- anistic study of in-context learning circuits and their forma- tion,”arXiv preprint arXiv:2404.07129, 2024. 5

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  16. [16]

    In-context Learning and Induction Heads

    C. Olsson, N. Elhage, N. Nanda, N. Joseph, N. DasSarma, T. Henighan, B. Mann, A. Askell, Y . Bai, A. Chen, T. Conerly, D. Drain, D. Ganguli, Z. Hatfield-Dodds, D. Hernandez, S. Johnston, A. Jones, J. Kernion, L. Lovitt, K. Ndousse, D. Amodei, T. Brown, J. Clark, J. Kaplan, S. McCandlish, and C. Olah, “In-context learning and induction heads,” 2022. [Onlin...

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  17. [17]

    Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small

    K. Wang, A. Variengien, A. Conmy, B. Shlegeris, and J. Steinhardt, “Interpretability in the wild: a circuit for indirect object identification in gpt-2 small,” 2022. [Online]. Available: https://arxiv.org/abs/2211.00593 6

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  18. [18]

    Investigating the Indirect Object Identification circuit in Mamba

    D. Ensign and A. Garriga-Alonso, “Investigating the indirect object identification circuit in mamba,” 2024. [Online]. Available: https://arxiv.org/abs/2407.14008 7

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  19. [19]

    Emergence of Minimal Circuits for Indirect Object Identification in Attention-Only Transformers

    R. Adhikari, “Emergence of minimal circuits for indirect object identification in attention-only transformers,” 2025. [Online]. Available: https://arxiv.org/abs/2510.25013 7

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  20. [20]

    Towards Automated Circuit Discovery for Mechanistic Interpretability

    A. Conmy, A. N. Mavor-Parker, A. Lynch, S. Heimer- sheim, and A. Garriga-Alonso, “Towards automated circuit discovery for mechanistic interpretability,” 2023. [Online]. Available: https://arxiv.org/abs/2304.14997 7

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  21. [21]

    Efficient automated circuit discovery in transformers using contextual decomposition,

    A. R. Hsu, G. Zhou, Y . Cherapanamjeri, Y . Huang, A. Y . Odisho, P. R. Carroll, and B. Yu, “Efficient automated circuit discovery in transformers using contextual decomposition,” 18

    work page

  22. [22]

    Efficient Automated Circuit Discovery in Transformers using Contextual Decomposition

    [Online]. Available: https://arxiv.org/abs/2407.00886 7

    work page internal anchor Pith review Pith/arXiv arXiv

  23. [23]

    Pahq: Accelerating automated circuit discovery through mixed-precision inference optimization,

    X. Wang, S. Yang, L. Wang, L. Zhang, H. Xie, L. Hu, and D. Wang, “Pahq: Accelerating automated circuit discovery through mixed-precision inference optimization,”

    work page

  24. [24]

    Available: https://arxiv.org/abs/2510.23264 7

    [Online]. Available: https://arxiv.org/abs/2510.23264 7

    work page arXiv

  25. [25]

    Reinforcement learning fine-tuning enhances activation intensity and diversity in the internal circuitry of llms,

    H. Zhang, Q. Hao, F. Xu, and Y . Li, “Reinforcement learning fine-tuning enhances activation intensity and diversity in the internal circuitry of llms,” 2026. [Online]. Available: https://arxiv.org/abs/2509.21044 8

    work page arXiv 2026

  26. [26]

    Information flow routes: Automatically interpreting language models at scale,

    J. Ferrando and E. V oita, “Information flow routes: Automatically interpreting language models at scale,” in Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Y . Al-Onaizan, M. Bansal, and Y .-N. Chen, Eds. Miami, Florida, USA: Association for Computational Linguistics, Nov. 2024, pp. 17 432–17 445. [Online]. Availa...

    work page 2024

  27. [27]

    A Mechanistic Analysis of a Transformer Trained on a Symbolic Multi-Step Reasoning Task

    J. Brinkmann, A. Sheshadri, V . Levoso, P. Swoboda, and C. Bartelt, “A mechanistic analysis of a transformer trained on a symbolic multi-step reasoning task,”arXiv preprint arXiv:2402.11917, 2024. 8

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  28. [28]

    Emergent symbolic mechanisms support abstract reasoning in large language models,

    Y . Yang, D. I. Campbell, K. Huang, M. Wang, J. D. Cohen, and T. W. Webb, “Emergent symbolic mechanisms support abstract reasoning in large language models,” inProceedings of the 42nd International Conference on Machine Learning (ICML), 2025. 8

    work page 2025

  29. [29]

    Improving Sparse Autoencoder with Dynamic Attention

    D. Wang, J. Zhang, D. Su, and H. Huang, “Improving sparse autoencoder with dynamic attention,” 2026. [Online]. Available: https://arxiv.org/abs/2604.14925 9

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  30. [30]

    Mechanistic interpretability with sparse autoencoder neural operators,

    B. Tolooshams, A. Shen, and A. Anandkumar, “Mechanistic interpretability with sparse autoencoder neural operators,”

    work page

  31. [31]

    Mechanistic Interpretability with Sparse Autoencoder Neural Operators

    [Online]. Available: https://arxiv.org/abs/2509.03738 9

    work page internal anchor Pith review Pith/arXiv arXiv

  32. [32]

    Toy Models of Superposition

    N. Elhage, T. Hume, C. Olsson, N. Schiefer, T. Henighan, S. Kravec, Z. Hatfield-Dodds, R. Lasenby, D. Drain, C. Chenet al., “Toy models of superposition,”arXiv preprint arXiv:2209.10652, 2022. 9

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  33. [33]

    Towards monosemanticity: Decomposing language models with dictionary learning,

    T. Bricken, A. Templeton, J. Batson, B. Chen, A. Jermyn, T. Conerly, N. Turner, C. Olsson, S. Douglas, B. McKenzie- Nelson, N. Elhage, S. Hume, J. Knight, B. Shlegeris, N. Ninan, A. Schlatter, C. Olah, and T. Henighan, “Towards monosemanticity: Decomposing language models with dictionary learning,”Transformer Circuits Thread, 2023, anthropic Research. [On...

    work page 2023

  34. [34]

    Finding Neurons in a Haystack: Case Studies with Sparse Probing

    W. Gurnee, N. Nanda, M. Pauly, K. Harvey, D. Troitskii, and D. Bertsimas, “Finding neurons in a haystack: Case stud- ies with sparse probing,”arXiv preprint arXiv:2305.01610,

    work page internal anchor Pith review Pith/arXiv arXiv

  35. [35]

    Scaling monosemanticity: Extracting interpretable features from claude 3 sonnet,

    A. Templeton, T. Conerly, J. Marcus, J. Lindsay, T. Bricken, B. Chen, A. Pearce, N. Ley, N. Russell, T. Henighan, and C. Olah, “Scaling monosemanticity: Extracting interpretable features from claude 3 sonnet,”Transformer Circuits Thread, 2024, anthropic. [Online]. Available: https://transformer- circuits.pub/2024/scaling-monosemanticity/ 9, 14

    work page 2024

  36. [36]

    Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2

    T. Lieberum, S. Rajamanoharan, A. Conmy, L. Smith, N. Sonnerat, V . Varma, J. Kram ´ar, A. Dragan, R. Shah, and N. Nanda, “Gemma scope: Open sparse autoencoders everywhere all at once on gemma 2,” 2024. [Online]. Available: https://arxiv.org/abs/2408.05147 10

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  37. [37]

    arXiv preprint arXiv:2503.05613 (2025)

    D. Shu, X. Wu, H. Zhao, D. Rai, Z. Yao, N. Liu, and M. Du, “A survey on sparse autoencoders: Interpreting the internal mechanisms of large language models,” 2025. [Online]. Available: https://arxiv.org/abs/2503.05613 10

    work page arXiv 2025

  38. [38]

    Improving Dictionary Learning with Gated Sparse Autoencoders

    S. Rajamanoharan, A. Conmy, L. Smith, T. Lieberum, V . Varma, J. Kram´ar, R. Shah, and N. Nanda, “Improving dictionary learning with gated sparse autoencoders,” 2024. [Online]. Available: https://arxiv.org/abs/2404.16014 10

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  39. [39]

    Scaling and evaluating sparse autoencoders

    L. Gao, T. D. la Tour, H. Tillman, G. Goh, R. Troll, A. Radford, I. Sutskever, J. Leike, and J. Wu, “Scaling and evaluating sparse autoencoders,” 2024. [Online]. Available: https://arxiv.org/abs/2406.04093 10

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  40. [40]

    BatchTopK Sparse Autoencoders

    B. Bussmann, P. Leask, and N. Nanda, “Batchtopk sparse autoencoders,” 2024. [Online]. Available: https: //arxiv.org/abs/2412.06410 10

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  41. [41]

    Entropy-lens: The information signature of transformer computations,

    R. Ali, F. Caso, C. Irwin, and P. Li `o, “Entropy-lens: The information signature of transformer computations,”arXiv preprint arXiv:2502.16570, 2025. 10

    work page arXiv 2025

  42. [42]

    Activation oracles: Training and evaluating llms as general-purpose activation explainers,

    A. Karvonen, J. Chua, C. Dumas, K. Fraser-Taliente, S. Kantamneni, J. Minder, E. Ong, A. S. Sharma, D. Wen, O. Evans, and S. Marks, “Activation oracles: Training and evaluating llms as general-purpose activation explainers,” Anthropic Alignment Science Blog, 2025, anthropic. [Online]. Available: https://alignment.anthropic.com/2025/ activation-oracles/ 10

    work page 2025

  43. [43]

    Sparse Autoencoders Reveal Interpretable and Steerable Features in VLA Models

    A. Swann, L. McGranahan, H. Buurmeijer, M. K. III, and M. Schwager, “Sparse autoencoders reveal interpretable and steerable features in vla models,” 2026. [Online]. Available: https://arxiv.org/abs/2603.19183 11

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  44. [44]

    Sparse feature circuits: Discovering and editing interpretable causal graphs in language models,

    S. Marks, C. Rager, E. J. Michaud, Y . Belinkov, D. Bau, and A. Mueller, “Sparse feature circuits: Discovering and editing interpretable causal graphs in language models,”

    work page

  45. [45]

    Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models

    [Online]. Available: https://arxiv.org/abs/2403.19647 11

    work page internal anchor Pith review Pith/arXiv arXiv

  46. [46]

    Evaluating explanation faithfulness in toy models,

    C. McDougall, T. Conerly, A. Templeton, J. Lindsey, T. Bricken, B. Chen, A. Pearce, C. Citro, E. Ameisen, A. Jones, H. Cunningham, N. L. Turner, M. MacDiarmid, A. Tamkin, E. Durmus, T. Hume, F. Mosconi, C. D. Freeman, T. R. Sumers, E. Rees, J. Batson, A. Jermyn, S. Carter, C. Olah, and T. Henighan, “Evaluating explanation faithfulness in toy models,”Trans...

    work page 2025

  47. [47]

    Transcoders Find Interpretable LLM Feature Circuits

    J. Dunefsky, P. Chlenski, and N. Nanda, “Transcoders find interpretable llm feature circuits,” 2024. [Online]. Available: https://arxiv.org/abs/2406.11944 11

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  48. [48]

    Circuit insights: Towards inter- pretability beyond activations,

    E. Golimblevskaia, A. Jain, B. Puri, A. Ibrahim, W. Samek, and S. Lapuschkin, “Circuit insights: Towards inter- pretability beyond activations,” 2026. [Online]. Available: https://arxiv.org/abs/2510.14936 11

    work page arXiv 2026

  49. [49]

    Small vectors, big effects: A mechanistic study of rl-induced reasoning via steering vectors,

    V . Sinii, N. Balagansky, G. Gerasimov, D. Laptev, Y . Ak- senov, V . Kurochkin, A. Gorbatovski, B. Shaposhnikov, and 19 D. Gavrilov, “Small vectors, big effects: A mechanistic study of rl-induced reasoning via steering vectors,” 2026. [Online]. Available: https://arxiv.org/abs/2509.06608 13

    work page arXiv 2026

  50. [50]

    Model whisper: Steering vectors unlock large language models’ potential in test- time,

    X. Kang, D. Shi, and L. Chen, “Model whisper: Steering vectors unlock large language models’ potential in test- time,” 2025. [Online]. Available: https://arxiv.org/abs/2512. 04748 13

    work page 2025

  51. [51]

    Toward a Flexible Framework for Linear Representation Hypothesis Using Maximum Likelihood Estimation

    T. Nguyen and Y . Leng, “Toward a flexible framework for linear representation hypothesis using maximum likelihood estimation,” 2025. [Online]. Available: https://arxiv.org/abs/ 2502.16385 13

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  52. [52]

    Representation Engineering for Large-Language Models: Survey and Research Challenges

    L. Bartoszcze, S. Munshi, B. Sukidi, J. Yen, Z. Yang, D. Williams-King, L. Le, K. Asuzu, and C. Maple, “Representation engineering for large-language models: Survey and research challenges,” 2025. [Online]. Available: https://arxiv.org/abs/2502.17601 13, 14

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  53. [53]

    Steering Language Models With Activation Engineering

    A. M. Turner, L. Thiergart, G. Leech, D. Udell, J. J. Vazquez, U. Mini, and M. MacDiarmid, “Steering language models with activation engineering,” 2024. [Online]. Available: https://arxiv.org/abs/2308.10248 13

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  54. [54]

    Steering Llama 2 via Contrastive Activation Addition

    N. Panickssery, N. Gabrieli, J. Schulz, M. Tong, E. Hubinger, and A. M. Turner, “Steering llama 2 via contrastive activation addition,” 2024. [Online]. Available: https: //arxiv.org/abs/2312.06681 13

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  55. [55]

    Emotion concepts and their function in a large language model,

    N. Sofroniew, J. Lindsey, W. Gurnee, E. Ameisen, B. Chen, A. Pearce, N. L. Turner, C. Citro, D. Abrahams, S. Carter, B. Hosmer, J. Marcus, M. Sklar, A. Templeton, T. Bricken, C. McDougall, H. Cunningham, T. Henighan, A. Jermyn, A. Jones, A. Persic, Z. Qi, T. B. Thompson, S. Zimmerman, K. Rivoire, T. Conerly, C. Olah, and J. Batson, “Emotion concepts and t...

    work page 2026

  56. [56]

    Interpretable Steering of Large Language Models with Feature Guided Activation Additions

    S. Soo, C. Guang, W. Teng, C. Balaganesh, T. Guoxian, and Y . Ming, “Interpretable steering of large language models with feature guided activation additions,” 2025. [Online]. Available: https://arxiv.org/abs/2501.09929 14

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  57. [57]

    Dspa: Dynamic sae steering for data-efficient preference alignment,

    J. Wedgwood, A. Muhamed, M. T. Diab, and V . Smith, “Dspa: Dynamic sae steering for data-efficient preference alignment,” 2026. [Online]. Available: https://arxiv.org/abs/ 2603.21461 14

    work page arXiv 2026

  58. [58]

    Fold-se: An efficient rule-based machine learning algorithm with scalable explainability,

    H. Wang and G. Gupta, “Fold-se: An efficient rule-based machine learning algorithm with scalable explainability,”

    work page

  59. [59]

    FOLD-SE: An Efficient Rule-based Machine Learning Algorithm with Scalable Explainability

    [Online]. Available: https://arxiv.org/abs/2208.07912 15

    work page internal anchor Pith review Pith/arXiv arXiv

  60. [60]

    Nesyfold: Neurosym- bolic framework for interpretable image classification,

    P. Padalkar, H. Wang, and G. Gupta, “Nesyfold: Neurosym- bolic framework for interpretable image classification,”

    work page

  61. [61]

    NeSyFOLD: Neurosymbolic Framework for Interpretable Image Classification

    [Online]. Available: https://arxiv.org/abs/2301.12667 15

    work page internal anchor Pith review Pith/arXiv arXiv

  62. [62]

    Theory and Practice of Logic Programming 25(4), 722–738 (2025)

    P. Padalkar and G. Gupta, “Symbolic rule extraction from attention-guided sparse representations in vision transformers,”Theory and Practice of Logic Programming, vol. 25, no. 4, pp. 722–738, Jul. 2025. [Online]. Available: http://dx.doi.org/10.1017/S1471068425100318 15

    work page doi:10.1017/s1471068425100318 2025

  63. [63]

    Logic-lm: Empowering large language models with symbolic solvers for faithful logical reasoning,

    L. Pan, A. Albalak, X. Wang, and W. Y . Wang, “Logic-lm: Empowering large language models with symbolic solvers for faithful logical reasoning,” inFindings of the Association for Computational Linguistics: EMNLP 2023, 2023. 16

    work page 2023

  64. [64]

    Llms can’t plan, but can help planning in llm-modulo frame- works,

    S. Kambhampati, K. Valmeekam, L. Guan, M. Verma, K. Stechly, S. Bhambri, L. Saldyt, and A. Murthy, “Llms can’t plan, but can help planning in llm-modulo frame- works,” inProceedings of the 41st International Conference on Machine Learning (ICML), 2024. 16

    work page 2024

  65. [65]

    Towards Principled Evaluations of Sparse Autoencoders for Interpretability and Control

    A. Makelov, G. Lange, and N. Nanda, “Towards principled evaluations of sparse autoencoders for interpretability and control,” 2024. [Online]. Available: https://arxiv.org/abs/ 2405.08366 17 20

    work page internal anchor Pith review Pith/arXiv arXiv 2024