Pith. sign in

REVIEW 3 major objections 5 minor 55 references

Are Arithmetic Heuristic Neurons Form-Invariant? A Mechanistic Analysis of Symbols, Text, and Code in LLMs

T0 review · 3 major / 5 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The same compact set of neurons performs arithmetic in equations, word problems, and code.

desk verdict A solid causal case for a shared late-layer arithmetic neuron set across formats, but the missing mismatched-donor control in the transfer experiment leaves the 'activation states' conclusion under-supported. read the letter →

arxiv 2607.16693 v1 pith:EARRQGTU submitted 2026-07-18 cs.CL

Sharath Naganna , Tanvir Ahmed Sijan , Uddipta Kalita This is my paper
classification cs.CL
keywords mechanisticinterpretabilityarithmeticheuristicsforminvarianceactivationpatchingMLPneuronscross-formattransferLLMLlama-3models
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether LLMs use the same internal arithmetic machinery for a symbolic equation, a natural-language word problem, and a Python program that express the same computation. It claims that a compact set of MLP neurons, the arithmetic heuristic neurons, is shared across all three formats in three Llama-3 models, and that this shared set is both necessary and sufficient for the arithmetic computation arising in late-layer MLPs. The strongest evidence is cross-format activation transfer: replacing the shared neurons' activations in a failed execution with those from a successful execution in another format corrects most errors, over 97% for addition and subtraction. The authors conclude that cross-format failures mostly reflect different activation states of one shared circuit rather than distinct circuits. If true, a mechanism discovered in one surface form generalizes to others, and format-dependent errors in LLMs may be repairable by aligning activation states.

What carries the argument

The central object is the shared arithmetic-heuristic circuit: the three-way intersection of format-specific sets of sparse MLP neurons that encode interpretable strategies (range, modulo, digit pattern, identical operands). The method is a two-stage discovery pipeline — attribution patching to shortlist candidates, exact activation patching to rank them — followed by keep-only and knockout interventions to test sufficiency and necessity, and by cross-format activation transfer to test whether failures are activation states. Measurements are taken at the final answer token in layers at or after the computation-onset layer, where a linear probe first decodes the answer; the shared set is comp

What would settle it

An experiment that would settle this: hold the shared late-layer MLP activations fixed across formats while varying attention-head or earlier-layer activations, or patch attention alone; if matching attention also recovers many failures, or if keeping shared MLP activations matched still leaves the model failing, then cross-format failures are not purely activation states of the shared MLP circuit.

Watch

Extended reading notes

Core claim

Arithmetic in LLMs is form-invariant at the neuron level: a compact set of late-layer MLP neurons performs the same computation in symbolic equations, word problems, and Python code. These arithmetic heuristic neurons — sparse neurons encoding strategies like range, modulo, digit pattern, or identical operands — are identified per format and intersected across formats. The intersection is necessary and sufficient within late-layer MLPs, and the same neurons carry the same heuristic roles in all three formats. The key evidence is cross-format activation transfer: transferring the shared neurons' activations from a successful run in one format into a failed run in another corrects over 97% of

Load-bearing premise

The conclusion rests on the assumption that the entire arithmetic-relevant signal, including whatever differs between a successful and a failed format, is present in MLP activations at the final token in late layers, so patching only those activations is a complete test of whether the circuits are shared.

Editorial extensions

If this is right

  • If the shared circuit is necessary and sufficient, the arithmetic computation that happens in late-layer MLPs is carried by a few hundred neurons per layer, not by a diffuse whole-network process.
  • Because transferring shared-neuron activations from a successful donor format repairs most cross-format failures, many wrong answers to word problems and code could be corrected by aligning activation states rather than retraining the model.
  • Since shared neurons keep the same heuristic roles across formats, mechanistic explanations built on symbolic arithmetic transfer directly to word problems and code.
  • The same qualitative pattern holds across three Llama-3 sizes, so the form-invariance result is not specific to one model scale.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The authors only patched final-token MLP activations in late layers; a natural extension is to patch attention-head activations or earlier-layer MLP activations across formats. If those also repair failures, the activation-state explanation would need to be widened beyond the shared MLP circuit.
  • The transfer results suggest a practical repair strategy for format failures: steer the shared neurons' activations toward the pattern the model produces on a format it handles better, potentially correcting errors at inference time without retraining.
  • The non-monotonic overlap curve hints at a core-plus-periphery architecture — a stable shared core surrounded by format-specific neurons. A testable extension would check whether low-ranked format-specific neurons are stable across different prompt templates within the same format.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper asks whether arithmetic in LLMs is form-invariant at the level of individual MLP neurons. Using three Llama-3 base models and three prompt formats—symbolic arithmetic, word problems, and Python code—the authors identify per-format top-k arithmetic neurons via attribution patching followed by activation patching, intersect them across formats, and then test the shared set with keep-only and knockout interventions. They also perform cross-format activation transfer from a successful execution in one format to a failed execution in another, and classify the shared neurons into heuristic families. The central claims are that a compact shared circuit is necessary and sufficient for late-layer arithmetic computation, that shared neurons implement the same heuristics across formats, and that cross-format failures are due to activation states of a shared circuit rather than distinct circuits.

Significance. If the conclusions hold, the paper would provide substantial evidence that arithmetic computation in LLMs is form-invariant at the neuron level, and would extend the 'bag of heuristics' framework beyond symbolic arithmetic. The study has real strengths: the causal interventions are direct, the shared-circuit definition is transparent rather than fitted to the conclusion, results are reported on three model scales, and heuristic-family agreement is tested against permutation nulls. The paper also ships code and data. However, the strongest conclusion—that cross-format failures arise from activation states rather than distinct circuits—rests on an experiment whose key control is described but never reported, and the causal claims are explicitly limited to late-layer, final-token MLP neurons. These gaps are fixable but currently leave the central claim under-supported.

major comments (3)
  1. [§4.4/§5.3, Figs. 7–8] The cross-format transfer experiment is the primary evidence for the abstract's claim that failures arise from activation states rather than distinct circuits. Section 4.4 defines a mismatched-donor condition—patching shared-neuron activations from a successful execution with a different operand pair—but this condition is never reported. The only comparisons shown are matched donor versus random neurons. The random control is uninformative for the failure-mechanism question because it patches neurons outside the shared set. Since the shared set includes result-value and result-pattern heuristics (§4.5, Appendix D), a matched donor may simply inject a direct correct-answer-promoting signal. Reporting the mismatched-donor condition is necessary to support the activation-state interpretation.
  2. [§4.3, §4.4, Limitations] All interventions are restricted to MLP neurons at the final token in layers ℓ≥16; attention heads and all earlier MLP layers are left unmodified. Thus the keep-only/knockout results establish necessity and sufficiency only within the late-layer MLP subset, not for the entire arithmetic circuit. The stronger conclusion in §5.3 and the abstract—'cross-format failures arise from activation states rather than distinct circuits'—does not follow, because format-specific routing or earlier-layer processing could differ while a late-layer patch still overshadows the failure. The paper's own Limitations paragraph acknowledges this scope, but the abstract and conclusion do not carry the same qualification. Either add interventions on the earlier/attention components or substantially narrow the claim.
  3. [§4.2, Figs. 4–8] Neuron importance is estimated from only 100 training prompts per (model, format, operator) split, and no error bars, confidence intervals, or rank-stability analyses are provided for the overlap curves or the transfer-recovery curves. Given that the shared circuit is defined by intersection of independently ranked top-k lists, small ranking noise could change the membership of the shared set and the shape of the non-monotonic Jaccard curves. This is particularly relevant for division, where the valid prompt pool is explicitly noted as limited. Bootstrap or split-half stability analyses over the neuron ranking would substantially strengthen the compactness and transfer claims.
minor comments (5)
  1. [Throughout] There are several typos and formatting artifacts, e.g., 'Arithmatic' in the Section 2.2 heading, and spaces inside author names in the bibliography ('Y usser', 'Y oav'). These should be cleaned up.
  2. [Figure 1] The word-problem example contains 'So he has 9 = 9', which is confusing. The figure caption says the answer is always generated as a single token, but the word-problem example appears to show the answer embedded mid-sentence. Clarifying the exact tokenization and final-token alignment would help.
  3. [§4.2, Eq. (4)] The notation in the indirect-effect definition is used before P and P* are clearly defined as clean/corrupt probabilities. Define the superscript convention before Eq. (4).
  4. [Appendix B, Table 2] Table 2 reports 'Mul Correct / 2520' and states the prompt pool is exhausted. It would be helpful to state explicitly that for addition, subtraction, and division the pool is also finite (bounded by operand choices and the [0,999] single-token constraint), and to report whether the 100 training prompts are sampled with or without replacement from the correct-prompt pool.
  5. [§5.3] The text says 'exceeding 97% for addition and subtraction' while multiplication reaches 80–85% and division is more variable. The abstract's phrasing is accurate but could be misread as applying to all operators; a one-sentence reminder in the conclusion would prevent overgeneralization.

Circularity Check

0 steps flagged · score 2.0 of 10

No definitional circularity; shared-circuit claims are causally tested, but the cross-format transfer conclusion lacks the mismatched-donor control needed to exclude answer injection.

full rationale

The paper's main derivation chain is not circular. The shared circuit is defined as the three-way intersection of independently ranked top-k MLP neurons per format, and its causal role is tested with keep-only/knockout mean-ablation interventions against size-matched random controls (Figures 5–6); the intersection being necessary and sufficient is not guaranteed by the per-format ranking alone, so those results carry independent content. Heuristic-family consistency is likewise tested against a permutation null (Appendix E), and no fitted parameter is renamed as a prediction. There is no self-citation chain or imported uniqueness theorem. The principal caveat is the cross-format transfer experiment (§4.4 vs. §5.3): the mismatched-donor condition is specified but never reported, and the random-neuron control cannot exclude the possibility that patching a matched donor's shared-neuron activations—selected by causal effect on the correct answer and including result-value/result-pattern heuristics—simply injects the correct-answer signal. This weakens the activation-state interpretation, but it is an omitted control and an unexcluded alternative, not a definitional circularity; the central shared-circuit and form-invariance findings do not reduce to their inputs by construction. Hence a low score reflecting a load-bearing gap rather than substantive circularity.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

No new physical entities, forces, or latent variables are postulated. The 'shared circuit' is an operational set of existing MLP neurons, not an invented entity. The analysis relies on several procedural choices and domain assumptions; the most consequential is that late-layer final-token MLP intersection captures the mechanism.

free parameters (4)
  • per-layer circuit size k = 50-100 (peak overlap / reported recovery)
    Central quantitative claims (>97% add/sub recovery, peak 3-way overlap) are stated at k≈50-100 neurons/layer; results for larger k are worse (division), and no principled criterion selects k.
  • heuristic matching threshold = 0.6
    Recommended threshold reused from Nikankin et al. (2025), Section 4.5; all heuristic-consistency statistics depend on this cutoff.
  • late-layer cutoff ℓ = ℓ≥16 (Llama-3-8B); ℓ≥14/18 for 1B/3B
    Neuron analysis is restricted to layers after probe accuracy exceeds 50% (§4.1); this choice defines which neurons can be shared and affects all overlap/faithfulness results.
  • attribution candidate / activation rerank counts = top 2000 then top 200 per layer
    Procedural cutoffs that determine the neuron pool in which shared circuits are sought (§4.2).
assumptions (5)
  • standard math MLP as key-value memory with neuron decomposition y = σ(xK)V (Eqs. 1-3)
    Evidence framework adopted from Geva et al. (2021, 2022); unproved background used throughout Sections 3-5.
  • domain assumption Arithmetic computation is localized to the final answer token and late layers
    Section 4.1 concludes from linear probe accuracy that computation onset is late-layer; all causal analysis then restricts to final-token MLP neurons.
  • domain assumption Mean ablation provides a neutral counterfactual baseline for faithfulness
    Section 4.3 uses a mean cache to ablate non-circuit neurons; mean ablation's neutrality is assumed, not validated.
  • domain assumption Correctly answered prompts suffice to identify circuits that also explain failures
    Dataset Construction in Section 4 retains only model-correct pairs for identification; failure analysis then targets donor-correct/target-wrong pairs outside the identification pool.
  • ad hoc to paper Three-way intersection of independently ranked top-k neurons defines the shared circuit
    Section 5.2 defines the shared circuit as the intersection of per-format top-k sets; assumes this intersection is a mechanistic unit rather than an artifact of ranking noise or format-common surface features.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Are Arithmetic Heuristic Neurons Form-Invariant? A Mechanistic Analysis of Symbols, Text, and Code in LLMs." pith.science (2026). https://pith.science/paper/EARRQGTU

@misc{pith2026260716693,
  author       = {Pith},
  title        = {Pith review of: Are Arithmetic Heuristic Neurons Form-Invariant? A Mechanistic Analysis of Symbols, Text, and Code in LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EARRQGTU}},
  note         = {Machine review of arXiv:2607.16693}
}
read the original abstract

Large language models often succeed on one formulation of a problem while failing on an equivalent formulation. Whether these failures arise from distinct internal circuits or different activation states of a shared circuit remains unknown. Recent mechanistic interpretability studies suggest that arithmetic in LLMs emerges from a "bag of heuristics," encoded by a sparse set of MLP neurons that represent distinct arithmetic strategies. We investigate whether arithmetic heuristic neurons are form-invariant across symbolic arithmetic, natural language word problems, and Python code in three Llama-3 models. In each format, we identify arithmetic heuristic neurons using a two-stage pipeline combining attribution patching and activation patching. A compact set of neurons is shared across all three formats, and targeted interventions show this shared circuit is both necessary and sufficient for late-layer arithmetic computation. Transferring the shared neurons' activations from a successful execution in one format to a failed execution in another recovers most incorrect predictions, exceeding 97% for addition and subtraction, indicating that cross-format failures arise from activation states rather than distinct circuits. Moreover, shared neurons consistently belong to the same heuristic families across formats, demonstrating that arithmetic computation in LLMs is largely form-invariant at the neuron level.

Figures

Figures reproduced from arXiv: 2607.16693 by the authors.

Figure 1
Figure 1. Three prompt formats used in our experi [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of the pipeline. (Left) We identify the top-k causal neurons independently for symbolic arithmetic, word problems, and code, computing their layer-wise intersection to isolate the shared circuit. (Middle) We evaluate whether these shared neurons are sufficient and necessary for arithmetic computation via Keep-Only (retaining only shared activations) and Knockout (ablating only shared activations) interventi… view at source ↗
Figure 3
Figure 3. Linear probe accuracy across transformer [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (13 more)
Figure 4
Figure 4. Figure 4: Three-way Jaccard overlap of the top-k neu￾rons across symbolic arithmetic, code, and word prob￾lems as a function of circuit size k. imately 50–100 neurons per layer are retained, be￾fore gradually decreasing as additional neurons are included. This suggests that the …
Figure 5
Figure 5. Figure 5: Sufficiency analysis using keep-only interventions. Faithfulness is measured after ablating all neurons [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Necessity analysis using knockout interventions. Faithfulness is measured after ablating only the shared [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Cross-format activation transfer from code to [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 9
Figure 9. Figure 9: Layer-wise probe accuracy for Llama-3.2- [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]
Figure 10
Figure 10. Figure 10: Layer-wise probe accuracy for Llama-3.2- [PITH_FULL_IMAGE:figures/full_fig_p012_10.png]
Figure 11
Figure 11. Figure 11: Three-way Jaccard overlap of the top-k neurons across arithmetic, code, and word formats for the additional Llama-3 models. Both models exhibit the same non-monotonic overlap pattern observed for Llama-3- 8B, with peak overlap occurring at relatively small circuit siz…
Figure 12
Figure 12. Figure 12: Knockout intervention for Llama-3.2-3B. 10 50 100 500 2000 neurons / layer (k) 0.0 0.2 0.4 0.6 0.8 1.0 faithfulness (keep-only) addition 10 50 100 500 2000 neurons / layer (k) subtraction 10 50 100 500 2000 neurons / layer (k) multiplication 10 50 100 500 2000 neurons…
Figure 13
Figure 13. Figure 13: Keep-only intervention for Llama-3.2-3B. [PITH_FULL_IMAGE:figures/full_fig_p013_13.png]
Figure 14
Figure 14. Figure 14: Keep-only intervention for Llama-3.2-1B. [PITH_FULL_IMAGE:figures/full_fig_p013_14.png]
Figure 15
Figure 15. Figure 15: Knockout intervention for Llama-3.2-1B. 10 20 50 100 200 500 1000 neurons patched per layer (k) 0.0 0.2 0.4 0.6 0.8 1.0 prompts recovered (wrong correct) donor addition subtraction multiplication division matched donor random control (a) Code → Arithmetic 10 20 50 100…
Figure 16
Figure 16. Figure 16: Cross-format activation transfer for Llama-3.2-3B. [PITH_FULL_IMAGE:figures/full_fig_p014_16.png]
Figure 17
Figure 17. Figure 17: Cross-format activation transfer for Llama-3.2-1B. [PITH_FULL_IMAGE:figures/full_fig_p014_17.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

55 extracted references · 5 canonical work pages

  1. [3]

    Stephen Cheng, Sarah Wiegreffe, and Dinesh Manocha. 2026. https://arxiv.org/abs/2604.08524 What drives representation steering? a mechanistic case study on steering refusal . Preprint, arXiv:2604.08524

    arXiv 2026

  2. [6]

    Mavor-Parker, Aengus Lynch, Stefan Heimersheim, and Adri \`a Garriga-Alonso

    Arthur Conmy, Augustine N. Mavor-Parker, Aengus Lynch, Stefan Heimersheim, and Adri \`a Garriga-Alonso. 2023. https://openreview.net/forum?id=89ia77nZ8u Towards automated circuit discovery for mechanistic interpretability . In Thirty-seventh Conference on Neural Information Processing Systems

    2023

  3. [13]

    Rhys Gould, Euan Ong, George Ogden, and Arthur Conmy. 2024. SUCCESSOR HEADS : RECURRING , INTERPRETABLE ATTENTION HEADS IN THE WILD

    2024

  4. [14]

    Michael Hanna, Sandro Pezzelle, and Yonatan Belinkov. 2024. https://openreview.net/forum?id=grXgesr5dT Have faith in faithfulness: Going beyond circuit overlap when finding model mechanisms . In ICML 2024 Workshop on Mechanistic Interpretability

    2024

  5. [18]

    Jack Lindsey, Wes Gurnee, Emmanuel Ameisen, Brian Chen, Adam Pearce, Nicholas L. Turner, Craig Citro, David Abrahams, Shan Carter, Basil Hosmer, Jonathan Marcus, Michael Sklar, Adly Templeton, Trenton Bricken, Callum McDougall, Hoagy Cunningham, Thomas Henighan, Adam Jermyn, Andy Jones, and 8 others. 2025. https://transformer-circuits.pub/2025/attribution...

    2025

  6. [21]

    Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022. Locating and editing factual associations in gpt. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS '22, Red Hook, NY, USA. Curran Associates Inc

    2022

  7. [22]

    Iman Mirzadeh, Keivan Alizadeh, Hooman Shahrokhi, Oncel Tuzel, Samy Bengio, and Mehrdad Farajtabar. 2025. GSM-SYMBOLIC : UNDERSTANDING THE LIMITA- TIONS OF MATHEMATICAL REASONING IN LARGE LANGUAGE MODELS

    2025

  8. [23]

    Neel Nanda. 2023. Attribution patching: Activation patching at industrial scale. https://www.neelnanda.io/mechanistic-interpretability/attribution-patching. Blog post. Accessed: 2026-07-12

    2023

Show all 55 references
  1. [26]

    nostalgebraist. 2020. Interpreting gpt: The logit lens. https://www.lesswrong.com/posts/AcKRB8wDpdaN6v6ru/interpreting-gpt-the-logit-lens. Blog post. Accessed: 2026-07-12

    2020

  2. [28]

    Nikhil Prakash, Tamar Rott Shaham, Tal Haklay, Yonatan Belinkov, and David Bau. 2024. https://openreview.net/forum?id=8sKcAWOf2D Fine-tuning enhances existing mechanisms: A case study on entity tracking . In The Twelfth International Conference on Learning Representations

    2024

  3. [32]

    Kevin Ro Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. 2023. https://openreview.net/forum?id=NpsVSN6o4ul Interpretability in the wild: a circuit for indirect object identification in GPT -2 small . In The Eleventh International Conference on L...

    2023

  4. [35]

    Fred Zhang and Neel Nanda. 2024. https://openreview.net/forum?id=Hf17y6u9BC Towards best practices of activation patching in language models: Metrics and methods . In The Twelfth International Conference on Learning Representations

    2024

  5. [36]

    Hugh Zhang, Jeff Da, Dean Lee, Vaughn Robinson, Catherine Wu, Will Song, Tiffany Zhao, Pranav Raja, Charlotte Zhuang, Dylan Slack, Qin Lyu, Sean Hendryx, Russell Kaplan, Michele (Mike) Lunati, and Summer Yue. 2024 a . A careful examination of large language model performance o...

    2024

  6. [38]

    Ziqian Zhong, Ziming Liu, Max Tegmark, and Jacob Andreas. 2023. https://proceedings.neurips.cc/paper_files/paper/2023/file/56cbfbf49937a0873d451343ddc8c57d-Paper-Conference.pdf The clock and the pizza: Two stories in mechanistic explanation of neural networks . In Advances in ...

    2023

  7. [39]

    Arithmetic

    Nikankin, Yaniv and Reusch, Anja and Mueller, Aaron and Belinkov, Yonatan , year = 2025, month = may, number =. Arithmetic. doi:10.48550/arXiv.2410.21272 , urldate =. arXiv , keywords =:2410.21272 , primaryclass =

    2025 doi arXiv

  8. [40]

    Training

    Cobbe, Karl and Kosaraju, Vineet and Bavarian, Mohammad and Chen, Mark and Jun, Heewoo and Kaiser, Lukasz and Plappert, Matthias and Tworek, Jerry and Hilton, Jacob and Nakano, Reiichiro and Hesse, Christopher and Schulman, John , year = 2021, month = nov, number =. Training. ...

    2021 doi arXiv

  9. [41]

    Measuring

    Hendrycks, Dan and Burns, Collin and Basart, Steven and Zou, Andy and Mazeika, Mantas and Song, Dawn and Steinhardt, Jacob , year = 2020, publisher =. Measuring. doi:10.48550/ARXIV.2009.03300 , urldate =

    2020 doi arXiv

  10. [42]

    Chain-of-

    Wei, Jason and Wang, Xuezhi and Schuurmans, Dale and Bosma, Maarten and Ichter, Brian and Xia, Fei and Chi, Ed and Le, Quoc and Zhou, Denny , year = 2023, month = jan, number =. Chain-of-. doi:10.48550/arXiv.2201.11903 , urldate =. arXiv , keywords =:2201.11903 , primaryclass =

    2023 doi arXiv

  11. [43]

    Stolfo, Alessandro and Belinkov, Yonatan and Sachan, Mrinmaya , editor =. A. Proceedings of the 2023. doi:10.18653/v1/2023.emnlp-main.435 , urldate =

    2023 doi

  12. [44]

    Wu, Zhengxuan and Geiger, Atticus and Arora, Aryaman and Huang, Jing and Wang, Zheng and Goodman, Noah and Manning, Christopher and Potts, Christopher , editor =. Pyvene:. Proceedings of the 2024. doi:10.18653/v1/2024.naacl-demo.16 , urldate =

    2024 doi

  13. [45]

    and Sucholutsky, Ilia , year = 2025, month = feb, number =

    Marjieh, Raja and Veselovsky, Veniamin and Griffiths, Thomas L. and Sucholutsky, Ilia , year = 2025, month = feb, number =. What Is a. doi:10.48550/arXiv.2502.01540 , urldate =. arXiv , keywords =:2502.01540 , primaryclass =

    2025 doi arXiv

  14. [46]

    doi:10.48550/arXiv.2402.19255 , urldate =

    Li, Qintong and Cui, Leyang and Zhao, Xueliang and Kong, Lingpeng and Bi, Wei , year = 2024, month = jul, number =. doi:10.48550/arXiv.2402.19255 , urldate =. arXiv , keywords =:2402.19255 , primaryclass =

    2024 doi arXiv

  15. [47]

    Mirzadeh, Iman and Alizadeh, Keivan and Shahrokhi, Hooman and Tuzel, Oncel and Bengio, Samy and Farajtabar, Mehrdad , year = 2025, langid =

    2025

  16. [48]

    Baeumel, Tanja and Gurgurov, Daniil and al Ghussin, Yusser and van Genabith, Josef and Ostermann, Simon , year = 2025, month = aug, number =. Modular. doi:10.48550/arXiv.2508.02513 , urldate =. arXiv , langid =:2508.02513 , primaryclass =

    2025 doi arXiv

  17. [49]

    Du, Wenyu and Zheng, Rui and Luo, Tongxu and Chung, Stephen and Fu, Jie , editor =. Fine-. Proceedings of the 8th. doi:10.18653/v1/2025.blackboxnlp-1.27 , urldate =

    2025 doi

  18. [50]

    Interpretability in the Wild: a Circuit for Indirect Object Identification in

    Kevin Ro Wang and Alexandre Variengien and Arthur Conmy and Buck Shlegeris and Jacob Steinhardt , booktitle=. Interpretability in the Wild: a Circuit for Indirect Object Identification in. 2023 , url=

    2023

  19. [51]

    The Twelfth International Conference on Learning Representations , year=

    Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity Tracking , author=. The Twelfth International Conference on Learning Representations , year=

  20. [52]

    Thirty-seventh Conference on Neural Information Processing Systems , year=

    Towards Automated Circuit Discovery for Mechanistic Interpretability , author=. Thirty-seventh Conference on Neural Information Processing Systems , year=

  21. [53]

    The Twelfth International Conference on Learning Representations , year=

    Towards Best Practices of Activation Patching in Language Models: Metrics and Methods , author=. The Twelfth International Conference on Learning Representations , year=

  22. [54]

    Attribution Patching Outperforms Automated Circuit Discovery

    Syed, Aaquib and Rager, Can and Conmy, Arthur. Attribution Patching Outperforms Automated Circuit Discovery. Proceedings of the 7th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP. 2024. doi:10.18653/v1/2024.blackboxnlp-1.25

    2024 doi

  23. [55]

    2024 , eprint=

    AtP*: An efficient and scalable method for localizing LLM behaviour to components , author=. 2024 , eprint=

    2024

  24. [56]

    ICML 2024 Workshop on Mechanistic Interpretability , year=

    Have Faith in Faithfulness: Going Beyond Circuit Overlap When Finding Model Mechanisms , author=. ICML 2024 Workshop on Mechanistic Interpretability , year=

    2024

  25. [57]

    2026 , eprint=

    What Drives Representation Steering? A Mechanistic Case Study on Steering Refusal , author=. 2026 , eprint=

    2026

  26. [58]

    Transformer

    Geva, Mor and Schuster, Roei and Berant, Jonathan and Levy, Omer , editor =. Transformer. Proceedings of the 2021. doi:10.18653/v1/2021.emnlp-main.446 , urldate =

    2021 doi

  27. [59]

    Unlocking

    Qiu, Zihan and Huang, Zeyu and Fu, Jie , editor =. Unlocking. Proceedings of the 2024. doi:10.18653/v1/2024.naacl-long.144 , url =

    2024 doi

  28. [60]

    Transformer

    Geva, Mor and Caciularu, Avi and Wang, Kevin and Goldberg, Yoav , editor =. Transformer. Proceedings of the 2022. doi:10.18653/v1/2022.emnlp-main.3 , urldate =

    2022 doi

  29. [61]

    Gould, Rhys and Ong, Euan and Ogden, George and Conmy, Arthur , year = 2024, abstract =

    2024

  30. [62]

    Direct and

    Pearl, Judea , year = 2013, month = jan, number =. Direct and. doi:10.48550/arXiv.1301.2300 , urldate =. arXiv , keywords =:1301.2300 , primaryclass =

    2013 doi arXiv

  31. [63]

    Proceedings of the 36th International Conference on Neural Information Processing Systems , articleno =

    Meng, Kevin and Bau, David and Andonian, Alex and Belinkov, Yonatan , title =. Proceedings of the 36th International Conference on Neural Information Processing Systems , articleno =. 2022 , isbn =

    2022

  32. [64]

    Proceedings of the 38th International Conference on Neural Information Processing Systems , articleno =

    Zhang, Hugh and Da, Jeff and Lee, Dean and Robinson, Vaughn and Wu, Catherine and Song, Will and Zhao, Tiffany and Raja, Pranav and Zhuang, Charlotte and Slack, Dylan and Lyu, Qin and Hendryx, Sean and Kaplan, Russell and Lunati, Michele (Mike) and Yue, Summer , title =. Proce...

    2024

  33. [65]

    Interpretability in the

    Wang, Kevin and Variengien, Alexandre and Conmy, Arthur and Shlegeris, Buck and Steinhardt, Jacob , year = 2022, month = nov, number =. Interpretability in the. doi:10.48550/arXiv.2211.00593 , urldate =. arXiv , keywords =:2211.00593 , primaryclass =

    2022 doi arXiv

  34. [66]

    2023 , month = mar, howpublished =

    Neel Nanda , title =. 2023 , month = mar, howpublished =

    2023

  35. [67]

    Proceedings of the 37th International Conference on Neural Information Processing Systems , articleno =

    Hanna, Michael and Liu, Ollie and Variengien, Alexandre , title =. Proceedings of the 37th International Conference on Neural Information Processing Systems , articleno =. 2023 , publisher =

    2023

  36. [68]

    2020 , month = jul, howpublished =

    nostalgebraist , title =. 2020 , month = jul, howpublished =

    2020

  37. [69]

    Eliciting

    Belrose, Nora and Ostrovsky, Igor and McKinney, Lev and Furman, Zach and Smith, Logan and Halawi, Danny and Biderman, Stella and Steinhardt, Jacob , year = 2025, month = nov, number =. Eliciting. doi:10.48550/arXiv.2303.08112 , urldate =. arXiv , keywords =:2303.08112 , primaryclass =

    2025 doi arXiv

  38. [70]

    doi:10.48550/arXiv.2304.05969 , urldate =

    Localizing. doi:10.48550/arXiv.2304.05969 , urldate =. arXiv , keywords =:2304.05969 , primaryclass =

    doi arXiv

  39. [71]

    Interpreting and

    Zhang, Wei and Wan, Chaoqun and Zhang, Yonggang and Cheung, Yiu-ming and Tian, Xinmei and Shen, Xu and Ye, Jieping , year = 2024, month = sep, number =. Interpreting and. doi:10.48550/arXiv.2409.01659 , urldate =. arXiv , keywords =:2409.01659 , primaryclass =

    2024 doi arXiv

  40. [72]

    Language

    Deng, Chunyuan and Li, Zhiqi and Xie, Roy and Chang, Ruidi and Chen, Hanjie , year = 2024, month = oct, number =. Language. doi:10.48550/arXiv.2410.15580 , urldate =. arXiv , keywords =:2410.15580 , primaryclass =

    2024 doi

  41. [73]

    Lindsey, Jack and Gurnee, Wes and Ameisen, Emmanuel and Chen, Brian and Pearce, Adam and Turner, Nicholas L. and Citro, Craig and Abrahams, David and Carter, Shan and Hosmer, Basil and Marcus, Jonathan and Sklar, Michael and Templeton, Adly and Bricken, Trenton and McDougall, ...

  42. [74]

    Language

    Kantamneni, Subhash and Tegmark, Max , year = 2025, month = feb, number =. Language. doi:10.48550/arXiv.2502.00873 , urldate =. arXiv , keywords =:2502.00873 , primaryclass =

    2025 doi arXiv

  43. [75]

    Cheng, Ziling and Cao, Meng and Pishdad, Leila and Cao, Yanshuai and Cheung, Jackie CK , editor =. Can. Proceedings of the 2025. doi:10.18653/v1/2025.emnlp-main.723 , urldate =

    2025 doi

  44. [76]

    doi:10.48550/arXiv.2301.05217 , urldate =

    Progress Measures for Grokking via Mechanistic Interpretability , author =. doi:10.48550/arXiv.2301.05217 , urldate =. arXiv , keywords =:2301.05217 , primaryclass =

    doi arXiv

  45. [77]

    The Clock and the Pizza: Two Stories in Mechanistic Explanation of Neural Networks , url =

    Zhong, Ziqian and Liu, Ziming and Tegmark, Max and Andreas, Jacob , booktitle =. The Clock and the Pizza: Two Stories in Mechanistic Explanation of Neural Networks , url =

  46. [78]

    and Liu, Ziming and Tegmark, Max , year = 2024, month = may, number =

    Ding, Xiaoman Delores and Guo, Zifan Carl and Michaud, Eric J. and Liu, Ziming and Tegmark, Max , year = 2024, month = may, number =. Survival of the. doi:10.48550/arXiv.2405.17420 , urldate =. arXiv , keywords =:2405.17420 , primaryclass =

    2024 doi arXiv

  47. [79]

    Mamidanna, Siddarth and Rai, Daking and Yao, Ziyu and Zhou, Yilun , editor =. All for. Proceedings of the 2025. doi:10.18653/v1/2025.emnlp-main.1565 , urldate =

    2025 doi

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.