Pith. sign in

REVIEW 1 major objections 2 minor 47 cited by

Localizing Model Behavior with Path Patching

T0 review · 1 major / 2 minor · reviewed 2026-05-16 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Path patching lets researchers test whether a neural network's behavior is localized to a specific set of paths through its components.

desk verdict Path patching gives a quantitative handle on localization claims in transformers, but residual-stream isolation needs explicit checks. read the letter →

arxiv 2304.05969 v2 pith:4A6VNY3D submitted 2023-04-12 cs.LG

Nicholas Goldowsky-Dill , Chris MacLeod , Lucas Sato , Aryaman Arora This is my paper · ORCID
classification cs.LG
keywords pathpatchingmechanisticinterpretabilitycausallocalizationinductionheadstransformercircuitsactivationinterventionsGPT-2analysis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces path patching as a method to express and quantitatively evaluate hypotheses that particular model behaviors arise from interactions along defined paths rather than from the full network. This approach moves beyond qualitative inspection by allowing direct causal interventions that measure how much a hypothesized set of paths contributes to an output. If the method works, it supplies a reproducible way to confirm or refute claims about where computation happens inside transformers and similar models. The authors demonstrate it by refining an existing account of induction heads and by characterizing a behavior in GPT-2. The technique also comes with an open-source implementation to support further experiments of the same kind.

What carries the argument

Path patching, an intervention that swaps activations along a hypothesized set of paths to measure their isolated causal contribution to behavior.

What would settle it

Run path patching on a set of paths hypothesized to produce a specific output and observe whether the output changes exactly as predicted while all other model activations remain untouched.

Watch

Extended reading notes

Core claim

Path patching replaces activations along selected paths while leaving other activations unchanged, thereby isolating the causal effect of those paths on the model's output. This provides a quantitative test for the claim that a given behavior is localized to the chosen paths rather than distributed across the network. The method is used to sharpen the description of induction heads and to examine a concrete behavior in GPT-2, showing that the localization hypotheses can be stated and measured with greater precision than before.

Load-bearing premise

Changing activations only along the chosen paths does not create unintended side effects or interactions that would alter the model's behavior through other routes.

Editorial extensions

If this is right

  • Researchers can state localization hypotheses in terms of explicit paths and obtain numerical evidence for or against them.
  • Existing qualitative accounts of induction heads can be refined by measuring the exact contribution of the relevant paths.
  • The same procedure can be applied to other behaviors in GPT-2 or similar models to produce comparable localization results.
  • An open implementation lowers the cost of running additional path-patching experiments on new hypotheses.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The technique could be extended to compare competing localization hypotheses by pitting their path sets against each other in the same experiment.
  • If path patching proves reliable, it might serve as a building block for automated search over possible localizations rather than manual hypothesis construction.
  • Similar path-based interventions could be tried on models outside the transformer family to check whether the localization pattern holds more generally.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

1 major / 2 minor

Summary. The paper introduces path patching, a technique for expressing and quantitatively testing hypotheses that neural network behaviors are localized to specific sets of paths through components. It applies the method to refine prior explanations of induction heads, characterizes a behavior in GPT-2, and releases an open-source framework for similar experiments.

Significance. If the isolation property holds, path patching would provide a useful quantitative tool for mechanistic interpretability, moving beyond ad-hoc qualitative localization claims. The open-sourced framework is a clear strength that supports reproducibility and extension by others.

major comments (1)
  1. [§3] §3 (Path Patching): The central claim that the intervention isolates causal contributions along hypothesized paths requires that residual-stream interactions with non-path components remain unchanged. No explicit measurement or ablation of cross-path leakage or downstream interference is reported, which is load-bearing for the quantitative evaluation of the induction-head and GPT-2 results.
minor comments (2)
  1. [Abstract] Abstract: The claim that the method 'refines' the induction-head explanation is stated without specifying the concrete change relative to prior work (e.g., what new quantitative evidence is added).
  2. [Experiments] Experiments: Figure captions and tables would benefit from explicit reporting of the exact quantitative metric (e.g., logit difference or accuracy delta) used to assess localization success.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for their constructive feedback. The concern about explicitly verifying the isolation property is substantive, and we have revised the manuscript to include new measurements addressing cross-path leakage and downstream interference.

read point-by-point responses

  1. Referee: [§3] §3 (Path Patching): The central claim that the intervention isolates causal contributions along hypothesized paths requires that residual-stream interactions with non-path components remain unchanged. No explicit measurement or ablation of cross-path leakage or downstream interference is reported, which is load-bearing for the quantitative evaluation of the induction-head and GPT-2 results.

    Authors: We agree that an explicit check on residual-stream interactions strengthens the quantitative claims. Path patching replaces activations only along the hypothesized path while running the remainder of the forward pass on the clean input; by construction this keeps non-path component inputs identical to the clean run except for the direct contributions arriving via the patched path. Nevertheless, to address the referee's point we have added Section 3.4, which reports an ablation measuring L2-norm changes to activations of all non-path components before versus after patching. For the induction-head experiments the median change is below 4% and does not alter the reported effect sizes; analogous results hold for the GPT-2 behavior. We also include a short discussion of why downstream interference is already captured by the path-patching metric itself. These additions are now load-bearing for the revised quantitative claims. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

Path patching introduced as independent experimental technique with no derivation chain

full rationale

The paper presents path patching as a new methodological tool for expressing and testing localization hypotheses in neural networks. No equations, parameters, or results are derived from prior fitted values or self-referential definitions. The abstract and description frame it as an experimental technique applied to induction heads and GPT-2 behaviors, without any load-bearing self-citations, ansatz smuggling, or renaming of known results as derivations. The central claim rests on the validity of the intervention method itself rather than reducing to its own inputs by construction. This is a standard case of a methods paper with self-contained content against external benchmarks.

Assumptions & free parameters 0 free parameters · 1 assumptions · 1 invented entities

The central claim rests on the domain assumption that network behaviors can be meaningfully localized to paths and that patching operations can isolate those paths. No free parameters or invented physical entities are described in the abstract.

assumptions (1)
  • domain assumption Neural network behaviors can be localized to subsets of paths through components
    This is the core hypothesis class the method is designed to test.
invented entities (1)
  • path patching
    purpose: Quantitative test for path-localized behavior hypotheses
    New experimental technique introduced in the paper

how reviews work

0 comments
Cite this review

Pith. "Pith review of Localizing Model Behavior with Path Patching." pith.science (2026). https://pith.science/paper/4A6VNY3D

@misc{pith2026230405969,
  author       = {Pith},
  title        = {Pith review of: Localizing Model Behavior with Path Patching},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4A6VNY3D}},
  note         = {Machine review of arXiv:2304.05969}
}
read the original abstract

Localizing behaviors of neural networks to a subset of the network's components or a subset of interactions between components is a natural first step towards analyzing network mechanisms and possible failure modes. Existing work is often qualitative and ad-hoc, and there is no consensus on the appropriate way to evaluate localization claims. We introduce path patching, a technique for expressing and quantitatively testing a natural class of hypotheses expressing that behaviors are localized to a set of paths. We refine an explanation of induction heads, characterize a behavior of GPT-2, and open source a framework for efficiently running similar experiments.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 47 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. WriteSAE: Sparse Autoencoders for Recurrent State

    cs.LG 2026-05 unverdicted novelty 8.0 of 10

    WriteSAE is the first sparse autoencoder that factors decoder atoms into the native d_k x d_v cache write shape of recurrent models and supplies a closed-form per-token logit shift for atom substitution.

  2. Test, then Route: How Language Models Execute In-Context Conditional Rules Across Models and Languages

    cs.CL 2026-08 conditional novelty 7.0 of 10

    In three open language models across six languages, the predicate test in an in-context conditional is a separable mid-stack circuit, but the learned answer-router direction is token-bound and does not transfer to new...

  3. A Theory of Conditional Collapse under Low-Rank Weight-Space Ablations: I. The Single-Block Theory and Synthetic Validation

    cs.LG 2026-08 conditional novelty 7.0 of 10

    Patching a component moves a decision by its donor-receiver contrast; ablating it moves the decision by its absolute level, and neither quantity bounds the other.

  4. Hidden APIs in Language Models: Discovering Reusable Causal Interfaces from Forked Futures

    cs.AI 2026-07 conditional novelty 7.0 of 10

    A Shared hidden-state interface beats Local, Mixture, and Distributed alternatives under held-out causal description length in Qwen2.5-1.5B and Llama-3-8B, with transplantation and mediation evidence of reuse.

  5. LAWFUL: Law-Aligned Witness for Faithful Use of Latents

    cs.LG 2026-07 conditional novelty 7.0 of 10

    LAWFUL defines coverage-aware physical-consistency scores and circuit tests, reporting that a MoCap-to-Radar transformer's 9-component temporal circuit carries Doppler-law consistency via attention patterns.

  6. A Mechanistic Lens on Semantic Conflicts: Using Activation Patching to Understand LLM Behavior

    cs.SE 2026-07 conditional novelty 7.0 of 10

    Semantic conflicts between code and cues cut execution-grounded correctness; residual-stream patching localizes the causal signal to the edit site, sparse carriers, and readout.

  7. The Curse of Multiple Mediators: Hidden Interaction Effects in Activation Patching

    cs.LG 2026-06 unverdicted novelty 7.0 of 10

    Re-derivation of activation patching NIE reveals it captures interaction effects in addition to direct causal effects, demonstrated via GPT-2 IOI circuit where INT explains component ranking issues and faithfulness in...

  8. Many Circuits, One Mechanism: Input Variation and Evaluation Granularity in Circuit Discovery

    cs.CL 2026-06 unverdicted novelty 7.0 of 10

    Structurally distinct circuits for literal sequence copying across token frequency bands implement the same computation, shown by broad transfer of band-specific edges, a shared core recovering 99% performance, and in...

  9. Function-Vector Heads Are Two Populations: Writers and Cancellers in In-Context Learning

    cs.CL 2026-05 unverdicted novelty 7.0 of 10

    The function-vector head population splits into sign-opposed writers and cancellers, so magnitude-only ranking averages a promoting and a suppressing mechanism.

  10. Transformer Field Theory: A Response-Theoretic Approach to Mechanistic Interpretability

    cs.LG 2026-05 unverdicted novelty 7.0 of 10

    Transformer Field Theory frames the residual stream as a field, models patching as source insertion, and uses first-order sensitivities plus Green functions to predict and describe responses, with empirical tests on G...

  11. Polymorphism Is Rotation: Operational Mechanistic Interpretability from a Two-Layer Transformer to Pythia-70m

    cs.LG 2026-05 unverdicted novelty 7.0 of 10

    Transformers trained from different random seeds exhibit residual-stream polymorphism that is exactly a uniform random rotation, which a Procrustes alignment removes to transfer SAEs and steering vectors.

  12. From Circuit Evidence to Mechanistic Theory: An Inductive Logic Approach

    cs.LG 2026-05 unverdicted novelty 7.0 of 10

    Introduces Causal Functional Signatures grounded in causal evidence and ILP-learned architectural signatures to enable explicit, comparable, and portable mechanistic claims across model scales.

  13. Where Does Authorship Signal Emerge in Encoder-Based Language Models?

    cs.CL 2026-05 unverdicted novelty 7.0 of 10

    Scoring mechanism determines the layer at which encoder-based models consolidate authorship signals, with mean pooling acting early and late interaction deferring to later layers.

  14. Language-Switching Triggers Take a Latent Detour Through Language Models

    cs.CL 2026-05 unverdicted novelty 7.0 of 10

    Researchers identify and decompose a language-switching backdoor circuit in an autoregressive LM into early attention composition, mid-layer orthogonal propagation, and final MLP conversion.

  15. Repeated-Token Counting Reveals a Dissociation Between Representations and Outputs

    cs.CL 2026-05 unverdicted novelty 7.0 of 10

    LLMs encode repeated token counts correctly in residual streams but a format-triggered MLP at 88-93% depth overwrites it with an incorrect fixed value.

  16. Can LLMs Reason Abstractly Over Math Word Problems Without CoT? Disentangling Abstract Formulation From Arithmetic Computation

    cs.CL 2025-05 conditional novelty 7.0 of 10

    A disentangled evaluation of math word problems finds arithmetic computation, not abstract formulation, is the main bottleneck for LLMs without chain-of-thought, and chain-of-thought mainly helps computation.

  17. Grounding latent algorithm routing in transformer reasoning

    cs.CL 2026-07 conditional novelty 6.5 of 10

    On ROUTEBENCH, 306M–612M dense transformers close most of the oracle routing gap with decodable, nuisance-stable, intervention-sensitive solver-family preferences.

  18. Induction in Both Directions: A Mechanistic Analysis of In-Context Learning in Masked Diffusion Language Models

    cs.CL 2026-07 conditional novelty 6.5 of 10

    Masked diffusion language models implement a bidirectional induction circuit with previous-token and next-token heads, and they compute the global mask fraction as an implicit timestep.

  19. Temporal Preference Concepts and their Functions in a Large Language Model

    cs.LG 2026-05 unverdicted novelty 6.5 of 10

    Temporal preference in Qwen3-4B-Instruct-2507 localizes to layers 17–35 (especially L24 attention), has curved residual-stream geometry, is behaviorally unstable, and can be bidirectionally steered.

  20. A Mechanistic Analysis of Gender Sensitivity in Dense Retrieval Models

    cs.IR 2026-08 conditional novelty 6.0 of 10

    Gender sensitivity in dense retrieval models originates in input embeddings and is carried by a small set of late-layer attention heads that jointly encode gender and term-matching signals.

  21. Persistent Sparse Autoencoders: Learning Feature Timescales in Language Models

    cs.LG 2026-07 conditional novelty 6.0 of 10

    Persistent SAEs learn per-feature persistence coefficients from reconstruction, splitting features into fast local detectors and slow topic-tracking states that retain prompt-injection signals over long contexts.

  22. Are Arithmetic Heuristic Neurons Form-Invariant? A Mechanistic Analysis of Symbols, Text, and Code in LLMs

    cs.CL 2026-07 conditional novelty 6.0 of 10

    Llama-3 arithmetic is computed by a shared neuron set across symbolic, text, and code formats, and cross-format failures are activation-state differences, not distinct circuits.

  23. Certified Interventional Fidelity: Anytime-Valid, Adaptive Evaluation of Causal Claims in Mechanistic Interpretability

    cs.LG 2026-07 accept novelty 6.0 of 10

    CIF certifies interventional interpretability metrics as bounded causal means with anytime-valid confidence sequences, including under adaptive sampling, cutting certification cost 10–30× with betting sequences.

  24. Surrogate Fidelity: When Can Open LLMs Explain Closed Ones?

    cs.LG 2026-06 unverdicted novelty 6.0 of 10

    Prediction agreement between open and closed LLMs substantially overstates agreement on attributions and causal reasons.

  25. ScAle: Attention Head Scaling as a Minimal Adapter for Spatial Reasoning in Vision Language Models

    cs.CV 2026-06 unverdicted novelty 6.0 of 10

    ScAle learns scalar coefficients to modulate last-token attention and MLP activations in frozen VLMs, achieving up to 134.1% relative accuracy gains on spatial benchmarks with only 1K parameters.

  26. Where Do Models Find Happiness? Emotion Vectors in Open-Source LLMs

    cs.CL 2026-06 unverdicted novelty 6.0 of 10

    Valence is geometrically encoded in Apertus-8B and Gemma-4-E4B with PC1 correlations of 0.76 and 0.83, but emerges at different depths than in Claude and arousal alignment varies by generated corpus.

  27. Factual Retrieval in LLMs Is a Redundant, Distributed and Non-Contiguous Process

    cs.CL 2026-06 unverdicted novelty 6.0 of 10

    Attribute retrieval in LLMs follows non-contiguous, redundant layer paths identified via iterative patching, implying highly distributed knowledge storage.

  28. Localizing Anchoring Pathways in Language Models

    cs.CL 2026-06 unverdicted novelty 6.0 of 10

    Attribution methods localize anchoring signals in Qwen and Llama models; edge-level circuits transfer within a model but show sparse transfer from base to instruction-tuned variants.

  29. One Lens, Many Worlds : A Capability-Typed Interface for World-Model Interpretability

    cs.LG 2026-06 unverdicted novelty 6.0 of 10

    WorldModelLens defines a typed adapter with four core methods and a capability descriptor to unify interpretability tooling across diverse world model architectures.

  30. Shared Semantics, Divergent Mechanisms: Unsupervised Feature Discovery by Aligning Semantics and Mechanisms

    cs.CL 2026-06 unverdicted novelty 6.0 of 10

    Proposes distribution-level unsupervised feature discovery for LLMs by clustering continuations on semantic content and mechanistic attributions without target outputs.

  31. Do Language Models Track Entities Across State Changes?

    cs.CL 2026-05 unverdicted novelty 6.0 of 10

    LMs solve entity tracking with state changes by parallel aggregation at the query token instead of incremental tracking, with REMOVE using a global suppression tag.

  32. Represented Is Not Computed: A Causal Test of Candidate Algorithmic Intermediates in a Transformer

    cs.LG 2026-05 unverdicted novelty 6.0 of 10

    Transformer represents but does not causally transmit staged algorithmic intermediates for base-digit extraction, diverging from probe predictions.

  33. Routing Subspaces: Auditing Evaluation-to-Deployment Mismatch in Fine-Tuned Language Models

    cs.CL 2026-05 conditional novelty 6.0 of 10

    Removing a single fitted activation direction at a mid-depth layer reduces the evaluation-vs-deployment behavioral gap on held-out prompts in 10 of 12 fine-tuned LLM settings, with matched controls staying flat.

  34. Certified Circuits: Stability Guarantees for Mechanistic Circuits

    cs.AI 2026-02 conditional novelty 6.0 of 10

    Certified Circuits uses deletion-based randomized smoothing to guarantee that circuit components stay included or excluded under bounded edits to the concept dataset, yielding more compact and more accurate circuits.

  35. All for One: LLMs Solve Mental Math at the Last Token With Information Transferred From Other Tokens

    cs.CL 2025-09 conditional novelty 6.0 of 10

    LLMs solve arithmetic in-context via an All-for-One pattern, with all input-specific computation occurring at the last token after a two-layer information transfer window.

  36. Dissecting Bias in LLMs: A Mechanistic Interpretability Perspective

    cs.CL 2025-06 conditional novelty 6.0 of 10

    Bias in GPT-2 and Llama-2 is localized to a small set of edges, and ablation of those edges reduces bias while impairing unrelated NLP tasks.

  37. Language Models Use Trigonometry to Do Addition

    cs.AI 2025-02 conditional novelty 6.0 of 10

    Three LLMs represent two-digit numbers as generalized helices and appear to compute addition by combining these helices into an answer helix.

  38. Circuit Claims Depend on What Is Extracted and How It Is Compared

    cs.LG 2026-07 conditional novelty 5.0 of 10

    On a synthetic Lean tactic-prediction task, exact circuit edge lists barely overlap across dense and sparse checkpoints while attention-head sets and size rankings do, so a circuit claim is well defined only once grap...

  39. Reading Task Failure Off the Activations: A Sparse-Feature Audit of GPT-2 Small on Indirect Object Identification

    cs.LG 2026-05 unverdicted novelty 5.0 of 10

    An empirical audit identifies a strong SAE feature correlate for GPT-2 small failures on 'keys' prompts in the IOI task, performs ablation and baseline controls showing it is not causal, and presents the audit pipelin...

  40. Rare Events, Real Signals: Functional Ensembles as Units of Computation in Deep Spiking Networks

    cs.NE 2026-05 unverdicted novelty 5.0 of 10

    In spiking ResNets, 1FC ensembles defined by pairwise correlations show ReLU-like cofiring-to-response mapping whose gain scales with ensemble size, with reliable class encoding restricted to infrequent high-cofiring events.

  41. Identifying a Circuit for Verb Conjugation in GPT-2

    cs.CL 2025-06 conditional novelty 5.0 of 10

    A candidate 12-head circuit for subject-verb agreement in GPT-2 Small is identified, but it generalizes poorly to more complex agreement conditions.

  42. How to use and interpret activation patching

    cs.LG 2024-04 accept novelty 5.0 of 10

    Activation patching provides evidence about neural network circuits when the choice of metric is aligned with the hypothesis and common interpretation errors are avoided.

  43. Towards Best Practices of Activation Patching in Language Models: Metrics and Methods

    cs.LG 2023-09 unverdicted novelty 5.0 of 10

    Varying evaluation metrics and corruption methods in activation patching produces different localization and circuit discovery outcomes in language models, leading to recommendations for preferred practices.

  44. Explainability of Large Language Models: Opportunities and Challenges toward Generating Trustworthy Explanations

    cs.CL 2025-10 conditional novelty 4.0 of 10

    LLM explanations split into local and mechanistic tracks; the paper argues they are trustworthy only if they pass causal and contrastive stress tests, adapt to the explainee, and satisfy eight trust principles.

  45. NEAT: Concept driven Neuron Attribution in LLMs

    cs.CL 2025-08 reject novelty 4.0 of 10

    NEAT identifies concept neurons by feeding a single mean hidden-state vector through the model and ranking neurons by their effect on concept-word probabilities.

  46. Unraveling Token Prediction Refinement and Identifying Essential Layers in Language Models

    cs.CL 2025-01 conditional novelty 4.0 of 10

    In GPT-2 multi-document QA, the layer gap between the first correct top-1 token prediction and its stable final form is larger when relevant information is in the middle of the context.

  47. High-Dimensional Statistics: Reflections on Progress and Open Problems

    math.ST 2026-05 unverdicted novelty 2.0 of 10

    This review synthesizes representative advances in high-dimensional statistics, highlights common themes and open problems, and points to key entry works.

Reference graph

Works this paper leans on

48 extracted references · 48 canonical work pages · cited by 47 Pith papers

  1. [1]

    2022 , eprint=

    Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small , author=. 2022 , eprint=

    work page 2022

  2. [2]

    2023 , archivePrefix=

    Towards Automated Circuit Discovery for Mechanistic Interpretability , author=. 2023 , archivePrefix=

    work page 2023

  3. [3]

    ArXiv , year=

    In-context Learning and Induction Heads , author=. ArXiv , year=

    work page

  4. [4]

    Distill , volume=

    Zoom in: An introduction to circuits , author=. Distill , volume=

    work page

  5. [5]

    2023 , eprint=

    A Toy Model of Universality: Reverse Engineering How Networks Learn Group Operations , author=. 2023 , eprint=

    work page 2023

  6. [6]

    Advances in Neural Information Processing Systems , volume=

    Causal abstractions of neural networks , author=. Advances in Neural Information Processing Systems , volume=

    work page

  7. [8]

    2022 , journal=

    Softmax Linear Units , author=. 2022 , journal=

    work page 2022

  8. [9]

    Proceedings of the 39th International Conference on Machine Learning , pages =

    Inducing Causal Structure for Interpretable Neural Networks , author =. Proceedings of the 39th International Conference on Machine Learning , pages =. 2022 , editor =

    work page 2022

Show all 48 references
  1. [12]

    Advances in neural information processing systems , volume=

    Investigating gender bias in language models using causal mediation analysis , author=. Advances in neural information processing systems , volume=

  2. [13]

    Advances in Neural Information Processing Systems , volume=

    Locating and editing factual associations in gpt , author=. Advances in Neural Information Processing Systems , volume=

  3. [15]

    Uncertainty in Artificial Intelligence , pages=

    Approximate causal abstractions , author=. Uncertainty in Artificial Intelligence , pages=. 2020 , organization=

    2020

  4. [16]

    Advances in neural information processing systems , volume=

    Residual networks behave like ensembles of relatively shallow networks , author=. Advances in neural information processing systems , volume=

  5. [17]

    2021 , journal=

    A Mathematical Framework for Transformer Circuits , author=. 2021 , journal=

    2021

  6. [19]

    Advances in neural information processing systems , volume=

    This looks like that: deep learning for interpretable image recognition , author=. Advances in neural information processing systems , volume=

  7. [20]

    2022 , eprint=

    Scale Efficiently: Insights from Pre-training and Fine-tuning Transformers , author=. 2022 , eprint=

    2022

  8. [21]

    2023 , eprint=

    Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks , author=. 2023 , eprint=

    2023

  9. [22]

    2023 , eprint=

    Tracr: Compiled Transformers as a Laboratory for Interpretability , author=. 2023 , eprint=

    2023

  10. [23]

    2023 , eprint=

    Eliciting Latent Predictions from Transformers with the Tuned Lens , author=. 2023 , eprint=

    2023

  11. [24]

    , author=

    Causal scrubbing: a method for rigorously testing interpretability hypotheses. , author=. 2022 , url=

    2022

  12. [26]

    2023 , originalyear =

    Scheurer, Jérémy Scheurer and Phil3 and tony and Thibodeau, Jacques and Lindner, David , title =. 2023 , originalyear =

    2023

  13. [27]

    Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence,

    Recent Advances in Adversarial Training for Adversarial Robustness , author =. Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence,. 2021 , month =. doi:10.24963/ijcai.2021/591 , url =

    2021 doi

  14. [29]

    Approximate causal abstractions

    Sander Beckers, Frederick Eberhardt, and Joseph Y Halpern. Approximate causal abstractions. In Uncertainty in Artificial Intelligence, pp.\ 606--615. PMLR, 2020

    2020

  15. [30]

    Eliciting latent predictions from transformers with the tuned lens, 2023

    Nora Belrose, Zach Furman, Logan Smith, Danny Halawi, Igor Ostrovsky, Lev McKinney, Stella Biderman, and Jacob Steinhardt. Eliciting latent predictions from transformers with the tuned lens, 2023

    2023

  16. [31]

    An interpretability illusion for bert

    Tolga Bolukbasi, Adam Pearce, Ann Yuan, Andy Coenen, Emily Reif, Fernanda Vi \'e gas, and Martin Wattenberg. An interpretability illusion for bert. arXiv preprint arXiv:2104.07143, 2021

    2021

  17. [32]

    Causal scrubbing: a method for rigorously testing interpretability hypotheses., 2022

    Lawrence Chan, Adrià Garriga-Alonso, Nicholas Goldowsky-Dill, Ryan Greenblatt, Jenny Nitishinskaya, Ansh Radhakrishnan, Buck Shlegeris, and Nate Thomas. Causal scrubbing: a method for rigorously testing interpretability hypotheses., 2022. URL https://bit.ly/3WRBhPD

    2022

  18. [33]

    This looks like that: deep learning for interpretable image recognition

    Chaofan Chen, Oscar Li, Daniel Tao, Alina Barnett, Cynthia Rudin, and Jonathan K Su. This looks like that: deep learning for interpretable image recognition. Advances in neural information processing systems, 32, 2019

    2019

  19. [34]

    A toy model of universality: Reverse engineering how networks learn group operations, 2023

    Bilal Chughtai, Lawrence Chan, and Neel Nanda. A toy model of universality: Reverse engineering how networks learn group operations, 2023

    2023

  20. [35]

    Mavor-Parker, Aengus Lynch, Stefan Heimersheim, and Adrià Garriga-Alonso

    Arthur Conmy, Augustine N. Mavor-Parker, Aengus Lynch, Stefan Heimersheim, and Adrià Garriga-Alonso. Towards automated circuit discovery for mechanistic interpretability, 2023

    2023

  21. [36]

    A mathematical framework for transformer circuits

    Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dari...

    2021

  22. [37]

    Softmax linear units

    Nelson Elhage, Tristan Hume, Catherine Olsson, Neel Nanda, Tom Henighan, Scott Johnston, Sheer ElShowk, Nicholas Joseph, Nova DasSarma, Ben Mann, Danny Hernandez, Amanda Askell, Kamal Ndousse, Andy Jones, Dawn Drain, Anna Chen, Yuntao Bai, Deep Ganguli, Liane Lovitt, Zac Hatfi...

    2022

  23. [38]

    Causal analysis of syntactic agreement mechanisms in neural language models

    Matthew Finlayson, Aaron Mueller, Sebastian Gehrmann, Stuart Shieber, Tal Linzen, and Yonatan Belinkov. Causal analysis of syntactic agreement mechanisms in neural language models. arXiv preprint arXiv:2106.06087, 2021

    2021

  24. [39]

    Neural natural language inference models partially embed theories of lexical entailment and negation

    Atticus Geiger, Kyle Richardson, and Christopher Potts. Neural natural language inference models partially embed theories of lexical entailment and negation. arXiv preprint arXiv:2004.14623, 2020

    2004

  25. [40]

    Causal abstractions of neural networks

    Atticus Geiger, Hanson Lu, Thomas Icard, and Christopher Potts. Causal abstractions of neural networks. Advances in Neural Information Processing Systems, 34: 0 9574--9586, 2021

    2021

  26. [41]

    Inducing causal structure for interpretable neural networks

    Atticus Geiger, Zhengxuan Wu, Hanson Lu, Josh Rozner, Elisa Kreiss, Thomas Icard, Noah Goodman, and Christopher Potts. Inducing causal structure for interpretable neural networks. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (e...

    2022

  27. [42]

    Causal abstraction for faithful model interpretation

    Atticus Geiger, Chris Potts, and Thomas Icard. Causal abstraction for faithful model interpretation. arXiv preprint arXiv:2301.04709, 2023 a

    2023

  28. [43]

    Finding alignments between interpretable causal variables and distributed neural representations

    Atticus Geiger, Zhengxuan Wu, Christopher Potts, Thomas Icard, and Noah D Goodman. Finding alignments between interpretable causal variables and distributed neural representations. arXiv preprint arXiv:2303.02536, 2023 b

    2023

  29. [44]

    Tracr: Compiled transformers as a laboratory for interpretability, 2023

    David Lindner, János Kramár, Matthew Rahtz, Thomas McGrath, and Vladimir Mikulik. Tracr: Compiled transformers as a laboratory for interpretability, 2023

    2023

  30. [45]

    Locating and editing factual associations in gpt

    Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in gpt. Advances in Neural Information Processing Systems, 35: 0 17359--17372, 2022

    2022

  31. [46]

    Zoom in: An introduction to circuits

    Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter. Zoom in: An introduction to circuits. Distill, 5 0 (3): 0 e00024--001, 2020

    2020

  32. [47]

    Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, T. J. Henighan, Benjamin Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Scott Johnston, Andy Jones, John Kernion, Liane Lovitt,...

    2022 arXiv

  33. [48]

    Direct and indirect effects

    Judea Pearl. Direct and indirect effects. CoRR, abs/1301.2300, 2013. URL http://arxiv.org/abs/1301.2300

    2013 arXiv

  34. [49]

    Shortformer: Better language modeling using shorter inputs

    Ofir Press, Noah A Smith, and Mike Lewis. Shortformer: Better language modeling using shorter inputs. arXiv preprint arXiv:2012.15832, 2020

    2012

  35. [50]

    Toward transparent ai: A survey on interpreting the inner structures of deep neural networks, 2023

    Tilman Räuker, Anson Ho, Stephen Casper, and Dylan Hadfield-Menell. Toward transparent ai: A survey on interpreting the inner structures of deep neural networks, 2023

    2023

  36. [51]

    Practical pitfalls of causal scrubbing, 2023

    Jérémy Scheurer Scheurer, Phil3, tony, Jacques Thibodeau, and David Lindner. Practical pitfalls of causal scrubbing, 2023. URL https://www.alignmentforum.org/posts/DFarDnQjMnjsKvW8s/practical-pitfalls-of-causal-scrubbing

    2023

  37. [52]

    Scale efficiently: Insights from pre-training and fine-tuning transformers, 2022

    Yi Tay, Mostafa Dehghani, Jinfeng Rao, William Fedus, Samira Abnar, Hyung Won Chung, Sharan Narang, Dani Yogatama, Ashish Vaswani, and Donald Metzler. Scale efficiently: Insights from pre-training and fine-tuning transformers, 2022

    2022

  38. [53]

    Residual networks behave like ensembles of relatively shallow networks

    Andreas Veit, Michael J Wilber, and Serge Belongie. Residual networks behave like ensembles of relatively shallow networks. Advances in neural information processing systems, 29, 2016

    2016

  39. [54]

    Investigating gender bias in language models using causal mediation analysis

    Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart Shieber. Investigating gender bias in language models using causal mediation analysis. Advances in neural information processing systems, 33: 0 12388--12401, 2020

    2020

  40. [55]

    Interpretability in the wild: a circuit for indirect object identification in gpt-2 small, 2022

    Kevin Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. Interpretability in the wild: a circuit for indirect object identification in gpt-2 small, 2022

    2022

Pith tools

Reviewed May 16, 2026 · model on record in the stance chip above.