Pith. sign in

REVIEW 3 major objections 5 minor 4 cited by

EAP-GP: Mitigating Saturation Effect in Gradient-based Automated Circuit Identification

T0 review · 3 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Gradient-based circuit discovery can be made more faithful by replacing straight-line integration paths with an adaptive, model-dependent path that avoids saturated regions.

desk verdict A useful saturation diagnosis and a plausible adaptive-path fix, but the central equation is not a valid path integral and the empirical claims outrun the tables; worth a serious referee, not a desk reject. read the letter →

arxiv 2502.06852 v1 pith:HTIEKPFQ submitted 2025-02-07 cs.LG cs.AI

Lin Zhang , Wenshuo Dong , Zhuoran Zhang , Shu Yang , Lijie Hu , Ninghao Liu , Pan Zhou , Di Wang This is my paper · ORCID
classification cs.LGcs.AI
keywords mechanisticinterpretabilitycircuitdiscoveryedgeattributionpatchingintegratedgradientssaturationeffecttransformerGPT-2faithfulnessmetrics
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that existing gradient-based circuit discovery methods, EAP and EAP-IG, produce unreliable edge attributions because their integration paths cross regions where the loss gradient is nearly zero, a problem the authors call the saturation effect. It proposes EAP-GP, which builds a path from clean to corrupted activations by stepping along the gradient of the difference between model outputs, thereby avoiding such saturated regions. Across six tasks and three GPT-2 sizes, the paper reports that circuits found by EAP-GP are more faithful under corruption than those from EAP and EAP-IG, with gains up to 17.7%. If correct, this gives practitioners a gradient-only way to find more trustworthy subgraph explanations of transformer behavior.

What carries the argument

GradPath, the adaptive integration path defined by Eq (8): starting at the clean activation $x_u$, each step moves in the direction of steepest descent of $\|G(\gamma) - G(x'_u)\|_2^2$ with step size normalized by the gradient norm. It does the work of replacing EAP-IG's model-agnostic straight line with a model-dependent curve that is supposed to stay out of low-gradient regions, and it is the only new mechanism in the method; the rest of EAP-GP follows the EAP-IG attribution and circuit-extraction pipeline.

What would settle it

Run Algorithm 1 for a fixed edge and record $\gamma_G(1)$ after $k$ steps; if $\gamma_G(1) \neq x'_u$ for typical edges, Eq (9) is not integrating along a path from $x_u$ to $x'_u$ and the claimed avoidance of saturation is not established. A second check: compare the faithfulness of circuits found by a variant that reparameterizes the path to force $\gamma_G(k)=x'_u$; if faithfulness drops or is unchanged, the advantage of EAP-GP is not due to path termination.

Watch

Extended reading notes

Core claim

The central claim is that saturation, not just a zero gradient at a single point, is what limits EAP-IG, and that replacing its fixed straight-line path with a model-dependent GradPath removes that limitation. EAP-GP scores an edge (u,v) by $(x_u - x'_u) \times \frac{1}{k}\sum_{j=1}^k \frac{\partial L(\gamma_G(j/k))}{\partial x_v}$, where $\gamma_G$ is built by iteratively moving from $x_u$ toward $x'_u$ along normalized gradients of $\|G(\gamma) - G(x'_u)\|_2^2$. The paper claims this path keeps gradients large enough that attributions stay responsive to activation changes, yielding better Normalized Faithfulness Scores at matched sparsity and precision/recall comparable to or better than prior methods against hand-annotated circuits.

Load-bearing premise

The method assumes that the adaptive path generated by Eq (8), starting at $x_u$, actually ends at the corrupted activation $x'_u$ and that Eq (9)'s product of total activation difference with average gradients along this non-straight path is a valid estimate of the loss change; if the path drifts away or fails to terminate at $x'_u$, the attribution scores are biased in an uncontrolled way.

Editorial extensions

If this is right

  • At matched sparsity, EAP-GP reports higher Normalized Faithfulness than EAP and EAP-IG on IOI, SVA, Hypernymy, Capital–Country, Gender-Bias, and Greater-Than on GPT-2 Small, and the advantage persists on GPT-2 Medium and XL.
  • Gradient-based circuit discovery can be made model-dependent without adding supervised labels, keeping the efficiency spirit of EAP while improving faithfulness.
  • The step count k is the main hyperparameter; the paper reports that k = 4 or 5 already gives high faithfulness on IOI and Gender-Bias.
  • EAP-GP is about five times slower than EAP-IG at k = 5 because path construction adds forward and backward passes over the activations.
  • Because the path is built per edge and per clean–corrupted pair, the method is directly applicable to any transformer whose activations can be patched, not just GPT-2.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the path-termination issue is real, a corrected version that forces $\gamma_G(1) = x'_u$ (for example, by a final linear interpolation step or an ODE solver) might preserve or improve the faithfulness gains and would cleanly separate the benefit of adaptive paths from the benefit of longer interpolation.
  • The saturation diagnosis suggests a general recipe for other attribution methods: whenever integrated gradients are used, the path should be chosen to maximize some measure of gradient magnitude, which could extend beyond circuit discovery to input-feature attribution for large language models.
  • Because the path depends on the model and the specific clean–corrupted pair, EAP-GP attributions may transfer less across similar prompts than straight-line IG; measuring stability across corruption samples would be a useful stress test.
  • The reported runtime overhead suggests a practical tradeoff: for very large models, the extra forward/backward passes per edge may make the method costly, so a cheaper approximation of GradPath (for example, shared path construction across edges) is a natural testable extension.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes EAP-GP, a gradient-based circuit identification method that replaces EAP-IG's straight-line interpolation path with an adaptive 'GradPath' obtained by normalized gradient descent on the squared distance between the current model output and the corrupted-activation model output. Edge scores are computed as the total activation difference times the average loss gradient along this path. The method is evaluated on six tasks with GPT-2 Small, Medium, and XL, reporting normalized faithfulness scores at several sparsity levels, precision-recall against manual IOI and Greater-Than circuits, and an ablation over the number of path steps. The abstract claims improvements up to 17.7% over EAP and EAP-IG.

Significance. If the central mechanism were established, the paper would make a useful contribution: an adaptive integration path for edge attribution is a natural extension of EAP-IG, and the breadth of the evaluation (six tasks, three model sizes, multiple sparsity levels, manual-circuit PR curves) is a genuine strength. The NFS gains on IOI, SVA, and Hypernymy are large in several regimes, and the k-ablation is informative. I also note that the evaluation is not circular: faithfulness is measured by intervention on the model, and precision/recall use externally annotated circuits. However, the method's core score is not derived as a path integral along the constructed path, and the path's connection to the saturation mechanism is asserted rather than shown. The empirical claims are also stronger than the tables support. Because these issues concern the paper's central claim, the current version needs substantial revision rather than minor polishing.

major comments (3)
  1. [Section 5, Eq (9) and Algorithm 1] For the non-straight path gammaG, the discrete path integral of Eq (1) would be sum_{j=0}^{k-1} grad L(gammaG(j/k)) . (gammaG((j+1)/k) - gammaG(j/k)) up to parameterization, not (x_u - x'_u) times the average gradient. The equality with (x_u - x'_u) times average gradient holds only for a straight line with gamma'(alpha) proportional to (x_u - x'_u). The paper neither derives the correct weighting for GradPath nor shows that the missing path-derivative term is negligible. In addition, because every update in Algorithm 1 is normalized by W_j and k is fixed, gammaG(1) is not guaranteed to equal x'_u and in fact will generally not. Thus Eq (9) is not an integrated gradient along the advertised path. Please provide the correct discrete path-integral formula with endpoint control, or explicitly characterize Eq (9) as a heuristic and give a separate justification for why it improves faithfulness.
  2. [Section 5, Eqs (7)-(8), and Section 4, Definition 4.1] The path is constructed by minimizing the squared output distance ||G(gammaG(j/k)) - G(x'_u)||^2, whereas saturation is defined in Definition 4.1 through near-zero values of the loss gradient dL/dx_v. No result connects these two quantities. For tasks whose loss is a logit difference or probability difference, moving toward G(x'_u) in Euclidean output space need not move along directions that keep dL/dx_v large. The paper should include direct evidence, for example a comparison of dL/dx_v norms along GradPath versus the straight-line path, or a formal statement under suitable assumptions, showing that the proposed path actually avoids saturation regions.
  3. [Section 6.2 and Table 4] The claim that 'EAP-GP identifies circuits that outperform other methods across all six tasks' is not supported by Table 4. On Greater-Than, EAP-IG is higher at 99.0% sparsity (95.35% vs 94.88%), at 98.4% sparsity (98.03% vs 98.98%), and at 97.5% sparsity (99.45% vs 98.98%). On Gender-Bias, EAP-IG is higher at 99.3% sparsity (70.06% vs 68.76%). The paper reports no seeds, confidence intervals, or significance tests, so the small differences at high sparsity and the headline 17.7% gain cannot be evaluated. Please report run-to-run variability and temper the claims, or provide statistical support for the comparative statements.
minor comments (5)
  1. [Section 3, Eq (6)] The interpolation factor in Eq (6) is written as 'k/m', but the surrounding text and Eq (3) indicate it should be 'j/k'. Please correct the notation.
  2. [Section 2 and References] The text attributes the argument that faithfulness metrics are preferable to circuit overlap to 'Hanna et al. (2024a)', but in the bibliography 2024a is the Greater-Than paper and 2024b is 'Have Faith in Faithfulness'. Please correct the citation.
  3. [Figures 4, 6, and 7] In the submitted text, Figures 4, 6, and 7 contain long literal '/uni000...' sequences instead of readable axis labels, legends, and plot content, making these figures effectively unreadable. Please regenerate the figures so that all text is embedded correctly.
  4. [Section 6.2, 'Effect on Number of Steps'] The explanation for the faithfulness decline at k > 5 (smaller gradient norms, reciprocal growth of W_j^{-1}, high-frequency oscillations near x') is stated without measurement. Please add diagnostics for the path, such as endpoint distance ||gammaG(1) - x'_u|| and gradient norms along the path, to support this explanation.
  5. [Abstract and Section 5] The text says the GradPath 'ends at the baseline input x'_u', but Algorithm 1 does not enforce this endpoint. Please align the wording with the algorithm or modify the algorithm, for example by adding a final projection onto x'_u or using a line search that guarantees endpoint convergence.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: EAP-GP is a new heuristic path attribution method evaluated against external ground-truth circuits, and its self-citations are not load-bearing.

full rationale

The paper's derivation chain does not reduce to its own inputs. EAP-GP replaces EAP-IG's straight-line averaging path (Eq. 6) with GradPath iterates (Eq. 8), and the resulting score (Eq. 9) is a heuristic extension, not a quantity fitted to the evaluation metric. The Normalized Faithfulness Score (Eq. 12) is computed post hoc by corrupting non-circuit edges (Eq. 11), and the precision-recall evaluation in Figure 5 uses manually annotated IOI and Greater-Than circuits from Syed et al. (2023), which are external to this paper and were not used to define the path. The self-citations in Section 1 (e.g., Cheng et al. 2024, Hong et al. 2024, Hu et al. 2024, Yang et al. 2024) are related-work pointers that do not justify the GradPath construction or the attribution formula. The substantive weakness identified by the skeptic—Eq. (9) omits the path-derivative weighting required by Eq. (1), and Eq. (8) minimizes model-output distance rather than loss-gradient saturation—is a validity/correctness concern about whether the heuristic achieves its stated goal, not a circular reduction: the adaptive path is not defined in terms of the faithfulness scores or the manual circuits it is evaluated against. No circular step can be exhibited from the paper's own equations, so the appropriate finding is no significant circularity.

Assumptions & free parameters 1 free parameters · 6 assumptions · 0 invented entities

The paper's central claim depends mainly on the assumed validity of the adaptive path integral in Eq (9), a step-size hyperparameter k, and the standard background of integrated gradients and faithfulness evaluation. No new physical or architectural entities are introduced.

free parameters (1)
  • k (number of GradPath steps) = 5
    Chosen by hand; the ablation in Section 6.2 shows k=4 or 5 is sufficient for IOI and Gender-Bias, while larger k degrades performance due to step normalization. The central results all use k=5.
assumptions (6)
  • standard math The integrated gradient formula in Eq (1) is a valid way to attribute edge importance when the path is known.
    The paper adopts IG from Sundararajan et al. (2017) without proof, which is acceptable background.
  • ad hoc to paper Equation (9) is a valid discrete approximation of the path integral for the GradPath path, even though the path is not straight.
    Section 5 uses (xu - x'_u) as an outer factor without showing that it equals the path derivative; this is assumed but not derived.
  • ad hoc to paper The one-step normalized gradient descent in Eq (8) produces an integration path that avoids saturation and stays in valid activation space.
    Algorithm 1 lines 4-5; no convergence, monotonicity, or endpoint guarantee is proven.
  • ad hoc to paper Minimizing the squared distance to the corrupted model output in Eq (7) is a good surrogate for avoiding saturated gradients.
    The paper motivates the GradPath objective intuitively but does not prove that low ||G(s)-G(x'_u)|| implies informative loss gradients.
  • domain assumption Normalized Faithfulness Score is an appropriate measure of circuit quality.
    Section 6.1 follows Hanna et al. (2024b); faithfulness is one of several possible circuit evaluation criteria.
  • domain assumption The task-specific logit and probability differences in Appendix A.1 correctly isolate the target behavior.
    The paper reuses these tasks and baselines from prior work without revalidating them.

how reviews work

0 comments
Cite this review

Pith. "Pith review of EAP-GP: Mitigating Saturation Effect in Gradient-based Automated Circuit Identification." pith.science (2026). https://pith.science/paper/HTIEKPFQ

@misc{pith2026250206852,
  author       = {Pith},
  title        = {Pith review of: EAP-GP: Mitigating Saturation Effect in Gradient-based Automated Circuit Identification},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HTIEKPFQ}},
  note         = {Machine review of arXiv:2502.06852}
}
read the original abstract

Understanding the internal mechanisms of transformer-based language models remains challenging. Mechanistic interpretability based on circuit discovery aims to reverse engineer neural networks by analyzing their internal processes at the level of computational subgraphs. In this paper, we revisit existing gradient-based circuit identification methods and find that their performance is either affected by the zero-gradient problem or saturation effects, where edge attribution scores become insensitive to input changes, resulting in noisy and unreliable attribution evaluations for circuit components. To address the saturation effect, we propose Edge Attribution Patching with GradPath (EAP-GP), EAP-GP introduces an integration path, starting from the input and adaptively following the direction of the difference between the gradients of corrupted and clean inputs to avoid the saturated region. This approach enhances attribution reliability and improves the faithfulness of circuit identification. We evaluate EAP-GP on 6 datasets using GPT-2 Small, GPT-2 Medium, and GPT-2 XL. Experimental results demonstrate that EAP-GP outperforms existing methods in circuit faithfulness, achieving improvements up to 17.7%. Comparisons with manually annotated ground-truth circuits demonstrate that EAP-GP achieves precision and recall comparable to or better than previous approaches, highlighting its effectiveness in identifying accurate circuits.

Figures

Figures reproduced from arXiv: 2502.06852 by the authors.

Figure 2
Figure 2. Illustration of the straight-line path and the dynamically adjusted path used in EAP-GP. GradPath starts at the original input xu and constructs a path in the direction of the steepest gradient descent toward the corrupted activation. The saturated area on the straight-line path is marked in red. regions cause the loss function to become insensitive to further input variations, reducing the attribution’s respon￾sive… view at source ↗
Figure 3
Figure 3. Faithfulness of circuits obtained using EAP-GP across different edge sparsity levels and step counts for IOI and gender-bias tasks. Method Sparsity (%) IOI Greater-Than NFS(%) ↑ Time (s) ↓ NFS(%) ↑ Time (s) ↓ EAP 97.5 ± 0.01 56.9 12.6 96.3 11.7 EAP-IG 97.5 ± 0.01 62.4 49.7 97.6 44.3 EAP-GP 97.5 ± 0.01 80.1 232.5 99.8 210.7 [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Comparison of circuit performance across different methods on GPT-2 Small. In all plots, a higher value indicates better performance. EAP-GP identifies circuits that outperform other methods across all six tasks. shown in [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figures from the paper (4 more)
Figure 5
Figure 5. Figure 5: Precision-recall curves for IOI (left) and Greater-Than (right) node / edge overlap it is greater than 0.8 and 0.25 for node and edge, respectively. However, for IOI, this assessment may be somewhat flawed due to the ambiguous role of MLPs in the manually found IOI cir…
Figure 6
Figure 6. Figure 6: Comparison of circuit performance across different methods on GPT-2 Medium. In all plots, a higher value indicates better performance. EAP-GP identifies circuits that outperform other methods across all six tasks. 96.0 96.5 97.0 97.5 98.0 98.5 99.0 99.5 100.0 Edge Spar…
Figure 7
Figure 7. Figure 7: Comparison of circuit performance across different methods on GPT-2 XL. In all plots, a higher value indicates better performance. EAP-GP identifies circuits that outperform other methods across all six tasks. 13 [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]
Figure 8
Figure 8. Figure 8: A circuit for Greater-Than with 99.9% sparsity, found by EAP-GP. 17 [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Query Circuits: Explaining How Language Models Answer User Prompts

    cs.AI 2025-09 conditional novelty 6.0 of 10

    Language models contain per-query circuits using a few percent of their edges that reproduce much of their answer behavior, and paraphrase-based Best-of-N search finds them.

  2. COMPKE: Complex Question Answering under Knowledge Editing

    cs.CL 2025-06 conditional novelty 6.0 of 10

    COMPKE is a new benchmark with 11,924 complex questions that tests knowledge editing through one-to-many relations and logical operations, where existing editing methods often fail.

  3. Understanding and Mitigating Cross-lingual Privacy Leakage via Language-specific and Universal Privacy Neurons

    cs.CL 2025-06 reject novelty 6.0 of 10

    Cross-lingual privacy leakage in LLMs is driven by a mix of language-universal and language-specific neurons, and deactivating those neurons lowers measured leakage by 23.3% to 31.6%.

  4. The Compositional Architecture of Regret in Large Language Models

    cs.CL 2025-06 reject novelty 5.0 of 10

    The paper claims that regret in LLMs is encoded by interacting neuron groups detectable in the final hidden layer, using new S-CDI, RDS, and GIC metrics.

Reference graph

Works this paper leans on

38 extracted references · 17 canonical work pages · cited by 4 Pith papers

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    GPT-4 Technical Report

    Achiam, J., Adler, S., Agarwal, S., and et al. GPT-4 Technical Report . arXiv preprint arXiv:2303.08774, 2023. URL https://arxiv.org/abs/2303.08774

    arXiv 2023

  3. [3]

    and Gavves, E

    Bereska, L. and Gavves, E. Mechanistic interpretability for ai safety--a review. arXiv preprint arXiv:2404.14082, 2024

    arXiv 2024

  4. [4]

    A., Yang, S., Lin, G., Zhai, Y., Fei, H., Xu, K., Yu, L., Hu, L., and Wang, D

    Cheng, K., Ali, M. A., Yang, S., Lin, G., Zhai, Y., Fei, H., Xu, K., Yu, L., Hu, L., and Wang, D. Leveraging logical rules in knowledge editing: A cherry on the top. arXiv preprint arXiv:2405.15452, 2024

    arXiv 2024

  5. [5]

    Identifying and adapting transformer-components responsible for gender bias in an english language model

    Chintam, A., Beloch, R., Zuidema, W., Hanna, M., and van der Wal, O. Identifying and adapting transformer-components responsible for gender bias in an english language model. In Belinkov, Y., Hao, S., Jumelet, J., Kim, N., McCarthy, A., and Mohebbi, H. (eds.), Proceedings of the 6th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP,...

    work page doi:10.18653/v1/2023.blackboxnlp-1.29 2023

  6. [6]

    Towards automated circuit discovery for mechanistic interpretability

    Conmy, A., Mavor-Parker, A., Lynch, A., et al. Towards automated circuit discovery for mechanistic interpretability. In Advances in Neural Information Processing Systems, volume 36, pp.\ 16318--16352. NeurIPS, 2023

    work page 2023

  7. [7]

    Sparse autoencoders find highly interpretable features in language models

    Cunningham, H., Ewart, A., Riggs, L., et al. Sparse autoencoders find highly interpretable features in language models. arXiv preprint arXiv:2309.08600, 2023

    arXiv 2023

  8. [8]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp.\ 4171--4186, Minneapolis, USA, 2019. Association for Computational Linguistics

    work page 2019

Show all 38 references
  1. [9]

    Causal abstractions of neural networks

    Geiger, A., Lu, H., Icard, T., Smith, J., and Doe, J. Causal abstractions of neural networks. In Advances in Neural Information Processing Systems, volume 34, pp.\ 9574--9586. NeurIPS, 2021

    2021

  2. [10]

    Finding alignments between interpretable causal variables and distributed neural representations

    Geiger, A., Wu, Z., Potts, C., Icard, T., and Goodman, N. Finding alignments between interpretable causal variables and distributed neural representations. In Locatello, F. and Didelez, V. (eds.), Proceedings of the Third Conference on Causal Learning and Reasoning, volume 236...

    2024

  3. [11]

    Transformer feed-forward layers are key-value memories

    Geva, M., Schuster, R., Berant, J., and Levy, O. Transformer feed-forward layers are key-value memories. arXiv preprint arXiv:2012.14913, 2020. URL https://arxiv.org/abs/2012.14913

    2012 arXiv

  4. [12]

    How does gpt-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model

    Hanna, M., Liu, O., and Variengien, A. How does gpt-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model. In Advances in Neural Information Processing Systems, volume 36. NeurIPS, 2024 a

    2024

  5. [13]

    Have faith in faithfulness: Going beyond circuit overlap when finding model mechanisms

    Hanna, M., Pezzelle, S., and Belinkov, Y. Have faith in faithfulness: Going beyond circuit overlap when finding model mechanisms. arXiv preprint arXiv:2403.17806, 2024 b . URL https://arxiv.org/abs/2403.17806

    2024 arXiv

  6. [14]

    Dictionary learning improves patch-free circuit discovery in mechanistic interpretability: A case study on othello-gpt

    He, Z., Ge, X., Tang, Q., et al. Dictionary learning improves patch-free circuit discovery in mechanistic interpretability: A case study on othello-gpt. arXiv preprint arXiv:2402.12201, 2024

    2024 arXiv

  7. [15]

    Dissecting fine-tuning unlearning in large language models

    Hong, Y., Zou, Y., Hu, L., Zeng, Z., Wang, D., and Yang, H. Dissecting fine-tuning unlearning in large language models. arXiv preprint arXiv:2410.06606, 2024

    2024 arXiv

  8. [16]

    A., and Wang, D

    Hu, L., Liu, L., Yang, S., Chen, X., Xiao, H., Li, M., Zhou, P., Ali, M. A., and Wang, D. A hopfieldian view-based interpretation for chain-of-thought reasoning. arXiv preprint arXiv:2406.12255, 2024

    2024 arXiv

  9. [17]

    Does circuit analysis interpretability scale? evidence from multiple choice capabilities in chinchilla

    Lieberum, T., Rahtz, M., Kram \'a r, J., et al. Does circuit analysis interpretability scale? evidence from multiple choice capabilities in chinchilla. arXiv preprint arXiv:2307.09458, 2023. URL https://arxiv.org/abs/2307.09458

    2023 arXiv

  10. [18]

    J., et al

    Marks, S., Rager, C., Michaud, E. J., et al. Sparse feature circuits: Discovering and editing interpretable causal graphs in language models. arXiv preprint arXiv:2403.19647, 2024. URL https://arxiv.org/abs/2403.19647

    2024 arXiv

  11. [19]

    Locating and editing factual associations in gpt

    Meng, K., Bau, D., Andonian, A., et al. Locating and editing factual associations in gpt. In Advances in Neural Information Processing Systems, volume 35, pp.\ 17359--17372. NeurIPS, 2022

    2022

  12. [20]

    Circuit component reuse across tasks in transformer language models

    Merullo, J., Eickhoff, C., and Pavlick, E. Circuit component reuse across tasks in transformer language models. In Proceedings of the Twelfth International Conference on Learning Representations. OpenReview.net, 2024

    2024

  13. [21]

    Transformer circuit faithfulness metrics are not robust

    Miller, J., Chughtai, B., and Saunders, W. Transformer circuit faithfulness metrics are not robust. arXiv preprint arXiv:2407.08734, 2024

    2024 arXiv

  14. [22]

    Evaluating brain-inspired modular training in automated circuit discovery for mechanistic interpretability

    Nainani, J. Evaluating brain-inspired modular training in automated circuit discovery for mechanistic interpretability. arXiv preprint arXiv:2401.03646, 2024. URL https://arxiv.org/abs/2401.03646

    2024 arXiv

  15. [23]

    Mechanistic interpretability quickstart guide

    Nanda, N. Mechanistic interpretability quickstart guide. Neel Nanda’s Blog, January 2023. Accessed: 2023-01-26

    2023

  16. [24]

    Mechanistic interpretability, variables, and the importance of interpretable bases

    Olah, C. Mechanistic interpretability, variables, and the importance of interpretable bases. https://www.transformer-circuits.pub/2022/mech-interp-essay, 2022

    2022

  17. [25]

    Zoom in: An introduction to circuits

    Olah, C., Cammarata, N., Schubert, L., et al. Zoom in: An introduction to circuits. Distill, 5 0 (3): 0 e00024.001, 2020. URL https://distill.pub/2020/circuits/zoom-in

    2020

  18. [26]

    and Bui, T

    O'Neill , C. and Bui, T. Sparse autoencoders enable scalable and reliable circuit identification in language models. arXiv preprint arXiv:2405.12522, 2024. URL https://arxiv.org/abs/2405.12522

    2024 arXiv

  19. [27]

    R., Haklay, T., et al

    Prakash, N., Shaham, T. R., Haklay, T., et al. Fine-tuning enhances existing mechanisms: A case study on entity tracking. arXiv preprint arXiv:2402.14811, 2024. URL https://arxiv.org/abs/2402.14811

    2024 arXiv

  20. [28]

    Visualizing the impact of feature attribution baselines

    Sturmfels, P., Lundberg, S., and Lee, S.-I. Visualizing the impact of feature attribution baselines. Distill, 5 0 (1): 0 e22, 2020. doi:10.23915/distill.00022

    2020 doi

  21. [29]

    Axiomatic attribution for deep networks

    Sundararajan, M., Taly, A., and Yan, Q. Axiomatic attribution for deep networks. In Proceedings of the International Conference on Machine Learning, volume 70, pp.\ 3319--3328. PMLR, 2017

    2017

  22. [30]

    Attribution patching outperforms automated circuit discovery

    Syed, A., Rager, C., and Conmy, A. Attribution patching outperforms automated circuit discovery. arXiv preprint arXiv:2310.10348, 2023. URL https://arxiv.org/abs/2310.10348

    2023 arXiv

  23. [31]

    Taori, R., Gulrajani, I., Zhang, T., Dubois, Y., Li, X., Guestrin, C., Liang, P., and Hashimoto, T. B. Stanford alpaca: An instruction-following llama model. https://github.com/tatsu-lab/stanford_alpaca, 2023

    2023

  24. [32]

    J., Geiger, A., et al

    Tigges, C., Hollinsworth, O. J., Geiger, A., et al. Linear representations of sentiment in large language models. arXiv preprint arXiv:2310.15154, 2023. URL https://arxiv.org/abs/2310.15154

    2023 arXiv

  25. [33]

    N., Kaiser, L., and Polosukhin, I

    Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, I. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30, 2017

    2017

  26. [34]

    Investigating gender bias in language models using causal mediation analysis

    Vig, J., Gehrmann, S., Belinkov, Y., Qian, S., Nevo, D., Singer, Y., and Shieber, S. Investigating gender bias in language models using causal mediation analysis. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M.-F., and Lin, H.-T. (eds.), Proceedings of the 33rd Advance...

    2020

  27. [35]

    R., Variengien, A., Conmy, A., Shlegeris, B., and Steinhardt, J

    Wang, K. R., Variengien, A., Conmy, A., Shlegeris, B., and Steinhardt, J. Interpretability in the wild: A circuit for indirect object identification in gpt-2 small. In International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=NpsVSN6o4ul

    2023

  28. [36]

    Interpretability at scale: Identifying causal mechanisms in alpaca

    Wu, Z., Geiger, A., Icard, T., et al. Interpretability at scale: Identifying causal mechanisms in alpaca. In Advances in Neural Information Processing Systems (NeurIPS), volume 36, 2024

    2024

  29. [37]

    What makes your model a low-empathy or warmth person: Exploring the origins of personality in llms

    Yang, S., Zhu, S., Bao, R., Liu, L., Cheng, Y., Hu, L., Li, M., and Wang, D. What makes your model a low-empathy or warmth person: Exploring the origins of personality in llms. arXiv preprint arXiv:2410.10863, 2024

    2024 arXiv

  30. [38]

    Locate-then-edit for multi-hop factual recall under knowledge editing

    Zhang, Z., Li, Y., Kan, Z., Cheng, K., Hu, L., and Wang, D. Locate-then-edit for multi-hop factual recall under knowledge editing. arXiv preprint arXiv:2410.06331, 2024

    2024 arXiv

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.