Pith. sign in

REVIEW 54 references

Language models contain per-query circuits using a few percent of their edges that reproduce much of their answer behavior, and paraphrase-based Best-of-N search finds them.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review

2026-08-04 13:52 UTC pith:ZCSBRK7M

arxiv 2509.24808 v2 pith:ZCSBRK7M submitted 2025-09-29 cs.AI

Query Circuits: Explaining How Language Models Answer User Prompts

Tung-Yu Wu , Fazl Barez This is my paper
classification cs.AI
keywords circuitsmodelquerycircuitexplanationsinputlanguagesparse
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Large language models are made of millions of connections. When a model answers a question like 'Which drug is safe for this patient?', most of those connections are busy with other things. This paper asks: which small set of connections actually carried the answer? The authors call that set a 'query circuit'.

Finding it is hard. Standard scoring methods, borrowed from older 'capability circuit' research, become noisy on realistic questions: their scores swing outside the normal range and do not improve as edges are added. The authors fix the measurement with a bounded faithfulness score called NDF, and fix the search with 'Best-of-N': generate several paraphrases of the question, find a candidate circuit for each, and keep the candidate that best reproduces the model's original answer. The paraphrases act like lottery tickets; the best one wins.

On multiple-choice exams (MMLU, ARC) and on arithmetic and sentence tasks, the winners are substantial: about 5,000 edges out of roughly 387,000, about 1.3%, reproduce on average about 60% of the model's per-question behavior. Removing these circuits instead of keeping them shows the critical information really does live there. Two faster approximations, iBoN and BoN-CSM, trade some faithfulness for speed.

The caveats: the reported 60% is the best result out of ten tries, so it flatters the method somewhat. No error bars are given, no code is released, and the score measures logit differences rather than whether the model picks the correct option. The results are most convincing on the simple IOI task, where the older, more standard metric also shows the effect.

Core claim

Abstract: 'we find that there exist extremely sparse query circuits within the model that can recover much of its performance on single queries. For example, a circuit covering only 1.3% of model connections can recover about 60% of performance on an MMLU questions.' Section 6.2: 'In MMLU, a circuit with only 5k edges (1.3% of Llama-3.2-1B-Instruct's all edges) achieves an average NDF of 0.6.' If the paper is correct, sparse per-query subgraphs of a transformer exist and can be found cheaply via paraphrase-based Best-of-N search, making local, faithful model explanations a practical target.

Load-bearing premise

The NDF normalization assumes the corrupted-query baseline is a fair counterfactual for 'the model without the question-understanding signal.' For MMLU/ARC this baseline is built by replacing the stem with 'Which is the most possible answer?' while preserving options. The paper's own Table 1 shows the baseline is not empty: L(M(q'))=0.53 for Query 3 versus 0.96 for the clean query, and Appendix B.4 concedes that both clean and corrupted queries contain option-ID signals that distribute logits. If position bias or option-format artifacts give the corrupted model a non-trivial margin, then the normalization L(M(q))-L(M(q')) is contaminated, and a circuit that partially reproduces option-format tendencies can score high NDF without recovering any question-answering computation. Every MMLU/ARC NDF value, including the headline 0.6 at 5k edges, inherits this scale. The IOI results use the standard name-replacement corruption and the classic NFS metric, so they do not carry this risk.

Share X LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Axiom & Free-Parameter Ledger

3 free parameters · 6 axioms · 0 invented entities

Everything the central claim rests on beyond the model weights themselves: the validity of logit-difference as the performance metric, the counterfactual baseline used to normalize faithfulness, the adequacy of integrated-gradient attribution as a proxy for causal patching, the adequacy of greedy top-N selection, and the semantic validity of GPT-4o paraphrases. No numerical parameters were fitted to data: the two hand-chosen hyperparameters (p=9, m=20) and the MMLU category-selection rule affect the magnitude but not the existence of the reported effects. No invented entities are introduced; a query circuit is a subgraph of the existing model, not a new mechanism.

free parameters (3)
  • number of paraphrases p = 9
    Hand-chosen in Section 6.1; Figure 7 shows diminishing returns from p=2 to 10, so the value is a cost-quality tradeoff, not an optimum.
  • IG discretization step m = 20
    Hand-chosen (Section 6.1); Figure 2c shows m=5, 20, 100, 500, 1000 perform similarly, so this choice does not drive results but is a free design constant.
  • MMLU category selection rule = categories where Claude 3.5 Sonnet scores >= 95% on HELM MMLU
    Appendix A: nine categories are drawn from this set; the rule constrains question difficulty and is a data-selection choice that shapes aggregate NDF levels.
axioms (6)
  • domain assumption Logit difference between the correct and average-incorrect tokens is a valid performance metric L for single-query faithfulness.
    Section 6.1 and Appendix A adopt logit difference as L for all tasks; NDF and NFS inherit it. On hard MMLU questions the model margin is often negative, so L is a signed logit margin, not accuracy or probability.
  • domain assumption Corrupted query 'Which is the most possible answer?' provides a clean counterfactual baseline for the NDF normalization on MCQs.
    Section 4.1 and Appendix A define NDF against L(M(q')) with this corruption; Table 1 and Appendix B.4 show the baseline retains option-ID signals (L(M(q'))=0.53), so the normalization is contaminated.
  • domain assumption Integrated-gradients edge scores with m=20 steps (Eq. 2) are adequate proxies for true indirect-effect scores (Eq. 1) on single queries.
    All methods use EAP-IG with m=20 (Section 6.1); Section 3.3.2 and Figure 2c show single-query scores are noisy, which is exactly why BoN is introduced, so the backbone scoring is approximate by the paper's own account.
  • standard math The residual-rewrite abstraction of a transformer into attention-head/MLP nodes with QKV-split edges (TransformerLens) is a valid circuit representation.
    Section 2.1, following Elhage et al. 2021 and Syed et al. 2024; this is the shared substrate of all circuit discovery the paper builds on.
  • domain assumption Greedy top-N edge selection with respect to attribution scores produces circuits no worse than Dijkstra-style construction.
    Section 6.1 and Appendix B.5 justify this on IOI, GT, and gender-bias in NFS, not on NDF and not on MMLU/ARC, but the paper leans on greedy selection for all main results.
  • domain assumption GPT-4o paraphrases of the question stem preserve meaning and do not change the query's underlying computation.
    Section 6.1: paraphrases are generated without validity checks or examples shown for MMLU/ARC; BoN's success rests on paraphrase-derived circuits transferring to the original query.

reviewed 2026-08-04 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Query Circuits: Explaining How Language Models Answer User Prompts." pith.science (2026). https://pith.science/paper/ZCSBRK7M

@misc{pith2026250924808,
  author       = {Pith},
  title        = {Pith review of: Query Circuits: Explaining How Language Models Answer User Prompts},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZCSBRK7M}},
  note         = {Machine review of arXiv:2509.24808}
}
Share X LinkedIn Reddit HN
read the original abstract

Explaining why a language model produces a particular output requires local, input-level explanations. Existing methods uncover global capability circuits (e.g., indirect object identification), but not why the model answers a specific input query in a particular way. We introduce query circuits, which directly trace the information flow inside a model that maps a specific input to the output. Unlike surrogate-based approaches (e.g., sparse autoencoders), query circuits are identified within the model itself, resulting in more faithful and computationally accessible explanations. To make query circuits practical, we address two challenges. First, we introduce Normalized Deviation Faithfulness (NDF), a robust metric to evaluate how well a discovered circuit recovers the model's decision for a specific input, and is broadly applicable to circuit discovery beyond our setting. Second, we develop sampling-based methods to efficiently identify circuits that are sparse yet faithfully describe the model's behavior. Across benchmarks (IOI, arithmetic, MMLU, and ARC), we find that there exist extremely sparse query circuits within the model that can recover much of its performance on single queries. For example, a circuit covering only 1.3% of model connections can recover about 60% of performance on an MMLU questions. Overall, query circuits provide a step towards faithful, scalable explanations of how language models process individual inputs. The project page is at https://tony10101105.github.io/query-circuit/.

Figures

Figures reproduced from arXiv: 2509.24808 by Fazl Barez, Tung-yu Wu.

Figure 1
Figure 1. Figure 1: Query circuit discovery aims to identify a sparse sub-network within the LLM that underlies the model response to a user input query. The LLM and circuit in this illustration are simplified for visualization. why the LLM produces a particular output for a given input prompt. Instead, query circuits provide local, prompt-level explanations by directly tracing the information flow inside the LLM ( [PITH_FUL… view at source ↗
Figure 2
Figure 2. Figure 2: Technical challenges of query circuit discovery. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Complete evaluation results for the three queries adopted in Figure [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: A case study on IOI dataset. Circuits discovered by the original input query’s paraphrases may recover model performance on the query. We find that while the circuit discovered on the orig￾inal query may fail to faithfully recover model per￾formance, circuits discovered on its paraphrases can succeed [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: The pipeline of Best-of-N sampling for discovering a faithful query circuit of N edges for an input query q, for which it generates p paraphrases. p = 3 in this illustration. Algorithm 1 iBoN Input: Circuits (edge sets) {E1, . . . , Ek} with size in ascending order and edge num￾ber constraint N. Output: An edge set (circuit) E 1: Initialize E as an empty edge set. 2: Find the largest i s.t. |Ei | < N. 3: E… view at source ↗
Figure 6
Figure 6. Figure 6: Main results of BoN sampling for query circuit discovery. BoN substantially outperforms all other methods. Although iBoN and BoN-CSM, two fast approximations to BoN, perform worse than it, they still clearly exceed both baselines. circuit by augmenting the best available smaller circuit with additional high-scoring edges from a larger one that are not already included. 5.4 BON WITH CONSTRAINT-ADAPTIVE SCOR… view at source ↗
Figure 7
Figure 7. Figure 7: Performance of BoN sampling with different numbers of paraphrases. As BoN selects the most faithful circuit, its performance exhibits monotonically increasing yet diminishing returns [PITH_FULL_IMAGE:figures/full_fig_p009_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

54 extracted references · 6 linked inside Pith

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Explainability for artificial intelligence in healthcare: a multidisciplinary perspective

    Julia Amann, Alessandro Blasimme, Effy Vayena, Dietmar Frey, Vince I Madai, and Precise4Q Consortium. Explainability for artificial intelligence in healthcare: a multidisciplinary perspective. BMC Medical Informatics and Decision Making, 20: 0 1--9, 2020

    2020

  3. [3]

    Circuit tracing: Revealing computational graphs in language models

    Emmanuel Ameisen, Jack Lindsey, Adam Pearce, Wes Gurnee, Nicholas L Turner, Brian Chen, Craig Citro, David Abrahams, Shan Carter, Basil Hosmer, et al. Circuit tracing: Revealing computational graphs in language models. Transformer Circuits Thread, 2025

    2025

  4. [4]

    Claude 3.5 sonnet

    Antropic. Claude 3.5 sonnet. 2025. URL https://www.anthropic.com/news/claude-3-5-sonnet

    2025

  5. [5]

    Mechanistic interpretability for AI safety - a review

    Leonard Bereska and Stratis Gavves. Mechanistic interpretability for AI safety - a review. Transactions on Machine Learning Research (TMLR), 2024. ISSN 2835-8856

    2024

  6. [6]

    Building and evaluating alignment auditing agents

    Trenton Bricken, Rowan Wang, Sam Bowman, Euan Ong, Johannes Treutlein, Jeff Wu, Evan Hubinger, and Samuel Marks. Building and evaluating alignment auditing agents. https://alignment.anthropic.com/2025/automated-auditing/, July 2025

    2025

  7. [7]

    How people use chatgpt

    Aaron Chatterji, Thomas Cunningham, David J Deming, Zoe Hitzig, Christopher Ong, Carl Yan Shan, and Kevin Wadman. How people use chatgpt. Working Paper 34255, National Bureau of Economic Research, September 2025

    2025

  8. [8]

    Selfie: self-interpretation of large language model embeddings

    Haozhe Chen, Carl Vondrick, and Chengzhi Mao. Selfie: self-interpretation of large language model embeddings. In Proceedings of the International Conference on Machine Learning (ICML), 2024

    2024

  9. [9]

    Think you have solved question answering? try arc, the ai2 reasoning challenge

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv:1803.05457v1, 2018

    Pith/arXiv arXiv 2018

  10. [10]

    Towards automated circuit discovery for mechanistic interpretability

    Arthur Conmy, Augustine Mavor-Parker, Aengus Lynch, Stefan Heimersheim, and Adri \`a Garriga-Alonso. Towards automated circuit discovery for mechanistic interpretability. In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS), volume 36, pp.\ 16318--16352, 2023

    2023

  11. [11]

    The llama 3 herd of models

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv e-prints, pp.\ arXiv--2407, 2024

    2024

  12. [12]

    A mathematical framework for transformer circuits

    Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olah. A...

    2021

  13. [13]

    N2g: A SCALABLE APPROACH FOR QUANTIFYING INTERPRETABLE NEURON REPRESENTATION IN LLMS

    Alex Foote, Neel Nanda, Esben Kran, Ioannis Konstas, and Fazl Barez. N2g: A SCALABLE APPROACH FOR QUANTIFYING INTERPRETABLE NEURON REPRESENTATION IN LLMS . In ICLR 2023 Workshop on Trustworthy and Reliable Large-Scale Machine Learning Models, 2023

    2023

  14. [14]

    The lottery ticket hypothesis: Finding sparse, trainable neural networks

    Jonathan Frankle and Michael Carbin. The lottery ticket hypothesis: Finding sparse, trainable neural networks. In Proceedings of the International Conference on Learning Representations (ICLR), 2019

    2019

  15. [15]

    Patchscopes: A unifying framework for inspecting hidden representations of language models

    Asma Ghandeharioun, Avi Caciularu, Adam Pearce, Lucas Dixon, and Mor Geva. Patchscopes: A unifying framework for inspecting hidden representations of language models. In Proceedings of the International Conference on Machine Learning (ICML), 2024

    2024

  16. [16]

    How does GPT -2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model

    Michael Hanna, Ollie Liu, and Alexandre Variengien. How does GPT -2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model. In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS), 2023

    2023

  17. [17]

    Have faith in faithfulness: Going beyond circuit overlap when finding model mechanisms

    Michael Hanna, Sandro Pezzelle, and Yonatan Belinkov. Have faith in faithfulness: Going beyond circuit overlap when finding model mechanisms. In Proceedings of the Conference on Language Modeling (COLM), 2024

    2024

  18. [19]

    Measuring massive multitask language understanding

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. In Proceedings of the International Conference on Learning Representations (ICLR), 2021

    2021

  19. [20]

    Sparse autoencoders find highly interpretable features in language models

    Robert Huben, Hoagy Cunningham, Logan Riggs Smith, Aidan Ewart, and Lee Sharkey. Sparse autoencoders find highly interpretable features in language models. In Proceedings of the International Conference on Learning Representations (ICLR), 2024

    2024

  20. [21]

    Gpt-4o system card

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024

    Pith/arXiv arXiv 2024

  21. [22]

    Guided integrated gradients: An adaptive path method for removing noise

    Andrei Kapishnikov, Subhashini Venugopalan, Besim Avci, Ben Wedin, Michael Terry, and Tolga Bolukbasi. Guided integrated gradients: An adaptive path method for removing noise. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 5050--5058, 2021

    2021

  22. [23]

    Scaling sparse feature circuit finding for in-context learning

    Dmitrii Kharlapenko, Stepan Shabalin, Fazl Barez, Arthur Conmy, and Neel Nanda. Scaling sparse feature circuit finding for in-context learning. In Proceedings of the International Conference on Machine Learning (ICML), 2025

    2025

  23. [24]

    Why are saliency maps noisy? cause of and solution to noisy saliency maps

    Beomsu Kim, Junghoon Seo, Seunghyeon Jeon, Jamyoung Koo, Jeongyeol Choe, and Taegyun Jeon. Why are saliency maps noisy? cause of and solution to noisy saliency maps. In 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), pp.\ 4149--4157, 2019

    2019

  24. [25]

    Granular concept circuits: Toward a fine-grained circuit discovery for concept representations

    Dahee Kwon, Sehyun Lee, and Jaesik Choi. Granular concept circuits: Toward a fine-grained circuit discovery for concept representations. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2025

    2025

  25. [26]

    Towards interpretable sequence continuation: Analyzing shared circuits in large language models

    Michael Lan, Philip Torr, and Fazl Barez. Towards interpretable sequence continuation: Analyzing shared circuits in large language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp.\ 12576--12601. Association for Computational Linguistics, November 2024

    2024

  26. [27]

    Reddi, Ke Ye, Felix Chern, Felix Yu, Ruiqi Guo, and Sanjiv Kumar

    Zonglin Li, Chong You, Srinadh Bhojanapalli, Daliang Li, Ankit Singh Rawat, Sashank J. Reddi, Ke Ye, Felix Chern, Felix Yu, Ruiqi Guo, and Sanjiv Kumar. The lazy neuron phenomenon: On emergence of activation sparsity in transformers. In Proceedings of the International Conference on Learning Representations (ICLR), 2023

    2023

  27. [28]

    Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, Benjamin Newman, Binhang Yuan, Bobby Yan, Ce Zhang, Christian Alexander Cosgrove, Christopher D Manning, Christopher Re, Diana Acosta-Navas, Drew Arad Hudson, Eric Zelikman, Esin Durmus, Faisal Ladhak, Frieda R...

    2023

  28. [29]

    Sparse crosscoders for cross-layer features and model diffing

    Jack Lindsey, Adly Templeton, Jonathan Marcus, Thomas Conerly, Joshua Batson, and Christopher Olah. Sparse crosscoders for cross-layer features and model diffing. Transformer Circuits Thread, 2024

    2024

  29. [30]

    Lundberg and Su-In Lee

    Scott M. Lundberg and Su-In Lee. A unified approach to interpreting model predictions. In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS), pp.\ 4768–4777, 2017

    2017

  30. [31]

    Enhancing neural network interpretability with feature-aligned sparse autoencoders

    Luke Marks, Alasdair Paren, David Krueger, and Fazl Barez. Enhancing neural network interpretability with feature-aligned sparse autoencoders. arXiv preprint arXiv:2411.01220, 2024

    Pith/arXiv arXiv 2024

  31. [32]

    Sparse feature circuits: Discovering and editing interpretable causal graphs in language models

    Samuel Marks, Can Rager, Eric J Michaud, Yonatan Belinkov, David Bau, and Aaron Mueller. Sparse feature circuits: Discovering and editing interpretable causal graphs in language models. In Proceedings of the International Conference on Learning Representations (ICLR), 2025

    2025

  32. [33]

    Transformer circuit evaluation metrics are not robust

    Joseph Miller, Bilal Chughtai, and William Saunders. Transformer circuit evaluation metrics are not robust. In Proceedings of the Conference on Language Modeling (COLM), 2024

    2024

  33. [34]

    MIB : A mechanistic interpretability benchmark

    Aaron Mueller, Atticus Geiger, Sarah Wiegreffe, Dana Arad, Iv \'a n Arcuschin, Adam Belfki, Yik Siu Chan, Jaden Fried Fiotto-Kaufman, Tal Haklay, Michael Hanna, Jing Huang, Rohan Gupta, Yaniv Nikankin, Hadas Orgad, Nikhil Prakash, Anja Reusch, Aruna Sankaranarayanan, Shun Shao, Alessandro Stolfo, Martin Tutek, Amir Zur, David Bau, and Yonatan Belinkov. MI...

    2025

  34. [35]

    Transformerlens

    Neel Nanda and Joseph Bloom. Transformerlens. https://github.com/TransformerLensOrg/TransformerLens, 2022

    2022

  35. [36]

    A toy model of mechanistic (un)faithfulness

    Chris Olah. A toy model of mechanistic (un)faithfulness. Transformer Circuits Thread, 2025

    2025

  36. [37]

    Explanations in autonomous driving: A survey

    Daniel Omeiza, Helena Webb, Marina Jirotka, and Lars Kunze. Explanations in autonomous driving: A survey. IEEE Transactions on Intelligent Transportation Systems, 23 0 (8): 0 10142--10162, 2022

    2022

  37. [38]

    Understanding addition in transformers

    Philip Quirke and Fazl Barez. Understanding addition in transformers. In In Proceedings of the International Conference on Learning Representations (ICLR), 2024

    2024

  38. [39]

    Language models are unsupervised multitask learners

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 1 0 (8): 0 9, 2019

    2019

  39. [40]

    A multimodal automated interpretability agent

    Tamar Rott Shaham, Sarah Schwettmann, Franklin Wang, Achyuta Rajaram, Evan Hernandez, Jacob Andreas, and Antonio Torralba. A multimodal automated interpretability agent. In Proceedings of the International Conference on Machine Learning (ICML). JMLR.org, 2024

    2024

  40. [41]

    A value for n-person games

    Lloyd S Shapley. A value for n-person games. In Harold W. Kuhn and Albert W. Tucker (eds.), Contributions to the Theory of Games II, pp.\ 307--317. Princeton University Press, 1953

    1953

  41. [42]

    Smoothgrad: removing noise by adding noise

    Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Vi \'e gas, and Martin Wattenberg. Smoothgrad: removing noise by adding noise. arXiv preprint arXiv:1706.03825, 2017

    Pith/arXiv arXiv 2017

  42. [43]

    Axiomatic attribution for deep networks

    Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In Proceedings of the International Conference on Machine Learning (ICML), pp.\ 3319--3328. PMLR, 2017

    2017

  43. [44]

    Attribution patching outperforms automated circuit discovery

    Aaquib Syed, Can Rager, and Arthur Conmy. Attribution patching outperforms automated circuit discovery. In Proceedings of the 7th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP, pp.\ 407--416, 2024

    2024

  44. [45]

    Universal properties of activation sparsity in modern large language models

    Filip Szatkowski, Patryk B e dkowski, Alessio Devoto, Jan Dubi \'n ski, Pasquale Minervini, Miko aj Pi \'o rczy \'n ski, Simone Scardapane, and Bartosz W \'o jcik. Universal properties of activation sparsity in modern large language models. arXiv preprint arXiv:2509.00454, 2025

    arXiv 2025

  45. [46]

    Daniel Freeman, Theodore R

    Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, Hoagy Cunningham, Nicholas L Turner, Callum McDougall, Monte MacDiarmid, C. Daniel Freeman, Theodore R. Sumers, Edward Rees, Joshua Batson, Adam Jermyn, Shan Carter, Chris Olah, and Tom Henighan. Scaling monosema...

    2024

  46. [47]

    Investigating gender bias in language models using causal mediation analysis

    Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart Shieber. Investigating gender bias in language models using causal mediation analysis. In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS), volume 33, pp.\ 12388--12401, 2020

    2020

  47. [48]

    Interpretability in the wild: a circuit for indirect object identification in GPT -2 small

    Kevin Ro Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. Interpretability in the wild: a circuit for indirect object identification in GPT -2 small. In Proceedings of the International Conference on Learning Representations (ICLR), 2023

    2023

  48. [49]

    Do LLM s overcome shortcut learning? an evaluation of shortcut challenges in large language models

    Yu Yuan, Lili Zhao, Kai Zhang, Guangting Zheng, and Qi Liu. Do LLM s overcome shortcut learning? an evaluation of shortcut challenges in large language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp.\ 12188--12200. Association for Computational Linguistics, 2024

    2024

  49. [50]

    Towards best practices of activation patching in language models: Metrics and methods

    Fred Zhang and Neel Nanda. Towards best practices of activation patching in language models: Metrics and methods. In Proceedings of the International Conference on Learning Representations (ICLR), 2024

    2024

  50. [51]

    Eap-gp: Mitigating saturation effect in gradient-based automated circuit identification

    Lin Zhang, Wenshuo Dong, Zhuoran Zhang, Shu Yang, Lijie Hu, Ninghao Liu, Pan Zhou, and Di Wang. Eap-gp: Mitigating saturation effect in gradient-based automated circuit identification. arXiv preprint arXiv:2502.06852, 2025

    Pith/arXiv arXiv 2025

  51. [52]

    Large language models are not robust multiple choice selectors

    Chujie Zheng, Hao Zhou, Fandong Meng, Jie Zhou, and Minlie Huang. Large language models are not robust multiple choice selectors. In Proceedings of the International Conference on Learning Representations (ICLR), 2024

    2024

  52. [53]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should not add it explicitly Type <Return> for now, but then later remove the command n...

  53. [54]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@first@sw \@firstoftwo \@ifundefined NAT@b*@#2 \@firstoftwo @num @NAT@ctr \@secondoft...

  54. [55]

    Attribution Patching Outperforms Automated Circuit Discovery

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibsetup #1 @NAT@ctr @ @openbib .11em \@plus.33em \@minus.07em 4000 4000 `\.\@m @bibit...

    Pith/arXiv arXiv 2024

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.