Pith. sign in

REVIEW 4 major objections 5 minor 2 cited by

Dissecting Bias in LLMs: A Mechanistic Interpretability Perspective

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that gender and nationality bias in GPT-2 and Llama-2 is concentrated in a few internal edges, is unstable under fine-tuning, and that ablating them degrades other NLP tasks.

desk verdict Useful empirical study, but the demographic-bias half rests on an unvalidated sentiment classifier and a circular debiasing metric; the gender-bias half and overlap analysis are more convincing. read the letter →

arxiv 2506.05166 v2 pith:VFW436TZ submitted 2025-06-05 cs.CL cs.AI

Bhavik Chandna , Zubair Bashir , Procheta Sen This is my paper
classification cs.CLcs.AI
keywords mechanisticinterpretabilityedgeattributionpatchingdemographicbiasgenderGPT-2Llama-2debiasingcircuitlocalization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that demographic and gender bias in large language models is not spread evenly through the network but is carried by a small subset of internal connections, and that this localization can be exploited for intervention. Using edge attribution patching (a cheap causal score for each connection between attention heads, MLP layers, and output logits) on GPT-2 Small, GPT-2 Large, and Llama-2, the authors identify the edges most responsible for nationality and gender bias. The edges cluster in a few layers, they shift when the model is fine-tuned or the sentence template changes, and demographic and gender circuits barely overlap. Replacing the top-scoring edges' activations at inference time reduces the bias metric by up to 71 percent in most settings, but also lowers performance on named-entity recognition and linguistic acceptability, so the bias circuit is entangled with general language competence.

What carries the argument

The central object is the edge attribution score from Edge Attribution Patching (EAP), which approximates how much the bias metric changes when a single edge's activation is replaced by the activation from a corrupted input. Edges are the connections between computational nodes such as attention heads, MLP layers, and the input or logits nodes. The paper ranks all edges by this score with the $L_2$ bias metric (cumulative probability of positive or male tokens among top-10 predictions) and the $C_2$ symmetric token replacement corruption, then tests localization by ablating the top edges and stability by comparing top-$K$ sets across conditions.

What would settle it

Run the same pipeline on the same demographic templates but label the completed sentences with a different sentiment classifier or with human annotators; if the top EAP edges change substantially, the localization claim is an artifact of the sentiment model rather than a property of the LLM. Additionally, if randomly chosen edge sets of the same size reduce the $L_2$ bias metric as much as the top-$K$ sets do, the attribution is not capturing a uniquely responsible circuit.

Watch

Extended reading notes

Core claim

Across GPT-2 Small, GPT-2 Large, and Llama-2, demographic and gender bias is encoded in a small, localized set of edges: most high-attribution edges sit in a few layers (roughly layers 2–6 in GPT-2 Small, scattered specific layers in GPT-2 Large, and layers 0–11 plus 30–31 in Llama-2), and ablating 40 percent of the top edges drops the bias metric by more than 90 percent for GPT-2 Small and Llama-2. These edges are not stable: they shift under fine-tuning on both bias-related and unrelated data and under grammatical variation of the prompt, and demographic and gender circuits are largely disjoint. When the top bias edges are corrupted at inference time, bias falls in most settings while CoLA and CoNLL-2003 performance also falls, indicating that bias-related edges overlap with components used by other language tasks.

Load-bearing premise

The demographic-bias labels, the split into positive- and negative-bias datasets, the edge scores, and the debiasing evaluation all inherit whatever errors the DistilBERT sentiment model makes when scoring completed nationality sentences.

Editorial extensions

If this is right

  • If bias is localized in a few edges, targeted inference-time interventions can reduce measured bias without retraining the model.
  • The instability of top edges under fine-tuning means a circuit found in one checkpoint is not a reliable target for permanent debiasing.
  • Because demographic and gender bias use mostly disjoint edges, debiasing one bias type will not automatically remove another.
  • The drop in CoLA and CoNLL-2003 accuracy after edge corruption implies that bias circuits share components with broader linguistic capabilities, so task-agnostic edge ablation has a real cost.
  • The finding suggests that circuit-level inspection can predict which downstream tasks will be harmed by a debiasing intervention.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If bias circuits are this unstable, then a debiasing method that only edits a fixed set of discovered edges may see the bias re-emerge under other prompts or after further fine-tuning; testing on held-out prompt distributions would reveal how quickly this happens.
  • The metric dependence is a plausible alternate explanation: the same pipeline with a different sentiment classifier or human labels might yield a different edge set, so the localization claim should be checked against an annotation-based bias measure.
  • One testable extension is to run the same edge-ranking procedure on newer model families or on other bias dimensions such as age or religion, which the paper itself notes may have entirely separate circuits.
  • The task-overlap result suggests a possible design rule for debiasing: identify and spare edges that are also important for downstream tasks, and measure the residual bias after sparing them.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper applies Edge Attribution Patching (EAP) to GPT-2 Small, GPT-2 Large, and Llama-2 to identify edges important for demographic and gender bias, defined through two metrics (L1, L2) and two corruption strategies (C1, C2). It reports that bias-related edges are concentrated in a few layers, that these edges shift under grammatical variation and fine-tuning, and that corrupting the top edges reduces the bias metric while degrading CoLA and CoNLL-2003 performance. The authors conclude that bias is localized, unstable across perturbations, and partially shared with general task competence, and they propose an inference-time debiasing method based on edge corruption.

Significance. If the localization and debiasing claims are sound, the paper offers a practical, retraining-free approach to bias mitigation and provides comparative evidence across model scales, which is a useful contribution to mechanistic interpretability. The release of code and the use of three models with two bias types are strengths. However, the central causal claims currently rest on unvalidated sentiment labels, a missing random-edge ablation baseline, and a debiasing evaluation that uses the same metric used for edge selection. These gaps weaken the support for the abstract's 'highly localized' and 'reduces biased outputs' statements, especially for demographic bias.

major comments (4)
  1. [Section 4, paragraph 2; Equations 1-3; Table 3] The demographic bias pipeline depends entirely on DistilBERT sentiment labels applied to completed sentences to classify each predicted token as positive or negative. No validation is provided against human judgment, a second sentiment model, or manual inspection. Because these labels feed directly into the L1/L2 metrics, the positive/negative dataset split, the EAP edge scores, and the δBias column of Table 3, any systematic error in sentiment classification for nationality templates propagates to the central demographic localization and debiasing claims. The authors should report label agreement rates, show example completions with their assigned labels, and test robustness with an alternative sentiment model or a lexicon-based labeler.
  2. [Section 3.3 and Figure 3] The localization claim that bias is encoded in a small set of edges is based on the drop in L2 when the top EAP-scored edges are ablated. However, the paper does not include a random-edge ablation baseline. Without comparing against ablation of an equal number of randomly chosen edges, the observed drop could reflect the general sensitivity of the model to any disruption or to globally important edges—a possibility that the paper's own CoLA/CoNLL degradation results make plausible. A random-edge control is necessary to attribute the drop specifically to bias-related localization.
  3. [Section 5, Table 3] The debiasing evaluation measures δBias using the same L2 metric that was used to select the edges via EAP. Since the edges are chosen for their effect on L2, corrupting their activations will reduce L2 at least partly by construction, making the reported bias reduction circular. The authors should evaluate debiasing with an independent bias measure—for example, a different set of templates, a different sentiment classifier or word-list assignment, or human-evaluated completions—to show that the intervention reduces bias and not merely the metric used for selection.
  4. [Section 5, Table 2] The choice of L2 and C2 for all subsequent experiments is made post-hoc, after observing which configuration in Table 2 yields the smallest 'Change in Metric' value. This is a selection on the test data, and the criterion itself is not a direct test of whether the important edges are causally responsible for bias: a small difference between the full model and a model restricted to important edges tests sufficiency, not necessity, yet the debiasing experiments later ablate these edges. The authors should either pre-specify the metric/corruption choice or show that the localization, stability, and debiasing conclusions are robust across L1/L2 and C1/C2 rather than artifacts of the selected configuration.
minor comments (5)
  1. [Section 5, Table 3 and caption] The table header 'NRE-CoNL2023' should be 'NER-CoNLL2003', and the caption uses 'NRE-CoNL2023' as well. Additionally, the paper should state whether δBias and performance changes are relative percentages or percentage-point changes.
  2. [Appendix E] The corruption descriptions for CoLA and CoNLL appear to be duplicated: both sentences begin 'For CoLA dataset...', and the first one says 'swap every noun token with XYZ' while the second says 'swap any two randomly chosen words'. One of these presumably applies to CoNLL-2003; please correct the assignment.
  3. [Section 6, Conclusion] The conclusion states that debiasing negatively impacts 'natural language inference', but the paper only evaluates CoLA and CoNLL-2003. Either add an NLI evaluation or remove this unsupported mention.
  4. [Section 3.4 and Figure 4] The stability analysis in Figure 4 does not specify the value of K used to define 'top K edges', nor whether K is held constant across models. State the K value and report whether the overlap results are sensitive to K.
  5. [Section 4, Table 4] For Llama-2 in the DSS1 negative condition there are only 8 samples; this small sample size should be explicitly flagged when interpreting the corresponding results in Figures 2, 3, and Table 3.

Circularity Check

1 steps flagged · score 6.0 of 10

Debiasing claim inherits the EAP selection metric; localization and downstream-task checks are independent.

  1. fitted input called prediction [Section 3 (Eq. 1) and Section 5 (Table 3)]
    "If a set of edges is important for a particular task then providing corrupted values to those edges will reduce the value of L(xclean) significantly. ... From Table 3, we can see that corrupting the top edges responsible for bias reduced bias in the original model in most of the cases, except for GPT-2 Large in DSS1."

    Equation 1 defines each edge's EAP score as the absolute change in the bias metric L when that edge is corrupted, and Section 5 fixes that metric to L2 (with C2) for 'all the remaining demographic and gender bias analysis experiments.' The 'top edges' whose corruption is reported in Table 3 are therefore precisely the edges selected to maximize the reduction in L2. Measuring 'δBias' as the change in that same L2 after corrupting those edges is not an independent test of debiasing: the reported bias drop is inherited from the selection objective rather than discovered from model behavior. The independent content in Table 3 is the CoLA and CoNLL-2003 degradation, which uses external benchmarks, but the headline bias-reduction claim is forced by the shared metric.

full rationale

The paper does not rely on any author self-citation chain; the EAP method, STR corruption, datasets, and downstream benchmarks are all external or independently specified. The localization and stability analyses (Figures 2-5) are self-contained empirical findings: EAP scores could in principle have been diffuse, and nothing in the method forces the observed layer concentration or the instability under fine-tuning. The demographic sentiment-labeling pipeline is an external-validity risk (if DistilBERT is nationality-biased, the labels, L, EAP scores, and δBias all inherit the error), but that is a correctness threat, not circularity. The one genuine circularity is the debiasing evaluation: the top edges are selected as the edges with the largest effect on L2, and Table 3 then reports the drop in that same L2 as δBias. That reduction is inherited from the selection objective, so the headline 'removing these components reduces biased outputs' is not an independent test; the CoLA/NER degradation is the only independent evidence in that table. Score 6 reflects partial circularity with independent anchors elsewhere.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new entities. It depends on several assumptions: the validity of the EAP approximation, the accuracy of the external sentiment classifier, the validity of the corruption counterfactuals, and the adequacy of the top-k token probability metrics as measures of bias.

free parameters (3)
  • top_k tokens = 10
    Number of next-token predictions used in the bias metric; chosen because the authors observed bias does not change much beyond 10 tokens (Appendix A).
  • top edges ablated = 400 (GPT-2 Small), 1000 (GPT-2 Large), 3000 (Llama-2)
    Threshold for bias-relevant edges used in debiasing; chosen by model size without sensitivity analysis (Section 5, Table 3).
  • localization layer threshold = 20%
    Layers with more than 20% of important edges are highlighted in Figure 2; arbitrary display threshold.
assumptions (4)
  • domain assumption EAP scores approximate true causal importance of edges
    The paper relies on Syed et al. (2023) that gradient-based attribution patching is a faithful approximation, without independent verification on these models or tasks.
  • domain assumption DistilBERT sentiment labels for template sentences are correct
    The demographic bias metric depends on classifying predicted tokens as positive or negative using DistilBERT on completed sentences; accuracy on nationality templates is not validated.
  • domain assumption Corruption strategies C1/C2 create valid counterfactuals
    Replacing country names with 'Emirati' or profession names with 'broadcaster' is assumed to remove bias-relevant content while preserving syntax and semantic coherence.
  • domain assumption L1/L2 computed on top-k tokens capture bias
    The paper defines bias as skew in top-k token probabilities; no external validation that this matches human judgments of bias.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Dissecting Bias in LLMs: A Mechanistic Interpretability Perspective." pith.science (2026). https://pith.science/paper/VFW436TZ

@misc{pith2026250605166,
  author       = {Pith},
  title        = {Pith review of: Dissecting Bias in LLMs: A Mechanistic Interpretability Perspective},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VFW436TZ}},
  note         = {Machine review of arXiv:2506.05166}
}
read the original abstract

Large Language Models (LLMs) are known to exhibit social, demographic, and gender biases, often as a consequence of the data on which they are trained. In this work, we adopt a mechanistic interpretability approach to analyze how such biases are structurally represented within models such as GPT-2 and Llama2. Focusing on demographic and gender biases, we explore different metrics to identify the internal edges responsible for biased behavior. We then assess the stability, localization, and generalizability of these components across dataset and linguistic variations. Through systematic ablations, we demonstrate that bias-related computations are highly localized, often concentrated in a small subset of layers. Moreover, the identified components change across fine-tuning settings, including those unrelated to bias. Finally, we show that removing these components not only reduces biased outputs but also affects other NLP tasks, such as named entity recognition and linguistic acceptability judgment because of the sharing of important components with these tasks.

Figures

Figures reproduced from arXiv: 2506.05166 by the authors.

Figure 1
Figure 1. Circuit Diagram for Positive Demographic Bias in a) GPT-2 Small, b) GPT-2 Large, and c) Llama-2. Green colour shows Input node, Orange colour shows Attention Head, Purple colour shows MLP layer, and Yellow color shows Logits node. The description of different types of nodes can be found in [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Layerwise important edge distribution for demographic bias (DSS1) and Gender bias (GSS1) across different models (i.e. GPT-2 Small, GPT-2 Large, LLAMA-2 from left to right). (a) GPT-2 Small (b) GPT-2 Large (c) Llama-2 [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Drop in L2 value with % of Edge Ablation from GPT-2 Small, GPT-2 Large and Llama-2 across different configurations (i.e. DSS1pos, DSS2pos, DSS1neg, DSS2neg, GSS1pos, GSS2pos, GSS1neg, GSS2neg) [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Plot showing overlap of top edges by EAP scores for GPT-2 Small, GPT-2 Large and Llama-2 over different bias and sentence structure variation. Pretained_Pos Pretained_Neg Shakespeare_Pos Shakespeare_Neg Positive_Pos Positive_Neg Pretained_Pos Pretained_Neg Shakespeare_…
Figure 5
Figure 5. Figure 5: Plot Showing Overlap of top edges by EAP scores for Untuned vs Finetuned GPT-2 Small, GPT-2 Large, and Llama-2 in DSS1 (Demographic) configuration. Pretrained_Pos, Pretrained_Neg show positive or negative bias in pretrained LLMs. Shakespeare_Pos and Shakespeare_Neg den…
Figure 6
Figure 6. Figure 6: Overlap Results from SAE Approach (a) (b) (c) (d) (e) (f) (g) (h) (i) (j) (k) (l) [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: Circuit Diagram: (a) GPT-2-Small-DSS1-Positive, (b) GPT-2-Small-DSS2-Positive, (c) GPT-2- [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Analysing Moral Bias in Finetuned LLMs through Mechanistic Interpretability

    cs.CL 2025-10 conditional novelty 5.0 of 10

    The Knobe effect in fine-tuned LLMs is localized to mid-to-late transformer layers and can be removed by patching in pretrained activations at a single layer.

  2. Explainability of Large Language Models: Opportunities and Challenges toward Generating Trustworthy Explanations

    cs.CL 2025-10 conditional novelty 4.0 of 10

    LLM explanations split into local and mechanistic tracks; the paper argues they are trustworthy only if they pass causal and contrastive stress tests, adapt to the explainee, and satisfy eight trust principles.

Reference graph

Works this paper leans on

39 extracted references · 17 canonical work pages · cited by 2 Pith papers

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    write newline

    " write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence 'output.state := if if FUNCTION not #0 #1 if FUNCTION and 'skip pop #0 if FUNCTIO...

  4. [4]

    Stubbersfield

    Alberto Acerbi and Joseph M. Stubbersfield. Large language models show human-like content biases in transmission chain experiments. Proceedings of the National Academy of Sciences, 120 0 (44), 2023

    work page 2023

  5. [5]

    Science in the age of large language models

    Abeba Birhane, Atoosa Kasirzadeh, David Leslie, and Sandra Wachter. Science in the age of large language models. Nature Reviews Physics, April 2023

    work page 2023

  6. [6]

    Quantifying and reducing stereotypes in word embeddings

    Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, and Adam Kalai. Quantifying and reducing stereotypes in word embeddings. arXiv preprint arXiv:1606.06121, 2016 a

    arXiv 2016

  7. [7]

    Man is to computer programmer as woman is to homemaker? debiasing word embeddings

    Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. Advances in neural information processing systems, 29, 2016 b

    work page 2016

  8. [8]

    Identifying and adapting transformer-components responsible for gender bias in an english language model

    Abhijith Chintam, Rahel Beloch, Willem Zuidema, Michael Hanna, and Oskar Van Der Wal. Identifying and adapting transformer-components responsible for gender bias in an english language model. arXiv preprint arXiv:2310.12611, 2023

    arXiv

Show all 39 references
  1. [9]

    Towards automated circuit discovery for mechanistic interpretability

    Arthur Conmy, Augustine Mavor-Parker, Aengus Lynch, Stefan Heimersheim, and Adri \`a Garriga-Alonso. Towards automated circuit discovery for mechanistic interpretability. Advances in Neural Information Processing Systems, 36: 0 16318--16352, 2023

    2023

  2. [10]

    Gallegos, Ryan A

    Isabel O. Gallegos, Ryan A. Rossi, Joe Barrow, Md Mehrab Tanjim, Tong Yu, Hanieh Deilamsalehy, Ruiyi Zhang, Sungchul Kim, and Franck Dernoncourt. Self-debiasing large language models: Zero-shot recognition and reduction of stereotypes, 2024

    2024

  3. [11]

    Causal abstractions of neural networks

    Atticus Geiger, Hanson Lu, Thomas Icard, and Christopher Potts. Causal abstractions of neural networks. Advances in Neural Information Processing Systems, 34: 0 9574--9586, 2021

    2021

  4. [12]

    Multimodal neurons in artificial neural networks

    Gabriel Goh, Nick Cammarata, Chelsea Voss, Shan Carter, Michael Petrov, Ludwig Schubert, Alec Radford, and Chris Olah. Multimodal neurons in artificial neural networks. Distill, 6 0 (3): 0 e30, 2021

    2021

  5. [13]

    Localizing model behavior with path patching

    Nicholas Goldowsky-Dill, Chris MacLeod, Lucas Sato, and Aryaman Arora. Localizing model behavior with path patching. arXiv preprint arXiv:2304.05969, 2023

    2023 arXiv

  6. [14]

    C hat GPT based data augmentation for improved parameter-efficient debiasing of LLM s

    Pengrui Han, Rafal Kocielnik, Adhithya Saravanan, Roy Jiang, Or Sharir, and Anima Anandkumar. C hat GPT based data augmentation for improved parameter-efficient debiasing of LLM s. In Proceedings of the Fourth Workshop on Language Technology for Equality, Diversity, Inclusion,...

    2024

  7. [15]

    distilbert-base-uncased-finetuned-sst-2-english (revision bfdd146), 2022

    HF Canonical Model Maintainers . distilbert-base-uncased-finetuned-sst-2-english (revision bfdd146), 2022. URL https://huggingface.co/distilbert-base-uncased-finetuned-sst-2-english

    2022

  8. [16]

    Shovon, and Gene Kim

    Mahammed Kamruzzaman, Md. Shovon, and Gene Kim. Investigating subtler biases in LLM s: Ageism, beauty, institutional, and nationality bias in generative models. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Findings of the Association for Computational Linguistics...

    2024

  9. [17]

    The impact of debiasing on the performance of language models in downstream tasks is underestimated

    Masahiro Kaneko, Danushka Bollegala, and Naoaki Okazaki. The impact of debiasing on the performance of language models in downstream tasks is underestimated. In Jong C. Park, Yuki Arase, Baotian Hu, Wei Lu, Derry Wijaya, Ayu Purwarianti, and Adila Alfa Krisnadhi, editors, Proc...

    2023

  10. [18]

    Backward lens: Projecting language model gradients into the vocabulary space

    Shahar Katz, Yonatan Belinkov, Mor Geva, and Lior Wolf. Backward lens: Projecting language model gradients into the vocabulary space. arXiv preprint arXiv:2402.12865, 2024

    2024 arXiv

  11. [19]

    Linear representations of political perspective emerge in large language models

    Junsol Kim, James Evans, and Aaron Schein. Linear representations of political perspective emerge in large language models. arXiv preprint arXiv:2503.02080, 2025

    2025 arXiv

  12. [20]

    Gender bias and stereotypes in large language models

    Hadas Kotek, Rikker Dockum, and David Sun. Gender bias and stereotypes in large language models. In Proceedings of The ACM Collective Intelligence Conference, page 12–24, New York, NY, USA, 2023. Association for Computing Machinery. ISBN 9798400701139

    2023

  13. [21]

    Sparse feature circuits: Discovering and editing interpretable causal graphs in language models

    Samuel Marks, Can Rager, Eric J Michaud, Yonatan Belinkov, David Bau, and Aaron Mueller. Sparse feature circuits: Discovering and editing interpretable causal graphs in language models. In The Thirteenth International Conference on Learning Representations, 2025. URL https://o...

    2025

  14. [22]

    Locating and editing factual associations in gpt

    Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in gpt. Advances in Neural Information Processing Systems, 35: 0 17359--17372, 2022

    2022

  15. [23]

    Progress measures for grokking via mechanistic interpretability

    Neel Nanda, Lawrence Chan, Tom Lieberum, Jess Smith, and Jacob Steinhardt. Progress measures for grokking via mechanistic interpretability. arXiv preprint arXiv:2301.05217, 2023

    2023 arXiv

  16. [24]

    Nationality bias in text generation

    Pranav Narayanan Venkit, Sanjana Gautam, Ruchi Panchanadikar, Ting-Hao Huang, and Shomir Wilson. Nationality bias in text generation. In Andreas Vlachos and Isabelle Augenstein, editors, Proceedings of the 17th Conference of the European Chapter of the Association for Computat...

    2023

  17. [25]

    Biases in large language models: Origins, inventory, and discussion

    Roberto Navigli, Simone Conia, and Bj\" o rn Ross. Biases in large language models: Origins, inventory, and discussion. J. Data and Information Quality, 15 0 (2), June 2023. ISSN 1936-1955

    2023

  18. [26]

    Mechanistic interpretability, variables, and the importance of interpretable bases

    Chris Olah. Mechanistic interpretability, variables, and the importance of interpretable bases. transformer circuits thread (june 27), 2022

    2022

  19. [27]

    Zoom in: An introduction to circuits

    Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter. Zoom in: An introduction to circuits. Distill, 5 0 (3): 0 e00024--001, 2020

    2020

  20. [28]

    Gender biases in automatic evaluation metrics for image captioning

    Haoyi Qiu, Zi-Yi Dou, Tianlu Wang, Asli Celikyilmaz, and Nanyun Peng. Gender biases in automatic evaluation metrics for image captioning. In The 2023 Conference on Empirical Methods in Natural Language Processing, 2023

    2023

  21. [29]

    Language models are unsupervised multitask learners

    Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. 2019

    2019

  22. [30]

    Introduction to the conll-2003 shared task: Language-independent named entity recognition

    Erik F Sang and Fien De Meulder. Introduction to the conll-2003 shared task: Language-independent named entity recognition. arXiv preprint cs/0306050, 2003

    2003 arXiv

  23. [31]

    Investigating gender bias in large language models through text generation

    Shweta Soundararajan and Sarah Jane Delany. Investigating gender bias in large language models through text generation. In Mourad Abbas and Abed Alhakim Freihat, editors, Proceedings of the 7th International Conference on Natural Language and Speech Processing (ICNLSP 2024), p...

    2024

  24. [32]

    Attribution patching outperforms automated circuit discovery

    Aaquib Syed, Can Rager, and Arthur Conmy. Attribution patching outperforms automated circuit discovery. arXiv preprint arXiv:2310.10348, 2023

    2023 arXiv

  25. [33]

    Attribution patching outperforms automated circuit discovery

    Aaquib Syed, Can Rager, and Arthur Conmy. Attribution patching outperforms automated circuit discovery. In Yonatan Belinkov, Najoung Kim, Jaap Jumelet, Hosein Mohebbi, Aaron Mueller, and Hanjie Chen, editors, Proceedings of the 7th BlackboxNLP Workshop: Analyzing and Interpret...

    2024

  26. [34]

    Llama 2: Open foundation and fine-tuned chat models, 2023

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, W...

    2023 arXiv

  27. [35]

    Investigating gender bias in language models using causal mediation analysis

    Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart Shieber. Investigating gender bias in language models using causal mediation analysis. Advances in neural information processing systems, 33: 0 12388--12401, 2020

    2020

  28. [36]

    Interpretability in the wild: a circuit for indirect object identification in gpt-2 small

    Kevin Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. Interpretability in the wild: a circuit for indirect object identification in gpt-2 small. arXiv preprint arXiv:2211.00593, 2022

    2022 arXiv

  29. [37]

    Neural network acceptability judgments

    A Warstadt. Neural network acceptability judgments. arXiv preprint arXiv:1805.12471, 2019

    2019 arXiv

  30. [38]

    Interpretability at scale: Identifying causal mechanisms in alpaca

    Zhengxuan Wu, Atticus Geiger, Thomas Icard, Christopher Potts, and Noah Goodman. Interpretability at scale: Identifying causal mechanisms in alpaca. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https://openreview.net/forum?id=nRfClnMhVX

    2023

  31. [39]

    Towards best practices of activation patching in language models: Metrics and methods, 2024

    Fred Zhang and Neel Nanda. Towards best practices of activation patching in language models: Metrics and methods, 2024. URL https://arxiv.org/abs/2309.16042

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.