Pith. sign in

REVIEW 3 major objections 6 minor 30 references

Identifying a Circuit for Verb Conjugation in GPT-2

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper isolates a 12-attention-head circuit in GPT-2 Small that matches most of the full model's subject-verb agreement performance, while showing that harder linguistic settings require far larger circuits.

desk verdict A well-scoped but undersupported circuit for subject-verb agreement: the key 0.65-vs-0.70 accuracy claim is made on the same prompts used to select the heads, so selection overfitting makes the sparse-circuit result unverified. read the letter →

arxiv 2506.22105 v1 pith:NHLECA2P submitted 2025-06-27 cs.CL cs.LG

David Demitri Africa This is my paper · ORCID
classification cs.CLcs.LG
keywords mechanisticinterpretabilitycircuitdiscoverysubject-verbagreementattentionheadspathpatchingresampleablationGPT-2transformer
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to show that GPT-2 Small's subject-verb agreement behavior is not spread uniformly across the network: a candidate circuit of just 12 attention heads, out of 144, reproduces most of the model's performance on simple prompts, reaching 0.65 accuracy versus the full model's 0.70. It then expands that circuit setting by setting and finds that complexity changes the picture: plural, negated, and pronoun constructions need only a few extra heads but the attention-head-only circuits stall well below the full model, while prefix, past-tense, irregular, and combined settings recruit up to 125 heads before approaching full accuracy. The point of isolating such circuits is to turn a black-box behavior into a checkable mechanism: which heads carry the agreement signal, whether the same heads are reused across constructions, and where the remaining behavior must be explained by other components such as the multilayer perceptron layers.

What carries the argument

The load-bearing object is the candidate circuit: a set of attention-head pairs, indexed by layer and head number, selected by ranking each head's causal effect measured with path patching under resample ablation, then greedily adding heads that improve the circuit's accuracy on the logit-difference metric until gains fall below a tolerance relative to the full model. Path patching replaces one head's activation with a resampled activation from counterfactual prompts, such as switching the subject's plurality, and the change in the correct-minus-incorrect verb logit is the head effect. The greedy iterative search on top of this ranking is what turns per-head effects into a circuit, and the same procedure, seeded from the base circuit, produces the expanded circuits for each linguistic setting.

What would settle it

Run a global search, exhaustive over small subsets or stochastic over all 144 heads, on the same base data and check whether a set of fewer than 12 heads, or a set that does not contain the paper's 12, matches or beats the full model's 0.70 accuracy; if it does, the identified circuit is not the minimal agreement mechanism. Alternatively, verify that patching just the 12 heads' activations from a plural-subject prompt flips the predicted verb on held-out base prompts; if it does not, the circuit is not causally responsible.

Watch

Extended reading notes

Core claim

The central discovery claimed is that subject-verb agreement in GPT-2 Small has a sparse, localizable attention-head circuit for the simplest case, and that this sparsity evaporates as linguistic complexity grows. In the author's controlled setup, a 12-head circuit spanning 7 layers attains 0.65 accuracy on the base singular-name task versus the full model's 0.70, and the same 12 heads are reused across all expansions. When settings are added one at a time, the greedy search adds at most 23 heads for plural but only reaches 0.53 accuracy against the full model's 0.88, whereas irregular verbs need 97 heads to reach 0.82 against 0.93, and the combined complex setting needs 125 heads for 0.80 against 1.00. The paper interprets the poor attention-only performance on plural, negation, and pronoun settings as evidence that those behaviors live partly outside attention heads, and the large expansions for prefix, past-tense, irregular, and combined settings as evidence that contextual and morphological processing is distributed across many heads. It also proposes that the combined setting's high full-model accuracy may come from redundant circuits or stronger dataset cues, not from a single compact algorithm.

Load-bearing premise

All the circuit sizes and accuracy figures depend on the assumption that a greedy, head-by-head search over individually measured effects recovers the network's real agreement mechanism; if heads only matter in combinations that the greedy order never visits, the reported circuits are artifacts.

Editorial extensions

If this is right

  • The base subject-verb agreement behavior of GPT-2 Small can be almost fully reproduced by 12 attention heads spanning 7 layers, with 0.65 accuracy versus the full model's 0.70.
  • Plural, negated, and pronoun constructions are not captured well by attention-head circuits: adding 10 to 17 heads still leaves circuit accuracy at 0.53 to 0.57 while the full model reaches 0.88 to 0.91, implying the missing computation lives outside attention heads.
  • Context-heavy constructions recruit much larger circuits: prefixes need 82 heads, past tense 55, irregular verbs 97, and the combined setting 125, reaching 0.78 to 0.82 against full-model scores of 0.86 to 1.00.
  • A small set of shared heads, especially in layer 11 and layer 0, appears in every setting and causes large performance drops when ablated, suggesting a common syntactic core.
  • The full model's best accuracy occurs on the most complex combined setting, which the paper attributes to redundancy across many heads or stronger combined cues rather than a compact mechanism.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the reported gap between attention-only circuits and the full model on plural, negation, and pronoun settings is consistent with multilayer perceptron layers carrying number and negation information; a direct test would patch MLP outputs the same way and see whether the circuit closes the gap.
  • Beyond the paper: the 12-head circuit may be tracking the first token or a proper-noun position rather than grammatical number; the paper itself notes this possibility, so a control that moves the subject later in the prompt would separate positional attention from true agreement.
  • Beyond the paper: the near-perfect full-model accuracy on the combined setting could reflect stronger logit margins on multi-cue prompts rather than a different mechanism; comparing circuit accuracy across settings at matched margin or difficulty would test whether the complexity paradox is real.
  • Beyond the paper: if the sparsity-to-diffusion pattern generalizes, it predicts that the same greedy procedure applied to other syntactic features, such as gender agreement or case, would find tiny base circuits that grow sharply when the feature interacts with other cues.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The manuscript reports a mechanistic interpretability study of subject–verb agreement in GPT-2 Small. Using a controlled prompt grid that varies plurality, negation, prefix, pronoun type, tense, and verb regularity, the author computes per-head contributions via path patching with resample ablation and then greedily selects attention heads into a 'base circuit' for simple singular prompts, subsequently expanding that circuit for each more complex setting. The headline results are that a 12-head circuit attains 0.65 accuracy versus 0.70 for the full model on the base task, and that expanded circuits for complex settings require far more heads (up to 125) while often still remaining far below full-model accuracy. The paper also gives a qualitative, attention-heatmap-based categorization of head functions and provides full per-permutation results in an appendix.

Significance. If validated, the work would contribute a concrete case study on the sparsity and scaling of syntactic circuits in a transformer language model, and the open-source implementation and detailed appendix tables are useful for replication. The study's design choices are largely sensible: it uses resample ablation, a controlled stimulus set, and an explicit, if heuristic, circuit-search procedure, and its limitations section is unusually candid about greedy search, attention-only analysis, and manual interpretation bias. The central quantitative claim, however, is currently not established: circuit accuracies appear to be evaluated on the same prompts used for head selection, the sample size per condition (n=100) makes the reported accuracy differences statistically fragile, and several expanded-circuit results contradict the paper's own complexity narrative. These issues are fixable with additional validation and reframing, but they are load-bearing.

major comments (3)
  1. [§3.1, §3.4, §4.1] The paper does not state whether the 100 prompts per condition used to report circuit accuracy are disjoint from the prompts used to select heads in the greedy search. If they are the same set, every reported circuit accuracy (including the headline 0.65 for the 12-head base circuit) is a fitted quantity, not an independent evaluation. With n=100 per condition, the binomial standard error is about 0.05, so the 5-point gap between the circuit's 0.65 and the full model's 0.70 is within noise. The authors should (i) explicitly disclose the split, (ii) evaluate on held-out prompts or use cross-validation, and (iii) report paired confidence intervals or at least standard errors for all accuracy figures.
  2. [Table 3, §4.2] The expanded circuits for the plural, negation, and pronoun settings reach lower accuracy (0.53, 0.53, and 0.57) than the 12-head base circuit (0.65), despite adding 11, 17, and 10 heads respectively. This pattern directly undermines the abstract's claim that 'substantially more [components] for more complex settings' is the route to near-model performance, since the additions do not close the gap in these settings. The explanations offered (diffuse encoding, redundancy, missing nonlinear interactions) are plausible but untested. The authors should either reframe the conclusion as 'attempted expansions failed to recover performance' and discuss why, or add control analyses (e.g., random head subsets of matched size, or an alternative search such as ACDC) to support the complexity-needs-more-heads interpretation.
  3. [§4.2, 'Complex' row] The full model's perfect 1.00 accuracy on the combined setting is reported without explanation, and the paper itself flags it as 'unexpected' and potentially due to dataset artifacts. Because the expanded circuit is optimized against full-model performance, an artifact-inflated target makes the 'Complex' comparison uninformative for the paper's central claim. The author should inspect the ALL prompts for confounds (e.g., imbalanced correct versus incorrect forms, token-frequency effects, or cues such as 'did not' that may trivially determine the base form) and either validate the setting or exclude it from the conclusions; the current text should not present the 125-head circuit as evidence for the complexity-scaling claim.
minor comments (6)
  1. [Abstract] The abstract's phrase 'component-token pairs' is not accurate: the analysis identifies attention heads, not head-token pairs; please rephrase to 'attention heads' or specify the granularity.
  2. [§3.1] No uncertainty or variability is reported for any accuracy or logit-difference metric; adding standard deviations or confidence intervals would be particularly important given the small per-condition sample size.
  3. [Figure 2 caption] The caption says circuit accuracy 'correlates directly' with size and complexity, but the plural/negation/pronoun rows in Table 3 show decreasing accuracy with added heads; the caption should be revised to avoid overstating the pattern.
  4. [§4.3] For consistency with the rest of the paper, use the (layer, head) notation (e.g., (11,6)) instead of the prose 'layer 11 heads 6, 4, and 7'.
  5. [Table 1] The 'Failure Implication' column describes each condition as a failure mode of GPT-2, but Table 2 shows accuracies mostly above 0.6 and often above 0.9; the column should be conditional (e.g., 'If GPT-2 fails here, this would imply...').
  6. [§4.4] The functional categorization in Table 4 is based on only four example sentences and is acknowledged as subjective; this is acceptable as hypothesis generation, but the text should state more clearly that these categories are illustrative and not validated.

Circularity Check

2 steps flagged · score 6.0 of 10

Base and expanded circuit accuracies are the same objective optimized by the greedy head-selection loop, so the headline 'near-model performance' partly restates the stopping rule rather than an independent prediction.

  1. fitted input called prediction [Section 3.4 (greedy iterative circuit search) and Section 4.1 (base circuit result)]
    "I greedily sample node-by-node starting from the highest ranked attention heads, and take in heads which improve the accuracy of the preexisting circuit. I stop when circuit performance no longer meaningfully improves (Fcircuit) within a specified tolerance relative to the full model’s performance (Ffull)."

    The 12-head base circuit is selected by greedily maximizing accuracy on the base SVA task, and Section 4.1 then reports that this circuit 'achieves an accuracy of 0.65, which is close to the full model’s accuracy of 0.70.' The reported accuracy is the same metric used as the selection objective, on prompts that the paper never states are held out from selection. The claimed 'near-model performance' is therefore partly a restatement of the stopping rule, which defines success as reaching the full model's accuracy within tolerance. With 100 samples per condition, the 0.05 accuracy gap is also within binomial sampling noise, so the base claim is not independently testable as written.

  2. fitted input called prediction [Section 3.5 (iterative circuit expansion) and Table 3 (expanded circuits)]
    "For each additional setting (e.g., negation, pronoun substitution, presence of a prefix, irregular verbs, or the full unfiltered dataset), I: 1. Evaluate the full model’s performance on the setting. 2. Evaluate the performance of the base circuit ... on the same setting. 3. Iteratively search for and add extra heads from the candidate pool that significantly reduce the performance gap relative to the full model."

    The expanded circuits are grown to close the accuracy gap on exactly those settings, and Table 3 then reports 'Circuit Accuracy / Full Model Accuracy' for each setting. The per-setting accuracy gap is the objective function of the search, so the expanded-circuit results are fitted quantities, not out-of-sample evaluations. The conclusion that 'substantially more' components are needed for complex settings is thus partly generated by the same optimization loop that produces the numbers; independent content would require validation on held-out prompts, which the paper does not state.

full rationale

This is not a self-citation or uniqueness-theorem case: the paper relies on standard circuit-discovery methods from Wang et al. and Conmy et al., and no load-bearing claim is justified by the author's own prior work. The circularity, where present, is the selection-evaluation conflation. Section 3.4 defines the circuit as the greedy set of heads that improve accuracy, and Sections 4.1 and 4.2 report the accuracy of that same selected set without stating any held-out split. The base result that 'only a small fraction ... is needed to achieve near-model performance' is especially close to the stopping criterion, which explicitly halts when circuit performance is within a specified tolerance of the full model. The expanded-circuit tables similarly report the gap that the greedy expansion was optimized to reduce. There is some independent content: the identity and number of heads actually selected is not logically forced, the base circuit is also evaluated on other settings, and the manual head-function analysis is separate from the accuracy loop. But the central quantitative evidence for the sparse-circuit claim is not an independent prediction on the base task, and the paper's own limitations section discusses greedy-search incompleteness but not the absence of held-out validation. Score 6 reflects partial circularity in the central reported accuracies, while stopping short of 8 because head identities and transfer behavior retain some empirical content.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new entities, forces, or particles. Its contributions rely on standard mech interp assumptions and on methodological free parameters (greedy tolerance, resample pool, sample counts) that are not fully specified. The main ad hoc assumption is that the greedily found subgraph represents the true causal circuit for the task.

free parameters (3)
  • Greedy search stopping tolerance
    Section 3.4 stops adding heads when circuit performance 'no longer meaningfully improves' within a specified tolerance relative to full model performance. The tolerance value is not reported, yet it determines circuit size and all accuracy figures.
  • Resample ablation pool size
    Resample ablation replaces head outputs with randomly selected activations from a pre-collected pool (Section 3.3). The pool size and sampling procedure are not specified, affecting the reliability of the estimated head effects.
  • Number of samples per permutation = 100
    The paper aggregates logit differences over 100 samples per permutation (Section 3.1). This fixed sample size is ad hoc and small enough that accuracy point estimates have wide confidence intervals, which are not provided.
assumptions (3)
  • domain assumption Path patching reveals the causal contribution of individual attention heads to model predictions.
    Standard mechanistic interpretability assumption, cited to Wang et al. (2022). It presumes that replacing a head's activation with a counterfactual value isolates that head's causal role, which can fail under nonlinear interactions.
  • domain assumption Resample ablation preserves realistic activation distributions and is a valid counterfactual intervention.
    The paper relies on resample ablation (Section 3.3) following Geiger et al. (2024). This assumes that sampled activations from a pool are a faithful counterfactual baseline, an assumption the paper itself notes may still introduce unrealistic patterns.
  • ad hoc to paper The greedily constructed circuit is a sufficient and meaningful subnetwork for subject-verb agreement.
    Section 3.4 defines the circuit as the result of greedy head addition. The paper admits in Section 5.1 that this does not guarantee optimality or necessity, yet all conclusions about circuit size and performance depend on this assumption.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Identifying a Circuit for Verb Conjugation in GPT-2." pith.science (2026). https://pith.science/paper/NHLECA2P

@misc{pith2026250622105,
  author       = {Pith},
  title        = {Pith review of: Identifying a Circuit for Verb Conjugation in GPT-2},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NHLECA2P}},
  note         = {Machine review of arXiv:2506.22105}
}
read the original abstract

I implement a procedure to isolate and interpret the sub-network (or "circuit") responsible for subject-verb agreement in GPT-2 Small. In this study, the model is given prompts where the subject is either singular (e.g. "Alice") or plural (e.g. "Alice and Bob"), and the task is to correctly predict the appropriate verb form ("walks" for singular subjects, "walk" for plural subjects). Using a series of techniques-including performance verification automatic circuit discovery via direct path patching, and direct logit attribution- I isolate a candidate circuit that contributes significantly to the model's correct verb conjugation. The results suggest that only a small fraction of the network's component-token pairs is needed to achieve near-model performance on the base task but substantially more for more complex settings.

Figures

Figures reproduced from arXiv: 2506.22105 by the authors.

Figure 1
Figure 1. Heatmap of average attention head effects on the base subject–verb agreement task. Pos [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Accuracy comparison between the full GPT-2 model and the iteratively expanded circuits [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Attention heatmaps illustrating a representative diffuse subject-scanner (layer 0, head 8). [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

30 extracted references · 11 canonical work pages

  1. [1]

    Broken agreement

    Kathryn Bock and Carol A Miller. Broken agreement. Cognitive Psychology, 23 0 (1): 0 45--93, 1991. ISSN 0010-0285. doi:https://doi.org/10.1016/0010-0285(91)90003-7. URL https://www.sciencedirect.com/science/article/pii/0010028591900037

    arXiv 1991

  2. [2]

    Thread: Circuits

    Nick Cammarata, Shan Carter, Gabriel Goh, Chris Olah, Michael Petrov, Ludwig Schubert, Chelsea Voss, Ben Egan, and Swee Kiat Lim. Thread: Circuits. Distill, 2020. doi:10.23915/distill.00024. https://distill.pub/2020/circuits

    doi:10.23915/distill.00024 2020

  3. [3]

    Red teaming deep neural networks with feature synthesis tools

    Stephen Casper, Tong Bu, Yuxiao Li, Jiawei Li, Kevin Zhang, Kaivalya Hariharan, and Dylan Hadfield-Menell. Red teaming deep neural networks with feature synthesis tools. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Processing Systems, volume 36, pp.\ 80470--80516. Curran Associates, Inc., 20...

    work page 2023

  4. [4]

    Towards automated circuit discovery for mechanistic interpretability

    Arthur Conmy, Augustine Mavor-Parker, Aengus Lynch, Stefan Heimersheim, and Adri\` a Garriga-Alonso. Towards automated circuit discovery for mechanistic interpretability. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Processing Systems, volume 36, pp.\ 16318--16352. Curran Associates, Inc., 2...

    work page 2023

  5. [5]

    Introduction: Canonical agreement

    Greville G Corbett. Introduction: Canonical agreement. In Agreement, pp.\ 1--34. Cambridge University Press, 2006

    work page 2006

  6. [6]

    Making syntax of sense: number agreement in sentence production

    Kathleen M Eberhard, J Cooper Cutting, and Kathryn Bock. Making syntax of sense: number agreement in sentence production. Psychological review, 112 0 (3): 0 531, 2005

    work page 2005

  7. [7]

    A mathematical framework for transformer circuits

    Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olah. A...

    2021

  8. [8]

    Object interference in subject--verb agreement: The role of intermediate traces of movement

    Julie Franck, Gabriela Soare, Ulrich H Frauenfelder, and Luigi Rizzi. Object interference in subject--verb agreement: The role of intermediate traces of movement. Journal of memory and language, 62 0 (2): 0 166--182, 2010

    work page 2010

Show all 30 references
  1. [9]

    How does gpt-2 predict acronyms? extracting and understanding a circuit via mechanistic interpretability

    Jorge Garc \' a-Carrasco, Alejandro Mat \'e , and Juan Carlos Trujillo. How does gpt-2 predict acronyms? extracting and understanding a circuit via mechanistic interpretability. In International Conference on Artificial Intelligence and Statistics, pp.\ 3322--3330. PMLR, 2024

    2024

  2. [10]

    Causal abstraction: A theoretical foundation for mechanistic interpretability, 2024

    Atticus Geiger, Duligur Ibeling, Amir Zur, Maheep Chaudhary, Sonakshi Chauhan, Jing Huang, Aryaman Arora, Zhengxuan Wu, Noah Goodman, Christopher Potts, and Thomas Icard. Causal abstraction: A theoretical foundation for mechanistic interpretability, 2024. URL https://arxiv.org...

    2024 arXiv

  3. [11]

    Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space

    Mor Geva, Avi Caciularu, Kevin Wang, and Yoav Goldberg. Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.), Proceedings of the 2022 Conference on Empirical Methods in Natura...

    2022 doi

  4. [12]

    Assessing bert's syntactic abilities

    Yoav Goldberg. Assessing bert's syntactic abilities. arXiv preprint arXiv:1901.05287, 2019

    1901 arXiv

  5. [13]

    Localizing model behavior with path patching, 2023

    Nicholas Goldowsky-Dill, Chris MacLeod, Lucas Sato, and Aryaman Arora. Localizing model behavior with path patching, 2023. URL https://arxiv.org/abs/2304.05969

    2023 arXiv

  6. [14]

    How does gpt-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model

    Michael Hanna, Ollie Liu, and Alexandre Variengien. How does gpt-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model. Advances in Neural Information Processing Systems, 36: 0 76033--76060, 2023

    2023

  7. [15]

    Negated and misprimed probes for pretrained language models: Birds can talk, but cannot fly

    Nora Kassner and Hinrich Sch \"u tze. Negated and misprimed probes for pretrained language models: Birds can talk, but cannot fly. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault (eds.), Proceedings of the 58th Annual Meeting of the Association for Computatio...

    2020 doi

  8. [16]

    Assessing the ability of lstms to learn syntax-sensitive dependencies

    Tal Linzen, Emmanuel Dupoux, and Yoav Goldberg. Assessing the ability of lstms to learn syntax-sensitive dependencies. Transactions of the Association for Computational Linguistics, 4: 0 521--535, 2016

    2016

  9. [17]

    Identifying a preliminary circuit for predicting gendered pronouns in gpt-2 small

    Chris Mathwin, Guillaume Corlouer, Esben Kran, Fazl Barez, and Neel Nanda. Identifying a preliminary circuit for predicting gendered pronouns in gpt-2 small. URL: https://itch. io/jam/mechint/rate/1889871, 2023

    2023

  10. [18]

    Locating and editing factual associations in gpt

    Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in gpt. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS '22, Red Hook, NY, USA, 2022. Curran Associates Inc. ISBN 9781713871088

    2022

  11. [19]

    Zoom in: An introduction to circuits

    Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter. Zoom in: An introduction to circuits. Distill, 2020. doi:10.23915/distill.00024.001. https://distill.pub/2020/circuits/zoom-in

    2020 doi

  12. [20]

    In-context learning and induction heads

    Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Scott Johnston, Andy Jones, Jackson Kernion, Liane Lovitt, Kam...

    2022

  13. [21]

    Language models are unsupervised multitask learners

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 1 0 (8): 0 9, 2019

    2019

  14. [22]

    Language models are not naysayers: an analysis of language models on negation benchmarks

    Thinh Hung Truong, Timothy Baldwin, Karin Verspoor, and Trevor Cohn. Language models are not naysayers: an analysis of language models on negation benchmarks. In Alexis Palmer and Jose Camacho-collados (eds.), Proceedings of the 12th Joint Conference on Lexical and Computation...

    2023 doi

  15. [23]

    Interpretability in the wild: a circuit for indirect object identification in GPT -2 small

    Kevin Ro Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. Interpretability in the wild: a circuit for indirect object identification in GPT -2 small. In The Eleventh International Conference on Learning Representations, 2022. URL https://openrevi...

    2022

  16. [24]

    Blimp: The benchmark of linguistic minimal pairs for english

    Alex Warstadt, Alicia Parrish, Haokun Liu, Anhad Mohananey, Wei Peng, Sheng-Fu Wang, and Samuel R Bowman. Blimp: The benchmark of linguistic minimal pairs for english. Transactions of the Association for Computational Linguistics, 8: 0 377--392, 2020

    2020

  17. [25]

    Knowledge circuits in pretrained transformers

    Yunzhi Yao, Ningyu Zhang, Zekun Xi, Mengru Wang, Ziwen Xu, Shumin Deng, and Huajun Chen. Knowledge circuits in pretrained transformers. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https://openreview.net/forum?id=YVXzZNxcag

    2024

  18. [26]

    Word frequency does not predict grammatical knowledge in language models

    Charles Yu, Ryan Sie, Nicolas Tedeschi, and Leon Bergen. Word frequency does not predict grammatical knowledge in language models. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (eds.), Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processi...

    2020 doi

  19. [27]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  20. [28]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  21. [29]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  22. [30]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.