Pith. sign in

REVIEW 4 major objections 6 minor 59 references

Disjoint Processing Mechanisms of Hierarchical and Linear Grammars in Large Language Models

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Pretrained LLMs process hierarchical and linear grammars with largely separate internal components.

desk verdict A serious localization study whose central disjointness claim is undermined by a stimulus confound and by non-significant linear-side ablations. read the letter →

arxiv 2501.08618 v1 pith:EOTUPBEA submitted 2025-01-15 cs.CL cs.AI

Aruna Sankaranarayanan , Dylan Hadfield-Menell , Aaron Mueller This is my paper · ORCID
classification cs.CLcs.AI MSC 68T50
keywords hierarchicalgrammarlinearfunctionalspecializationmechanisticinterpretabilityattributionpatchinggrammaticalityjudgmentlargelanguagemodelsJabberwockysentences
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether a general-purpose learner exposed only to text can acquire the kind of functional segregation that human brains show for hierarchical language structure. Using synthetic grammars in English, Italian, Japanese, and nonce words, it first shows that pretrained LLMs are more accurate at grammaticality judgments for hierarchical than for linear/positional rules. It then locates the top 1% of MLP and attention neurons most influential for each judgment via attribution patching, and reports that hierarchy-selective neurons overlap strongly across hierarchical grammars but only weakly with linear-selective neurons; ablating them selectively degrades hierarchical judgments. Hierarchy-selective components also drive judgments on meaningless Jabberwocky sentences. The paper concludes that functional specialization toward hierarchical syntax can emerge from distributional exposure alone, and that the responsible components are localizable and partially abstract.

What carries the argument

The central object is a controlled grammar battery: eighteen synthetic grammars, with three hierarchical structures (declarative, subordinate, passive) and three linear structures (negation, inversion, and a language-specific third rule such as wh-word insertion, last-noun agreement, or past-tense placement), each generated in English, Italian, Japanese, and again with nonce words. On each grammar the model performs an in-context grammaticality judgment, and component importance is scored by attribution patching, a first-order Taylor approximation of the indirect effect of each MLP and attention neuron on the logit difference between the correct and incorrect answer tokens. The top 1% of neurons by estimated effect define the hierarchy-sensitive set H and the linearity-sensitive set L; pairwise overlap between these sets and mean-activation ablation experiments test whether the sets are causally distinct.

What would settle it

Construct a version of the experiment where linear positives are as natural-sounding as hierarchical positives, using attested word-order variants, while hierarchical negatives are made exactly as unnatural as the linear negatives; if the accuracy gap and the hierarchical-versus-linear component disjointness then vanish, the central claim would be falsified.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that pretrained large language models contain largely disjoint, causally verified component sets for processing hierarchical versus linear grammars. Across six open-weight models, the top 1% of neurons by estimated indirect effect on hierarchical grammaticality judgments show high pairwise overlap within the hierarchical family and across English, Italian, and Japanese, but significantly lower overlap with the top 1% for linear grammars. Ablating the hierarchy-sensitive set to its mean activation reduces accuracy on hierarchical inputs more than ablating the linear-sensitive set or a random set, and the reverse holds for linear inputs. The same hierarchy-sensitive components also influence grammaticality judgments on nonce-word sentences, which the paper takes as evidence that the specialization tracks abstract structure rather than lexical meaning or training-distribution familiarity.

Load-bearing premise

The load-bearing premise is that the only systematic difference between hierarchical and linear stimuli is structural type; but because hierarchical positives are natural word-order sentences and hierarchical negatives are made by swapping the final two words, the models might be distinguishing natural from scrambled word order rather than hierarchy from linearity, a confound the paper acknowledges only in a footnote.

Editorial extensions

If this is right

  • If correct, the paper implies that pretrained LLMs develop localizable subnetworks specialized for hierarchical syntax, so syntax processing is not smeared uniformly across the network.
  • Hierarchy-selective components are shared across languages and transfer to nonce inputs, implying the specialization is a structural abstraction rather than a lexicon effect.
  • The selective ablation results give causal, not merely correlational, evidence that these component sets are functionally distinct.
  • The selectivity is not uniform: smaller models and Japanese inputs show weaker, less selective effects, indicating that specialization strengthens with model scale and with language prevalence in the training distribution.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural next experiment would balance word-order naturalness directly: construct linear-positive sentences using word orders that actually occur in some dialect or register, and hierarchical negatives using swaps that preserve attested word order; if the accuracy gap and component disjointness then vanish, the results would be explained by natural-versus-artificial order rather than hierarchy-ve
  • The same attribution-patching protocol could test other structural contrasts, such as center-embedding, cross-serial dependencies, or artificial grammars matched for surprisal, to see whether hierarchy is the operative variable or a proxy for some other distributional property.
  • Because the analysis operates on individual neurons and attention outputs, future work with more fine-grained units might reveal that the 'disjoint' sets are not separate mechanisms but different overlapping multifunctional circuits; the localization claim may be coarser than the true causal structure.
  • If confirmed, the finding suggests LLM pretraining could serve as a controlled testbed for how functional specialization in language arises from exposure, complementing human neuroimaging studies where such controlled exposure is impossible.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper asks whether pretrained LLMs develop separable mechanistic resources for processing hierarchical versus linear/positional grammatical structure, inspired by Musso et al. (2003). It constructs three hierarchical and three linear grammars in English, Italian, Japanese, and nonce versions, evaluates six LLMs on in-context grammaticality judgments, localizes the top 1% of attention/MLP units by attribution patching, measures pairwise overlap between grammars, and tests causal specificity through ablations. The authors report higher accuracy on hierarchical than linear grammars for English and Italian, greater component overlap within hierarchical grammars than across hierarchical-linear pairs, and selective ablation effects for hierarchical components, including on Jabberwocky stimuli; they conclude that LLMs acquire localizable, largely disjoint processing mechanisms for hierarchical versus linear structure from distributional exposure alone.

Significance. If the conclusions are valid, the paper would be an important contribution to mechanistic interpretability and to debates about inductive biases in language models: it would show functional specialization for syntax-like structure arising in general-purpose sequence models without explicit human-like biases. The study has real strengths: it spans six open-weight models, three natural languages plus Jabberwocky controls; attribution patching makes component localization computationally tractable; ablations go beyond correlational overlap; the authors state that data and code are released; and the Limitations section candidly discusses polysemanticity, task specificity, and an alternative behavioral explanation. However, the stimulus design conflates hierarchy with natural word order, and the ablation statistics in Table 9 do not support linear-selective components. These issues are central rather than cosmetic, so the current manuscript cannot be accepted as-is.

major comments (4)
  1. [§2.2 and §3.4] The stimulus construction conflates grammatical type with naturalness. For every hierarchical grammar, positives are ordinary natural-language sentences and negatives are formed by swapping the final two words; for every linear grammar, both positives and negatives are arbitrary positional manipulations (e.g., inserting a word at position 4 or reversing the sentence). Therefore higher accuracy on hierarchical grammars, and the top-1% neurons identified for them, could reflect detection of 'natural word order' versus 'artificial word order' rather than hierarchy per se. Experiment 4 replaces the lexicon with Jabberwocky words but preserves exactly the same asymmetry: hierarchical Jabberwocky positives follow natural SVO/SOV order, hierarchical negatives are final-two-word swaps, and linear items are positional patterns. Hence the Jabberwocky control removes only lexical in-distributionness, not the naturalness confound. Footnote 9 acknowledges this alternative for behavioral results, but the same alternative threatens Experiments 2 and 3. A concrete control would be to generate hierarchical negatives by moving a word to an arbitrary non-final position, or to match hierarchical and linear items on surface plausibility, and then re-run the overlap and ablation analyses.
  2. [§3.3 / Table 9] The causal claim for linear-selective components is not supported by the reported statistics. Table 9 shows that on linear grammars, ablating H-components versus L-components is non-significant in every language: EN(L) p=0.73, IT(L) p=0.33, JP(L) p=0.06. Appendix B.3 also states that 'relative accuracy decreases are not significantly different between ablations of hierarchical/linear components' for linear grammars. The main text says 'Ablating components from L decreases the model's accuracy on linear structures more than ablating H,' which contradicts these data. Since the disjointness claim requires selectivity in both directions, the paper currently supports only hierarchical-selective components, and that support is largely relative to random ablation rather than to L-components.
  3. [§3.1 / Table 4] The behavioral claim is language-dependent, but the paper's framing overgeneralizes. For Japanese, the Mann-Whitney U test in Table 4 is U=203, p=0.2, so there is no significant accuracy advantage for hierarchical over linear grammars in Japanese. The text in §3.1 carefully limits the p<0.001 statement to English and Italian, yet RQ1 and the Conclusion say that models 'show distinct behaviors' on hierarchical versus linear inputs without the Japanese qualifier. The authors should either state RQ1 as restricted to English and Italian or explain why the Japanese null does not undermine the cross-linguistic claim.
  4. [§3.2 and §3.4] The interpretation of the overlap results needs a chance baseline. The paper reports that all pairwise overlaps are significantly different from zero and that H-H overlaps exceed H-L overlaps. However, because top-1% sets are selected using the same task format and answer tokens, a non-zero baseline overlap is expected even for fully shared mechanisms; the H-H vs H-L contrast is the relevant comparison, and the significance there is encouraging. Still, the statement that overlaps are 'significantly different from 0' is not evidence of specialization. A permutation baseline or a random-subsample overlap calculation should be reported so that the reader can see how much of the absolute overlap is expected by chance.
minor comments (6)
  1. [§2.1] The model list contains six models, but Appendix Tables 7, 8, and 12 refer to '7 models' or report N=108 without explaining the count; please reconcile the model count and the sample-size notation.
  2. [Table 1] The negative examples in Table 1 are missing spaces (e.g., 'a woman readschaptera'), which is presumably a typesetting artifact but should be corrected for readability.
  3. [§2.2] The github link is given only as 'github'; a proper URL is needed to substantiate the reproducibility claim.
  4. [Limitations] The Limitations section discusses polysemanticity and task generalization but does not list the naturalness confound of the stimulus construction as a limitation; given its central role, it should be acknowledged there.
  5. [§3.4 / Table 12] Labels such as 'H(ZZ x EN)' are not defined in the main text; please add a sentence explaining this cross-grammar component-overlap notation.
  6. [§3.4 / Table 11] The prose around Figure 4d should be more careful: Table 11 shows a significant effect of English hierarchical ablations on Jabberwocky hierarchical accuracy (p=0.004) but not on Jabberwocky linear accuracy (p=0.41), so the claim should be restricted to the hierarchical condition.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the behavioral and causal pipeline is self-contained; the naturalness confound flagged in footnote 9 is a validity limitation, not a constructional identity.

full rationale

No step in the paper's derivation chain reduces by construction to its own input. The accuracy gaps, top-1% indirect-effect neuron sets, pairwise overlap statistics, and ablation outcomes are all measured quantities on held-out prompts and held-out grammaticality judgments; no parameter is fitted to the disjointness conclusion, and the ablation predictions could have failed (and in fact partially do fail for smaller models and for Japanese, as shown in Tables 9 and 12). The only serious validity concern is the stimulus-construction confound that the authors themselves flag in footnote 9: "these behavioral results may be explainable using teleological approaches such as those in McCoy et al. (2024): linear grammaticality judgment is a low-probability task and contains low-probability inputs (assuming a pretraining distribution based on Internet text), and will therefore be more difficult for a language model to perform, even if the model used a shared mechanism." This is a genuine confounding and alternative explanation for the hierarchical-versus-linear accuracy gap and for the component localizations, and the paper acknowledges rather than hides it; but it is a limitation, not circularity, because the grammar labels are not defined in terms of the measured accuracies or neuron sets, and the confound does not make the conclusion an identity. Experiment 4's Jabberwocky results actually contradict the in-distribution-only interpretation, providing independent content. Self-citations (Mueller et al. 2022/2024, and the attribution-patching citations) are motivational or methodological and are not load-bearing; no uniqueness theorem is imported from the authors' own prior work, and no known result is renamed. Therefore no circular step can be exhibited, and the score is 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

No fitted constants or invented entities support the claim; however, the conclusion depends on several domain assumptions that are acknowledged only in part, especially the validity of the grammar templates and of attribution patching, plus a hand-chosen 1% threshold.

free parameters (3)
  • top_1pct_neuron_threshold = 1% of MLP and attention dimensions per layer
    Chosen by hand (Section 3.2); all overlap and ablation results depend on which components count as sensitive.
  • L_subsample_size_matching_H = |L| reduced to |H| per layer
    Section 3.3; ablating unequal set sizes would confound comparisons, but the subsampling rule (sort by IE, keep top per layer) is a design choice affecting results.
  • demonstration_count = 10 demonstrations, 4 seeds
    Few-shot prompt design (Section 3.1); behavior and attribution magnitudes may depend on these settings.
assumptions (4)
  • domain assumption The Musso et al. (2003) grammar templates are valid instances of hierarchical (UG-compliant) versus linear (non-UG) rule types.
    Section 2.2; the entire H/L dichotomy rests on this mapping, imported from the human neuroimaging literature.
  • domain assumption Attribution patching (first-order Taylor approximation of indirect effect) faithfully identifies causally relevant components.
    Equation (2), Section 3.2; authors note correlation with activation patching is weaker at first and last layers.
  • ad hoc to paper Swapping the final two words of a hierarchical sentence breaks hierarchical but not linear structure.
    Section 2.2; a positional swap is itself a linear transformation, which blurs the claimed H/L contrast.
  • domain assumption The pretraining corpora are English-dominated, making English in-distribution and Japanese relatively out-of-distribution.
    Section 2.1 and footnote 3; exact training data composition is unknown for all tested models.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Disjoint Processing Mechanisms of Hierarchical and Linear Grammars in Large Language Models." pith.science (2026). https://pith.science/paper/EOTUPBEA

@misc{pith2026250108618,
  author       = {Pith},
  title        = {Pith review of: Disjoint Processing Mechanisms of Hierarchical and Linear Grammars in Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EOTUPBEA}},
  note         = {Machine review of arXiv:2501.08618}
}
read the original abstract

All natural languages are structured hierarchically. In humans, this structural restriction is neurologically coded: when two grammars are presented with identical vocabularies, brain areas responsible for language processing are only sensitive to hierarchical grammars. Using large language models (LLMs), we investigate whether such functionally distinct hierarchical processing regions can arise solely from exposure to large-scale language distributions. We generate inputs using English, Italian, Japanese, or nonce words, varying the underlying grammars to conform to either hierarchical or linear/positional rules. Using these grammars, we first observe that language models show distinct behaviors on hierarchical versus linearly structured inputs. Then, we find that the components responsible for processing hierarchical grammars are distinct from those that process linear grammars; we causally verify this in ablation experiments. Finally, we observe that hierarchy-selective components are also active on nonce grammars; this suggests that hierarchy sensitivity is not tied to meaning, nor in-distribution inputs.

Figures

Figures reproduced from arXiv: 2501.08618 by the authors.

Figure 1
Figure 1. Few-shot accuracy on the grammaticality judgment task on hierarchical and linear inputs. On average, [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Mean pairwise overlap percentage of the top [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Mean relative change in accuracy across mod [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (12 more)
Figure 4
Figure 4. Figure 4: Results on Jabberwocky grammars. We show grammaticality judgment task performance (a), mean neuron [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Experiment 1. Model-wise accuracy on the grammaticality judgments task given hierarchical and linear inputs from English, Italian and Japanese(See § 3.1 and Tables 5 and 2) [PITH_FULL_IMAGE:figures/full_fig_p018_5.png]
Figure 6
Figure 6. Figure 6: Experiment 1. Model-wise accuracy on the grammaticality judgments task given hierarchical and linear Jabberwocky inputs (See § 3.1 and Tables 6 and 3) (a) English, Italian, and Japanese grammars (b) Jabberwocky grammars [PITH_FULL_IMAGE:figures/full_fig_p018_6.png]
Figure 7
Figure 7. Figure 7: Experiment 2. Mean pairwise neuron overlaps, by model, for the top 1% of MLP and attention neurons by ˆIE between hierarchical and linear inputs. (See § 3.2) 18 [PITH_FULL_IMAGE:figures/full_fig_p018_7.png]
Figure 8
Figure 8. Figure 8: Experiment 2. MLP neuron overlaps by model for English, Italian and Japanese grammars. 19 [PITH_FULL_IMAGE:figures/full_fig_p019_8.png]
Figure 9
Figure 9. Figure 9: Experiment 2. Attention neuron overlaps by model for English, Italian and Japanese grammars. 20 [PITH_FULL_IMAGE:figures/full_fig_p020_9.png]
Figure 10
Figure 10. Figure 10: Experiment 2. MLP neuron overlaps by model for Jabberwocky grammars. 21 [PITH_FULL_IMAGE:figures/full_fig_p021_10.png]
Figure 11
Figure 11. Figure 11: Experiment 2. Attention Overlaps by model for Jabberwocky grammars. 22 [PITH_FULL_IMAGE:figures/full_fig_p022_11.png]
Figure 12
Figure 12. Figure 12: Experiment 3. Mean relative change in accuracy by model, when ablating the top 1% of attention and MLP neurons by ˆIE between hierarchical and linear inputs. (See § 3.3) Ablation Structure Type Test-statistic P-Value EN (H) vs EN (L) ZZ (H) 69.5 0.004 EN (H) vs EN (Ra…
Figure 13
Figure 13. Figure 13: Experiment 4. Mean relative change in accuracy by model, when ablating the top 1% of attention and MLP neurons pertaining to English hierarchical and linear grammars. Model is tested on Jabberwocky grammars post ablation, and performance decrease is measured on hierar…
Figure 14
Figure 14. Figure 14: Experiment 4. MLP Overlaps by model between English and Jabberwocky grammars 26 [PITH_FULL_IMAGE:figures/full_fig_p026_14.png]
Figure 15
Figure 15. Figure 15: Experiment 4. Attention Overlaps by model between English and Jabberwocky grammars 27 [PITH_FULL_IMAGE:figures/full_fig_p027_15.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

59 extracted references · 34 canonical work pages

  1. [1]

    Smith, Navin Goyal, and Yulia Tsvetkov

    Kabir Ahuja, Vidhisha Balachandran, Madhur Panwar, Tianxing He, Noah A. Smith, Navin Goyal, and Yulia Tsvetkov. 2024. https://openreview.net/forum?id=YwLgSimUIT Learning syntax without planting trees: Understanding when and why transformers generalize hierarchically . In ICML 2024 Workshop on Mechanistic Interpretability

    work page 2024

  2. [2]

    Badr AlKhamissi, Greta Tuckute, Antoine Bosselut, and Martin Schrimpf. 2024. https://arxiv.org/abs/2411.02280 The LLM language network: A neuroscientific approach for identifying causally task-relevant units . arXiv preprint arXiv:2411.02280

    arXiv 2024

  3. [3]

    Marie Amalric and Stanislas Dehaene. 2019. A distinct cortical network for mathematical knowledge in the human brain. NeuroImage, 189:19--31

    work page 2019

  4. [4]

    Ian A Apperly, Dana Samson, Naomi Carroll, Shazia Hussain, and Glyn Humphreys. 2006. Intact first-and second-order false belief reasoning in a patient with severely impaired grammar. Social neuroscience, 1(3-4):334--348

    work page 2006

  5. [5]

    Baker, Jia Liu, Lawrence L

    Chris I. Baker, Jia Liu, Lawrence L. Wald, Kenneth K. Kwong, Thomas Benner, and Nancy Kanwisher. 2007. https://doi.org/10.1073/pnas.0703300104 Visual word processing and experiential origins of functional selectivity in human extrastriate cortex . Proceedings of the National Academy of Sciences, 104(21):9087--9092

    work page doi:10.1073/pnas.0703300104 2007

  6. [6]

    Deniz Bayazit, Negar Foroutan, Zeming Chen, Gail Weiss, and Antoine Bosselut. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.376 Discovering knowledge-critical subnetworks in pretrained language models . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 6549--6583, Miami, Florida, USA. Association for Comp...

    work page doi:10.18653/v1/2024.emnlp-main.376 2024

  7. [7]

    Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Hatfield-Dodds, Alex Tamkin, Karina Nguyen, Brayden McLean, Josiah E Burke, Tristan Hume, Shan Carter, Tom Henighan, and Ch...

    2023

  8. [8]

    Paul Broca. 1861. Remarques sur le siége de la faculté langage articulé; suives d'une observation d'aphémie. Bulletins et mémoires de la Société Anatomique de Paris, 6:330--357

    work page

Show all 59 references
  1. [9]

    broca’s area,

    Xuanyi Chen, Josef Affourtit, Rachel Ryskin, Tamar I Regev, Samuel Norman-Haignere, Olessia Jouravlev, Saima Malik-Moraleda, Hope Kean, Rosemary Varley, and Evelina Fedorenko. 2023. The human language system, including its inferior frontal component in “broca’s area,” does not...

    2023

  2. [10]

    Noam Chomsky. 1957. Syntactic structures. De Gruyter Mouton

    1957

  3. [11]

    Noam Chomsky. 1965. Aspects of the theory of syntax. The MIT Press

    1965

  4. [12]

    Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. 2024. https://openreview.net/forum?id=F76bwRSLeK Sparse autoencoders find highly interpretable features in language models . In The Twelfth International Conference on Learning Representations

    2024

  5. [13]

    Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield-Dodds, Robert Lasenby, Dawn Drain, Carol Chen, Roger Grosse, Sam McCandlish, Jared Kaplan, Dario Amodei, Martin Wattenberg, and Christopher Olah. 2022. https://transform...

    2022

  6. [14]

    Martin BH Everaert, Marinus AC Huybregts, Noam Chomsky, Robert C Berwick, and Johan J Bolhuis. 2015. Structures, not strings: Linguistics as part of the cognitive sciences. Trends in cognitive sciences, 19(12):729--743

    2015

  7. [15]

    Evelina Fedorenko, Michael K Behr, and Nancy Kanwisher. 2011. Functional specificity for high-level linguistic processing in the human brain. Proceedings of the National Academy of Sciences, 108(39):16428--16433

    2011

  8. [16]

    Scott, Peter Brunner, William G

    Evelina Fedorenko, Terri L. Scott, Peter Brunner, William G. Coon, Brianna Pritchett, Gerwin Schalk, and Nancy Kanwisher. 2016. https://doi.org/10.1073/pnas.1612132113 Neural correlate of the construction of sentence meaning . Proceedings of the National Academy of Sciences, 1...

    2016 doi

  9. [17]

    Evelina Fedorenko and Rosemary Varley. 2016. Language and thought are not the same thing: evidence from neuroimaging and neurological patients. Annals of the New York Academy of Sciences, 1369(1):132--153

    2016

  10. [18]

    Matthew Finlayson, Aaron Mueller, Sebastian Gehrmann, Stuart Shieber, Tal Linzen, and Yonatan Belinkov. 2021. https://doi.org/10.18653/v1/2021.acl-long.144 Causal analysis of syntactic agreement mechanisms in neural language models . In Proceedings of the 59th Annual Meeting o...

    2021 doi

  11. [19]

    Atticus Geiger, Kyle Richardson, and Christopher Potts. 2020. https://doi.org/10.18653/v1/2020.blackboxnlp-1.16 Neural natural language inference models partially embed theories of lexical entailment and negation . In Proceedings of the Third BlackboxNLP Workshop on Analyzing ...

    2020 doi

  12. [20]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Art...

    2024 arXiv

  13. [21]

    Michael Hanna, Ollie Liu, and Alexandre Variengien. 2023. https://openreview.net/forum?id=p4PckNQR8k How does GPT -2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model . In Thirty-seventh Conference on Neural Information Processing Systems

    2023

  14. [22]

    John Hewitt and Christopher D. Manning. 2019. https://doi.org/10.18653/v1/N19-1419 A structural probe for finding syntax in word representations . In Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Langu...

    2019 doi

  15. [23]

    Anna A Ivanova, Zachary Mineroff, Vitor Zimmerer, Nancy Kanwisher, Rosemary Varley, and Evelina Fedorenko. 2021. https://pubmed.ncbi.nlm.nih.gov/37216147/ The language network is recruited but not required for nonverbal event semantics . Neurobiology of Language, 2(2):176--201

    2021

  16. [24]

    Anna A Ivanova, Shashank Srikant, Yotaro Sueoka, Hope H Kean, Riva Dhamala, Una-May O'reilly, Marina U Bers, and Evelina Fedorenko. 2020. https://elifesciences.org/articles/58906 Comprehension of computer code relies primarily on domain-general executive brain regions . elife,...

    2020

  17. [25]

    Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023. https://arxiv.org/abs/2310.06825 Mistral 7b . arXiv preprint arXiv:2310.06825

    2023 arXiv

  18. [26]

    Julie Kallini, Isabel Papadimitriou, Richard Futrell, Kyle Mahowald, and Christopher Potts. 2024. https://doi.org/10.18653/v1/2024.acl-long.787 Mission: Impossible language models . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Vol...

    2024 doi

  19. [27]

    Nancy Kanwisher. 2010. Functional specificity in the human brain: a window into the functional architecture of the mind. Proceedings of the national academy of sciences, 107(25):11163--11170

    2010

  20. [28]

    J \'a nos Kram \'a r, Tom Lieberum, Rohin Shah, and Neel Nanda. 2024. AtP *: An efficient and scalable method for localizing LLM behaviour to components. arXiv preprint arXiv:2403.00745

    2024 arXiv

  21. [29]

    Y Liu, J Kim, C Wilson, and M Bedny. 2020. Computer code comprehension shares neural resources with formal logical inference in the fronto-parietal network. biorxiv, 2020.05. 24.096180

    2020

  22. [30]

    Saima Malik-Moraleda, Maya Taliaferro, Steve Shannon, Niharika Jhingan, Sara Swords, David J Peterson, Paul Frommer, Marc Okrand, Jessie Sams, Ramsey Cardwell, et al. 2023. Constructed languages are processed by the same brain mechanisms as natural languages. bioRxiv

    2023

  23. [31]

    Michaud, Yonatan Belinkov, David Bau, and Aaron Mueller

    Samuel Marks, Can Rager, Eric J. Michaud, Yonatan Belinkov, David Bau, and Aaron Mueller. 2024. https://arxiv.org/abs/2403.19647 Sparse feature circuits: Discovering and editing interpretable causal graphs in language models . arXiv preprint arXiv:2403.19647

    2024 arXiv

  24. [32]

    Thomas McCoy, Robert Frank, and Tal Linzen

    R. Thomas McCoy, Robert Frank, and Tal Linzen. 2018. Revisiting the poverty of the stimulus: hierarchical generalization without a hierarchical bias in recurrent neural networks. In Proceedings of the 40th Annual Meeting of the Cognitive Science Society, CogSci 2018, Proceedin...

    2018

  25. [33]

    Thomas McCoy, Shunyu Yao, Dan Friedman, Mathew D

    R. Thomas McCoy, Shunyu Yao, Dan Friedman, Mathew D. Hardy, and Thomas L. Griffiths. 2024. https://doi.org/10.1073/pnas.2322420121 Embers of autoregression show how large language models are shaped by the problem they are trained to solve . Proceedings of the National Academy ...

    2024 doi

  26. [34]

    Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022. Locating and editing factual associations in gpt. Advances in Neural Information Processing Systems, 35:17359--17372

    2022

  27. [35]

    Jack Merullo, Carsten Eickhoff, and Ellie Pavlick. 2024. https://openreview.net/forum?id=fpoAYV6Wsk Circuit component reuse across tasks in transformer language models . In The Twelfth International Conference on Learning Representations

    2024

  28. [36]

    Thomas McCoy, Dipanjan Das, Emily Pitler, and Tal Linzen

    Junghyun Min, R. Thomas McCoy, Dipanjan Das, Emily Pitler, and Tal Linzen. 2020. https://doi.org/10.18653/v1/2020.acl-main.212 Syntactic data augmentation increases robustness to inference heuristics . In Proceedings of the 58th Annual Meeting of the Association for Computatio...

    2020 doi

  29. [37]

    Martin M Monti, Lawrence M Parsons, and Daniel N Osherson. 2009. The boundaries of language and thought in deductive inference. Proceedings of the National Academy of Sciences, 106(30):12554--12559

    2009

  30. [38]

    Aaron Mueller, Robert Frank, Tal Linzen, Luheng Wang, and Sebastian Schuster. 2022. https://doi.org/10.18653/v1/2022.findings-acl.106 Coloring the blank slate: Pre-training imparts a hierarchical inductive bias to sequence-to-sequence models . In Findings of the Association fo...

    2022 doi

  31. [39]

    Shikhar Murty, Pratyusha Sharma, Jacob Andreas, and Christopher Manning. 2023. https://doi.org/10.18653/v1/2023.acl-short.38 Grokking of hierarchical structure in vanilla transformers . In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics ...

    2023 doi

  32. [40]

    u rgen Reichenbach, Christian B \

    Mariacristina Musso, Andrea Moro, Volkmar Glauche, Michel Rijntjes, J \"u rgen Reichenbach, Christian B \"u chel, and Cornelius Weiller. 2003. Broca's area and the language instinct. Nature neuroscience, 6(7):774--781

    2003

  33. [41]

    Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Scott Johnston, Andy Jones, Jackson Kernion, Liane Lovitt, Kam...

    2022

  34. [42]

    Judea Pearl. 2001. Direct and indirect effects. In Proceedings of the Seventeenth Conference on Uncertainty in Artificial Intelligence, pages 411--420. Morgan Kaufmann

    2001

  35. [43]

    Nikhil Prakash, Tamar Rott Shaham, Tal Haklay, Yonatan Belinkov, and David Bau. 2024. https://openreview.net/forum?id=8sKcAWOf2D Fine-tuning enhances existing mechanisms: A case study on entity tracking . In The Twelfth International Conference on Learning Representations

    2024

  36. [44]

    Robins and Sander Greenland

    James M. Robins and Sander Greenland. 1992. http://www.jstor.org/stable/3702894 Identifiability and exchangeability for direct and indirect effects . Epidemiology, 3(2):143--155

    1992

  37. [45]

    Koustuv Sinha, Robin Jia, Dieuwke Hupkes, Joelle Pineau, Adina Williams, and Douwe Kiela. 2021. https://doi.org/10.18653/v1/2021.emnlp-main.230 Masked language modeling and the distributional hypothesis: Order word matters pre-training for little . In Proceedings of the 2021 C...

    2021 doi

  38. [46]

    Haiyang Sun, Lin Zhao, Zihao Wu, Xiaohui Gao, Yutao Hu, Mengfei Zuo, Wei Zhang, Junwei Han, Tianming Liu, and Xintao Hu. 2024. https://arxiv.org/abs/2410.19542 Brain-like functional organization within large language models . arXiv preprint arXiv:2410.19542

    2024 arXiv

  39. [47]

    Aaquib Syed, Can Rager, and Arthur Conmy. 2024. https://openreview.net/forum?id=RysbaxAnc6 Attribution patching outperforms automated circuit discovery . In The 7th BlackboxNLP Workshop

    2024

  40. [48]

    Li, Arnab Sen Sharma, Aaron Mueller, Byron C

    Eric Todd, Millicent L. Li, Arnab Sen Sharma, Aaron Mueller, Byron C. Wallace, and David Bau. 2024. Function vectors in large language models. In Proceedings of the 2024 International Conference on Learning Representations

    2024

  41. [49]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. https://arxiv.org/abs/2307.09288 Llama 2: Open foundation and fine-tuned chat models . arXiv preprint arXiv:...

    2023 arXiv

  42. [50]

    Rosemary Varley and Michael Siegal. 2000. Evidence for cognition without grammar from causal reasoning and ‘theory of mind’in an agrammatic aphasic patient. Current Biology, 10(12):723--726

    2000

  43. [51]

    Rosemary A Varley, Nicolai JC Klessinger, Charles AJ Romanowski, and Michael Siegal. 2005. Agrammatic but numerate. Proceedings of the National Academy of Sciences, 102(9):3519--3524

    2005

  44. [52]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in Neural Information Processing Systems, 30

    2017

  45. [53]

    Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart Shieber. 2020. https://proceedings.neurips.cc/paper_files/paper/2020/file/92650b2e92217715fe312e6fa7b90d82-Paper.pdf Investigating gender bias in language models using causal me...

    2020

  46. [54]

    W3Techs . 2024. https://w3techs.com/technologies/overview/content_language Usage statistics and market share of content languages for websites, may 2024 . Accessed: 2024-05-18

    2024

  47. [55]

    Kevin Ro Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. 2023. https://openreview.net/forum?id=NpsVSN6o4ul Interpretability in the wild: A circuit for indirect object identification in GPT -2 small . In The Eleventh International Conference on L...

    2023

  48. [56]

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jianxin Yang, Jin Xu, Jingren Zhou, Jinze...

    2024 arXiv

  49. [57]

    Qinan Yu, Jack Merullo, and Ellie Pavlick. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.615 Characterizing mechanisms for factual recall in language models . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 9924--9959, Singa...

    2023 doi

  50. [58]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  51. [59]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.