What, Where, and How: Disentangling the Roles of Task, Language, and Model in Code Model Representations

Piotr Wilam

REVIEW 3 major objections 4 minor 28 references

What, Where, and How: Disentangling the Roles of Task, Language, and Model in Code Model Representations

T0 review · 3 major / 4 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that universality in code model representations splits into three independent axes: which concepts get dedicated circuitry is conserved across models, but where those circuits sit and how they grow are model-specific.

desk verdict Solid, well-scoped 2x2 study that productively splits the universality question into What/Where/How; the core dissociation looks real, but the single threshold and post-hoc clustering need scrutiny before the numbers are trusted. read the letter →

arxiv 2607.21491 v1 pith:RNXJVF62 submitted 2026-07-23 cs.CL cs.LG

Piotr Wilam This is my paper

classification cs.CLcs.LG

keywords conceptcircuitsmechanisticinterpretabilitycodemodelsuniversalitycross-modeltransferlayerlocalizationPythonRust

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether two independently trained code models represent the same grammatical concepts in the same way. Using a 2x2 design (Python and Rust crossed with two models), it measures a complete inventory of 58 Python and 57 Rust concepts with a concept-circuit extraction method. The central result is a dissociation: the ranking of which concepts earn dedicated circuitry transfers across models (What is conserved), while the layer where concepts are processed and the way circuits grow over layers differ sharply (Where and How diverge). A sympathetic reader would care because this refines the universality debate: representational content may converge, but computational organization does not.

What carries the argument

The concept-circuit extraction method: for each grammatical concept, generate many prompts containing the construct and a second family of checker prompts where the keyword appears outside its structural role. A neuron is active if its absolute activation exceeds epsilon = 0.5; the universal circuit is the intersection of active-neuron sets over the concept prompts, and the concept-only circuit is the part of that intersection not shared with the checker mask. The concept fraction (size of concept-only set divided by total active set) serves as a per-concept, per-layer measurement defined identically across models and languages, making cross-model comparison possible.

What would settle it

Apply the same concept-circuit extraction to a third independently trained code model and test whether its processing band, atomicity flow type, and cross-language sharing level co-vary as predicted by the two-style fingerprint (e.g., an early band should come with smooth atomicity onset and higher sharing). Alternatively, re-run the full pipeline with a fine threshold sweep to see if the cross-model Spearman rho and the layer-band gap persist at epsilon values away from 0.5.

Watch

Extended reading notes

Core claim

Independently trained code models agree on which grammatical constructs merit dedicated neural circuitry, but they do not agree on where or how that circuitry is organized. In the paper's 2x2 design, concept-fraction rankings correlate across models (Spearman rho = 0.638 for Python, 0.673 for Rust, p < 10^-7), yet Qwen concentrates concept-specific processing in a late band around layers 17-19 while DeepSeek peaks at layers 6-7 for both languages, a 12-13 layer gap. The growth pattern also differs: Qwen gives atomic concepts an early circuit spike at layers 2-3, which DeepSeek lacks. The paper concludes that 'are circuits universal?' has a split answer: yes for What, no for Where and How.

Load-bearing premise

The entire measurement depends on the binary activation threshold epsilon = 0.5, a parameter chosen by inspection rather than derived from theory, and the paper reports only a three-point threshold sweep without showing whether the dissociation survives at neighboring values.

Editorial extensions

If this is right

An inventory of which concepts earn circuitry can be reused from one model to another, because the What ranking transfers.
Layer-indexed techniques (fixed-depth probing, activation patching, circuit editing) are model-specific and must be re-localized per model, with the band measurements indicating where to look.
Language design determines representation strength: Rust constructs receive roughly 2-3x more concept-specific circuitry than Python equivalents in both models, so how strongly a construct is represented is language-determined, not architecture-determined.
Cross-language neuron sharing is a real but variable phenomenon: the ranking of which constructs share is conserved, but the amount is model-specific (DeepSeek shares about 1.94x more than Qwen).
The per-model fingerprint predicts that a third, independently trained model should show co-varying features (early band, smooth atomicity onset, higher cross-language sharing) rather than mixing them freely.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Inference: If the What/Where/How dissociation holds, interpretability results are less portable across models than the field often hopes; this favors inventory-based comparison methods over single-model circuit studies.
Inference: The binary activation threshold at epsilon = 0.5 is load-bearing; a finer threshold sweep might reveal whether the dissociation persists or is an artifact of that specific operating point.
Inference: The Qwen-specific cluster of Rust type-and-trait keywords suggests models can form abstract semantic categories not present in surface syntax; this could generalize to other type-system-heavy languages and be tested directly.
Inference: The paper's own sub-threshold caveat implies a continuous-geometry extension might capture low-amplitude distributed structure, potentially altering the Where conclusion if such structure differs between models.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The genuinely new thing here is the fixed-protocol inventory across two languages and two independently trained models, and using it to split universality into three axes: What (which concepts get circuits) is conserved, Where and How are not. That is a real step beyond the usual yes/no framing, and the direction of the result — rho ~0.65, not 0.3 or 0.9 — is informative. The paper is also honest in the right places: the ablation failure on Break is reported, the random cluster in the permutation test is labeled random, and the Limitations section names the degenerate consistency parameter and the causal-validation scope. Released code and frozen artifacts are a plus; the claim numbering is verifiable.

The soft spots are real but not disqualifying. The biggest is the threshold. Everything downstream sits on epsilon=0.5, chosen because signal “emerges” there after a three-point sweep. That is post-hoc, and the paper gives no sensitivity analysis around it. I’m not convinced the dissociation would vanish at other thresholds — the layer-band difference and the cross-model ranking could well persist — but I’d want to see the same metrics at, say, 0.3 and 0.7 before trusting the precise values. The frozen artifacts make this a cheap check, so it’s an addressable flaw, not a fatal one.

The semantic clustering concern is real but smaller. The four groups were hypothesized after looking at the dendrogram, so the p<0.001 for the type-trait cluster is conditional on selection. The cluster itself is semantically coherent and spans four syntactic categories, which makes it plausible, but the reported p-value overstates the evidence. A replication on a third model or a pre-registered grouping would fix this. The “two processing styles” claim rests on n=2 models; the paper is appropriately cautious, framing it as a fingerprint with falsifiable predictions. That’s fine.

Who gets value: definitely mechanistic interpretability people, especially anyone working on cross-model comparison or portability of interpretability results. It deserves a serious referee. I’d recommend major revision rather than acceptance: require a threshold sensitivity analysis, a stability check on the clustering, and ideally a third model or at least an acknowledgment that the fingerprint claim is exploratory. I’d engage with it again after those are addressed.

Referee Report

3 major / 4 minor

Summary. This paper extends a concept-circuit extraction method to a 2x2 design (Python/Rust × Qwen2.5-Coder-7B / DeepSeek-Coder-V1-6.7B) and measures, for 58 Python and 57 Rust grammatical concepts, which neurons are active and how concept-specific masks evolve across layers. The central claim is a triple dissociation: the ranking of which concepts receive dedicated circuitry is conserved across models (Spearman ρ≈0.64–0.67, p<10^-7), while the layer band in which concepts are processed (Qwen late L17–19, DeepSeek early L6–7) and the growth dynamics (Qwen early atomicity spike vs DeepSeek smooth late onset) are model-specific. Additional findings are that Rust constructs receive 2–3× more concept-specific circuitry than Python, cross-language neuron sharing is 1.94× higher in DeepSeek, and Qwen clusters Rust type-and-trait keywords into a tight neuron group. The paper is carefully scoped, reports negative results (Break ablation fails; one semantic cluster is random), and releases frozen artifacts and locked-number tests.

Significance. If the dissociation is real, the paper makes a substantive contribution to the universality debate by showing that 'universal' is not a single yes/no property: representational content can transfer across models while computational organization does not. The methodology is innovative in using a structured grammar-derived inventory as a common yardstick, and the reproducibility apparatus (frozen artifacts, regenerable figures, locked numerical tests) is exemplary. The paper also reports honest negative results and explicit limitations. However, the central measurements all depend on a post-hoc binary activation threshold, and the headline semantic cluster is tested on the same data used to form it; these issues must be resolved before the claims are fully supported.

major comments (3)

[§3 Binarisation; §4.2–4.4; §6.2] The central measurements all flow through the binary activation threshold ε=0.5 fixed in §3. The reported properties—concept-fraction rankings (Table 1), layer-band peaks (Figure 2), early-bias contrast (Figure 3), and cluster cohesion (Figure 6)—are computed from masks defined by |activation|>ε, and the paper’s own three-point scan shows that the qualitative regime changes with ε: at 0.001 the circuits are trivially full, at 0.1 concept-only fractions are near-zero, and only at 0.5 does 'structural signal emerge.' No fine-grained sensitivity analysis is reported, so the What/Where/How dissociation may be an artifact of this operating point rather than a property of the models. Please add a systematic ε-sweep (e.g., ε∈{0.2,0.3,0.4,0.5,0.6,0.7}) and report the Spearman ρ, the layer-peak positions, the early-bias values, and the cluster p-values across the sweep, or provide a principled ar
[§6.2 Semantic Clustering] The four clusters in Figure 5 are read off the dendrogram and then 'tested' with a permutation test on the same data. Because the group definitions were selected after inspecting the data, the reported p-values (type-system traits p<0.001, memory/ownership p=0.044, data definition p=0.035, control-flow p=0.292) are selection-conditioned and not valid as confirmatory tests. The 'strong' type-trait cluster may be a real effect, but the current analysis cannot distinguish it from an overfit grouping. Please use a pre-specified clustering (or a holdout split: define groups on one subset, test on the other) and report the false-discovery rate over the set of possible clusterings.
[§8 Discussion / Table 5] The 'two processing styles' fingerprint is presented as a dimensionality finding ('six measurements... cohere into two styles'). With only two models, any set of binary contrasts can be made to align after the fact; the paper itself concedes 'could in principle be coincidence.' This is not disqualifying if the fingerprint is explicitly a hypothesis, but the current wording overstates the evidence. Please reframe §8 as generating a falsifiable prediction (e.g., a third model with an early peak band should show smooth atomicity onsets and higher cross-language sharing) and specify the pre-registered test that would confirm or refute it.

minor comments (4)

[§3 and Figure 2] The 'cons=0.8' consistency filter is inert for these dense SwiGLU models, yet it remains in axis labels (e.g., Figure 2). Remove or annotate to avoid implying a second active parameter.
[§3, Appendix A] Section 3 refers to 'Appendix R' for reproducibility; the appendix is labelled Appendix A, and the Availability section also says 'Appendix A.' Fix the cross-reference.
[§4.1 vs Table 2] Section 4.1 gives DeepSeek Python builtins concept fraction 0.068, but Table 2 reports 0.060 for P×DS Builtin/Object. If these are the same quantity, reconcile the discrepancy; the locked-number test should also catch this.
[Figure 11] The axis label 'Jacc.~vs Cos.' contains a LaTeX artifact; define 'Jaccard' and 'cosine' clearly in the caption. Also, the figure caption says 'Pearson r (Jacc.~vs Cos.)' but the text says 'Pearson r peaking at 0.645 at L20' without stating the number of pairs; consider adding n=276 in the caption.

Circularity Check

2 steps flagged · score 4.0 of 10

Partial circularity at the operating-point and cluster-selection stages; the central What/Where/How dissociation retains independent content.

fitted input called prediction [Section 3, Binarisation]
"We sweep ε∈ {0.001,0.1,0.5} to probe permissive to strict: at 0.001 circuits are trivially full, at 0.1 concept-only fractions are near-zero, and structural signal emerges at ε= 0.5 , which we use throughout."

All central quantities—concept fraction |A\B|/|A|, layer-band peaks, circuit-size growth curves, and cluster memberships—are computed from masks at ε=0.5. That threshold was selected because 'structural signal emerges' there, so the reported existence of the signal is partly an artifact of choosing the operating point at which it appears. The paper notes the consistency filter is degenerate, making the sweep effectively over ε alone, and provides no fine-grained sensitivity analysis showing the What/Where/How dissociation is stable across ε. Thus the headline structure is conditioned on a parameter selected using the very phenomenon it is used to demonstrate.
fitted input called prediction [Section 6.2, Semantic Clustering in Rust]
"Hierarchical clustering at Qwen × Rust × L14 (Figure 5) suggests four groups; a permutation test (10,000 same-size random draws, fixed seed; Figure 6) sorts them: ... Type-system traits ... 0.535 ... <0.001 strong"

The 'hypothesised' groups are extracted from the dendrogram of the same data, then the same data are used to compute within-group Jaccard cohesion and its permutation null. The null distribution is for a random pre-specified group, not for the most cohesive group selected by inspecting the dendrogram; selection is not accounted for. Consequently the p<0.001 for the type-trait cluster is partly forced by the selection procedure rather than an independent confirmation of the cluster's semantic status.

full rationale

The paper's central dissociation is not wholly circular: the Spearman correlations for 'What', the layer-band peaks for 'Where', and the early-bias contrast for 'How' are empirical quantities that could have landed differently, and they are not fitted to target values. The method self-citation to Wilam (2026) is load-bearing, but the current paper applies the method to new models and validates it with ablation and linear probes, so this is not a case of a uniqueness theorem or ansatz smuggled in solely by self-citation. However, two steps do show a fitting-selection loop. First, the binarisation threshold ε=0.5 is chosen because 'structural signal emerges' there, and every downstream measurement is computed at that threshold; the Limitations confirm the sweep reduces to ε alone because the consistency parameter is degenerate, and no sensitivity analysis establishes that the dissociation is a property of the models rather than of the operating point. Second, the Rust semantic clusters are hypothesized after inspecting the dendrogram and then tested on the same data, making the reported p<0.001 overconfident. These issues are partial: they affect the strength and framing of some claims, but the main What/Where/How comparisons remain empirically grounded. The released code and frozen artifacts enable re-analysis at other thresholds, but that does not remove the selection bias in the reported numbers.

Assumptions & free parameters 3 free parameters · 3 assumptions · 1 invented entities

The paper's central claims rest on a small number of manually chosen thresholds and on assumptions about what the intersection method isolates. There are no fitted physical constants, but the ε threshold and the L14 clustering layer are load-bearing choices that affect the quantitative claims. The two-processing-styles construct is an inferred latent with no independent evidence yet.

free parameters (3)

epsilon (activation threshold) = 0.5
Chosen from the sweep {0.001, 0.1, 0.5} because 'structural signal emerges' only at 0.5; all concept fractions, layer bands, and circuit-size curves depend on it. Section 3, Binarisation.
clustering layer for semantic clusters = L14
The Rust type-trait cluster (Figure 5) and Python cluster are computed at layer 14, with no stated selection criterion and no stability analysis across layers. Section 6.2.
cross-language sharing pass threshold = 10%
A pairing counts as sharing if the sharing fraction exceeds 10%; authors say the count is insensitive to the cutoff but show no sensitivity analysis. Section 5.3.

assumptions (3)

domain assumption A neuron that fires above threshold on all 50 varied prompts is responding to the concept rather than to incidental features (marginalisation precondition).
The entire measurement depends on this; the paper validates indirectly via calibration positives/negatives, but it is not proven. Section 3, Marginalisation.
domain assumption The binary threshold view at epsilon=0.5 captures the functionally relevant concept circuitry; sub-threshold distributed structure is ignorable.
Paper acknowledges sub-threshold structure is missed; linear probes show 97%+ decodability even where binary sets are small. Sections 7.2 and Limitations.
domain assumption Two models and two languages are sufficient to operationalize task-, language-, and model-determined variation in the intended sense.
Authors define 'determined' operationally within the 2x2 and state a third model is needed for prediction; the attribution of variation to 'model' conflates architecture, corpus, and training-run differences. Section 4.5.

invented entities (1)

Two processing styles (late/sharp vs early/smooth)
purpose: To explain why six measurements co-segregate across the two models and to serve as a per-model fingerprint.
No third model is tested; with two models, coherence of any set of differing measurements is weak evidence. The paper itself frames this as a prediction for future work, not an established latent. Section 8.

how reviews work

0 comments

Cite this review

Pith. "Pith review of What, Where, and How: Disentangling the Roles of Task, Language, and Model in Code Model Representations." pith.science (2026). https://pith.science/paper/RNXJVF62

@misc{pith2026260721491,
  author       = {Pith},
  title        = {Pith review of: What, Where, and How: Disentangling the Roles of Task, Language, and Model in Code Model Representations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RNXJVF62}},
  note         = {Machine review of arXiv:2607.21491}
}

abstract

Do independently trained language models come to represent the same thing in the same way? We answer for code, extending a recently introduced concept-circuit extraction method to a 2x2 design -- Python and Rust crossed with Qwen2.5-Coder-7B and DeepSeek-Coder-V1-6.7B -- and measuring a complete inventory of grammatical concepts (58 Python, 57 Rust) identically in all four cells: the smallest design that separates what depends on the task, the language, and the model. The answer splits into three parts. What earns dedicated circuitry is set by the task: the models agree on which concepts receive circuits (Spearman $\rho$ = 0.638 for Python, 0.673 for Rust, both p < $10^{-7}$). Where those circuits sit is set by the model: Qwen processes concepts in a late band (~L17-19), DeepSeek at L6-7, for both languages. How circuits grow across layers is also set by the model: Qwen gives its atomic concepts an early spike that DeepSeek does not. "Are circuits universal?" thus has no single answer: yes for What, no for Where and How -- universality is a property of representational content, not of computational organisation. None of this structure was fixed in advance. The agreement could have landed anywhere between independence and identity; it lands at $\rho \approx 0.65$. Rust constructs receive 2-3x more concept-specific circuitry than their Python equivalents, in both models. Both models share neurons between the languages (6/7 and 7/7 paired constructs), DeepSeek 1.94x more than Qwen -- a direction no prior result predicts. And Qwen binds nine keywords of Rust's type-and-trait machinery into one tight neuron cluster (Jaccard 0.535 vs null 0.112, p < 0.001), a semantic dimension invisible in surface syntax. Ablation and linear probes confirm the circuits are functional. All claims are scoped to this 2x2; whether the per-model profile predicts a third model is the designed next test.

Figures

Figures reproduced from arXiv: 2607.21491 by the authors.

**Figure 2.** Layer-resolved concept-fraction profiles. Qwen peaks in a late band ( [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

**Figure 3.** Circuit size by layer for the six Python atomicity concepts. Qwen shows an early spike at L2–3 [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Figures from the paper (8 more)

**Figure 4.** Figure 4: Cross-language neuron sharing by equivalence class. DeepSeek shares 1.94× more than Qwen; the 10% threshold is shown as a dashed line (§5.3). shareable a pairing is, is conserved across models (task-determined); the magnitude difference is model-determined, plausibly…

**Figure 5.** Figure 5: Rust semantic clustering at Qwen × L14 — Ward dendrogram on 1−Jaccard over the non-empty concept-only sets. The four hypothesised groups (type-system traits, memory/ownership, data definition, control-flow/module) are tested for cohesion in [PITH_FULL_IMAGE:figures/fu…

**Figure 6.** Figure 6: Permutation test (10,000 same-size random draws, seed = 42) on the four hypothesised groups from [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]

**Figure 7.** Figure 7: Double dissociation by concept at the peak-effect layer. Four concepts (Import, Try, While, Assert) pass with concept-only ablation more dissociative than random null; Break fails; small-n concepts show no measurable effect (§7.1). at 0.645 at L20. The correlation is m…

**Figure 8.** Figure 8: Per-layer circuit size for every testable concept, coloured by flow type (§ [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]

**Figure 9.** Figure 9: Python concept clustering, Qwen × L14. The atomicity super-cluster and control-flow groupings emerge in the green tree (§6.2). Try While AsyncFor AsyncWith ClassDef With Nonlocal Raise AsyncFunc YieldFrom For Return Global Lambda If Yield bool map propertyall zip bytes…

**Figure 10.** Figure 10: Python concept clustering, DeepSeek × L14. Direct companion to [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]

**Figure 11.** Figure 11: gives the per-layer probe accuracy and the Jaccard–cosine correlation underlying §7.2. 0 10 20 Layer 0.5 0.6 0.7 0.8 0.9 1.0 Probe 1 Accuracy (object vs checker) P_QW: Probe accuracy by layer chance (a) Probe accuracy by layer (97.6– 99.7%, 28 layers). 0 20 Layer 0.0 …

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

28 extracted references · 5 linked inside Pith

[1]

Belinkov

Y. Belinkov. Probing classifiers: Promises, shortcomings, and advances. Computational Linguistics 48(1), 2022

2022
[2]

Bricken et al

T. Bricken et al. Towards monosemanticity: Decomposing language models with dictionary learning. Transformer Circuits Thread, 2023

2023
[3]

Chughtai, L

B. Chughtai, L. Chan, and N. Nanda. A toy model of universality: Reverse engineering how networks learn group operations. ICML, 2023

2023
[4]

Conmy, A

A. Conmy, A. Mavor-Parker, A. Lynch, S. Heimersheim, and A. Garriga-Alonso. Towards automated circuit discovery for mechanistic interpretability. NeurIPS, 2023

2023
[5]

Conneau, S

A. Conneau, S. Wu, H. Li, L. Zettlemoyer, and V. Stoyanov. Emerging cross-lingual structure in pretrained language models. ACL, 2020

2020
[6]

Cunningham, A

H. Cunningham, A. Ewart, L. Riggs, R. Huben, and L. Sharkey. Sparse autoencoders find highly interpretable features in language models. arXiv:2309.08600, 2023

arXiv 2023
[7]

Elhage et al

N. Elhage et al. Toy models of superposition. Transformer Circuits Thread, 2022

2022
[8]

Gurnee, N

W. Gurnee, N. Nanda, M. Pauly, K. Harvey, D. Troitskii, and D. Bertsimas. Finding neurons in a haystack: Case studies with sparse probing. TMLR, 2023

2023

Show all 28 references

[9]

Gurnee, T

W. Gurnee, T. Horsley, Z. C. Guo, T. Rezaei Kheirkhah, Q. Sun, W. Hathaway, N. Nanda, and D. Bertsimas. Universal neurons in GPT2 language models. arXiv:2401.12181, 2024

2024 arXiv
[10]

Z. Yin, X. Gu, and B. Shen. Neuron-guided interpretation of code LLMs: Where, why, and how? arXiv:2512.19980, 2025

2025
[11]

Heimersheim and N

S. Heimersheim and N. Nanda. How to use and interpret activation patching. arXiv, 2024

2024
[12]

Hernandez, A

E. Hernandez, A. Sen Sharma, T. Haklay, K. Meng, M. Wattenberg, J. Andreas, Y. Belinkov, and D. Bau. Linearity of relation decoding in transformer LMs. ICLR, 2024

2024
[13]

M. Huh, B. Cheung, T. Wang, and P. Isola. The Platonic representation hypothesis. ICML, 2024

2024
[14]

Kornblith, M

S. Kornblith, M. Norouzi, H. Lee, and G. Hinton. Similarity of neural network representations revisited. ICML, 2019

2019
[15]

orner, M. M\

F. K\"orner, M. M\"uller-Eberstein, A. Korhonen, and B. Plank. When meanings meet: Investigating the emergence and quality of shared concept spaces during multilingual language model training. EACL, 2026

2026
[16]

M. Lan, P. Torr, A. Meek, A. Khakzar, D. Krueger, and F. Barez. Quantifying feature space universality across large language models via sparse autoencoders. arXiv:2410.06981, 2024

2024 arXiv
[17]

K. Meng, D. Bau, A. Andonian, and Y. Belinkov. Locating and editing factual associations in GPT. NeurIPS, 2022

2022
[18]

Muller, A

B. Muller, A. Anastasopoulos, B. Sagot, and D. Seddah. When being unseen from mBERT is just the beginning. NAACL, 2021

2021
[19]

C. Olah, N. Cammarata, L. Schubert, G. Goh, M. Petrov, and S. Carter. Zoom in: An introduction to circuits. Distill, 2020

2020
[20]

Pires, E

T. Pires, E. Schlinger, and D. Garrette. How multilingual is multilingual BERT? ACL, 2019

2019
[21]

T. Tang, W. Luo, H. Huang, D. Zhang, X. Wang, X. Zhao, F. Wei, and J.-R. Wen. Language-specific neurons: The key to multilingual capabilities in large language models. ACL, 2024

2024
[22]

Templeton et al

A. Templeton et al. Scaling monosemanticity: Extracting interpretable features from Claude 3 Sonnet. Transformer Circuits Thread, 2024

2024
[23]

Tenney, D

I. Tenney, D. Das, and E. Pavlick. BERT rediscovers the classical NLP pipeline. ACL, 2019

2019
[24]

Y. Wan, W. Zhao, H. Zhang, Y. Sui, G. Xu, and H. Jin. What do they capture? A structural analysis of pre-trained language models for source code. ICSE, 2022

2022
[25]

K. Wang, A. Variengien, A. Conmy, B. Shlegeris, and J. Steinhardt. Interpretability in the wild: A circuit for indirect object identification in GPT-2 small. ICLR, 2023

2023
[26]

J. Wang, X. Ge, W. Shu, Q. Tang, Y. Zhou, Z. He, and X. Qiu. Towards universality: Studying mechanistic similarity across language model architectures. arXiv:2410.06672, 2024

2024 arXiv
[27]

P. Wilam. CSP-Atlas: Concept-Specific Neural Circuits in a Sparse Python Transformer. arXiv:2605.24603, 2026

2026 arXiv
[28]

Zhang, Q

R. Zhang, Q. Yu, M. Zang, C. Eickhoff, and E. Pavlick. The same but different: Structural similarities and differences in multilingual language modeling. ICLR, 2025

2025

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.

[1] [1]

Belinkov

Y. Belinkov. Probing classifiers: Promises, shortcomings, and advances. Computational Linguistics 48(1), 2022

2022

[2] [2]

Bricken et al

T. Bricken et al. Towards monosemanticity: Decomposing language models with dictionary learning. Transformer Circuits Thread, 2023

2023

[3] [3]

Chughtai, L

B. Chughtai, L. Chan, and N. Nanda. A toy model of universality: Reverse engineering how networks learn group operations. ICML, 2023

2023

[4] [4]

Conmy, A

A. Conmy, A. Mavor-Parker, A. Lynch, S. Heimersheim, and A. Garriga-Alonso. Towards automated circuit discovery for mechanistic interpretability. NeurIPS, 2023

2023

[5] [5]

Conneau, S

A. Conneau, S. Wu, H. Li, L. Zettlemoyer, and V. Stoyanov. Emerging cross-lingual structure in pretrained language models. ACL, 2020

2020

[6] [6]

Cunningham, A

H. Cunningham, A. Ewart, L. Riggs, R. Huben, and L. Sharkey. Sparse autoencoders find highly interpretable features in language models. arXiv:2309.08600, 2023

arXiv 2023

[7] [7]

Elhage et al

N. Elhage et al. Toy models of superposition. Transformer Circuits Thread, 2022

2022

[8] [8]

Gurnee, N

W. Gurnee, N. Nanda, M. Pauly, K. Harvey, D. Troitskii, and D. Bertsimas. Finding neurons in a haystack: Case studies with sparse probing. TMLR, 2023

2023