Pith. sign in

REVIEW 4 major objections 5 minor 29 references

Semantic Convergence: Investigating Shared Representations Across Scaled LLMs

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Gemma-2-2B and Gemma-2-9B, fourfold apart in size, converge on strongly aligned internal feature representations, with middle layers most similar.

desk verdict A modest but honest extension of Lan et al. to a 4x size gap; the mid-layer alignment is likely real, but the shared tokenizer and missing code/data keep the universality claim conditional. read the letter →

arxiv 2507.22918 v1 pith:VMR5KC6Q submitted 2025-07-21 cs.CL cs.LG

Daniel Son , Sanjana Rathore , Andrew Rufail , Adrian Simon , Daniel Zhang , Soham Dave , Cole Blondin , Kevin Zhu
show 1 more author
Sean O'Brien
This is my paper
classification cs.CLcs.LG
keywords featureuniversalitysparseautoencoderssingularvaluecanonicalcorrelationanalysisrepresentationalsimilarityGemma-2semanticsubspacesmulti-tokenrepresentationscross-modelinterpretability
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether two language models from the same family but with a fourfold difference in size—Gemma-2-2B and Gemma-2-9B—develop the same internal concepts. The authors decompose each model's residual-stream activations into interpretable features with sparse autoencoders, match features across models by activation correlation, and measure how well the matched spaces align using SVCCA and RSA. They report that middle layers align most strongly, with a peak paired SVCCA of 0.73 against a random-pair baseline of 0.005–0.034, while early and late layers align much less. They also report that semantically coherent groups of concepts and multi-token phrases align across the two scales. If these results hold, they strengthen the case that universality of internal features is a structural property of at least this model family, and that interpretability tools trained on a small model could transfer to a larger sibling.

What carries the argument

The load-bearing machinery is the pipeline that turns polysemantic neuron activations into comparable feature dictionaries: a sparse autoencoder (SAE) trained on each model's residual stream produces sparse, monosemantic features; features are then paired across models by Pearson correlation of their activations, using either a 1-to-1 bijective matching or a many-to-1 matching that lets one feature in the larger model serve several features in the smaller one; finally, rotation-invariant similarity measures—SVCCA, which averages canonical correlations after SVD denoising, and RSA, which correlates representational dissimilarity matrices—quantify how well the matched spaces align. This pipeline is what allows the paper to compare internal spaces across architectures of different widths and depths.

What would settle it

Train or obtain SAEs for two models from different families with different tokenizers while keeping scale comparable, feed the same prompts, and run the identical pairing and SVCCA/RSA pipeline; if the middle-layer SVCCA peak drops to the random-pair baseline region (below roughly 0.05), the reported universality would be a tokenizer artifact rather than a general property.

Watch

Extended reading notes

Core claim

The paper's central claim is that Gemma-2-2B and Gemma-2-9B converge on shared internal feature representations despite a fourfold difference in scale. The evidence is layer-wise: after aligning SAE features by Pearson correlation, singular value canonical correlation analysis (SVCCA) and representational similarity analysis (RSA) both show a diagonal band of high similarity across contiguous middle-layer pairs, peaking at SVCCA 0.73 for layer 14 of the small model and layer 19 of the large one, while random-pair baselines stay below 0.034. The middle-stack overlap is interpreted as the point where both models most similarly organize concepts; early and late layers, including the final decoder layer of the large model, show far weaker alignment. The paper extends this to semantic subspaces, showing that compatible concept pairs such as nature–people and emotion–time align with paired SVCCA above 0.6 while incompatible pairs like country–people do not, and that a multi-token emotion–time subspace aligns strongly in early and middle layers, suggesting phrases, not just single tokens, are shared.

Load-bearing premise

The comparison assumes that matching features across Gemma-2-2B and Gemma-2-9B reveals shared concepts, but both models share the same tokenizer and see the same prompts, so the mid-layer alignment could be driven by identical surface tokens rather than by genuinely shared internal concepts.

Editorial extensions

If this is right

  • If mid-layer SAE features transfer from the 2B model to the 9B model, interpretability tools such as feature probes and SAE-based circuit analysis can be developed and validated on the smaller model before being applied to the larger one.
  • Cross-scale safety interventions could target the same aligned features in both models, so a mechanistic intervention found in one model has a candidate counterpart in the other.
  • The semantic subspace results identify concept pairs whose internal geometry is shared, providing concrete test cases for checking whether a third model exhibits the same conceptual alignment.
  • The observation that multi-token phrases align better than single tokens in early layers challenges the assumption that meaning lives in a single token vector, and shifts analysis toward phrase-level features.
  • The consistent mid-layer peak across both matching strategies and both similarity metrics suggests that the alignment is not an artifact of the particular pairing algorithm.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the same-tokenizer confound means the strongest test of universality is a cross-tokenizer comparison; if mid-layer alignment persists between models with different vocabularies, the claim would extend beyond the Gemma family.
  • Beyond the paper: the fact that 1-to-1 matching gives higher scores than many-to-1 suggests that duplicate features in the larger model dilute alignment; measuring whether many-to-1 matches concentrate on a few features or spread across many would distinguish true shared structure from a single over-used feature.
  • Beyond the paper: because all comparisons use the same prompt set, the alignment could reflect shared task statistics rather than shared latent geometry; an out-of-distribution prompt corpus would separate these.
  • Beyond the paper: if universality holds across model families, as the paper hints from prior results, a cross-family SAE feature atlas could be built, letting interpretability results migrate between unrelated architectures; the paper's single-family evidence makes this a testable conjecture rather than a demonstrated fact.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper investigates whether two Gemma-2 models of different sizes (2B and 9B) develop shared internal semantic features. Using pretrained sparse autoencoders on residual-stream activations, the authors match SAE features between models by Pearson activation correlation and then compare the matched feature spaces with SVCCA and RSA. They report that middle layers show the strongest cross-model alignment, that semantic subspaces such as 'emotion' and 'time' align better than unrelated concept pairs, and that a single multi-token subspace also shows alignment. The paper interprets these results as evidence for feature universality across model scales and as support for transferring interpretability tools between the 2B and 9B models.

Significance. If the empirical findings withstand the methodological concerns below, the paper would provide a useful data point for the feature-universality literature: it is a clean scale comparison within one model family, and it uses two complementary rotation-invariant similarity metrics. The inclusion of random-pair baselines and multiple runs of the many-to-one matching is a strength that makes the raw mid-layer alignment pattern credible. However, the central interpretive claim that this alignment reflects shared semantic concepts rather than shared token statistics or the matching procedure itself is not yet established, so the broader significance is conditional on additional controls.

major comments (4)
  1. [Section 3.1-3.2, Figs. 2, 4, 6] The main similarity scores are circularly conditioned on the matching criterion. Features are first greedily paired by maximum Pearson correlation, and then SVCCA/RSA is computed only on those selected pairs (Section 3.1 and 3.2). This means the reported alignment does not measure global feature-space overlap, as claimed in Section 3.2.1, but rather the similarity of a subset that was chosen precisely because its elements are highly correlated. The random-pair baseline (0.005-0.034) only shows that selected pairs are more aligned than arbitrary pairs; it does not control for selection-induced upper bias. The authors should either report SVCCA/RSA on the full SAE feature spaces without preselection or construct a null distribution by applying the same greedy matching procedure to unrelated activation datasets.
  2. [Section 7 and Section 3.1] The shared-tokenizer confound is not controlled. Both models use the identical Gemma tokenizer and are evaluated on identical input strings, so matched features can share token-level identity and input-embedding statistics rather than abstract semantic concepts. Section 7 explicitly concedes that same-tokenizer pairings 'lead to higher quality pairings and thus higher accuracy results,' which directly undercuts the semantic-universality interpretation. A paraphrase control with non-overlapping token sequences, or a comparison across models with different tokenizers, is needed before the conclusion 'reinforcing universality as a foundation for cross-model interpretability' can be supported. Without such a control, the mid-layer peak is a real empirical pattern but its semantic interpretation is underdetermined.
  3. [Section 4.1, 'Effect of filtering non-concept features'] The non-concept feature filter is never defined. The text states that 'unimportant features' were removed (Fig. 3b, Fig. 5b, Fig. 7b), and the caption says this is 'filtering non-concept features,' but no threshold, importance measure, or count of removed features is provided. This is a free parameter that directly affects the reported correlation increases (e.g., peak from 0.70 to 0.74), so the claim that filtering 'did not artificially boost scores' cannot be evaluated. The authors must specify the exact criterion, or remove this analysis from the main results.
  4. [Section 4.2, Tables 1-3] The semantic-subspace tables report p-values of exactly 0.0 without stating the number of shuffles, the variance of the shuffling distribution, or any multiple-comparison correction across the many tested concept pairs, layers, and metrics. In addition, the multi-token conclusion rests on a single subspace ('Emotion and Time,' Table 3), yet the text and abstract describe multi-token semantic subspaces as a general result. The conclusion in Section 4.2 that 'semantic subspaces are the better level of analysis' is not supported by n=1. The authors should report full distributions or confidence intervals, apply correction for multiple tests, and temper the multi-token claims to what the single-subspace evidence allows.
minor comments (5)
  1. [Section 3.2.1] The notation for the representational dissimilarity matrix should be R^{n×n}, not R^{nxn}.
  2. [Section 4.2, Table 3 caption] The Table 3 caption says 'for bad pairs of concepts' but the text and the table describe multi-token inputs that are 'encoded similarly'; the caption appears to be copied from Table 2 and should be corrected.
  3. [Sections 1 and 5] There is a typo: 'existance' should be 'existence.'
  4. [Section 3.1.1] The phrase 'p-value score' is informal; 'p-value' would suffice.
  5. [Appendix A] The figure ordering in the appendix is confusing: Figure 9 appears before Figures 7 and 8 in the text, and the numbering does not follow the order of citation. Renumbering the appendix figures would improve readability.

Circularity Check

1 steps flagged · score 4.0 of 10

Partial circularity: feature pairs are greedily selected by Pearson correlation and then scored with SVCCA/RSA on the same selected subset, inflating absolute alignment; the layer pattern retains independent content.

  1. fitted input called prediction [Section 3.1.1, Feature Pairings (1-to-1 and many-to-1 matching); results reported in Section 4.1, Fig. 2a and random-baseline Fig. 8a]
    "1-to-1 (bijective) matching. We iteratively build a one–to–one assignment: at each step we pick the still–unmatched pair of features with the highest Pearson correlation. ... Unless stated otherwise, all correlations are computed with Pearson correlation; the aligned pairs returned by the chosen strategy are then fed into the subsequent SVCCA and RSA calculations."

    The pairs that enter the SVCCA/RSA computation are not an unbiased sample of the two feature spaces; they are explicitly selected to maximize Pearson activation correlation, which is itself a similarity measure. The reported 'paired SVCCA' is therefore a similarity score computed on a subset that was chosen for similarity, so the high absolute values (peak 0.73) are inflated by construction. The random-pair baseline (0.005–0.034) only demonstrates that greedy matching beats random pairing; it does not remove the selection bias. The middle-layer-versus-edge layer pattern is not fully forced, because matching is applied independently to each layer pair, so the qualitative 'middle layers most similar' claim retains independent empirical content; the circularity is partial rather than total.

full rationale

The paper is a primarily empirical study of feature alignment between Gemma-2-2B and Gemma-2-9B, following the SAE-based pipeline of Lan et al. (2024). There is no load-bearing self-citation chain: the cited method is external, and the Gemma-2 measurements are new. The main circularity concern is in the pairing protocol: features are greedily matched by highest Pearson correlation, and SVCCA/RSA is then computed only on those matched pairs. Because the matching criterion is itself a similarity measure, the absolute paired-SVCCA values are upwardly biased by construction; the random-pair baseline only controls for chance-level alignment of arbitrary pairs, not for the selection of maximally correlated pairs. The paper's strongest qualitative finding—that mid-layer pairs align more than early or late layer pairs—is not an artifact of this selection, since matching is performed separately for every layer pair, so the layer-contrast pattern is an empirical result. The acknowledged same-tokenizer confound (Section 7) threatens the semantic-universality interpretation but is a validity concern, not a circularity. Overall, the central claim is partially constructed by the matching procedure, but the layer-wise pattern and the cross-model comparison retain independent content, warranting a moderate score rather than a high one.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

No new entities are introduced. The main load-bearing inputs are the SAE dictionaries and the matching/filtering choices; the filter threshold is an unstated free parameter. The axioms above capture the unstated assumptions the central claim depends on.

free parameters (1)
  • Non-concept feature importance threshold = unspecified
    The paper filters out 'unimportant' features before computing activation correlations (Fig. 3b) but never defines the criterion or threshold; the reported improvement from 0.70 to 0.74 depends on this post-hoc choice.
assumptions (4)
  • domain assumption SAE features trained on residual-stream activations are monosemantic and correspond to shared interpretable concepts.
    Invoked throughout Sections 3 and 4; this is the dictionary-learning premise inherited from Bricken et al. 2023 and Cunningham et al. 2023, and it is not independently verified in this paper.
  • domain assumption Shared tokenizer and identical input prompts provide a valid basis for cross-model feature matching.
    The paper relies on the same Gemma tokenizer for both models and acknowledges in Section 7 that a shared tokenizer 'leads to higher quality pairings.' This assumption is load-bearing for the universality claim.
  • ad hoc to paper Pearson-correlation greedy matching produces feature pairs whose aligned similarity reflects global feature-space overlap.
    Section 3.1.1 matches features by maximum Pearson correlation on a prompt set, then feeds the matched subset into SVCCA/RSA; the subset is therefore selected for high correlation, which inflates the subsequent similarity scores.
  • domain assumption Random-pair baselines control for the selection bias introduced by the matching procedure.
    Section 4.1 compares paired scores to randomly paired features with p-value buckets (e.g., p=0.0), but random pairs are not matched for high correlation, so the baseline may understate the selection effect.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Semantic Convergence: Investigating Shared Representations Across Scaled LLMs." pith.science (2026). https://pith.science/paper/VMR5KC6Q

@misc{pith2026250722918,
  author       = {Pith},
  title        = {Pith review of: Semantic Convergence: Investigating Shared Representations Across Scaled LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VMR5KC6Q}},
  note         = {Machine review of arXiv:2507.22918}
}
read the original abstract

We investigate feature universality in Gemma-2 language models (Gemma-2-2B and Gemma-2-9B), asking whether models with a four-fold difference in scale still converge on comparable internal concepts. Using the Sparse Autoencoder (SAE) dictionary-learning pipeline, we utilize SAEs on each model's residual-stream activations, align the resulting monosemantic features via activation correlation, and compare the matched feature spaces with SVCCA and RSA. Middle layers yield the strongest overlap, while early and late layers show far less similarity. Preliminary experiments extend the analysis from single tokens to multi-token subspaces, showing that semantically similar subspaces interact similarly with language models. These results strengthen the case that large language models carve the world into broadly similar, interpretable features despite size differences, reinforcing universality as a foundation for cross-model interpretability.

Figures

Figures reproduced from arXiv: 2507.22918 by the authors.

Figure 1
Figure 1. Workflow of pairing rows with the highest correlated features between two models (Gemma-2-2B and [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. (a) SVCCA and (b) RSA 1-to-1 paired scores of SAEs for layers in Gemma-2-2B vs Gemma-2-9B. Note [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. (a) 1-to-1 Mean Activation Correlation before and (b) after filtering non-concept features for Gemma-2-2B [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (13 more)
Figure 9
Figure 9. Figure 9: (a) Mean Randomly Paired SVCCA Many-to￾1 scores and (b) SVCCA Many-to-1 P-values of SAEs for layers in Gemma-2-2B vs Gemma-2-9B [PITH_FULL_IMAGE:figures/full_fig_p011_9.png]
Figure 4
Figure 4. Figure 4: (a) SVCCA and (b) RSA Many-to-1 paired scores of SAEs for layers in Gemma-2-2B vs Gemma￾2-9B. Note the pattern of higher scores in the middle layers indicating similarity in middle layers between both models [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]
Figure 5
Figure 5. Figure 5: (a) Many-to-1 Mean Activation Correlation [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 6
Figure 6. Figure 6: (a) SVCCA and (b) RSA Many-to-1 (5 run average) paired scores of SAEs for layers in Gemma-2- 2B vs Gemma-2-9B. Note the pattern of higher scores in the middle layers indicating similarity in middle layers between both models. This pattern is a trend in layer similarity…
Figure 13
Figure 13. Figure 13: Highest similarity sits in the center stack; [PITH_FULL_IMAGE:figures/full_fig_p012_13.png]
Figure 14
Figure 14. Figure 14: Alignment remains centered; deep and early [PITH_FULL_IMAGE:figures/full_fig_p012_14.png]
Figure 15
Figure 15. Figure 15: Mid-layer RSA peaks near 0.17; edge layers [PITH_FULL_IMAGE:figures/full_fig_p012_15.png]
Figure 16
Figure 16. Figure 16: Mid-to-deep pairs score higher: Biology, Month-Names reach ∼0.25 at L19 [PITH_FULL_IMAGE:figures/full_fig_p013_16.png]
Figure 17
Figure 17. Figure 17: Highest scores at 9B L19: Biology ∼0.30, Countries ∼0.25. L2 and L39 remain near zero [PITH_FULL_IMAGE:figures/full_fig_p013_17.png]
Figure 18
Figure 18. Figure 18: Strongest alignment at L19; outer layers [PITH_FULL_IMAGE:figures/full_fig_p013_18.png]
Figure 22
Figure 22. Figure 22: paired SVCCA 1-to-1 for country-time con [PITH_FULL_IMAGE:figures/full_fig_p014_22.png]
Figure 23
Figure 23. Figure 23: paired SVCCA 1-to-1 for nature-people con [PITH_FULL_IMAGE:figures/full_fig_p014_23.png]
Figure 24
Figure 24. Figure 24: paired SVCCA 1-to-1 for emotion-time con [PITH_FULL_IMAGE:figures/full_fig_p014_24.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

29 extracted references · 8 canonical work pages

  1. [4]

    Preprint, arXiv:2304.01373

    Pythia: A suite for analyzing large language models across training and scaling. Preprint, arXiv:2304.01373. Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Hatfield-Dodds...

    arXiv

  2. [5]

    Transformer Circuits Thread

    Towards monosemanticity: Decom- posing language models with dictionary learning. Transformer Circuits Thread. Https://transformer- circuits.pub/2023/monosemantic- features/index.html. S´ebastien Bubeck, Varun Chandrasekaran, Ronen El- dan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Pe- ter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, Harsha Nori, Hamid Pa...

    work page 2023

  3. [6]

    Preprint, arXiv:2303.12712

    Sparks of artificial general in- telligence: Early experiments with gpt-4. Preprint, arXiv:2303.12712. Bilal Chughtai, Lawrence Chan, and Neel Nanda

    arXiv

  4. [7]

    Preprint, arXiv:2302.03025

    A toy model of universality: Reverse engineering how networks learn group operations. Preprint, arXiv:2302.03025. Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey

    arXiv

  5. [8]

    Preprint, arXiv:2309.08600

    Sparse autoencoders find highly interpretable features in language models. Preprint, arXiv:2309.08600. Mathieu Dehouck

    arXiv

  6. [11]

    Preprint, arXiv:2305.19911

    Neuron to graph: Interpreting language model neurons at scale. Preprint, arXiv:2305.19911. Leo Gao, Tom Dupr´e la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu

    arXiv

  7. [12]

    Preprint, arXiv:2406.04093

    Scaling and evaluating sparse autoencoders. Preprint, arXiv:2406.04093. Albert Garde, Esben Kran, and Fazl Barez

    arXiv

  8. [13]

    DeepDecipher: Accessing and Investigating Neuron Activation in Large Language Models

    Deepdecipher: Accessing and investigating neu- ron activation in large language models. Preprint, arXiv:2310.01870. Ryan Greenblatt, Carson Denison, Benjamin Wright, Fabien Roger, Monte MacDiarmid, Sam Marks, Jo- hannes Treutlein, Tim Belonax, Jack Chen, David Du- venaud, Akbir Khan, Julian Michael, S¨oren Minder- mann, Ethan Perez, Linda Petrini, Jonatha...

    work page Pith review arXiv

Show all 29 references
  1. [14]

    arXiv preprint

    Alignment faking in large language models. arXiv preprint. ArXiv:2412.14093 [cs]. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al

    arXiv

  2. [15]

    arXiv preprint arXiv:2501.12948

    Deepseek-r1: In- centivizing reasoning capability in llms via reinforce- ment learning. arXiv preprint arXiv:2501.12948. Wes Gurnee, Theo Horsley, Zifan Carl Guo, Tara Rezaei Kheirkhah, Qinyi Sun, Will Hathaway, Neel Nanda, and Dimitris Bertsimas

    arXiv

  3. [16]

    Preprint, arXiv:2401.12181

    Universal neurons in gpt2 language models. Preprint, arXiv:2401.12181. Dan Hendrycks, Mantas Mazeika, and Thomas Wood- side

    arXiv

  4. [17]

    Preprint, arXiv:2306.12001

    An overview of catastrophic ai risks. Preprint, arXiv:2306.12001. Harold Hotelling

    arXiv

  5. [18]

    arXiv preprint

    Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training. arXiv preprint. ArXiv:2401.05566 [cs]. Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton

    arXiv

  6. [20]

    arXiv preprint arXiv:2410.06981

    Sparse autoencoders reveal universal feature spaces across large language models. arXiv preprint arXiv:2410.06981. Lauro Langosco Di Langosco, Jack Koch, Lee D. Sharkey, Jacob Pfau, and David Krueger

    arXiv

  7. [22]

    Accessed: 2024-06-19

    Progress up- date #1 from the gdm mech interp team: Full update. Accessed: 2024-06-19. Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter

    2024

  8. [25]

    arXiv preprint

    Goal Misgeneralization: Why Correct Specifications Aren’t Enough For Correct Goals. arXiv preprint. ArXiv:2210.01790 [cs]. Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivi `ere, Mihir Sanjay Kale, Juliett...

    arXiv

  9. [26]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth´ee Lacroix, Baptiste Rozi `ere, Naman Goyal, Eric Hambro, Faisal Azhar, et al

    Do sparse autoencoders find ’true features’? Accessed: 2025-01-29. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth´ee Lacroix, Baptiste Rozi `ere, Naman Goyal, Eric Hambro, Faisal Azhar, et al

    2025

  10. [27]

    arXiv preprint arXiv:2302.13971

    Llama: Open and effi- cient foundation language models. arXiv preprint arXiv:2302.13971. Iulia Turc, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova

    arXiv

  11. [28]

    arXiv preprint arXiv:1908.08962

    Well-read students learn better: On the importance of pre-training compact models. arXiv preprint arXiv:1908.08962. Pedro HV Valois, Lincon S Souza, Erica K Shimo- moto, and Kazuhiro Fukui

    1908 arXiv

  12. [29]

    arXiv preprint arXiv:2412.07334

    Frame represen- tation hypothesis: Multi-token llm interpretability and concept-guided text generation. arXiv preprint arXiv:2412.07334. Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson

  13. [2010]

    https://wordnet.princeton.edu/

    Wordnet: Lexical database for english. https://wordnet.princeton.edu/. Ac- cessed: 2025-05-18. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al

    2025

  14. [2013]

    arXiv preprint arXiv:1312.5663

    k-sparse autoencoders. arXiv preprint arXiv:1312.5663. Neel Nanda and Arthur Conmy

    arXiv

  15. [2017]

    In Neural Information Processing Systems

    Svcca: Sin- gular vector canonical correlation analysis for deep learning dynamics and interpretability. In Neural Information Processing Systems. Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Tom Lieberum, Vikrant Varma, J´anos Kram´ar, Rohin Shah, and Neel Nanda. 2024...

    arXiv

  16. [2019]

    Preprint, arXiv:1905.00414

    Similarity of neu- ral network representations revisited. Preprint, arXiv:1905.00414. Nikolaus Kriegeskorte, Marieke Mur, and Peter Ban- dettini

    1905 arXiv

  17. [2020]

    Zoom in: An introduction to circuits. Distill. Https://distill.pub/2020/circuits/zoom-in. Bruno A. Olshausen and David J. Field

    2020

  18. [2022]

    Transformer Circuits Thread

    Toy models of superposition. Transformer Circuits Thread. Https://transformer- circuits.pub/2022/toy model/index.html. Joshua Engels, David D Baek, Subhash Kantamneni, and Max Tegmark

    2022

  19. [2023]

    arXiv preprint arXiv:2303.08774

    Gpt-4 technical report. arXiv preprint arXiv:2303.08774. Leonard Bereska and Efstratios Gavves

    arXiv

  20. [2024]

    Preprint, arXiv:2404.14082

    Mech- anistic interpretability for ai safety – a review. Preprint, arXiv:2404.14082. Stella Biderman, Hailey Schoelkopf, Quentin Anthony, Herbie Bradley, Kyle O’Brien, Eric Hallahan, Mo- hammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, Aviya Skowron, Lin...

    arXiv

  21. [2025]

    arXiv preprint arXiv:2504.18530

    Scaling laws for scalable oversight. arXiv preprint arXiv:2504.18530. Alex Foote, Neel Nanda, Esben Kran, Ioannis Kon- stas, Shay Cohen, and Fazl Barez

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.