Pith. sign in

REVIEW 4 major objections 6 minor 3 cited by

Evaluating SAE interpretability without explanations

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that SAE latent interpretability can be measured by intruder detection—spotting the one non-activating example among activating ones—without generating any natural-language explanation, and that LLM judges agree with…

desk verdict A genuinely useful idea for SAE evaluation — but the human validation is too thin to carry the 0.85 correlation. read the letter →

arxiv 2507.08473 v1 pith:75TSUC5W submitted 2025-07-11 cs.LG

Gonçalo Paulo , Nora Belrose This is my paper
classification cs.LG
keywords sparseautoencodersinterpretabilityevaluationintruderdetectionactivationdecilesexampleembeddingscoringLLMlatentmechanistic
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Sparse autoencoders (SAEs) are neural-network tools that try to split a model's messy internal states into discrete, human-readable pieces, but measuring how interpretable those pieces are has relied on generating written explanations first. This paper argues that interpretability can instead be measured directly from activations, by asking an evaluator to pick out the one activating example that does not belong—an intruder-detection task adapted from topic-model word intrusion. The load-bearing evidence is a Spearman correlation of 0.85 between LLM and human intruder-detection accuracy across 56 latents, with humans averaging 65% accuracy. If this holds, SAE interpretability becomes something that can be scored without the extra steps, prompt choices, and philosophical assumption that interpretable latents must be expressible in a sentence.

What carries the argument

The intruder sentence detection task is the central instrument: for each latent, four activating contexts and one non-activating intruder are shown in a numbered list, activating tokens are highlighted with << and >>, and the evaluator (human or LLM) must report which item is the intruder; the latent's score is average accuracy across deciles. The companion example-embedding scoring replaces the judge with cosine-similarity distances: two query examples (one activating, one not) are compared against sets of activating and non-activating examples, and the separation is summarized as an AUROC. These tasks carry the argument by removing explanation generation entirely, so that interpretability is a property of the latent's activation behavior rather than of a paraphrase.

What would settle it

Recruit independent human participants with no prior exposure to the SAE or the examples, give them the same intruder-detection prompts, and compare their per-latent accuracy with the authors' scores: if the correlation falls well below the reported inter-human agreement of 0.87, the validation of LLM intruder scores as a measure of interpretability is unsupported.

Watch

Extended reading notes

Core claim

The paper claims that interpretability of a sparse-autoencoder latent is the ability of a judge to distinguish examples that activate the latent from examples that do not, and that this ability can be measured by intruder detection without any natural-language explanation of the latent. Humans were given five 32-token contexts—four activating examples sampled from the same activation decile plus one non-activating intruder—and asked to find the intruder; a strong LLM doing the same task agreed with the human scores at Spearman correlation 0.85 over 56 latents drawn from four SAEs trained on SmolLM2 135M. The paper also introduces example-embedding scoring, which measures whether activating examples cluster in embedding space, and finds it faster but much weaker, correlating with human scores at 0.48. Across activation deciles, both methods show high-activation examples are substantially easier to judge, and an intruder-decile variant reveals that most latents do not behave like binary monosemantic features: accuracy depends on which decile pairs are used and is strongly asymmetric.

Load-bearing premise

The load-bearing assumption is that the human scores—produced by the authors on 56 latents with roughly 10–20 prompts per latent—are representative of what naive human judges would say, and were not inflated or anchored by the authors' prior knowledge of the latents and examples.

Editorial extensions

If this is right

  • SAE interpretability can be benchmarked by direct activation tests, so evaluation no longer depends on explanation generation prompts or on the existence of a good short verbal description of each latent.
  • LLM judges can stand in for human judges on intruder detection, since their per-latent scores correlate at 0.85 with human scores; different strong LLMs also agree closely with one another.
  • Interpretability varies with activation strength: examples from the highest activation decile score up to 20% higher than examples from the lowest decile, while even low-decile examples remain above chance.
  • The intruder-decile variant rules out the simple picture that most SAE features are binary and monosemantic, because accuracy between activation deciles is far above random and highly asymmetric.
  • Example embedding scoring is fast and uses a small model, but in these experiments its AUROCs sit close to random and it correlates with human scores at only 0.48, so it is a weaker proxy than LLM intruder detection.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the 0.85 human–LLM agreement generalizes to other SAE architectures and models, intruder detection could become a standardized, cheap screening metric for SAE training runs, letting practitioners compare latent quality across checkpoints without writing explanations.
  • The intruder-decile asymmetry could be developed into a diagnostic for feature type: symmetric distance-dependent confusion would indicate scalar features, close-to-chance confusion would indicate binary features, and the observed pattern suggests a graded or context-dependent structure that might be formalized as a soft feature hierarchy.
  • Because the method uses only highlighted tokens and contexts, it should transfer to vision, audio, or protein-model SAEs, where natural-language explanations are less natural; this is an extension the paper motivates but does not test.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This paper proposes two methods for evaluating the interpretability of sparse autoencoder (SAE) latents without generating natural-language explanations: intruder detection, where an evaluator (human or LLM) must pick the one non-activating example among five candidate contexts, and example embedding scoring, which measures whether activating examples cluster in the embedding space of a small sentence transformer. The authors apply the methods to SAEs trained on SmolLM2 135M, report that Claude Sonnet 3.5's intruder-detection accuracy correlates with human accuracy at Spearman ρ=0.85 across 56 latents (Section 4.1), and use the intruder setup to study how interpretability varies with activation decile. They conclude that explanation-free evaluation can provide a standardized measure of interpretability and that example embedding scoring, though currently weak, is a promising fast alternative.

Significance. The contribution is methodologically fresh: replacing explanation-based simulation with a direct discrimination task sidesteps a known confound between explanation generation and latent interpretability, and the intruder task is simple enough to administer to both humans and LLMs on the same inputs. The authors should be credited for transparently reporting negative results (example embedding AUROCs near 0.5, below-chance accuracy in some decile-intruder conditions) and for comparing multiple LLMs. However, the central validity claim rests almost entirely on a small human gold standard: 56 latents from a single model and SAE configuration, rated by the two authors with 10-20 prompts per latent. If the headline correlation survives independent, decile-balanced human evaluation, the method would be a valuable contribution; the current evidence is suggestive but not yet conclusive.

major comments (4)
  1. [Sections 3.1 and 3.1.1, Figure 3] The comparison between human and LLM intruder scores is confounded by the decile sampling scheme. In Section 3.1, the latent's intruder detection score is defined as accuracy averaged over deciles, so the LLM score is balanced across the activation distribution. For the human evaluation (Section 3.1.1), however, the decile for each prompt is chosen randomly and not shown to the participant, and each latent receives only 10-20 prompts. Because accuracy varies strongly with decile (Figure 3, left panel), a latent that happens to draw mostly high-activation deciles in the human session will have an inflated human score, independently of its interpretability. The paper does not report the per-latent decile composition of human prompts or control for decile in the reported Spearman correlation of 0.85. To support the headline correlation, the authors should either match the human prompt deciles to the LLM's decile distribution, compute the correlation separately per decile or with decile as a covariate, or demonstrate that the decile composition is roughly uniform across latents.
  2. [Sections 3.1.1 and 4.1.2] The human gold standard is produced by the two authors themselves, with only 10-20 prompts per latent in the main set and 3-5 prompts per latent in the inter-rater subset. Section 4.1.2 reports a Pearson inter-rater correlation of 0.87 on 40 latents, and Claude's correlations with the two raters of 0.86 and 0.69. The gap between the two per-rater correlations is large and unexplained, and the headline 0.85 Spearman value is close to the higher per-rater correlation. Because the authors designed the task and trained the SAEs, their judgments may reflect prior knowledge of the latents or the examples rather than naive perception. The validity claim would be substantially strengthened by recruiting independent naive raters on at least a subset of latents, by reporting per-rater correlations (both Spearman and Pearson) with confidence intervals, and by reporting the agreement on the binned interpretability categories shown in Figure 2.
  3. [Section 3.1] The procedure for selecting non-activating intruder examples is under-specified. The manuscript states that the intruder is "drawn from a pool of examples that do activate other latents but do not trigger the latent being evaluated," but it does not describe how that pool is constructed: whether the examples come from the same corpus, whether they are matched on token length, format, topic, or perplexity, and whether they are random contexts or adversarially selected. If the non-activating examples are not matched on surface features, the task can be solved by detecting format, topic, or lexical outliers rather than by understanding the latent's semantics. The above-chance accuracy in the lowest deciles (Figure 3, left panel) is interpreted as evidence that low-activating examples are informative, but it could instead reflect such low-level cues. Please specify the intruder sampling procedure in detail and add a control condition (e.g., intruders matched on perplexity or topic) to demonstrate that the task measures latent semantics rather than superficial text properties.
  4. [Sections 4.1 and 4.1.2, Tables 1 and 2] The paper reports several Pearson and Spearman correlations without confidence intervals, p-values, or tests of differences between correlations. Given the small sample of 56 latents and their clustering within four layers, the effective sample size is likely smaller than 56. The headline correlation between human and LLM intruder accuracy should be accompanied by bootstrap or permutation confidence intervals, and the analysis should be stratified by layer to ensure that the 0.85 correlation is not driven primarily by between-layer differences in interpretability. Without such measures, the reader cannot assess the precision of the central estimate or the claimed superiority over previous evaluation metrics.
minor comments (6)
  1. [Section 4.1] There are several typos: "wether" should be "whether", "Has found" should be "As found", and the Figure 1 caption contains "colection" instead of "collection".
  2. [Figure 2 caption] The caption states that the dot size represents the number of prompts the human saw, with smaller dots representing 8 prompts, but the text says humans saw on average 10 to 20 prompts per latent. Please clarify the range and the meaning of the dot size.
  3. [Table 1] The table lists both "Human 1" and "Human" as columns/rows without explaining whether "Human" is the second author, a pooled score, or an average. Define all evaluator labels in the caption or text.
  4. [Section 4.1] The statement that the 0.85 correlation is "a higher agreement than what was seen in previous SAE evaluation metrics (Paulo et al., 2024)" compares correlations across different tasks, different human evaluators, and different compute budgets; this should be qualified as not directly comparable.
  5. [Code Availability] The code availability statement says the scorers "can be used in our pipeline ... delphi," but no URL or repository details are provided. Please provide a concrete link or release instructions to make the code and human ratings accessible.
  6. [Section 4.2] The explanation for why example embedding scoring fails to reach high AUROC, namely that the embedding model does not understand the << and >> highlighting, is plausible but untested. A simple ablation removing the highlights or fine-tuning the embedding model on highlighted contexts would clarify the mechanism.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular reduction: LLM intruder scores are validated against human judgments and not against fitted parameters or a self-cited theorem; author-produced human labels are an independence concern, not circularity.

full rationale

The central empirical chain is not circular. The paper operationalizes interpretability as intruder-detection accuracy and reports a Spearman correlation of 0.85 between Claude Sonnet 3.5 and human scores on 56 latents; the human scores are a measured gold standard, not a fitted parameter, and the LLM scores are not constructed from them (Section 4.1). The inter-human correlation of 0.87 on a separate set of 40 latents is likewise an independent measurement (Section 4.1.2). The definition of interpretability as a human being able to distinguish activating from non-activating examples is a stipulated operationalization, and the intruder task implements that definition; the paper does not derive the metric from a theorem or from a self-cited uniqueness claim. Self-citations to Paulo et al. (2024) and Paulo and Belrose (2025) appear for comparison metrics (fuzzing, detection), for the inspiration behind example-embedding scoring, and for the philosophical assumption about natural-language explanations, but none of these citations carries the load of the human-LLM validation. The most significant weakness is that the human raters are the authors themselves, with only 10-20 prompts per latent in the main set and 3-5 prompts per latent in the inter-human subset; this threatens the independence and generalizability of the gold standard but does not constitute a by-construction reduction of a prediction to its inputs. The score of 2 reflects these minor self-referential elements, not substantive circularity.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The mechanism draws no new entities and fits no free parameters to data; its dependence is methodological: a specific definition of interpretability, a human gold standard supplied by the authors, and the assumption that highlighting and embeddings carry the relevant signal.

free parameters (1)
  • non-activating threshold (activation level defining 'does not trigger the latent')
    Section 3.1 builds the intruder pool from examples that do not trigger the evaluated latent, but the exact activation cutoff is never stated; different thresholds would change intruder difficulty and scores.
assumptions (4)
  • domain assumption Interpretability is operationalized as the ability to distinguish activating from non-activating examples.
    Stated in Section 1; the intruder task directly implements this definition, so the metric inherits its validity from this assumption about what interpretability is.
  • domain assumption Human evaluation is the gold standard for interpretability.
    Section 2.1 asserts this; all metric validation is relative to human scores, making the reliability of the human baseline load-bearing.
  • domain assumption Highlighting activating tokens with << >> provides sufficient signal and does not bias the intruder task.
    Section 3.1 highlights tokens in all examples; the paper does not ablate highlighting, so scores may depend on this visual cue rather than on latent behavior alone.
  • domain assumption Cosine similarity in the MiniLM embedding space reflects semantic relatedness of activating contexts.
    Section 3.2 relies on this for AUROC computation; the near-random results suggest the assumption holds only weakly for this embedding model.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Evaluating SAE interpretability without explanations." pith.science (2026). https://pith.science/paper/75TSUC5W

@misc{pith2026250708473,
  author       = {Pith},
  title        = {Pith review of: Evaluating SAE interpretability without explanations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/75TSUC5W}},
  note         = {Machine review of arXiv:2507.08473}
}
read the original abstract

Sparse autoencoders (SAEs) and transcoders have become important tools for machine learning interpretability. However, measuring how interpretable they are remains challenging, with weak consensus about which benchmarks to use. Most evaluation procedures start by producing a single-sentence explanation for each latent. These explanations are then evaluated based on how well they enable an LLM to predict the activation of a latent in new contexts. This method makes it difficult to disentangle the explanation generation and evaluation process from the actual interpretability of the latents discovered. In this work, we adapt existing methods to assess the interpretability of sparse coders, with the advantage that they do not require generating natural language explanations as an intermediate step. This enables a more direct and potentially standardized assessment of interpretability. Furthermore, we compare the scores produced by our interpretability metrics with human evaluations across similar tasks and varying setups, offering suggestions for the community on improving the evaluation of these techniques.

Figures

Figures reproduced from arXiv: 2507.08473 by the authors.

Figure 1
Figure 1. Evaluating the interpretability of SAE latents. To evaluate the interpretability of SAE latents, activations of the latents are collected over a colection of text. Traditionally these are then used to generate explanations, which are then scored. This process is an indirect measurement of the latents interpretability, and specific choices when generating explanations can influence the scores. Instead, we propose to … view at source ↗
Figure 2
Figure 2. Correlation between human and LLM intruder detection accuracy. In the left panel we compare the accuracy on the intruder task for the LLM - in this case Claude Sonnet 3.5 - and that of a human. In the right panel we show a more coarse grained classification. All latents which have less than that 0.2 accuracy are considered non interpretable, and different degrees of interpretability are assigned to the other 4 bins … view at source ↗
Figure 3
Figure 3. Interpretability of the distribution of activations. The activation examples used on the intruder task come from different deciles of the distribution of the latent’s activations. On the lower activating deciles, it’s harder to distinguish which is the intruder example, but the accuracy is still significantly above random. If instead of using non-activating examples we use examples from a different decile, we can ob… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Interpretability evaluation using example embedding scoring. In the left panel, we show that the highest activating decile are easier to distinguish from non activating examples, as was the case with intruder detection. On the panel on the right, it is possible to see …

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Descriptive Collision in Sparse Autoencoder Auto-Interpretability: When One Explanation Describes Many Features

    cs.LG 2026-05 accept novelty 8.0 of 10

    Many distinct SAE features share identical explanations, with the average annotation resolving only 70% of feature identity in a large annotated dataset.

  2. Sign-Aware Gated Sparse Autoencoders: Modeling Anticorrelated Features with Bi-Jump-ReLU Activations

    cs.LG 2026-05 conditional novelty 7.0 of 10

    SA-GSAE with Bi-Jump-ReLU enables one latent to encode both polarities of anticorrelated features, Pareto-dominating or matching full-width gated SAEs while reducing dead latents by up to 500x on some LLM hookpoints.

  3. Sign-Aware Gated Sparse Autoencoders: Modeling Anticorrelated Features with Bi-Jump-ReLU Activations

    cs.LG 2026-05 conditional novelty 6.0 of 10

    A half-width sign-aware gated sparse autoencoder matches full-width Gated SAE reconstruction on six LLM hookpoints while cutting dead features by 0.35–0.82 absolute at matched sparsity.

Reference graph

Works this paper leans on

31 extracted references · 10 canonical work pages · cited by 2 Pith papers

  1. [1]

    From mechanistic interpretability to mechanistic biology: Training, evaluating, and interpreting sparse autoencoders on protein language models

    Etowah Adams, Liam Bai, Minji Lee, Yiyang Yu, and Mohammed AlQuraishi. From mechanistic interpretability to mechanistic biology: Training, evaluating, and interpreting sparse autoencoders on protein language models. bioRxiv, pp.\ 2025--02, 2025

    work page 2025

  2. [2]

    Smollm2: When smol goes big--data-centric training of a small language model

    Loubna Ben Allal, Anton Lozhkov, Elie Bakouch, Gabriel Mart \' n Bl \'a zquez, Guilherme Penedo, Lewis Tunstall, Andr \'e s Marafioti, Hynek Kydl \' c ek, Agust \' n Piqueres Lajar \' n, Vaibhav Srivastav, et al. Smollm2: When smol goes big--data-centric training of a small language model. arXiv preprint arXiv:2502.02737, 2025

    arXiv 2025

  3. [3]

    Interpretability as compression: Reconsidering sae explanations of neural activations with mdl-saes

    Kola Ayonrinde, Michael T Pearce, and Lee Sharkey. Interpretability as compression: Reconsidering sae explanations of neural activations with mdl-saes. arXiv preprint arXiv:2410.11179, 2024

    arXiv 2024

  4. [4]

    Language models can explain neurons in language models

    Steven Bills, Nick Cammarata, Dan Mossing, Henk Tillman, Leo Gao, Gabriel Goh, Ilya Sutskever, Jan Leike, Jeff Wu, and William Saunders. Language models can explain neurons in language models. URL https://openaipublic. blob. core. windows. net/neuron-explainer/paper/index. html.(Date accessed: 14.05. 2023), 2, 2023

    work page 2023

  5. [5]

    Exemplary Natural Images Explain CNN Activations Better than State-of-the-Art Feature Visualization

    Judy Borowski, Roland S. Zimmermann, Judith Schepers, Robert Geirhos, Thomas S. A. Wallis, Matthias Bethge, and Wieland Brendel. Exemplary natural images explain cnn activations better than state-of-the-art feature visualization, 2020. URL https://arxiv.org/abs/2010.12606

    work page Pith review arXiv 2020

  6. [6]

    Reading tea leaves: How humans interpret topic models

    Jonathan Chang, Sean Gerrish, Chong Wang, Jordan Boyd-Graber, and David Blei. Reading tea leaves: How humans interpret topic models. Advances in neural information processing systems, 22, 2009

    work page 2009

  7. [7]

    A is for absorption: Studying feature splitting and absorption in sparse autoencoders, 2024

    David Chanin, James Wilken-Smith, Tomáš Dulka, Hardik Bhatnagar, and Joseph Bloom. A is for absorption: Studying feature splitting and absorption in sparse autoencoders, 2024. URL https://arxiv.org/abs/2409.14507

    arXiv 2024

  8. [8]

    Sparse autoencoders find highly interpretable features in language models

    Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. Sparse autoencoders find highly interpretable features in language models. arXiv preprint arXiv:2309.08600, 2023

    arXiv 2023

Show all 31 references
  1. [9]

    Scaling and evaluating sparse autoencoders

    Leo Gao, Tom Dupr \'e la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. Scaling and evaluating sparse autoencoders. arXiv preprint arXiv:2406.04093, 2024

    2024 arXiv

  2. [10]

    The missing curve detectors of inceptionv1: Applying sparse autoencoders to inceptionv1 early vision, 2024

    Liv Gorton. The missing curve detectors of inceptionv1: Applying sparse autoencoders to inceptionv1 early vision, 2024. URL https://arxiv.org/abs/2406.03662

    2024 arXiv

  3. [11]

    Enhancing automated interpretability with output-centric feature descriptions, 2025

    Yoav Gur-Arieh, Roy Mayan, Chen Agassy, Atticus Geiger, and Mor Geva. Enhancing automated interpretability with output-centric feature descriptions, 2025. URL https://arxiv.org/abs/2501.08319

    2025 arXiv

  4. [12]

    Saebench: A comprehensive benchmark for sparse autoencoders in language model interpretability, 2025

    Adam Karvonen, Can Rager, Johnny Lin, Curt Tigges, Joseph Bloom, David Chanin, Yeu-Tong Lau, Eoin Farrell, Callum McDougall, Kola Ayonrinde, Matthew Wearden, Arthur Conmy, Samuel Marks, and Neel Nanda. Saebench: A comprehensive benchmark for sparse autoencoders in language mod...

    2025 arXiv

  5. [13]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization, 2017. URL https://arxiv.org/abs/1412.6980

    2017 arXiv

  6. [14]

    From superposition to sparse codes: interpretable representations in neural networks

    David Klindt, Charles O'Neill, Patrik Reizinger, Harald Maurer, and Nina Miolane. From superposition to sparse codes: interpretable representations in neural networks. arXiv preprint arXiv:2503.01824, 2025

    2025 arXiv

  7. [15]

    Concept bottleneck models

    Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept bottleneck models. ArXiv, abs/2007.04612, 2020. URL https://api.semanticscholar.org/CorpusID:220424448

    2007 arXiv

  8. [16]

    Learning biologically relevant features in a pathology foundation model using sparse autoencoders, 2024

    Nhat Minh Le, Ciyue Shen, Neel Patel, Chintan Shah, Darpan Sanghavi, Blake Martin, Alfred Eng, Daniel Shenker, Harshith Padigela, Raymond Biju, Syed Ashar Javed, Jennifer Hipp, John Abel, Harsha Pokkalla, Sean Grullon, and Dinkar Juyal. Learning biologically relevant features ...

    2024 arXiv

  9. [17]

    Gemma scope: Open sparse autoencoders everywhere all at once on gemma 2, 2024

    Tom Lieberum, Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Nicolas Sonnerat, Vikrant Varma, János Kramár, Anca Dragan, Rohin Shah, and Neel Nanda. Gemma scope: Open sparse autoencoders everywhere all at once on gemma 2, 2024. URL https://arxiv.org/abs/2408.05147

    2024 arXiv

  10. [18]

    Compositional explanations of neurons

    Jesse Mu and Jacob Andreas. Compositional explanations of neurons. ArXiv, abs/2006.14032, 2020. URL https://api.semanticscholar.org/CorpusID:220055965

    2006 arXiv

  11. [19]

    Partially rewriting a transformer in natural language, 2025

    Gon c alo Paulo and Nora Belrose. Partially rewriting a transformer in natural language, 2025. URL https://arxiv.org/abs/2501.18838

    2025 arXiv

  12. [20]

    Automatically interpreting millions of features in large language models, 2024

    Gon c alo Paulo, Alex Mallen, Caden Juang, and Nora Belrose. Automatically interpreting millions of features in large language models, 2024. URL https://arxiv.org/abs/2410.13928

    2024 arXiv

  13. [21]

    Spine: Sparse interpretable neural embeddings

    Anant Subramanian, Danish Pruthi, Harsh Jhamtani, Taylor Berg-Kirkpatrick, and Eduard Hovy. Spine: Sparse interpretable neural embeddings. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018

    2018

  14. [22]

    Unpacking sdxl turbo: Interpreting text-to-image models with sparse autoencoders, 2024

    Viacheslav Surkov, Chris Wendler, Mikhail Terekhov, Justin Deschenaux, Robert West, and Caglar Gulcehre. Unpacking sdxl turbo: Interpreting text-to-image models with sparse autoencoders, 2024. URL https://arxiv.org/abs/2410.22366

    2024

  15. [23]

    The llama 3 herd of models, 2024

    Llama Team. The llama 3 herd of models, 2024. URL https://arxiv.org/abs/2407.21783

    2024 arXiv

  16. [24]

    Daniel Freeman, Theodore R

    Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, Hoagy Cunningham, Nicholas L Turner, Callum McDougall, Monte MacDiarmid, C. Daniel Freeman, Theodore R. Sumers, Edward Rees, Joshua ...

    2024

  17. [25]

    A sample survey study of poly-semantic neurons in deep cnns

    Changwan Zhang and Yue Wang. A sample survey study of poly-semantic neurons in deep cnns. In Conference on Computer Graphics, Artificial Intelligence, and Data Processing, 2023. URL https://api.semanticscholar.org/CorpusID:258884786

    2023

  18. [26]

    Zimmermann, Thomas Klein, and Wieland Brendel

    Roland S. Zimmermann, Thomas Klein, and Wieland Brendel. Scale alone does not improve mechanistic interpretability in vision models. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https://openreview.net/forum?id=OZ7aImD4uQ

    2023

  19. [27]

    Zimmermann, David A

    Roland S. Zimmermann, David A. Klindt, and Wieland Brendel. Measuring mechanistic interpretability at scale without humans. In ICLR 2024 Workshop on Representational Alignment, 2024. URL https://openreview.net/forum?id=M8yBcvRvwn

    2024

  20. [28]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  21. [29]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  22. [30]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  23. [31]

    f.[RQ^ UWM9 =qNղl 8sj W s sn|x` Wy i G2 q N

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

    2000

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.