Pith. sign in

REVIEW 3 major objections 5 minor 35 cited by

BatchTopK Sparse Autoencoders

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read BatchTopK sparse autoencoders beat per-sample TopK by relaxing sparsity to the batch.

desk verdict The BatchTopK training idea is a genuine improvement over TopK, but the inference threshold in Eq. 7 does not reproduce batch-level top-nk selection, so the 'same average sparsity' claim is unsubstantiated until fixed. read the letter →

arxiv 2412.06410 v1 pith:DJMEJBMP submitted 2024-12-09 cs.LG cs.AIstat.ML

Bart Bussmann , Patrick Leask , Neel Nanda This is my paper · ORCID
classification cs.LGcs.AIstat.ML
keywords BatchTopKsparseautoencoderstop-ksparsitylanguagemodelinterpretabilityreconstructionfidelityJumpReLUdictionarylearningvariable
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Sparse autoencoders that fix a per-sample top-$k$ budget waste latents on easy inputs and starve hard ones. This paper relaxes the constraint to the batch: keep the top $n \times k$ activations across a whole batch, so easy samples use few latents and hard ones use many. The authors claim BatchTopK SAEs consistently reconstruct activations of GPT-2 Small and Gemma 2 2B better than standard TopK SAEs at the same average sparsity, and match or beat JumpReLU SAEs in most settings. A bonus is that average sparsity is set directly by choosing $k$, avoiding the costly hyperparameter sweep JumpReLU requires.

What carries the argument

The central object is the BatchTopK function, which keeps the top $n \times k$ latent activations by value across a batch of $n$ samples rather than the top $k$ per sample. Training uses this batch-level mask on the encoder output, with the same auxiliary dead-latent loss as TopK SAEs. At inference, batch dependence is removed by converting the mask to a scalar threshold $\theta$, estimated as the average minimum positive activation across batches, and applying it as a JumpReLU; the paper argues this preserves the learned sparsity pattern.

What would settle it

On a held-out batch from the training distribution, compare the active-latent mask produced by the batch-level top-$nk$ selection against the mask produced by the scalar threshold $\theta$; if the masks disagree on many samples, or if evaluating with the true batch mask gives materially lower NMSE than evaluating with $\theta$, then the reported performance does not reflect the method as trained.

Watch

Extended reading notes

Core claim

The central claim is that replacing the per-sample TopK selection with a batch-level selection of the top $n \times k$ activations yields a sparse autoencoder with better reconstruction fidelity at the same average number of active latents. On residual-stream activations from GPT-2 Small and Gemma 2 2B, BatchTopK SAEs report lower normalized MSE and lower downstream cross-entropy degradation than TopK SAEs across dictionary sizes and sparsity levels, and reconstruction comparable to JumpReLU SAEs. The authors also show the method actually uses its flexibility: the number of active latents per sample varies widely, with one-latent samples and samples using more than 80 latents in the same batch.

Load-bearing premise

The inference-time threshold $\theta$, estimated as the average minimum positive activation across batches, must reproduce the training-time batch-level top-$nk$ selection; if it does not, the evaluated model is not the model that was trained.

Editorial extensions

If this is right

  • For a fixed average $L_0$, BatchTopK SAEs reconstruct activations with lower NMSE than TopK SAEs on both GPT-2 Small and Gemma 2 2B.
  • BatchTopK SAEs match or improve on JumpReLU reconstruction while letting the practitioner specify average sparsity directly, instead of tuning a sparsity penalty.
  • Because the active-latent count adapts to each input, BatchTopK allocates almost no latents to low-information tokens such as the BOS token, where TopK wastes its fixed budget.
  • On GPT-2 Small, BatchTopK also beats JumpReLU on downstream cross-entropy degradation; on Gemma 2 2B it beats JumpReLU only at the lowest sparsity setting ($k=16$).
  • Because BatchTopK keeps the TopK architecture up to the selection rule, its latents should remain about as interpretable as TopK latents, though interpretability is not directly measured.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the inference-time threshold faithfully reproduces batch selection, BatchTopK is a near drop-in improvement for any existing TopK SAE training pipeline.
  • The threshold estimate in Eq. 7 is an average over the minimum positive activation; calibrating $\theta$ on held-out data instead might close any residual gap between training-time and inference-time behavior.
  • The observed per-sample $L_0$ variation suggests a natural extension: make the latent budget a function of input difficulty (token position, attention entropy) rather than a global batch quantile.
  • A direct interpretability study, probing monosemanticity or firing patterns, would test whether the added flexibility preserves the interpretability that motivates SAEs.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces BatchTopK sparse autoencoders, which replace the per-sample top-k selection of TopK SAEs with a batch-level top-nk selection. This allows the number of active latents per sample to vary while keeping the average number under explicit control. At inference, the batch dependency is removed by applying a fixed threshold θ estimated from the average minimum positive activation in batches. Experiments on GPT-2 Small (layer 8) and Gemma 2 2B (layer 12) report that BatchTopK achieves lower NMSE and less CE degradation than TopK at matched average sparsity, and comparable or better performance than JumpReLU SAEs depending on the model and setting. The authors provide code and training details in the appendix.

Significance. If the central claims hold, this is a practically useful contribution: BatchTopK offers a simple modification with direct control of average L0, avoiding the costly hyperparameter sweep required for JumpReLU SAEs. The experimental scope is reasonable (two model families, several dictionary sizes), and the paper ships code, which aids reproducibility. The main weaknesses are the unvalidated inference-time threshold, which determines whether the evaluated model is actually the trained model, and the lack of reported L0 values for the JumpReLU baseline, which is required for the 'comparable sparsity' claim. These issues are local in the sense that they can be addressed with additional experiments and reporting, but they are load-bearing for the paper's central message.

major comments (3)
  1. [§3, Eq. (7)] The inference-time threshold θ defined in Eq. (7) is not shown to reproduce the training-time batch-level top-nk selection, and as written it is not the correct cutoff. Training selects the top n×k activations in a batch, so the relevant cutoff is the n×k-th largest activation value; the expression θ = EX[min{zi,j(X) | zi,j(X) > 0}] computes the expected smallest positive activation, which is generally much smaller than that cutoff. A fixed scalar threshold also cannot reproduce the batch-adaptive boundary, since per-batch selection depends on the content of the entire batch. The manuscript reports no inference-time L0 for BatchTopK, no comparison between the fixed-θ rule and the true batch-level selection rule on held-out batches, and no sensitivity of the results to the number of batches used to estimate θ. Because all evaluations in Figures 1 and 2 use the fixed-θ rule, the headline comparisons to TopK and JumpReLU may not describe the trained BatchTopK objective.
  2. [§4 and Appendix A.1] The JumpReLU comparison is not verified at matched sparsity. The paper states that the sparsity coefficient was varied so that the resulting sparsity would match k, but it never reports the actual L0 of the JumpReLU SAEs on the evaluation data. Since L0 is an outcome of JumpReLU training rather than a setting, the 'comparable performance' claim is only meaningful if the effective L0 is close to the stated k. Please report L0 for all models on the evaluation set, and for BatchTopK at inference as well, so that the average-sparsity axis of Figures 1 and 2 is explicit.
  3. [§4, Figures 1 and 2] The reported point estimates have no error bars and no information about repeated seeds. The abstract claims BatchTopK 'consistently' outperforms TopK, but with a single run per configuration it is impossible to tell whether the differences, which are often small, are within run-to-run variation. Please provide at least three seeds (or bootstrap confidence intervals) for the main comparisons, or clearly state that only single runs were performed.
minor comments (5)
  1. [Eq. (6)] The loss expression has mismatched parentheses; it should be ‖X − (BatchTopK(WencX + benc)Wdec + bdec)‖^2_2, and the norm notation should be consistently written.
  2. [Eq. (7)] The notation should specify that the minimum is taken over all samples i and latents j in a batch, and that the expectation is over batches.
  3. [§3] Please state how many batches are used to estimate θ, and whether θ is computed on training activations or held-out activations.
  4. [Abstract] The GitHub link in the abstract appears without a separating space before the URL; fix the formatting.
  5. [Figure 1 (left)] Figure 1 (left) omits JumpReLU because its L0 cannot be fixed; the caption could still report the achieved L0 values for all plotted models to make the sparsity matching explicit.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: BatchTopK's central claims are empirical comparisons against external baselines, and the inference threshold is a practical conversion rather than a fitted prediction.

full rationale

The paper's derivation chain is self-contained. BatchTopK defines its training objective (Eq. 6) with a batch-level top-nk selection, which fixes the average number of selected latents per sample to k by construction; the reported 'same average sparsity' comparison therefore follows from the definition rather than from a fitted parameter. The central claims (lower NMSE and CE degradation than TopK, comparable to JumpReLU) are empirical results evaluated on GPT-2 Small and Gemma 2 2B activations against independently trained baselines, not quantities derived from the method's own inputs. The inference-time threshold theta in Eq. 7 is estimated from training batches, but it is used as a practical mechanism to remove batch dependency, not as evidence for a predicted theoretical quantity, and no result is claimed to be a consequence of that threshold. Self-citations to [6] and [7] are baseline and motivation references that do not carry the argument: the comparisons are measured, not imported. No equation is shown to reduce to an input by construction, and no fitted value is renamed as a prediction. Potential concerns about how faithfully Eq. 7 reproduces batch-level selection are correctness risks, not circularity, since the method's empirical comparison does not depend on a derivation from the threshold.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central method rests on the standard SAE assumption of sparse decomposition, the unquantified approximation of batch-level selection by a single threshold, and unverified sparsity matching for JumpReLU baselines. No new entities are introduced.

free parameters (2)
  • inference threshold theta = average minimum positive latent activation across batches
    Defined in Eq. 7 to convert batch-dependent training to a per-sample activation function. It is estimated from the same data distribution, and its value affects evaluation results but is not optimized.
  • JumpReLU sparsity penalty (baseline) = 0.004/0.0018/0.0008 (GPT-2); 0.02/0.005/0.001 (Gemma)
    Chosen to target active latent counts k. Actual achieved L0 is not reported, which weakens the comparability of the JumpReLU baseline.
assumptions (3)
  • domain assumption Language model activations can be approximated as sparse linear combinations of learned directions
    Standard SAE premise stated in Section 2; the entire evaluation depends on this being a meaningful decomposition target.
  • ad hoc to paper The batch-level top-nk selection can be approximated at inference by a single threshold theta
    Eq. 7 defines theta as the average minimum positive activation; the paper assumes this preserves BatchTopK behavior without quantifying the approximation error.
  • domain assumption Chosen JumpReLU sparsity coefficients produce sparsity levels comparable to the BatchTopK k values
    Stated in Appendix A.1; actual L0 values are not reported, so this assumption is unverified.

how reviews work

0 comments
Cite this review

Pith. "Pith review of BatchTopK Sparse Autoencoders." pith.science (2026). https://pith.science/paper/DJMEJBMP

@misc{pith2026241206410,
  author       = {Pith},
  title        = {Pith review of: BatchTopK Sparse Autoencoders},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DJMEJBMP}},
  note         = {Machine review of arXiv:2412.06410}
}
read the original abstract

Sparse autoencoders (SAEs) have emerged as a powerful tool for interpreting language model activations by decomposing them into sparse, interpretable features. A popular approach is the TopK SAE, that uses a fixed number of the most active latents per sample to reconstruct the model activations. We introduce BatchTopK SAEs, a training method that improves upon TopK SAEs by relaxing the top-k constraint to the batch-level, allowing for a variable number of latents to be active per sample. As a result, BatchTopK adaptively allocates more or fewer latents depending on the sample, improving reconstruction without sacrificing average sparsity. We show that BatchTopK SAEs consistently outperform TopK SAEs in reconstructing activations from GPT-2 Small and Gemma 2 2B, and achieve comparable performance to state-of-the-art JumpReLU SAEs. However, an advantage of BatchTopK is that the average number of latents can be directly specified, rather than approximately tuned through a costly hyperparameter sweep. We provide code for training and evaluating BatchTopK SAEs at https://github.com/bartbussmann/BatchTopK

Figures

Figures reproduced from arXiv: 2412.06410 by the authors.

Figure 1
Figure 1. On GPT-2 Small activations, BatchTopK largely achieves better NMSE and CE than [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. On Gemma 2 2B activations, BatchTopK outperforms TopK SAEs across different values [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Distribution of the number of active latents per sample for a BatchTopK model. The [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 35 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Where You Measure Decides What You Measure: Position Selection in Ablation-Based SAE Evaluation

    cs.LG 2026-08 accept novelty 7.0 of 10

    In ablation-based SAE evaluation, the measurement token is selected by the dictionary under test, and holding that token fixed collapses most of the variance that is usually attributed to differences between dictionaries.

  2. What do Reward Models Memorize?

    cs.LG 2026-07 conditional novelty 7.0 of 10

    Counterfactual memorization maps show RMs misallocate capacity to easy pairs, memorize dataset artifacts, and overgeneralize length/compliance on unseen pairs.

  3. Beyond the Hard Budget: Sparsity Regularizers for More Interpretable Top-k Sparse Autoencoders

    cs.LG 2026-06 unverdicted novelty 7.0 of 10

    Sparsity regularizers applied before Top-k selection in SAEs improve monosemanticity and make reconstruction robust to inference-time k across vision models and datasets.

  4. Reference Feature Atlases for Mechanistic Auditing of Language Models

    cs.AI 2026-06 conditional novelty 7.0 of 10

    A reference feature atlas attaches a new language model with a linear decoder and surfaces panel-uncovered structure via a separate residual dictionary.

  5. Sign-Aware Gated Sparse Autoencoders: Modeling Anticorrelated Features with Bi-Jump-ReLU Activations

    cs.LG 2026-05 conditional novelty 7.0 of 10

    SA-GSAE with Bi-Jump-ReLU enables one latent to encode both polarities of anticorrelated features, Pareto-dominating or matching full-width gated SAEs while reducing dead latents by up to 500x on some LLM hookpoints.

  6. Incorporating Hierarchical Semantics in Sparse Autoencoder Architectures

    cs.CL 2025-06 conditional novelty 7.0 of 10

    A two-level mixture-of-experts sparse autoencoder models parent and child concepts together, improving reconstruction and reducing feature redundancy on Gemma 2-2B activations compared to flat top-k SAEs.

  7. Sparse Autoencoders Do Not Find Canonical Units of Analysis

    cs.LG 2025-02 conditional novelty 7.0 of 10

    Across different dictionary sizes, SAE latents are neither complete nor atomic, so SAEs do not learn a canonical set of features.

  8. Interpretability in Parameter Space: Minimizing Mechanistic Description Length with Attribution-based Parameter Decomposition

    cs.LG 2025-01 conditional novelty 7.0 of 10

    Attribution-based Parameter Decomposition splits a network's parameters into faithful, minimal, and simple components and recovers ground-truth mechanisms in toy models of superposition and compressed computation.

  9. LaPrune: Controllable Differentiable Sparsity at Million Scale

    cs.LG 2026-08 accept novelty 6.0 of 10

    A differentiable top-k mask layer that enforces an exact selection budget and uses a normalized hardness parameter to interpolate from equal-weight masks to hard binary masks, with saturation theory and million-scale results.

  10. ChronoLens: Measuring Language Change Across Time, Languages, and Linguistic Levels

    cs.CL 2026-08 conditional novelty 6.0 of 10

    ChronoLens uses feature-aligned crosscoders to show that historical language change has comparable magnitude across linguistic levels within a language, but divergent timing and direction across five parliamentary languages.

  11. ECG-InterpBench: Benchmarking the Interpretability of ECG Foundation Models with Matched-Scale Sparse Autoencoders

    cs.LG 2026-07 conditional novelty 6.0 of 10

    With matched-scale sparse autoencoders, HuBERT-ECG best preserves its ECG representation while ECG-JEPA best exposes clinical measurements through single features — a leader split that repeats on MIMIC-IV-ECG.

  12. CADENCE: A Cardiac Atom Dictionary for Interpretable Neural Concept Extraction from ECG Foundation Models

    cs.AI 2026-07 conditional novelty 6.0 of 10

    A sparse dictionary learned from an ECG foundation model's embeddings recovers interpretable cardiac concepts—PVCs, atrial fibrillation, bundle branch blocks, ST/T-wave segments—and transfers to an external dataset wi...

  13. SADe: Sparse-Atom Support Decontamination for Few-Shot Segmentation with Weak Support Annotations

    cs.CV 2026-07 conditional novelty 6.0 of 10

    A frozen support-only cleaner that uses SAE atom contrast plus dense similarity to strip distractors from weak FSS supports and lifts query mIoU across heterogeneous predictors, especially under expanded boxes.

  14. Decoder-Preserving Sparse Autoencoders: Which Readouts Survive Sparse Compression?

    cs.LG 2026-07 accept novelty 6.0 of 10

    A new SAE objective penalizes disagreement between ridge prediction operators, preserving more linear readouts at equal reconstruction error.

  15. Interpreting Video Representations with Spatio-Temporal Sparse Autoencoders

    cs.CV 2026-04 conditional novelty 6.0 of 10

    Spatio-temporal contrastive SAEs recover temporal coherence lost by hard TopK, improve action probes by +3.9% and retrieval by up to 2.8× R@1, and expose a monosemanticity metric artifact.

  16. Less is Enough: Synthesizing Diverse Data in LLM Feature Space with Sparse Autoencoders

    cs.CL 2026-02 conditional novelty 6.0 of 10

    Coverage of sparse-autoencoder-identified task features predicts post-training performance and can guide synthesis of small, high-impact datasets (2,000 vs. 300,000 samples).

  17. AudioSAE: Towards Understanding of Audio-Processing Models with Sparse AutoEncoders

    cs.SD 2026-02 conditional novelty 6.0 of 10

    SAE features from Whisper and HuBERT are seed-stable, interpretable, and steerable: cutting false speech detections by 70% and correlating with EEG responses to speech.

  18. Bridging Mechanistic Interpretability and Prompt Engineering with Gradient Ascent for Interpretable Persona Control

    cs.LG 2026-01 unverdicted novelty 6.0 of 10

    Gradient-guided token search against internal persona directions yields gibberish prompts that reduce sycophancy, hallucination, and myopic reward in three tested LLMs.

  19. Sparse but Wrong: Incorrect L0 Leads to Incorrect Features in Sparse Autoencoders

    cs.LG 2025-08 conditional novelty 6.0 of 10

    Incorrect L0 makes sparse autoencoders mix correlated features rather than disentangling them, and a decoder projection metric can identify the correct L0.

  20. Teach Old SAEs New Domain Tricks with Boosting

    cs.LG 2025-07 conditional novelty 6.0 of 10

    Training a small secondary sparse autoencoder on the reconstruction error of a pretrained SAE improves domain-specific reconstruction and language-model perplexity without hurting general performance.

  21. Insights into a radiology-specialised multimodal large language model with sparse autoencoders

    cs.LG 2025-07 conditional novelty 6.0 of 10

    Applying Matryoshka sparse autoencoders to a radiology-specialised multimodal LLM reveals a minority of interpretable clinical features, while steering them produces unreliable and often off-target report changes.

  22. Decoding Dense Embeddings: Sparse Autoencoders for Interpreting and Discretizing Dense Retrieval

    cs.IR 2025-05 conditional novelty 6.0 of 10

    Dense retrieval embeddings can be decomposed into interpretable latent concepts that serve both as explanations and as efficient sparse indexing units for retrieval.

  23. Position: Mechanistic Interpretability Should Prioritize Feature Consistency in SAEs

    cs.LG 2025-05 conditional novelty 6.0 of 10

    A position paper proposing feature consistency, measured by PW-MCC, as a core SAE evaluation criterion, with evidence that TopK SAEs achieve high consistency on LLM activations.

  24. Low-Rank Adapting Models for Sparse Autoencoders

    cs.LG 2025-01 conditional novelty 6.0 of 10

    LoRA fine-tuning of the language model around a fixed SAE reduces the SAE-insertion loss gap by 30-55% and matches end-to-end SAEs 2-20x faster.

  25. Transcoders Beat Sparse Autoencoders for Interpretability

    cs.LG 2025-01 conditional novelty 6.0 of 10

    Skip transcoders beat sparse autoencoders on both reconstruction fidelity and automated interpretability scores for transformer MLP layers.

  26. SAeUron: Interpretable Concept Unlearning in Diffusion Models with Sparse Autoencoders

    cs.LG 2025-01 conditional novelty 6.0 of 10

    SAeUron removes concepts from text-to-image diffusion models by ablating concept-specific sparse autoencoder features during inference, achieving state-of-the-art unlearning on UnlearnCanvas and I2P without weight updates.

  27. Prof-K: Probabilistic One-Pass Filtering for Efficient Top-k Selection

    cs.LG 2026-08 reject novelty 5.0 of 10

    Prof-K is a sample-then-filter top-k algorithm that reduces exact selection over N values to exact selection over a small candidate buffer with a distribution-free probabilistic guarantee.

  28. Stable and Steerable Sparse Autoencoders with Weight Regularization

    stat.ML 2026-03 conditional novelty 5.0 of 10

    L2 weight regularization in TopK SAEs increases cross-seed feature overlap and roughly doubles measured steering success on Pythia-70M, at the cost of collapsing most latents to zero.

  29. Understanding sparse autoencoder scaling in the presence of feature manifolds

    cs.LG 2025-09 conditional novelty 5.0 of 10

    When loss improvement from tiling a common feature manifold decays slower than feature frequency decays, sparse autoencoders allocate most latents to that manifold and discover sublinearly many features.

  30. BlueGlass: A Framework for Composite AI Safety

    cs.AI 2025-07 conditional novelty 5.0 of 10

    BlueGlass provides composite AI safety infrastructure; its case studies on object-detection VLMs reveal dataset trade-offs, a decoder-layer phase transition in probe accuracy, and SAE-discovered concepts including spu...

  31. Reviving Your MNEME: Predicting The Side Effects of LLM Unlearning and Fine-Tuning via Sparse Model Diffing

    cs.CL 2025-06 conditional novelty 5.0 of 10

    Sparse model diffing on task-agnostic text can flag which academic, safety, or behavioral categories a fine-tuned or unlearned LLM has unintentionally changed.

  32. Train One Sparse Autoencoder Across Multiple Sparsity Budgets to Preserve Interpretability and Accuracy

    cs.LG 2025-05 conditional novelty 5.0 of 10

    HierarchicalTopK trains a single sparse autoencoder that reconstructs transformer activations well at many sparsity levels, matching or beating separate per-level models.

  33. Analyze Feature Flow to Enhance Interpretation and Steering in Language Models

    cs.LG 2025-02 conditional novelty 5.0 of 10

    Cosine similarity between sparse autoencoder features across layers and modules builds flow graphs that explain feature evolution and enable multi-layer steering of language model generation.

  34. Sparsification and Reconstruction from the Perspective of Representation Geometry

    cs.LG 2025-05 reject novelty 4.0 of 10

    Sparse encoding appears to stratify and compress feature representations, but the claimed causal link between cluster separation and reconstruction is not supported.

  35. Mechanistic Interpretability for Neural Networks: Circuits, Sparse Features and Symbolic Reasoning

    cs.LG 2026-07 accept

    A scoping review surveying circuit analysis, sparse autoencoders, activation steering, and neurosymbolic frameworks for interpreting and controlling Transformer-based neural networks.

Reference graph

Works this paper leans on

9 extracted references · 3 canonical work pages · cited by 35 Pith papers

  1. [1]

    Towards monosemanticity: Decomposing language models with dictionary learning

    Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, et al. Towards monosemanticity: Decomposing language models with dictionary learning. Transformer Circuits Thread, 2, 2023

    2023

  2. [2]

    Sparse autoen- coders find highly interpretable features in language models

    Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. Sparse autoen- coders find highly interpretable features in language models. arXiv preprint arXiv:2309.08600, 2023

    arXiv 2023

  3. [3]

    Scaling and evaluating sparse autoencoders, 2024

    Leo Gao, Tom Dupré la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. Scaling and evaluating sparse autoencoders, 2024. 5

    work page 2024

  4. [4]

    k-sparse autoencoders, 2014

    Alireza Makhzani and Brendan Frey. k-sparse autoencoders, 2014

    2014

  5. [5]

    Language models are unsupervised multitask learners

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019

    2019

  6. [6]

    Improving dictionary learning with gated sparse autoen- coders, 2024

    Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Tom Lieberum, Vikrant Varma, János Kramár, Rohin Shah, and Neel Nanda. Improving dictionary learning with gated sparse autoen- coders, 2024

    work page 2024

  7. [7]

    Jumping ahead: Improving reconstruction fidelity with jumprelu sparse autoencoders, 2024

    Senthooran Rajamanoharan, Tom Lieberum, Nicolas Sonnerat, Arthur Conmy, Vikrant Varma, János Kramár, and Neel Nanda. Jumping ahead: Improving reconstruction fidelity with jumprelu sparse autoencoders, 2024

    2024

  8. [8]

    Gemma 2: Improving open language models at a practical size

    Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, et al. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118, 2024

    arXiv 2024

Show all 9 references
  1. [9]

    Scaling monosemanticity: Extracting interpretable features from claude 3 sonnet

    Adly Templeton, Tom Conerly, and et al. Scaling monosemanticity: Extracting interpretable features from claude 3 sonnet. Anthropic, 2024. A Supplemental Material A.1 Experimental Details In this appendix, we provide details about the datasets used, model configurations, and hy...

    2024

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.