Pith. sign in

REVIEW 5 major objections 5 minor 25 references

Empirical Evaluation of Progressive Coding for Sparse Autoencoders

T0 review · 5 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Nested sparse autoencoders beat pruned ones as progressive coders

desk verdict Useful comparison of progressive SAE methods, but the claimed superiority over pruning leans on a single unoptimized baseline. read the letter →

arxiv 2505.00190 v1 pith:IRXZ4NHC submitted 2025-04-30 cs.LG cs.AI

Hans Peter , Anders S{o}gaard This is my paper · ORCID
classification cs.LGcs.AI
keywords sparseautoencodersprogressivecodingMatryoshkarepresentationlearningdictionarypowerlawfeaturepruninginterpretabilitylanguagemodelrepresentationsreconstructionfidelity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks how to obtain sparse autoencoders (SAEs) of many sizes for language-model interpretability without training each size separately. It claims that feature importance in a vanilla SAE's dictionary follows a power law, so a pretrained SAE can be turned into a progressive coder by permuting its columns and truncating the code at any granularity. It then compares that pruning approach with jointly trained nested "Matryoshka" SAEs, and reports that on the granularity-versus-reconstruction frontier the Matryoshka SAE wins: lower reconstruction loss, lower recaptured language-modeling loss, and higher representational similarity at every granularity tested. The trade-off is interpretability: pruned vanilla features keep their original meaning, while Matryoshka features are harder for a language model to explain, especially in the outer granularities. The paper also derives an empirical scaling law for reconstruction loss as a function of model size, sparsity, and granularity.

What carries the argument

The load-bearing object is the progressive coder: an SAE in which decoding only the first $G$ latent dimensions, for any $G \leq N$, yields a usable reconstruction, so computation scales with $G$ rather than $N$. The paper builds this in two ways. The pruning route exploits permutation invariance: because each feature activation depends only on its own row of the encoder, reordering latent dimensions does not change the reconstruction, so sorting features by mean squared activation, $E[\text{activation}^2]$, and keeping the first $G$ converts any pretrained SAE into a progressive coder. The Matryoshka route jointly trains all granularities at once with a shared encoder and decoder, adding a weighted reconstruction loss for each nested head $m \in M$; because the smallest code is a prefix of the largest, one encoding pass serves all granularities, so training $|M|$ nested models costs roughly what the largest one costs. The dictionary power law is what makes the hierarchy learnable: the decoder's covariance eigenvalues and activation importance decay as power laws, so the first columns can, in principle, capture most of the variance.

What would settle it

Compute the optimal subset of $G$ features for each granularity on a pretrained TopK SAE, using exhaustive search for small dictionaries or greedy/orthogonal matching pursuit for large ones, and compare its FVU and recaptured language-modeling loss against the Matryoshka SAE at the same granularity. If the optimized pruned SAE matches or beats the Matryoshka SAE on the frontier, the paper's conclusion that Matryoshka SAEs are more efficient progressive coders would be overturned.

Watch

Extended reading notes

Core claim

The paper's central claim is that jointly training nested SAEs makes them strictly better progressive coders than pruning an already-trained vanilla SAE by activation importance. For all granularities in $\{2^{14}, 2^{15}, 2^{16}\}$ and at fixed relative sparsity, the Matryoshka SAE outperforms both the baseline TopK SAE and the column-permuted baseline on fraction-of-variance-unexplained, recaptured language-modeling loss, and representational similarity analysis (RSA), the correlation between distance matrices of original and reconstructed activations. A second, separate finding is that dictionary importance in vanilla SAEs follows a power law: a small number of decoder directions account for most of the variance, with eigenvalues of the decoder covariance decaying with exponents around $-0.54$ to $-0.72$ and mean squared activations decaying with exponents around $-1.03$ to $-1.27$. This supports the pruning baseline, but also explains why pruning degrades at small granularities through feature splitting, where one semantic feature in a small SAE is fragmented across many latents in a large one. The paper further shows that applying the permutation ordering to a Matryoshka SAE improves it slightly, and that a Matryoshka SAE truncated to 16K or 32K latents matches an SAE trained from scratch at that size, suggesting one large model can be distilled into smaller ones efficiently.

Load-bearing premise

The pruning baseline is assumed to be strong: sorting features by how strongly they fire on average is the best tested ordering, but no optimal subset selection or alternative ranking heuristics are compared, so a better pruning method could close the gap with Matryoshka SAEs.

Editorial extensions

If this is right

  • A single Matryoshka SAE can replace several separately trained SAEs: at a fixed computational budget, train one large nested model and truncate it to any granularity, with training overhead roughly 1.25 times the largest model.
  • At 16K and 32K granularities, both a 64K Matryoshka SAE and a permuted 64K TopK SAE match a same-size-from-scratch SAE, so distillation from one large SAE is a viable cheaper alternative.
  • The gap between Matryoshka and pruned baselines shrinks as the ratio of granularity to model size decreases, due to feature splitting; methods that recombine split features could recover the lost performance.
  • Sampling granularities uniformly during training concentrates activation mass in the first features and improves progressive-coding metrics further.
  • Pruned vanilla features are more interpretable than Matryoshka features, but within Matryoshka SAEs the innermost granularities are more interpretable than the outermost, suggesting the loss already imposes a meaningful ordering.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Extension: The paper's pruning baseline is heuristic; an optimal-subset baseline, such as greedy selection on the decoder or solving the subset-selection problem per granularity, could establish whether the Matryoshka advantage is real or just reflects a weak comparator.
  • Extension: The power-law finding suggests that feature ordering by $E[\text{activation}^2]$ could be combined with feature merging to reverse feature splitting; this is a natural next experiment not run in the paper.
  • Extension: If the frontier result transfers to larger dictionaries with millions of features and to other activation functions such as JumpReLU, then progressive coding could become the default way to serve SAEs at multiple resolutions, since only one model needs to be trained and stored.
  • Extension: The interpretability gap might be reducible by initializing the Matryoshka decoder with a permuted pretrained dictionary or by adding a feature-identity regularization; the paper does not test either, so the trade-off may not be intrinsic.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper addresses the practical problem of obtaining sparse autoencoders (SAEs) of many sizes efficiently, by inducing progressive coders whose first G latents give a useful reconstruction. It proposes and evaluates two approaches: pruning a pretrained vanilla TopK SAE after permuting its dictionary columns by a per-feature importance statistic, and jointly training nested Matryoshka SAEs with shared encoder/decoder weights. Experiments on Gemma-2-2b residual-stream activations compare these methods on reconstruction FVU, recaptured language-modeling loss, and representational similarity, and report that Matryoshka SAEs dominate the granularity-versus-fidelity frontier, while pruned vanilla features remain more interpretable. The paper also reports a power-law structure in SAE dictionaries and fits a scaling law for progressive reconstruction loss.

Significance. If the central comparison is robust, the paper gives useful practical guidance: jointly trained nested SAEs can serve as better progressive coders than a simple pruning baseline at roughly the training cost of the largest model, at the price of some interpretability. The evaluation is not circular, because the compared methods are scored on held-out metrics that are not used to fit them. The paper also credits concurrent Matryoshka SAE work and is candid about limitations such as model scale, kernel inefficiency, and feature splitting. The main weakness is that the pruning comparator is a single heuristic ordering, so the central frontier claim is not yet established against stronger baselines; variance estimates are also missing.

major comments (5)
  1. [§4, §6] The central comparison is against a single pruning heuristic: Section 4 selects E[activation^2] over activation-frequency ranking, and Section 6 then concludes that Matryoshka SAEs outperform "the baseline column permuted SAE" on the granularity-versus-reconstruction-fidelity frontier. The paper does not compare against other natural orderings (decoder-column norms, leave-one-out FVU increase, or an oracle subset selected by reconstruction loss), and it reports in Section 6 that applying column permutation to the Matryoshka SAE itself further improves performance, which shows the learned Matryoshka ordering is not optimal. If a stronger vanilla-dictionary ordering closes the gap, the claim that joint nested training is a more efficient progressive coding strategy is weakened. Please add at least one stronger ordering and an oracle comparison, and report whether the frontier claim survives.
  2. [§4 (Eq. 2)] The text states that "Each feature activation z_j depends solely on the dot product between the j-th row of W_enc and the centered input..., plus its bias term" and uses this to justify per-feature importance sorting. For TopK SAEs this is incorrect: the TopK activation selects the K largest pre-activations globally, so z_j depends on all rows of W_enc through the selection mechanism. The permutation-invariance construction remains valid, but the stated conditional independence does not hold, and the per-feature statistic E[activation^2] ignores the coupling introduced by global top-K selection. Please correct the derivation and discuss how this affects the interpretation of the ranking.
  3. [§6 (Results)] All headline comparisons in Section 6 are based on single training runs. The claim that "For all granularities, the Matryoshka SAE outperforms..." is reported without error bars, confidence intervals, or multiple seeds; given the stochasticity of SAE training, the observed differences on the frontier may be within run-to-run noise. Please report results over at least three independent seeds or initializations (or provide a variance estimate) for FVU, recaptured cross-entropy loss, and RSA.
  4. [§6 (Interpretability)] The interpretability evaluation uses 10 test sequences and 20 training sequences for explanation generation, and Figure 14 is based on 400 randomly selected features. The abstract's claim that pruned vanilla SAEs are more interpretable rests on this small sample and on a single automated-interpretability pipeline. Please provide confidence intervals or increase the evaluation sample, and temper the interpretability conclusion if the estimates remain this noisy.
  5. [§7 (Eq. 9)] The scaling law in Eq. (9) is fit to the 16k, 32k, and 65k TopK SAEs and displayed against the same data in Figure 15; no held-out evaluation or cross-validation is reported, and parameter estimates are given without uncertainty. Since the section frames the law as predictive ("how well can the performance of the model be predicted"), please validate on held-out configurations (e.g., unseen granularity/sparsity combinations) or report cross-validated prediction error.
minor comments (5)
  1. [§6] The notation for granularities is inconsistent: Section 6 defines M={214, 215, 216} and later G∈M={5000,10000,...}; please unify the notation (likely 2^14, 2^15, 2^16) and define all symbols once.
  2. [§6, Appendix B] Several cross-references are broken: the text refers to "Appendix 21" and "Section ??", and the correlation of about 0.8 between FVU and recaptured LM loss is attributed to a footnote that does not exist; please repair these references.
  3. [Figure captions 7 and 8] The captions say "Fvu per token for gemma-2-2b divided by the cross-entropy loss using the SAE reconstruction," which does not describe the plotted quantity; the captions should state the metric actually shown.
  4. [§3] Section 3 reports power-law exponents and R2 values for eigenvalue and activation statistics but gives no confidence intervals or alternative distribution fits; because the tail deviates visibly from a power law, the phrase "dictionary importance follows a power law" is stronger than the evidence shown.
  5. [Table 1] The hyperparameter table omits batch size, number of training steps, and the exact TopK/k schedule; please add these details to make the training setup reproducible.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central Matryoshka-versus-pruning comparison is evaluated on held-out metrics, and the only fitted scaling law is peripheral.

full rationale

This is an empirical comparison paper, and its central claims are not circular. The main result, that Matryoshka SAEs outperform pruned and column-permuted vanilla SAEs on the granularity-versus-reconstruction-fidelity frontier, is supported by held-out FVU, recaptured LM cross-entropy, and RSA computed on 10^5 held-out tokens; these metrics are not the quantities used to rank features for the pruning baseline (E[activation^2]), nor are they identical to the Matryoshka training loss on the training set. The choice of the pruning ordering is justified by an empirical comparison of two heuristics, which is a legitimate though limited baseline choice rather than a definitional identity; the concern that a stronger ordering might close the gap is a correctness or baseline-strength concern, not circularity. The Section 7 scaling law is explicitly fitted to validation data from the 16k, 32k, and 65k TopK SAEs, so the 'predicted' losses in Figure 15 are in-sample evaluations of a fitted curve; this is a peripheral modeling exercise and is not load-bearing for the abstract's comparative claim. The only co-authored self-citation, Li et al. (2024a), is used to motivate RSA as a representational-similarity metric and does not supply any assumption needed for the comparative result. Appendix C explicitly lists limitations such as modest SAE scale, kernel overhead, and feature-splitting, which shows the authors distinguish limitations from claimed results. Under the rules that self-citation matters only when load-bearing and that a fitted empirical law is not a derivation, I find no circular step and assign score 0.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The free parameters are the fitted scaling-law coefficients and the unreported granularity weights. The axioms capture the main unverified premises: the pruning ordering heuristic, the metric suite, the sparsity regime, and the extrapolation from small SAEs.

free parameters (2)
  • Scaling law parameters = alpha=-3.60, beta_k=0.69, beta_n=0.19, beta_g=0.08, gamma_n=0.02, gamma_g=-0.10, zeta=-2.13, eta=-0.13
    Fitted in Section 7 to FVU for 16k/32k/65k TopK SAEs at four sparsity levels; used to claim predictive scaling laws but not validated on held-out sizes.
  • Matryoshka granularity weights c_m = unreported
    Equation 8 weights each granularity's reconstruction loss; the paper never states the values (assumed equal or tuned), and the results depend on this design choice.
assumptions (4)
  • ad hoc to paper E[activation^2] ranking is a good proxy for feature importance in pruning
    Section 4 adopts it after comparing to activation frequency only; the central comparison of Matryoshka vs pruned vanilla depends on this ranking.
  • domain assumption FVU, recaptured CE loss, and RSA are jointly adequate metrics for progressive coder quality
    Section 6 uses all three; they report correlation about 0.8, so disagreements are possible; Braun et al. 2024 (cited) show discrepancies between FVU and recaptured loss.
  • domain assumption Relative sparsity (K/G constant) is the right comparison regime for progressive coders
    Section 6 footnote 1 defines fixed relative sparsity; alternative regimes might alter the trade-off.
  • domain assumption Results on 65k-wide SAEs trained on 50M tokens generalize to larger SAEs
    Limitations section acknowledges modest sizes; the power law at multiple scales is the evidence used, but the conclusion is extrapolation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Empirical Evaluation of Progressive Coding for Sparse Autoencoders." pith.science (2026). https://pith.science/paper/IRXZ4NHC

@misc{pith2026250500190,
  author       = {Pith},
  title        = {Pith review of: Empirical Evaluation of Progressive Coding for Sparse Autoencoders},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IRXZ4NHC}},
  note         = {Machine review of arXiv:2505.00190}
}
read the original abstract

Sparse autoencoders (SAEs) \citep{bricken2023monosemanticity,gao2024scalingevaluatingsparseautoencoders} rely on dictionary learning to extract interpretable features from neural networks at scale in an unsupervised manner, with applications to representation engineering and information retrieval. SAEs are, however, computationally expensive \citep{lieberum2024gemmascopeopensparse}, especially when multiple SAEs of different sizes are needed. We show that dictionary importance in vanilla SAEs follows a power law. We compare progressive coding based on subset pruning of SAEs -- to jointly training nested SAEs, or so-called {\em Matryoshka} SAEs \citep{bussmann2024learning,nabeshima2024Matryoshka} -- on a language modeling task. We show Matryoshka SAEs exhibit lower reconstruction loss and recaptured language modeling loss, as well as higher representational similarity. Pruned vanilla SAEs are more interpretable, however. We discuss the origins and implications of this trade-off.

Figures

Figures reproduced from arXiv: 2505.00190 by the authors.

Figure 1
Figure 1. Illustrating progressive coding, the dark part highlight the ressources not used by the model at inference time. preserving all pairwise distances between the points up to a factor of (1 + ϵ). In dictionary learning, the goal is to find an overcomplete set of basis vectors D ∈ R D×N , with N >> D, and a set of representations R = [r1, . . . , rN ], where ri ∈ R n, that jointly minimizes reconstruction and sparsity w… view at source ↗
Figure 2
Figure 2. Power law fits for eigenvalues of the covari￾ance matrix, E[activation2 ] and activation frequency (E[1|activation| > 0]). We fit a linear regression model to the logarithmically transformed values and display the coefficient and fit for each. We analyze three models of various sizes (65k, 32k, 16k) with consistent sparsity ratios (256-65k, 128-32k, 64-16k). The eigenvalues of the decoder matrix’s covariance matrix … view at source ↗
Figure 3
Figure 3. An illustration of dictionary permutation with function π, Both models will produce the same output given the same input Xˆ′ = z1:GWDec[: G, :] + Bcenter, where G represents the granularity, or the length of the code the decoder receives. We propose two ranking methods for determining π: sorting by mean squared activation: E[activation2 ]; or sorting by mean activation frequency: E[1|activation| > 0] [PITH_FULL_IMA… view at source ↗
Figures from the paper (14 more)
Figure 4
Figure 4. Figure 4: Granularity vs FVU (normalized reconstruction loss) for non-permuted(baseline), permuted based on E[activation2 ] and E[1{activation > 0}]. Relative sparsity is fixed such that k non￾zero / granularity is constant for all granularities Our results demonstrate that sort…
Figure 6
Figure 6. Figure 6: Mean cross-entropy loss per token for gemma-2-2b di￾vided by the cross-entropy loss using the SAE reconstruction, computed over 105 tokens on the pile-uncopyrighted dataset. K refers to the sparsity mechanism, for the topk activation function, which all our models use,…
Figure 9
Figure 9. Figure 9: Sparsity vs Reconstruction fidelity (FVU) [PITH_FULL_IMAGE:figures/full_fig_p006_9.png]
Figure 8
Figure 8. Figure 8: RSA per token for gemma-2-2b divided by the cross￾entropy loss using the SAE reconstruction, computed over 105 tokens on the pile-uncopyrighted dataset. Sparsity-Fidelity Frontier Next, we evaluate the sparsity vs fidelity frontier for our different approaches. For a f…
Figure 11
Figure 11. Figure 11: Sparsity vs Reconstruction fidelity for models, only using the first 10k, 16k or 32k latents. Lower fvu is better, higher recaptured ce-loss is better. Interpretability As Matryoshka SAEs are a new method for training SAEs, we find it important to evaluate whether thi…
Figure 13
Figure 13. Figure 13: The Pearson correlation between Llama-3 simulated and ground truth activations for different granularities of a Ma￾tryoshka SAE. Note that the granularities of 16k are a subset of 32k etc. Values above 1 are an artifact of the kernel density estimation process We find…
Figure 15
Figure 15. Figure 15: Loss vs. predicted loss for SAE (32k and 65k latents) Substantial evidence supports that Matryoshka SAEs learn a hierarchy of features, placing the most important features in the first mk columns of the decoder. Earlier work on MRL(Devvrit et al., 2024) has suggested …
Figure 14
Figure 14. Figure 14: Balanced accuracy for feature indices grouped into quantiles 0-100 for 400 randomly selected features Matryoshka SAEs slightly underperform on this task: The first latents seem to perform better than the average, but scores quickly drop. 7. Discussion: Scaling and Gra…
Figure 16
Figure 16. Figure 16: Sampled, non-sampled Matryoshka and baseline (105 t) Sampling improves Matryoshka SAE metrics. The sampled Matryoshka SAE concentrates most of its activation mass in the first features, while the non-sampled exhibits distinct plateaus for each granularity level. The b…
Figure 17
Figure 17. Figure 17: Mean activation squared by interval in latent space: sampled, non-sampled Matryoshka and baseline. 8 [PITH_FULL_IMAGE:figures/full_fig_p008_17.png]
Figure 18
Figure 18. Figure 18: We compute the mean index of the top 5 closest feature for each feature for the Matryoshka TopK SAE [PITH_FULL_IMAGE:figures/full_fig_p013_18.png]
Figure 19
Figure 19. Figure 19: We compute the mean index of the top 5 closest feature for each feature for the TopK SAE The natural locality of features in the standard TopK SAE can be attributed to the random initialization process, where nearby features in the latent space tend to develop related…
Figure 20
Figure 20. Figure 20: We compute the mean index of the top 5 closest feature for each feature for a random initialized decoder B. Figures [PITH_FULL_IMAGE:figures/full_fig_p014_20.png]
Figure 21
Figure 21. Figure 21: Correlation analysis between different evaluation metrics (FVU, CE Loss, and RSA). The scatter plots show pairwise relationships with linear regression fits, displaying both Pearson correlation coefficients (r) and coefficients of determination (R²). C. Limitations an…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

25 extracted references · 6 canonical work pages

  1. [4]

    Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H

    URL https://arxiv.org/ abs/2405.17430. Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., Schuh, P., Shi, K., Tsvyashchenko, S., Maynez, J., Rao, A., Barnes, P., Tay, Y ., Shazeer, N., Prabhakaran, V ., Reif, E., Du, N., Hutchinson, B., Pope, R., Bradbury, J., Austin, J., Isard, M...

    arXiv

  2. [6]

    Devvrit, Kudugunta, S., Kusupati, A., Dettmers, T., Chen, K., Dhillon, I., Tsvetkov, Y ., Hajishirzi, H., Kakade, S., Farhadi, A., and Jain, P

    URL https:// arxiv.org/abs/2309.08600. Devvrit, Kudugunta, S., Kusupati, A., Dettmers, T., Chen, K., Dhillon, I., Tsvetkov, Y ., Hajishirzi, H., Kakade, S., Farhadi, A., and Jain, P. Matformer: Nested transformer for elastic inference,

    arXiv

  3. [7]

    org/abs/2310.07707

    URL https://arxiv. org/abs/2310.07707. EleutherAI. sae-auto-interp. https://github.com/ EleutherAI/sae-auto-interp,

    arXiv

  4. [8]

    pub/2022/toy_model/index.html

    URL https://transformer-circuits. pub/2022/toy_model/index.html. Gao, L., la Tour, T. D., Tillman, H., Goh, G., Troll, R., Radford, A., Sutskever, I., Leike, J., and Wu, J. Scaling and evaluating sparse autoencoders,

    work page 2022

  5. [9]

    URL https: //arxiv.org/abs/2406.04093. Grattafiori, A., Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Vaughan, A., Yang, A., Fan, A., Goyal, A., Hartshorn, A., Yang, A., Mitra, A., Sravankumar, A., Korenev, A., Hinsvark, A., Rao, A., Zhang, A., Rodriguez, A., Gregerson, A., Spataru, A., Roziere, B., Bir...

    arXiv

  6. [10]

    Gu, J., Zhai, S., Zhang, Y ., Susskind, J., and Jaitly, N

    URL https://arxiv.org/abs/2407.21783. Gu, J., Zhai, S., Zhang, Y ., Susskind, J., and Jaitly, N. Matryoshka diffusion models,

    arXiv

  7. [11]

    Hoffmann, J., Borgeaud, S., Mensch, A., Buchatskaya, E., Cai, T., Rutherford, E., de Las Casas, D., Hendricks, L

    URL https: //arxiv.org/abs/2310.15111. Hoffmann, J., Borgeaud, S., Mensch, A., Buchatskaya, E., Cai, T., Rutherford, E., de Las Casas, D., Hendricks, L. A., Welbl, J., Clark, A., Hennigan, T., Noland, E., Millican, K., van den Driessche, G., Damoc, B., Guy, A., Osindero, S., Simonyan, K., Elsen, E., Rae, J. W., 10 Empirical Evaluation of Progressive SAEs ...

    arXiv

  8. [12]

    Hurley, N

    URL https://arxiv.org/ abs/2203.15556. Hurley, N. P. and Rickard, S. T. Comparing measures of spar- sity,

    arXiv

Show all 25 references
  1. [13]

    URL https://arxiv.org/abs/2001. 08361. Klabunde, M., Schumacher, T., Strohmaier, M., and Lem- merich, F. Similarity of neural network models: A survey of functional and representational measures,

    2001

  2. [14]

    Kusupati, A., Bhatt, G., Rege, A., Wallingford, M., Sinha, A., Ramanujan, V ., Howard-Snyder, W., Chen, K., Kakade, S., Jain, P., and Farhadi, A

    URL https://arxiv.org/abs/2305.06329. Kusupati, A., Bhatt, G., Rege, A., Wallingford, M., Sinha, A., Ramanujan, V ., Howard-Snyder, W., Chen, K., Kakade, S., Jain, P., and Farhadi, A. Matryoshka represen- tation learning,

    arXiv

  3. [15]

    Li, J., Kementchedjhieva, Y ., Fierro, C., and Søgaard, A

    URL https://arxiv.org/ abs/2205.13147. Li, J., Kementchedjhieva, Y ., Fierro, C., and Søgaard, A. Do vision and language models share concepts? a vec- tor space alignment study. Transactions of the Asso- ciation for Computational Linguistics , 12:1232–1249, 2024a. doi: 10.1162...

    arXiv

  4. [16]

    Makelov, A., Lange, G., and Nanda, N

    URL https: //arxiv.org/abs/2408.05147. Makelov, A., Lange, G., and Nanda, N. Towards princi- pled evaluations of sparse autoencoders for interpretabil- ity and control,

    arXiv

  5. [17]

    Mudide, A., Engels, J., Michaud, E

    URL https://arxiv.org/ abs/2405.08366. Mudide, A., Engels, J., Michaud, E. J., Tegmark, M., and de Witt, C. S. Efficient dictionary learning with switch sparse autoencoders,

    arXiv

  6. [18]

    org/abs/2410.08201

    URL https://arxiv. org/abs/2410.08201. Nabeshima, N. Matryoshka sparse autoencoders. AI Align- ment Forum, 12

    arXiv

  7. [19]

    Rajamanoharan, S., Lieberum, T., Sonnerat, N., Conmy, A., Varma, V ., Kram´ar, J., and Nanda, N

    URL https://arxiv.org/ abs/2410.13928. Rajamanoharan, S., Lieberum, T., Sonnerat, N., Conmy, A., Varma, V ., Kram´ar, J., and Nanda, N. Jumping ahead: Improving reconstruction fidelity with jumprelu sparse au- toencoders,

    arXiv

  8. [20]

    Rippel, O., Gelbart, M

    URL https://arxiv.org/abs/ 2407.14435. Rippel, O., Gelbart, M. A., and Adams, R. P. Learning ordered representations with nested dropout,

    arXiv

  9. [21]

    The scatter plots show pairwise relationships with linear regression fits, displaying both Pearson correlation coefficients (r) and coefficients of determination (R²)

    Correlation analysis between different evaluation metrics (FVU, CE Loss, and RSA). The scatter plots show pairwise relationships with linear regression fits, displaying both Pearson correlation coefficients (r) and coefficients of determination (R²). C. Limitations and Future ...

    2024

  10. [22]

    Templeton, A., Conerly, T., Marcus, J., Lindsey, J., Bricken, T., Chen, B., Pearce, A., Citro, C., Ameisen, E., Jones, A., Cunningham, H., Turner, N

    URL https://arxiv.org/abs/2408.00118. Templeton, A., Conerly, T., Marcus, J., Lindsey, J., Bricken, T., Chen, B., Pearce, A., Citro, C., Ameisen, E., Jones, A., Cunningham, H., Turner, N. L., McDougall, C., MacDiarmid, M., Freeman, C. D., Sumers, T. R., Rees, E., Batson, J., J...

    arXiv

  11. [23]

    Yun, Z., Chen, Y ., Olshausen, B

    URL https: //transformer-circuits.pub/2024/ scaling-monosemanticity/index.html. Yun, Z., Chen, Y ., Olshausen, B. A., and LeCun, Y . Trans- former visualization via dictionary learning: contextual- ized embedding as a linear superposition of transformer factors,

    2024

  12. [24]

    12 Empirical Evaluation of Progressive SAEs A

    URL https://arxiv.org/abs/ 2103.15949. 12 Empirical Evaluation of Progressive SAEs A. Feature Splitting Feature splitting is the phenomenon where as the dictionary grows in size, one basis vector gets decomposed into multiple separate basis vectors. In contrast, the standard T...

    arXiv

  13. [2014]

    Skodras, A., Christopoulos, C., and Ebrahimi, T

    URL https://arxiv.org/abs/1402.0915. Skodras, A., Christopoulos, C., and Ebrahimi, T. The jpeg 2000 still image compression standard. IEEE Signal Processing Magazine, 18(5):36–58,

    2000 arXiv

  14. [2020]

    Bussmann, B., Leask, P., and Nanda, N

    URL https:// arxiv.org/abs/2005.14165. Bussmann, B., Leask, P., and Nanda, N. Learn- ing multi-level features with matryoshka saes. AI Alignment Forum, 12

    2005 arXiv

  15. [2022]

    Cunningham, H., Ewart, A., Riggs, L., Huben, R., and Sharkey, L

    URL https://arxiv.org/abs/2204.02311. Cunningham, H., Ewart, A., Riggs, L., Huben, R., and Sharkey, L. Sparse autoencoders find highly interpretable features in language models,

    arXiv

  16. [2023]

    Brown, T

    https://transformer- circuits.pub/2023/monosemantic-features/index.html. Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-V oss, A., Krueger, G., Henighan, T., Child, R., Ramesh, ...

    2023

  17. [2024]

    URL https: //arxiv.org/abs/2405.12241. Bricken, T., Templeton, A., Batson, J., Chen, B., Jermyn, A., Conerly, T., Turner, N., Anil, C., Denison, C., Askell, A., Lasenby, R., Wu, Y ., Kravec, S., Schiefer, N., Maxwell, T., Joseph, N., Hatfield-Dodds, Z., Tamkin, A., Nguyen, K.,...

    arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.