{"id":"11a0cc20-c59b-4829-a8ea-fcf33575699d","arxiv_id":"2505.00190","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Nested 'Matryoshka' sparse autoencoders outperform pruned vanilla sparse autoencoders on reconstruction and recaptured language-model loss, but pruned vanilla features remain more interpretable.","lead":"Two ways to make sparse autoencoders (the feature-finders used to interpret large language models) usable at multiple sizes are compared: jointly trained nested versions and simply pruning a big pretrained one. The nested versions reconstruct activations and preserve language-model behavior better, while the pruned versions keep features more interpretable.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed superiority of Matryoshka SAEs over pruning rests on a single heuristic ordering (E[activation^2]); a stronger pruning baseline could close the gap.","rationale":"The reader's weakest_assumption identifies exactly the same load-bearing concern: the pruning baseline's ordering heuristic is not validated against stronger subset-selection methods. I agree with that assessment. The central claim is an empirical superiority claim, and its scope is only as broad as the baseline it is compared against. While the paper's experiments are internally consistent, the absence of any oracle or reconstruction-aware ordering leaves open the possibility that a better pruning method would match or beat Matryoshka SAEs. The paper's own observation that permuting the Matryoshka SAE further improves performance supports this concern, as it shows the feature order is not already optimal. The concrete test proposed would settle the question by measuring whether any reasonable alternative ordering closes the gap. Since this is an addressable weakness rather than a demonstrated fatal flaw, the reader's CONDITIONAL verdict remains appropriate, and I would not change it.","tokens_in":16233,"tokens_out":12741,"duration_ms":131662,"concrete_test":"Using the same 65k TopK SAE and held-out activations, for G in {2^14, 2^15} and the same relative sparsity ratios, compute FVU for the pruned baseline under several alternative orderings: decoder column L2 norm; per-feature leave-one-out FVU increase; and greedy forward selection (add the column that most reduces validation FVU, with 100 steps of local search). If any alternative ordering achieves FVU at or below the Matryoshka SAE at the same G, the headline comparison is not robust to the choice of baseline; report the best and mean FVU over 10 random restarts.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4 justifies the pruning baseline by comparing E[activation^2] ordering only against activation-frequency ordering and no reordering; Section 6 then concludes that Matryoshka SAEs outperform the baseline column permuted SAE on the granularity-versus-reconstruction fidelity frontier. The load-bearing assumption is that E[activation^2] is a strong (or at least representative) way to select the first G features of a pretrained vanilla SAE. This is not established. E[activation^2] is a marginal statistic that ignores correlations among features and decoder geometry; for TopK SAEs the global top-K selection couples features, so the Section 4 claim that each activation depends solely on its own row of W_enc is incorrect. More importantly, the paper never compares against other natural orderings: decoder column norms, reconstruction-loss-based importance (e.g., leave-one-out FVU increase), or an oracle subset. The paper itself notes that applying the column permutation to the Matryoshka SAE increases performance further, which shows the learned Matryoshka order is not optimal. If a better ordering of the vanilla dictionary closes the gap, the central claim that joint nested training is a more efficient progressive coding strategy is weakened to a statement about one particular heuristic.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper addresses the practical problem of obtaining sparse autoencoders (SAEs) of many sizes efficiently, by inducing progressive coders whose first G latents give a useful reconstruction. It proposes and evaluates two approaches: pruning a pretrained vanilla TopK SAE after permuting its dictionary columns by a per-feature importance statistic, and jointly training nested Matryoshka SAEs with shared encoder/decoder weights. Experiments on Gemma-2-2b residual-stream activations compare these methods on reconstruction FVU, recaptured language-modeling loss, and representational similarity, and report that Matryoshka SAEs dominate the granularity-versus-fidelity frontier, while pruned vanilla features remain more interpretable. The paper also reports a power-law structure in SAE dictionaries and fits a scaling law for progressive reconstruction loss.","tokens_in":16497,"tokens_out":6317,"duration_ms":64568,"significance":"If the central comparison is robust, the paper gives useful practical guidance: jointly trained nested SAEs can serve as better progressive coders than a simple pruning baseline at roughly the training cost of the largest model, at the price of some interpretability. The evaluation is not circular, because the compared methods are scored on held-out metrics that are not used to fit them. The paper also credits concurrent Matryoshka SAE work and is candid about limitations such as model scale, kernel inefficiency, and feature splitting. The main weakness is that the pruning comparator is a single heuristic ordering, so the central frontier claim is not yet established against stronger baselines; variance estimates are also missing.","major_comments":[{"comment":"The central comparison is against a single pruning heuristic: Section 4 selects E[activation^2] over activation-frequency ranking, and Section 6 then concludes that Matryoshka SAEs outperform \"the baseline column permuted SAE\" on the granularity-versus-reconstruction-fidelity frontier. The paper does not compare against other natural orderings (decoder-column norms, leave-one-out FVU increase, or an oracle subset selected by reconstruction loss), and it reports in Section 6 that applying column permutation to the Matryoshka SAE itself further improves performance, which shows the learned Matryoshka ordering is not optimal. If a stronger vanilla-dictionary ordering closes the gap, the claim that joint nested training is a more efficient progressive coding strategy is weakened. Please add at least one stronger ordering and an oracle comparison, and report whether the frontier claim survives.","section":"§4, §6"},{"comment":"The text states that \"Each feature activation z_j depends solely on the dot product between the j-th row of W_enc and the centered input..., plus its bias term\" and uses this to justify per-feature importance sorting. For TopK SAEs this is incorrect: the TopK activation selects the K largest pre-activations globally, so z_j depends on all rows of W_enc through the selection mechanism. The permutation-invariance construction remains valid, but the stated conditional independence does not hold, and the per-feature statistic E[activation^2] ignores the coupling introduced by global top-K selection. Please correct the derivation and discuss how this affects the interpretation of the ranking.","section":"§4 (Eq. 2)"},{"comment":"All headline comparisons in Section 6 are based on single training runs. The claim that \"For all granularities, the Matryoshka SAE outperforms...\" is reported without error bars, confidence intervals, or multiple seeds; given the stochasticity of SAE training, the observed differences on the frontier may be within run-to-run noise. Please report results over at least three independent seeds or initializations (or provide a variance estimate) for FVU, recaptured cross-entropy loss, and RSA.","section":"§6 (Results)"},{"comment":"The interpretability evaluation uses 10 test sequences and 20 training sequences for explanation generation, and Figure 14 is based on 400 randomly selected features. The abstract's claim that pruned vanilla SAEs are more interpretable rests on this small sample and on a single automated-interpretability pipeline. Please provide confidence intervals or increase the evaluation sample, and temper the interpretability conclusion if the estimates remain this noisy.","section":"§6 (Interpretability)"},{"comment":"The scaling law in Eq. (9) is fit to the 16k, 32k, and 65k TopK SAEs and displayed against the same data in Figure 15; no held-out evaluation or cross-validation is reported, and parameter estimates are given without uncertainty. Since the section frames the law as predictive (\"how well can the performance of the model be predicted\"), please validate on held-out configurations (e.g., unseen granularity/sparsity combinations) or report cross-validated prediction error.","section":"§7 (Eq. 9)"}],"minor_comments":[{"comment":"The notation for granularities is inconsistent: Section 6 defines M={214, 215, 216} and later G∈M={5000,10000,...}; please unify the notation (likely 2^14, 2^15, 2^16) and define all symbols once.","section":"§6"},{"comment":"Several cross-references are broken: the text refers to \"Appendix 21\" and \"Section ??\", and the correlation of about 0.8 between FVU and recaptured LM loss is attributed to a footnote that does not exist; please repair these references.","section":"§6, Appendix B"},{"comment":"The captions say \"Fvu per token for gemma-2-2b divided by the cross-entropy loss using the SAE reconstruction,\" which does not describe the plotted quantity; the captions should state the metric actually shown.","section":"Figure captions 7 and 8"},{"comment":"Section 3 reports power-law exponents and R2 values for eigenvalue and activation statistics but gives no confidence intervals or alternative distribution fits; because the tail deviates visibly from a power law, the phrase \"dictionary importance follows a power law\" is stronger than the evidence shown.","section":"§3"},{"comment":"The hyperparameter table omits batch size, number of training steps, and the exact TopK/k schedule; please add these details to make the training setup reproducible.","section":"Table 1"}],"recommendation":"major_revision","confidential_remarks":"The paper is an honest empirical study with clear value if the pruning-baseline concern is addressed. The main risk is that the central frontier claim may reduce to a statement about one heuristic ordering; the authors should be asked to add oracle and alternative orderings, as well as variance estimates. I do not see a circularity problem: the compared methods are scored on held-out metrics, and the only fitted prediction (the scaling law) is not load-bearing for the abstract's main claim. The presentation needs editorial cleanup, but that is secondary."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. The paper compares two ways to get multiple SAE sizes out of one training run: jointly-trained nested (Matryoshka) SAEs and pruning a pretrained vanilla SAE by feature importance. The comparison is genuine, but the pruning side rests on a single heuristic ordering (E[activation^2]) that is never tested against decoder column norms, leave-one-out reconstruction importance, or an oracle subset. If a stronger pruning baseline closes the gap, the abstract's categorical claim that Matryoshka SAEs 'exhibit lower reconstruction loss' becomes a statement about one particular baseline.\n\nCredit where due. The permutation-based pruning method is a natural but concrete baseline, and the five-protocol evaluation (FVU, recaptured LM loss, RSA, simulation scoring, fuzzing) is more thorough than most SAE papers. The paper is honest about the power law replicating Li et al. (2024b) and about Matryoshka SAEs being concurrent work. Naming the interpretability trade-off — pruned vanilla features keep their identities, Matryoshka features do not — is useful.\n\nSoft spots, in rough order of severity. The pruning baseline is load-bearing; Section 4 only compares E[activation^2] against activation-frequency and never against a stronger selector. The stress-test note is right that the conditional-independence statement in Section 4 is overstated for TopK SAEs: whether a feature fires depends on the global top-K competition, so marginal activation statistics do not fully determine importance under pruning. The paper's own result that permuting the Matryoshka SAE improves it further shows the learned order is not optimal either, which cuts both ways. Second, no error bars or multiple seeds anywhere, and the interpretability study uses 10 test / 20 training sequences and 400 features — pilot-scale for an abstract-level claim. Third, the scaling law in Section 7 is fit and evaluated in-sample; calling it predictive is a stretch. Fourth, no code or checkpoints are released, which limits reproducibility. The paper itself, in the limitations appendix, flags the small SAE scale and the feature-splitting problem, which is honest.\n\nWho this is for: practitioners training SAEs for interpretability or representation engineering who want flexible multi-size readout. It deserves a serious referee. Send it out, but the referee should ask for stronger pruning baselines, error bars or seeded repeats, a held-out scaling-law check, and code release.","headline":"Useful comparison of progressive SAE methods, but the claimed superiority over pruning leans on a single unoptimized baseline.","tokens_in":16993,"tokens_out":3944,"would_cite":true,"duration_ms":39086,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Nested sparse autoencoders beat pruned ones as progressive coders","keywords":["sparse autoencoders","progressive coding","Matryoshka representation learning","dictionary power law","feature pruning","interpretability","language model representations","reconstruction fidelity"],"falsifier":"Compute the optimal subset of $G$ features for each granularity on a pretrained TopK SAE, using exhaustive search for small dictionaries or greedy/orthogonal matching pursuit for large ones, and compare its FVU and recaptured language-modeling loss against the Matryoshka SAE at the same granularity. If the optimized pruned SAE matches or beats the Matryoshka SAE on the frontier, the paper's conclusion that Matryoshka SAEs are more efficient progressive coders would be overturned.","tokens_in":2032,"feed_emoji":"🧠","tokens_out":3429,"duration_ms":115864,"temperature":0.7,"pith_summary":"The paper asks how to obtain sparse autoencoders (SAEs) of many sizes for language-model interpretability without training each size separately. It claims that feature importance in a vanilla SAE's dictionary follows a power law, so a pretrained SAE can be turned into a progressive coder by permuting its columns and truncating the code at any granularity. It then compares that pruning approach with jointly trained nested \"Matryoshka\" SAEs, and reports that on the granularity-versus-reconstruction frontier the Matryoshka SAE wins: lower reconstruction loss, lower recaptured language-modeling loss, and higher representational similarity at every granularity tested. The trade-off is interpretability: pruned vanilla features keep their original meaning, while Matryoshka features are harder for a language model to explain, especially in the outer granularities. The paper also derives an empirical scaling law for reconstruction loss as a function of model size, sparsity, and granularity.","feed_headline":"Nested sparse autoencoders beat pruned ones as progressive coders","feed_subtitle":"Nested SAEs reconstruct activations better at all granularities; pruned features keep interpretability.","key_machinery":"The load-bearing object is the progressive coder: an SAE in which decoding only the first $G$ latent dimensions, for any $G \\leq N$, yields a usable reconstruction, so computation scales with $G$ rather than $N$. The paper builds this in two ways. The pruning route exploits permutation invariance: because each feature activation depends only on its own row of the encoder, reordering latent dimensions does not change the reconstruction, so sorting features by mean squared activation, $E[\\text{activation}^2]$, and keeping the first $G$ converts any pretrained SAE into a progressive coder. The Matryoshka route jointly trains all granularities at once with a shared encoder and decoder, adding a weighted reconstruction loss for each nested head $m \\in M$; because the smallest code is a prefix of the largest, one encoding pass serves all granularities, so training $|M|$ nested models costs roughly what the largest one costs. The dictionary power law is what makes the hierarchy learnable: the decoder's covariance eigenvalues and activation importance decay as power laws, so the first columns can, in principle, capture most of the variance.","core_discovery":"The paper's central claim is that jointly training nested SAEs makes them strictly better progressive coders than pruning an already-trained vanilla SAE by activation importance. For all granularities in $\\{2^{14}, 2^{15}, 2^{16}\\}$ and at fixed relative sparsity, the Matryoshka SAE outperforms both the baseline TopK SAE and the column-permuted baseline on fraction-of-variance-unexplained, recaptured language-modeling loss, and representational similarity analysis (RSA), the correlation between distance matrices of original and reconstructed activations. A second, separate finding is that dictionary importance in vanilla SAEs follows a power law: a small number of decoder directions account for most of the variance, with eigenvalues of the decoder covariance decaying with exponents around $-0.54$ to $-0.72$ and mean squared activations decaying with exponents around $-1.03$ to $-1.27$. This supports the pruning baseline, but also explains why pruning degrades at small granularities through feature splitting, where one semantic feature in a small SAE is fragmented across many latents in a large one. The paper further shows that applying the permutation ordering to a Matryoshka SAE improves it slightly, and that a Matryoshka SAE truncated to 16K or 32K latents matches an SAE trained from scratch at that size, suggesting one large model can be distilled into smaller ones efficiently.","pith_inferences":["Extension: The paper's pruning baseline is heuristic; an optimal-subset baseline, such as greedy selection on the decoder or solving the subset-selection problem per granularity, could establish whether the Matryoshka advantage is real or just reflects a weak comparator.","Extension: The power-law finding suggests that feature ordering by $E[\\text{activation}^2]$ could be combined with feature merging to reverse feature splitting; this is a natural next experiment not run in the paper.","Extension: If the frontier result transfers to larger dictionaries with millions of features and to other activation functions such as JumpReLU, then progressive coding could become the default way to serve SAEs at multiple resolutions, since only one model needs to be trained and stored.","Extension: The interpretability gap might be reducible by initializing the Matryoshka decoder with a permuted pretrained dictionary or by adding a feature-identity regularization; the paper does not test either, so the trade-off may not be intrinsic."],"forward_implications":["A single Matryoshka SAE can replace several separately trained SAEs: at a fixed computational budget, train one large nested model and truncate it to any granularity, with training overhead roughly 1.25 times the largest model.","At 16K and 32K granularities, both a 64K Matryoshka SAE and a permuted 64K TopK SAE match a same-size-from-scratch SAE, so distillation from one large SAE is a viable cheaper alternative.","The gap between Matryoshka and pruned baselines shrinks as the ratio of granularity to model size decreases, due to feature splitting; methods that recombine split features could recover the lost performance.","Sampling granularities uniformly during training concentrates activation mass in the first features and improves progressive-coding metrics further.","Pruned vanilla features are more interpretable than Matryoshka features, but within Matryoshka SAEs the innermost granularities are more interpretable than the outermost, suggesting the loss already imposes a meaningful ordering."],"supporting_citations":[{"why":"Supplies the TopK SAE architecture, the normalized MSE (FVU) metric, and the sparse-dense matmul kernel that all experiments rely on.","marker":"(Gao et al., 2024)"},{"why":"Proposes Matryoshka SAEs concurrently; the paper's central method is the same jointly trained nested architecture.","marker":"(Bussmann et al., 2024)"},{"why":"Also proposes Matryoshka SAEs independently; grounds the central method in concurrent work.","marker":"(Nabeshima, 2024)"},{"why":"Introduces Matryoshka representation learning, whose nested loss the Matryoshka SAE adapts.","marker":"(Kusupati et al., 2024)"},{"why":"Reports power-law decay of decoder covariance eigenvalues, the observation the dictionary power-law hypothesis extends.","marker":"(Li et al., 2024b)"},{"why":"Supplies a family of open JumpReLU SAEs used as comparison baselines and as training configuration references.","marker":"(Lieberum et al., 2024)"},{"why":"Defines feature splitting and the SAE interpretability framing that explains the performance limits of pruning.","marker":"(Bricken et al., 2023)"},{"why":"Defines progressive coding, the evaluation objective the paper adopts.","marker":"(Skodras et al., 2001)"}],"fun_headline_variants":["Nested SAEs beat pruning for progressive coding","Matryoshka SAEs better than pruning for coding","Progressive coding: nested SAEs beat pruning","Nested SAEs win on coding, pruned on interpretability"],"cache_read_input_tokens":19200,"weakest_assumption_plain":"The pruning baseline is assumed to be strong: sorting features by how strongly they fire on average is the best tested ordering, but no optimal subset selection or alternative ranking heuristics are compared, so a better pruning method could close the gap with Matryoshka SAEs.","fun_headline_variants_meta":{"raw":{"variants":["Nested SAEs beat pruning for progressive coding","Matryoshka SAEs better than pruning for coding","Progressive coding: nested SAEs beat pruning","Nested SAEs win on coding, pruned on interpretability"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001125,"raw_usage":{"total_tokens":4722,"prompt_tokens":1031,"completion_tokens":3691,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":647,"completion_tokens_details":{"reasoning_tokens":3626}},"tokens_in":647,"tokens_out":3691,"duration_ms":25427,"temperature":1.0,"reasoning_tokens":3626,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T04:48:52.722494+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the optimal subset of $G$ features for each granularity on a pretrained TopK SAE, using exhaustive search for small dictionaries or greedy/orthogonal matching pursuit for large ones, and compare its FVU and recaptured language-modeling loss against the Matryoshka SAE at the same granularity. If the optimized pruned SAE matches or beats the Matryoshka SAE on the frontier, the paper's conclusion that Matryoshka SAEs are more efficient progressive coders would be overturned.","supporting_citations":[],"review_version":1}