{"id":"86a8bec5-4543-45f7-bfce-61ec49f42b35","arxiv_id":"2508.16560","paper_version":4,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Incorrect L0 makes sparse autoencoders mix correlated features rather than disentangling them, and a decoder projection metric can identify the correct L0.","lead":"Sparse autoencoders that unpack language model activations can learn wrong, mixed concepts when their sparsity level L0 is set too low or too high. The paper proposes a decoder projection score to pick the right L0 and argues that many popular open-source SAEs are trained too sparsely.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"In Gemma layer 5, s_dec_n's global minimum lies far above the claimed L0≈200; the paper substitutes an undefined visual 'elbow,' so the LLM validation of the metric is not established.","rationale":"The toy-model experiment is a solid and important result: it shows that a low-L0 SAE can mix correlated features and achieve better MSE than a ground-truth dictionary. That part should be credited. My concern is not with the toy result but with the transfer used for the headline LLM claim. The paper's own Figure 9 shows that s_dec_n's global minimum in Gemma-2-2b layer 5 is not at the claimed correct L0; the match to sparse probing is made with a visual elbow. This is an internal inconsistency rather than a disagreement with external consensus. Because the conclusion that public SAEs are too sparse depends on this elbow, the paper needs either a formal change-point estimator or a different validation. This concern is addressable and does not warrant rejection; it strengthens the reader's conditional verdict, so I keep the verdict unchanged.","tokens_in":22327,"tokens_out":5830,"duration_ms":69634,"concrete_test":"Using the released Gemma-2-2b layer 5 BatchTopK SAEs (or retraining a small sweep), recompute s_dec_n (Eq. 5) for n=12k, 14k, and 16k. Fit a piecewise-linear change-point model to s_dec_n vs L0 and compare the estimated knee to (a) the global minimum of s_dec_n and (b) the L0 maximizing k=16 sparse probing F1, with bootstrap or seed-based confidence intervals. If the knee, rather than the global minimum, is required to match probing, then the paper's central LLM claim is unsupported unless the elbow estimator is formally justified.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4 and Figure 9 claim that s_dec_n identifies the correct L0 in Gemma-2-2b layer 5 because the 'elbow' before the low-L0 jump coincides with peak k=16 sparse probing. However, the same figure shows that s_dec_n does not have a global minimum at that elbow: the global minimum is in the shallow high-L0 region (roughly L0≈750–1000). Thus the LLM experiments do not reproduce the toy-model result in which s_dec_n is minimized at the true L0. The paper instead reads the correct L0 from a visual elbow, choosing n=16k by hand, and Appendix A.9 confirms the elbow is consistent across n but is not a formal optimum. Because the headline conclusion that 'most SAEs have L0 too low' uses this elbow as ground truth, the LLM validation is circular unless the elbow is given a precise definition and is shown to be the actual optimum of a well-defined criterion.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper argues that the L0 sparsity level of a sparse autoencoder is not a free hyperparameter but has a correct value determined by the underlying data. Using toy models with ground-truth correlated features, it shows that when L0 is set too low, SAEs mix positively and negatively correlated features to improve reconstruction, and when L0 is set too high, they find degenerate mixed solutions. The authors propose an nth-decoder-projection metric, s_dec_n, that is minimized at the true L0 in toy models, and they report that an 'elbow' in this metric in LLM SAEs (Gemma-2-2b, Llama-3.2-1b) coincides with peak k=16 sparse probing performance. They conclude that most publicly available SAEs are trained with L0 too low and that sparsity–reconstruction tradeoff plots are misleading.","tokens_in":22554,"tokens_out":5846,"duration_ms":76518,"significance":"If the central claim holds, the paper would be a useful corrective to the common practice of treating L0 as a purely neutral axis in an evaluation tradeoff. The toy experiments are well-controlled: they use known ground truth, multiple seeds, and explicit comparison of learned SAEs against a ground-truth SAE, and the demonstration that a lower-MSE SAE can have less monosemantic latents is concrete and reproducible. The authors also ship code and, in the appendices, honestly document where their metric does not behave cleanly. The most valuable contribution is the identification of feature-mixing as a failure mode of low-L0 SAEs and the empirical warning that sparsity–reconstruction plots can reward incorrect solutions. However, the LLM-side validation of the proposed metric is not yet established, and the broad practical claim about public SAEs rests on that validation.","major_comments":[{"comment":"The LLM 'correct L0' is read from a visual elbow, not from a defined optimum. For Gemma-2-2b layer 5, Figure 9 shows that the global minimum of s_dec_n lies in the shallow high-L0 region (roughly L0≈750–1000), not at the claimed L0≈200. Appendix A.9 explicitly states that 'These plots never have a clear global minimum at the “elbow” point, but the “elbow” is always at the same point regardless of choice of N.' This undercuts the abstract's claim that the method 'finds the correct L0' in LLM SAEs. Because the same s_dec_n elbow is later used to conclude that public SAEs with L0<100 are too low, the LLM validation is circular unless the elbow is replaced by a precise, reproducible criterion and shown to be the relevant optimum of a well-defined objective. The independent k=16 sparse-probing peak helps, but then the metric is not what identifies the L0; the probing benchmark is.","section":"§4, Fig. 9; Appendix A.9"},{"comment":"The metric depends on a free hyperparameter n, and the LLM results are not robust to that choice. In the paper, n=16k (about h/2) is selected after inspecting the curves; for Gemma-2-2b layer 12 JumpReLU SAEs, only n=16k gives a clear s_dec_n minimum at the claimed 200–250, while other n values do not. For Gemma-2-2b layer 5, no n gives a global minimum at the elbow. A method whose output changes qualitatively with an unspecified hyperparameter needs either a principled way to choose n or a sensitivity analysis demonstrating the conclusion is invariant. As written, the toy-model result (n=12 and n=18 both minimized at the true L0) does not transfer to the LLM setting, so the claim that s_dec_n can 'guide the search for the correct L0' is not supported by the LLM experiments.","section":"§3.5, Eq. (5); §4 and Figs. 17–19"},{"comment":"The broad practical conclusion that 'most commonly used SAEs have an L0 that is too low' is based on two small LLMs (Gemma-2-2b, Llama-3.2-1b) and a few layers, with the toy model restricted to orthogonal linear features and correlated Bernoulli firing. Appendix A.8 acknowledges the non-linear feature exclusion and the limited layer coverage, but the Discussion and abstract do not temper the conclusion accordingly. This is not an internal inconsistency, but it means the headline claim is considerably stronger than the empirical support. The authors should either add evidence across more models/layers or explicitly reframe the claim as a hypothesis to be tested in broader settings.","section":"§6 and §A.8"}],"minor_comments":[{"comment":"Typographical issues: 'monsemanticity' in the Figure 1 caption should be 'monosemanticity'; 'afterall' should be 'after all'; 'saes' should be 'SAEs'; 'n th' in Section 3.5 should be 'nth'.","section":"Throughout"},{"comment":"The idealized histogram contains unrendered text ('0 sdec n sdec n'), which makes the figure hard to interpret. Please clean up the labels.","section":"Fig. 6"},{"comment":"The hyperparameter is sometimes called n and sometimes N; the caption of Figure 7 says 'slopes ... depend on N' while the text uses n. Please standardize.","section":"Fig. 7 and Appendix A.6"},{"comment":"The rationale for choosing n 'sufficiently larger than a reasonable guess at the correct L0' is vague. A more formal statement—e.g., in terms of the expected number of active latents above a projection threshold—would help practitioners implement the metric.","section":"§3.5"}],"recommendation":"major_revision","confidential_remarks":"The toy-model contribution is solid and likely publishable, but the LLM validation is the load-bearing part of the headline claims. The elbow-versus-global-minimum issue in Gemma layer 5 is not a presentation nit; it directly affects whether the proposed metric can be said to 'find' L0 in real models. If the authors can formalize the elbow criterion and show it is stable and coincides with an independent benchmark across layers, the paper would be much stronger. The current version asks the reader to accept the elbow as the correct L0 because the authors say so."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things. First, the toy-model half is genuinely good: with known ground-truth features, the paper shows that a BatchTopK SAE with L0 below the true value mixes correlated features, and that this incorrect SAE actually beats the ground-truth dictionary on MSE. That is a clean counterexample to how sparsity-reconstruction tradeoff plots get used, and it is the part worth keeping. Second, the LLM half is weaker than the abstract suggests. The 'correct L0' for Gemma layer 5 is read from a visual elbow in s_dec_n, not from a formal optimum, and the paper's own appendix admits these plots never have a global minimum at the elbow.\n\nThe toy experiments are well-controlled: orthogonal features, correlated Bernoulli firing, known true L0, and they check both BatchTopK and JumpReLU. The MSE result is load-bearing and it holds. The hysteresis experiment in Appendix A.4 — start too low and the SAE never fully recovers even after L0 is raised — is a good bonus. Code is released, so the toy results are reproducible. The s_dec_n metric works in the toy, minimized at the true L0 for the n values shown, and the decoder pairwise cosine similarity alternative gives a consistent picture.\n\nThe stress-test note is on target. In Gemma-2-2b layer 5, the global minimum of s_dec_n sits in the shallow high-L0 region around L0 750-1000, not at the claimed 200. The paper is transparent about this, but then still uses the elbow as the definition of correct L0 without a formal criterion. The independent support is the sparse probing correlation, which is real, but the Gemma layer 5 probing curve is a flat plateau from roughly L0 200 to 1000, so 'peak at 200' is generous. Llama layer 7 gives a cleaner minimum and a clearer probing peak, which helps the metric's case. The hyperparameter n (16k, near h/2) is hand-picked, though the elbow location is consistent across n in the appendix, which mitigates it.\n\nIs it circular? Only partially. The toy half is externally anchored. The LLM half uses the metric's own elbow to declare L0 200-250 correct and then concludes public SAEs with L0 under 100 are too low — that is circular unless the elbow is formalized. But the sparse probing correlation breaks the pure circle, and the authors are honest about limitations (linear features only, two small models, few layers).\n\nBottom line: the cautionary message — low L0 causes feature mixing and MSE actively rewards it — is solid and practically important. Treat the specific L0 numbers for real LLMs as a heuristic. This deserves peer review: the toy results are worth citing, and a serious referee should push for a precise elbow definition and error bars on probing, not reject the premise. I'd bring it to reading group.","headline":"The toy-model result is real and worth keeping — low-L0 SAEs beat the ground-truth dictionary on MSE — but the LLM-side 'correct L0' is a visual elbow, not a formal optimum, so 'most SAEs are too sparse' is plausible, not proven.","tokens_in":23084,"tokens_out":6420,"would_cite":true,"duration_ms":66523,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Setting an SAE's sparsity (L0) too low or too high makes it mix correlated features instead of isolating them, and a new decoder-projection metric can find the correct setting.","keywords":["sparse autoencoders","L0 sparsity","feature mixing","feature hedging","monosemanticity","decoder projection metric","linear representation hypothesis","mechanistic interpretability"],"falsifier":"Run the paper's toy-model experiment with L0 below the true L0 but with the feature correlation matrix modified so that correlated features never co-occur in the same training sample; the claim predicts latents stop mixing, since the reconstruction incentive to hedge disappears. Separately, on a layer not studied in the paper, sweep L0 and compare the s_dec_n elbow with k-sparse probing F1: divergence much larger than the alignment shown for Gemma-2-2b and Llama-3.2-1b would falsify the metric's general validity.","tokens_in":22154,"feed_emoji":"🧩","tokens_out":6350,"duration_ms":61494,"temperature":0.7,"pith_summary":"The paper claims that the L0 hyperparameter of a sparse autoencoder—the average number of latents that fire per token—is not a free design knob but has a correct value set by the data. When L0 is set too low, the SAE cheats by blending correlated features together to improve reconstruction; when set too high, it finds degenerate solutions that also blend features. Only at the correct L0 does the SAE recover the near-orthogonal \"true\" features of the model. The paper proposes a metric, the nth decoder projection score (s_dec_n), that locates the correct L0 in toy models and whose \"elbow\" coincides with peak sparse-probing performance in two small LLMs, and reports that most commonly released SAEs run at an L0 that is too low. If right, the work implies the standard sparsity–reconstruction tradeoff plot, the usual way SAEs are compared, is actively misleading.","feed_headline":"Wrong sparsity setting corrupts SAE features","feed_subtitle":"Too-low L0 makes sparse autoencoders mix correlated concepts; a decoder projection metric finds the right level.","key_machinery":"Nth decoder projection score (s_dec_n): compute projections of (x − b_dec) onto each decoder column for a batch, sort all values, and take the nth-highest (with n below h/2). At the correct L0, most non-firing latents have near-zero projection on arbitrary inputs, while mixed latents project strongly, so a low s_dec_n signals monosemanticity. The companion object is the toy model itself: orthogonal feature directions with correlated Bernoulli firing, in which the true L0 is known and the ground-truth SAE can be constructed, providing the controlled demonstration that low-L0 SAEs are driven by MSE to hedge correlated features.","core_discovery":"Working from the Linear Representation Hypothesis, the paper builds toy models with known orthogonal features whose firing is correlated, so the true L0 is known. It shows that when the SAE's L0 is below the true value, the reconstruction loss rewards latents that absorb positive components of positively correlated features and negative components of anti-correlated ones; a deliberately corrupted SAE beats the ground-truth dictionary on variance explained. When L0 is too high, the SAE converges to degenerate mixed solutions. The paper introduces s_dec_n, the nth-highest projection of decoder directions onto training activations, which is minimized at the true L0 in toy models and whose elbow","pith_inferences":["If the mixing mechanism transfers, a concrete fingerprint of too-low L0 in real SAEs is latents carrying negative components of anti-correlated features; inspecting latents for topically unrelated concepts (the paper's 'French poetry' vs 'Harry Potter' example) is a testable diagnostic.","The argument implies that any sparse dictionary learner on correlated data—not just LLM SAEs—has a data-dependent correct sparsity, so similar decoder-projection diagnostics may transfer to other sparse coding settings.","Because the appendix shows low-L0 damage is hard to reverse, automatic L0 search should be built as a high-to-low annealing; the paper's own heuristic embodies this but it is not stated as a general design principle.","The metric's stable elbow across small models suggests a cheaper validation protocol: train a small SAE suite on a few layers, locate the elbow, and extrapolate L0 for the full suite—an extension the paper does not claim."],"forward_implications":["Most open-source SAEs have L0 below 100 while the paper's proxy points to roughly 200–250 for Gemma-2-2b layer 12; if the claim holds, widely used feature dictionaries are partially mixed and noisier than reported.","Sparsity–reconstruction tradeoff plots should not be the primary SAE evaluation: they can rank a feature-mixing SAE above a perfect one.","Practitioners should sweep L0 and select it with s_dec_n or decoder pairwise cosine similarity, rather than treating sparsity as a free choice.","JumpReLU SAEs tolerate high L0 better than BatchTopK SAEs because per-latent thresholds adapt and \"stick\" near the correct firing rate.","Starting training with L0 too low can permanently trap the SAE in a bad local minimum, so L0 schedules should approach the target from above."],"supporting_citations":[{"why":"Supplies the feature-hedging mechanism the paper claims low L0 triggers: SAEs mix correlated features when resources are insufficient.","marker":"(Chanin et al., 2025)"},{"why":"Foundational for the Linear Representation Hypothesis and superposition, which justify the toy model of near-orthogonal features.","marker":"(Elhage et al., 2022)"},{"why":"BatchTopK SAEs, the architecture whose L0 is set directly and used for most experiments.","marker":"(Bussmann et al., 2024)"},{"why":"The JumpReLU training procedure the paper follows, including sparsity and dead-latent losses.","marker":"(Conerly et al., 2025)"},{"why":"The sparse probing benchmark used to validate that the s_dec_n optimum matches downstream performance in LLMs.","marker":"(Kantamneni et al., 2025)"},{"why":"Exemplifies the sparsity–reconstruction tradeoff evaluation practice the paper argues is unsound.","marker":"(Gao et al., 2024)"},{"why":"Introduces JumpReLU SAEs and is another instance of tradeoff-based evaluation the paper critiques.","marker":"(Rajamanoharan et al., 2024)"},{"why":"The Pile dataset used to train the LLM SAE sweeps.","marker":"(Gao et al., 2020)"}],"fun_headline_variants":["Wrong L0 mixes features in sparse autoencoders","SAEs need the right sparsity or features get mixed","A metric finds the correct L0 for sparse autoencoders","Most SAEs set sparsity too low, corrupting features","Find the right L0 to keep SAE features clean"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The toy model—orthogonal feature directions firing via a correlated Bernoulli process—is assumed to capture enough of how real LLM features behave that the low-L0 mixing mechanism and the s_dec_n minimum transfer to actual models; the paper validates this on only two small LLMs and a few layers, and explicitly leaves non-linear features (Appendix A.8) out of scope.","fun_headline_variants_meta":{"raw":{"variants":["Wrong L0 mixes features in sparse autoencoders","SAEs need the right sparsity or features get mixed","A metric finds the correct L0 for sparse autoencoders","Most SAEs set sparsity too low, corrupting features","Find the right L0 to keep SAE features clean"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000171,"raw_usage":{"total_tokens":1110,"prompt_tokens":746,"completion_tokens":364,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":490,"completion_tokens_details":{"reasoning_tokens":292}},"tokens_in":490,"tokens_out":364,"duration_ms":4012,"temperature":1.0,"reasoning_tokens":292,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T17:12:58.266228+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the paper's toy-model experiment with L0 below the true L0 but with the feature correlation matrix modified so that correlated features never co-occur in the same training sample; the claim predicts latents stop mixing, since the reconstruction incentive to hedge disappears. Separately, on a layer not studied in the paper, sweep L0 and compare the s_dec_n elbow with k-sparse probing F1: divergence much larger than the alignment shown for Gemma-2-2b and Llama-3.2-1b would falsify the metric's general validity.","supporting_citations":[{"cited_title":"Dictionary learning optimization techniques.https: //transformer-circuits.pub/2025/january-update,","cited_arxiv_id":null,"evidence_quote":"The JumpReLU training procedure the paper follows, including sparsity and dead-latent losses."}],"review_version":1}