{"id":"cd90ed17-6e11-457a-9e0f-d4d1c44509d6","arxiv_id":"2505.20254","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A position paper proposing feature consistency, measured by PW-MCC, as a core SAE evaluation criterion, with evidence that TopK SAEs achieve high consistency on LLM activations.","lead":"The paper argues that mechanistic interpretability should treat run-to-run feature consistency as a primary evaluation criterion for sparse autoencoders, and proposes PW-MCC as the metric. It reports that TopK SAEs reach about 0.80 consistency on LLM activations, linking higher consistency to more semantically stable features.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claim that PW-MCC is a reliable proxy for ground-truth feature recovery is only validated on a linear synthetic model; without a more realistic known-ground-truth check, high real-world consistency may not imply meaningful recovery.","rationale":"The reader's weakest assumption focuses on the reachability condition in the theoretical argument, and I agree that this is a genuine weakness. However, the more load-bearing downstream consequence is that the empirical proxy claim, namely that PW-MCC tracks ground-truth recovery, has only been demonstrated in a narrow linear synthetic regime. If that proxy fails on more realistic data, the headline 0.80 number loses its interpretive value and the recommendation to adopt PW-MCC as a primary evaluation metric becomes harder to justify. The paper deserves credit for releasing code, performing multi-seed sweeps for several architectures, and including a synthetic model organism with Zipfian structure; these are real contributions that support the attainability of consistency. The concern is not that the authors are dishonest or that the empirical results are fabricated, but that an important extrapolation from synthetic to real data is underevidenced. Because the paper is framed as a position paper, a conditional acceptance is the right level of scrutiny: the central normative claim can stand, but the supporting evidence for PW-MCC as a ground-truth proxy should be either extended to a more realistic known-ground-truth setting or explicitly scoped. My read does not change the reader's verdict, so I leave it unchanged.","tokens_in":30501,"tokens_out":11504,"duration_ms":126870,"concrete_test":"Train TopK SAEs on activations of a board-game model with known ground-truth concepts (e.g., the Othello or chess models used in Karvonen et al., [27]), using the same dictionary width and sparsity choices as in the Pythia-160M experiments. Compute PW-MCC across 5 seeds and GT-MCC by matching learned dictionary vectors to known concept directions. If PW-MCC remains high (about 0.8) while GT-MCC is low (below about 0.6) in a compressive, Zipfian regime analogous to real LLM data, then the proxy claim does not transfer beyond the linear synthetic model and the central argument would need to be weakened.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 5.1 states that 'PW-MCC acts as a justifiable proxy for ground-truth alignment in unsupervised settings,' citing the synthetic experiments of Section 4.2. Those experiments use a linear generative model with exact k-sparse codes and a matched-capacity regime; Figure 2 shows PW-MCC tracking GT-MCC only there. The theoretical bridge (Theorem 1 plus Corollary 1 in Appendix C) requires zero reconstruction error and reachability of every k-sparse code; Appendix C.5 concedes exact reachability cannot hold on finite real data. Consequently, on real LLM activations there is no direct evidence that a PW-MCC of about 0.80 (Section 5.2) reflects recovery of underlying features rather than mere run-to-run agreement on some arbitrary dictionary. The semantic validation in Table 2 additionally uses the same LLM to generate and judge explanations, so high vector similarity mechanically yields similar top-activating contexts and thus similar explanations; the correlation is partly circular. If PW-MCC is not a valid proxy outside the linear synthetic regime, the paper's call to prioritize consistency as a key evaluation metric is premature. This is load-bearing because the practical force of the central position depends on consistency being meaningful, not merely reproducible.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper argues that mechanistic interpretability (MI) should prioritize feature consistency in sparse autoencoders (SAEs), operationalized as the reliable convergence to equivalent feature sets across independent training runs. The authors propose the Pairwise Dictionary Mean Correlation Coefficient (PW-MCC) as a practical metric, provide a theoretical argument connecting SAE training to dictionary learning identifiability, present synthetic experiments showing PW-MCC tracks ground-truth recovery (GT-MCC), and report real-world experiments on Pythia-160M and Gemma-2-2B activations in which TopK SAEs achieve PW-MCC of about 0.80 and feature-level similarity correlates with semantic explanation similarity. The paper concludes with a call for the community to routinely report consistency metrics.","tokens_in":30766,"tokens_out":5442,"duration_ms":57408,"significance":"If the empirical findings hold, the paper offers a constructive resolution to a well-documented problem: learned SAE features are often unstable across runs, undermining reproducibility and cumulative progress. The release of code, the multi-seed protocol, and the comparison across seven architectures on two LLMs are strengths, as is the synthetic validation of PW-MCC against a known ground truth in a matched-capacity setting. The central position is plausible and potentially influential. However, the theoretical guarantee is conditional on idealized reachability and zero-error conditions that are not met in the real-world experiments, and the validity of PW-MCC as a proxy for ground-truth recovery is demonstrated only in a linear synthetic regime. The paper's practical force thus depends on additional empirical support that it does not yet provide.","major_comments":[{"comment":"The theoretical guarantee rests on conditions that are not satisfied in the experiments that the theory is invoked to support. Corollary 1 requires zero reconstruction error on the training set and reachability of every k-sparse code f, i.e., for all f in Sigma_k there exists x in X with E(x)=f. Appendix C.5 concedes that exact reachability cannot be verified on finite datasets, and the real-world SAE training (Section 5.2) does not achieve zero reconstruction error. Consequently, the claim that the theoretical analysis 'explains why TopK SAEs can achieve consistent features' (Section 4.1) is an analogy rather than a proof. Please either empirically check the conditions to the extent possible and report how far they are from being satisfied, or explicitly frame the theorem as an idealized sufficient condition and state that the real-data results are only suggestive.","section":"Section 4.1 and Appendix C, Corollary 1"},{"comment":"The statement that 'PW-MCC acts as a justifiable proxy for ground-truth alignment in unsupervised settings' is supported only by synthetic experiments with a linear generative model, exact k-sparse codes, and a matched-capacity regime (Section 4.2). Real LLM activations are not produced by such a generator, and the SAE operates in a globally compressive regime where d_SAE is far smaller than the number of latent concepts. The paper does not demonstrate that PW-MCC tracks GT-MCC in any setting resembling the LLM experiments. The semantic validation in Table 2 is partly circular: the same LLM (gpt-4.1) both generates and evaluates the explanations, and features with high dictionary-vector cosine similarity will have similar top-activating contexts mechanically, so high GPT scores are expected even if the learned features do not correspond to meaningful latent factors. Please add an external validation, e.g., a different judge LLM or human evaluation, or a synthetic benchmark with a more realistic generative process and known ground truth.","section":"Section 5.1 and Section 4.2"},{"comment":"The headline PW-MCC values are the maximum over a hyperparameter sweep that was designed to maximize PW-MCC: 'For each SAE, we performed a hyperparameter sweep, selecting the configuration that yielded the highest final PW-MCC across three independent training runs.' Thus the reported 0.80 for TopK SAEs is an upper extreme, not a typical or average outcome, and the claim that 'high consistency is achievable' is inflated by selection. This matters because the paper's position is to prioritize consistency as an evaluation metric; showing that consistency can be optimized by searching hyperparameters is weaker than demonstrating that a standard configuration reliably yields high consistency. Please report the distribution of PW-MCC across the sweep (e.g., median and range) and state explicitly that the reported values were selected to maximize the metric.","section":"Section 5.2 and Appendix F.1"}],"minor_comments":[{"comment":"The sentence 'The training objective simultaneously encourages ... (2) Zero reconstruction error by minimizing ||X-AF||_F' is imprecise: minimizing an objective does not guarantee zero error unless the data lies in the span of the learned dictionary. Rephrase as 'aims to achieve zero reconstruction error.'","section":"Section 4.1"},{"comment":"The caption states that GPT-score is averaged over 20 pairs, but the number of feature pairs per similarity range is 34, 311, 975, 1,423, and 13,640. Please clarify how the 20-pair average is computed and whether the reported scores are based on a random subsample.","section":"Table 2"},{"comment":"In the definition of the TopK encoder, the tie-breaking rule is specified as 'lexicographically' but the encoder selects the k largest magnitude inner products. It would be clearer to state that ties are broken deterministically by feature index.","section":"Appendix C.1"},{"comment":"The figure caption describes 'Token frequency in 1M tokens from Pile' but does not state which model's tokenizer or dataset split was used. Please add this detail for reproducibility.","section":"Figure 4"}],"recommendation":"major_revision","confidential_remarks":"This is a position paper, and the empirical comparison of SAE architectures on real LLM activations is a useful contribution. However, the theoretical section overclaims the connection between the identifiability theorem and the observed results, and the proxy validity of PW-MCC needs strengthening. The authors should either soften the theoretical language or add empirical checks of the theory's assumptions. The semantic validation in Table 2 should be redone with a separate judge model to avoid circularity; currently it largely reflects that similar vectors produce similar top-activating contexts."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nRead this if you have any stake in SAE evaluation or dictionary learning for interpretability. It is a position paper that earns its place: the case for measuring run-to-run feature consistency is well-argued, the metric is practical, and the empirical findings are the real contribution.\n\nWhat is new: the systematic demonstration that TopK SAEs reach PW-MCC around 0.80 on LLM activations while L1-penalized variants sit around 0.5–0.6; the synthetic model organism linking feature frequency, local capacity regimes, and consistency; and the frequency-consistency spectrum reproduced on real data across architectures. PW-MCC is borrowed from ICA—they credit that—but the adaptation to SAE evaluation is sensible, and the metric is defined independently of fitted hyperparameters. The synthetic experiments are clean, code is released, and the frequency-consistency finding is robust across architectures. That is more than most position papers deliver.\n\nSoft spots, in proportion. The theoretical guarantee (Section 4.1 and Appendix C) assumes zero reconstruction error and reachability of every k-sparse code on the training data; the paper concedes in C.5 that exact reachability cannot hold on finite real data. The theorem is a motivating analogy for the real experiments, not a proof, and that framing belongs in the main text rather than the appendix. The headline 0.80 comes from a hyperparameter sweep that selects the best configuration, disclosed in 5.2 but still the max. For a position paper this is acceptable, but readers should read it as an upper envelope rather than typical performance.\n\nThe stress-test concern that PW-MCC is validated as a ground-truth proxy only on a linear synthetic model is fair, and the real-data semantic validation only partially answers it: vector-similar matched features get near-identical explanations, and vector-dissimilar pairs get divergent ones. The weakness is that the explanations and the judge come from the same pipeline (100 scored pairs, 20 per bin), so the semantic check is suggestive rather than airtight. The claim that PW-MCC is a justifiable proxy for ground-truth alignment is overstated; the evidence supports a useful diagnostic that correlates with functional similarity.\n\nThe central position holds up: consistency is worth prioritizing even if perfect consistency is unattainable, and the alternative-views section engages the community counterarguments honestly.\n\nRecommendation: send this to serious referees. With the theoretical claims reframed as motivation and the sweep selection and semantic validation disclosed more honestly, it becomes a solid contribution. I would bring it to reading group and would cite it in my own SAE evaluation work.","headline":"A genuinely useful position paper with solid empirical contributions, a theory that is motivation rather than proof, and a headline number that is best-case rather than typical.","tokens_in":31317,"tokens_out":5670,"would_cite":true,"duration_ms":56982,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Mechanistic interpretability should benchmark sparse autoencoders on run-to-run feature consistency, and TopK SAEs can pass the test.","keywords":["sparse autoencoders","feature consistency","mechanistic interpretability","dictionary learning","identifiability","PW-MCC","TopK SAE","Zipfian feature frequencies"],"falsifier":"Train two TopK SAEs with identical data and architecture on LLM activations and check the semantic-similarity result: if matched feature pairs with high dictionary cosine similarity are frequently judged semantically unrelated by the same explanation pipeline, the claim that PW-MCC tracks meaningful semantic stability would fail. Alternatively, in a synthetic setting with known ground truth, find a configuration where PW-MCC is high while GT-MCC stays low, which would break the proxy claim that pairwise consistency indicates ground-truth recovery.","tokens_in":30265,"feed_emoji":"🧩","tokens_out":5623,"duration_ms":62382,"temperature":0.7,"pith_summary":"This position paper argues that mechanistic interpretability should treat run-to-run feature consistency as a first-class evaluation criterion for sparse autoencoders (SAEs), not an afterthought. It proposes the Pairwise Dictionary Mean Correlation Coefficient (PW-MCC), a matching-based score that asks whether two independently trained dictionaries contain the same feature directions up to permutation and scaling. The paper claims high consistency is achievable, around 0.80 for TopK SAEs on large language model activations, and that PW-MCC tracks ground-truth feature recovery in synthetic settings, making it a usable proxy when no ground truth exists. A reader should care because inconsistent SAE features make circuit explanations, steering, and unlearning unreliable: if the feature vocabulary shifts between runs, then interpretations built on it cannot be reproduced or reused.","feed_headline":"TopK SAEs hit 0.80 run-to-run feature consistency","feed_subtitle":"A matching-based score, PW-MCC, turns reproducibility into a checkable number for interpretability research.","key_machinery":"The load-bearing object is the Mean Correlation Coefficient (MCC) and its pairwise variant PW-MCC: a Hungarian-algorithm-matched one-to-one assignment that maximizes the sum of absolute cosine similarities between dictionary columns, normalized to the unit interval. Strong Feature Consistency is defined as alignment of two dictionaries up to permutation and per-feature nonzero scaling, and PW-MCC measures precisely that. On the theoretical side, the round-trip property $E(Af)=f$ for all $k$-sparse $f$, together with the spark condition from dictionary learning, turns TopK SAE training into a procedure whose solutions are identifiable up to permutation and scaling; this is the mechanism that explains why TopK SAEs, which by construction produce exactly $k$-sparse codes and are trained to reconstruct, can converge to equivalent feature sets.","core_discovery":"On the paper's own terms, the discovery is that feature consistency is not an unsolvable pathology but an attainable, measurable property, and that the path to it runs through architectural choices that enforce exact sparsity and zero reconstruction error. Theory adapted from sparse dictionary learning says a dictionary satisfying the spark condition, meaning no two distinct $k$-sparse codes produce the same output, is identifiable up to permutation and scaling once the data covers a certain witness set. The paper proves that a round-trip property, encoder(decoder($f$)) $=$ $f$ for every $k$-sparse $f$, implies the spark condition, and argues that TopK SAEs' training objective promotes exactly that. In synthetic matched settings TopK SAEs reach PW-MCC around 0.97; on LLM activations they reach around 0.80, with frequent features more consistent than rare ones, and high dictionary-vector similarity between matched features correlates with semantic similarity of their generated explanations.","pith_inferences":["A natural but untested corollary is that consistency can also serve as a selection signal for data quality: if the same TopK recipe yields high PW-MCC on one layer or corpus and low on another, the lower value may indicate that the activation distribution lacks the coverage the identifiability theorem needs, not merely that training failed.","The frequency-consistency spectrum suggests a division of labor for future SAE designs: reserve the dictionary's capacity for the frequent, stable core and treat the long tail as explicitly approximate, rather than expecting one architecture to be uniformly consistent.","An obvious extension is to turn PW-MCC into a training objective or regularizer: because it is evaluated on a matched-pair basis, one could test whether adding a consistency-promoting penalty between concurrent runs raises PW-MCC, something the paper motivates but does not implement.","Because the theory's reachability assumption is untestable on real data, the LLM results should be read as empirical evidence for an analogy rather than as a theorem instantiated."],"forward_implications":["If PW-MCC is adopted as a standard evaluation number, SAE papers can be compared on reproducibility alongside reconstruction loss, and hyperparameters such as the TopK sparsity $k$ can be chosen where consistency peaks.","Routine reporting of PW-MCC contextualized by feature frequency would turn feature interpretation from a one-off per-run activity into reusable annotations: stably matched features can be labeled once and carried across runs.","Consistency-based evaluation would surface a frequency-dependent quality spectrum, with frequent features highly reproducible and rare-tail features less so, guiding where interpretability effort is best spent.","Building interpretability claims such as circuits, steering, and unlearning on consistent dictionaries would make those claims robust to random initialization, addressing a documented failure mode where overlap between runs can be as low as 30 percent for standard SAEs."],"supporting_citations":[{"why":"Supplies the identifiability theorem, adapted as Theorem 1, guaranteeing two dictionaries coincide up to permutation and scaling when spark and data coverage hold.","marker":"[22]"},{"why":"Defines the spark condition and the equivalence between spark(A)>2k and injectivity, used in the round-trip proof.","marker":"[12]"},{"why":"Introduces the TopK SAE architecture whose exact k-sparsity is the mechanism behind the consistency result.","marker":"[18]"},{"why":"Documents that standard SAEs trained on the same data learn different features, the failure mode the paper responds to.","marker":"[44]"},{"why":"Establishes the standard L1-penalized ReLU SAE used as the baseline architecture.","marker":"[6]"},{"why":"Provides the Mean Correlation Coefficient from independent component analysis that PW-MCC and GT-MCC are built on.","marker":"[24]"},{"why":"Supplies the automated feature explanation pipeline used to show that high vector similarity correlates with semantic similarity.","marker":"[45]"},{"why":"Provides the language model whose activations are used for the real-data consistency experiments.","marker":"[4]"}],"fun_headline_variants":["SAEs: Prioritize feature consistency for reliable MI","PW-MCC: A metric for reproducible SAE features","TopK SAEs ensure 0.80 feature consistency","Feature consistency: The missing SAE priority","Make SAE features consistent: PW-MCC scores"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The consistency guarantee for TopK SAEs rests on assuming the training data contains, for every possible k-sparse code, an activation that the encoder maps to that code; on finite real datasets this exact reachability cannot hold, so the real-world experiments are an analogy to the theorem rather than an instance of it.","fun_headline_variants_meta":{"raw":{"variants":["SAEs: Prioritize feature consistency for reliable MI","PW-MCC: A metric for reproducible SAE features","TopK SAEs ensure 0.80 feature consistency","Feature consistency: The missing SAE priority","Make SAE features consistent: PW-MCC scores"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000183,"raw_usage":{"total_tokens":1313,"prompt_tokens":943,"completion_tokens":370,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":559,"completion_tokens_details":{"reasoning_tokens":308}},"tokens_in":559,"tokens_out":370,"duration_ms":4185,"temperature":1.0,"reasoning_tokens":308,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T13:55:51.453522+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train two TopK SAEs with identical data and architecture on LLM activations and check the semantic-similarity result: if matched feature pairs with high dictionary cosine similarity are frequently judged semantically unrelated by the same explanation pipeline, the claim that PW-MCC tracks meaningful semantic stability would fail. Alternatively, in a synthetic setting with known ground truth, find a configuration where PW-MCC is high while GT-MCC stays low, which would break the proxy claim that pairwise consistency indicates ground-truth recovery.","supporting_citations":[{"cited_title":"When can dictionary learning uniquely recover sparse data from subsamples?IEEE Transactions on Information Theory, 61(11):6290–6297, 2015","cited_arxiv_id":null,"evidence_quote":"Supplies the identifiability theorem, adapted as Theorem 1, guaranteeing two dictionaries coincide up to permutation and scaling when spark and data coverage hold."},{"cited_title":"Optimally sparse representation in general (nonorthogonal) dictionaries via l1 minimization.Proceedings of the National Academy of Sciences, 100(5):2197– 2202, 2003","cited_arxiv_id":null,"evidence_quote":"Defines the spark condition and the equivalence between spark(A)>2k and injectivity, used in the round-trip proof."},{"cited_title":"Towards monosemanticity: Decomposing language models with dictionary learning","cited_arxiv_id":null,"evidence_quote":"Establishes the standard L1-penalized ReLU SAE used as the baseline architecture."},{"cited_title":"Pythia: A suite for analyzing large language models across training and scaling","cited_arxiv_id":null,"evidence_quote":"Provides the language model whose activations are used for the real-data consistency experiments."}],"review_version":1}