{"id":"df4fe691-2f4e-4765-94f6-d18710ac3d2a","arxiv_id":"2506.20790","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"SPD uses stochastic masking and a learned causal importance function to decompose neural network parameters into sparsely active rank-one subcomponents, recovering ground-truth mechanisms in toy models where APD struggled.","lead":"This paper introduces Stochastic Parameter Decomposition (SPD), a method that breaks a neural network's weight matrices into small rank-one pieces and learns which pieces matter for each input by randomly ablating them and training to match the original outputs.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"With β3 too low, SPD finds faithful but non-minimal decompositions that pass reconstruction-based validation, yet Appendix A.3 recommends selecting β3 via known ground truth and the working range is never quantified; the robustness/identification claims lack ground-truth-free evidence.","rationale":"The reader's CONDITIONAL verdict with medium correctness risk is well-aligned with the evidence. I examined three candidate weaknesses: (i) the expressivity of the per-subcomponent causal importance functions (the reader's pick), (ii) seed sensitivity and the unexplained skew in Section 3.4, and (iii) the circularity between β3 selection and known ground truth. The expressivity limitation is real but explicitly conceded by the authors (Section 2.3.3) and does not threaten the demonstrated claims, since per-subcomponent inner activations suffice in all toy models; it is a generalization caveat, not a threat to the demonstration. Seed sensitivity and the unexplained skew are self-reported and do weaken the cross-layer flagship results, but the paper already hedges those as qualitative findings. The most load-bearing issue is (iii): the minimality objective is the sole force toward sparse mechanisms, its failure mode (β3 too low) exactly passes reconstruction-based validation, and the paper's own recommended heuristic for breaking this ambiguity uses known ground truth (Appendix A.3). With the working range of β3 unquantified and per-model β3 differing by orders of magnitude (Appendix A.4), the headline claims rest on ground-truth-informed tuning. This does not falsify the method — the TMS results are strong, quantified, and reported across seeds, and the code is released — but it does mean the demonstration's central claims are not yet established in the setting the method is meant to serve (no known ground truth). My proposed check, a wide β3 sweep with a ground-truth-free selection rule, would settle whether the claimed robustness is intrinsic or an artifact of tuning with the answer in hand. Since the reader's CONDITIONAL verdict already requires addressing these gaps, my analysis does not move the verdict.","tokens_in":21394,"tokens_out":15073,"duration_ms":168461,"concrete_test":"Sweep β3 over at least two orders of magnitude (e.g., 1e-6 to 1e-2) on TMS40-10 and the 1-layer compressed computation model, with 5 seeds per value, and select β3 using only the ground-truth-free heuristics of Appendix A.3 (negligible performance loss plus the 'some importance values ≈ 1' sanity check). Then measure MMCS/ML2R on TMS40-10 and the fraction of the 100 input features whose neuron contributions are recovered (Eq. 12) for compressed computation. If a ground-truth-free selection rule lands inside the working range in a majority of seeds and that range spans at least a factor of 3 in β3, the robustness claim survives; if selection misses the range or the range is narrower, the identification and robustness claims remain unsubstantiated for settings where ground truth is unknown.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central demonstration claim is that SPD 'identifies known ground-truth mechanisms up to a small error' and is 'considerably more scalable and robust to hyperparameters' than APD (Sections 3 and 5). The load-bearing assumption is that the SPD loss (Eq. 8) selects the minimal decomposition, but the only term pushing toward sparsity is the importance-minimality loss (Eq. 7) with coefficient β3. If β3 is too small, the causal importance functions can collapse to g≈1 everywhere: masks in Eq. 3 then concentrate at 1, the stochastic reconstruction losses (Eqs. 4 and 5) are trivially satisfied because the masked model equals the faithful sum of subcomponents, and the decomposition is faithful yet non-minimal. This failure mode is invisible to the first Appendix A.3 heuristic (negligible performance loss) and to the sanity check that 'some importance values should take values of 1', since in the bad regime g=1 everywhere. Figure 8 shows only three β3 values for one model (too low / just right / too high) and never quantifies the width of the working range; the per-model β3 values in A.4.1 (3e-3 vs 1e-4, with p=1 vs p=2) differ by more than an order of magnitude. So the quantitative MMCS=1.000 results (Table 1) were obtained with hyperparameters that, per the paper's own recommended heuristic (A.3, 'Recovering known mechanisms'), were selected using the known ground truth. Without a ground-truth-free selection rule, the experiments do not separate 'SPD finds ground-truth mechanisms' from 'β3 was tuned until the known answer appeared'. The authors' own reports of seed sensitivity and a systematic, unexplained skew in the 2-layer model (Section 3.4) corroborate that robustness is not yet established.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Stochastic Parameter Decomposition (SPD), a method for decomposing a neural network's weight matrices into rank-one subcomponents that sum to the original weights (faithfulness), are active as sparsely as possible per input (minimality), and are simple by construction. SPD trains a per-subcomponent causal importance function that predicts how ablatable each subcomponent is on a given input, and uses stochastically sampled masks to encourage the masked model to match the target model. The full loss is Eq. (8), combining a faithfulness term, two stochastic reconstruction losses, and an importance-minimality penalty with coefficient beta3. The paper evaluates SPD on several toy models with known ground-truth mechanisms (Toy Model of Superposition, TMS with a hidden identity, Toy Model of Compressed Computation, and two- and three-layer Cross-Layer Distributed Representations), reporting MMCS and ML2R near 1 for the superposition models and qualitative agreement for the others. The authors claim SPD is more scalable and robust to hyperparameters than Attribution-based Parameter Decomposition (APD), and release code and training details.","tokens_in":21794,"tokens_out":4277,"duration_ms":51868,"significance":"If the empirical claims hold, SPD is a meaningful step for linear parameter decomposition and mechanistic interpretability: it replaces gradient-based attributions with a directly trained causal importance model, avoids the top-k hyperparameter of APD, and is demonstrated on slightly larger and deeper toy models than APD could handle. The paper is commendably concrete: it releases code, provides pseudocode (Algorithm 1), reports hyperparameters in Appendix A.4, and includes the authors' own admission of unresolved issues, notably an unexplained skew in the two-layer model (Section 3.4, Figure 10) and the lack of a general clustering procedure. The central advertised property, however, is robustness to hyperparameters, and the evidence for that property is currently entangled with knowledge of the ground truth, as detailed in the major comments. The significance is therefore conditional: the method has clear promise, but the paper does not yet establish the robustness claim it advertises.","major_comments":[{"comment":"The central empirical claim that SPD identifies ground-truth mechanisms is not yet separated from knowledge of the ground truth. Section 3.3, Figure 8 shows that when beta3 is too small, the learned masks become non-monosemantic, but the decomposition remains faithful in the sense that the masked model reconstructs the target output; the Appendix A.3 heuristics 'Negligible performance loss' and 'Other sanity checks' do not rule out this regime, because the bad regime has importance values equal to 1 everywhere and no reconstruction failure. Appendix A.4.1 then reports beta3=3e-3 with p=1 for TMS5-2/TMS5-2+ID and beta3=1e-4 with p=2 for TMS40-10/TMS40-10+ID, a 30-fold variation, and the 'Recovering known mechanisms' heuristic explicitly uses the known decomposition to select beta3. Consequently, the MMCS=1.000 results in Table 1 may reflect selecting hyperparameters from the answer rather than a robust property of SPD. Please quantify the beta3 working range for at least one model and provide a ground-truth-free selection rule, for example a validation criterion that penalizes non-minimality without access to known mechanisms.","section":"§2.3.3, Eq. (7); §3.3, Fig. 8; Appendix A.3"},{"comment":"The two-layer Cross-Layer Distributed Representations result is presented as a success, yet Figure 10 shows a systematic skew toward higher values on the Y-axis, and the text states 'We currently do not understand the source of this discrepancy' and notes sensitivity to random seeds. Because the paper's central claim includes that SPD succeeds on this model, the unexplained systematic error and seed sensitivity should be resolved or quantified. Please report the number of seeds, the spread of the neuron-contribution agreement and MMCS across seeds, and whether the skew persists after hyperparameter search; without this, the reported result may reflect a favorable run rather than a stable property of SPD.","section":"§3.4, Fig. 10"},{"comment":"The MMCS metric in Eq. (9) takes a maximum over all C subcomponents. When C is large, as in the 2-layer and 3-layer models where C=400 and C=500 (Appendix A.4.2), random or spurious subcomponents can inflate the max, and the metric does not penalize the presence of extra subcomponents. Table 1 reports MMCS=1.000 to three decimal places for four models, which is strong; please report the full distribution of cosine similarities, the number of non-negligible subcomponents learned per model, and a precision/recall-style statistic that penalizes spurious subcomponents, so the quantitative claim is not driven by the max over many candidates.","section":"§3, Eq. (9)"},{"comment":"The identification argument in Section 5, footnote 6 asserts that in an idealized setting, any beta3 infinitesimally larger than zero makes the desired decomposition the global optimum, but this claim is not proved and is stated conditionally on several idealizations. The toy experiments do not establish robustness to beta3; they establish that a few hand-picked beta3 values recover known mechanisms. Please either prove the global-optimum claim for the specific linear and ReLU toy models, or replace it with a clearly delimited empirical claim about the beta3 working range, and report how the quantitative metrics vary across beta3 values for each model.","section":"§5, footnote 6"}],"minor_comments":[{"comment":"The causal importance function is defined as outputting values in [0,1], but the leaky hard sigmoids in Appendix A.2 deliberately produce outputs outside this range; the text should clarify how the mask sampling in Eq. (3) and the importance penalty in Eq. (7) treat out-of-range values.","section":"§2.3.1 and Appendix A.2"},{"comment":"The sentence 'APD fails to learn ground truth mechanisms in this model' is stated without a quantitative comparison or a reference to a figure/table; please add a quantitative comparison or move the claim to a discussion section.","section":"§3.2"},{"comment":"The statement that Eq. (5) is equivalent to Eq. (4) 'if the subcomponents sum to the original weights' should be expanded: the equivalence also requires that the layerwise losses are combined with weights that match the sampling distribution, and the current wording is too terse.","section":"§2.3.2, Eq. (5)"},{"comment":"The reference to 'unpublished work, forthcoming' should be replaced by a citation or removed, and the 'WandB report here' links should include stable URLs or DOIs.","section":"§3.3"},{"comment":"The caption and text mix 'left'/'right' with 'left two columns' and 'right two columns'; please standardize the direction labels so the too-low, just-right, and too-high regimes are unambiguous.","section":"Figure 8"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid engineering contribution and the code release is a strong point. My main reservation is that the advertised robustness to hyperparameters is not established without ground-truth-free beta3 selection; this is fixable with additional experiments and analysis, so I recommend major revision rather than rejection. I would also ask the editor to ensure that the authors address the two-layer seed sensitivity and the MMCS inflation concern before the paper is accepted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my take. The genuine novelty is SPD's stochastic-masking objective: instead of APD's attribution-plus-top-k, it learns per-subcomponent causal importance functions that predict ablatability, then trains masked subcomponents to match the output. That is a real departure, and the toy results show that with appropriate beta3, SPD recovers ground-truth mechanisms across all tested models, including two new ones (TMS+ID and a 3-layer cross-layer model) where APD struggled. The code is released, and the authors are unusually candid about limitations: they explicitly flag seed sensitivity, an unexplained systematic skew in the 2-layer results, the limited expressivity of their per-subcomponent importance functions (Section 2.3.3), and the need for clustering in real models (Section 5). That honesty counts for something.\n\nThe soft spot is the robustness claim. Minimality is carried entirely by beta3 * L_importance-minimality. If beta3 is too small, the importance functions collapse to g≈1 everywhere; masks then concentrate at 1, both stochastic reconstruction losses are trivially satisfied because the masked model equals the faithful sum, and you get a faithful but non-minimal decomposition. This failure mode is invisible to the paper's own heuristics: the 'negligible performance loss' check in A.3 is satisfied, and the 'some importance values equal 1' sanity check is also satisfied in the collapsed regime. Meanwhile, beta3 values in A.4.1 were chosen with known ground truth (A.3's 'Recovering known mechanisms' heuristic), and differ by more than an order of magnitude across TMS sizes. So Table 1's MMCS=1.000 demonstrates 'SPD can find the answer when beta3 is set right,' not 'SPD robustly identifies mechanisms without prior knowledge.' The working range of beta3 is never quantified, and the multi-layer results show seed sensitivity and a skew the authors say they don't understand. These issues don't sink the method, but they undercut the abstract's 'robust to hyperparameters' claim.\n\nI also agree with the reader's point that MMCS, by maxing over many subcomponents, can inflate alignment scores; the near-exact values are still suggestive, but the metric has upward bias.\n\nWho is this for? Mechanistic interpretability researchers working on parameter-space decomposition or extending APD. It's a subfield-level contribution, not a general-audience ML result. It deserves a serious referee: the method is novel, clearly presented, and the limitations are honestly stated. A referee should ask for a ground-truth-free beta3 selection rule or a quantified working range, and for repeated-seed error bars on the multi-layer models. With those, the paper would be substantially stronger.\n\nRecommendation: send it to peer review. Treat the central claims as promising but conditional: the method works, but the evidence for robustness is thinner than the abstract suggests.","headline":"A genuine, clearly-written method advance over APD, but the robustness claim is only as strong as the beta3 selection rule, which still leans on known ground truth.","tokens_in":22313,"tokens_out":4326,"would_cite":true,"duration_ms":46590,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"Stochastic Parameter Decomposition learns, per rank-one piece of a network's weights, how much that piece can be randomly ablated without changing outputs, and this suffices to recover known mechanisms in every toy model tested.","keywords":["stochastic parameter decomposition","linear parameter decomposition","mechanistic interpretability","causal importance","ablatability","rank-one factorization","causal mediation analysis","network decomposition"],"falsifier":"Construct a small network with a known mechanism whose causal importance genuinely depends on activity elsewhere in the network, for example a gated circuit where a downstream layer decides whether an upstream rank-one component matters on a given input. Apply SPD with its per-subcomponent, own-activation-only importance MLPs: if the learned importance values systematically disagree with brute-force measurement of how much each subcomponent can actually be ablated on inputs where the cross-layer dependence binds, the expressivity assumption behind the method gives way.","tokens_in":21204,"feed_emoji":"🧩","tokens_out":12434,"duration_ms":125544,"temperature":0.7,"pith_summary":"This paper introduces Stochastic Parameter Decomposition (SPD), a method for splitting a trained network's weights into a sum of rank-one subcomponents (each a single outer product of two vectors) so that on any given input only a small number of them are causally required. The central move is to replace gradient attributions with a learned causal importance function that predicts, for each subcomponent, how much it can be randomly ablated without changing the network's output, and then to train the network to behave identically under such random ablations while penalizing any subcomponent for staying important. If the method works as claimed, it removes the two barriers that kept linear parameter decomposition impractical, computational cost and sensitivity to the number-of-active-components hyperparameter, and extends the approach to deeper and more complex models. Across every toy model tested, including two new ones that defeated the earlier attribution-based method, SPD recovers the known ground-truth mechanisms with near-perfect cosine alignment between learned and true components and no parameter shrinkage.","feed_headline":"Random-ablation training recovers a network's true mechanisms","feed_subtitle":"SPD replaces fragile attributions and top-k tuning with learned ablatability, reaching models APD could not.","key_machinery":"The load-bearing object is the learned causal importance function: for each rank-one subcomponent $U^l_{:,c}V^l_{c,:}$ of weight matrix $W^l$, a small MLP maps the subcomponent's inner activation $h^l_c(x)=\\sum_j V^l_{c,j}a^l_j(x)$ to an importance value $g^l_c(x)\\in[0,1]$ through a leaky hard sigmoid. The importance value sets the lower end of a uniform mask $m^l_c(x,r)=g^l_c(x)+(1-g^l_c(x))r$ with $r\\sim U(0,1)$, so a subcomponent predicted unimportant is aggressively and randomly ablated, while a predicted-important one stays near full strength. Three objectives compete: faithfulness (the subcomponents sum to the original weights), stochastic reconstruction (the randomly masked network reproduces the target's outputs, both globally and layerwise), and minimality (the $\\ell^p$ magnitude of the importance values is penalized). The mechanism works because the stochastic masks probe every joint ablation pattern with some probability, so a subcomponent that is genuinely load-bearing on an input cannot afford a high ablation probability and keeps its importance high, whereas superfluous subcomponents are driven to importance zero.","core_discovery":"The paper claims that a neural network can be decomposed into its causal mechanisms without computing any attributions. For each weight matrix $W^l$ the method learns a set of rank-one subcomponents $\\sum_c U^l_{:,c}V^l_{c,:}$ together with a small learned predictor that outputs an importance value $g^l_c(x)\\in[0,1]$ for each subcomponent on each input; importance is defined operationally as the complement of ablatability, the amount by which the subcomponent can be randomly masked while leaving the output unchanged. On every input the subcomponents are masked by random amounts between their predicted importance and full strength, and the masked network is trained to reproduce the target output while a minimality penalty drives all importance values toward zero. The demonstrated result is that this stochastic ablation objective aligns subcomponents with ground-truth mechanisms: in toy models of superposition, compressed computation, and cross-layer distributed representations the learned decompositions match the known mechanisms up to small error, with mean max cosine similarity of 1.000 and mean L2 ratio close to 1, and without the shrinkage the attribution-based predecessor exhibited. In the compressed-computation model the method also reverses a mechanistic conclusion of the earlier method, finding that the MLP output matrix is one high-rank component rather than one per input feature.","pith_inferences":["If the local-importance assumption survives scaling, the practical bottleneck of the approach shifts to clustering: SPD produces per-layer rank-one pieces, and a general algorithmic grouping step, which the toy-model study avoids because ground truth names the clusters, will decide whether the method works on real networks.","The learned importance values double as a basis-free causal map: the masks specify which parameter vectors matter for which inputs, so SPD outputs could serve as circuit hypotheses that activation-patching or causal-scrubbing tests would then verify, without choosing units in advance.","A stress test the paper does not run: applying SPD to the embedding and unembedding matrices of a small transformer, where the one-component-per-token expectation can be checked directly and would give the first evidence the method transfers beyond toy MLPs.","The reported mean L2 ratios slightly exceed 1 (up to 1.031), a mild magnitude overshoot opposite in sign to the predecessor's shrinkage; whether this bias grows with depth and couples to the importance penalty $\\beta_3$ is left open."],"forward_implications":["SPD decomposes a three-layer residual MLP that the attribution-based method could not handle, with masks showing one subcomponent per input feature and a single output component spread across all three layers.","In every toy model tested the learned subcomponents recover the ground-truth mechanisms with mean max cosine similarity 1.000 and mean L2 ratio near 1, indicating correct orientation and no shrinkage of magnitudes.","On the Toy Model of Compressed Computation SPD finds that the MLP output matrix is a single rank-50 component, contradicting the earlier attribution-based decomposition's claim of one component per input feature.","SPD decomposes an identity matrix inserted into a superposition model into exactly $m_1$ rank-one subcomponents that sum to it, demonstrating the method does not suffer the feature-splitting failure of activation-space dictionary learning.","Because it needs no estimate of how many components are active per datapoint and lets gradients reach every subcomponent on every step, the authors expect SPD to scale to larger, non-toy models such as language models."],"supporting_citations":[{"why":"Defines the linear parameter decomposition framework, supplies the APD baseline and the toy models of compressed computation and cross-layer representations that SPD must beat, and is the source of the shrinkage and feature-splitting comparisons.","marker":"[Braun et al., 2025]"},{"why":"Supplies the Toy Model of Superposition, the first ground-truth test bed whose weight-matrix columns SPD must recover.","marker":"[Elhage et al., 2022]"},{"why":"Provides the reparameterization trick, the mechanism that lets gradients flow through the stochastic ablation masks into the causal importance functions.","marker":"Kingma and Welling [2013]"},{"why":"Documents feature splitting in sparse dictionary learning, which motivates the hidden-identity experiment showing SPD does not suffer the same failure.","marker":"[Bricken et al., 2023]"},{"why":"Provides evidence that gradient attributions are poor approximations of true causal importance, the premise for replacing attributions with learned ablatability.","marker":"[Syed et al., 2024]"},{"why":"Supplies the mean max cosine similarity metric used to quantify alignment between learned subcomponents and ground-truth weights.","marker":"[Sharkey et al., 2022]"},{"why":"Supplies the interference-term framework for computation in superposition, used both to interpret the compressed-computation model and to set reconstruction-loss targets.","marker":"[Hänni et al., 2024]"},{"why":"Shows that attribution methods can fail basic sanity checks, background for the paper's decision to abandon attribution-based importance estimates.","marker":"[Adebayo et al., 2018]"}],"fun_headline_variants":["No-attribution decomposition reveals true neural mechanisms","Stochastic masking decomposes nets without costly attributions","Learned ablatability scales network decomposition to bigger models","Training with random masks uncovers causal mechanisms in models"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that a subcomponent's ablatability on a given input is predictable from a single scalar, that subcomponent's own inner activation, and the authors concede that arbitrary models will likely require more expressive importance functions that take in more of the network.","fun_headline_variants_meta":{"raw":{"variants":["No-attribution decomposition reveals true neural mechanisms","Stochastic masking decomposes nets without costly attributions","Learned ablatability scales network decomposition to bigger models","Training with random masks uncovers causal mechanisms in models"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000247,"raw_usage":{"total_tokens":1578,"prompt_tokens":1015,"completion_tokens":563,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":631,"completion_tokens_details":{"reasoning_tokens":501}},"tokens_in":631,"tokens_out":563,"duration_ms":5899,"temperature":1.0,"reasoning_tokens":501,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T22:41:06.180224+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Construct a small network with a known mechanism whose causal importance genuinely depends on activity elsewhere in the network, for example a gated circuit where a downstream layer decides whether an upstream rank-one component matters on a given input. Apply SPD with its per-subcomponent, own-activation-only importance MLPs: if the learned importance values systematically disagree with brute-force measurement of how much each subcomponent can actually be ablated on inputs where the cross-layer dependence binds, the expressivity assumption behind the method gives way.","supporting_citations":[{"cited_title":"Towards monosemanticity: Decomposing language models with dictionary learning","cited_arxiv_id":null,"evidence_quote":"Documents feature splitting in sparse dictionary learning, which motivates the hidden-identity experiment showing SPD does not suffer the same failure."},{"cited_title":"Taking features out of superposition with sparse autoencoders, Dec 2022","cited_arxiv_id":null,"evidence_quote":"Supplies the mean max cosine similarity metric used to quantify alignment between learned subcomponents and ground-truth weights."},{"cited_title":"Sanity checks for saliency maps","cited_arxiv_id":null,"evidence_quote":"Shows that attribution methods can fail basic sanity checks, background for the paper's decision to abandon attribution-based importance estimates."}],"review_version":1}