{"id":"e616f9f2-80ce-49a2-a7f5-c3a32d4494bb","arxiv_id":"2608.08168","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"Chain-of-thought reasoning in DeepSeek-R1-Distill-Qwen-7B depends on a small set of high-intensity sparse features; suppressing them breaks both reasoning and LaTeX formatting.","lead":"DeepSeek-R1-Distill-Qwen-7B's math reasoning is studied with sparse autoencoders, comparing chain-of-thought ('Thinking') with direct answering ('NoThinking'). The paper finds Thinking uses a stable, high-intensity set of features, while NoThinking is more diffuse, and suppressing key features degrades both reasoning and output formatting.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Causal intervention lacks a reconstruction-only (alpha=0) control, so the observed degradation could be an artifact of SAE decoding rather than feature suppression.","rationale":"The reader's weakest_assumption correctly identifies SAE faithfulness and intervention validity as load-bearing. My concern sharpens this to a specific missing control: the paper never runs a reconstruction-only condition (alpha=0). This is the most direct way to test whether the intervention protocol itself introduces artifacts. If the alpha=0 control shows no metric shift, then the observed degradation is indeed due to feature suppression, supporting the causal claims. If it shows a shift, then the entire causal analysis is confounded, and the three principles could be reconstruction artifacts rather than properties of the model's reasoning. This concern is more precise than a general call for SAE fidelity metrics because it isolates the exact step where the artifact would enter. The reader's rationale also mentions lack of control baselines, so there is partial agreement. The verdict remains CONDITIONAL because the paper is otherwise addressable by adding this control and possibly re-running the analysis; it does not necessarily require rejection if the control passes. Thus UNCHANGED is appropriate.","tokens_in":10582,"tokens_out":4989,"duration_ms":50638,"concrete_test":"Run the intervention hook for Thinking mode on the same benchmark problems with alpha=0 (i.e., decode the SAE latents without modification and replace the layer-13 residual stream with the reconstruction). Compare output length, LaTeX density, boxed-answer retention, and metacognitive density against the baseline (no hook). If these metrics shift beyond a pre-specified threshold (e.g., >5% relative change), the intervention protocol is confounded by reconstruction error. Additionally, include a control where a randomly selected low-TAV feature (e.g., rank > 1000) is suppressed at alpha=1.0; if this produces degradation comparable to the top-3 features, the effects are not specific to 'specialized' features.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's causal conclusions (Section 4.3) rest on a single intervention protocol (Eq. 6) that multiplies a latent activation by (1-alpha). To affect the model, this modified latent must be decoded back to the residual stream, but the manuscript never states whether the residual stream is replaced by the SAE reconstruction of the modified latents, nor does it report the SAE's reconstruction fidelity. Crucially, no control condition is reported in which the hook is applied with alpha=0 (i.e., the residual stream is replaced by the SAE reconstruction of the original activations). If reconstruction error is non-negligible (plausible for a Top-K SAE with K=20), then even alpha=0 would distort the model's hidden states, and any subsequent metric changes would be attributable to the reconstruction operation, not to the suppression of the targeted feature. Since the selected features are the most active by TAV, suppressing them further increases the reconstruction discrepancy, conflating feature importance with reconstruction sensitivity. Without this control, the three principles (coupling of reasoning and syntax, compensatory over-generation, fragile coordination) are not causally identified.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper trains Top-K Sparse Autoencoders on layer-13 residual-stream activations of DeepSeek-R1-Distill-Qwen-7B, separately for the model's Thinking and NoThinking modes, using DeepMath-103K as the training corpus. It compares the top-20 features ranked by Total Activation Volume across Easy, Medium, and Hard mathematical benchmarks and reports that Thinking mode relies on sparse, high-intensity feature activations that remain stable across difficulty, whereas NoThinking mode exhibits a more diffuse and adaptive activation pattern. The paper then causally suppresses the top-3 Thinking-mode features by Total Activation Volume and reports three principles: coupling of reasoning and syntactic structure, compensatory sequence extension, and fragile coordination among specialized features. The central conclusion is that Chain-of-Thought is a low-redundancy, feature-level control regime rather than a standalone reasoning module.","tokens_in":10831,"tokens_out":4360,"duration_ms":44308,"significance":"If the observational and causal claims hold, the paper would make a useful contribution to mechanistic interpretability by providing a comparative feature-dynamics account of Thinking versus NoThinking modes in a reasoning-specialized LLM. The explicit intervention protocol and the attempt to characterize CoT as a control regime rather than a module are valuable conceptual framing, and the manuscript formulates several falsifiable predictions about feature-level suppression effects. However, the current evidence is substantially weaker than the claims require: there are no error bars or significance tests for any quantitative result, the causal intervention lacks a crucial reconstruction-only control, and the feature-selection procedure is entangled with the metrics used to characterize the selected features. The paper's value is therefore conditional on addressing these load-bearing methodological gaps.","major_comments":[{"comment":"The causal intervention protocol is missing the essential alpha=0 reconstruction-only control, and the manuscript does not report the SAE's reconstruction fidelity. Equation (6) modifies a latent activation, but to affect the model this modified latent must be decoded back into the residual stream; the paper never states whether the residual stream is replaced by the full SAE reconstruction of the modified latents. Without an alpha=0 condition, any degradation observed under suppression could be attributable to the reconstruction operation rather than to the targeted feature. This is load-bearing for all three causal principles reported in Section 4.3, and it must be addressed by reporting reconstruction loss (e.g., L2/L0) and by including an alpha=0 control that replaces the residual stream with the SAE reconstruction of the original activations.","section":"§4.1.6, Eq. (6); §4.3"},{"comment":"The token-category percentages in Table 2 do not sum to 100 for any column (e.g., Thinking-Easy sums to 74.0% and NoThinking-Easy sums to 69.5%). This suggests either an incomplete taxonomy, overlapping categories, or an unreported residual category, and it undermines the quantitative comparisons of verbal-deduction versus symbolic-manipulation tokens across difficulty levels. The authors should either report a complete and mutually exclusive categorization that sums to 100% per column or explicitly state that the remaining tokens fall into an 'other' category, and they should provide per-cell confidence intervals or significance tests for the differences they claim.","section":"§4.2.2, Table 2"},{"comment":"The feature-selection metric is not independent of the claims it is used to support. The observational claim that Thinking mode relies on sparse, high-intensity features is based on features selected by Total Activation Volume, and the same metric is then used to choose the top-3 intervention targets in Section 4.3. The causal results therefore cannot distinguish the importance of the selected features from the selection criterion itself. I would like to see robustness checks that suppress randomly chosen features, lower-ranked features, or features selected by an independent criterion, and an analysis that reports whether the observed effects are specific to the TAV-selected set.","section":"§4.1.5, §4.1.6, §4.3"},{"comment":"All causal results in Table 4 are reported as single values with no indication of the number of problems, number of decoding runs, or variance across runs. For example, 'Boxed Answer Retention 0%' versus '10%' is reported without the denominator, and 'Output Length Change +454%' is reported without a confidence interval. These results cannot be evaluated as statistical evidence for the three mechanisms without repeated sampling, error bars, and a statement of how many samples underlie each number. The authors should report full experimental repetitions, standard deviations or confidence intervals, and the underlying sample sizes for every metric in Table 4.","section":"§4.3, Table 4"}],"minor_comments":[{"comment":"The title 'Casual Evaluation Metrics' should read 'Causal Evaluation Metrics'.","section":"Appendix A.4"},{"comment":"Reference [24] is incomplete and reference [25] is malformed; both should be completed in the final bibliography.","section":"References"},{"comment":"The examples in Table 3 contain renderings such as 'P\\in fty' and '\\impl iessin' that appear to be corrupted LaTeX; these should be fixed so the activation contexts are readable.","section":"Table 3"},{"comment":"The TopK operator in Eq. (4) is described with an absolute-value ranking but the formal definition would benefit from stating precisely how ties are broken and how the ReLU non-linearity is applied.","section":"§3.3"},{"comment":"The paper should state how many tokens or sequences from each benchmark are used for the activation statistics and whether the same problem instances are evaluated in both Thinking and NoThinking modes.","section":"§4.1.3"},{"comment":"For the Boxed Answer Retention metric, the paper reports only post-intervention values (0% or 10%); the baseline retention rate on the same problems should also be reported, since a low baseline would change the interpretation.","section":"§4.3"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely and interesting question, and the comparative Thinking/NoThinking SAE design is a reasonable contribution to mechanistic interpretability. However, the current manuscript does not provide code, data, or sufficient experimental detail to verify the quantitative claims, and the missing alpha=0 control is a fundamental gap in the causal inference. The editor may wish to ask for reproducibility artifacts in revision. The citation list is broad but includes two malformed references; there is no evidence of citation manipulation beyond those formatting issues."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The genuinely new thing here is the comparison design: same model, same math problems, Thinking vs NoThinking, with both observational feature statistics and causal suppression. Prior SAE work mostly studies CoT alone; this paper shows that Thinking mode runs on sparse high-intensity features while NoThinking is more diffuse, and that this holds across difficulty levels. That is a real result for DeepSeek-R1-Distill-Qwen-7B, and the qualitative token analysis (F4416 as reasoning monitor, F10770 as structural formatter) is a useful illustration. Credit where due: the framework is sensible, the three governing principles are clearly stated, and the paper is honest about what it observes.\n\nThe soft spots are concentrated where the causal claims live. The intervention protocol in Eq. 6 never says whether the residual stream is replaced with the SAE reconstruction of the modified latents, and no alpha=0 control is reported. If reconstruction error is non-negligible for this Top-K SAE, then even replacing the residual stream with the SAE's own reconstruction of the original activations would distort behavior, and the observed degradation in LaTeX density and boxed retention could be an artifact of the decoding operation, not of feature suppression. This is not a minor omission; it is the difference between \"suppressing feature 28634 causes over-generation\" and \"any perturbation of the hidden state via SAE causes degradation.\" The paper also reports no error bars or significance tests anywhere, so the observational contrasts in Figures 1-3 could be noise. Table 2's token categories do not sum to 100%, which suggests an unreported \"other\" bucket or an inconsistent taxonomy. The feature selection is partly circular: TAV picks the top features and TAV is also used to select intervention targets, though the causal suppression does at least provide independent evidence. The marker sets are hand-picked, which is fine if disclosed, and the lack of released code or SAE checkpoints makes the results hard to verify.\n\nNone of this kills the paper. The flaws are addressable: add an alpha=0 reconstruction control, report reconstruction fidelity, provide error bars and a per-condition number of samples, fix the token taxonomy, and release artifacts. With those, the three principles could become a solid empirical claim.\n\nThis deserves a serious referee, with the understanding that revision should be substantial. I would not cite it in its current form, but I would bring it to reading group to argue about the intervention protocol.","headline":"A genuinely comparative Thinking vs NoThinking SAE study with an important mechanism hypothesis, but the causal intervention lacks the one control that would make the headline claims stick.","tokens_in":11330,"tokens_out":1431,"would_cite":false,"duration_ms":16250,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Chain-of-thought in DeepSeek-R1-Distill-Qwen-7B is carried by a sparse, low-redundancy set of features whose suppression simultaneously breaks reasoning and mathematical formatting.","keywords":["sparse autoencoders","chain-of-thought","mechanistic interpretability","causal intervention","mathematical reasoning","DeepSeek-R1-Distill-Qwen-7B","total activation volume","feature ablation"],"falsifier":"Run the same suppression protocol on randomly selected SAE features matched in Total Activation Volume; if those random features produce the same LaTeX collapse and the same 454%-scale verbosity expansion, then the top-3 feature selection is not the cause and the coupling result is an artifact of the intervention rather than evidence about reasoning. A complementary check would be to verify that suppressing the same features in NoThinking mode produces no comparable formatting collapse, as the paper's Thinking-specific account predicts.","tokens_in":10377,"feed_emoji":"🧠","tokens_out":7371,"duration_ms":69923,"temperature":0.7,"pith_summary":"The paper sets out to show that chain-of-thought reasoning in DeepSeek-R1-Distill-Qwen-7B is not a separate reasoning module layered on top of ordinary generation. Using a Top-K sparse autoencoder on layer-13 residual-stream activations, it claims that Thinking mode runs on a small set of sparse, high-intensity features whose peak activation stays roughly constant as math problems get harder, while NoThinking mode spreads activation across a broader, difficulty-dependent set of features that favor symbolic manipulation. The stakes are practical: if a handful of features carry both the logical deduction and the LaTeX formatting of the answer, then feature-level intervention becomes a plausible lever for controlling or repairing reasoning behavior. The paper's causal claim is that suppressing the three most active features by Total Activation Volume degrades mathematical formatting, triggers verbose and repetitive continuation, and reveals that coherent chain-of-thought depends on fragile coordination among a few specialized features.","feed_headline":"Chain-of-thought is a sparse control regime, not a reasoning module","feed_subtitle":"Suppressing three sparse features collapses LaTeX output and triggers verbose, repetitive continuations.","key_machinery":"The central object is a Top-K Sparse Autoencoder trained separately on Thinking and NoThinking residual-stream activations from layer 13 of DeepSeek-R1-Distill-Qwen-7B, with a dictionary of $2^{16}$ features. Feature importance is scored by Total Activation Volume (TAV), the sum of a feature's activation magnitudes over the validation corpus, and the causal intervention multiplies a target feature's latent activation by $(1-\\alpha)$ for $\\alpha \\in \\{0.1, 0.3, 0.5, 1.0\\}$ during thinking-block generation only. This combination does the paper's work: the SAE makes the residual stream legible as a sparse set of interpretable directions, TAV picks which directions are functionally important, and the $(1-\\alpha)$ suppression converts the observed activation differences into claims about what the model needs in order to reason.","core_discovery":"On the paper's own terms, the discovery is that Thinking and NoThinking modes separate cleanly in sparse-feature space, and that the separation is causally load-bearing. Thinking mode activates a small number of features at high magnitude, with the dominant feature F4416 holding nearly constant intensity across easy, medium, and hard problems, whereas NoThinking mode runs at higher mean activation with lower maxima, recruiting different feature mixtures that consolidate toward single-feature dominance only on hard tasks. Suppressing the top-three TAV features F28634, F4416, and F8893 inside the thinking block produces a consistent drop in LaTeX density, near-zero boxed-answer retention, and a 454% output-length expansion with a 63% drop in Distinct-1 when the core feature F28634 is removed. The paper interprets these results as showing that reasoning and syntactic structure share the same sparse representations, that Thinking responds to disruption with compensatory low-information verbosity, and that coherent chain-of-thought is a finely tuned, low-redundancy control regime rather than a standalone module.","pith_inferences":["A test the paper does not run: suppressing the same top-3 features in NoThinking mode should, under the paper's account, produce weaker formatting collapse because NoThinking relies on a different, diffuse feature coalition; running it would directly test the mode-specificity of the coupling claim.","The compensatory over-generation pattern (longer output, more metacognitive markers, lower lexical diversity) resembles a generic failure mode of next-token predictors when their preferred computational path is blocked; a natural extension is to check whether low-TAV or random feature suppression triggers the same verbosity in non-reasoning generation.","The paper's TAV-based ranking could become a practical steering recipe for reasoning models, for example suppressing F28634 to curtail overthinking or amplifying F4416 to encourage self-correction, but the paper does not demonstrate such control.","The NoThinking mode's convergence to single-feature dominance on hard tasks hints that extreme difficulty may force even direct-answer models into a sparse regime, which would blur the dichotomy the paper draws; this is a speculation, not a paper claim."],"forward_implications":["If one small set of sparse features carries both deduction and formatting, then steering a single feature should be able to push the model between concise, well-formatted reasoning and verbose, repetitive degeneration.","The stability of dominant-feature intensity across difficulty levels implies that feature-level control points found on easy tasks remain useful on hard tasks, without retuning per benchmark.","Because reasoning and syntactic structure share representations, attempts to improve CoT by adding or removing formatting constraints will also alter the reasoning process itself.","The fragile-coordination result predicts that multi-feature suppression will produce qualitatively different failure modes rather than additive degradation, so intervention studies need to map the feature-interaction graph, not just single-feature effects."],"supporting_citations":[{"why":"Supplies the model under study, DeepSeek-R1-Distill-Qwen-7B, whose thinking-block structure defines the two inference modes.","marker":"[1]"},{"why":"Introduces the dictionary-learning view of SAE features as interpretable monosemantic directions, the conceptual basis for reading reasoning off sparse features.","marker":"[12]"},{"why":"Provides the Top-K activation mechanism and the scaling law used to set the number of features and training protocol.","marker":"[15]"},{"why":"Establishes the scaled SAE training recipe and the intermediate-layer motivation adopted for layer 13.","marker":"[16]"},{"why":"Introduces k-sparse autoencoders, the architecture family the Top-K variant is built on.","marker":"[21]"},{"why":"Supplies DeepMath-103K, the corpus on which the mode-specific SAEs are trained.","marker":"[22]"},{"why":"Defines the Thinking versus NoThinking inference modes used throughout the comparison.","marker":"[23]"},{"why":"Supports the choice of an intermediate residual-stream layer as the site of high-level reasoning features.","marker":"[26]"},{"why":"Provides the Distinct-1 metric used to measure lexical diversity and detect repetitive degeneration after intervention.","marker":"[27]"}],"fun_headline_variants":["CoT is a sparse control regime, not a reasoning module","Suppressing 3 sparse features collapses CoT and inflates verbosity","CoT uses few high-intensity features; blocking them breaks reasoning","Sparse features drive both reasoning and LaTeX formatting in CoT","CoT's fragile sparse features: 3 suppressions wreak havoc on output"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the Top-K SAE trained on layer-13 residual-stream activations reconstructs the model faithfully enough that multiplying one of its latent activations by $(1-\\alpha)$ changes the model's actual behavior in a way that reflects the model's own circuitry rather than an artifact of the intervention method.","fun_headline_variants_meta":{"raw":{"variants":["CoT is a sparse control regime, not a reasoning module","Suppressing 3 sparse features collapses CoT and inflates verbosity","CoT uses few high-intensity features; blocking them breaks reasoning","Sparse features drive both reasoning and LaTeX formatting in CoT","CoT's fragile sparse features: 3 suppressions wreak havoc on output"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000709,"raw_usage":{"total_tokens":3213,"prompt_tokens":986,"completion_tokens":2227,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":602,"completion_tokens_details":{"reasoning_tokens":2134}},"tokens_in":602,"tokens_out":2227,"duration_ms":16660,"temperature":1.0,"reasoning_tokens":2134,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T00:19:00.032446+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same suppression protocol on randomly selected SAE features matched in Total Activation Volume; if those random features produce the same LaTeX collapse and the same 454%-scale verbosity expansion, then the top-3 feature selection is not the cause and the coupling result is an artifact of the intervention rather than evidence about reasoning. A complementary check would be to verify that suppressing the same features in NoThinking mode produces no comparable formatting collapse, as the paper's Thinking-specific account predicts.","supporting_citations":[{"cited_title":"Scaling monosemanticity: Extracting interpretable features from claude 3 sonnet","cited_arxiv_id":null,"evidence_quote":"Supports the choice of an intermediate residual-stream layer as the site of high-level reasoning features."},{"cited_title":"wait\", \"hmm","cited_arxiv_id":null,"evidence_quote":"Provides the Distinct-1 metric used to measure lexical diversity and detect repetitive degeneration after intervention."}],"review_version":1}