Pith. sign in

REVIEW 4 major objections 6 minor 27 references

Thinking vs. NoThinking: Towards Interpreting Reasoning Mechanisms of Large Language Models via Sparse Autoencoders

T0 review · 4 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Chain-of-thought in DeepSeek-R1-Distill-Qwen-7B is carried by a sparse, low-redundancy set of features whose suppression simultaneously breaks reasoning and mathematical formatting.

desk verdict A genuinely comparative Thinking vs NoThinking SAE study with an important mechanism hypothesis, but the causal intervention lacks the one control that would make the headline claims stick. read the letter →

arxiv 2608.08168 v1 pith:A7AQC5G5 submitted 2026-08-08 cs.CL

Bo Cheng , Qiaolin Lu , Yi Chang , Yuan Wu This is my paper · ORCID
classification cs.CL
keywords sparseautoencoderschain-of-thoughtmechanisticinterpretabilitycausalinterventionmathematicalreasoningDeepSeek-R1-Distill-Qwen-7Btotalactivationvolumefeatureablation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that chain-of-thought reasoning in DeepSeek-R1-Distill-Qwen-7B is not a separate reasoning module layered on top of ordinary generation. Using a Top-K sparse autoencoder on layer-13 residual-stream activations, it claims that Thinking mode runs on a small set of sparse, high-intensity features whose peak activation stays roughly constant as math problems get harder, while NoThinking mode spreads activation across a broader, difficulty-dependent set of features that favor symbolic manipulation. The stakes are practical: if a handful of features carry both the logical deduction and the LaTeX formatting of the answer, then feature-level intervention becomes a plausible lever for controlling or repairing reasoning behavior. The paper's causal claim is that suppressing the three most active features by Total Activation Volume degrades mathematical formatting, triggers verbose and repetitive continuation, and reveals that coherent chain-of-thought depends on fragile coordination among a few specialized features.

What carries the argument

The central object is a Top-K Sparse Autoencoder trained separately on Thinking and NoThinking residual-stream activations from layer 13 of DeepSeek-R1-Distill-Qwen-7B, with a dictionary of $2^{16}$ features. Feature importance is scored by Total Activation Volume (TAV), the sum of a feature's activation magnitudes over the validation corpus, and the causal intervention multiplies a target feature's latent activation by $(1-\alpha)$ for $\alpha \in \{0.1, 0.3, 0.5, 1.0\}$ during thinking-block generation only. This combination does the paper's work: the SAE makes the residual stream legible as a sparse set of interpretable directions, TAV picks which directions are functionally important, and the $(1-\alpha)$ suppression converts the observed activation differences into claims about what the model needs in order to reason.

What would settle it

Run the same suppression protocol on randomly selected SAE features matched in Total Activation Volume; if those random features produce the same LaTeX collapse and the same 454%-scale verbosity expansion, then the top-3 feature selection is not the cause and the coupling result is an artifact of the intervention rather than evidence about reasoning. A complementary check would be to verify that suppressing the same features in NoThinking mode produces no comparable formatting collapse, as the paper's Thinking-specific account predicts.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that Thinking and NoThinking modes separate cleanly in sparse-feature space, and that the separation is causally load-bearing. Thinking mode activates a small number of features at high magnitude, with the dominant feature F4416 holding nearly constant intensity across easy, medium, and hard problems, whereas NoThinking mode runs at higher mean activation with lower maxima, recruiting different feature mixtures that consolidate toward single-feature dominance only on hard tasks. Suppressing the top-three TAV features F28634, F4416, and F8893 inside the thinking block produces a consistent drop in LaTeX density, near-zero boxed-answer retention, and a 454% output-length expansion with a 63% drop in Distinct-1 when the core feature F28634 is removed. The paper interprets these results as showing that reasoning and syntactic structure share the same sparse representations, that Thinking responds to disruption with compensatory low-information verbosity, and that coherent chain-of-thought is a finely tuned, low-redundancy control regime rather than a standalone module.

Load-bearing premise

The load-bearing premise is that the Top-K SAE trained on layer-13 residual-stream activations reconstructs the model faithfully enough that multiplying one of its latent activations by $(1-\alpha)$ changes the model's actual behavior in a way that reflects the model's own circuitry rather than an artifact of the intervention method.

Editorial extensions

If this is right

  • If one small set of sparse features carries both deduction and formatting, then steering a single feature should be able to push the model between concise, well-formatted reasoning and verbose, repetitive degeneration.
  • The stability of dominant-feature intensity across difficulty levels implies that feature-level control points found on easy tasks remain useful on hard tasks, without retuning per benchmark.
  • Because reasoning and syntactic structure share representations, attempts to improve CoT by adding or removing formatting constraints will also alter the reasoning process itself.
  • The fragile-coordination result predicts that multi-feature suppression will produce qualitatively different failure modes rather than additive degradation, so intervention studies need to map the feature-interaction graph, not just single-feature effects.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A test the paper does not run: suppressing the same top-3 features in NoThinking mode should, under the paper's account, produce weaker formatting collapse because NoThinking relies on a different, diffuse feature coalition; running it would directly test the mode-specificity of the coupling claim.
  • The compensatory over-generation pattern (longer output, more metacognitive markers, lower lexical diversity) resembles a generic failure mode of next-token predictors when their preferred computational path is blocked; a natural extension is to check whether low-TAV or random feature suppression triggers the same verbosity in non-reasoning generation.
  • The paper's TAV-based ranking could become a practical steering recipe for reasoning models, for example suppressing F28634 to curtail overthinking or amplifying F4416 to encourage self-correction, but the paper does not demonstrate such control.
  • The NoThinking mode's convergence to single-feature dominance on hard tasks hints that extreme difficulty may force even direct-answer models into a sparse regime, which would blur the dichotomy the paper draws; this is a speculation, not a paper claim.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper trains Top-K Sparse Autoencoders on layer-13 residual-stream activations of DeepSeek-R1-Distill-Qwen-7B, separately for the model's Thinking and NoThinking modes, using DeepMath-103K as the training corpus. It compares the top-20 features ranked by Total Activation Volume across Easy, Medium, and Hard mathematical benchmarks and reports that Thinking mode relies on sparse, high-intensity feature activations that remain stable across difficulty, whereas NoThinking mode exhibits a more diffuse and adaptive activation pattern. The paper then causally suppresses the top-3 Thinking-mode features by Total Activation Volume and reports three principles: coupling of reasoning and syntactic structure, compensatory sequence extension, and fragile coordination among specialized features. The central conclusion is that Chain-of-Thought is a low-redundancy, feature-level control regime rather than a standalone reasoning module.

Significance. If the observational and causal claims hold, the paper would make a useful contribution to mechanistic interpretability by providing a comparative feature-dynamics account of Thinking versus NoThinking modes in a reasoning-specialized LLM. The explicit intervention protocol and the attempt to characterize CoT as a control regime rather than a module are valuable conceptual framing, and the manuscript formulates several falsifiable predictions about feature-level suppression effects. However, the current evidence is substantially weaker than the claims require: there are no error bars or significance tests for any quantitative result, the causal intervention lacks a crucial reconstruction-only control, and the feature-selection procedure is entangled with the metrics used to characterize the selected features. The paper's value is therefore conditional on addressing these load-bearing methodological gaps.

major comments (4)
  1. [§4.1.6, Eq. (6); §4.3] The causal intervention protocol is missing the essential alpha=0 reconstruction-only control, and the manuscript does not report the SAE's reconstruction fidelity. Equation (6) modifies a latent activation, but to affect the model this modified latent must be decoded back into the residual stream; the paper never states whether the residual stream is replaced by the full SAE reconstruction of the modified latents. Without an alpha=0 condition, any degradation observed under suppression could be attributable to the reconstruction operation rather than to the targeted feature. This is load-bearing for all three causal principles reported in Section 4.3, and it must be addressed by reporting reconstruction loss (e.g., L2/L0) and by including an alpha=0 control that replaces the residual stream with the SAE reconstruction of the original activations.
  2. [§4.2.2, Table 2] The token-category percentages in Table 2 do not sum to 100 for any column (e.g., Thinking-Easy sums to 74.0% and NoThinking-Easy sums to 69.5%). This suggests either an incomplete taxonomy, overlapping categories, or an unreported residual category, and it undermines the quantitative comparisons of verbal-deduction versus symbolic-manipulation tokens across difficulty levels. The authors should either report a complete and mutually exclusive categorization that sums to 100% per column or explicitly state that the remaining tokens fall into an 'other' category, and they should provide per-cell confidence intervals or significance tests for the differences they claim.
  3. [§4.1.5, §4.1.6, §4.3] The feature-selection metric is not independent of the claims it is used to support. The observational claim that Thinking mode relies on sparse, high-intensity features is based on features selected by Total Activation Volume, and the same metric is then used to choose the top-3 intervention targets in Section 4.3. The causal results therefore cannot distinguish the importance of the selected features from the selection criterion itself. I would like to see robustness checks that suppress randomly chosen features, lower-ranked features, or features selected by an independent criterion, and an analysis that reports whether the observed effects are specific to the TAV-selected set.
  4. [§4.3, Table 4] All causal results in Table 4 are reported as single values with no indication of the number of problems, number of decoding runs, or variance across runs. For example, 'Boxed Answer Retention 0%' versus '10%' is reported without the denominator, and 'Output Length Change +454%' is reported without a confidence interval. These results cannot be evaluated as statistical evidence for the three mechanisms without repeated sampling, error bars, and a statement of how many samples underlie each number. The authors should report full experimental repetitions, standard deviations or confidence intervals, and the underlying sample sizes for every metric in Table 4.
minor comments (6)
  1. [Appendix A.4] The title 'Casual Evaluation Metrics' should read 'Causal Evaluation Metrics'.
  2. [References] Reference [24] is incomplete and reference [25] is malformed; both should be completed in the final bibliography.
  3. [Table 3] The examples in Table 3 contain renderings such as 'P\in fty' and '\impl iessin' that appear to be corrupted LaTeX; these should be fixed so the activation contexts are readable.
  4. [§3.3] The TopK operator in Eq. (4) is described with an absolute-value ranking but the formal definition would benefit from stating precisely how ties are broken and how the ReLU non-linearity is applied.
  5. [§4.1.3] The paper should state how many tokens or sequences from each benchmark are used for the activation statistics and whether the same problem instances are evaluated in both Thinking and NoThinking modes.
  6. [§4.3] For the Boxed Answer Retention metric, the paper reports only post-intervention values (0% or 10%); the baseline retention rate on the same problems should also be reported, since a low baseline would change the interpretation.

Circularity Check

1 steps flagged · score 4.0 of 10

The 'sparse' descriptor in the central observational claim is imposed by the Top-K SAE probe rather than discovered; the causal suppression analysis is independent and not circular.

  1. self definitional [Section 3.3 (Eq. 4-5) and Section 4.2.1; echoed in the Abstract]
    "This architecture enforces sparsity directly through the activation mechanism... the model imposes a hard constraint by retaining only the k most significant latents... z=TopK(h)... Since sparsity is strictly enforced by k... This indicates a highly sparse distribution where a small number of features activate intensely and the majority remain suppressed."

    The Top-K SAE fixes exactly K=20 nonzero latents per token for both Thinking and NoThinking modes (Eq. 4), so 'sparse' and 'majority remain suppressed' are properties of the probe rather than empirical differences between modes. The abstract's headline that Thinking 'relies on sparse ... feature activations' is therefore an architectural input, not a derived finding. The reported mean/max/std contrasts are comparisons of activation magnitude among the preselected top-20 TAV features and do not establish per-token sparsity as a mode-specific property. The causal suppression results in Section 4.3 do not depend on this label and remain independent.

full rationale

The paper's main causal contribution -- targeted suppression of the top-3 TAV features and the resulting behavioral changes -- is a genuine intervention test and is not derived by construction from any fitted equation or from self-citation. The three principles (reasoning-syntax coupling, compensatory over-generation, fragile coordination) are empirical observations of perturbed outputs, and their identification does not reduce to the TAV metric used to choose the targets. However, the observational headline that Thinking mode 'relies on sparse and high-intensity feature activations' is partly circular: sparsity is enforced by the chosen Top-K architecture (K=20) for both modes, so calling the mode 'sparse' restates the probe's design rather than a measured distinction. The high-intensity characterization is also influenced by selecting features via Total Activation Volume, though the specific magnitudes and cross-difficulty stability are not forced by that criterion. The missing alpha=0 reconstruction-only control noted by the skeptic is an experimental validity concern, not a circularity, and does not affect this score. No load-bearing self-citation or imported uniqueness theorem is present.

Assumptions & free parameters 8 free parameters · 7 assumptions · 0 invented entities

The paper introduces no new physical or theoretical entities; its ledgers rest on standard SAE assumptions, hand-chosen analysis dimensions, and a small set of curated markers. The causal claims depend critically on the faithfulness of the SAE intervention, which is not validated with reconstruction metrics or control ablations.

free parameters (8)
  • Layer index = 13
    Chosen as a representative intermediate layer based on prior work (Appendix A.1); all analyses depend on this layer.
  • Feature dimension C = 2^16 = 65536
    Derived from scaling law C∝Z^γ with γ≈0.60-0.65, but the exact constant is not specified.
  • Top-K sparsity schedule = K anneals from 200 to 20 over first 50% of initial epoch
    Chosen to balance reconstruction and sparsity; not fitted to target outcomes.
  • Number of top features analyzed = 20
    Top-20 features by TAV used for all observational comparisons.
  • Number of features suppressed = 3
    Top-3 features by TAV on easy task selected for causal intervention.
  • Suppression strength alpha = grid {0.1, 0.3, 0.5, 1.0}
    Intervention strengths explored; reported results likely at one strength, not specified.
  • Metacognitive marker set = 16 markers
    Hand-curated list in Appendix A.4, used as a density metric.
  • Uncertainty marker set = 9 markers
    Hand-curated list in Appendix A.4, used as a density metric.
assumptions (7)
  • domain assumption SAE features are monosemantic and interpretable directions in the residual stream.
    Adopted from prior SAE literature (Sections 3.2-3.3) and used to interpret features like F4416 as 'reasoning monitor' and F10770 as 'structural formatter'.
  • domain assumption The layer-13 residual stream is a representative locus for high-level reasoning.
    Appendix A.1 states this follows [26,16], but it is not empirically verified for this model; all analyses depend on this layer choice.
  • domain assumption Modifying SAE latents via Eq. (6) yields a valid causal intervention on the model.
    The paper does not validate reconstruction fidelity or the injection method; the intervention's construct validity is assumed.
  • domain assumption The scaling law C ∝ Z^γ (γ≈0.60-0.65) from prior studies applies to this model and justifies C=2^16.
    Used in Appendix A.1 to rationalize feature dimension; an empirical law treated as given.
  • domain assumption NoThinking mode with an empty thinking box is a valid control that bypasses explicit reasoning.
    Section 4.1.3 adopts this from [23]; differences between modes could also reflect decoding constraints rather than reasoning mechanisms.
  • domain assumption Benchmark difficulty categories (AMC23: Easy, AIME: Medium, OlympiadBench: Hard) form a comparable ordinal scale across modes.
    Section 4.1.4 and Appendix A.2; evaluation subsets differ in size and provenance.
  • ad hoc to paper Hand-crafted marker lists (16 metacognitive, 9 uncertainty) and LaTeX environment counts are valid measures of the intended constructs.
    Appendix A.4 defines these metrics without validation against human judgments or external benchmarks.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Thinking vs. NoThinking: Towards Interpreting Reasoning Mechanisms of Large Language Models via Sparse Autoencoders." pith.science (2026). https://pith.science/paper/A7AQC5G5

@misc{pith2026260808168,
  author       = {Pith},
  title        = {Pith review of: Thinking vs. NoThinking: Towards Interpreting Reasoning Mechanisms of Large Language Models via Sparse Autoencoders},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/A7AQC5G5}},
  note         = {Machine review of arXiv:2608.08168}
}
read the original abstract

While Large Language Models (LLMs) employing Chain-of-Thought (CoT) exhibit superior reasoning capabilities, the neural mechanisms distinguishing this explicit Thinking mode from direct answer generation (NoThinking mode) remain poorly understood. To deconstruct this cognitive process, we apply Top-K Sparse Autoencoders (SAEs) to the intermediate representations of DeepSeek-R1-Distill-Qwen-7B and examine the model's divergent behaviors across math-solving tasks of three distinct difficulty levels. Observationally, we identify a clear distinction in how the model functions under two reasoning modes: Thinking mode relies on sparse and high-intensity feature activations driving verbal deduction independent of problem complexity, whereas NoThinking mode exhibits an adaptive and diffuse pattern prioritizing symbolic manipulation. Causally, suppressing the three most active sparse features by Total Activation Volume reveals three principles: (i) reasoning and syntactic structure are tightly coupled, as interventions consistently degrade \LaTeX{} and boxed-solution formatting; (ii) Thinking responds to disruption with compensatory over-generation marked by increased metacognitive cues and repetitive, low-information continuations; and (iii) coherent CoT behavior depends on a fragile coordination among specialized features, yielding distinct failure modes under perturbation but a consistently impaired output structure.

Figures

Figures reproduced from arXiv: 2608.08168 by the authors.

Figure 1
Figure 1. The mean, maximum, and standard deviation (variability) of activations for the top-20 feature vectors across [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Heatmaps showing the normalized activations of the top-20 features across difficulty levels for [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. The activation trajectories of the top-20 SAE features across difficulty levels for both [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

27 extracted references · 6 canonical work pages

  1. [1]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    arXiv 2025

  2. [2]

    Chain-of-thought prompting elicits reasoning in large language models.Advances in neural information processing systems, 35:24824–24837, 2022

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models.Advances in neural information processing systems, 35:24824–24837, 2022

    2022

  3. [3]

    Language models don’t always say what they think: Unfaithful explanations in chain-of-thought prompting.Advances in Neural Information Processing Systems, 36:74952–74965, 2023

    Miles Turpin, Julian Michael, Ethan Perez, and Samuel Bowman. Language models don’t always say what they think: Unfaithful explanations in chain-of-thought prompting.Advances in Neural Information Processing Systems, 36:74952–74965, 2023

    2023

  4. [4]

    Let’s verify step by step

    Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. InThe Twelfth International Conference on Learning Representations, 2023

    2023

  5. [5]

    How does chain of thought think? mechanistic interpretability of chain-of-thought reasoning with sparse autoencoding.arXiv preprint arXiv:2507.22928, 2025

    Xi Chen, Aske Plaat, and Niki van Stein. How does chain of thought think? mechanistic interpretability of chain-of-thought reasoning with sparse autoencoding.arXiv preprint arXiv:2507.22928, 2025

    arXiv 2025

  6. [6]

    Finding sparse autoencoder representations of errors in cot prompting

    Justin Theodorus, V Swaytha, Shivani Gautam, Adam Ward, Mahir Shah, Cole Blondin, and Kevin Zhu. Finding sparse autoencoder representations of errors in cot prompting. InICLR 2025 Workshop on Building Trust in Language Models and Applications

    work page 2025

  7. [7]

    Progress measures for grokking via mechanistic interpretability.arXiv preprint arXiv:2301.05217, 2023

    Neel Nanda, Lawrence Chan, Tom Lieberum, Jess Smith, and Jacob Steinhardt. Progress measures for grokking via mechanistic interpretability.arXiv preprint arXiv:2301.05217, 2023

    arXiv 2023

  8. [8]

    Emergent world representations: Exploring a sequence model trained on a synthetic task.arXiv preprint arXiv:2210.13382, 2022

    Kenneth Li, Aspen K Hopkins, David Bau, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. Emergent world representations: Exploring a sequence model trained on a synthetic task.arXiv preprint arXiv:2210.13382, 2022

    arXiv 2022

Show all 27 references
  1. [9]

    Representation engineering: A top-down approach to ai transparency.arXiv preprint arXiv:2310.01405, 2023

    Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, et al. Representation engineering: A top-down approach to ai transparency.arXiv preprint arXiv:2310.01405, 2023

    arXiv

  2. [10]

    Improving dictionary learning with gated sparse autoencoders.arXiv preprint arXiv:2404.16014, 2024

    Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Tom Lieberum, Vikrant Varma, János Kramár, Ro- hin Shah, and Neel Nanda. Improving dictionary learning with gated sparse autoencoders.arXiv preprint arXiv:2404.16014, 2024

    2024 arXiv

  3. [11]

    Sparse autoencoders find highly interpretable features in language models.arXiv preprint arXiv:2309.08600, 2023

    Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. Sparse autoencoders find highly interpretable features in language models.arXiv preprint arXiv:2309.08600, 2023

    2023 arXiv

  4. [12]

    Towards monosemanticity: Decomposing language models with dictionary learning.Transformer Circuits Thread, 2, 2023

    Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, et al. Towards monosemanticity: Decomposing language models with dictionary learning.Transformer Circuits Thread, 2, 2023

    2023

  5. [13]

    Feature extraction and steering for enhanced chain-of-thought reasoning in language models.arXiv preprint arXiv:2505.15634, 2025

    Zihao Li, Xu Wang, Yuzhe Yang, Ziyu Yao, Haoyi Xiong, and Mengnan Du. Feature extraction and steering for enhanced chain-of-thought reasoning in language models.arXiv preprint arXiv:2505.15634, 2025

    2025 arXiv

  6. [14]

    Locating and editing factual associations in gpt

    Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in gpt. Advances in neural information processing systems, 35:17359–17372, 2022

    2022

  7. [15]

    Scaling and evaluating sparse autoencoders.arXiv preprint arXiv:2406.04093, 2024

    Leo Gao, Tom Dupré la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. Scaling and evaluating sparse autoencoders.arXiv preprint arXiv:2406.04093, 2024. 9

    2024 arXiv

  8. [16]

    Gemma scope: Open sparse autoencoders everywhere all at once on gemma 2.arXiv preprint arXiv:2408.05147, 2024

    Tom Lieberum, Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Nicolas Sonnerat, Vikrant Varma, János Kramár, Anca Dragan, Rohin Shah, and Neel Nanda. Gemma scope: Open sparse autoencoders everywhere all at once on gemma 2.arXiv preprint arXiv:2408.05147, 2024

    2024 arXiv

  9. [17]

    Sparse autoencoder features for classifications and transferability

    Jack Gallifant, Shan Chen, Kuleen Sasse, Hugo Aerts, Thomas Hartvigsen, and Danielle Bitterman. Sparse autoencoder features for classifications and transferability. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 29927–29951, 2025

    2025

  10. [18]

    Saes are good for steering–if you select the right features

    Dana Arad, Aaron Mueller, and Yonatan Belinkov. Saes are good for steering–if you select the right features. arXiv preprint arXiv:2505.20063, 2025

    2025

  11. [19]

    Lingualens: Towards interpreting linguistic mechanisms of large language models via sparse auto-encoder

    Yi Jing, Zijun Yao, Hongzhu Guo, Lingxu Ran, Xiaozhi Wang, Lei Hou, and Juanzi Li. Lingualens: Towards interpreting linguistic mechanisms of large language models via sparse auto-encoder. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,...

    2025

  12. [20]

    Decoding dense embeddings: Sparse autoencoders for interpreting and discretizing dense retrieval.arXiv preprint arXiv:2506.00041, 2025

    Seongwan Park, Taeklim Kim, and Youngjoong Ko. Decoding dense embeddings: Sparse autoencoders for interpreting and discretizing dense retrieval.arXiv preprint arXiv:2506.00041, 2025

    2025 arXiv

  13. [21]

    K-sparse autoencoders.arXiv preprint arXiv:1312.5663, 2013

    Alireza Makhzani and Brendan Frey. K-sparse autoencoders.arXiv preprint arXiv:1312.5663, 2013

    2013 arXiv

  14. [22]

    Deepmath-103k: A large-scale, challenging, decontaminated, and verifiable mathematical dataset for advancing reasoning.arXiv preprint arXiv:2504.11456, 2025

    Zhiwei He, Tian Liang, Jiahao Xu, Qiuzhi Liu, Xingyu Chen, Yue Wang, Linfeng Song, Dian Yu, Zhenwen Liang, Wenxuan Wang, et al. Deepmath-103k: A large-scale, challenging, decontaminated, and verifiable mathematical dataset for advancing reasoning.arXiv preprint arXiv:2504.11456, 2025

    2025 arXiv

  15. [23]

    Reasoning models can be effective without thinking.arXiv preprint arXiv:2504.09858, 2025

    Wenjie Ma, Jingxuan He, Charlie Snell, Tyler Griggs, Sewon Min, and Matei Zaharia. Reasoning models can be effective without thinking.arXiv preprint arXiv:2504.09858, 2025

    2025 arXiv

  16. [24]

    Interpreting and steering llm representations with mutual information-based explanations on sparse autoencoders

    Xuansheng Wu, Jiayi Yuan, Wenlin Yao, Xiaoming Zhai, and Ninghao Liu. Interpreting and steering llm representations with mutual information-based explanations on sparse autoencoders

  17. [25]

    A method for stochastic optimization.arXiv preprint arXiv:1412.6980, 1412(6), 2014

    Kingma DP Ba J Adam et al. A method for stochastic optimization.arXiv preprint arXiv:1412.6980, 1412(6), 2014

    2014 arXiv

  18. [26]

    Scaling monosemanticity: Extracting interpretable features from claude 3 sonnet

    Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, Hoagy Cunningham, Nicholas L Turner, Callum McDougall, Monte MacDiarmid, Alex Tamkin, Eaker E, Sanyam Kapoor, Josh Kaplan, Stanislav...

    2024

  19. [27]

    wait", "hmm

    Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and William B Dolan. A diversity-promoting objective function for neural conversation models. InProceedings of the 2016 conference of the North American chapter of the association for computational linguistics: human langu...

    2016

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.