Pith. sign in

REVIEW 3 major objections 5 minor 8 cited by

A single sparse-autoencoder feature direction can act as a control vector that shifts a reasoning model into a chosen reasoning strategy, beating prompt- and vector-based baselines and correcting more already-wrong answers.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 12:15 UTC pith:BDYRVN4B

load-bearing objection SAE-Steering has a genuinely useful idea and a real error-correction result, but the headline +15% advantage is partly selection-on-the-metric. the 3 major comments →

arxiv 2601.03595 v2 pith:BDYRVN4B submitted 2026-01-07 cs.AI cs.CL

Controllable LLM Reasoning via Sparse Autoencoder-Based Steering

Yi Fang , Wenjie Wang , Mingfeng Xue , Boyi Deng , Fengli Xu , Dayiheng Liu , Fuli Feng This is my paper
classification cs.AI cs.CL
keywords sparse autoencodersreasoning strategy controlactivation steeringfeature identificationlarge reasoning modelslogit lenstest-time interventionerror correction
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper seeks to make the reasoning strategies of large reasoning models—problem understanding, procedural planning, backtracking, multi-perspective verification, and hypothesis reasoning—controllable at fine grain. It claims that sparse autoencoders trained on the residual stream disentangle these strategies into separate feature directions, and that its two-stage search, SAE-Steering, can find one effective direction per strategy: first keep features that amplify strategy-keyword logits, filtering out over 99% of candidates, then rank survivors by how often an LLM judge says a steered continuation shows the target strategy more explicitly than an unsteered one. Steering with the top feature as a control vector is reported to outperform prompt-based and contrastive-vector baselines by more than 15% in control effectiveness. When a trained router picks among strategy features during extended reasoning after a wrong answer, the method corrects about 7% more errors than simply forcing further thinking. If true, this gives a cheap, test-time handle on how a reasoning model thinks.

Core claim

The central claim is that strategy concepts in a reasoning model's hidden states are not hopelessly entangled: a sparse autoencoder can isolate directions that correspond to individual reasoning strategies, and injecting one such direction into the residual stream for a block of tokens reliably pushes subsequent reasoning toward that strategy. The identification method is deliberately two-stage: a logit-contribution readout filters the tens of thousands of SAE features down to a few dozen candidates that amplify strategy-specific keyword logits, and an intervention-based judge then ranks those candidates by how much more explicitly the steered continuation exhibits the target strategy compar

What carries the argument

Sparse-autoencoder feature directions—the decoder columns of an SAE trained on residual-stream activations, each intended to be a monosemantic direction. One such direction serves as the control vector added to a middle-to-late layer. The other load-bearing mechanism is the two-stage identification pipeline: a logit-contribution matrix (decoder weights times the unembedding matrix) recalls features that amplify keyword logits, filtering out over 99% of features; then a validation-set intervention, scored by a judge, ranks the survivors by actual control effectiveness. This pipeline converts a search over tens of thousands of features into a search over a few dozen.

Load-bearing premise

The load-bearing premise is that an LLM judge's binary verdict—that a steered continuation 'more explicitly demonstrates' the target strategy than an unsteered one—measures actual strategy use; the same verdict is used both to select the feature and to report the headline effectiveness, so if the judge mainly keys on surface marker words like 'wait' or 'another approach', the reported gains could be largely cosmetic.

What would settle it

Take the released models and the five strategies, generate matched steered and unsteered continuations, and have independent human annotators mark concrete reasoning moves—backtracking to an earlier step, applying a second method, restating the problem—after stripping the strategy keywords from the texts. If the steered continuations do not contain more such moves than the baselines at a rate comparable to the reported control effectiveness, the central claim is false. Alternatively, run the error-correction comparison against unguided extended thinking with the router's strategy labels hidden

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • One feature direction per strategy is sufficient to shift a reasoning trajectory into the target strategy, at least for the five strategies tested.
  • Strategy control can operate as a test-time intervention without retraining the reasoning model.
  • Features found on math reasoning transfer to out-of-domain science reasoning, suggesting the directions capture general strategy concepts.
  • Logit-based recall is a cheaper and more precise way to find steering-relevant SAE features than activation-strength-based selection.
  • Middle-to-late transformer layers are the right place to inject strategy control; shallow-layer features are scarce and weak.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The authors leave implicit that the same recall-and-rank recipe could be reused for any controllable attribute—tone, length, factuality, cautiousness—wherever a keyword proxy and an intervention judge are available.
  • Because the judge metric is used both to select features and to measure success, an independent behavioral test (e.g., counting actual backtracking moves or verification steps after masking strategy words) would tell whether the control is substantive or partly surface-level.
  • A natural next step the paper does not test is steering at the very start of reasoning, or switching strategies mid-chain based on router confidence, which could turn error correction into a closed-loop controller.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes SAE-Steering, a two-stage method for identifying sparse-autoencoder features that control specific reasoning strategies in large reasoning models. Stage 1 recalls SAE features whose direct logit contribution amplifies strategy-specific keywords, filtering out over 99% of features; Stage 2 ranks the survivors by an LLM-judge-based control-effectiveness score on a validation set. The selected feature directions are then used as steering vectors during decoding. The authors claim that this method outperforms existing prompt- and activation-based baselines by over 15% in control effectiveness on AIME and GPQA, and that a strategy router built from these features improves error-correction accuracy by 7% absolute over Budget Forcing.

Significance. If the claims hold, the paper would be a useful contribution to interpretability and controllable reasoning: it proposes a practical way to search a large SAE feature space, releases code and data, and demonstrates a downstream application (error correction) with an objective accuracy-based metric. The RQ3 error-correction results are the strongest independent evidence, since they do not rely on the same LLM judge used for feature selection. However, the headline control-effectiveness comparison is currently not fully supported because the same judge-based metric is used both for feature selection and for Table 1 evaluation, and all quantitative claims lack uncertainty quantification. The core mechanism is plausible and the paper is internally consistent, but the main comparative claim needs stronger evidence.

major comments (3)
  1. [§3.2, Eq. (7); §4.1; Table 1] The headline control-effectiveness comparison is not independent of the selection criterion. Stage 2 ranks features by Effectiveness(f_j), computed with the LLM judge protocol in Figure 7, and §4.1 states that evaluation 'follow[s] the procedure described in Stage 2.' Thus SAE-Steering is selecting features on the same judge-based metric that Table 1 reports, while the baselines receive no equivalent selection. The reported >15% margin therefore conflates method quality with selection alignment. I request an evaluation with a genuinely independent metric (e.g., human structural annotation or a judge not involved in selection), and/or a baseline that is also selected by the same Stage-2 procedure. The 0.82 human-agreement check on 200 samples is partial mitigation but does not remove the selection/evaluation overlap.
  2. [Tables 1–3; §4.4] All headline numbers are point estimates without variance or significance testing. The sample sizes make this important: in Table 3, AIME25 R1-Llama-8B improves from 0.0123 (2/163) to 0.0552 (9/163); with 163 samples, a bootstrap CI would likely overlap substantially. Similarly, Table 2 precision values are single draws from a candidate pool. The paper should report confidence intervals or paired significance tests (e.g., bootstrap or McNemar) for the 15% control-effectiveness margin and the 7% error-correction improvement. Without this, the 'consistently outperforms' claim is not yet established.
  3. [Figure 7 prompt; §3.2 Stage 1] The Figure 7 judge prompt explicitly tells the model to 'count the number of times the target cognitive reasoning strategy appears,' and Stage 1 selects features that amplify the logits of strategy keywords. This creates a route by which the judge rewards keyword-frequency increases without genuine strategy restructuring. The Logit Boosting example in Figure 3 is reassuring but anecdotal. The paper should provide a quantitative analysis that separates surface keyword matching from structural strategy change, e.g., by measuring performance under an instruction that forbids counting keywords, or by ablating keyword-frequency-matched baselines.
minor comments (5)
  1. [§4.1; Appendix A] The hyperparameters n=2, τ=0.1, top-10 tokens, T=512, and the α search starting at 15 are described but not analyzed for sensitivity. Since τ and the keyword list directly determine the candidate pool, a small sensitivity study would strengthen the robustness claims.
  2. [§4.3, Figures 4–5] The layer-wise analysis reports average control effectiveness of the top-3 features without error bars or per-layer variance. Also, the analysis is only on Qwen3-8B; the authors should state whether the same pattern is expected for R1-Llama-8B.
  3. [Table 1; §4.2] The observation that GPQA is easier because reasoning length is shorter ('7k vs. 15k tokens') is not supported by any quantitative correlation. Consider reporting per-sample reasoning lengths or a stratified analysis.
  4. [Appendix C] Human agreement is assessed on 200 samples (40 per strategy) with Krippendorff's alpha 0.78. This is acceptable but not high; reporting per-strategy agreement with confidence intervals, and ideally a larger sample, would increase confidence in the judge-based evaluation.
  5. [§4.1 baselines] The description of the baselines is brief. In particular, it is unclear whether Vector Steering's steering strength and injection layer were tuned with the same procedure as SAE-Steering, and where Think Intervention inserts its instructions. This is needed for a fair comparison.

Circularity Check

1 steps flagged

Table 1's 15% control-effectiveness advantage is the same LLM-judge construct Stage 2 optimizes (Eq. 7): RQ1 partially reports the fitted objective; RQ3's accuracy gains are independent.

specific steps
  1. fitted input called prediction [Section 3.2, Eq. (7); Section 4.1 'Evaluation Protocol'; Section 4.2, Table 1]
    "The control effectiveness of a feature fj is then calculated as the control success rate over the validation set: Effectiveness(fj) = 1/|P| Σ_{p∈P} Jp,j. (7) ... We evaluate control effectiveness following the procedure described in Stage 2 of Section 3.2."

    Stage 2 ranks candidate features by Effectiveness(f_j), an LLM-judge binary verdict on whether the steered output 'more explicitly demonstrates the target strategy', and Section 4.1 reuses 'the procedure described in Stage 2' as the evaluation protocol. Table 1's RQ1 claim (SAE-Steering outperforms Vector Steering by >15%) therefore reports the same construct the method optimizes, while the baselines receive no analogous metric-tuned selection. Part of the reported margin is selection alignment: the fitted quantity is reported as the headline prediction. The circularity is partial because the test data (AIME24/25, GPQA) and the three-judge majority vote differ from the validation set and the single GPT-4o selection judge.

full rationale

One genuine, partial circularity sits at the center of the paper: the RQ1 headline. Stage 2 (Section 3.2) selects strategy-specific features by maximizing Effectiveness(f_j) (Eq. 7) — an LLM judge's binary verdict, via the Figure 7 prompt, on whether the steered continuation 'more explicitly demonstrates the target strategy' than the unsteered baseline. Section 4.1 explicitly evaluates control effectiveness 'following the procedure described in Stage 2 of Section 3.2,' and Section 4.2 (and the Abstract) convert that same judge protocol into the claim that SAE-Steering 'outperforms existing methods by over 15% in control effectiveness' (Table 1). Thus the quantity being predicted is the quantity the pipeline was fitted to maximize; the compared baselines (Logit Boosting, Think Intervention, Vector Steering) are not selected or tuned against this metric, so part of the reported margin is objective alignment rather than intrinsic control quality. The identity is not literal: selection uses a validation set judged by GPT-4o alone, while the test report uses a distinct judge ensemble (GPT-4o, Gemini-2.5-flash, Deepseek-V3.2) on held-out problems, so this is construct-level rather than equation-level circularity. The related surface-marker concern — Stage 1 recalls features whose top-10 logit tokens are strategy keywords, and the Figure 7 judge is told to 'count the number of times the target cognitive reasoning strategy appears' — is partially refuted by the paper's own control: Logit Boosting, a pure keyword booster, scores far below SAE-Steering (e.g., 0.21 vs 0.88 for PU on AIME), showing the judge is not merely counting keywords. Independent content remains: RQ3 error correction is measured by final-answer accuracy (e.g., AIME25 R1-Llama-8B: 0.0552 vs 0.0123 for Budget Forcing), a metric never used in feature selection, and Table 2's precision comparison against ReasonScore uses a different recall criterion. Self-citations are not load-bearing: the one overlapping-author citation (Deng et al. 2025, SAE interpretability) is contextual. Score 5: the central RQ1 measure is the fitted objective (partial circularity), while RQ3 and the overall method retain independent, externally measurable content.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

No new physical/causal entities are postulated; 'strategy-specific features' are existing SAE dictionary directions that are labeled and selected, and the strategy router is a trained module. The paper's contribution rests instead on free parameters (alpha, recall thresholds, keyword lists, T) and four domain assumptions about SAE monosemanticity, the logit-lens proxy, the LLM-judge metric, and the manual strategy corpora.

free parameters (4)
  • Steering strength alpha = per-feature; searched downward from alpha=15 until repetition disappears, averaged over validation samples
    Appendix A: chosen on the 50-response validation set; directly determines how strongly the strategy is imposed and the reported effectiveness.
  • Stage 1 recall thresholds (n, tau, top-10 tokens) = n=2, tau=0.1, top-10 tokens per feature
    Section 4.1: hand-set; they determine the candidate pool size (143/357 features) and therefore which features reach the effectiveness ranking.
  • Strategy keyword lists = 5 keywords per strategy (Table 4)
    Appendix B: derived from manually segmented corpora (top-20 frequency + manual curation); Stage 1 recall and the keyword proxy are defined entirely by these lists.
  • Continuation length T (Stage 2) = 512 tokens
    Section 4.1: length of the steered continuation judged for control effectiveness; affects the metric.
axioms (4)
  • domain assumption SAE features learned on LMSYS-CHAT-1M + OPENTHOUGHTS activations (TopK, K=50, last layer) decompose strategy-entangled hidden states into monosemantic, linearly actionable directions
    Section 3.1: the method assumes sparsity-trained features are disentangled enough that adding one feature direction reliably induces a reasoning strategy; this borrows the monosemanticity thesis of Bricken et al./Huben et al.
  • domain assumption The LM-head logit lens L=W_dec^T U (Eq. 6) is a valid proxy for a feature's causal steering capacity
    Eq. 6 ignores the final layer norm and any residual computation between the SAE layer and the head; Stage 1's recall (the 99% filter) rests entirely on this linear proxy.
  • domain assumption An LLM judge's verdict that a steered continuation 'more explicitly demonstrates the target strategy' is a valid operationalization of reasoning-strategy control
    Figure 7/Section 4.1: this is both the selection objective (Stage 2) and the reported RQ1/RQ2 metric; only 200 samples were human-checked (0.82 agreement), and the prompt asks the judge to count strategy occurrences, so surface usage may satisfy it.
  • domain assumption Manual segmentation of LRM responses into the five strategy corpora is accurate and the five strategies are separable
    Appendix B: keyword lists come from manually identified reasoning segments; overlapping/ambiguous keywords (e.g., 'approach', 'if', 'let') suggest the strategies are not cleanly separable, which feeds both recall and the judge metric.

pith-pipeline@v1.3.0-alltime-deepseek · 4299 in / 4239 out tokens · 241450 ms · 2026-08-03T12:15:29.775690+00:00 · methodology

0 comments
read the original abstract

Large Reasoning Models (LRMs) exhibit human-like cognitive reasoning strategies (\eg backtracking, cross-verification) during the reasoning process, which improves their performance on complex tasks. Currently, reasoning strategies are autonomously selected by LRMs themselves. However, such autonomous selection often produces inefficient or even erroneous reasoning paths. To make reasoning more reliable and flexible, it is important to develop methods for controlling reasoning strategies. Existing methods struggle to control fine-grained reasoning strategies due to conceptual entanglement in LRMs' hidden states. To address this, we leverage Sparse Autoencoders (SAEs) to decompose strategy-entangled hidden states into a disentangled feature space. To identify the few strategy-specific features from the vast pool of SAE features, we propose SAE-Steering, an efficient two-stage feature identification pipeline. SAE-Steering first recalls features that amplify the logits of strategy-specific keywords, filtering out over 99\% of features, and then ranks the remaining features by their control effectiveness. Using the identified strategy-specific features as control vectors, SAE-Steering outperforms existing methods by over 15\% in control effectiveness. Furthermore, controlling reasoning strategies can redirect LRMs from erroneous paths to correct ones, achieving a 7\% absolute accuracy improvement. Our code and data are available at https://github.com/Peter-Fy/SAE-Steering.

Figures

Figures reproduced from arXiv: 2601.03595 by Boyi Deng, Dayiheng Liu, Fengli Xu, Fuli Feng, Mingfeng Xue, Wenjie Wang, Yi Fang.

Figure 1
Figure 1. Figure 1: An illustration of reasoning strategy control. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: (a) Overview of the SAE architecture. (b) Feature identification pipeline of SAE-Steering. Numbers below [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Case study: SAE-Steering changes reasoning [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 5
Figure 5. Figure 5: Control effectiveness across layers. sert a “wait” token at the end of the initial, flawed reasoning to induce further thinking. During this extended reasoning phase, we apply SAE-Steering to control the LRM’s subsequent reasoning strat￾egy. To select the most appropriate strategy for different problems, we train a strategy router (Ap￾pendix E). We compare our approach with two common self-correction basel… view at source ↗
Figure 6
Figure 6. Figure 6: A case study of SAE-based steering. By steering with different strategy-specific features as control vectors, [PITH_FULL_IMAGE:figures/full_fig_p013_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: The prompt used to evaluate the control effectiveness. [PITH_FULL_IMAGE:figures/full_fig_p014_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 8 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Do Unified Multimodal Models Think in One Space? A Lens Through Cross-Branch Steering

    cs.CV 2026-07 conditional novelty 7.0

    Steering vectors from the understanding branch can control image generation, but vectors from the generation branch cannot control understanding, showing UMMs are architecturally unified but representationally asymmetric.

  2. The Tell-Tale Norm: $\ell_2$ Magnitude as a Signal for Reasoning Dynamics in Large Language Models

    cs.CL 2026-06 unverdicted novelty 6.0

    The L2 norm of LLM hidden states signals reasoning intensity, with a theoretical bound on SAE feature activations, enabling three new test-time scaling techniques that boost performance.

  3. Latent Reward Steering: An Adaptive Inference-Time Framework that Implicitly Promotes Cognitive Behaviors in Reasoning LLMs

    cs.AI 2026-05 conditional novelty 6.0

    Reward-guided optimization of SAE latents, gated by reward and confidence, improves multi-benchmark reasoning and is post-hoc associated with more verification and course correction.

  4. Latent Reward Steering: An Adaptive Inference-Time Framework that Implicitly Promotes Cognitive Behaviors in Reasoning LLMs

    cs.AI 2026-05 unverdicted novelty 6.0

    LRS trains a latent reward model on final-answer correctness to steer SAE states during inference, improving reasoning performance and implicitly encouraging better cognitive behaviors.

  5. Transcoders Trace Visual Grounding and Hallucinations in Vision-Language Models

    cs.LG 2026-05 unverdicted novelty 6.0

    Transcoders decompose MLP layers in Gemma 3-4B-IT to trace visual grounding more effectively than SAEs and predict hallucinations from circuit graph features at AUC 0.68.

  6. Steered Generation via Gradient-Based Optimization on Sparse Query Features

    cs.LG 2026-05 unverdicted novelty 5.0

    Prototype-Based Sparse Steering decomposes query activations with SAEs and optimizes sparse features via gradients to steer LLM outputs toward specific behaviors.

  7. Qwen-Scope: Turning Sparse Features into Development Tools for Large Language Models

    cs.CL 2026-05 unverdicted novelty 4.0

    Qwen-Scope provides open-source sparse autoencoders for Qwen models that function as practical interfaces for steering, evaluating, data workflows, and optimizing large language models.

  8. Discovering Millions of Interpretable Features with Sparse Autoencoders

    cs.LG 2026-06 unverdicted novelty 3.0

    Trains and releases SAEs for Qwen3-1.7B/4B/8B models with layer-wise coverage and demonstrates causal steering of refusal via selected features.

Reference graph

Works this paper leans on

48 extracted references · 18 linked inside Pith · cited by 7 Pith papers

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    AIME. 2025. https://artofproblemsolving.com/wiki/index.php/AIME_Problems_and_Solutions. Aime problems and solutions

    2025

  4. [4]

    Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Hatfield-Dodds, Alex Tamkin, Karina Nguyen, and 6 others. 2023. Towards monosemanticity: Decomposing language models with d...

    2023

  5. [5]

    Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, Rui Wang, Zhaopeng Tu, Haitao Mi, and Dong Yu. 2025. Do NOT think that much for 2+3=? on the overthinking of long reasoning models. In ICML . OpenReview.net

    2025

  6. [6]

    Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, and 1 others. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.06261

    Pith/arXiv arXiv 2025

  7. [7]

    Boyi Deng, Yu Wan, Baosong Yang, Yidan Zhang, and Fuli Feng. 2025. Unveiling language-specific features in large language models via sparse autoencoders. In ACL (1) , pages 4563--4608. Association for Computational Linguistics

    2025

  8. [8]

    Grosse, Sam McCandlish, Jared Kaplan, Dario Amodei, Martin Wattenberg, and Christopher Olah

    Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield - Dodds, Robert Lasenby, Dawn Drain, Carol Chen, Roger B. Grosse, Sam McCandlish, Jared Kaplan, Dario Amodei, Martin Wattenberg, and Christopher Olah. 2022. Toy models of superposition. arXiv preprint arXiv:2209.10652

    Pith/arXiv arXiv 2022

  9. [9]

    Zihao Fu, Wai Lam, Anthony Man-Cho So, and Bei Shi. 2021. A theoretical analysis of the repetition problem in text generation. In AAAI

    2021

  10. [10]

    Galichin, Alexey Dontsov, Polina Druzhinina, Anton Razzhigaev, Oleg Y

    Andrey V. Galichin, Alexey Dontsov, Polina Druzhinina, Anton Razzhigaev, Oleg Y. Rogov, Elena Tutubalina, and Ivan V. Oseledets. 2025. I have covered all the bases here: Interpreting reasoning features in large language models via sparse autoencoders. arXiv preprint arXiv:2503.18878

    Pith/arXiv arXiv 2025

  11. [11]

    Kanishk Gandhi, Ayush Chakravarthy, Anikait Singh, Nathan Lile, and Noah D. Goodman. 2025. Cognitive behaviors that enable self-improving reasoners, or, four habits of highly effective stars. Second Conference on Language Modeling

    2025

  12. [12]

    Sandeep Reddy Gantla. 2025. Exploring mechanistic interpretability in large language models: Challenges, approaches, and insights. In 2025 International Conference on Data Science, Agents & Artificial Intelligence (ICDSAAI)

    2025

  13. [13]

    Leo Gao, Tom Dupr \' e la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. 2025. Scaling and evaluating sparse autoencoders. In ICLR

    2025

  14. [14]

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948

    Pith/arXiv arXiv 2025

  15. [15]

    Robert Huben, Hoagy Cunningham, Logan Riggs Smith, Aidan Ewart, and Lee Sharkey. 2024. Sparse autoencoders find highly interpretable features in language models. In ICLR

    2024

  16. [16]

    Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen - tau Yih. 2020. Dense passage retrieval for open-domain question answering. In EMNLP (1) . Association for Computational Linguistics

    2020

  17. [17]

    Jia LI, Edward Beeching, Lewis Tunstall, Ben Lipkin, Roman Soletskyi, Shengyi Costa Huang, Kashif Rasul, Longhui Yu, Albert Jiang, Ziju Shen, Zihan Qin, Bin Dong, Li Zhou, Yann Fleureau, Guillaume Lample, and Stanislas Polu. 2024. https://huggingface.co/AI-MO/NuminaMath-1.5 Numinamath

    2024

  18. [18]

    Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. Let's verify step by step. arXiv preprint arXiv:2305.20050

    Pith/arXiv arXiv 2023

  19. [19]

    Zhengkai Lin, Zhihang Fu, Ze Chen, Chao Chen, Liang Xie, Wenxiao Wang, Deng Cai, Zheng Wang, and Jieping Ye. 2025. Controlling thinking speed in reasoning models. NeurIPS

    2025

  20. [20]

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, and 1 others. 2024. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437

    Pith/arXiv arXiv 2024

  21. [21]

    Sara Vera Marjanović, Arkil Patel, Vaibhav Adlakha, Milad Aghajohari, Parishad BehnamGhader, Mehar Bhatia, Aditi Khandelwal, Austin Kraft, Benno Krojer, Xing Han Lù, Nicholas Meade, Dongchan Shin, Amirhossein Kazemnejad, Gaurav Kamath, Marius Mosbach, Karolina Stańczak, and Siva Reddy. 2025. Deepseek-r1 thoughtology: Let's think about llm reasoning. arXiv...

    arXiv 2025

  22. [22]

    Cand \` e s, and Tatsunori Hashimoto

    Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei - Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel J. Cand \` e s, and Tatsunori Hashimoto. 2025. s1: Simple test-time scaling. arXiv preprint arXiv:2501.19393

    Pith/arXiv arXiv 2025

  23. [23]

    nostalgebraist. 2020. Interpreting gpt: The logit lens. Less-Wrong (blog)

    2020

  24. [24]

    OpenAI. 2024. https://arxiv.org/abs/2410.21276 Gpt-4o system card . Preprint, arXiv:2410.21276

    Pith/arXiv arXiv 2024

  25. [25]

    OpenAI. 2025. https://openai.com/index/openai-o1-system-card/ Openai o1 system card . Accessed: 2025-02-21

    2025

  26. [26]

    Qianjun Pan, Wenkai Ji, Yuyang Ding, Junsong Li, Shilian Chen, Junyi Wang, Jie Zhou, Qin Chen, Min Zhang, Yulan Wu, and Liang He. 2025. A survey of slow thinking-based reasoning llms using reinforced learning and inference-time scaling law. arXiv preprint arXiv:2505.02665

    Pith/arXiv arXiv 2025

  27. [27]

    Yunjia Qi, Hao Peng, Xiaozhi Wang, Amy Xin, Youfeng Liu, Bin Xu, Lei Hou, and Juanzi Li. 2025. AGENTIF: benchmarking instruction following of large language models in agentic scenarios. CoRR, abs/2505.16944

    Pith/arXiv arXiv 2025

  28. [28]

    David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. 2023. GPQA: A graduate-level google-proof q & a benchmark. CoRR, abs/2311.12022

    Pith/arXiv arXiv 2023

  29. [29]

    Leheng Sheng, An Zhang, Zijian Wu, Weixiang Zhao, Changshuo Shen, Yi Zhang, Xiang Wang, and Tat - Seng Chua. 2025. On reasoning strength planning in large reasoning models. NeurIPS

    2025

  30. [30]

    Wei Shi, Sihang Li, Tao Liang, Mingyang Wan, Gojun Ma, Xiang Wang, and Xiangnan He. 2025. Route sparse autoencoder to interpret large language models. CoRR, abs/2503.08200

    Pith/arXiv arXiv 2025

  31. [31]

    Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: language agents with verbal reinforcement learning. In NeurIPS

    2023

  32. [32]

    Chandan Singh, Jeevana Priya Inala, Michel Galley, Rich Caruana, and Jianfeng Gao. 2024. Rethinking interpretability in the era of large language models. arXiv preprint arXiv:2402.01761

    Pith/arXiv arXiv 2024

  33. [33]

    Charlie Victor Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. 2025. Scaling LLM test-time compute optimally can be more effective than scaling parameters for reasoning. In ICLR . OpenReview.net

    2025

  34. [34]

    Xinyu Tang, Xiaolei Wang, Zhihao Lv, Yingqian Min, Xin Zhao, Binbin Hu, Ziqi Liu, and Zhiqiang Zhang. 2025. Unlocking general long chain-of-thought reasoning capabilities of large language models via representation engineering. In ACL (1) , pages 6832--6849. Association for Computational Linguistics

    2025

  35. [35]

    Openthoughts Team. 2025 a . Openthoughts: Data recipes for reasoning models

    2025

  36. [36]

    Qwen Team. 2025 b . Qwen3 technical report

    2025

  37. [37]

    A \" a ron van den Oord, Yazhe Li, and Oriol Vinyals. 2018. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748

    Pith/arXiv arXiv 2018

  38. [38]

    Constantin Venhoff, Iv \' a n Arcuschin, Philip Torr, Arthur Conmy, and Neel Nanda. 2025. Understanding reasoning in thinking language models via steering vectors. CoRR, abs/2506.18167

    arXiv 2025

  39. [39]

    Yue Wang, Qiuzhi Liu, Jiahao Xu, Tian Liang, Xingyu Chen, Zhiwei He, Linfeng Song, Dian Yu, Juntao Li, Zhuosheng Zhang, Rui Wang, Zhaopeng Tu, Haitao Mi, and Dong Yu. 2025. Thoughts are all over the place: On the underthinking of o1-like llms. arXiv preprint arXiv:2501.18585

    Pith/arXiv arXiv 2025

  40. [40]

    Wang, and Prateek Mittal

    Tong Wu, Chong Xiang, Jiachen T. Wang, and Prateek Mittal. 2025. Effectively controlling reasoning models through thinking intervention. arXiv preprint arXiv:2503.24370

    Pith/arXiv arXiv 2025

  41. [41]

    Chenxu Yang, Qingyi Si, Mz Dai, Dingyu Yao, Mingyu Zheng, Minghui Chen, Zheng Lin, and Weiping Wang. 2025 a . Test-time prompt intervention

    2025

  42. [42]

    Jingyuan Yang, Rongjun Li, Weixuan Wang, Ziyu Zhou, Zhiyong Feng, and Wei Peng. 2025 b . Lf-steering: Latent feature activation steering for enhancing semantic consistency in large language models. arXiv preprint arXiv:2501.11036

    Pith/arXiv arXiv 2025

  43. [43]

    Olshausen, and Yann LeCun

    Zeyu Yun, Yubei Chen, Bruno A. Olshausen, and Yann LeCun. 2021. Transformer visualization via dictionary learning: contextualized embedding as a linear superposition of transformer factors. In DeeLIO@NAACL-HLT, pages 1--10. Association for Computational Linguistics

    2021

  44. [44]

    Wojciech Zaremba, Evgenia Nitishinskaya, Boaz Barak, Stephanie Lin, Sam Toyer, Yaodong Yu, Rachel Dias, Eric Wallace, Kai Xiao, Johannes Heidecke, and 1 others. 2025. Trading inference-time compute for adversarial robustness. arXiv preprint arXiv:2501.18841

    Pith/arXiv arXiv 2025

  45. [45]

    Xingsheng Zhang, Luxi Xing, Chen Zhang, Yanbing Liu, Yifan Deng, Yunpeng Li, Yue Hu, and Chenxu Niu. 2025. Can we steer reasoning direction by thinking intervention? In Findings of EMNLP 2025

    2025

  46. [46]

    Xing, Joseph E

    Lianmin Zheng, Wei - Lin Chiang, Ying Sheng, Tianle Li, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zhuohan Li, Zi Lin, Eric P. Xing, Joseph E. Gonzalez, Ion Stoica, and Hao Zhang. 2024. Lmsys-chat-1m: A large-scale real-world LLM conversation dataset. In ICLR . OpenReview.net

    2024

  47. [47]

    Qihuang Zhong, Kang Wang, Ziyang Xu, Juhua Liu, Liang Ding, and Bo Du. 2024. Achieving> 97\ Frontiers of Computer Science

    2024

  48. [48]

    Le, Ed H

    Pei Zhou, Jay Pujara, Xiang Ren, Xinyun Chen, Heng - Tze Cheng, Quoc V. Le, Ed H. Chi, Denny Zhou, Swaroop Mishra, and Huaixiu Steven Zheng. 2024. SELF-DISCOVER: large language models self-compose reasoning structures. In NeurIPS

    2024