Pith. sign in

REVIEW 3 major objections 6 minor 81 references

Reasoning fine-tuning reorganizes how language models structure computation over time, into discrete latent policy states that can be measured, transplanted, and pruned.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 15:09 UTC pith:C6RBPSOP

load-bearing objection A credible empirical study with a real model-order confound in the headline metric and an internal contradiction in the CEBRA claim; still worth refereeing, with specific fixes. the 3 major comments →

arxiv 2607.18532 v1 pith:C6RBPSOP submitted 2026-07-20 cs.CL

Reasoning Fine-Tuning Induces Persistent Latent Policy States

Abir Harrasse , Michael Lan , Hunar Batra , Fateme Hashemi Chaleshtori , Chaithanya Bandi This is my paper
classification cs.CL
keywords chain-of-thought reasoninglatent policy statesswitching dynamical systemsreasoning fine-tuningactivation trajectoriesmechanistic interpretabilitymodel interventionsreasoning prefix pruning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper aims to show that reasoning fine-tuning does not merely improve token-level prediction but globally reorganizes how a language model structures computation over time, into discrete, persistent, switchable latent policy states. It models chain-of-thought activation trajectories as a switching dynamical system with a small number of hidden regimes, learned from sentence-level last-token representations. Across four benchmarks and model sizes from 1.5B to 32B parameters, reasoning-fine-tuned models display more differentiated transition structure than base models, with model-dependent changes in state usage, persistence, and mixing. The regimes align with reasoning stages, and causal tests show the recovered dynamics matter: swapping states hurts prediction, transplanting reasoning dynamics into base models improves accuracy on hard problems, and pruning trajectories that dwell in failure-associated states beats self-consistency in 11 of 12 settings. A sympathetic reader would care because the paper offers a measurable, actionable organizational account of why reasoning fine-tuning works rather than treating it as a black-box performance gain.

Core claim

The central claim is that reasoning fine-tuning globally reorganizes latent dynamics: chain-of-thought reasoning unfolds through a small number of discrete, persistent, switchable policy states, and fine-tuning makes these states more differentiated in transition structure without uniformly increasing persistence. Using a switching dynamical system fitted to sentence-level activation trajectories, the paper reports that across four benchmarks and model scales from 1.5B to 32B parameters, reasoning models consistently show higher transition differentiation than base models, while state utilization, persistence, and mixing change in model-dependent ways. The regimes are functionally specialize

What carries the argument

The load-bearing object is a linear-Gaussian switching dynamical system (SDS): a discrete Markov chain of latent regimes, each with its own affine dynamics z(t+1) = A_k z(t) + b_k + noise, governing the evolution of a low-dimensional embedding of the model's residual stream. The embeddings come from a time-aware contrastive encoder trained with temporal adjacency, and the system is fit with expectation-maximization and forward-backward smoothing, with regime count selected by Bayesian information criterion. The paper measures organizational differences through transition differentiation (mean total-variation distance from uniform switching), persistence (self-transition probability), effecti

Load-bearing premise

The load-bearing premise is that sentence-level, last-token activation trajectories, fit as a first-order linear-Gaussian switching system with BIC-selected regime counts, produce regime sequences whose base-versus-reasoning differences reflect true latent policy organization rather than artifacts of model order or temporal resolution; the paper itself concedes that persistence is not identifiable once the selected number of regimes exceeds the true order, which is the regime

What would settle it

Fit the same SDS pipeline to a base model and its reasoning-fine-tuned counterpart at every candidate K, and compare the transition-structure and persistence gaps at matched K values across all model-dataset pairs. If the gaps vanish or reverse when K is held equal, or when the pipeline is applied to two random seeds of the same model with no fine-tuning difference, the claim that reasoning fine-tuning reorganizes latent dynamics collapses.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the central claim is correct, reasoning gains are explainable and measurable as latent policy organization, not just improved token competence.
  • The recovered regimes are functionally meaningful: misapplying them degrades predictive fit, while transplanting them into base models improves performance on hard problems.
  • Trajectory-level structure can be exploited at inference: SDS-guided pruning of failure-prone prefixes outperforms self-consistency in 11 of 12 settings.
  • The theory predicts that trajectory-level rewards induce persistent states by making continuation reward-relevant, offering a mechanism connecting fine-tuning to latent dynamics.
  • The framework provides a new lens for process-level control of reasoning models, beyond output-level interventions.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A fairer comparison would hold the state budget fixed between base and reasoning models across the full K sweep, isolating whether reorganization is about richer state repertoires or more differentiated transitions among the same number of states.
  • Because transplantation improves a base model without weight changes, a natural extension is to use the recovered policy as a lightweight inference-time controller for models that have not been reasoning-fine-tuned.
  • The framework's temporal resolution is one sentence; re-running the SDS recovery at token or thought level could reveal whether persistent states survive finer granularity or are partly an artifact of sentence-level aggregation.
  • If the trajectory-level reward explanation is right, other reward-based post-training schemes should also induce persistent latent states, a prediction testable by running the same analysis on instruction-tuned or RLHF'd checkpoints.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes a switching dynamical system (SDS) framework, with CEBRA-based dimensionality reduction and EM estimation, to infer discrete latent policy states from sentence-level activation trajectories of base and reasoning-fine-tuned LLMs. Across four model pairs and four benchmarks, it reports that reasoning models exhibit 'richer latent-policy organization,' most consistently expressed as higher TVD of the transition matrix, along with model-dependent changes in effective state usage, persistence, and mixing. Controls include time shuffling, randomized CEBRA pairs, Dirichlet-prior sensitivity, paired-correct traces, and cross-dataset transfer. Causal claims are supported by state-swap ablations, cross-model predictive transfer, and a steering intervention that transplants reasoning SDS dynamics into base models. A practical application, PREFIXGUARD, prunes failure-prone prefixes and outperforms self-consistency in 11/12 settings.

Significance. If correct, the paper offers a novel and actionable description of how reasoning fine-tuning reorganizes internal dynamics, with a publicly available codebase and a rich set of controls. The cross-model transfer asymmetry and the prefix-pruning results are particularly interesting. However, the primary evidence for the headline claim—higher TVD—is entangled with the model-order selection procedure, and the causal steering experiment lacks a key control. With additional fixed-order analyses and control interventions, the paper could substantially strengthen its conclusions.

major comments (3)
  1. [3.4, Appendix A (Eq. 14), Table 1, Figure 4] The headline TVD result is computed at BIC-selected K (Eq. 14), and Figure 4 shows reasoning models systematically select larger K than base models. Because TVD measures distance of each transition row to a K-dimensional uniform distribution, the metric is not comparable across models with different K. The fixed-K=5 control in Appendix E.2.3 (Table 6) uses randomized non-adjacent CEBRA positive pairs rather than the default adjacent-positive encoder, so it does not directly rule out a model-order confound for the default pipeline. Please recompute Tables 1 and 3 at a fixed K (e.g., K=5) using the default CEBRA objective, and report whether the TVD separation persists; also report the distribution of BIC-selected K across trajectories.
  2. [Appendix C.1, Remark C.5] Theorem C.4/Remark C.5 prove that for K>K⋆ there exist observationally equivalent solutions with strictly lower persistence and different transition structure. Since the pipeline selects K per trajectory and aggregates metrics across trajectories, it is unclear how many trajectories are in the over-specified regime where persistence/TVD are not identified. Please provide a sensitivity analysis—e.g., a simulation study showing BIC recovery of K⋆ and the effect on TVD under model-order mis-specification, plus an empirical breakdown of selected K per model/dataset.
  3. [§8.1, Appendix I.2 (Table 10)] The transplantation experiment reports that steering the base model with reasoning-SDS dynamics improves pass@8, but it lacks a control condition. Without a control using the base model's own SDS dynamics or a shuffled/random transition matrix, the improvement cannot be attributed specifically to the recovered reasoning dynamics. Add such a control to support the causal interpretation.
minor comments (6)
  1. [Abstract and §4] The term 'richer latent-policy organization' is operationalized through several metrics whose directions differ across model families; consider restating the central claim to emphasize TVD, which is the only consistent metric, and clearly separate the model-dependent findings.
  2. [Figure 2] The CK-residual plot lacks a quantitative test; please report the fraction of runs below the diagonal and a permutation test p-value.
  3. [Appendix E.2.3, Table 6] Report standard deviations over the three seeds for the randomized-pair results.
  4. [Section 8.2] The fixed scoring rule for PREFIXGUARD is not fully specified; provide the exact formula or a precise reference to the appendix.
  5. [Section 5.2] State-swap ablations are reported for two depth groups, but the main text says 'across all model families and datasets'; consider adding a summary table or pointing explicitly to Figures 16–17.
  6. [Running header] The running header on page 2 reads 'Reasoning Fine-T uning'; please correct.

Circularity Check

0 steps flagged

No significant circularity: the central claim is not forced by construction or by self-citation.

full rationale

The paper’s headline comparison (Table 1) is a fitted descriptive metric, not a prediction of a fitted parameter. The TVD/persistence gap is computed from EM-fitted transition matrices on CEBRA embeddings; the only prior touching the transition matrix (Eq. 13, kappa=1) smooths rows toward uniform rather than inflating self-transitions, and Appendix E.2.4 reports that removing it leaves the base/reasoning separation unchanged. The main confound raised by the skeptic—that BIC-selected K differs between base and reasoning models and TVD’s uniform reference depends on K—is a comparability caveat, not a circular reduction: Table 6 provides a fixed-K=5 control in which reasoning models still exceed base models in pstay and TVD across all 12 settings. Identifiability is imported from external results (Douc et al. 2004; Yakowitz & Spragins 1968), and the paper explicitly disclaims that its idealized assumptions are imposed on the empirical pipeline. No load-bearing step cites prior work by the same authors. The out-of-sample handles (state-swap, cross-model transfer, transplantation on pass@8=0 problems, PREFIXGUARD gains) are independent of the fitting procedure and give the central claim external content. Remaining concerns (per-trajectory BIC aggregation, missing pstay rows in Table 7, in-sample state-swap sanity check) are robustness issues, not by-construction circularity.

Axiom & Free-Parameter Ledger

8 free parameters · 6 axioms · 1 invented entities

The framework contributes an SDS+CEBRA pipeline, but nearly all quantities that determine the headline metrics are fit or hand-chosen: BIC-selected K (which differs between base and reasoning models and interacts with the non-identifiability of persistence, Remark C.5), a persistence prior aimed at pstay (with pstay missing from the sensitivity table), hand-picked layers chosen partly for high TVD (the headline metric), and an unspecified PREFIXGUARD scoring rule. The theory claims identifiability and MLE consistency only under assumptions the authors explicitly say are not imposed by the empirical pipeline. Independent support comes from the temporal-order controls (time-shuffle, randomized CEBRA pairs, fixed-K analysis) and from the causal interventions.

free parameters (8)
  • Latent regime count K (BIC-selected per trajectory) = varies; reasoning models select larger K than base (Fig. 4)
    Model order chosen per trajectory by in-sample BIC (§3.4, §5.1). Since persistence is non-identifiable for K above the true order (Remark C.5), and base vs reasoning models land at different K, the headline TVD/persistence differences are partly co-determined by model-order choice; the fixed-K=5 control (Appendix E.2.3) is the main mitigation.
  • Dirichlet persistence prior κ (default 1) = 1
    Added to transition counts in the EM M-step (Eq. 13) to encourage self-transition — the exact quantity the 'persistence' headline reports. Sensitivity is reported for Keff, TVD, spectral gap, but pstay itself is omitted from Table 7.
  • CEBRA embedding dimension and temperature τ = not stated in text
    Hyperparameters of the InfoNCE objective (Eq. 4); values are not reported, hindering replication.
  • EM ridge regularization λ = 1e-4
    Hand-set in Eqs. 11–12.
  • Layer selection per model = L22/L28/L20 middle; L31/L47/L27 final (used in main tables)
    Layers chosen for stable bands where 'TVD and Keff are high, spectral gap is low' (Appendix E.3); the headline metric (TVD) is one of the selection criteria, a mild selection-bias risk.
  • State utility V(s) fitted from correctness outcomes = estimated from trajectory correctness
    Feeds the detour score (Eq. 5) and PREFIXGUARD; fitted on the same benchmark data used for evaluation.
  • Transplantation steering hyperparameters = α=8, β=8, τ=20 tokens, T=0.7, 8 samples
    Hand-set in Appendix H; no sensitivity analysis reported.
  • PREFIXGUARD scoring weights = unspecified
    The scoring rule is described only as 'a fixed combination of CEBRA-based reasoning quality, SDS transition value, transition likelihood, and a penalty for time spent in detour states' (§8.2); without weights the controller cannot be replicated.
axioms (6)
  • domain assumption Sentence-level last-token activations at a single layer are a sufficient statistic for the reasoning state
    §3.1: h_t is the final-token representation of each punctuation-split sentence. If the true regime operates at token or multi-sentence granularity, the recovered regimes alias it. The paper asserts token-level is 'too fine-grained' but provides no external validation of sentence granularity.
  • domain assumption Switching dynamical system with first-order Markov latent states and linear-Gaussian emissions approximates LLM activation trajectories
    §3.2 Eq. (3). Tested only indirectly via Chapman–Kolmogorov residuals and order-2 BIC on the decoded regime sequence (Fig. 2), not on raw activations.
  • ad hoc to paper Identifiability/MLE-consistency assumptions (A1)–(A4): ergodicity, distinct emissions, compactness, uniform filter forgetting
    Appendix C. The paper's own preamble states these are 'stronger, idealized assumptions ... not imposed by the empirical pipeline.' The regime-recovery guarantees therefore do not formally cover the empirical claims.
  • standard math Gaussian-mixture identifiability (Yakowitz & Spragins 1968) and MLE consistency (Douc et al. 2004)
    Invoked in Proposition C.1(i)–(ii) for regime-parameter recovery.
  • ad hoc to paper Under RLVR, the continuation margin γi(xt)=Vi(xt)-max_j Vj(xt) is ≥ γ with probability ≥ 1-δ
    Appendix C.2, Proposition C.7 premise. The margin is assumed, not measured on the trained models; the conclusion that RLVR induces persistence restates the premise.
  • domain assumption BIC is a valid model-order selector for these trajectories
    §3.4 Eq. (14). BIC compares in-sample likelihoods; with persistence non-identifiable above the true K (Remark C.5), BIC may select unidentifiable clone solutions.
invented entities (1)
  • Latent policy states (regimes) independent evidence
    purpose: Discrete latent variables claimed to govern activation dynamics during CoT reasoning; used for comparison, steering, and pruning
    State-swap ablations (drop in R² under label permutation), transplantation into base models (pass@8 rises from 0 to 0.60), and PREFIXGUARD gains provide out-of-model handles. However, the states live inside a fitted CEBRA+EM model, and identifiability is not guaranteed for the empirical pipeline (Appendix C preamble), so the evidence is predictive/causal rather than a proof that the regimes are 'the' true policies.

pith-pipeline@v1.3.0-alltime-deepseek · 32092 in / 23938 out tokens · 247872 ms · 2026-08-01T15:09:12.196440+00:00 · methodology

0 comments
read the original abstract

Reasoning-specialized language models show large performance gains over base models, yet the internal changes responsible for improved multi-step reasoning remain poorly understood. It is unclear whether reasoning fine-tuning improves local token-level competence or globally reorganizes how models structure inference over time. We address this question by modeling Chain-of-Thought reasoning as a switching dynamical system (SDS), in which internal representations evolve under discrete latent policy states. Our framework combines time-aware contrastive representation learning with discrete regime discovery to recover latent policies from activation trajectories. Across four benchmarks and model scales from 1.5B to 32B parameters, reasoning-fine-tuned models exhibit richer latent-policy organization than their base counterparts, characterized by more differentiated transition structure and model-dependent changes in state utilization, persistence, and mixing. The recovered regimes exhibit functional specialization aligned with distinct reasoning stages, and extensive controls confirm that their structure is not explained by correctness, representation learning, or modeling priors, but depends on the coherent temporal organization of reasoning trajectories. Causal interventions further show that the regimes are functionally meaningful: state-swap ablations reduce one-step predictive fit, while transplanting reasoning dynamics into base models improves performance on challenging reasoning problems. Finally, SDS-guided pruning of failure-prone reasoning prefixes outperforms self-consistency in 11 of 12 model-dataset settings, with gains of up to 12.5 percentage points. Together, our results suggest that reasoning fine-tuning globally reorganizes latent dynamics, offering a new lens for mechanistic analysis and process-level control of reasoning models.

Figures

Figures reproduced from arXiv: 2607.18532 by Abir Harrasse, Chaithanya Bandi, Fateme Hashemi Chaleshtori, Hunar Batra, Michael Lan.

Figure 1
Figure 1. Figure 1: Left: Overview of our framework. A base model Mθ is fine-tuned for reasoning to produce Mθ ′ . We fit a Switching Dynamical System (SDS) to each model’s activation trajectories and recover sequences of latent regimes. Reasoning fine-tuning reorganizes state utilization, persistence, and mixing while producing more differentiated transition dynamics. The recovered reasoning policy can be transplanted onto t… view at source ↗
Figure 2
Figure 2. Figure 2: CK residuals against the permu￾tation null. Points below the diagonal indi￾cate that the decoded regime sequence sat￾isfies the Chapman–Kolmogorov relation more closely than a memoryless baseline. A core modeling assumption is that the de￾coded regime sequence {st} follows a first￾order Markov chain. We verify this using two complementary tests. The first is a Chapman– Kolmogorov residual test: for a first… view at source ↗
Figure 3
Figure 3. Figure 3: Predictive gain from increasing la￾tent regimes. The y-axis shows ∆R 2 vs. a linear autoregressive baseline, with markers indicating BIC-selected K. Across models, performance improves with a small number of regimes, supporting discrete latent policy states over a single-mode baseline. To test whether additional latent regimes provide genuinely useful predictive struc￾ture, we sweep the number of regimes K… view at source ↗
Figure 4
Figure 4. Figure 4: BIC curves comparing reasoning and base models across model families. Bold lines [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Per-dataset BIC curves for reasoning and base models across all model families. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Representative CEBRA trajectories for Qwen-14B on GSM8K. [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Row-normalized contingency matrix P(regime | stage) across three models (left: Llama-8B, middle: Qwen-1.5B, right: Qwen-14B), showing how human-defined reasoning stages distribute across discovered latent regimes and revealing distinct specialization patterns across the recovered policy states. traces with both a reasoning stage via Qwen2.5-7B-Instruct (Yang et al., 2024b) and a state via SDS. This begets … view at source ↗
Figure 8
Figure 8. Figure 8: Empirical verification of Corollary B.3 for Reasoning (top) and base (bottom) models across all model–dataset pairs. Each point is one model–dataset case; marker shape denotes model family. Filled markers satisfy B(fC) − B(fP) > log(M(fC)/M(fP)); hollow markers do not. (A3) Nondegeneracy and compactness. Each Σk ≻ 0. The parameter space Θ is compact. Moments sufficient for stationarity and log-likelihood i… view at source ↗
Figure 9
Figure 9. Figure 9: Latent policy structure across training for Llama-8B (layer 31). Grey band: base [PITH_FULL_IMAGE:figures/full_fig_p025_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Layer ablation of SDS metrics across depth for each model family. Bold lines [PITH_FULL_IMAGE:figures/full_fig_p031_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: ∆R 2 over linear autoregressive baselines for alternative latent-state discov￾ery pipelines. CEBRA+EM consistently improves predictive fit over CEBRA-MoE, while PCA+SLDS achieves competitive fit on some settings but does so with substantially different structural tradeoffs [PITH_FULL_IMAGE:figures/full_fig_p032_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Effective number of occupied regimes for alternative pipelines. CEBRA-MoE [PITH_FULL_IMAGE:figures/full_fig_p032_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Persistence under alternative latent-state discovery pipelines. CEBRA-MoE at [PITH_FULL_IMAGE:figures/full_fig_p032_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Transition-structure TVD relative to a reference distribution. CEBRA+EM yields [PITH_FULL_IMAGE:figures/full_fig_p038_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Spectral-gap proxy across alternative pipelines. Lower values correspond to [PITH_FULL_IMAGE:figures/full_fig_p038_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: State-swap ablation in early layers. The identity assignment preserves sub [PITH_FULL_IMAGE:figures/full_fig_p038_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: State-swap ablation in late layers. The drop in [PITH_FULL_IMAGE:figures/full_fig_p039_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: Cross-dataset consistency of recovered latent policy states under a shared frozen [PITH_FULL_IMAGE:figures/full_fig_p039_18.png] view at source ↗
Figure 19
Figure 19. Figure 19: Representative CEBRA trajectories for Llama-8B on GSM8K. [PITH_FULL_IMAGE:figures/full_fig_p040_19.png] view at source ↗
Figure 20
Figure 20. Figure 20: Representative CEBRA trajectories for Llama-8B on MATH500. [PITH_FULL_IMAGE:figures/full_fig_p041_20.png] view at source ↗
Figure 21
Figure 21. Figure 21: Representative CEBRA trajectories for Qwen-1.5B on GSM8K. [PITH_FULL_IMAGE:figures/full_fig_p042_21.png] view at source ↗
Figure 22
Figure 22. Figure 22: Representative CEBRA trajectories for Qwen-1.5B on MATH500. [PITH_FULL_IMAGE:figures/full_fig_p043_22.png] view at source ↗
Figure 23
Figure 23. Figure 23: Representative CEBRA trajectories for Qwen-14B on GSM8K. [PITH_FULL_IMAGE:figures/full_fig_p044_23.png] view at source ↗
Figure 24
Figure 24. Figure 24: Representative CEBRA trajectories for Qwen-14B on MATH500. [PITH_FULL_IMAGE:figures/full_fig_p045_24.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

81 extracted references · 1 canonical work pages

  1. [1]

    2023 , eprint=

    Chain-of-Thought Prompting Elicits Reasoning in Large Language Models , author=. 2023 , eprint=

    2023

  2. [2]

    ICML 2025 Workshop on Reliable and Responsible Foundation Models , year=

    A Statistical Physics of Language Model Reasoning , author=. ICML 2025 Workshop on Reliable and Responsible Foundation Models , year=

    2025

  3. [3]

    Proceedings of the 41st International Conference on Machine Learning , articleno =

    Balsells-Rodas, Carles and Wang, Yixin and Li, Yingzhen , title =. Proceedings of the 41st International Conference on Machine Learning , articleno =. 2024 , publisher =

    2024

  4. [5]

    2026 , eprint=

    Learning a Generative Meta-Model of LLM Activations , author=. 2026 , eprint=

    2026

  5. [6]

    The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

    Quantifying Elicitation of Latent Capabilities in Language Models , author=. The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

  6. [7]

    2025 , eprint=

    Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model? , author=. 2025 , eprint=

    2025

  7. [8]

    On the Limits of

    Fang Wu and Yejin Choi , booktitle=. On the Limits of. 2025 , url=

    2025

  8. [9]

    2025 , eprint=

    Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoning , author=. 2025 , eprint=

    2025

  9. [10]

    2025 , eprint=

    Thought Branches: Interpreting LLM Reasoning Requires Resampling , author=. 2025 , eprint=

    2025

  10. [11]

    2025 , eprint=

    Reasoning-Finetuning Repurposes Latent Representations in Base Models , author=. 2025 , eprint=

    2025

  11. [12]

    2025 , eprint=

    Base Models Know How to Reason, Thinking Models Learn When , author=. 2025 , eprint=

    2025

  12. [13]

    2025 , eprint=

    Rank-1 LoRAs Encode Interpretable Reasoning Signals , author=. 2025 , eprint=

    2025

  13. [14]

    in Hidden Markov Models , author=

  14. [16]

    and Spragins, John D

    Yakowitz, Sidney J. and Spragins, John D. , title =. The Annals of Mathematical Statistics , volume =. 1968 , month =

    1968

  15. [17]

    On the Direction of

    Kexin Huang and Haoming Meng and Junkang Wu and Jinda Lu and Chiyu Ma and Ziqian Chen and Xue Wang and Bolin Ding and Jiancan Wu and Xiang Wang and Xiangnan He and Guoyin Wang and Jingren Zhou , booktitle=. On the Direction of. 2026 , url=

    2026

  16. [19]

    2026 , eprint=

    On the Entropy Dynamics in Reinforcement Fine-Tuning of Large Language Models , author=. 2026 , eprint=

    2026

  17. [20]

    2025 , eprint=

    The CoT Encyclopedia: Analyzing, Predicting, and Controlling how a Reasoning Model will Think , author=. 2025 , eprint=

    2025

  18. [21]

    and Fu, K

    Ackerson, G. and Fu, K. , journal=. On state estimation in switching environments , year=

  19. [22]

    Neural computation , volume=

    Variational learning for switching state-space models , author=. Neural computation , volume=. 2000 , publisher=

    2000

  20. [23]

    2017 , editor =

    Linderman, Scott and Johnson, Matthew and Miller, Andrew and Adams, Ryan and Blei, David and Paninski, Liam , booktitle =. 2017 , editor =

    2017

  21. [24]

    2025 , eprint=

    The Reasoning Boundary Paradox: How Reinforcement Learning Constrains Language Models , author=. 2025 , eprint=

    2025

  22. [25]

    The Fourteenth International Conference on Learning Representations , year=

    Narrow Finetuning Leaves Clearly Readable Traces in the Activation Differences , author=. The Fourteenth International Conference on Learning Representations , year=

  23. [27]

    Mechanistic Interpretability Workshop at NeurIPS 2025 , year=

    Towards a Mechanistic Understanding of Robustness in Finetuned Reasoning Models , author=. Mechanistic Interpretability Workshop at NeurIPS 2025 , year=

    2025

  24. [28]

    Towards Understanding Fine-Tuning Mechanisms of

    Xu Wang and Yan Hu and Wenyu Du and Reynold Cheng and Benyou Wang and Difan Zou , booktitle=. Towards Understanding Fine-Tuning Mechanisms of. 2025 , url=

    2025

  25. [29]

    ICLR 2025 Workshop on Building Trust in Language Models and Applications , year=

    Towards Understanding Distilled Reasoning Models: A Representational Approach , author=. ICLR 2025 Workshop on Building Trust in Language Models and Applications , year=

    2025

  26. [30]

    Demystifying Reasoning Dynamics with Mutual Information: Thinking Tokens are Information Peaks in

    Chen Qian and Dongrui Liu and Haochen Wen and Zhen Bai and Yong Liu and Jing Shao , booktitle=. Demystifying Reasoning Dynamics with Mutual Information: Thinking Tokens are Information Peaks in. 2025 , url=

    2025

  27. [31]

    2025 , eprint=

    Thought Anchors: Which LLM Reasoning Steps Matter? , author=. 2025 , eprint=

    2025

  28. [32]

    Workshop on Reasoning and Planning for Large Language Models , year=

    Understanding Reasoning in Thinking Language Models via Steering Vectors , author=. Workshop on Reasoning and Planning for Large Language Models , year=

  29. [34]

    Measuring Chain of Thought Faithfulness by Unlearning Reasoning Steps

    Tutek, Martin and Hashemi Chaleshtori, Fateme and Marasovic, Ana and Belinkov, Yonatan. Measuring Chain of Thought Faithfulness by Unlearning Reasoning Steps. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025. doi:10.18653/v1/2025.emnlp-main.504

    doi:10.18653/v1/2025.emnlp-main.504 2025

  30. [36]

    2021 , eprint=

    Measuring Mathematical Problem Solving With the MATH Dataset , author=. 2021 , eprint=

    2021

  31. [38]

    2024 , eprint=

    MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark , author=. 2024 , eprint=

    2024

  32. [39]

    2024 , url =

    Llama 3 Model Card , author=. 2024 , url =

    2024

  33. [42]

    QwQ-32B: Embracing the Power of Reinforcement Learning , url =

    Qwen Team , month =. QwQ-32B: Embracing the Power of Reinforcement Learning , url =

  34. [44]

    2025 , eprint=

    1.4 Million Open-Source Distilled Reasoning Dataset to Empower Large Language Model Training , author=. 2025 , eprint=

    2025

  35. [45]

    2025 , eprint=

    DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning , author=. 2025 , eprint=

    2025

  36. [46]

    2025 , eprint=

    Tulu 3: Pushing Frontiers in Open Language Model Post-Training , author=. 2025 , eprint=

    2025

  37. [47]

    2024 , eprint=

    DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models , author=. 2024 , eprint=

    2024

  38. [48]

    2024 , month = aug, journal =

    GPT-4o System Card , author =. 2024 , month = aug, journal =

    2024

  39. [49]

    2026 , url=

    Junda Wu and Yuxin Xiong and Xintong Li and Sheldon Yu and Zhengmian Hu and Tong Yu and Rui Wang and Xiang Chen and Jingbo Shang and Julian McAuley , booktitle=. 2026 , url=

    2026

  40. [50]

    2025 , url=

    Runjin Chen and Zhenyu Zhang and Junyuan Hong and Souvik Kundu and Zhangyang Wang , booktitle=. 2025 , url=

    2025

  41. [51]

    Ackerson and K

    G. Ackerson and K. Fu. On state estimation in switching environments. IEEE Transactions on Automatic Control, 15 0 (1): 0 10--17, 1970. doi:10.1109/TAC.1970.1099359

    arXiv 1970

  42. [52]

    Llama 3 model card

    AI@Meta. Llama 3 model card. 2024. URL https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md

    2024

  43. [53]

    Qwen technical report

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, Peng Wang, Shijie Wang, Wei Wang, Shengguang Wu, Benfeng X...

    Pith/arXiv arXiv 2023

  44. [54]

    On the identifiability of switching dynamical systems

    Carles Balsells-Rodas, Yixin Wang, and Yingzhen Li. On the identifiability of switching dynamical systems. In Proceedings of the 41st International Conference on Machine Learning, ICML'24. JMLR.org, 2024

    2024

  45. [55]

    Bogdan, Uzay Macar, Neel Nanda, and Arthur Conmy

    Paul C. Bogdan, Uzay Macar, Neel Nanda, and Arthur Conmy. Thought anchors: Which llm reasoning steps matter?, 2025. URL https://arxiv.org/abs/2506.19143

    arXiv 2025

  46. [56]

    A statistical physics of language model reasoning

    Jack David Carson. A statistical physics of language model reasoning. In ICML 2025 Workshop on Reliable and Responsible Foundation Models, 2025. URL https://openreview.net/forum?id=MbJXVbwSir

    2025

  47. [57]

    SEAL : Steerable reasoning calibration of large language models for free

    Runjin Chen, Zhenyu Zhang, Junyuan Hong, Souvik Kundu, and Zhangyang Wang. SEAL : Steerable reasoning calibration of large language models for free. In Second Conference on Language Modeling, 2025. URL https://openreview.net/forum?id=klPszYDIRT

    2025

  48. [58]

    Training verifiers to solve math word problems

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021

    Pith/arXiv arXiv 2021

  49. [59]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025

    DeepSeek-AI. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025. URL https://arxiv.org/abs/2501.12948

    Pith/arXiv arXiv 2025

  50. [60]

    Asymptotic properties of the maximum likelihood estimator in autoregressive models with markov regime

    Randal Douc, Éric Moulines, and Tobias Rydén. Asymptotic properties of the maximum likelihood estimator in autoregressive models with markov regime. The Annals of Statistics, 32 0 (5), October 2004. ISSN 0090-5364. doi:10.1214/009053604000000021. URL http://dx.doi.org/10.1214/009053604000000021

    work page doi:10.1214/009053604000000021 2004

  51. [61]

    Variational learning for switching state-space models

    Zoubin Ghahramani and Geoffrey E Hinton. Variational learning for switching state-space models. Neural computation, 12 0 (4): 0 831--864, 2000

    2000

  52. [62]

    Measuring mathematical problem solving with the math dataset, 2021

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset, 2021. URL https://arxiv.org/abs/2103.03874

    Pith/arXiv arXiv 2021

  53. [63]

    On the direction of RLVR updates for LLM reasoning: Identification and exploitation

    Kexin Huang, Haoming Meng, Junkang Wu, Jinda Lu, Chiyu Ma, Ziqian Chen, Xue Wang, Bolin Ding, Jiancan Wu, Xiang Wang, Xiangnan He, Guoyin Wang, and Jingren Zhou. On the direction of RLVR updates for LLM reasoning: Identification and exploitation. In The Fourteenth International Conference on Learning Representations, 2026. URL https://openreview.net/forum...

    2026

  54. [64]

    Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena D

    Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V. Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena D. Hwang, Jiangjiang Yang, Ronan Le Bras, Oyvind Tafjord, Chris Wilhelm, Luca Soldaini, Noah A. Smith, Yizhong Wang, Pradeep Dasigi, and Hannaneh Hajishirzi...

    Pith/arXiv arXiv 2025

  55. [65]

    The cot encyclopedia: Analyzing, predicting, and controlling how a reasoning model will think, 2025

    Seongyun Lee, Seungone Kim, Minju Seo, Yongrae Jo, Dongyoung Go, Hyeonbin Hwang, Jinho Park, Xiang Yue, Sean Welleck, Graham Neubig, Moontae Lee, and Minjoon Seo. The cot encyclopedia: Analyzing, predicting, and controlling how a reasoning model will think, 2025. URL https://arxiv.org/abs/2505.10185

    Pith/arXiv arXiv 2025

  56. [66]

    Clue: Non-parametric verification from experience via hidden-state clustering

    Zhenwen Liang, Ruosen Li, Yujun Zhou, Linfeng Song, Dian Yu, Xinya Du, Haitao Mi, and Dong Yu. Clue: Non-parametric verification from experience via hidden-state clustering. arXiv preprint arXiv:2510.01591, 2025

    arXiv 2025

  57. [67]

    Bayesian Learning and Inference in Recurrent Switching Linear Dynamical Systems

    Scott Linderman, Matthew Johnson, Andrew Miller, Ryan Adams, David Blei, and Liam Paninski. Bayesian Learning and Inference in Recurrent Switching Linear Dynamical Systems . In Aarti Singh and Jerry Zhu (eds.), Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, volume 54 of Proceedings of Machine Learning Research,...

    2017

  58. [68]

    Learning a generative meta-model of llm activations, 2026

    Grace Luo, Jiahai Feng, Trevor Darrell, Alec Radford, and Jacob Steinhardt. Learning a generative meta-model of llm activations, 2026. URL https://arxiv.org/abs/2602.06964

    arXiv 2026

  59. [69]

    Bogdan, Senthooran Rajamanoharan, and Neel Nanda

    Uzay Macar, Paul C. Bogdan, Senthooran Rajamanoharan, and Neel Nanda. Thought branches: Interpreting llm reasoning requires resampling, 2025. URL https://arxiv.org/abs/2510.27484

    Pith/arXiv arXiv 2025

  60. [70]

    Narrow finetuning leaves clearly readable traces in the activation differences

    Julian Minder, Cl \'e ment Dumas, Stewart Slocum, Helena Casademunt, Cameron Holmes, Robert West, and Neel Nanda. Narrow finetuning leaves clearly readable traces in the activation differences. In The Fourteenth International Conference on Learning Representations, 2026. URL https://openreview.net/forum?id=qyVzZsrsnS

    2026

  61. [71]

    Diab, Virginia Smith, and Dawn Song

    Aashiq Muhamed, Xuandong Zhao, Mona T. Diab, Virginia Smith, and Dawn Song. Towards a mechanistic understanding of robustness in finetuned reasoning models. In Mechanistic Interpretability Workshop at NeurIPS 2025, 2025. URL https://openreview.net/forum?id=rqe4zQCURe

    2025

  62. [72]

    La, Duy M

    Phuc Minh Nguyen, Chinh D. La, Duy M. H. Nguyen, Nitesh V. Chawla, Binh T. Nguyen, and Khoa D. Doan. The reasoning boundary paradox: How reinforcement learning constrains language models, 2025. URL https://arxiv.org/abs/2510.02230

    arXiv 2025

  63. [73]

    Arkil Patel, Satwik Bhattamishra, and Navin Goyal. Are NLP models really able to solve simple math word problems? In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp.\ 2080--2094, Online, June 2021. Association for Computational Linguistics. doi:10.18653/v1/2...

    doi:10.18653/v1/2021.naacl-main.168 2021

  64. [74]

    Demystifying reasoning dynamics with mutual information: Thinking tokens are information peaks in LLM reasoning

    Chen Qian, Dongrui Liu, Haochen Wen, Zhen Bai, Yong Liu, and Jing Shao. Demystifying reasoning dynamics with mutual information: Thinking tokens are information peaks in LLM reasoning. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025. URL https://openreview.net/forum?id=E1FrjgaG1J

    2025

  65. [75]

    Learnable latent embeddings for joint behavioural and neural analysis

    Steffen Schneider, Jin Hwa Lee, and Mackenzie Weygandt Mathis. Learnable latent embeddings for joint behavioural and neural analysis. Nature, 617 0 (7960): 0 360–368, May 2023. ISSN 1476-4687. doi:10.1038/s41586-023-06031-6. URL http://dx.doi.org/10.1038/s41586-023-06031-6

    doi:10.1038/s41586-023-06031-6 2023

  66. [76]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024. URL https://arxiv.org/abs/2402.03300

    Pith/arXiv arXiv 2024

  67. [77]

    Understanding reasoning in thinking language models via steering vectors

    Constantin Venhoff, Iv \'a n Arcuschin, Philip Torr, Arthur Conmy, and Neel Nanda. Understanding reasoning in thinking language models via steering vectors. In Workshop on Reasoning and Planning for Large Language Models, 2025 a . URL https://openreview.net/forum?id=OwhVWNOBcz

    2025

  68. [78]

    Base models know how to reason, thinking models learn when, 2025 b

    Constantin Venhoff, Iván Arcuschin, Philip Torr, Arthur Conmy, and Neel Nanda. Base models know how to reason, thinking models learn when, 2025 b . URL https://arxiv.org/abs/2510.07364

    Pith/arXiv arXiv 2025

  69. [79]

    Beyond the 80/20 rule: High-entropy minority tokens drive effective reinforcement learning for llm reasoning, 2025 a

    Shenzhi Wang, Le Yu, Chang Gao, Chujie Zheng, Shixuan Liu, Rui Lu, Kai Dang, Xionghui Chen, Jianxin Yang, Zhenru Zhang, Yuqiong Liu, An Yang, Andrew Zhao, Yang Yue, Shiji Song, Bowen Yu, Gao Huang, and Junyang Lin. Beyond the 80/20 rule: High-entropy minority tokens drive effective reinforcement learning for llm reasoning, 2025 a . URL https://arxiv.org/a...

    Pith/arXiv arXiv 2025

  70. [80]

    Towards understanding fine-tuning mechanisms of LLM s via circuit analysis

    Xu Wang, Yan Hu, Wenyu Du, Reynold Cheng, Benyou Wang, and Difan Zou. Towards understanding fine-tuning mechanisms of LLM s via circuit analysis. In ICLR 2025 Workshop on Building Trust in Language Models and Applications, 2025 b . URL https://openreview.net/forum?id=Z9qzta1yiK

    2025

  71. [81]

    Mmlu-pro: A more robust and challenging multi-task language understanding benchmark, 2024

    Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, Tianle Li, Max Ku, Kai Wang, Alex Zhuang, Rongqi Fan, Xiang Yue, and Wenhu Chen. Mmlu-pro: A more robust and challenging multi-task language understanding benchmark, 2024. URL https://arxiv.org/abs/2406.01574

    Pith/arXiv arXiv 2024

  72. [82]

    Reasoning-finetuning repurposes latent representations in base models, 2025 a

    Jake Ward, Chuqiao Lin, Constantin Venhoff, and Neel Nanda. Reasoning-finetuning repurposes latent representations in base models, 2025 a . URL https://arxiv.org/abs/2507.12638

    Pith/arXiv arXiv 2025

  73. [83]

    Rank-1 loras encode interpretable reasoning signals

    Jake Ward, Paul Riechers, and Adam Shai. Rank-1 loras encode interpretable reasoning signals. arXiv preprint arXiv:2511.06739, 2025 b

    arXiv 2025

  74. [84]

    On the limits of RLVR : Support, entropy, and the illusion of reasoning

    Fang Wu and Yejin Choi. On the limits of RLVR : Support, entropy, and the illusion of reasoning. In 2nd AI for Math Workshop @ ICML 2025, 2025. URL https://openreview.net/forum?id=KXtLWJAzgh

    2025

  75. [85]

    CTRLS : Chain-of-thought reasoning via latent state transition

    Junda Wu, Yuxin Xiong, Xintong Li, Sheldon Yu, Zhengmian Hu, Tong Yu, Rui Wang, Xiang Chen, Jingbo Shang, and Julian McAuley. CTRLS : Chain-of-thought reasoning via latent state transition. In The 29th International Conference on Artificial Intelligence and Statistics, 2026. URL https://openreview.net/forum?id=afld4XGbQe

    2026

  76. [86]

    Yakowitz and John D

    Sidney J. Yakowitz and John D. Spragins. On the identifiability of finite mixtures. The Annals of Mathematical Statistics, 39 0 (1): 0 209--214, February 1968. doi:10.1214/aoms/1177698520

    arXiv 1968

  77. [87]

    Qwen2 technical report

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng He, Junyang Lin, Kai Dang, Keming Lu, Keqin Chen, Kexin Yang, Mei Li, Mingfeng ...

    Pith/arXiv arXiv 2024

  78. [88]

    Qwen2.5 technical report

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tianyi T...

    Pith/arXiv arXiv 2024

  79. [89]

    Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model?, 2025

    Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Yang Yue, Shiji Song, and Gao Huang. Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model?, 2025. URL https://arxiv.org/abs/2504.13837

    Pith/arXiv arXiv 2025

  80. [90]

    1.4 million open-source distilled reasoning dataset to empower large language model training, 2025

    Han Zhao, Haotian Wang, Yiping Peng, Sitong Zhao, Xiaoyu Tian, Shuaiting Chen, Yunjie Ji, and Xiangang Li. 1.4 million open-source distilled reasoning dataset to empower large language model training, 2025. URL https://arxiv.org/abs/2503.19633

    Pith/arXiv arXiv 2025

Showing first 80 references.