SLPO: Scaling Latent Reasoning via a Surrogate Policy

Runyang You; Wenjie Li; Yongqi Li; Zhiyuan Liu

REVIEW 3 major objections 5 minor 27 references

Latent reasoners, which think in continuous vectors rather than decoded tokens, can be trained with outcome-reward RL if each hidden transition is scored under a dropout-built Gaussian surrogate and the stopping time is learned as a policy

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 11:59 UTC pith:2AQI6GAA

load-bearing objection SLPO introduces a genuinely new RL interface for latent reasoning, but the headline Pass@k gains are partly confounded by a doubled latent budget in one of the four main rows, and the theory's key alignment condition is never estimated. the 3 major comments →

arxiv 2607.19691 v2 pith:2AQI6GAA submitted 2026-07-22 cs.CL cs.AIcs.LG

SLPO: Scaling Latent Reasoning via a Surrogate Policy

Runyang You , Zhiyuan Liu , Yongqi Li , Wenjie Li This is my paper

classification cs.CL cs.AIcs.LG

keywords latent reasoningoutcome-reward reinforcement learningsurrogate policyGaussian surrogateMC-dropouttest-time scalingadaptive stoppingcontinuous chain of thought

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Latent reasoning models carry intermediate computation as continuous vectors instead of decoded tokens, so they have been stuck at imitation learning: unlike explicit chain-of-thought, they lacked the token-level likelihood through which outcome rewards could assign credit, and their thinking budget was fixed. SLPO claims to supply both missing pieces — a surrogate likelihood that scores each latent transition under an isotropic Gaussian fitted to K Monte-Carlo dropout forwards, and a stopping head that turns the reasoning horizon into a learnable decision. Trained with RLOO or GRPO on verifiable final-answer rewards, SLPO improves Pass@8 and Pass@16 in all twelve backbone–dataset settings (gains up to 12.07 percentage points), transfers to soft-token latent inference, and learns to allocate longer latent trajectories to harder problems. If the paper is right, continuous reasoning gets the same reward-driven test-time scaling as explicit chain-of-thought, but without paying per-token decoding cost.

Core claim

The paper's central claim is that the obstacle to outcome-reward RL in latent reasoners is not the absence of any per-step likelihood but the absence of a tractable one. SLPO turns each dropout-induced latent transition into a differentiable score: K stochastic forwards define an isotropic Gaussian, the realized state is scored under it, and an advantage-weighted objective pulls the surrogate toward high-reward states. A correctness-supervised stopping head, cold-started before RL, makes the rollout horizon a policy action inside the same objective. Empirically, this yields latent test-time scaling: Pass@8 and Pass@16 improve in all twelve backbone–dataset settings, under RLOO and GRPO, and

What carries the argument

An isotropic Gaussian surrogate over latent transitions: for each step, K dropout forwards give an empirical mean μ and floored variance σ², and the realized latent state is scored as log π̃_θ(h_t) = −(d/2)log(2πσ²) − ‖h_t−μ‖²/(2σ²); with variance held fixed, a positive advantage pulls μ toward the realized state (Prop. 1). A first-stop gate s_θ(h)=σ(g_θ(h)) defines a stopping-time likelihood P_θ(τ=t)=ρ_t ∏_{k<t}(1−ρ_k) that enters the objective as a policy action, seeded by a correctness-supervised cold start. The composite score — latent surrogate + answer-token likelihood + stopping-time log-probability, weighted by an RLOO or GRPO advantage — routes verifiable outcome credit into vector

Load-bearing premise

Everything rests on the unmeasured premise that the isotropic Gaussian surrogate built from only four dropout forwards at the current parameters approximates the true score of the dropout-induced rollout distribution closely enough (small gradient error ε_score); if that alignment fails, SLPO's update is not a policy gradient of the expected reward and the proven local-ascent guarantee has no force.

What would settle it

Estimate, from held-out rollouts, the alignment quantity ε²_score = E_ξ‖∇_θ ℓ̃_θ(ξ) − ∇_θ log q_θ(ξ)‖² that Prop. 3 requires to be small; if it is large relative to √E[A²], the guaranteed bound on the surrogate-gradient error is vacuous. A complementary probe: retrain SLPO with the stopping gate's cold-start supervision replaced by length randomization matched to the same rollout distribution — if Pass@k gains persist, they come from the gate's supervision rather than from outcome-reward credit assignment.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

Latent reasoners gain a real outcome-optimization stage: Pass@8 and Pass@16 improve in all twelve backbone–dataset settings, with one Pass@8 gain reaching +12.07 percentage points (26.8% relative).
The surrogate is a reusable credit-assignment layer: RLOO and GRPO yield closely aligned scaling curves, so SLPO is not tied to one advantage estimator.
A fixed thinking budget becomes a learned policy: the stopping gate allocates longer latent trajectories to harder problems (Pearson r ≈ 0.26–0.30, p < 0.001) and short ones to easy problems.
The same objective transfers to soft-token latent inference (thinking on vocabulary-weighted embeddings), lifting AIME 2025 Pass@1 from 0.96 to 3.33 and AMC23 Pass@1 from 27.03 to 32.50 at 3B scale.
Surrogate construction and K dropout forwards run only at training time; at inference the cost is just the realized adaptive trajectory length.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The score-alignment premise behind Prop. 3–4 — that the Gaussian surrogate's gradient is within small ε_score of the true dropout-rollout score gradient — is never estimated; if K=4 samples misalign in high-dimensional hidden space, the update is not provably a policy gradient of the expected reward. Directly measuring ε²_score on held-out rollouts would settle whether the guarantee holds.
The gate's difficulty correlation suggests a constrained formulation the paper does not take: maximize accuracy under a fixed expected latent budget, which would turn the observed difficulty–length correlation into an explicitly Pareto-optimal compute allocation.
If the claim holds at larger scale, it predicts a cheaper scaling frontier — a model could spend most of its reasoning in cheap continuous steps and decode only the final answer — and the natural test is pass-rate per unit FLOP against explicit CoT.
The G-sweep finding (group size, not surrogate sample count K, is the main scaling axis) hints that exploration diversity, not score fidelity, currently binds performance, and suggests swapping dropout for other stochasticity sources under the same surrogate.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

SLPO introduces a genuinely new RL interface for latent reasoning, but the headline Pass@k gains are partly confounded by a doubled latent budget in one of the four main rows, and the theory's key alignment condition is never estimated.

read the letter

SLPO is worth reading carefully: it is the first method I know of that applies outcome-reward RL directly to unconstrained hidden-state latent reasoners, using a moment-matched Gaussian surrogate over MC-dropout transitions plus a correctness-initialized stopping gate. The combination is new relative to LEPO, Latent-GRPO, and CoLaR, which route through vocabulary or embedding spaces. The paper is also honest: the theory in App. A.5 explicitly conditions on a score-alignment epsilon that is never estimated, and the empirical claims are mostly internally consistent—the 12 Pass@k improvements in Table 1 check out, and the RLOO/GRPO curves align.

The main soft spot is the compute budget. Ungated baselines run at a fixed T=6; +SLPO trains with T_max=12 and lets the gate choose. For GPT-2 CODI, the gate realizes ~11.8 latent steps—nearly double the baseline—so the large gains in that row are uninterpretable as an RL effect, and no fixed-12 control is reported. But the stress-test overstates the scope: the largest claimed gain (+12.07 pp on MultiArith Pass@8) comes from Llama COCONUT, which runs at ~7.5 steps, and Llama CODI runs at ~5.8 steps with modest gains. So the confound is real but partial, not load-bearing across all 12 settings. A compute-matched control would settle it.

Second, the theory. Prop 4's local ascent depends on an epsilon_score that is never estimated. For a Gaussian surrogate built from K=4 MC-dropout forwards in d-dimensional hidden space, that alignment could be poor. The paper states the condition honestly, but it doesn't check it. This is a gap, though it doesn't invalidate the empirical interface—the surrogate is a legitimate self-consistency score, as the reader notes.

Third, the main table has no error bars or significance tests. Pass@k is a mean over three seeds; the spread is not shown. The soft-token transfer is mixed: at 3B, SLPO loses to LEPO on MATH500 P@1/P@32 and GSM8K P@32, and the headline AIME gain is 3.33 vs 0.96—on a 30-problem benchmark, that's one correct answer. The difficulty-adaptive correlation (r=0.26–0.30) is plausible, but the cold-start objective has an unhandled log(0) edge case when the valid stopping set is empty.

Overall: this is a serious paper for the latent reasoning subfield. The method is novel and specified precisely enough to replicate, and the partial controls give me confidence the direction is real. I would send it to peer review—a good referee will ask for a compute-matched control and a sensitivity analysis on the surrogate alignment. With those, the claims would be considerably stronger.

Referee Report

3 major / 5 minor

Summary. The paper proposes SLPO, a method to apply outcome-reward RL to autoregressive latent reasoners. Two components are introduced: a stopping head that is cold-started from correctness signals and then refined by RL, and a Gaussian surrogate policy over latent transitions built from MC-dropout forwards, which supplies a differentiable per-step likelihood for credit assignment. Training uses RLOO or GRPO with verifiable final-answer rewards. Experiments on COCONUT and CODI with GPT-2 and Llama backbones report Pass@8/16 improvements in all 12 backbone–dataset settings, adaptive latent lengths, and transfer to soft-token inference. The appendix gives propositions intended to justify the surrogate update and a local expected-reward ascent guarantee.

Significance. If the claims hold, SLPO addresses a real bottleneck: latent reasoners are mostly imitation-trained and lack a tractable policy likelihood, so outcome-reward RL has been difficult to apply. The paper is clearly written and the empirical effort is substantial: two latent reasoners, two backbones, three held-out benchmarks, RLOO/GRPO comparisons, hyperparameter sweeps, a computational profile, and a transfer experiment to soft-token inference. Table 1 is internally consistent in the sense that all reported Pass@k cells improve as stated and the headline 12.07 pp / 26.8% arithmetic checks out. However, the central scaling claim is currently confounded by an unequal latent-compute budget, and the theoretical guarantee rests on an unmeasured score-alignment condition. Both issues are load-bearing for the paper's main contribution, so the manuscript needs targeted additional experiments or a substantially weakened interpretation.

major comments (3)

[§5 Models and baselines; Table 2; App. A.2/A.6]
[App. A.5, Prop. 3–4]
[Table 4; §7.3]

minor comments (5)

[Table 1]
[§7.4 / App. D.5]
[§7.5 / Fig. 7]
[Alg. 1 / §4.1]
[App. A.9]

Circularity Check

0 steps flagged

No significant circularity: the surrogate is an on-policy reward-weighted score rather than a fitted prediction; the theory's alignment condition is explicit and the empirical claims are held-out.

full rationale

SLPO's surrogate transition likelihood is constructed from MC-dropout forwards at the current parameters and used to score the same rollout distribution; this is a reward-weighted on-policy score, not a hidden fit whose output is claimed as a prediction. The detached empirical update in App. A.5 Prop. 2 is exactly the objective the paper says is optimized. Props. 3-4 state a conditional local-ascent guarantee subject to an unmeasured score-alignment epsilon_score; the absence of an estimate for epsilon is a correctness/verification gap, but the theorem is not circular because it does not assume the conclusion. The stopping-gate cold start supervises the gate with answer correctness at candidate lengths, so the difficulty-length correlation in Fig. 7 is plausibly driven by that label; however it is evaluated on held-out data and is not equal by construction to the training loss. Prop. 1 is a mathematical identity describing the mean-gradient component of the surrogate loss, not an empirical prediction. The only self-citation that could raise concern is reference [20] (same-group prior work on MC-dropout latent rollouts and GSM-Hard/MultiArith), but MC-dropout stochasticity is standard and externally testable, and none of SLPO's central derivations reduce to that citation. The Tmax=6 vs Tmax=12 comparison is a possible experimental confound, not a definitional reduction of a prediction to an input. Verdict: no significant circularity.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 2 invented entities

Everything the central claim rests on: the behavior distribution q_θ must be explorable (MC-dropout p=0.1, inherited from the authors' prior [20]); the Gaussian surrogate must approximately match the true dropout score (ε_score, unmeasured); the exact-answer reward must be a valid signal; standard inequalities fill in Props 1-5. Free parameters: K=4, G=8, p=0.1, T_min=3, T_max=12, variance floor ε (value unreported in main setup), and the validation-selected stop threshold. The budget increase from T_max=6 to 12 is the main confound in interpreting the empirical gains.

free parameters (6)

K (surrogate forward passes) = 4 (swept over {2,4,8} in Fig. 3)
Number of MC-dropout forwards used to estimate the Gaussian surrogate moments μ and σ² at each step (Sec 4.2). The paper reports performance is 'substantially less sensitive' to K, but the surrogate estimator is defined by it.
G (group/rollout size) = 8 (swept over {2,4,8})
Number of latent trajectories sampled per problem for RLOO advantages; identified as the primary scaling axis (Sec 7.1, Fig. 3).
MC-dropout rate p = 0.1
Dropout probability during rollout generation; it defines the stochastic behavior distribution q_θ that is the only exploration substrate (Sec 4.2, App A.3).
Thinking budget T_max / T_min = 12 / 3
Maximum latent steps for +SLPO (raised from the ungated baselines' fixed 6) and the earliest allowed stop. This budget increase is a confound with the reported gains (Table 2).
Variance floor ε = not stated for continuous setting; ε=1e-6 in soft-token (App A.9)
Floors σ² in the isotropic Gaussian; Prop 5 shows the surrogate objective is unbounded without it, so the floor is load-bearing yet its value is unreported in the main setup.
Stop threshold τ = swept {0.5,...,0.9}, selected by validation accuracy (App A.6)
Deterministic stopping threshold at inference; chosen by a validation sweep, a mild selection step whose choice affects all reported Acc/#L numbers.

axioms (5)

domain assumption MC-dropout at p=0.1 generates sufficient exploration and its distribution is the behavior policy q_θ
The entire SLPO update assumes dropout-injected stochastic rollouts provide useful exploration for RL. This substrate is inherited from the authors' prior work [20] and not independently validated here.
domain assumption The isotropic Gaussian surrogate (with K samples) approximates the true dropout-transition score: E||∇ℓ̃_θ − ∇log q_θ||² ≤ ε²_score with small ε_score
Prop 3 and 4 (App A.5) bound the gradient error and guarantee local expected-reward ascent only under this unmeasured score-alignment condition. No estimate of ε_score is reported.
domain assumption Latent trajectories lack a tractable per-step likelihood, motivating the surrogate
Sec 4 and related work assert that continuous hidden transitions bypass the vocabulary distribution, so no token-level log-prob exists (cites [16,17]). This is the motivating premise of the method.
domain assumption Exact final-answer match is a valid verifiable reward for grade-school math
RLVR-style binary reward R = 1[â = a*] after answer parsing (App A.3); all credit assignment in cold start and SLPO rests on this signal.
standard math Standard analysis tools: Cauchy-Schwarz, L-smoothness descent lemma, Gaussian density identities (Props 1-5)
Straightforward inequalities and calculus; statements are correct as given.

invented entities (2)

Gaussian surrogate policy π̃_θ over latent transitions no independent evidence
purpose: Provides a tractable per-step log-likelihood for advantage-weighted credit assignment over continuous hidden states (Sec 4.2)
A modeling construct defined by the model's own MC-dropout moments; its validity is only evidenced by the paper's downstream task gains. It is a fitted/constructed object, not an independently checkable entity.
Stopping head g_θ / gate policy no independent evidence
purpose: Learns variable-horizon stopping (continue/stop) so the rollout length is optimizable (Sec 4.1, Sec 4.3)
Method component; initialized by correctness cold start and refined by RL. Its behavior is evaluated only within this paper.

pith-pipeline@v1.3.0-alltime-deepseek · 18863 in / 22244 out tokens · 207644 ms · 2026-08-01T11:59:35.282560+00:00 · methodology

0 comments

read the original abstract

Reinforcement learning with verifiable rewards has become the predominant recipe for eliciting test-time scaling in explicit Chain-of-Thought reasoners. Yet this scaling path remains computationally costly, since every intermediate step must be decoded as a language token. Latent reasoning instead carries intermediate computation as continuous vectors and already matches or surpasses explicit CoT at far shorter horizons. Despite this promise, latent reasoners remain largely imitation-bound, while explicit CoT has already moved past imitation via outcome-reward RL. Latent trajectories lack a tractable per-step likelihood and an adaptive stopping interface under fixed thinking budgets, so outcome rewards cannot elicit latent test-time scaling. We introduce Surrogate Latent Policy Optimization (SLPO) to bring outcome-reward RL to autoregressive latent reasoners: an empirical surrogate policy density over latent transitions for trajectory-level credit assignment, and a correctness-supervised stopping head that outcome-reward optimization refines into a variable-horizon policy. Across continuous and soft thinking settings, SLPO improves Pass@$k$ under parallel sampling and allocates longer latent computation to harder instances with higher deterministic accuracy.

discussion (0)

Reference graph

Works this paper leans on

27 extracted references · 1 canonical work pages

[1]

Chi, Quoc V

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. NIPS ’22, Red Hook, NY, USA, 2022. Curran Associates Inc. ISBN 9781713871088

2022
[2]

Le, Ed H

Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. InThe Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net,

2023
[3]

s1: Simple test-time scaling

NiklasMuennighoff,ZitongYang,WeijiaShi,XiangLisaLi,LiFei-Fei,HannanehHajishirzi,LukeZettlemoyer,Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. s1: Simple test-time scaling. In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng, editors,Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processin...

work page doi:10.18653/v1/2025.em 2025
[4]

Scaling LLM test-time compute optimally can be more effective than scaling parameters for reasoning

Charlie Victor Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling LLM test-time compute optimally can be more effective than scaling parameters for reasoning. InThe Thirteenth International Conference on Learning Representations, 2025. URLhttps://openreview.net/forum?id=4FWAwZtd2n

2025
[5]

Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

Pith/arXiv arXiv 2025
[6]

Reinforcement learning with verifiable rewards implicitly incentivizes correct reasoning in base llms.arXiv preprint arXiv:2506.14245, 2025

Xumeng Wen, Zihan Liu, Shun Zheng, Shengyu Ye, Zhirong Wu, Yang Wang, Zhijian Xu, Xiao Liang, Junjie Li, Ziming Miao, Jiang Bian, and Mao Yang. Reinforcement learning with verifiable rewards implicitly incentivizes correct reasoning in base llms.arXiv preprint arXiv:2506.14245, 2025

Pith/arXiv arXiv 2025
[7]

A survey of reinforcement learning for large reasoning models.arXiv preprint arXiv:2509.08827, 2025

Kaiyan Zhang, Yuxin Zuo, Bingxiang He, Youbang Sun, Runze Liu, Che Jiang, et al. A survey of reinforcement learning for large reasoning models.arXiv preprint arXiv:2509.08827, 2025

Pith/arXiv arXiv 2025
[8]

Stop overthinking: A survey on efficient reasoning for large language models, 2025

Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Shaochen Zhong, Na Zou, Hanjie Chen, and Xia Hu. Stop overthinking: A survey on efficient reasoning for large language models, 2025. URLhttps://arxiv.org/abs/2503.16419

Pith/arXiv arXiv 2025
[9]

Regular: Variational latent reasoning guided by rendered chain-of-thought, 2026

Fanmeng Wang, Haotian Liu, Guojiang Zhao, Hongteng Xu, and Zhifeng Gao. Regular: Variational latent reasoning guided by rendered chain-of-thought, 2026. URLhttps://arxiv.org/abs/2601.23184

arXiv 2026
[10]

Training large language models to reason in a continuous latent space

Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason E Weston, and Yuandong Tian. Training large language models to reason in a continuous latent space. InSecond Conference on Language Modeling, 2025. URLhttps://openreview.net/forum?id=Itxz7S4Ip3. 12 SLPO: Scaling Latent Reasoning via a Surrogate Policy

2025
[11]

Reasoning beyond language: A comprehensive survey on latent chain-of-thought reasoning,

Xinghao Chen, Anhao Zhao, Heming Xia, Xuan Lu, Hanlin Wang, Yanjun Chen, Wei Zhang, Jian Wang, Wenjie Li, and Xiaoyu Shen. Reasoning beyond language: A comprehensive survey on latent chain-of-thought reasoning,
[12]

CODI:Compressingchain-of-thought into continuous space via self-distillation

ZhenyiShen, HanqiYan, LinhaiZhang, ZhanghaoHu, YaliDu, andYulanHe. CODI:Compressingchain-of-thought into continuous space via self-distillation. In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng, editors,Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 677–693, Suzhou, China, Nov...

doi:10.18653/v1/2025.emnlp-main.36 2025
[13]

Think silently, think fast: Dynamic latent compression of LLM reasoning chains

Wenhui Tan, Jiaze Li, Jianzhong Ju, Zhenbo Luo, Ruihua Song, and Jian Luan. Think silently, think fast: Dynamic latent compression of LLM reasoning chains. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025. URLhttps://openreview.net/forum?id=AQsko3PPUe

2025
[14]

Sim-cot: Supervised implicit chain-of-thought, 2025

Xilin Wei, Xiaoran Liu, Yuhang Zang, Xiaoyi Dong, Yuhang Cao, Jiaqi Wang, Xipeng Qiu, and Dahua Lin. Sim-cot: Supervised implicit chain-of-thought, 2025. URLhttps://arxiv.org/abs/2509.20317

arXiv 2025
[15]

Render-of-thought: Rendering textual chain-of-thought as images for visual latent reasoning, 2026

Yifan Wang, Shiyu Li, Peiming Li, Xiaochen Yang, Yang Tang, and Zheng Wei. Render-of-thought: Rendering textual chain-of-thought as images for visual latent reasoning, 2026. URLhttps://arxiv.org/abs/2601.14750

Pith/arXiv arXiv 2026
[16]

Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model?arXiv preprint arXiv:2504.13837, 2025

Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Yang Yue, Shiji Song, and Gao Huang. Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model?arXiv preprint arXiv:2504.13837, 2025

Pith/arXiv arXiv 2025
[17]

La, Duy M

Phuc Minh Nguyen, Chinh D. La, Duy M. H. Nguyen, Nitesh V. Chawla, Binh T. Nguyen, and Khoa D. Doan. The reasoning boundary paradox: How reinforcement learning constrains language models.arXiv preprint arXiv:2510.02230, 2025

arXiv 2025
[18]

DART:Distillingautoregressivereasoning to silent thought

NanJiang, ZimingWu, De-ChuanZhan, FumingLai, andShaobingLian. DART:Distillingautoregressivereasoning to silent thought. In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng, editors, Proceedingsofthe2025ConferenceonEmpiricalMethodsinNaturalLanguageProcessing, pages5100–5108, Suzhou, China, November 2025. Association for Comput...

doi:10.18653/v1/2025.emnlp-main.256 2025
[19]

LLM latent reasoning as chain of superposition, 2025

Jingcheng Deng, Liang Pang, Zihao Wei, Shichen Xu, Zenghao Duan, Kun Xu, Yang Song, Huawei Shen, and Xueqi Cheng. LLM latent reasoning as chain of superposition, 2025. URLhttps://arxiv.org/abs/2510.15522

arXiv 2025
[20]

Parallel test-time scaling for latent reasoning models

Runyang You, Yongqi Li, Meng Liu, Wenjie Wang, Liqiang Nie, and Wenjie Li. Parallel test-time scaling for latent reasoning models. InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), San Diego, USA, November 2026. Association for Computational Linguistics

2026
[21]

Deepseekmath: Pushing the limits of mathematical reasoning in open language models

Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024

Pith/arXiv arXiv 2024
[22]

Effective reinforcement learning for reasoning in language models.arXiv preprint arXiv:2505.17218, 2025

Lianghuan Huang, Shuo Li, Sagnik Anupam, Insup Lee, and Osbert Bastani. Effective reinforcement learning for reasoning in language models.arXiv preprint arXiv:2505.17218, 2025

Pith/arXiv arXiv 2025
[23]

LEPO: Latent reasoning policy optimization for large language models, 2026

Yuyan Zhou, Jiarui Yu, Hande Dong, Zhezheng Hao, Hong Wang, Jianqing Zhang, and Qiang Lin. LEPO: Latent reasoning policy optimization for large language models, 2026. URLhttps://arxiv.org/abs/2604.17892

Pith/arXiv arXiv 2026
[24]

Latent- GRPO: Group relative policy optimization for latent reasoning, 2026

Jingcheng Deng, Zihao Wei, Liang Pang, Junhong Wu, Shicheng Xu, Zenghao Duan, and Huawei Shen. Latent- GRPO: Group relative policy optimization for latent reasoning, 2026. URLhttps://arxiv.org/abs/2604.27998

Pith/arXiv arXiv 2026
[25]

Soft thinking: Unlocking the reasoning potential of LLMs in continuous concept space

Zhen Zhang, Xuehai He, Weixiang Yan, Ao Shen, Chenyang Zhao, Shuohang Wang, Yelong Shen, and Xin Eric Wang. Soft thinking: Unlocking the reasoning potential of LLMs in continuous concept space. InAdvances in Neural Information Processing Systems, 2025. URLhttps://arxiv.org/abs/2505.15778. A Implementation Details This appendix supplements Sec. 5 with mode...

Pith/arXiv arXiv 2025
[2023]

URLhttps://openreview.net/forum?id=1PL1NIMMrw
[2025]

URLhttps://arxiv.org/abs/2505.16782

arXiv

[1] [1]

Chi, Quoc V

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. NIPS ’22, Red Hook, NY, USA, 2022. Curran Associates Inc. ISBN 9781713871088

2022

[2] [2]

Le, Ed H

Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. InThe Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net,

2023

[3] [3]

s1: Simple test-time scaling

NiklasMuennighoff,ZitongYang,WeijiaShi,XiangLisaLi,LiFei-Fei,HannanehHajishirzi,LukeZettlemoyer,Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. s1: Simple test-time scaling. In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng, editors,Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processin...

work page doi:10.18653/v1/2025.em 2025

[4] [4]

Scaling LLM test-time compute optimally can be more effective than scaling parameters for reasoning

Charlie Victor Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling LLM test-time compute optimally can be more effective than scaling parameters for reasoning. InThe Thirteenth International Conference on Learning Representations, 2025. URLhttps://openreview.net/forum?id=4FWAwZtd2n

2025

[5] [5]

Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

Pith/arXiv arXiv 2025

[6] [6]

Reinforcement learning with verifiable rewards implicitly incentivizes correct reasoning in base llms.arXiv preprint arXiv:2506.14245, 2025

Xumeng Wen, Zihan Liu, Shun Zheng, Shengyu Ye, Zhirong Wu, Yang Wang, Zhijian Xu, Xiao Liang, Junjie Li, Ziming Miao, Jiang Bian, and Mao Yang. Reinforcement learning with verifiable rewards implicitly incentivizes correct reasoning in base llms.arXiv preprint arXiv:2506.14245, 2025

Pith/arXiv arXiv 2025

[7] [7]

A survey of reinforcement learning for large reasoning models.arXiv preprint arXiv:2509.08827, 2025

Kaiyan Zhang, Yuxin Zuo, Bingxiang He, Youbang Sun, Runze Liu, Che Jiang, et al. A survey of reinforcement learning for large reasoning models.arXiv preprint arXiv:2509.08827, 2025

Pith/arXiv arXiv 2025

[8] [8]

Stop overthinking: A survey on efficient reasoning for large language models, 2025

Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Shaochen Zhong, Na Zou, Hanjie Chen, and Xia Hu. Stop overthinking: A survey on efficient reasoning for large language models, 2025. URLhttps://arxiv.org/abs/2503.16419

Pith/arXiv arXiv 2025

[9] [9]

Regular: Variational latent reasoning guided by rendered chain-of-thought, 2026

Fanmeng Wang, Haotian Liu, Guojiang Zhao, Hongteng Xu, and Zhifeng Gao. Regular: Variational latent reasoning guided by rendered chain-of-thought, 2026. URLhttps://arxiv.org/abs/2601.23184

arXiv 2026

[10] [10]

Training large language models to reason in a continuous latent space

Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason E Weston, and Yuandong Tian. Training large language models to reason in a continuous latent space. InSecond Conference on Language Modeling, 2025. URLhttps://openreview.net/forum?id=Itxz7S4Ip3. 12 SLPO: Scaling Latent Reasoning via a Surrogate Policy

2025

[11] [11]

Reasoning beyond language: A comprehensive survey on latent chain-of-thought reasoning,

Xinghao Chen, Anhao Zhao, Heming Xia, Xuan Lu, Hanlin Wang, Yanjun Chen, Wei Zhang, Jian Wang, Wenjie Li, and Xiaoyu Shen. Reasoning beyond language: A comprehensive survey on latent chain-of-thought reasoning,

[12] [12]

CODI:Compressingchain-of-thought into continuous space via self-distillation

ZhenyiShen, HanqiYan, LinhaiZhang, ZhanghaoHu, YaliDu, andYulanHe. CODI:Compressingchain-of-thought into continuous space via self-distillation. In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng, editors,Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 677–693, Suzhou, China, Nov...

doi:10.18653/v1/2025.emnlp-main.36 2025

[13] [13]

Think silently, think fast: Dynamic latent compression of LLM reasoning chains

Wenhui Tan, Jiaze Li, Jianzhong Ju, Zhenbo Luo, Ruihua Song, and Jian Luan. Think silently, think fast: Dynamic latent compression of LLM reasoning chains. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025. URLhttps://openreview.net/forum?id=AQsko3PPUe

2025

[14] [14]

Sim-cot: Supervised implicit chain-of-thought, 2025

Xilin Wei, Xiaoran Liu, Yuhang Zang, Xiaoyi Dong, Yuhang Cao, Jiaqi Wang, Xipeng Qiu, and Dahua Lin. Sim-cot: Supervised implicit chain-of-thought, 2025. URLhttps://arxiv.org/abs/2509.20317

arXiv 2025

[15] [15]

Render-of-thought: Rendering textual chain-of-thought as images for visual latent reasoning, 2026

Yifan Wang, Shiyu Li, Peiming Li, Xiaochen Yang, Yang Tang, and Zheng Wei. Render-of-thought: Rendering textual chain-of-thought as images for visual latent reasoning, 2026. URLhttps://arxiv.org/abs/2601.14750

Pith/arXiv arXiv 2026

[16] [16]

Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model?arXiv preprint arXiv:2504.13837, 2025

Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Yang Yue, Shiji Song, and Gao Huang. Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model?arXiv preprint arXiv:2504.13837, 2025

Pith/arXiv arXiv 2025

[17] [17]

La, Duy M

Phuc Minh Nguyen, Chinh D. La, Duy M. H. Nguyen, Nitesh V. Chawla, Binh T. Nguyen, and Khoa D. Doan. The reasoning boundary paradox: How reinforcement learning constrains language models.arXiv preprint arXiv:2510.02230, 2025

arXiv 2025

[18] [18]

DART:Distillingautoregressivereasoning to silent thought

NanJiang, ZimingWu, De-ChuanZhan, FumingLai, andShaobingLian. DART:Distillingautoregressivereasoning to silent thought. In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng, editors, Proceedingsofthe2025ConferenceonEmpiricalMethodsinNaturalLanguageProcessing, pages5100–5108, Suzhou, China, November 2025. Association for Comput...

doi:10.18653/v1/2025.emnlp-main.256 2025

[19] [19]

LLM latent reasoning as chain of superposition, 2025

Jingcheng Deng, Liang Pang, Zihao Wei, Shichen Xu, Zenghao Duan, Kun Xu, Yang Song, Huawei Shen, and Xueqi Cheng. LLM latent reasoning as chain of superposition, 2025. URLhttps://arxiv.org/abs/2510.15522

arXiv 2025

[20] [20]

Parallel test-time scaling for latent reasoning models

Runyang You, Yongqi Li, Meng Liu, Wenjie Wang, Liqiang Nie, and Wenjie Li. Parallel test-time scaling for latent reasoning models. InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), San Diego, USA, November 2026. Association for Computational Linguistics

2026

[21] [21]

Deepseekmath: Pushing the limits of mathematical reasoning in open language models

Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024

Pith/arXiv arXiv 2024

[22] [22]

Effective reinforcement learning for reasoning in language models.arXiv preprint arXiv:2505.17218, 2025

Lianghuan Huang, Shuo Li, Sagnik Anupam, Insup Lee, and Osbert Bastani. Effective reinforcement learning for reasoning in language models.arXiv preprint arXiv:2505.17218, 2025

Pith/arXiv arXiv 2025

[23] [23]

LEPO: Latent reasoning policy optimization for large language models, 2026

Yuyan Zhou, Jiarui Yu, Hande Dong, Zhezheng Hao, Hong Wang, Jianqing Zhang, and Qiang Lin. LEPO: Latent reasoning policy optimization for large language models, 2026. URLhttps://arxiv.org/abs/2604.17892

Pith/arXiv arXiv 2026

[24] [24]

Latent- GRPO: Group relative policy optimization for latent reasoning, 2026

Jingcheng Deng, Zihao Wei, Liang Pang, Junhong Wu, Shicheng Xu, Zenghao Duan, and Huawei Shen. Latent- GRPO: Group relative policy optimization for latent reasoning, 2026. URLhttps://arxiv.org/abs/2604.27998

Pith/arXiv arXiv 2026

[25] [25]

Soft thinking: Unlocking the reasoning potential of LLMs in continuous concept space

Zhen Zhang, Xuehai He, Weixiang Yan, Ao Shen, Chenyang Zhao, Shuohang Wang, Yelong Shen, and Xin Eric Wang. Soft thinking: Unlocking the reasoning potential of LLMs in continuous concept space. InAdvances in Neural Information Processing Systems, 2025. URLhttps://arxiv.org/abs/2505.15778. A Implementation Details This appendix supplements Sec. 5 with mode...

Pith/arXiv arXiv 2025

[26] [2023]

URLhttps://openreview.net/forum?id=1PL1NIMMrw

[27] [2025]

URLhttps://arxiv.org/abs/2505.16782

arXiv