Pith. sign in

REVIEW 4 major objections 4 minor 3 cited by

Learn to Reason Efficiently with Adaptive Length-based Reward Shaping

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Adaptive length rewards can make reasoning models both more accurate and shorter-winded.

desk verdict A solid, practical RL method for compressing reasoning traces; the adaptive difficulty mechanism is the new bit, but the paper overstates its ECR guarantee and needs error bars. read the letter →

arxiv 2505.15612 v1 pith:JAQK2NIJ submitted 2025-05-21 cs.CL cs.AIcs.LG

Wei Liu , Ruochen Zhou , Yiyun Deng , Yuzhen Huang , Junteng Liu , Yuntian Deng , Yizhe Zhang , Junxian He This is my paper
classification cs.CLcs.AIcs.LG
keywords reinforcementlearningrewardshapingchain-of-thoughtcompressionreasoningefficiencyoverthinkingmathematicallengthcontrolParetotrade-off
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the over-thinking of large reasoning models can be corrected by reinforcement-learning reward shaping rather than by truncation. It first shows that truncation, group-based brevity rewards, and budget-based rewards all fit one formula: a correctness term plus a length term controlled by a coefficient. On that view it proposes LASER, a step reward that gives correct short responses a bonus, and then LASER-D, which makes the target length dynamic and difficulty-aware by choosing the shortest length at which at least one correct response is expected for easy, medium, and hard questions. The reported upshot is a Pareto improvement: on AIME2024 a 1.5B model gains +6.1 points of accuracy while using 63% fewer tokens, and the trained models show less repetitive self-reflection. If correct, the claim is that efficient reasoning is not a tax on accuracy but a learnable skill.

What carries the argument

The central object is the length-based step reward under the unified decomposition $\hat{R}(x,y) = C(y) + \lambda(y) \cdot S(y)$. LASER uses a step indicator $S(y) = \alpha \cdot I(L(y) \leq L_T)$ gated by correctness; LASER-D's automatic adapting mechanism uses $ECR_d = P_{l,d} \cdot |C_d|$ to set per-difficulty target lengths $L_A$, where difficulty is bucketed into easy, medium, and hard by per-question correctness rate over 8 rollouts. This mechanism is what carries the argument: it converts the vague goal 'be concise but correct' into a concrete, periodically updated number for each difficulty class, and everything else—the reward function, the GRPO updates, and the Pareto curves—hangs off that number.

What would settle it

Re-run LASER-D on the same models but compute target lengths from a larger held-out set, or from oracle per-question optimal lengths found by grid search; if the accuracy-versus-tokens frontier worsens or no longer dominates truncation, the ECR criterion is the failure point.

Watch

Extended reading notes

Core claim

The central claim is that response length in long-chain-of-thought reasoning models can be treated as a trainable reward target rather than a fixed constraint. The paper proposes a step reward function $S(y) = \alpha \cdot I(L(y) \leq L_T)$ applied only to correct responses, which preserves the correctness signal while giving a bonus for brevity; this is LASER. LASER-D replaces the fixed target $L_T$ with difficulty-group-specific targets $L_A$ that are recomputed periodically on a 500-sample monitoring set, using an expected-correct-responses criterion $ECR_d = P_{l,d} \cdot |C_d| \geq 1$ to pick the smallest length that still permits a correct answer. For hard questions the adaptive target stays near the context limit, for easy questions it shrinks sharply, so the model learns to answer quickly when it can and to deliberate when it must. The variant LASER-DE additionally softens the penalty for long incorrect responses, encouraging exploration. Extensive runs on DeepSeek-R1-Distill-Qwen-1.5B and 7B show these methods dominate prior length-reward baselines on the accuracy-versus-token Pareto frontier, including a +6.1 accuracy gain on AIME2024 at 63% fewer tokens for the 1.5B model.

Load-bearing premise

The whole dynamic mechanism assumes that the shortest length with at least one correct answer on a 500-sample monitoring set, for each difficulty bucket defined by 8-rollout correctness rates, is the right training budget for that bucket.

Editorial extensions

If this is right

  • If LASER-D and LASER-DE are correct, long reasoning traces are partly a reward artifact, meaning future reasoning-model training can target efficiency directly instead of paying an accuracy penalty for shorter outputs.
  • The unified formulation means fixed truncation, group-based rewards, and budget rewards are all points in one design space, so new variants can be constructed by changing the correctness term, the length term, or their gate.
  • The monitoring heuristic gives a practical recipe: a small held-out set, periodic evaluation every N training steps, and per-difficulty budgets suffice to automate length control without manual target-length tuning.
  • On out-of-domain benchmarks such as GPQA, MMLU, and LSAT, the trained models keep or improve the efficiency-accuracy trade-off, suggesting the compression transfers beyond the math training distribution.
  • On the 32B model, the tested LASER-DE variant keeps accuracy within about one point while sharply reducing output length, indicating the mechanism does not require small models to work.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The ECR criterion is effectively a test-time compute allocation policy, so it could be lifted out of RL and used as a decoding-time controller that sets an allowed budget per predicted difficulty for frozen models.
  • Difficulty bucketing by 8-rollout correctness may conflate question difficulty with model uncertainty; replacing it with calibrated uncertainty or verifier scores would be a direct, testable refinement of the method.
  • The reduced 'self-reflection' keywords and backtracking suggest some apparent reasoning behaviors are decorative; length rewards could therefore serve as a probe for identifying which chain-of-thought tokens are actually load-bearing.
  • Because the reported gains are on math-heavy benchmarks where correctness is a single boxed answer, extending the same mechanism to code generation or agentic tasks would test whether the reward-shaping view transfers when evaluation is less clean.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes a unified reward-shaping view of RL-based chain-of-thought compression, then introduces LASER, a step-function length reward with a fixed target length, and LASER-D/LASER-DE, variants that adapt the target length per difficulty group during training. Experiments on DeepSeek-R1-Distill-Qwen-1.5B/7B/32B across MATH500, AIME2024, AMC2023, OlympiadBench, and out-of-domain GPQA/LSAT/MMLU report that LASER-D and LASER-DE achieve better Pareto-optimal accuracy-versus-token trade-offs than truncation, group-based, budget-based, and ThinkPrune baselines; e.g., +6.1 AIME2024 accuracy with 63% fewer tokens on the 1.5B model. Analysis of reasoning traces indicates reduced backtracking and fewer self-reflection keywords after training.

Significance. If the empirical claims hold, the paper makes a useful contribution: it offers a simple unifying framework for length-based reward shaping, introduces an automated difficulty-aware mechanism, and backs it with a broad multi-model, multi-benchmark study as well as released code and models. The Pareto-frontier comparisons are more informative than single-point accuracy comparisons, and the analysis of reasoning-behavior changes is a useful addition. However, the central explanatory mechanism—the automatic target-length selection via ECR—contains a statistical flaw that needs to be fixed or re-scoped before the paper's main claims about adaptive difficulty-aware rewards are supported.

major comments (4)
  1. [Section 5.2, Eq. (3)] The quantity ECR_d = P_{l,d} * |C_d| does not estimate the expected number of correct responses within length l unless correctness and length are independent and |C_d| equals the actual number of correct responses, neither of which is established. P_{l,d} is the marginal coverage ratio over all responses, while the paper's own analysis (Section 7) shows that incorrect responses are systematically longer, so P_{l,d} is not a valid proxy for P(correct and L <= l). For the hard group with |C_d| = 1, the condition ECR_d >= 1 forces L_A to be the maximum response length in the group, regardless of whether that longest response is correct; for the easy group, replacing the observed correct count by the fixed threshold 6 means L_A is determined by an overall quantile of all responses rather than by correct-response lengths. The paper provides no check that the selected L_A matches the minimal length at which correct responses actually appear, so the claimed 'ensuring at least one complete and correct response' guarantee is unsupported. Please either redefine ECR over correct responses, use the observed correct counts, empirically validate the selected L_A against the correct-response length distribution, or revise the claim to state what the heuristic actually computes.
  2. [Section 6.1 and Appendix E.2] The evaluation protocol reports 4 samples for MATH500/OlympiadBench and 16 samples for AIME2024/AMC2023, but it does not specify how the multiple samples are converted into an accuracy number (per-sample accuracy, pass@k, or majority vote), nor how the reported 'average token usage' is computed across correct and incorrect responses. Since both axes of the Pareto front depend on the sampling and aggregation procedure, this omission makes it difficult to compare the reported numbers across methods. Please specify the aggregation rule and, for the headline comparisons, report results under at least one alternative aggregation to show the conclusions are not an artifact of the rule.
  3. [Tables 3, 4 and Figures 1, 5] Each Pareto point is a single training run, with no seeds, standard deviations, or significance tests. Several of the comparisons that support the central claim involve small accuracy gaps (e.g., LASER-D versus LASER on AIME at comparable length, or LASER-DE versus group-based methods in the low-token regime), and RL training variance could plausibly exceed these gaps. At minimum, please provide multiple seeds or variance estimates for the 1.5B model's main comparisons, or explicitly state that the reported differences are single-run observations without error bars.
  4. [Section 6.3, Table 4 (32B)] For the 32B model, only LASER-DE is compared against the original model, with no LASER-D, LASER, or length-reward baseline at that scale. As reported, the 32B result supports only that LASER-DE roughly preserves accuracy while reducing length, not that the full Pareto-optimality claim holds at 32B. Please either add the missing comparisons or explicitly restrict the 32B claim to the reported comparison.
minor comments (4)
  1. [Table 3] The LASER-D LT=1024 row reports AMC accuracy as '256', while the full results in Table 6 list '2556'; this appears to be a typo and should be corrected.
  2. [Appendix E.2] The sentence 'Notably, we do not tune α with fixed value 0.5 in all experiments of our methods' is ambiguous; it should be rephrased to say that α is fixed at 0.5 and not tuned.
  3. [Appendix D.1 and Appendix J] Several typos remain, including 'mannually' instead of 'manually', 'can viewed' instead of 'can be viewed', 'paramters' instead of 'parameters', and 'Behaviros' instead of 'Behaviors'.
  4. [Section 5.2] The phrase 'there is a minimum number of correct responses for each group (e.g., 6, 3, and 1 for easy, medium, and hard levels, respectively, when K = 8)' is inaccurate for hard questions with zero correct responses; if such queries exist, |C_d| = 1 is not a minimum but an assigned constant.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the headline gains are measured on held-out benchmarks after training, and the ECR-based target-length selector is model selection rather than a fitted prediction.

full rationale

The central empirical claims—+6.1 accuracy on AIME2024, 63% token reduction, and the Pareto-optimal trade-offs—are reported from evaluations on held-out benchmarks (MATH500, AIME2024, AMC2023, OlympiadBench, GPQA, LSAT, MMLU) after RL training. None of these numbers is computed from the paper's own equations; they are observed outcomes of trained models. The unified formulation in Eq. 2 and Table 2 is a notational rewrite of existing truncation, group-based, and budget-based reward methods; rewriting a known method in a common notation is not circular, and LASER is presented as a distinct step-reward design rather than as a derived prediction. The ECR mechanism in Eq. 3 selects target lengths LA from a 500-sample monitoring subset of the training data; this is hyperparameter/model selection, not a fitted input that is later relabeled as a predicted result. No equation maps LA or the monitoring statistics into the reported test accuracies by construction. The paper's self-citations, e.g., to SimplerL-Zoo for the behavior-analysis prompt and to DAPO for the clip-higher training heuristic, are methodological and not load-bearing for the central Pareto-efficiency claim. The manuscript's own limitations section notes that further validation in code generation and agentic tasks is needed; that is a scope limitation, not evidence of circularity. The skeptic concern about Eq. 3—that multiplying an all-response coverage ratio by a fixed minimum-correct-count assumes correctness and length are independent—is a validity or correctness risk: the 'at least one complete and correct response' guarantee may not follow from the stated formula. But an unsupported or incorrect justification is not the same as circularity, because the reported results do not reduce to that justification by definition. Overall, the derivation chain is self-contained and the headline results are empirically measured, so the circularity score is 0.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

No new physical or structural entities are introduced. The method rests on several hand-chosen hyperparameters and two domain assumptions about rollout-based difficulty estimation and ECR-based target selection. The evaluation is external to training, so the central empirical claim is not circular, but the design choices are numerous.

free parameters (6)
  • length-reward coefficient alpha = 0.5
    Chosen by hand for all LASER variants; controls how strongly the length bonus competes with the correctness reward (Sections 4.3 and 6.1).
  • lower bound target length L_T = 1024, 2048, 4096, or 8192 depending on method
    A swept hyperparameter used to trace the Pareto frontier; LASER-D and LASER-DE results depend on the enumeration starting point (Section 6.2, Table 5).
  • difficulty thresholds = k/3 and 2k/3 with k=8 rollouts
    Hand-chosen cutoffs dividing queries into easy, medium, and hard groups based on rollout correctness (Section 5.1).
  • minimum correct counts |C_d| = 6, 3, and 1 for easy, medium, and hard
    Set to the theoretical minima implied by the difficulty thresholds; these values are used in the ECR computation (Section 5.2).
  • monitoring interval N and monitoring set size = N=20, |D_M|=500
    Periodicity of target-length updates and monitoring dataset size are chosen without reported ablations (Section 5.2).
  • candidate length enumeration step I = not reported
    Used to scan target lengths from L_T to 16,384 tokens (Section 5.2); the step size is never stated, which affects the selected adaptive target length L_A.
assumptions (4)
  • standard math GRPO and the KL-regularized RL objective in Eq. 1 provide a valid optimization framework.
    Adopted from Shao et al. [20] and Schulman et al. [19]; not proved in this paper.
  • domain assumption Rule-based correctness reward (+1, -0.5, -1) is a sufficient training signal for mathematical reasoning.
    Inherited from DeepSeek-R1 [4]; it underlies every experiment in the paper.
  • ad hoc to paper Correctness rate within k rollout responses is a reliable per-query difficulty measure.
    Difficulty buckets are defined by k/3 and 2k/3 thresholds on rollout accuracy (Section 5.1), which can be noisy with only 8 rollouts per question.
  • ad hoc to paper The smallest target length with ECR_d >= 1 on the monitoring set is the right training budget for each difficulty group.
    This is the core design principle of the automatic adapting mechanism (Section 5.2, Eq. 3); if short correct responses are unrepresentative, target lengths will be too tight.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Learn to Reason Efficiently with Adaptive Length-based Reward Shaping." pith.science (2026). https://pith.science/paper/JAQK2NIJ

@misc{pith2026250515612,
  author       = {Pith},
  title        = {Pith review of: Learn to Reason Efficiently with Adaptive Length-based Reward Shaping},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JAQK2NIJ}},
  note         = {Machine review of arXiv:2505.15612}
}
read the original abstract

Large Reasoning Models (LRMs) have shown remarkable capabilities in solving complex problems through reinforcement learning (RL), particularly by generating long reasoning traces. However, these extended outputs often exhibit substantial redundancy, which limits the efficiency of LRMs. In this paper, we investigate RL-based approaches to promote reasoning efficiency. Specifically, we first present a unified framework that formulates various efficient reasoning methods through the lens of length-based reward shaping. Building on this perspective, we propose a novel Length-bAsed StEp Reward shaping method (LASER), which employs a step function as the reward, controlled by a target length. LASER surpasses previous methods, achieving a superior Pareto-optimal balance between performance and efficiency. Next, we further extend LASER based on two key intuitions: (1) The reasoning behavior of the model evolves during training, necessitating reward specifications that are also adaptive and dynamic; (2) Rather than uniformly encouraging shorter or longer chains of thought (CoT), we posit that length-based reward shaping should be difficulty-aware i.e., it should penalize lengthy CoTs more for easy queries. This approach is expected to facilitate a combination of fast and slow thinking, leading to a better overall tradeoff. The resulting method is termed LASER-D (Dynamic and Difficulty-aware). Experiments on DeepSeek-R1-Distill-Qwen-1.5B, DeepSeek-R1-Distill-Qwen-7B, and DeepSeek-R1-Distill-Qwen-32B show that our approach significantly enhances both reasoning performance and response length efficiency. For instance, LASER-D and its variant achieve a +6.1 improvement on AIME2024 while reducing token usage by 63%. Further analysis reveals our RL-based compression produces more concise reasoning patterns with less redundant "self-reflections". Resources are at https://github.com/hkust-nlp/Laser.

Figures

Figures reproduced from arXiv: 2505.15612 by the authors.

Figure 1
Figure 1. Left: Accuracy and response length on AIME2024. For the figure of more benchmarks, please refer to Appendix A. Each point represents a single training run with different hyper-parameters. Given the high computational cost of obtaining this figure, the base model used is DeepSeek-R1-Distill-Qwen-1.5B. Results on 7B and 32B models are in §6.3. Our methods, LASER, LASER-D, and LASER-DE achieve a Pareto-optimal trade-of… view at source ↗
Figure 2
Figure 2. Performance on out-of-domain benchmarks: GPQA and average performance across all three [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Average keyword amount and response length over RL training on AIME24. The truncation method [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figures from the paper (12 more)
Figure 4
Figure 4. Figure 4: Changes in reasoning behaviors ratio and response length over RL training iterations on AIME2024. [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]
Figure 5
Figure 5. Figure 5: Pareto-optimal trade-off between accuracy and response length across various methods. Each point [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: The ratio of truncated responses in training data with 8192 tokens limit. [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: (a) Accuracy on training dataset across training iterations for different methods (b) Rewards across [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: Training prompt for our training. E.2 Training and Evaluation Details We employ the verl [21] framework for model training and Qwen-Math-Eval [25] for evaluation. During training, we set the rollout batch size to 128, conduct 8 rollouts per prompt, use a temperature of…
Figure 9
Figure 9. Figure 9: Budget-forcing inference with different methods. (a) Average accuracy with different output budget on [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]
Figure 10
Figure 10. Figure 10: Dynamics of adaptive target lengths during the training process of [PITH_FULL_IMAGE:figures/full_fig_p019_10.png]
Figure 11
Figure 11. Figure 11: Performance on out-of-domain benchmarks including GPQA [17], LSAT [29, 23], and MMLU [8]. [PITH_FULL_IMAGE:figures/full_fig_p020_11.png]
Figure 12
Figure 12. Figure 12: Prompt used to identify and analyze reasoning behaviors with [PITH_FULL_IMAGE:figures/full_fig_p023_12.png]
Figure 13
Figure 13. Figure 13: The full example of Figure 1 [PITH_FULL_IMAGE:figures/full_fig_p024_13.png]
Figure 14
Figure 14. Figure 14: Additional case study demonstrating the evolution of reasoning efficiency. In this example, the [PITH_FULL_IMAGE:figures/full_fig_p025_14.png]
Figure 15
Figure 15. Figure 15: Further example demonstrating improvements in reasoning approach [PITH_FULL_IMAGE:figures/full_fig_p026_15.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Reconsidering Overthinking: Penalizing Internal and External Redundancy in CoT Reasoning

    cs.AI 2025-08 conditional novelty 6.0 of 10

    A dual-penalty RL method that compresses chain-of-thought traces by separately penalizing internal semantic stagnation and external post-answer continuation reduces reasoning length by about 40% while preserving accur...

  2. Garbage In, Reasoning Out? Why Benchmark Scores are Unreliable and What to Do About It

    cs.CL 2025-06 conditional novelty 6.0 of 10

    A systematic human audit of SocialIQa, FauxPas-EAI and ToMi shows that benchmark scores are inflated or distorted by data flaws, rigid scoring, and sensitivity to phrasing.

  3. Numerically Optimizing Shortcuts to Adiabaticity: A Hybrid Control Strategy

    quant-ph 2026-04 unverdicted novelty 4.0 of 10

    Combining analytical shortcuts to adiabaticity with numerical optimization yields up to 1000× better two-ion separation protocols without extra experimental cost.

Reference graph

Works this paper leans on

33 extracted references · 9 canonical work pages · cited by 3 Pith papers

  1. [1]

    Aggarwal and S

    P. Aggarwal and S. Welleck. L1: Controlling how long a reasoning model thinks with reinforcement learning, 2025. URL https://arxiv.org/abs/2503.04697

    arXiv 2025

  2. [2]

    Arora and A

    D. Arora and A. Zanette. Training language models to reason efficiently, 2025. URL https://arxiv. org/abs/2502.04463

    arXiv 2025

  3. [3]

    X. Chen, J. Xu, T. Liang, Z. He, J. Pang, D. Yu, L. Song, Q. Liu, M. Zhou, Z. Zhang, R. Wang, Z. Tu, H. Mi, and D. Yu. Do not think that much for 2+3=? on the overthinking of o1-like llms, 2025. URL https://arxiv.org/abs/2412.21187

    arXiv 2025

  4. [4]

    DeepSeek-AI, D. Guo, D. Yang, H. Zhang, J. Song, R. Zhang, R. Xu, Q. Zhu, S. Ma, P. Wang, X. Bi, X. Zhang, X. Yu, Y . Wu, Z. F. Wu, Z. Gou, Z. Shao, Z. Li, Z. Gao, A. Liu, B. Xue, B. Wang, B. Wu, B. Feng, C. Lu, C. Zhao, C. Deng, C. Zhang, C. Ruan, D. Dai, D. Chen, D. Ji, E. Li, F. Lin, F. Dai, F. Luo, G. Hao, G. Chen, G. Li, H. Zhang, H. Bao, H. Xu, H. W...

    arXiv 2025

  5. [5]

    Gandhi, A

    K. Gandhi, A. Chakravarthy, A. Singh, N. Lile, and N. D. Goodman. Cognitive behaviors that enable self-improving reasoners, or, four habits of highly effective stars, 2025. URL https://arxiv.org/abs/2503. 01307

    work page 2025

  6. [6]

    S. Hao, S. Sukhbaatar, D. Su, X. Li, Z. Hu, J. Weston, and Y . Tian. Training large language models to reason in a continuous latent space, 2024. URL https://arxiv.org/abs/2412.06769

    arXiv 2024

  7. [7]

    C. He, R. Luo, Y . Bai, S. Hu, Z. Thai, J. Shen, J. Hu, X. Han, Y . Huang, Y . Zhang, J. Liu, L. Qi, Z. Liu, and M. Sun. OlympiadBench: A challenging benchmark for promoting AGI with olympiad-level bilingual 11 multimodal scientific problems. In L.-W. Ku, A. Martins, and V . Srikumar, editors,Proceedings of the 62nd Annual Meeting of the Association for C...

    2024

  8. [8]

    Hendrycks, C

    D. Hendrycks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, and J. Steinhardt. Measuring massive multitask language understanding, 2021. URL https://arxiv.org/abs/2009.03300

    arXiv 2021

Show all 33 references
  1. [9]

    Hendrycks, C

    D. Hendrycks, C. Burns, S. Kadavath, A. Arora, S. Basart, E. Tang, D. Song, and J. Steinhardt. Measuring mathematical problem solving with the math dataset. InThirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2021

    2021

  2. [10]

    B. Hou, Y . Zhang, J. Ji, Y . Liu, K. Qian, J. Andreas, and S. Chang. Thinkprune: Pruning long chain-of- thought of llms via reinforcement learning, 2025. URL https://arxiv.org/abs/2504.01296

    2025 arXiv

  3. [11]

    Kimi, A. Du, B. Gao, B. Xing, C. Jiang, C. Chen, C. Li, C. Xiao, C. Du, C. Liao, C. Tang, C. Wang, D. Zhang, E. Yuan, E. Lu, F. Tang, F. Sung, G. Wei, G. Lai, H. Guo, H. Zhu, H. Ding, H. Hu, H. Yang, H. Zhang, H. Yao, H. Zhao, H. Lu, H. Li, H. Yu, H. Gao, H. Zheng, H. Yuan, J....

    2025 arXiv

  4. [12]

    M. Luo, S. Tan, J. Wong, X. Shi, W. Y . Tang, M. Roongta, C. Cai, J. Luo, L. E. Li, R. A. Popa, and I. Stoica. Deepscaler: Surpassing o1-preview with a 1.5b model by scaling rl. https://pretty-radio-b75.notion.site/DeepScaleR-Surpassing-O1-Preview\ -with-a-1-5B-Model-by-Scalin...

    2025

  5. [13]

    Muennighoff, Z

    N. Muennighoff, Z. Yang, W. Shi, X. L. Li, L. Fei-Fei, H. Hajishirzi, L. Zettlemoyer, P. Liang, E. Candès, and T. Hashimoto. s1: Simple test-time scaling, 2025. URL https://arxiv.org/abs/2501.19393

    2025 arXiv

  6. [14]

    Munkhbat, N

    T. Munkhbat, N. Ho, S. H. Kim, Y . Yang, Y . Kim, and S.-Y . Yun. Self-training elicits concise reasoning in large language models, 2025. URL https://arxiv.org/abs/2502.20122

    2025 arXiv

  7. [15]

    Jaech, A

    OpenAI, :, A. Jaech, A. Kalai, A. Lerer, A. Richardson, A. El-Kishky, A. Low, A. Helyar, A. Madry, A. Beutel, A. Carney, A. Iftimie, A. Karpenko, A. T. Passos, A. Neitz, A. Prokofiev, A. Wei, A. Tam, A. Bennett, A. Kumar, A. Saraiva, A. Vallone, A. Duberstein, A. Kondrich, A. ...

    2024 arXiv

  8. [16]

    X. Qu, Y . Li, Z. Su, W. Sun, J. Yan, D. Liu, G. Cui, D. Liu, S. Liang, J. He, P. Li, W. Wei, J. Shao, C. Lu, Y . Zhang, X.-S. Hua, B. Zhou, and Y . Cheng. A survey of efficient reasoning for large reasoning models: Language, multimodality, and beyond, 2025. URL https://arxiv....

    2025

  9. [17]

    D. Rein, B. L. Hou, A. C. Stickland, J. Petty, R. Y . Pang, J. Dirani, J. Michael, and S. R. Bowman. Gpqa: A graduate-level google-proof q&a benchmark, 2023. URL https://arxiv.org/abs/2311.12022

    2023 arXiv

  10. [18]

    Y . Ruan, N. Band, C. J. Maddison, and T. Hashimoto. Reasoning to learn from latent thoughts, 2025. URL https://arxiv.org/abs/2503.18866

    2025

  11. [19]

    Schulman, F

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  12. [20]

    Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, M. Zhang, Y . Li, Y . Wu, and D. Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024

    2024 arXiv

  13. [21]

    Sheng, C

    G. Sheng, C. Zhang, Z. Ye, X. Wu, W. Zhang, R. Zhang, Y . Peng, H. Lin, and C. Wu. Hybridflow: A flexible and efficient rlhf framework. arXiv preprint arXiv: 2409.19256, 2024

    2024 arXiv

  14. [22]

    R. Wang, H. Wang, B. Xue, J. Pang, S. Liu, Y . Chen, J. Qiu, D. F. Wong, H. Ji, and K.-F. Wong. Harnessing the reasoning economy: A survey of efficient reasoning for large language models, 2025. URL https://arxiv.org/abs/2503.24377

    2025 arXiv

  15. [23]

    S. Wang, Z. Liu, W. Zhong, M. Zhou, Z. Wei, Z. Chen, and N. Duan. From lsat: The progress and challenges of complex reasoning. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2022

    2022

  16. [24]

    H. Xia, Y . Li, C. T. Leong, W. Wang, and W. Li. Tokenskip: Controllable chain-of-thought compression in llms, 2025. URL https://arxiv.org/abs/2502.12067

    2025

  17. [25]

    A. Yang, B. Zhang, B. Hui, B. Gao, B. Yu, C. Li, D. Liu, J. Tu, J. Zhou, J. Lin, K. Lu, M. Xue, R. Lin, T. Liu, X. Ren, and Z. Zhang. Qwen2.5-math technical report: Toward mathematical expert model via self-improvement,

  18. [26]

    E. Yeo, Y . Tong, M. Niu, G. Neubig, and X. Yue. Demystifying long chain-of-thought reasoning in llms,

  19. [27]

    Q. Yu, Z. Zhang, R. Zhu, Y . Yuan, X. Zuo, Y . Yue, T. Fan, G. Liu, L. Liu, X. Liu, H. Lin, Z. Lin, B. Ma, G. Sheng, Y . Tong, C. Zhang, M. Zhang, W. Zhang, H. Zhu, J. Zhu, J. Chen, J. Chen, C. Wang, H. Yu, W. Dai, Y . Song, X. Wei, H. Zhou, J. Liu, W.-Y . Ma, Y .-Q. Zhang, L....

    2025 arXiv

  20. [28]

    W. Zeng, Y . Huang, Q. Liu, W. Liu, K. He, Z. Ma, and J. He. Simplerl-zoo: Investigating and taming zero reinforcement learning for open base models in the wild, 2025. URL https://arxiv.org/abs/2503.18892

    2025 arXiv

  21. [29]

    <think>...</think>

    W. Zhong, S. Wang, D. Tang, Z. Xu, D. Guo, J. Wang, J. Yin, M. Zhou, and N. Duan. Ar-lsat: Investigating analytical reasoning of text, 2021. 13 A Pareto-Optimality We illustrate the efficacy-efficiency trade-off in Figure 5. Our proposed methods, LASER , LASER -D, and LASER -D...

    2021

  22. [32]

    Wait, subtracting a negative is like adding the positive, so that would be 3 + 6, which is 9

    Next, subtract y1 from y2, which is (3 - (-6)). Wait, subtracting a negative is like adding the positive, so that would be 3 + 6, which is 9. So now, I have the differences: -6 for the x-coordinate and 9 for the y-coordinate. But since we're going to square them, the negative ...

  23. [33]

    Calculate \( f(-1) \):\[f(-1) = \frac{3(-1) - 2}{-1 - 2} = \frac{-3 - 2}{-3} = \frac{-5}{-3} = \frac{5}{3}\]3

    Calculate \( f(-2) \):\[f(-2) = \frac{3(-2) - 2}{-2 - 2} = \frac{-6 - 2}{-4} = \frac{-8}{-4} = 2\]2. Calculate \( f(-1) \):\[f(-1) = \frac{3(-1) - 2}{-1 - 2} = \frac{-3 - 2}{-3} = \frac{-5}{-3} = \frac{5}{3}\]3. Calculate \( f(0) \):\[f(0) = \frac{3(0) - 2}{0 - 2} = \frac{-2}{...

  24. [2024]

    URL https://arxiv.org/abs/2409.12122

    arXiv

  25. [2025]

    URL https://arxiv.org/abs/2502.03373

    arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.