Pith. sign in

REVIEW 5 major objections 6 minor 1 cited by

Why Distillation can Outperform Zero-RL: The Role of Flexible Reasoning

T0 review · 5 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Distilling 920 AIME solutions into Qwen2.5-32B beats zero-RL on hard math benchmarks.

desk verdict A cheap 920-example distillation result that likely holds, but the zero-RL comparison needs an extraction audit and the cognitive-behavior story is confounded by response length. read the letter →

arxiv 2505.21067 v1 pith:TFN6GMG3 submitted 2025-05-27 cs.AI

Xiao Hu , Xingyu Lu , Liyuan Mao , YiFan Zhang , Tianke Zhang , Bin Wen , Fan Yang , Tingting Gao
show 1 more author
Guorui Zhou
This is my paper
classification cs.AI
keywords distillationzero-RLlargelanguagemodelsmathematicalreasoningmulti-perspectivethinkingmetacognitiveawarenessflexiblesupervisedfine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that a simple distillation recipe—supervised fine-tuning the Qwen2.5-32B base model on 920 AIME problems solved by a stronger teacher—can beat zero-RL, which trains the same base with tens of thousands of prompts and far more compute. The claim is not just a performance comparison: the paper identifies why distillation wins. Its answer is flexible reasoning, marked by anthropomorphic expressions and logical connectors that accompany two advanced cognitive behaviors, Multi-Perspective Thinking or Attempting and Metacognitive Awareness, which zero-RL fails to amplify. Blocking the distinctive tokens degrades the distilled model but does not erase its advantage, suggesting the behaviors are internalized rather than surface imitation.

What carries the argument

The load-bearing object is the 920-example distillation set constructed from all historical AIME problems (1983-2023), each paired with one DeepSeek R1 solution and used for supervised fine-tuning of Qwen2.5-32B. The argument then runs through two analytic instruments: token-frequency categorization (anthropomorphic tokens, logical connectors, and mathematical reasoning tokens) and GPT-4o-based counting of two advanced cognitive behaviors, Multi-Perspective Thinking or Attempting and Metacognitive Awareness. The token-restriction decoding experiment is the mechanism that separates surface imitation from internalized behavior: banning 'wait', 'maybe', 'alternatively', and related tokens drops performance but leaves the model ahead of zero-RL, and the model works around the ban, sometimes switching to Chinese, to express the same reasoning shifts.

What would settle it

Train a distilled model on the same 920 prompts but with teacher responses stripped of all anthropomorphic tokens and logical connectors; if that model still matches the full distilled model's AIME2025 score and cognitive-behavior counts, the paper's central mechanism is wrong.

Watch

Extended reading notes

Core claim

The central discovery is that fewer than a thousand unfiltered teacher examples can transfer a reasoning style that large-scale zero-RL does not produce. Starting from the same Qwen2.5-32B base, the authors distill all 920 AIME problems from 1983 to 2023 using DeepSeek R1 responses, without filtering for correctness, and fine-tune for five epochs. The resulting model scores 50.0 Avg@32 on AIME2025 versus 32.9 for DAPO-32B, and leads across AIME2024, HMMT Feb 2025, GPQA Diamond, and MATH500. Token-frequency analysis shows the distilled model mirrors the teacher's use of words such as 'wait', 'maybe', 'but', and 'alternatively', while zero-RL models keep the base model's rigid step-by-step style. The authors trace these surface differences to deeper habits: the distilled model exhibits many more instances of Multi-Perspective Thinking or Attempting and Metacognitive Awareness, and these behaviors correlate with benchmark scores. Even when the distinctive tokens are blocked during decoding, the model preserves part of its advantage and finds new ways to express the same behaviors, which the authors take as evidence that distillation teaches reasoning processes, not just word patterns.

Load-bearing premise

The central comparison assumes that the three released zero-RL checkpoints are fair, representative instances of zero-RL on Qwen2.5-32B and that the evaluation does not systematically disadvantage them.

Editorial extensions

If this is right

  • On the same Qwen2.5-32B base, supervised fine-tuning on 920 unfiltered AIME solutions outperforms three zero-RL baselines on AIME2024, AIME2025, HMMT Feb 2025, GPQA Diamond, and MATH500.
  • The distilled model's advantage holds at temperature 0.6 and extends to MMLU-Pro and MMLU-STEM, despite the training data being mathematics-only.
  • Blocking distinctive tokens such as 'wait', 'maybe', 'alternatively', and 'but' during decoding lowers the distilled model's scores, with the largest drop on harder problems, but the restricted model still remains comparable to zero-RL.
  • The token-restricted model retains more instances of the two advanced cognitive behaviors than zero-RL and works around the ban, sometimes switching to Chinese, which the paper reads as evidence that the behaviors are internalized.
  • Zero-RL checkpoints show signs of reward hacking and overfitting, such as ending reasoning right after 'let's' or emitting integer answers when the training set contained only integer answers.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A concrete testable extension is to run zero-RL with an explicit reward term for Multi-Perspective Thinking or Attempting and Metacognitive Awareness; if the gap with distillation narrows, the paper's behavioral explanation is confirmed rather than stylistic.
  • The token-restriction results imply that answer-extraction and prompt-template choices can change who appears to win, so a fair head-to-head should report each zero-RL checkpoint under its own intended decoding format, which the paper attempts but the community could audit independently.
  • Because the distillation data are unfiltered historical AIME problems from a single teacher, the result suggests that data quantity is not the bottleneck; a cheap recipe for other domains is to collect a few hundred hard, representative problems solved by a strong teacher and look for the two behaviors before scaling data.
  • The observed transfer beyond mathematics, to MMLU-Pro and MMLU-STEM, hints that the two cognitive behaviors are domain-general; testing on code and science reasoning benchmarks would show whether the same mechanism explains improvements there.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper reports that supervised fine-tuning of Qwen2.5-32B on 920 DeepSeek-R1-generated solutions to historical AIME problems (with no correctness filtering) produces a model that outperforms three released zero-RL checkpoints (DAPO-32B, ORZ-32B, SimpleRL-32B) on AIME2024, AIME2025, HMMT, GPQA Diamond, and MATH500. The authors attribute the gap to 'flexible reasoning', operationalized through token-frequency differences (anthropomorphic tokens and logical connectors) and two GPT-4o-counted cognitive behaviors (Multi-Perspective Thinking or Attempting; Metacognitive Awareness), and they support this with a token-blocking decoding ablation. The paper also discusses reward hacking and overfitting in zero-RL outputs and proposes that distilled models are better starting points for subsequent RL.

Significance. If the comparison is fair, the headline result is practically significant: it suggests that a small amount of teacher-distilled SFT can match or beat large-scale RL on a 32B base model, with much lower compute. The paper's strengths include a direct benchmark against public checkpoints, detailed evaluation settings (Appendix B and C), a temperature-0.6 robustness check, and a falsifiable token-blocking probe. However, the zero-RL arm relies on third-party checkpoints evaluated with a single answer-extraction harness, and the cognitive-behavior analyses are confounded by response length. These issues currently prevent the paper from establishing its strongest claims as stated.

major comments (5)
  1. [Section 3, Appendix C.1, Table 1] The comparison against zero-RL is the load-bearing evidence for the abstract, but it rests on an incomplete answer-extraction audit. Appendix C.1 documents that the Qwen2.5-Math harness, which prioritizes answers inside \boxed, was adapted for DAPO-32B by extracting the text following 'Answer:', yet no corresponding adaptation is described for ORZ-32B, whose official prompt (Table 11) requires the answer inside <answer> tags. If ORZ-32B outputs are not boxed, even a modest extraction failure rate on the 30-problem AIME/HMMT sets would move its scores by several points and erode the margins shown in Table 1 (e.g., 41.9 vs 61.2 on AIME2024). Please report per-model raw extraction success rates and, if needed, use each model's native answer format with a matching parser.
  2. [Section 3, Table 1] The headline 'clearly outperform' is supported only by point estimates on 30-problem benchmarks, with no confidence intervals or significance tests. With only 30 problems, the 61.2 vs 50.6 gap on AIME2024 corresponds to roughly three to four additional solved problems, and the observed difference is within the range of sampling variability. Please report bootstrap confidence intervals (or per-problem pass rates) for Avg@32 and Pass@8(40), and temper the abstract's 'clearly' wording accordingly.
  3. [Section 4.1, Tables 1 and 3] Table 3 reports a baseline AIME2025 Avg@32 of 52.9 for the distilled model, while Table 1 reports 50.0 for the same model and same evaluation setting. This discrepancy is not explained. Because the token-restriction delta (-14.9) is computed against this 52.9 baseline, the internal consistency of the ablation depends on resolving which number is correct; if the baseline differs by evaluation seed or sampling, the delta must be recomputed on a matched baseline.
  4. [Section 4.2, Figure 4, Table 1] The cognitive-behavior counts are reported per response, but the distilled model's responses are substantially longer than those of the zero-RL models (e.g., AIME2024 average length 13,975 vs 7,916 for DAPO and 10,174 for ORZ). Without normalizing by output length, the higher counts in Figure 4 may simply reflect the fact that the distilled model writes more tokens. Please report per-token densities (or a length-controlled analysis) before concluding that distillation 'enhances' these behaviors.
  5. [Section 4.2, Tables 3-4, Appendix D.4] The explanatory claim that the two cognitive behaviors 'give rise to' flexible reasoning is partly circular and partly correlational. The behaviors are operationalized through the same linguistic markers (e.g., 'wait', 'alternatively', 'maybe') that are used to define the distinctive tokens, and the token-restriction ablation blocks exactly those markers; the drop in GPT-4o counts in Table 4 is therefore in part definitional. Moreover, no evidence distinguishes 'behaviors cause performance' from 'longer, more exploratory responses contain more such phrases'. A falsifiable operationalization that is not tied to the banned-token list, or a mediation analysis controlling for length and base rates, would be needed to support the causal language in Section 5.
minor comments (6)
  1. [Section 3] 'GQPA Diamond' should be 'GPQA Diamond'.
  2. [Appendix B.2] 'consine' should be 'cosine'; Table 9's '3s hours' should be '3 hours'; and 'the the rest' contains a duplicated article.
  3. [Section 4.1] 'the distill model' should be 'the distilled model'.
  4. [Appendix C.1] The in-text references to Table 10 and Table 11 are swapped relative to the table captions: the caption for Table 10 is DAPO-32B, but the text assigns DAPO-32B to Table 11. Please correct the cross-references.
  5. [Section 4.2, Figure 4] No error bars or inter-annotator statistics are reported for the GPT-4o counts, despite the acknowledged instability of LLM-as-a-judge; please add variance estimates.
  6. [Section 3] The sentence noting that some zero-RL training samples include a subset of historical AIME problems should be reconciled with the abstract's 'much more data' claim, since distillation data include teacher solutions and are not directly comparable in sample count.

Circularity Check

1 steps flagged · score 5.0 of 10

Central benchmark comparison is not circular, but the explanatory mechanism is partially circular: the 'advanced cognitive behaviors' are measured with the same linguistic markers that defined the distilled/zero-RL style difference.

  1. self definitional [Section 4.2 (paragraph after the two behavior definitions) and Appendix D.4 (GPT-4o prompt)]
    "Both of the advanced cognitive behaviors are reflected through certain key phrases, which can be interpreted in context. For example, expressions like 'let's try another angle...' or 'but I need a better strategy ... here's an idea, let's try...' often indicate Multi-Perspective Thinking or Attempting; and expressions such as 'wait, maybe my approach is wrong here' or 'it seems not correct, step back' typically indicate Metacognitive Awareness. We use GPT-4o to identify the number of occurrences of advanced cognitive behaviors in model responses."

    These behaviors are operationalized in the GPT-4o prompt of Appendix D.4 with example phrases composed almost entirely of the 'distinctive tokens' that Section 4.1 selected as having the largest frequency differences between the distilled and zero-RL models ('wait', 'maybe', 'alternatively', 'but', 'another', 'seems'). Hence the finding that the distilled model shows more Multi-Perspective Thinking and Metacognitive Awareness is in large part a restatement of the token-frequency difference, not an independent confirmation that distillation enhanced these behaviors. The token-restriction control (Table 4) partially mitigates this because behavior counts remain above zero-RL even when those tokens are banned, so the reduction is partial rather than total.

full rationale

The paper's headline claim (920 distilled examples outperform zero-RL on AIME2024/2025, HMMT, GPQA) rests on a direct external benchmark comparison with released third-party checkpoints; no parameter is fitted to the reported scores and no self-citation is load-bearing, so that claim is not circular. Concerns that the ORZ-32B arm may have been disadvantaged by answer-extraction choices are evaluation-fairness/correctness risks, not circularity. The one partial circularity is in the explanatory chapter: the two 'advanced cognitive behaviors' are counted by GPT-4o using example phrases built from the same distinctive tokens that defined the distilled-vs-zero-RL stylistic difference, so the conclusion 'distillation enhances these behaviors' is partly descriptive of the token-frequency analysis. The token-restriction experiment provides some independent evidence, which keeps this from being a full reduction. Overall score 5.

Assumptions & free parameters 1 free parameters · 5 assumptions · 2 invented entities

The performance comparison itself introduces no fitted numeric parameters. The explanatory story introduces two hand-defined behavioral categories and a hand-chosen token ban list, neither of which has external validation.

free parameters (1)
  • Distinctive token ban list = 'wait', 'me', 'perhaps', 'maybe', 'alternatively', 'but', 'another', 'hold on', 'hmm', 'alternate', 'alternately'…
    Hand-selected as the tokens with the largest frequency gap between distilled and zero-RL outputs; the measured performance drop in Table 3 depends on this selection.
assumptions (5)
  • domain assumption The three zero-RL checkpoints are representative, fairly evaluated instances of zero-RL on Qwen2.5-32B.
    Section 3 compares only against released DAPO-32B, ORZ-32B, SimpleRL-32B; any weakness in those models or mismatches in their intended evaluation reduces the strength of the outperformance claim.
  • domain assumption The Qwen2.5-Math evaluation harness with per-model answer extraction yields unbiased accuracy estimates.
    Section C.1 describes adaptations to DAPO's 'Answer:' format; the assumption is that no systematic bias favors the distilled model.
  • domain assumption GPT-4o can reliably count the two cognitive behaviors from model responses.
    Section 4.2 and Appendix D.4 rely on GPT-4o annotations sampled over a few responses; no human inter-annotator agreement is reported.
  • ad hoc to paper The two cognitive behaviors are the correct decomposition of flexible reasoning.
    Multi-Perspective Thinking or Attempting and Metacognitive Awareness are defined by the authors for this paper; the mapping from them to performance is assumed, not derived.
  • domain assumption Training on 920 AIME solutions, including about 14.6 percent incorrect ones, is a valid distillation signal.
    The paper keeps all samples regardless of correctness; the high downstream performance is used to infer that the reasoning style matters, but this is an assumption about what SFT learns.
invented entities (2)
  • Multi-Perspective Thinking or Attempting
    purpose: A named cognitive behavior used to explain the distilled model's superior performance; it is counted by GPT-4o from phrases like 'try another angle'.
    No external, validated measurement exists; the entity is operationalized only through the authors' prompt to GPT-4o.
  • Metacognitive Awareness (as operationalized here)
    purpose: Used to explain why the distilled model catches and corrects errors; counted by GPT-4o using phrases like 'wait, maybe my approach is wrong'.
    Although metacognition is an existing psychological construct, the paper's counting procedure is paper-specific and not validated against human annotation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Why Distillation can Outperform Zero-RL: The Role of Flexible Reasoning." pith.science (2026). https://pith.science/paper/TFN6GMG3

@misc{pith2026250521067,
  author       = {Pith},
  title        = {Pith review of: Why Distillation can Outperform Zero-RL: The Role of Flexible Reasoning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TFN6GMG3}},
  note         = {Machine review of arXiv:2505.21067}
}
read the original abstract

Reinforcement learning (RL) has played an important role in improving the reasoning ability of large language models (LLMs). Some studies apply RL directly to \textit{smaller} base models (known as zero-RL) and also achieve notable progress. However, in this paper, we show that using only 920 examples, a simple distillation method based on the base model can clearly outperform zero-RL, which typically requires much more data and computational cost. By analyzing the token frequency in model outputs, we find that the distilled model shows more flexible reasoning. It uses anthropomorphic tokens and logical connectors much more often than the zero-RL model. Further analysis reveals that distillation enhances the presence of two advanced cognitive behaviors: Multi-Perspective Thinking or Attempting and Metacognitive Awareness. Frequent occurrences of these two advanced cognitive behaviors give rise to flexible reasoning, which is essential for solving complex reasoning problems, while zero-RL fails to significantly boost the frequency of these behaviors.

Figures

Figures reproduced from arXiv: 2505.21067 by the authors.

Figure 1
Figure 1. Comparison of token usage between the Distilled and zero-RL models responses to [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Token usage in Qwen2.5-32B-base’s responses to AIME2024 problems across anthropo [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Token usage in DeepSeek R1’s responses to AIME2024 problems across anthropomorphic [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Comparison of the number of advanced cognitive behaviors per response across benchmarks. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Response length distribution of DeepSeek R1 on 920 distillation problems. [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: Training curves for our distillation [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: Comparison of the number of advanced cognitive behaviors per response on HMMT Feb [PITH_FULL_IMAGE:figures/full_fig_p021_7.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. rStar2-Agent: Agentic Reasoning Technical Report

    cs.CL 2025-08 conditional novelty 7.0 of 10

    A 14B model trained with agentic RL and a resample-on-correct rollout strategy scores 80.6% on AIME24 and 69.8% on AIME25, nearly matching DeepSeek-R1 (671B) in one week on 64 GPUs.

Reference graph

Works this paper leans on

42 extracted references · 12 canonical work pages · cited by 1 Pith paper

  1. [1]

    Openai o1 system card.arXiv preprint arXiv:2412.16720, 2024

    Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai o1 system card.arXiv preprint arXiv:2412.16720, 2024

    arXiv 2024

  2. [2]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    arXiv 2025

  3. [3]

    Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi k1. 5: Scaling reinforcement learning with llms.arXiv preprint arXiv:2501.12599, 2025

    arXiv 2025

  4. [4]

    Qwq-32b: Embracing the power of reinforcement learning, March 2025

    Qwen Team. Qwq-32b: Embracing the power of reinforcement learning, March 2025. Accessed: 2025-05-09

    work page 2025

  5. [5]

    Gemini 2.5 pro: Our most advanced reasoning model, March 2025

    Google DeepMind. Gemini 2.5 pro: Our most advanced reasoning model, March 2025. Accessed: 2025-05-09

    work page 2025

  6. [6]

    Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437, 2024

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437, 2024

    arXiv 2024

  7. [7]

    Simplerl-zoo: Investigating and taming zero reinforcement learning for open base models in the wild.arXiv preprint arXiv:2503.18892, 2025

    Weihao Zeng, Yuzhen Huang, Qian Liu, Wei Liu, Keqing He, Zejun Ma, and Junxian He. Simplerl-zoo: Investigating and taming zero reinforcement learning for open base models in the wild.arXiv preprint arXiv:2503.18892, 2025

    arXiv 2025

  8. [8]

    Limr: Less is more for rl scaling.arXiv preprint arXiv:2502.11886, 2025

    Xuefeng Li, Haoyang Zou, and Pengfei Liu. Limr: Less is more for rl scaling.arXiv preprint arXiv:2502.11886, 2025

    arXiv 2025

Show all 42 references
  1. [9]

    Dapo: An open-source llm reinforcement learning system at scale.arXiv preprint arXiv:2503.14476, 2025

    Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, et al. Dapo: An open-source llm reinforcement learning system at scale.arXiv preprint arXiv:2503.14476, 2025

    2025 arXiv

  2. [10]

    Open-reasoner-zero: An open source approach to scaling up reinforcement learning on the base model.arXiv preprint arXiv:2503.24290, 2025

    Jingcheng Hu, Yinmin Zhang, Qi Han, Daxin Jiang, Xiangyu Zhang, and Heung-Yeung Shum. Open-reasoner-zero: An open source approach to scaling up reinforcement learning on the base model.arXiv preprint arXiv:2503.24290, 2025

    2025 arXiv

  3. [11]

    Understanding r1-zero-like training: A critical perspective.arXiv preprint arXiv:2503.20783, 2025

    Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. Understanding r1-zero-like training: A critical perspective.arXiv preprint arXiv:2503.20783, 2025

    2025 arXiv

  4. [12]

    Vapo: Efficient and reliable reinforcement learning for advanced reasoning tasks.arXiv preprint arXiv:2504.05118, 2025

    Yufeng Yuan, Qiying Yu, Xiaochen Zuo, Ruofei Zhu, Wenyuan Xu, Jiaze Chen, Chengyi Wang, TianTian Fan, Zhengyin Du, Xiangpeng Wei, et al. Vapo: Efficient and reliable reinforcement learning for advanced reasoning tasks.arXiv preprint arXiv:2504.05118, 2025

    2025 arXiv

  5. [13]

    Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model? arXiv preprint arXiv:2504.13837, 2025

    Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Shiji Song, and Gao Huang. Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model? arXiv preprint arXiv:2504.13837, 2025

    2025 arXiv

  6. [14]

    s1: Simple test-time scaling.arXiv preprint arXiv:2501.19393, 2025

    Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. s1: Simple test-time scaling.arXiv preprint arXiv:2501.19393, 2025

    2025 arXiv

  7. [15]

    Limo: Less is more for reasoning.arXiv preprint arXiv:2502.03387, 2025

    Yixin Ye, Zhen Huang, Yang Xiao, Ethan Chern, Shijie Xia, and Pengfei Liu. Limo: Less is more for reasoning.arXiv preprint arXiv:2502.03387, 2025

    2025 arXiv

  8. [16]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report.arXiv preprint arXiv:2412.15115, 2024

    2024 arXiv

  9. [17]

    Step- dpo: Step-wise preference optimization for long-chain reasoning of llms.arXiv preprint arXiv:2406.18629, 2024

    Xin Lai, Zhuotao Tian, Yukang Chen, Senqiao Yang, Xiangru Peng, and Jiaya Jia. Step- dpo: Step-wise preference optimization for long-chain reasoning of llms.arXiv preprint arXiv:2406.18629, 2024. 11

    2024 arXiv

  10. [18]

    Training language models to follow instructions with human feedback.Advances in neural information processing systems, 35:27730–27744, 2022

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback.Advances in neural information processing systems, 35:27730...

    2022

  11. [19]

    Training a helpful and harmless assistant with reinforcement learning from human feedback.arXiv preprint arXiv:2204.05862, 2022

    Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. Training a helpful and harmless assistant with reinforcement learning from human feedback.arXiv preprint arXiv:2204.05862, 2022

    2022 arXiv

  12. [20]

    Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347, 2017

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  13. [21]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

    2024 arXiv

  14. [22]

    Understanding aha moments: from external observations to internal mechanisms.arXiv preprint arXiv:2504.02956, 2025

    Shu Yang, Junchao Wu, Xin Chen, Yunze Xiao, Xinyi Yang, Derek F Wong, and Di Wang. Understanding aha moments: from external observations to internal mechanisms.arXiv preprint arXiv:2504.02956, 2025

    2025 arXiv

  15. [23]

    Bespoke-stratos: The unreasonable effectiveness of reasoning distilla- tion

    Bespoke Labs. Bespoke-stratos: The unreasonable effectiveness of reasoning distilla- tion. www.bespokelabs.ai/blog/bespoke-stratos-the-unreasonable-effectiveness-of-reasoning- distillation, 2025. Accessed: 2025-01-22

    2025

  16. [24]

    Redstar: Does scaling long-cot data unlock better slow-reasoning systems?arXiv preprint arXiv:2501.11284, 2025

    Haotian Xu, Xing Wu, Weinong Wang, Zhongzhi Li, Da Zheng, Boyuan Chen, Yi Hu, Shijia Kang, Jiaming Ji, Yingying Zhang, et al. Redstar: Does scaling long-cot data unlock better slow-reasoning systems?arXiv preprint arXiv:2501.11284, 2025

    2025 arXiv

  17. [25]

    An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, et al. Qwen2. 5-math technical report: Toward mathematical expert model via self-improvement.arXiv preprint arXiv:2409.12122, 2024

    2024 arXiv

  18. [26]

    American invitational mathematics examination 2024 part 1, 2024

    part I. American invitational mathematics examination 2024 part 1, 2024

    2024

  19. [27]

    American invitational mathematics examination 2024 part 2, 2024

    part II. American invitational mathematics examination 2024 part 2, 2024

    2024

  20. [28]

    American invitational mathematics examination 2025 part 1, 2025

    part I. American invitational mathematics examination 2025 part 1, 2025

    2025

  21. [29]

    American invitational mathematics examination 2025 part 2, 2025

    part II. American invitational mathematics examination 2025 part 2, 2025

    2025

  22. [30]

    Hmmt february 2025 dataset

    MathArena Team. Hmmt february 2025 dataset. https://huggingface.co/datasets/ MathArena/hmmt_feb_2025, 2025. Accessed: 2025-05-16

    2025

  23. [31]

    Gpqa: A graduate-level google-proof q&a benchmark

    David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. Gpqa: A graduate-level google-proof q&a benchmark. InFirst Conference on Language Modeling, 2024

    2024

  24. [32]

    Measuring mathematical problem solving with the math dataset

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874, 2021

    2021 arXiv

  25. [33]

    A sober look at progress in language model reasoning: Pitfalls and paths to reproducibility.arXiv preprint arXiv:2504.07086, 2025

    Andreas Hochlehnert, Hardik Bhatnagar, Vishaal Udandarao, Samuel Albanie, Ameya Prabhu, and Matthias Bethge. A sober look at progress in language model reasoning: Pitfalls and paths to reproducibility.arXiv preprint arXiv:2504.07086, 2025

    2025

  26. [34]

    Evaluating large language models trained on code.arXiv preprint arXiv:2107.03374, 2021

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code.arXiv preprint arXiv:2107.03374, 2021

    2021 arXiv

  27. [35]

    Cognitive behaviors that enable self-improving reasoners, or, four habits of highly effective stars.arXiv preprint arXiv:2503.01307, 2025

    Kanishk Gandhi, Ayush Chakravarthy, Anikait Singh, Nathan Lile, and Noah D Goodman. Cognitive behaviors that enable self-improving reasoners, or, four habits of highly effective stars.arXiv preprint arXiv:2503.01307, 2025. 12

    2025 arXiv

  28. [36]

    Assessing metacognitive awareness.Contem- porary educational psychology, 19(4):460–475, 1994

    Gregory Schraw and Rayne Sperling Dennison. Assessing metacognitive awareness.Contem- porary educational psychology, 19(4):460–475, 1994

    1994

  29. [37]

    Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024

    2024 arXiv

  30. [38]

    Mmlu-pro: A more robust and challenging multi-task language understanding benchmark

    Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, et al. Mmlu-pro: A more robust and challenging multi-task language understanding benchmark. InThe Thirty-eight Conference on Neural Information Pro...

    2024

  31. [39]

    Qwen2.5-math-cot

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding.arXiv preprint arXiv:2009.03300, 2020. 13 A Limitation Our work also has some limitations. First, our work highlight the im...

    2009 arXiv

  32. [40]

    Qwen-boxed

    demonstrates that performing SFT with 800,000 examples can significantly enhance the base model’s performance. C Evaluation Details and More Results As pointed out in [33], many detailed evaluation parameters can influence the results, especially on datasets like AIME or HMMT,...

    2025

  33. [41]

    let’s try another angle

    *Multi-Perspective Thinking or Attempting*: Viewing a problem from diverse perspectives to gain fresh insights, or exploring different ideas and alternative approaches to make meaningful progress. For example, expressions like "let’s try another angle..." and "but I need a bet...

  34. [42]

    wait, maybe my approach is wrong here

    *Metacognitive Awareness*: Actively reflecting on your reasoning process during problem-solving to assess progress, evaluate current strategies, and identify potential errors in real time. Any reflective hesitation, backtracking, and verification are indicative of this awarene...

    2025

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.