Pith. sign in

REVIEW 4 major objections 6 minor 43 references

One Missing Piece for Open-Source Reasoning Models: A Dataset to Mitigate Cold-Starting Short CoT LLMs in RL

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that a 100K long chain-of-thought dataset can be generated by short-CoT LLMs, guided by 1K reasoning-flow outlines from o1, and that this dataset matches R1-level quality while serving as a cold-start foundation for…

desk verdict A genuinely useful dataset and construction pipeline for distillation-free long CoT, but the headline RLVR gain is mostly a reward-compatibility artifact and needs a controlled comparison. read the letter →

arxiv 2506.02338 v1 pith:LJMYUFHB submitted 2025-06-03 cs.CL

Hyungjoo Chae , Dongjin Kang , Jihyuk Kim , Beong-woo Kwak , Sunghyun Park , Haeju Park , Jinyoung Yeo , Moontae Lee
show 1 more author
Kyungjae Lee
This is my paper
classification cs.CL
keywords longchain-of-thoughtreasoningflowdatasetconstructionreinforcementlearningwithverifiablerewardsLLMthoughtbudgetoverthinkingcold-start
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether long chain-of-thought (CoT) reasoning data needed for training reasoning models can be produced by ordinary short-CoT LLMs, without distilling from an existing large reasoning model like R1. It answers yes: a 100K-instance dataset, the Long CoT Collection, is generated by GPT-4o guided by 1K reasoning-flow outlines harvested from o1. The dataset reaches quality judged comparable to R1's outputs and, when used to cold-start a small policy before reinforcement learning with verifiable rewards, yields 2-3x larger gains than starting from the base model. This matters because it decouples open-source reasoning-model development from dependence on closed long-CoT teachers.

What carries the argument

The reasoning flow, a sequence of outline phrases such as 'Analyzing the problem statement' and 'Okay, let's verify', summarizes the high-level reasoning strategy. The pipeline extracts 1K such flows with thought budgets from o1, retrieves domain- and budget-matched demonstrations for each new question, has GPT-4o generate a flow by imitating the retrieved structure, expands each flow step into an actual long CoT rationale conditioned on previous and next steps, and filters out wrong-answer rationales. The flow carries the teacher's verification and exploration strategies, and the number of outline steps provides the thought-budget control knob.

What would settle it

Take the Long CoT Collection and shuffle the order of the outline steps before expansion, or replace the o1-extracted outlines with generic templates while keeping everything else identical; if the downstream SFT-then-RLVR gains on MATH500 and GPQA remain the same, the specific reasoning flow is not the causal ingredient, whereas if they drop sharply, the flow is doing the work.

Watch

Extended reading notes

Core claim

The central claim is that o1's reasoning competence can be captured at the level of a 'reasoning flow'—a compact sequence of outline steps describing how a problem is decomposed and checked—and that this outline, plus a thought-budget estimate, is enough to make a short-CoT LLM (GPT-4o) produce long CoT rationales that rival R1's in quality. The resulting Long CoT Collection is not only useful for supervised fine-tuning; it is a reliable cold-start foundation for RLVR, with models initialized on it showing 2-3x greater performance gains on MATH500 and GPQA than base-model-initialized policies. The paper also demonstrates thought-budget controllability, letting the data generator adjust how much thinking is spent per problem, which addresses LRM overthinking.

Load-bearing premise

The load-bearing premise is that a compact reasoning-flow outline plus a thought-budget number captures enough of o1's reasoning competence that GPT-4o's step-by-step expansion preserves the real strategies rather than producing plausible filler; if the outline loses the substance of o1's reasoning, the dataset inherits only surface patterns and the RL gains would not come from learned reasoning strategies.

Editorial extensions

If this is right

  • Long CoT datasets can be built by short-CoT LLMs, so future open reasoning models need not distill from R1 or other closed long-CoT systems.
  • Thought budget can be set at data-generation time; training on 100%-budget data beats 50% and 25% budgets on MATH500, suggesting the budget knob trades compute against reasoning quality.
  • Cold-starting RLVR with the Long CoT Collection gives 2-3x larger performance gains than starting from the base model, reducing the sparse-reward cold-start problem.
  • The collection transfers to general reasoning benchmarks (GPQA, MMLU-Pro), not just math.
  • Models trained on the collection show higher length, answer, and format rewards than the base in a LIMO-based reward analysis, indicating a better starting policy for RL.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If reasoning-flow outlines can be obtained cheaply from any capable model or even from text heuristics, the pipeline could be applied to domains beyond math and general reasoning, such as coding or scientific QA, without needing a long-CoT teacher.
  • The thought-budget mechanism suggests a potential curriculum: train first on high-budget data, then distill or compress to low budget, which might yield models that think just long enough and mitigate overthinking.
  • A direct test of the paper's causal story would be to shuffle the outline order or replace o1 flows with generic problem-solving templates; if downstream RL gains persist, then the specific o1 strategies are not the active ingredient.
  • Because the dataset is generated by GPT-4o, its ceiling is bounded by GPT-4o's base competence; the method's value likely grows as the generator model's base reasoning improves.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces the Long CoT Collection, a 100K-instance dataset of long chain-of-thought (CoT) rationales generated by short-CoT LLMs (GPT-4o) under the guidance of a 1K seed dataset of o1 reasoning flows and thought budgets. The authors propose a three-stage pipeline (retrieval, reasoning-flow generation, step-by-step rationale generation) that transfers o1's reasoning structure to short-CoT LLMs while allowing control over thought length. They claim the resulting dataset is comparable in quality to DeepSeek-R1, improves SFT performance on GPQA and MMLU-Pro, and acts as a reliable foundation for reinforcement learning, with models initialized on it achieving 2-3x larger RLVR gains than base-model initialization. The paper publicly releases code, data, and models.

Significance. If the claims hold, the work provides a method to create long-CoT reasoning data without direct distillation from existing LRMs, which is a meaningful step toward independent open-source LRM development. The thought-budget controllability is a genuine and useful contribution, and the detailed pipeline description, public artifact release, and BoN/SFT results are strengths. However, the headline RLVR claim is currently confounded by the length-reward design, and the quality comparison against R1 is biased by question selection. The paper itself concedes in Section 7 that SFT on the dataset is not dramatically better than direct distillation, so the RL-initialization claim is the central differentiator and must be supported by properly controlled experiments. With additional controls, the dataset could be a valuable community resource.

major comments (4)
  1. [Section 5.3, Figure 1] The headline RLVR comparison is confounded by the length reward. The length reward is defined as 1 - |min(x,y)/max(x,y)-1| with y being o1-mini's thought length, while the Long CoT Collection is constructed using thought budgets matched to the same reference (Sections 3.2.1 and 6.1). Consequently, the LC-initialized policy already starts with outputs near the length target, whereas the base Qwen-2.5-0.5B does not. Under GRPO's group-relative advantages, the base model's early learning signal is dominated by length matching rather than by solving the problem, so the reported 2-3x larger gains likely measure reward-shaping compatibility rather than a better cold-start foundation. The authors should add RLVR runs with answer+format rewards only (no length reward) and, ideally, a control initialization on R1-distilled or other long-CoT SFT data. Without these controls, the central claim that the collection is a 'reliable foundation for RL' is not supported.
  2. [Section 4.1, Figure 4] The quality comparison against R1 is based on 100 questions selected 'for which R1-generated solutions have the correct answer.' This conditioning removes all cases where R1 is incorrect, biasing the comparison in R1's favor and preventing any conclusion about the dataset's quality on the underlying distribution. To support the claim that the Long CoT Collection is 'comparable to—or slightly below—R1,' the evaluation should be run on a random sample or stratified by R1 correctness, with results reported separately for cases where R1 is correct and incorrect.
  3. [Section 4.1, Table 3] Table 3 reports that self-verification triggers appear far less frequently in the generated rationales than in R1: 'Wait' appears in only 4% of Ours versus 100% of DeepSeek-R1, and 'Verif-' in 27% versus 60%. This is hard to reconcile with the paper's claim that the dataset contains 'rich reasoning triggers' that 'help explore diverse reasoning paths.' The o3-mini-based qualitative evaluation may reward outline-like structure rather than genuine reasoning behavior. A behavioral test, such as measuring the frequency of self-correction or the accuracy improvement on problems requiring verification, would provide more direct evidence that the reasoning-flow transfer preserves o1's strategies.
  4. [Section 5.3 and Limitations] The RLVR experiment is conducted only on Qwen-2.5-0.5B, while the SFT and BoN experiments use 7B-8B models. The abstract and Section 5.3 make an unqualified claim that 'models initialized on our data achieve 2-3x larger gains with RLVR,' which is not demonstrated at the model scale where the dataset is most likely to be deployed. While the limitation section acknowledges the GPU constraint, the claim should be qualified to the 0.5B setting or replicated at a larger scale before generalizing.
minor comments (6)
  1. [Section 2] 'dataset volumns' should be 'dataset volumes.'
  2. [Section 5.3] 'the parable answer span' should be 'the parseable answer span' or 'the answer span.'
  3. [Figure 7 caption] The caption says 'Pearson correlation (R2)'—Pearson correlation is conventionally denoted r, while R2 is the coefficient of determination; please clarify which statistic is reported.
  4. [Section 3.1.1] The notation Dref ∈ {q, Sref, bref} is ambiguous; it should be Dref = {(q, Sref, bref)} or similar to indicate a set of tuples.
  5. [Appendix A.1] The dataset is referred to as both 'Magpie-Reasoning-150K' and 'magpie-reasoning-V1'; use a single consistent name throughout.
  6. [Table 5 caption] 'NunimaMath-CoT' should be 'NuminaMath-CoT.'

Circularity Check

1 steps flagged · score 6.0 of 10

The headline 2-3x RLVR gain is confounded: the same o1-mini thought-budget distance is used both to construct the SFT dataset and as the RL length reward.

  1. self definitional [Section 3.2.1 / Section 5.3 (Figure 1)]
    "(2) Thought budget control: To align with reference LRMs, the thought budget is controlled by retrieving reasoning flows of similar length for demonstration. We measure this similarity using 1 − |min(x,y)/max(x,y) − 1|, where x and y represent the reference and candidate budgets, respectively. ... (1) Length Reward: We use the function 1 − |min(x,y)/max(x,y) − 1| that measures the difference between the length of sampled thought and o1-mini’s thought on a scale of 0 to 1."

    The same budget-distance formula is used twice: first as the retrieval criterion that makes the Long CoT Collection match o1-mini thought lengths (Section 3.2.1), and then as the RLVR length reward whose target is o1-mini's thought length (Section 5.3). The LC-initialized policy therefore starts near the length-reward maximum by construction, while the base short-CoT policy starts far below it. Under GRPO's group-relative advantages, the base model's early gradients are dominated by lengthening responses, so the reported 2-3x larger MATH500/GPQA gains partly measure how well the SFT data pre-satisfy the length reward rather than a better reasoning cold-start.

full rationale

The dataset quality comparisons (Section 4) are externally grounded via o3-mini evaluation, and the BoN and GPQA/MMLU-Pro results do not depend on the length reward. However, the paper's primary quantitative claim—that models initialized on the Long CoT Collection achieve 2-3x larger RLVR gains—is not self-contained: the length reward in Section 5.3 is literally the same closeness-to-o1-mini-budget function used to retrieve demonstrations during dataset construction (Section 3.2.1), and Appendix C confirms the LC model scores highest on this length reward. This makes a substantial part of the RLVR improvement a reward-shaping artifact rather than an independent demonstration of reasoning cold-start benefits. The paper does not run an RL baseline initialized on R1-distilled or another long-CoT SFT dataset to control for the length-reward confound. No load-bearing self-citation or imported uniqueness theorem was found; the circularity is confined to the length-reward component of the RL claim.

Assumptions & free parameters 2 free parameters · 5 assumptions · 1 invented entities

The central claim rests on the unvalidated assumption that o1's outline-level reasoning flow, combined with a token budget, is enough to let GPT-4o produce high-quality long CoT data. The pipeline introduces two hand-chosen retrieval parameters and relies on LLM-based correctness filtering and LLM-based quality judgment without independent verification.

free parameters (2)
  • Domain matching score weights = 1 for main category, 0.2 per subcategory
    Chosen by hand in Appendix A.2 to retrieve demonstrations; no sensitivity analysis is provided.
  • Seed dataset size = 1K
    Chosen by hand as the amount of o1 demonstration data to bootstrap generation (Section 3.1); no scaling analysis on seed size.
assumptions (5)
  • domain assumption o1's reasoning flows, extracted from ChatGPT outputs, capture the reasoning strategies responsible for o1's performance.
    The entire dataset is built to imitate o1's reasoning flow (Section 3.1). If o1's performance comes from something other than the flow content, the dataset may not transfer.
  • domain assumption GPT-4o can expand outline steps into coherent, sufficiently long rationales without additional reasoning training.
    Section 3.2.3 relies on GPT-4o to generate each rationale step; the method fails if short CoT LLMs cannot sustain coherence.
  • domain assumption o3-mini is an unbiased judge of reasoning flow, strategy, and correctness.
    The qualitative comparison in Section 4.1 uses o3-mini as evaluator; no human agreement study is provided.
  • ad hoc to paper The thought budget similarity function (min/max ratio) is a valid proxy for matching reasoning effort.
    Section 3.2.1 introduces the function without justification from cognitive science or prior work.
  • domain assumption Correctness filtering by GPT-4o with reference answer comparison removes harmful rationales without introducing bias.
    Section 3.2.4; if GPT-4o accepts wrong answers or rejects correct novel solutions, dataset quality and diversity are affected.
invented entities (1)
  • Reasoning flow
    purpose: Intermediate sequence of outline steps used to guide long CoT generation and thought budget control.
    Introduced as a new representation in Section 3.1.1; no external benchmark validates that reasoning flows capture reasoning quality. Evidence is indirect via downstream generation and model performance.

how reviews work

0 comments
Cite this review

Pith. "Pith review of One Missing Piece for Open-Source Reasoning Models: A Dataset to Mitigate Cold-Starting Short CoT LLMs in RL." pith.science (2026). https://pith.science/paper/LJMYUFHB

@misc{pith2026250602338,
  author       = {Pith},
  title        = {Pith review of: One Missing Piece for Open-Source Reasoning Models: A Dataset to Mitigate Cold-Starting Short CoT LLMs in RL},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LJMYUFHB}},
  note         = {Machine review of arXiv:2506.02338}
}
read the original abstract

With the release of R1, a publicly available large reasoning model (LRM), researchers commonly train new LRMs by training language models on R1's long chain-of-thought (CoT) inferences. While prior works show that LRMs' capabilities can be reproduced through direct distillation, the continued reliance on the existing models (e.g., R1) remains a critical limitation in advancing the field. As a first step toward independent LRM development, this paper explores the possibility of constructing a long CoT dataset with LLMs that are not trained for inference-time scaling. To this end, we present the Long CoT Collection, a dataset of 100K CoT rationales annotated using existing short CoT LLMs. We develop a pipeline that induces o1's novel reasoning strategies into short CoT LLMs, enabling them to think longer and introducing controllability over the thought budget to better manage the overthinking problem. Our extensive analyses validate that our dataset achieves quality comparable to--or slightly below--R1. Furthermore, our experiments demonstrate that training on our dataset not only strengthens general reasoning skills, but also provides a strong foundation for reinforcement learning--models initialized on our data achieve 2-3x larger gains with RLVR.

Figures

Figures reproduced from arXiv: 2506.02338 by the authors.

Figure 1
Figure 1. Comparison of RLVR performance between the base model (Qwen-2.5-0.5B) and the model trained on the Long CoT Collection (Qwen-2.5-0.5B-LC) on MATH500 and GPQA. limit real-world applications (Hendrycks et al., 2022), while the closed-source approach potentially prohibits academic progress in the field. To address these issues, DeepSeek-AI et al. (2025) release an open-source version of o1 and detail their methodology … view at source ↗
Figure 2
Figure 2. Overview of our data construction pipeline. First, we collect an 1K seed dataset of reasoning flow and [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 4
Figure 4. Head-to-head comparison of the generated [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figures from the paper (17 more)
Figure 3
Figure 3. Figure 3: The top 15 most common root verbs and their [PITH_FULL_IMAGE:figures/full_fig_p004_3.png]
Figure 5
Figure 5. Figure 5: Comparison of the number of reasoning tokens [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Results of best-of-n experiments with Llama-3.1-8B-Instruct and Qwen-2.5-7B-Instruct. Model Size GPQA Diamond MMLU Pro API only o1-mini N/A 60.0 80.3 o1 N/A 77.3 - Open Weights Qwen-2.5-32B-Instruct 32B 49.0 69.2 QwQ-32B 32B 65.2 71.0 R1 671B 71.5 84.0 Qwen-7B-R1-disti…
Figure 7
Figure 7. Figure 7: The Pearson correlation (R2 ) between generated tokens and o1-mini thought tokens. We leverage 100% budget (left), 50% budget (mid), and 25% budget (right) to generate the collections of long CoT rationales [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: The token count distribution for each collec [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]
Figure 9
Figure 9. Figure 9: Distribution of the number of reasoning out [PITH_FULL_IMAGE:figures/full_fig_p010_9.png]
Figure 10
Figure 10. Figure 10: Length, answer, and format rewards across three models in LIMO dataset ( [PITH_FULL_IMAGE:figures/full_fig_p012_10.png]
Figure 11
Figure 11. Figure 11: Results of Llama-3.1-8B-LC on MMLU-Pro broken down by domain. Overall business law psychology biology chemistry history other health economics math physics computer science philosophy engineering 30.00% 40.00% 50.00% 60.00% Qwen-2.5-7B-Instruct Ours [PITH_FULL_IMAGE:…
Figure 13
Figure 13. Figure 13: Heatmap of the thought budget function, defined as 1 − [PITH_FULL_IMAGE:figures/full_fig_p013_13.png]
Figure 12
Figure 12. Figure 12: Results of Qwen-2.5-7B-LC on MMLU-Pro broken down by domain [PITH_FULL_IMAGE:figures/full_fig_p013_12.png]
Figure 14
Figure 14. Figure 14: An example instance from the Long CoT Collection. [PITH_FULL_IMAGE:figures/full_fig_p014_14.png]
Figure 15
Figure 15. Figure 15: An example response from Llama-3.1-8B-LC (Ours), which trained on the Long CoT Collection. [PITH_FULL_IMAGE:figures/full_fig_p015_15.png]
Figure 16
Figure 16. Figure 16: Prompt used for stepwise long CoT generation. [PITH_FULL_IMAGE:figures/full_fig_p016_16.png]
Figure 17
Figure 17. Figure 17: Prompt used for filtering the incorrect rationales. [PITH_FULL_IMAGE:figures/full_fig_p016_17.png]
Figure 18
Figure 18. Figure 18: Prompt used for qualitative analysis on Reasoning Flow. We assign the position of A/B randomly. [PITH_FULL_IMAGE:figures/full_fig_p017_18.png]
Figure 19
Figure 19. Figure 19: Prompt used for qualitative analysis on Reasoning Strategy. We assign the position of A/B randomly. [PITH_FULL_IMAGE:figures/full_fig_p017_19.png]
Figure 20
Figure 20. Figure 20: Prompt used for qualitative analysis on Correctness. We assign the position of A/B randomly. [PITH_FULL_IMAGE:figures/full_fig_p017_20.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

43 extracted references · 6 canonical work pages

  1. [1]

    Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jackson Kernion, Tom Conerly, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Tristan Hume, and 12 others. 2022. https://arxiv.org/abs/2204.05862 Training a helpful an...

    arXiv 2022

  2. [2]

    Le, Christopher Ré, and Azalia Mirhoseini

    Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoc V. Le, Christopher Ré, and Azalia Mirhoseini. 2024. https://arxiv.org/abs/2407.21787 Large language monkeys: Scaling inference compute with repeated sampling . Preprint, arXiv:2407.21787

    arXiv 2024

  3. [3]

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, and 181 others. 2025. https://arxiv.org/abs/2501.12948 Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement lea...

    arXiv 2025

  4. [4]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, and 3 others. 2024. https://arxiv.org/abs/2407.21783 The llama 3 herd of mod...

    arXiv 2024

  5. [5]

    Xinyu Guan, Li Lyna Zhang, Yifei Liu, Ning Shang, Youran Sun, Yi Zhu, Fan Yang, and Mao Yang. 2025. https://arxiv.org/abs/2501.04519 rstar-math: Small llms can master math reasoning with self-evolved deep thinking . Preprint, arXiv:2501.04519

    arXiv 2025

  6. [6]

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021 a . https://arxiv.org/abs/2009.03300 Measuring massive multitask language understanding . Preprint, arXiv:2009.03300

    arXiv 2021

  7. [7]

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021 b . https://arxiv.org/abs/2103.03874 Measuring mathematical problem solving with the math dataset . Preprint, arXiv:2103.03874

    arXiv 2021

  8. [8]

    Dan Hendrycks, Nicholas Carlini, John Schulman, and Jacob Steinhardt. 2022. https://arxiv.org/abs/2109.13916 Unsolved problems in ml safety . Preprint, arXiv:2109.13916

    arXiv 2022

Show all 43 references
  1. [9]

    Jian Hu, Xibin Wu, Zilin Zhu, Xianyu, Weixun Wang, Dehao Zhang, and Yu Cao. 2024. Openrlhf: An easy-to-use, scalable and high-performance rlhf framework. arXiv preprint arXiv:2405.11143

    2024 arXiv

  2. [10]

    Zhen Huang, Haoyang Zou, Xuefeng Li, Yixiu Liu, Yuxiang Zheng, Ethan Chern, Shijie Xia, Yiwei Qin, Weizhe Yuan, and Pengfei Liu. 2024. https://arxiv.org/abs/2411.16489 O1 replication journey -- part 2: Surpassing o1-preview through simple distillation, big progress or bitter l...

    2024 arXiv

  3. [11]

    Bespoke Labs. 2025. https://hf.co/bespokelabs/Bespoke-Stratos-32B Bespoke-stratos: The unreasonable effectiveness of reasoning distillation . Accessed: 2025-01-22

    2025

  4. [12]

    Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, and 1 others. 2024. T " ulu 3: Pushing frontiers in open language model post-training. arXiv preprint arXiv:2411.15124

    2024 arXiv

  5. [13]

    Jia LI, Edward Beeching, Lewis Tunstall, Ben Lipkin, Roman Soletskyi, Shengyi Costa Huang, Kashif Rasul, Longhui Yu, Albert Jiang, Ziju Shen, Zihan Qin, Bin Dong, Li Zhou, Yann Fleureau, Guillaume Lample, and Stanislas Polu. 2024. https://github.com/project-numina/aimo-progres...

    2024

  6. [14]

    Zhenwen Liang, Ye Liu, Tong Niu, Xiangliang Zhang, Yingbo Zhou, and Semih Yavuz. 2024. Improving llm reasoning through scaling inference computation with collaborative verification. arXiv preprint arXiv:2410.05318

    2024 arXiv

  7. [15]

    Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. https://arxiv.org/abs/2305.20050 Let's verify step by step . Preprint, arXiv:2305.20050

    2023 arXiv

  8. [16]

    Hao Liu, Matei Zaharia, and Pieter Abbeel. 2023. https://arxiv.org/abs/2310.01889 Ring attention with blockwise transformers for near-infinite context . Preprint, arXiv:2310.01889

    2023 arXiv

  9. [18]

    Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. 2025 b . https://arxiv.org/abs/2501.19393 s1: Simple test-time scaling . Preprint, arXiv:2501.19393

    2025 arXiv

  10. [19]

    Mathematical Association of America. 2024. https://artofproblemsolving.com/wiki/index.php/AIME_Problems_and_Solutions/ Aime

    2024

  11. [20]

    OpenAI. 2024. https://openai.com/index/learning-to-reason-with-llms/ Learning to reason with llms

    2024

  12. [21]

    Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and...

    2022 arXiv

  13. [22]

    Bo Pang, Hanze Dong, Jiacheng Xu, Silvio Savarese, Yingbo Zhou, and Caiming Xiong. 2025. https://arxiv.org/abs/2502.03860 Bolt: Bootstrap long chain-of-thought in language models without distillation . Preprint, arXiv:2502.03860

    2025 arXiv

  14. [23]

    Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, and 24 others. 2024. https://arxiv.org/abs/2412.15115 Qwe...

    2024 arXiv

  15. [24]

    David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. 2023. https://arxiv.org/abs/2311.12022 Gpqa: A graduate-level google-proof q&a benchmark . Preprint, arXiv:2311.12022

    2023 arXiv

  16. [25]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. https://arxiv.org/abs/2402.03300 Deepseekmath: Pushing the limits of mathematical reasoning in open language models . Preprint, arXiv:2402.03300

    2024 arXiv

  17. [26]

    Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. 2024. https://arxiv.org/abs/2408.03314 Scaling llm test-time compute optimally can be more effective than scaling model parameters . Preprint, arXiv:2408.03314

    2024 arXiv

  18. [27]

    NovaSky Team. 2025 a . https://novasky-ai.github.io/posts/sky-t1 Sky-t1: Fully open-source reasoning model with o1-preview performance in \ 450 budget . Accessed: 2025-01-09

    2025

  19. [28]

    Open Thoughts Team. 2025 b . Open Thoughts

    2025

  20. [29]

    Qwen Team. 2024. https://qwenlm.github.io/blog/qwq-32b-preview/ Qwq: Reflect deeply on the boundaries of the unknown

    2024

  21. [30]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. https://arxiv.org/abs/2302.13971 Llama:...

    2023 arXiv

  22. [31]

    Chaojie Wang, Yanchen Deng, Zhiyi Lyu, Liang Zeng, Jujie He, Shuicheng Yan, and Bo An. 2024 a . Q*: Improving multi-step reasoning for llms with deliberative planning. arXiv preprint arXiv:2406.14283

    2024 arXiv

  23. [32]

    Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, and 1 others. 2024 b . Mmlu-pro: A more robust and challenging multi-task language understanding benchmark. arXiv preprint arXiv:2406.01574

    2024 arXiv

  24. [33]

    Yue Wang, Qiuzhi Liu, Jiahao Xu, Tian Liang, Xingyu Chen, Zhiwei He, Linfeng Song, Dian Yu, Juntao Li, Zhuosheng Zhang, and 1 others. 2025. Thoughts are all over the place: On the underthinking of o1-like llms. arXiv preprint arXiv:2501.18585

    2025 arXiv

  25. [34]

    Haotian Xu, Xing Wu, Weinong Wang, Zhongzhi Li, Da Zheng, Boyuan Chen, Yi Hu, Shijia Kang, Jiaming Ji, Yingying Zhang, Zhijiang Guo, Yaodong Yang, Muhan Zhang, and Debing Zhang. 2025. https://arxiv.org/abs/2501.11284 Redstar: Does scaling long-cot data unlock better slow-reaso...

    2025 arXiv

  26. [35]

    Zhangchen Xu, Fengqing Jiang, Luyao Niu, Yuntian Deng, Radha Poovendran, Yejin Choi, and Bill Yuchen Lin. 2024. Magpie: Alignment data synthesis from scratch by prompting aligned llms with nothing. arXiv preprint arXiv:2406.08464

    2024 arXiv

  27. [36]

    Yixin Ye, Zhen Huang, Yang Xiao, Ethan Chern, Shijie Xia, and Pengfei Liu. 2025. https://arxiv.org/abs/2502.03387 Limo: Less is more for reasoning . Preprint, arXiv:2502.03387

    2025 arXiv

  28. [37]

    Edward Yeo, Yuxuan Tong, Morry Niu, Graham Neubig, and Xiang Yue. 2025. https://arxiv.org/abs/2502.03373 Demystifying long chain-of-thought reasoning in llms . Preprint, arXiv:2502.03373

    2025 arXiv

  29. [38]

    Hongzhou Yu, Tianhao Cheng, Ying Cheng, and Rui Feng. 2025. https://arxiv.org/abs/2501.09213 Finemedlm-o1: Enhancing the medical reasoning ability of llm from supervised fine-tuning to test-time training . Preprint, arXiv:2501.09213

    2025 arXiv

  30. [39]

    Di Zhang, Jianbo Wu, Jingdi Lei, Tong Che, Jiatong Li, Tong Xie, Xiaoshui Huang, Shufei Zhang, Marco Pavone, Yuqiang Li, and 1 others. 2024 a . Llama-berry: Pairwise optimization for o1-like olympiad-level mathematical reasoning. arXiv preprint arXiv:2410.02884

    2024 arXiv

  31. [40]

    Yuxiang Zhang, Shangxi Wu, Yuqi Yang, Jiangming Shu, Jinlin Xiao, Chao Kong, and Jitao Sang. 2024 b . https://arxiv.org/abs/2412.00154 o1-coder: an o1 replication for coding . Preprint, arXiv:2412.00154

    2024 arXiv

  32. [41]

    Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. 2024. https://arxiv.org/abs/2403.13372 Llamafactory: Unified efficient fine-tuning of 100+ language models . Preprint, arXiv:2403.13372

    2024 arXiv

  33. [42]

    Banghua Zhu, Evan Frick, Tianhao Wu, Hanlin Zhu, and Jiantao Jiao. 2023. Starling-7b: Improving llm helpfulness & harmlessness with rlaif

    2023

  34. [43]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  35. [44]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.