Pith. sign in

REVIEW 3 major objections 4 minor 37 references

Mining Hidden Thoughts from Texts: Evaluating Continual Pretraining with Synthetic Data for LLM Reasoning

T0 review · 3 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Prepending LLM-generated hidden thoughts to expert texts during continual pretraining improves reasoning on MMLU, with the largest gains — about 8 points over standard CPT — on the hardest questions.

desk verdict The MMLU evaluation prompt gives Reasoning CPT a home-field advantage, so the headline gains over standard CPT are not yet trustworthy. read the letter →

arxiv 2505.10182 v1 pith:PKYU6UHI submitted 2025-05-15 cs.CL cs.LG

Yoichi Ishibashi , Taro Yano , Masafumi Oyamada This is my paper · ORCID
classification cs.CLcs.LG
keywords continualpretraininghiddenthoughtssyntheticdataLLMreasoningMMLUcross-domaintransferchain-of-thoughtefficiency
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that a language model can learn to reason better by continuing to pretrain on synthetic text in which an LLM-generated 'hidden thought' is placed before an expert passage, on the premise that every text is the residue of its author's thinking. On MMLU, this Reasoning CPT beats standard continual pretraining on the same passages by about 1.4–1.8 points overall and by roughly 8 points on the hardest questions, with up to 3.3 points over the base model. The gains appear in domains never seen during training, and the trained models spontaneously spend fewer tokens on easy questions and more on hard ones. If the effect is real, it offers a way to build reasoning data from ordinary expert text without task-specific rewards.

What carries the argument

The central object is the training sequence $X = \langle\text{start\_of\_thought}\rangle \oplus H \oplus \langle\text{end\_of\_thought}\rangle \oplus S$, where $H$ is an LLM-generated hidden-thought segment and $S$ is the original expert text, trained with the standard autoregressive next-token loss. Hidden thoughts are produced by Gemma2-9B-it under a prompt that elicits goal setting, background-knowledge recall, decision-making, and self-verification, so ordinary STEM and legal passages become explicit reasoning traces. The mechanism also includes a corpus-level correlation between original-text length and hidden-thought length (Spearman $\rho=0.348$ for STEM, $\rho=0.486$ for Law), which the paper identifies as the plausible driver of the models' difficulty-adaptive thinking length.

What would settle it

A control that replaces the hidden-thought segment with same-length, same-format filler thoughts (or shuffled thoughts from other texts) would settle the claim: if it matches Reasoning CPT's gains, the reasoning content is not what drives the improvement. A second check would have domain experts judge whether the generated thoughts recover actual omitted reasoning steps from the source texts; low fidelity would undermine the proposed mechanism.

Watch

Extended reading notes

Core claim

Continual pretraining on synthetic sequences that prepend an LLM-generated hidden thought to an expert text — called Reasoning CPT — improves MMLU accuracy more than standard continual pretraining on the same texts, and the gap widens with problem difficulty. Reasoning CPT trained on STEM text reaches 69.1% overall versus 67.3% for standard CPT; trained on legal text it reaches 68.1% versus 66.7%. On Very Hard questions the advantage over standard CPT is about 8 points in both domains, with gains of 10.5–11.2 points over the base model. The skill transfers across domains: a model trained on legal hidden thoughts improves MMLU-STEM by 4.3 points, and models trained with hidden thoughts generate shorter thinking on easy questions and longer thinking on hard ones, matching the positive correlation between source-text length and thought length in the training corpus.

Load-bearing premise

The load-bearing premise is that the LLM-generated hidden thoughts faithfully reconstruct the reasoning behind the source texts, so the measured improvements come from learning that reasoning content rather than from simply seeing more tokens in a thought-tagged format.

Editorial extensions

If this is right

  • Reasoning CPT on either STEM or legal text raises MMLU across all four subject groups, not just the training domain.
  • The advantage over standard CPT grows with problem difficulty, reaching about 8 points on Very Hard questions in both training domains.
  • Models trained this way adapt reasoning length: fewer thinking tokens than CPT on easy questions and more on hard ones, with no accuracy loss on easy items.
  • Because the method needs no correctness labels or verifiable rewards, it can be applied to any high-quality text corpus.
  • The trained model retains diverse reasoning paths, improving GSM8k Pass@5 to 91.7% versus 81.2% Pass@1 for the instruction-tuned baseline.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The adaptive-length behaviour may be a corpus-level heuristic: the training data's text-length/thought-length correlation could teach 'think until confident' rather than genuine difficulty awareness; a controlled corpus with decorrelated lengths would test this.
  • A confound remains: the few-shot evaluation prompts themselves include hidden-thought examples, so part of the gain could come from format familiarity rather than reasoning content; an evaluation with plain CoT prompts would separate the two.
  • If the mechanism generalizes, the same recipe could turn fiction, history, or scientific prose into reasoning data, extending the approach beyond reward-rich domains.
  • The results also suggest a cheaper alternative to RL for building reasoning: mine thoughts once with a strong generator, then continual-pretrain a base model, preserving output diversity that instruction tuning tends to narrow.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces Reasoning CPT, a continual-pretraining method in which each original text is prepended with LLM-generated 'hidden thoughts' produced by Gemma2-9B-it from the source text. The authors compare Reasoning CPT with standard CPT on the same 150k-example STEM and Law corpora, using Gemma2-9B + LoRA, and evaluate on MMLU plus GSM8k diversity. They report consistent MMLU gains over standard CPT, larger gains on difficult questions, cross-domain transfer, adaptive thought length, and improved Pass@k. The central claim is that learning synthetic reconstructions of latent author reasoning is more effective than learning the texts alone.

Significance. If the central claim survives, the paper is a useful empirical contribution because it proposes a reward-free way to create reasoning-oriented training data from abundant text. The presentation is transparent: exact prompts, data sizes, hyperparameters, loss curves, token-matched comparisons, and concrete examples are included. The matched-token comparison in Figure 4 is a genuine attempt to control for the extra-token confound, and the Pass@k analysis in Section 5 is a meaningful check on output diversity. However, all headline results rest on a single base model, a single benchmark, and a single evaluation prompt, and the main comparison is vulnerable to a format-familiarity confound. The contribution is therefore promising but not yet established at the level claimed.

major comments (3)
  1. [§3.1, Appendix E.2, Table 3] The skeptical concern about evaluation-prompt format is real. Section 3.1 states that the few-shot prompts follow Ruan et al. (2025) and include hidden thoughts enclosed by thought tags, and Appendix E.2 confirms that the MMLU evaluation prompt contains <start_of_thought> and <end_of_thought> exemplars. Equation (2) trains Reasoning CPT on exactly this tag structure, whereas standard CPT is trained on raw text without tags. Consequently, the 1.4-8 point gaps in Tables 1 and 2 may measure fluency with the hidden-thought prompt format rather than the reasoning content of the synthetic thoughts. This is not a neutral evaluation condition: Appendix A, Table 3 shows that on GSM8k the base model improves from 58.3 to 65.4 when the hidden-thought style is used. The authors should evaluate all models with a standard CoT prompt, or train a control that inserts syntactically similar but content-free filler between the same thought tags, before the central claim can be attributed to hidden-thought content.
  2. [Tables 1-2, Figure 4] All accuracy numbers are single-run point estimates from one base model and one training configuration. The claimed advantages are mostly 1-3 points overall and about 8 points on Very Hard questions; without variance estimates or multiple seeds it is impossible to tell whether the smaller gaps, such as 68.1 vs 66.7 for Law overall, are reliable. The authors should report standard deviations across at least three seeds, or otherwise bound the noise, before claiming that Reasoning CPT 'consistently outperforms' standard CPT.
  3. [§2.2] The quality of the generated hidden thoughts is asserted rather than measured. The thoughts are produced by Gemma2-9B-it, the instruction-tuned sibling of the base model, with no human evaluation, no validation against the source text's actual reasoning, and no ablation against an alternative augmentation such as Ruan et al.'s background-knowledge style. Because the whole method hinges on the reconstructed thoughts being faithful and useful, the paper needs either a validation study or a control condition that keeps the thought-tag format constant while varying only the content.
minor comments (4)
  1. [Abstract and Table 2] The abstract says 'gains of up to 8 points on the most challenging problems,' which is the gap versus standard CPT, but the table also shows 10.5-11.2 point gains versus the base model; the basis for the '8 points' should be stated explicitly to avoid ambiguity.
  2. [§3.1 and references] The paper cites Gemma2-9B and Gemma2-9B-it as [Shao et al., 2024], but that reference is DeepSeekMath; the Gemma model card or the correct Google model citation should be used.
  3. [Appendix E.2] The MMLU prompt in Appendix E.2 appears to have missing or misplaced <start_of_thought> tags in the first two exemplars, which makes the exact evaluation condition ambiguous; the prompt should be reproduced exactly as used.
  4. [§3.2, Table 2] The difficulty labels come from GPT-4o with no validation or inter-annotator agreement; reporting a secondary labeling or at least a sample of human-checked labels would strengthen the difficulty-ordering claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: empirical comparisons on external benchmarks; evaluation-format similarity is a confound, not a circular derivation.

full rationale

The paper's central claims are empirical comparisons on external benchmarks (MMLU, GSM8k) rather than quantities derived from their own inputs. Reasoning CPT and standard CPT are trained on the same source corpus with the only difference being inserted hidden thoughts (Eq. 2), and MMLU accuracy is measured on a held-out benchmark; no fitted parameter is renamed as a prediction. There are no load-bearing self-citations: the evaluation-prompt convention is attributed to external prior work (Ruan et al. 2025), and the authors' own prior work is not cited as justification. Appendix A (Table 3) does show that the hidden-thought prompt format itself shifts Gemma2-9B's GSM8k accuracy by about 7 points, and the 2-shot MMLU prompt includes hidden-thought exemplars, so format matching is a legitimate experimental confound that a standard-prompt or filler-thought control would resolve. But a confound is not circularity under the defined criteria: the measured gain is not equivalent by construction to the training input, and the paper does not claim to derive its results from a self-citation or a uniqueness theorem. The difficulty-adaptive reasoning length analysis is also an empirical observation with an acknowledged plausible driver (token-count correlation in the training corpus, Figure 6), not a result defined into existence. Verdict: no significant circularity.

Assumptions & free parameters 7 free parameters · 6 assumptions · 1 invented entities

The central empirical comparison relies on several hand-chosen training hyperparameters and on the unvalidated premise that LLM-generated hidden thoughts capture useful reasoning. The evaluation against external benchmarks keeps circularity low, but the lack of ablations and the single-seed setup make the exact contributions of each component uncertain.

free parameters (7)
  • LoRA rank r = 64
    Hand-chosen for all runs; no sweeps reported, and the magnitude of MMLU gains could depend on this capacity.
  • Learning rate = 3e-5 with cosine decay
    Hand-chosen; no ablation, could affect relative gains.
  • Training epochs = 6
    Hand-chosen; all models use the same setting, but overfitting or underfitting could shape the reported differences.
  • Maximum sequence length = 1024 tokens
    Chosen so that original text plus hidden thoughts fits; truncation choices are part of the data pipeline.
  • Hidden-thought generation temperature = 0.3
    Hand-chosen for synthetic data generation; affects diversity and quality of thoughts.
  • Maximum hidden-thought length = 512 tokens
    Imposed to fit within the 1024-token training window; interacts with the reported correlation between text length and thought length.
  • Training corpus size = 150,000 examples per domain
    Arbitrary subsample; no scaling study to show whether gains saturate or would grow with more data.
assumptions (6)
  • domain assumption Every text is the result of an author's implicit thinking process, and reconstructing it with an LLM yields useful training signal.
    Stated in §1 and §2.2; not validated against real human thought traces.
  • domain assumption LLM-generated hidden thoughts are faithful enough to improve reasoning.
    No human evaluation or comparison to actual author notes; quality only indirectly supported by downstream MMLU gains.
  • domain assumption MMLU accuracy is a valid proxy for general reasoning ability.
    Standard but imperfect; used as the sole main benchmark for the central claims.
  • domain assumption GPT-4o-assigned difficulty levels are correct.
    No validation of the difficulty ratings; all difficulty analyses depend on them.
  • domain assumption LoRA fine-tuning behaves like full fine-tuning for this comparison.
    All models use LoRA, so relative comparisons are fair, but absolute claims about continual pretraining may not transfer to full fine-tuning.
  • domain assumption Autoregressive language modeling on thought-plus-text sequences teaches reasoning rather than just memorization of the format.
    The paper infers this from results; it is not directly proven, and the evaluation prompts also include hidden-thought examples.
invented entities (1)
  • hidden thoughts
    purpose: LLM-generated reconstructions of the implicit reasoning behind a text, used as a pretraining signal.
    Conceptual construct introduced in §1. No direct evidence of correspondence with actual human thoughts; only indirect downstream evidence.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Mining Hidden Thoughts from Texts: Evaluating Continual Pretraining with Synthetic Data for LLM Reasoning." pith.science (2026). https://pith.science/paper/PKYU6UHI

@misc{pith2026250510182,
  author       = {Pith},
  title        = {Pith review of: Mining Hidden Thoughts from Texts: Evaluating Continual Pretraining with Synthetic Data for LLM Reasoning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PKYU6UHI}},
  note         = {Machine review of arXiv:2505.10182}
}
read the original abstract

Large Language Models (LLMs) have demonstrated significant improvements in reasoning capabilities through supervised fine-tuning and reinforcement learning. However, when training reasoning models, these approaches are primarily applicable to specific domains such as mathematics and programming, which imposes fundamental constraints on the breadth and scalability of training data. In contrast, continual pretraining (CPT) offers the advantage of not requiring task-specific signals. Nevertheless, how to effectively synthesize training data for reasoning and how such data affect a wide range of domains remain largely unexplored. This study provides a detailed evaluation of Reasoning CPT, a form of CPT that uses synthetic data to reconstruct the hidden thought processes underlying texts, based on the premise that texts are the result of the author's thinking process. Specifically, we apply Reasoning CPT to Gemma2-9B using synthetic data with hidden thoughts derived from STEM and Law corpora, and compare it to standard CPT on the MMLU benchmark. Our analysis reveals that Reasoning CPT consistently improves performance across all evaluated domains. Notably, reasoning skills acquired in one domain transfer effectively to others; the performance gap with conventional methods widens as problem difficulty increases, with gains of up to 8 points on the most challenging problems. Furthermore, models trained with hidden thoughts learn to adjust the depth of their reasoning according to problem difficulty.

Figures

Figures reproduced from arXiv: 2505.10182 by the authors.

Figure 1
Figure 1. Performance differences from the base model across MMLU problem difficulty levels [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Example of synthetic data (Law domain). Black text represents the original text. Green text indicates the hidden thoughts reconstructed from the original text. Hidden Thoughts Generation We generate hidden thoughts for the preprocessed texts using an LLM. For each sample in both domains (STEM and Law), we add hidden thoughts to the text. Using the prompt in Appendix E, we generate hidden thoughts marked by thought t… view at source ↗
Figure 3
Figure 3. Example of synthetic data (STEM domain). [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (15 more)
Figure 4
Figure 4. Figure 4: MMLU accuracy trends with training token count for (standard) CPT and Reasoning CPT. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Relationship between problem difficulty, length of generated hidden thoughts, and accuracy. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Correlation between original text token count and hidden thoughts token count [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Comparison of Pass@k accuracy for each model on GSM8k. Gemma2-9B and Reasoning CPT show large improvements as k increases, indicating more problems can be solved with more sampling attempts. In contrast, the instruction-tuned model (Gemma2-9B-it) shows high initial Pas…
Figure 8
Figure 8. Figure 8: Training loss observed during continual pretraining. The left plot shows the loss over [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]
Figure 9
Figure 9. Figure 9: Example of synthetic data (STEM domain) with hidden thoughts [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]
Figure 10
Figure 10. Figure 10: Example of synthetic data (STEM domain) with hidden thoughts [PITH_FULL_IMAGE:figures/full_fig_p019_10.png]
Figure 11
Figure 11. Figure 11: Example of synthetic data (STEM domain) with hidden thoughts [PITH_FULL_IMAGE:figures/full_fig_p020_11.png]
Figure 12
Figure 12. Figure 12: Example of synthetic data (STEM domain) with hidden thoughts [PITH_FULL_IMAGE:figures/full_fig_p021_12.png]
Figure 13
Figure 13. Figure 13: Example of synthetic data (STEM domain) with hidden thoughts [PITH_FULL_IMAGE:figures/full_fig_p022_13.png]
Figure 14
Figure 14. Figure 14: Example of synthetic data (Law domain) with hidden thoughts [PITH_FULL_IMAGE:figures/full_fig_p023_14.png]
Figure 15
Figure 15. Figure 15: Example of synthetic data (Law domain) with hidden thoughts [PITH_FULL_IMAGE:figures/full_fig_p024_15.png]
Figure 16
Figure 16. Figure 16: Example of synthetic data (Law domain) with hidden thoughts [PITH_FULL_IMAGE:figures/full_fig_p025_16.png]
Figure 17
Figure 17. Figure 17: Example of synthetic data (Law domain) with hidden thoughts [PITH_FULL_IMAGE:figures/full_fig_p026_17.png]
Figure 18
Figure 18. Figure 18: Example of synthetic data (Law domain) with hidden thoughts [PITH_FULL_IMAGE:figures/full_fig_p027_18.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

37 extracted references · 10 canonical work pages

  1. [1]

    Openai o1 system card

    Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El - Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, Alex Iftimie, Alex Karpenko, Alex Tachard Passos, Alexander Neitz, Alexander Prokofiev, Alexander Wei, Allison Tam, Ally Bennett, Ananya Kumar, Andre Saraiva, Andrea Vallone, Andrew Duberstein, Andrew Kondrich, Andre...

    arXiv doi:10.48550/arxiv.2412.16720 2024

  2. [2]

    DeepSeek - AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai...

    arXiv doi:10.48550/arxiv.2501.12948 2025

  3. [3]

    Aime, February 2024

    Mathematical Association of America . Aime, February 2024. URL https://artofproblemsolving.com/wiki/index.php/AIME_Problems_and_Solutions/

    work page 2024

  4. [4]

    David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. GPQA: A graduate-level google-proof q & a benchmark. CoRR, abs/2311.12022, 2023. doi:10.48550/ARXIV.2311.12022. URL https://doi.org/10.48550/arXiv.2311.12022

    arXiv doi:10.48550/arxiv.2311.12022 2023

  5. [5]

    Cand \` e s, and Tatsunori Hashimoto

    Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei - Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel J. Cand \` e s, and Tatsunori Hashimoto. s1: Simple test-time scaling. CoRR, abs/2501.19393, 2025. doi:10.48550/ARXIV.2501.19393. URL https://doi.org/10.48550/arXiv.2501.19393

    arXiv doi:10.48550/arxiv.2501.19393 2025

  6. [6]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. CoRR, abs/2402.03300, 2024. doi:10.48550/ARXIV.2402.03300. URL https://doi.org/10.48550/arXiv.2402.03300

    arXiv doi:10.48550/arxiv.2402.03300 2024

  7. [7]

    Openwebmath: An open dataset of high-quality mathematical web text

    Keiran Paster, Marco Dos Santos, Zhangir Azerbayev, and Jimmy Ba. Openwebmath: An open dataset of high-quality mathematical web text. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024 . OpenReview.net, 2024. URL https://openreview.net/forum?id=jKHmjlpViu

    work page 2024

  8. [8]

    The pile: An 800gb dataset of diverse text for language modeling

    Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, Shawn Presser, and Connor Leahy. The pile: An 800gb dataset of diverse text for language modeling. CoRR, abs/2101.00027, 2021. URL https://arxiv.org/abs/2101.00027

    arXiv 2021

Show all 37 references
  1. [9]

    Reasoning to learn from latent thoughts

    Yangjun Ruan, Neil Band, Chris J Maddison, and Tatsunori Hashimoto. Reasoning to learn from latent thoughts. arXiv preprint arXiv:2503.18866, 2025

    2025

  2. [10]

    Gemini 2.5: Our most intelligent ai model, March 2025

    Google. Gemini 2.5: Our most intelligent ai model, March 2025. URL https://blog.google/technology/google-deepmind/gemini-model-thinking-updates-march-2025/#gemini-2-5-pro

    2025

  3. [11]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen - Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen - Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022 ....

    2022

  4. [12]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019 . OpenReview.net, 2019. URL https://openreview.net/forum?id=Bkg6RiCqY7

    2019

  5. [13]

    Measuring massive multitask language understanding

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021 . OpenReview...

    2021

  6. [14]

    Stop overthinking: A survey on efficient reasoning for large language models

    Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Hanjie Chen, Xia Hu, et al. Stop overthinking: A survey on efficient reasoning for large language models. arXiv preprint arXiv:2503.16419, 2025

    2025 arXiv

  7. [15]

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Pond \' e de Oliveira Pinto, Jared Kaplan, Harrison Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, ...

    2021 arXiv

  8. [16]

    Training verifiers to solve math word problems

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. CoRR, abs/2110.14168, 2021. URL https://...

    arXiv

  9. [17]

    Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model?, 2025

    Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Yang Yue, Shiji Song, and Gao Huang. Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model?, 2025. URL https://arxiv.org/abs/2504.13837

    2025 arXiv

  10. [18]

    T " ulu 3: Pushing frontiers in open language model post-training

    Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, et al. T " ulu 3: Pushing frontiers in open language model post-training. arXiv preprint arXiv:2411.15124, 2024

    2024 arXiv

  11. [19]

    Optimizing language models for inference time objectives using reinforcement learning

    Yunhao Tang, Kunhao Zheng, Gabriel Synnaeve, and R \' e mi Munos. Optimizing language models for inference time objectives using reinforcement learning. CoRR, abs/2503.19595, 2025. doi:10.48550/ARXIV.2503.19595. URL https://doi.org/10.48550/arXiv.2503.19595

    2025 doi arXiv

  12. [20]

    Le, Ed H

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali,...

    2023

  13. [21]

    Bradley C. A. Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoc V. Le, Christopher R \' e , and Azalia Mirhoseini. Large language monkeys: Scaling inference compute with repeated sampling. CoRR, abs/2407.21787, 2024. doi:10.48550/ARXIV.2407.21787. URL https://doi.org/10...

    2024 doi arXiv

  14. [22]

    Self-refine: Iterative refinement with self-feedback

    Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. Self-refine: Iterativ...

    2023

  15. [23]

    Reflexion: language agents with verbal reinforcement learning

    Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: language agents with verbal reinforcement learning. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine, editors, Advances in Neural Information...

    2023

  16. [24]

    Teaching large language models to self-debug

    Xinyun Chen, Maxwell Lin, Nathanael Sch \" a rli, and Denny Zhou. Teaching large language models to self-debug. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024 . OpenReview.net, 2024 a . URL https://openreview.net...

    2024

  17. [25]

    Tree of thoughts: Deliberate problem solving with large language models

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine, editors, Adva...

    2023

  18. [26]

    Mutual reasoning makes smaller llms stronger problem-solvers

    Zhenting Qi, Mingyuan Ma, Jiahang Xu, Li Lyna Zhang, Fan Yang, and Mao Yang. Mutual reasoning makes smaller llms stronger problem-solvers. CoRR, abs/2408.06195, 2024. doi:10.48550/ARXIV.2408.06195. URL https://doi.org/10.48550/arXiv.2408.06195

    2024 doi arXiv

  19. [27]

    Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah D. Goodman. Star: Bootstrapping reasoning with reasoning. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems 35: Annual Conference on Neural Info...

    2022

  20. [28]

    If llm is the wizard, then code is the wand: A survey on how code empowers large language models to serve as intelligent agents

    Ke Yang, Jiateng Liu, John Wu, Chaoqi Yang, Yi Ren Fung, Sha Li, Zixuan Huang, Xu Cao, Xingyao Wang, Yiquan Wang, Heng Ji, and Chengxiang Zhai. If llm is the wizard, then code is the wand: A survey on how code empowers large language models to serve as intelligent agents. ArXi...

    2024 arXiv

  21. [29]

    Locatelli, Marzieh Fadaee, A

    Viraat Aryabumi, Yixuan Su, Raymond Ma, Adrien Morisot, Ivan Zhang, Acyr F. Locatelli, Marzieh Fadaee, A. Ustun, and Sara Hooker. To code, or not to code? exploring impact of code in pre-training. ArXiv, abs/2408.10914, 2024. URL https://api.semanticscholar.org/CorpusID:271909530

    2024 arXiv

  22. [30]

    How does code pretraining affect language model task performance? ArXiv, abs/2409.04556, 2024

    Jackson Petty, Sjoerd van Steenkiste, and Tal Linzen. How does code pretraining affect language model task performance? ArXiv, abs/2409.04556, 2024. URL https://api.semanticscholar.org/CorpusID:272525123

    2024 arXiv

  23. [31]

    Code pretraining improves entity tracking abilities of language models

    Najoung Kim, Sebastian Schuster, and Shubham Toshniwal. Code pretraining improves entity tracking abilities of language models. ArXiv, abs/2405.21068, 2024. URL https://api.semanticscholar.org/CorpusID:270199578

    2024 arXiv

  24. [32]

    Fumiya Uchiyama, Takeshi Kojima, Andrew Gambardella, Qi Cao, Yusuke Iwasawa, and Yutaka Matsuo. Which programming language and what features at pre-training stage affect downstream logical inference performance? In Conference on Empirical Methods in Natural Language Processing...

    2024

  25. [33]

    Lexpam: Legal procedure awareness-guided mathematical reasoning

    Kepu Zhang, Guofu Xie, Weijie Yu, Mingyue Xu, Xu Tang, Yaxin Li, and Jun Xu. Lexpam: Legal procedure awareness-guided mathematical reasoning. 2025. URL https://api.semanticscholar.org/CorpusID:277510367

    2025

  26. [34]

    Eric Zelikman, Georges Harik, Yijia Shao, Varuna Jayasiri, Nick Haber, and Noah D. Goodman. Quiet-star: Language models can teach themselves to think before speaking. CoRR, abs/2403.09629, 2024. doi:10.48550/ARXIV.2403.09629. URL https://doi.org/10.48550/arXiv.2403.09629

    2024 doi arXiv

  27. [35]

    Explain yourself! leveraging language models for commonsense reasoning

    Nazneen Fatema Rajani, Bryan McCann, Caiming Xiong, and Richard Socher. Explain yourself! leveraging language models for commonsense reasoning. In Anna Korhonen, David R. Traum, and Llu \' s M \` a rquez, editors, Proceedings of the 57th Conference of the Association for Compu...

    2019 doi

  28. [36]

    Reinstruct: Building instruction data from unlabeled corpus

    Shu Chen, Xinyan Guan, Yaojie Lu, Hongyu Lin, Xianpei Han, and Le Sun. Reinstruct: Building instruction data from unlabeled corpus. In Lun - Wei Ku, Andre Martins, and Vivek Srikumar, editors, Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thaila...

    2024 doi

  29. [37]

    Learning to reason and memorize with self-notes

    Jack Lanchantin, Shubham Toshniwal, Jason Weston, Arthur Szlam, and Sainbayar Sukhbaatar. Learning to reason and memorize with self-notes. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine, editors, Advances in Neural Information Proces...

    2023

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.