Pith. sign in

REVIEW 5 major objections 6 minor 46 references

ReaLM: Reflection-Enhanced Autonomous Reasoning with Small Language Models

T0 review · 5 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A reinforcement learning framework, ReaLM, trains small language models to reason more reliably by learning from both correct and incorrect reasoning traces, and then removes the need for external reasoning prompts at inference.

desk verdict ReaLM is a well-executed recipe with a promising training signal, but the autonomy mechanism is not actually isolated and the external CoTs contain answer leakage, so the claims outrun the evidence. read the letter →

arxiv 2508.12387 v1 pith:TJGWXQC2 submitted 2025-08-17 cs.CL

Yuanfeng Xu , Zehui Dai , Jian Liang , Jiapeng Guan , Guangrun Wang , Liang Lin , Xiaohui Lv This is my paper · ORCID
classification cs.CL
keywords smalllanguagemodelsreinforcementlearningchain-of-thoughtdistillationprocesssupervisionautonomousreasoningverticaldomainsmulti-route
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ReaLM is a reinforcement learning framework that aims to give small language models (SLMs, ≤7B parameters) three properties at once: stronger multi-step reasoning, the ability to reason without externally supplied chain-of-thought prompts at inference time, and better generalization to vertical domains with proprietary rules. The central idea is to feed the model a bundle of unfiltered reasoning traces from a large teacher model, including many wrong ones, reward it both for judging each trace and for producing the correct final answer, and then gradually fade the traces out of the training input using a cosine schedule. On nine academic benchmarks and a private industrial ad-relevance set, the paper reports average gains of 4.2% over prior methods when external CoTs are available and 2.6% when they are not. The paper's claim, stated fairly, is that comparative reflection over failed reasoning paths is what teaches a small model to reason, and that fading external guidance converts that skill into autonomous inference.

What carries the argument

The load-bearing mechanism is a contrastive two-stage reward inside a GRPO reinforcement loop, coupled with a cosine-decay gate that withdraws external chain-of-thought prompts during training. Stage 1 rewards answer and format correctness; Stage 2, active only when Stage 1 succeeds, rewards agreement between the model's per-chain judgment and the chain's own conclusion ($r_{gain}=\mathrm{scale}_{gain}\cdot\tfrac{1}{k}\sum_{i} s_i$). EAAI selects which external CoTs enter the context according to threshold $0.5\,(1+\cos(\pi\,t))$, and disables Stage 2 when the selection probability becomes very low to avoid reward sparsity. The theoretical support is Proposition 2.1.1, a reward bound showing $E[R(C_{out})] \ge E[R(C_{SLM})]$ when reference CoTs are provided, and Proposition 2.1.2, which decomposes the reward into a learned weighted combination of reference rewards plus a non-negative generation gain.

What would settle it

At intermediate training checkpoints, evaluate the model both with and without external CoTs in the prompt while the cosine threshold decays; if the with/without accuracy gap does not shrink as the threshold falls, the fading schedule is not teaching internalized reasoning but simply exposing the model to fewer hints.

Watch

Extended reading notes

Core claim

The central discovery is that a small model's reasoning improves when reinforcement learning is supervised to contrast whole reasoning routes rather than to imitate only correct ones or to use external chains as static features. MRPV presents the model with k unfiltered teacher-generated chains (correct and incorrect together), asks it to predict both the answer and a per-chain correctness judgment, and gates a process reward r_gain behind a correct-answer reward r_base so that the auxiliary judgment sharpens main-task learning without displacing it. EAAI then withdraws the chains during training by sampling them with probability 0.5(1+cos(πt)), so at inference the model sees only the question; the paper reports that this preserves most of the benefit (e.g., 94.4 vs 96.0 on GSM8K for DeepSeek-R1-7B) while removing the external dependency. The paper also contributes a vertical-domain recipe: guided chain-of-thought distillation that injects expert comments into teacher prompts, validated on a newly introduced industrial ad-search relevance benchmark. Two formal propositions, a reward bound and a reward decomposition, state that referencing diverse CoTs raises the expected reward at least to the level of reasoning from scratch.

Load-bearing premise

The load-bearing premise is that gradually fading external reasoning traces during training makes the small model internalize the reasoning patterns, so that at test time accuracy without those traces stays close to accuracy with them.

Editorial extensions

If this is right

  • If ReaLM is correct, SLMs can be trained to match or beat GRPO-based RL and distillation baselines on math, commonsense, and factual reasoning while needing no external CoTs at inference.
  • Performance is highest when 20–80% of the training CoTs are wrong, which empirically argues against reject-sampling-only supervision and supports learning from failure.
  • The method transfers to a proprietary vertical domain, with ReaLM-R1 reaching 82.61 accuracy on the new ad-search relevance benchmark, suggesting rule-aware CoT distillation is a workable way to inject non-public rules.
  • The two propositions imply that any SLM trained with contrastive multi-route reference CoTs enjoys an expected-reward bound above unaided reasoning, making MRPV a generally applicable RL supervision layer.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Going beyond the paper: the EAAI cosine decay is never compared against fixed-probability or abrupt CoT dropping, so the shape of the fade-out curve is not isolated; a direct ablation would test whether smoothness matters.
  • Going beyond the paper: the SAME/DIFF error analysis could be extended by directly probing how much the final model attends to external traces when they are present, to verify internalization rather than residual dependence.
  • Going beyond the paper: the theoretical reward bound assumes teacher CoTs are at least as good as the student's unaided reasoning and that their embedding deviations are mildly negatively correlated; a weaker-teacher test would show where the bound breaks.
  • Going beyond the paper: rule-aware CoT distillation suggests a general recipe for RL training in any vertical domain that has expert annotations, so the industrial-dataset result is a template rather than a one-off.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes ReaLM, a reinforcement learning framework for small language models (SLMs) that combines three components: Multi-Route Process Verification (MRPV), which contrasts positive and negative external chain-of-thought (CoT) paths; Enabling Autonomy via Asymptotic Induction (EAAI), which fades the external CoT input over training; and guided CoT distillation with an industrial benchmark for vertical-domain generalization. The main empirical claim is that ReaLM improves SLM reasoning by 4.2% on average when external CoTs are available at inference and by 2.6% when they are not, while removing the need for external CoTs at test time. Experiments are conducted on GSM8K, BBH, MMLU, MRPC, and their out-of-domain variants, plus a proprietary ad-search relevance dataset, with two base models and three random seeds.

Significance. If the claims hold, ReaLM would be a meaningful step toward SLMs that reason robustly, autonomously, and in domain-specific settings without sacrificing efficiency. The study's strengths include ablations that isolate the process-verification reward, error analysis that examines learning from incorrect CoTs, and evaluation on two base models with multiple seeds. The public-benchmark results are plausible, but the central autonomy mechanism (EAAI) is not cleanly isolated, and the theoretical justification in Appendix C is circular at the load-bearing point. The industrial results lack error bars and are not directly reproducible due to the proprietary dataset.

major comments (5)
  1. [§3.3, Tables 1 and 3] The autonomy claim for EAAI is not isolated by the reported experiments. No condition trains with external CoTs at a constant rate and then evaluates without them; the comparison in Table 3 is between MRPV+EAAI (trained with a fade schedule, evaluated without CoTs) and RL (trained and evaluated without CoTs). The 2.6% average gain in Table 1 could therefore be explained by CoT-augmented RL training alone, rather than by the fade schedule specifically. Please add a control that keeps the CoT inclusion probability constant (e.g., p=1 or p=0.5) for the full training run and then evaluates without CoTs. In addition, report checkpoint evaluations showing whether the accuracy gap between with-CoT and without-CoT evaluation shrinks as the cosine threshold decays, which would directly test the internalization mechanism.
  2. [Appendix C, Propositions 2.1.1 and 2.1.2] The proof sketch assumes the property it sets out to establish. Eq. (19) requires the cross terms δ_i^T H δ_j to be nonnegative, which is justified by the assumption that the CoT deviation vectors are 'mildly negatively correlated' in a negative-definite reward landscape. That is precisely the contrastive-benefit property that MRPV is designed to induce; it is not shown to hold a priori for arbitrary CoT samples. Proposition 2.1.2 similarly posits Δ_gen ≥ 0 as 'additional gains' rather than deriving it from the stated assumptions. Since these assumptions are untested, the theoretical analysis does not provide independent support for MRPV. Please either validate the assumptions empirically (e.g., measure the Hessian-vector products on actual model embeddings) or reposition the discussion as an intuition rather than a theorem.
  3. [Footnote 3 vs. Appendix Tables A5 and A6] Footnote 3 states that external CoTs are 'without final answers,' but the GPT responses shown in Tables A5 and A6 end with tokens such as '#### 8' and '#### 9' (and similar answer strings). If these CoTs are used in the prompts exactly as shown, the with-CoT results in Table 1 include answer leakage, and the 4.2% gain over CoT-Synthesizer could be partly a selection effect rather than a reasoning improvement. Please clarify whether the final-answer tokens were stripped from the external CoTs before being fed to the SLM. If they were not stripped, rerun the with-CoT experiments with answer tokens removed and report the difference.
  4. [§3.1, Table 1] The baseline suite omits RFT (reject-sampling fine-tuning), which is the central comparison in the paper's motivation: the introduction explicitly criticizes RFT for filtering out negative reasoning paths. Without an RFT baseline, the claim that MRPV's contrastive supervision improves over reject-sampling is not directly demonstrated. Please add RFT (or an analogous reject-sampling SFT baseline) to Table 1 and the corresponding analysis.
  5. [§3.1, Table 5] The industrial benchmark results in Table 5 are reported without error bars or statistical significance tests, and the comparison includes only two baselines (SKIntern and CoT-Synthesizer). Because the dataset is proprietary and the expert comments are not publicly available, the reproducibility of this result is limited. Please report multiple seeds with variance, and consider adding a public vertical-domain task or releasing a sanitized subset to support the generalization claim.
minor comments (6)
  1. [Table A1] In Table A1, 'GSK8K' appears to be a typo for 'GSM8K'; the same typo appears in the dataset description in Section 3.1.
  2. [Table 2] The last column header in Table 2, 'ARAI', appears to be a typo for 'EAAI'.
  3. [Appendix C, after Eq. (16)] The sentence 'Assuming uniform weights wi = 1/k wi≠1/k' is garbled; it should read 'Assuming uniform weights, i.e., wi = 1/k'.
  4. [Section 3.3] The statement that 'MR+EAAI in Tab. 3 significantly outperforms the RL method' uses the word 'significantly' without a reported statistical test; please either provide a significance test or replace 'significantly' with a more neutral phrase.
  5. [Algorithm 1 and Eq. (6)] The description 'Only CoTs with sampled probability below this threshold are used' is slightly confusing because 'prob' in Algorithm 1 is a uniform random draw, not a property of the CoT. Clarify that the random draw determines inclusion probability.
  6. [Abstract and Section 2.1] The term 'reflection-enhanced' in the abstract is not explicitly defined. The method largely involves contrastive reasoning over provided CoTs and assessing their utility; consider defining 'reflection' in Section 2.1 to avoid ambiguity.

Circularity Check

1 steps flagged · score 6.0 of 10

Partial circularity: the theoretical reward decomposition is an assumed residual renamed as a proposition, while the empirical benchmark results remain independent measurements.

  1. self definitional [Sec. 2.1, Proposition 2.1.2 (Eq. 5); Appendix C, Eq. (12)]
    "Proposition 2.1.2 (Reward Decomposition) ... E[R(Cout)] = E[R(Pk i=1 wi·Ci)] + ∆gen, where ∆gen≥ 0 denotes additional gains from novel or refined generation by the SLM beyond direct imitation of the references. [Appendix C:] Following Assumption. 3, the SLM may either directly adopt or reject inputs from ei, and can further improve through refinement or novel reasoning. Formally, eR(eout) = eR(Pk i=1 wiei) + △gen ≥ eR(Pk i=1 wiei), where △gen ≥ 0 represents additional gain from novel or refined generation."

    For any output eout and any weights {wi} with Σwi = 1, the equality in (5) holds simply by defining Δgen as the residual; the only substantive content is the sign Δgen ≥ 0. The 'proof' of Proposition 2.1.2 does not derive this sign—it restates Eq. (12), where Δgen ≥ 0 is asserted 'following Assumption 3', i.e., the assumption that a well-trained SLM captures extra gains beyond direct combination of inputs. The decomposition is therefore the assumed gain renamed as a proposition, and Proposition 2.1.1 inherits the same assumption through Eqs. (11)-(12).

full rationale

The circular step is confined to the theoretical motivation, not the experimental measurements. Proposition 2.1.2's reward decomposition is a definitional identity plus an assumed nonnegative residual; since the 'additional gain' Δgen is exactly what the proposition claims to establish, the first-principles justification of MRPV reduces to its own input at the load-bearing point. Proposition 2.1.1 then inherits this problem through Eq. (12), and its proof further relies on an unstated complementarity condition in Eq. (19), so the reward bound is not derived from premises independent of the intended conclusion. However, the paper's central numerical claims are direct accuracy measurements against external benchmarks (GSM8K, BBH, MMLU, MRPC, GSM8KPlus, AGIEval, BB-sub, ARC-C/E) and independent baselines; no benchmark number is fitted from the propositions or generated by construction. The ablation tables also provide empirical comparisons that stand apart from the theory. The missing control for a constant-CoT-rate training condition is a genuine experimental confound for the EAAI autonomy mechanism, but it is not a circular reduction, so it does not raise the score to 8 or above. The self-citations in the paper are not load-bearing for the empirical results.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The training recipe relies on several unspecified coefficients (scale_gain, k, rule-injection probability, reward-disable threshold). The theoretical motivation rests on five assumptions, at least two of which (Assumptions 3 and 4) read as restatements of the result. No new physical or conceptual entities are introduced.

free parameters (4)
  • scale_gain = not specified
    Multiplier for rgain in Eq. (3); no value given in main text or Appendix B, yet it controls the strength of the CoT-judgment reward.
  • k (number of sampled external CoTs per prompt) = not specified
    Section 2.1 says k ≤ N CoTs are sampled; N=10 is given in Appendix B, but k and the sampling distribution over the 10 candidates are not reported.
  • rule injection probability = not specified
    Section 2.3 and Fig. 2(a).1 inject expert comments into teacher prompts 'with a certain probability'; the probability is not given.
  • EAAI reward-disable threshold = not specified
    Section 2.2 disables Stage 2 rewards when the CoT sampling probability becomes 'too low'; the threshold is not specified.
assumptions (5)
  • ad hoc to paper Continuous reward eR(e)=E_{C∼p(C|e)}[R(C)] exists and optimizing policy gradient is equivalent to optimizing E[eR(φ(C))].
    Appendix C Assumption 1; introduces a smooth reward over semantic embeddings without proof.
  • domain assumption The teacher LLM used for CoT generation is not weaker than the SLM when no external CoTs are provided.
    Appendix C Assumption 2; plausible for GPT-4-turbo vs 7B models, but not measured on each benchmark.
  • ad hoc to paper A well-trained SLM can capture extra gains beyond direct combination of inputs.
    Appendix C Assumption 3; this is the Δgen≥0 term in Eq. (12), so it already assumes the conclusion of Proposition 2.1.2.
  • ad hoc to paper Sampled CoT embeddings are scattered around the optimum and deviations are mildly negatively correlated.
    Appendix C Assumption 4 and the paragraph before Eq. (19); the sign of δ_i^T H δ_j is asserted without proof.
  • standard math The reward surface is locally quadratic with negative definite Hessian at the optimum.
    Used for the second-order Taylor expansion in Appendix C Eq. (14).

how reviews work

0 comments
Cite this review

Pith. "Pith review of ReaLM: Reflection-Enhanced Autonomous Reasoning with Small Language Models." pith.science (2026). https://pith.science/paper/TJGWXQC2

@misc{pith2026250812387,
  author       = {Pith},
  title        = {Pith review of: ReaLM: Reflection-Enhanced Autonomous Reasoning with Small Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TJGWXQC2}},
  note         = {Machine review of arXiv:2508.12387}
}
read the original abstract

Small Language Models (SLMs) are a cost-effective alternative to Large Language Models (LLMs), but often struggle with complex reasoning due to their limited capacity and a tendency to produce mistakes or inconsistent answers during multi-step reasoning. Existing efforts have improved SLM performance, but typically at the cost of one or more of three key aspects: (1) reasoning capability, due to biased supervision that filters out negative reasoning paths and limits learning from errors; (2) autonomy, due to over-reliance on externally generated reasoning signals; and (3) generalization, which suffers when models overfit to teacher-specific patterns. In this paper, we introduce ReaLM, a reinforcement learning framework for robust and self-sufficient reasoning in vertical domains. To enhance reasoning capability, we propose Multi-Route Process Verification (MRPV), which contrasts both positive and negative reasoning paths to extract decisive patterns. To reduce reliance on external guidance and improve autonomy, we introduce Enabling Autonomy via Asymptotic Induction (EAAI), a training strategy that gradually fades external signals. To improve generalization, we apply guided chain-of-thought distillation to encode domain-specific rules and expert knowledge into SLM parameters, making them part of what the model has learned. Extensive experiments on both vertical and general reasoning tasks demonstrate that ReaLM significantly improves SLM performance across aspects (1)-(3) above.

Figures

Figures reproduced from arXiv: 2508.12387 by the authors.

Figure 1
Figure 1. Comparison with typical SLM reasoning methods. (i) RFT uses only correct CoTs, limiting learning from errors (capabil￾ity). (ii) CoT-based Synthesizer requires external CoTs at inference, reducing independence (autonomy). (iii) LLMs-as-Instructor tailors data but struggles in new domains (generalization). (iv) ReaLM addresses all three by learning from contrastive CoTs, fading ex￾ternal signals, and distilling exper… view at source ↗
Figure 2
Figure 2. Overview of ReaLM. (a) The architecture comprises three stages. 1 External CoTs are generated with optional rule injection. 2 EAAI samples k CoTs via a guidance-fading schedule. 3 MRPV performs RL with two key designs: (i) scoring each CoT alongside typical reasoning, and (ii) applying a two-stage reward to encourage comparison and reflection. (b) It supports two strategies: ReaLM-Zero (pure RL) and ReaLM-R1 (iterat… view at source ↗
Figure 3
Figure 3. Impact of Incorrect CoT Ratio on Performance. Variants are initialled with Deepseek-R1-7B and evaluated on GSM8K. (a) With external CoTs provided during inference, and (b) without external CoTs provided during inference. Methods Acc↑ F1 Score↑ # Other Advanced Methods SKIntern 80.19 79.05 CoT-synthesizer 81.54 80.26 # Different Variants of ReaLM ReaLM-Zero w/o Expert Comments 82.30 81.40 ReaLM-Zero 82.56 81.80 ReaLM… view at source ↗

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 5 canonical work pages

  1. [1]

    Chen, H.; Wu, S.; Quan, X.; Wang, R.; Yan, M.; and Zhang, J. 2023 a . Mcc-kd: Multi-cot consistent knowledge distillation. arXiv preprint arXiv:2310.14747

    arXiv 2023

  2. [2]

    Chen, S.; Chen, Y.; Li, Z.; Jiang, Y.; Wan, Z.; He, Y.; Ran, D.; Gu, T.; Li, H.; Xie, T.; et al. 2025. Recent advances in large langauge model benchmarks against data contamination: From static to dynamic evaluation. arXiv preprint arXiv:2502.17521

    arXiv 2025

  3. [3]

    Chen, X.; Aksitov, R.; Alon, U.; Ren, J.; Xiao, K.; Yin, P.; Prakash, S.; Sutton, C.; Wang, X.; and Zhou, D. 2023 b . Universal self-consistency for large language model generation. arXiv preprint arXiv:2311.17311

    arXiv 2023

  4. [4]

    Chenglin, L.; Chen, Q.; Li, L.; Wang, C.; Tao, F.; Li, Y.; Chen, Z.; and Zhang, Y. 2024. Mixed distillation helps smaller language models reason better. In Findings of the Association for Computational Linguistics: EMNLP 2024, 1673--1690

    work page 2024

  5. [5]

    Clark, P.; Cowhey, I.; Etzioni, O.; Khot, T.; Sabharwal, A.; Schoenick, C.; and Tafjord, O. 2018. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457

    arXiv 2018

  6. [6]

    Cobbe, K.; Kosaraju, V.; Bavarian, M.; Chen, M.; Jun, H.; Kaiser, L.; Plappert, M.; Tworek, J.; Hilton, J.; Nakano, R.; et al. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168

    arXiv 2021

  7. [7]

    Dai, C.; Li, K.; Zhou, W.; and Hu, S. 2024. Improve Student's Reasoning Generalizability through Cascading Decomposed CoTs Distillation. arXiv preprint arXiv:2405.19842

    arXiv 2024

  8. [8]

    L.; and Mali, A

    Dave, N.; Kifer, D.; Giles, C. L.; and Mali, A. 2024. Investigating symbolic capabilities of large language models. arXiv preprint arXiv:2405.13209

    arXiv 2024

Show all 46 references
  1. [9]

    Dolan, B.; and Brockett, C. 2005. Automatically constructing a corpus of sentential paraphrases. In Third international workshop on paraphrasing (IWP2005)

    2005

  2. [10]

    Geva, M.; Khashabi, D.; Segal, E.; Khot, T.; Roth, D.; and Berant, J. 2021. Did aristotle use a laptop? a question answering benchmark with implicit reasoning strategies. Transactions of the Association for Computational Linguistics, 9: 346--361

    2021

  3. [11]

    Guo, D.; Yang, D.; Zhang, H.; Song, J.; Zhang, R.; Xu, R.; Zhu, Q.; Ma, S.; Wang, P.; Bi, X.; et al. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948

    2025 arXiv

  4. [12]

    Ham, G.; Cho, Y.; Lee, J.-H.; Kang, M.; Choi, G.; and Kim, D. 2024. Difficulty level-based knowledge distillation. Neurocomputing, 606: 128375

    2024

  5. [13]

    Han, H.; Kim, S.; Choi, H.-S.; and Yoon, S. 2023. On the impact of knowledge distillation for model interpretability. arXiv preprint arXiv:2305.15734

    2023 arXiv

  6. [14]

    Hendrycks, D.; Burns, C.; Basart, S.; Zou, A.; Mazeika, M.; Song, D.; and Steinhardt, J. 2020. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300

    2020 arXiv

  7. [15]

    Hsieh, C.-Y.; Li, C.-L.; Yeh, C.-K.; Nakhost, H.; Fujii, Y.; Ratner, A.; Krishna, R.; Lee, C.-Y.; and Pfister, T. 2023. Distilling step-by-step! outperforming larger language models with less training data and smaller model sizes. arXiv preprint arXiv:2305.02301

    2023 arXiv

  8. [16]

    Hui, B.; Yang, J.; Cui, Z.; Yang, J.; Liu, D.; Zhang, L.; Liu, T.; Zhang, J.; Yu, B.; Lu, K.; et al. 2024. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409.12186

    2024 arXiv

  9. [17]

    L.; Anthony, Q.; Lesort, T.; Belilovsky, E.; and Rish, I

    Ibrahim, A.; Th \'e rien, B.; Gupta, K.; Richter, M. L.; Anthony, Q.; Lesort, T.; Belilovsky, E.; and Rish, I. 2024. Simple and scalable strategies to continually pre-train large language models. arXiv preprint arXiv:2403.08763

    2024 arXiv

  10. [18]

    Jaech, A.; Kalai, A.; Lerer, A.; Richardson, A.; El-Kishky, A.; Low, A.; Helyar, A.; Madry, A.; Beutel, A.; Carney, A.; et al. 2024. Openai o1 system card. arXiv preprint arXiv:2412.16720

    2024 arXiv

  11. [19]

    B.; Chess, B.; Child, R.; Gray, S.; Radford, A.; Wu, J.; and Amodei, D

    Kaplan, J.; McCandlish, S.; Henighan, T.; Brown, T. B.; Chess, B.; Child, R.; Gray, S.; Radford, A.; Wu, J.; and Amodei, D. 2020. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361

    2020 arXiv

  12. [20]

    J.; Kim, D.; Jang, J.; Ye, S.; Shin, J.; and Seo, M

    Kim, S.; Joo, S. J.; Kim, D.; Jang, J.; Ye, S.; Shin, J.; and Seo, M. 2023. The cot collection: Improving zero-shot and few-shot learning of language models via chain-of-thought fine-tuning. arXiv preprint arXiv:2305.14045

    2023 arXiv

  13. [21]

    Li, Q.; Cui, L.; Zhao, X.; Kong, L.; and Bi, W. 2024. Gsm-plus: A comprehensive benchmark for evaluating the robustness of llms as mathematical problem solvers. arXiv preprint arXiv:2402.19255

    2024 arXiv

  14. [22]

    Li, S.; Chen, J.; Shen, Y.; Chen, Z.; Zhang, X.; Li, Z.; Wang, H.; Qian, J.; Peng, B.; Mao, Y.; et al. 2022. Explanations from large language models make small reasoners better. arXiv preprint arXiv:2210.06726

    2022 arXiv

  15. [23]

    Li, Y.; Ma, Y.; Yan, S.; Zhang, C.; Liu, J.; Lu, J.; Xu, Z.; Chen, M.; Wang, M.; Zhan, S.; et al. 2025. Model Merging in Pre-training of Large Language Models. arXiv preprint arXiv:2505.12082

    2025 arXiv

  16. [24]

    Liao, H.; He, S.; Hao, Y.; Li, X.; Zhang, Y.; Zhao, J.; and Liu, K. 2024. SKIntern: Internalizing Symbolic Knowledge for Distilling Better CoT Capabilities into Small Language Models. arXiv preprint arXiv:2409.13183

    2024 arXiv

  17. [25]

    Lightman, H.; Kosaraju, V.; Burda, Y.; Edwards, H.; Baker, B.; Lee, T.; Leike, J.; Schulman, J.; Sutskever, I.; and Cobbe, K. 2023. Let's verify step by step. In The Twelfth International Conference on Learning Representations

    2023

  18. [26]

    Liu, W.; Li, G.; Zhang, K.; Du, B.; Chen, Q.; Hu, X.; Xu, H.; Chen, J.; and Wu, J. 2023. Mind's mirror: Distilling self-evaluation capability and comprehensive thinking from large language models. arXiv preprint arXiv:2311.09214

    2023 arXiv

  19. [27]

    Rajbhandari, S.; Rasley, J.; Ruwase, O.; and He, Y. 2020. Zero: Memory optimizations toward training trillion parameter models. In SC20: International Conference for High Performance Computing, Networking, Storage and Analysis, 1--16. IEEE

    2020

  20. [28]

    Rajpurkar, P.; Zhang, J.; Lopyrev, K.; and Liang, P. 2016. Squad: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250

    2016 arXiv

  21. [29]

    Ramos, D.; Mamede, C.; Jain, K.; Canelas, P.; Gamboa, C.; and Le Goues, C. 2025. Are large language models memorizing bug benchmarks? In 2025 IEEE/ACM International Workshop on Large Language Models for Code (LLM4Code), 1--8. IEEE

    2025

  22. [30]

    Sener, O.; and Savarese, S. 2017. Active learning for convolutional neural networks: A core-set approach. arXiv preprint arXiv:1708.00489

    2017 arXiv

  23. [31]

    Setlur, A.; Nagpal, C.; Fisch, A.; Geng, X.; Eisenstein, J.; Agarwal, R.; Agarwal, A.; Berant, J.; and Kumar, A. 2024. Rewarding progress: Scaling automated process verifiers for llm reasoning. arXiv preprint arXiv:2410.08146

    2024 arXiv

  24. [32]

    Shao, Z.; Wang, P.; Zhu, Q.; Xu, R.; Song, J.; Bi, X.; Zhang, H.; Zhang, M.; Li, Y.; Wu, Y.; et al. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300

    2024 arXiv

  25. [33]

    A.; Abid, A.; Fisch, A.; Brown, A

    Srivastava, A.; Rastogi, A.; Rao, A.; Shoeb, A. A.; Abid, A.; Fisch, A.; Brown, A. R.; Santoro, A.; Gupta, A.; Garriga-Alonso, A.; et al. 2023. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. Transactions on machine learning research

    2023

  26. [34]

    W.; Chowdhery, A.; Le, Q

    Suzgun, M.; Scales, N.; Sch \"a rli, N.; Gehrmann, S.; Tay, Y.; Chung, H. W.; Chowdhery, A.; Le, Q. V.; Chi, E. H.; Zhou, D.; et al. 2022. Challenging big-bench tasks and whether chain-of-thought can solve them. arXiv preprint arXiv:2210.09261

    2022 arXiv

  27. [35]

    Touvron, H.; Martin, L.; Stone, K.; Albert, P.; Almahairi, A.; Babaei, Y.; Bashlykov, N.; Batra, S.; Bhargava, P.; Bhosale, S.; et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

    2023 arXiv

  28. [36]

    Wang, H.; Xiong, W.; Xie, T.; Zhao, H.; and Zhang, T. 2024. Interpretable preferences via multi-objective reward modeling and mixture-of-experts. arXiv preprint arXiv:2406.12845

    2024 arXiv

  29. [37]

    Wang, X.; Wei, J.; Schuurmans, D.; Le, Q.; Chi, E.; Narang, S.; Chowdhery, A.; and Zhou, D. 2022. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171

    2022 arXiv

  30. [38]

    V.; Zhou, D.; et al

    Wei, J.; Wang, X.; Schuurmans, D.; Bosma, M.; Xia, F.; Chi, E.; Le, Q. V.; Zhou, D.; et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35: 24824--24837

    2022

  31. [39]

    Xu, J.; Zhou, M.; Liu, W.; Liu, H.; Han, S.; and Zhang, D. 2025. TwT: Thinking without Tokens by Habitual Reasoning Distillation with Multi-Teachers' Guidance. arXiv preprint arXiv:2503.24198

    2025

  32. [40]

    Ying, J.; Lin, M.; Cao, Y.; Tang, W.; Wang, B.; Sun, Q.; Huang, X.; and Yan, S. 2024. Llms-as-instructors: Learning from errors toward automating model improvement. arXiv preprint arXiv:2407.00497

    2024 arXiv

  33. [41]

    Yuan, Z.; Yuan, H.; Li, C.; Dong, G.; Lu, K.; Tan, C.; Zhou, C.; and Zhou, J. 2023. Scaling Relationship on Learning Mathematical Reasoning with Large Language Models. arXiv:2308.01825

    2023 arXiv

  34. [42]

    Zhang, B.; Zhang, X.; Zhang, J.; Yu, J.; Luo, S.; and Tang, J. 2025 a . Cot-based synthesizer: Enhancing llm performance through answer synthesis. arXiv preprint arXiv:2501.01668

    2025 arXiv

  35. [43]

    Zhang, C.; Song, D.; Ye, Z.; and Gao, Y. 2023. Towards the law of capacity gap in distilling language models. arXiv preprint arXiv:2311.07052

    2023 arXiv

  36. [44]

    Zhang, X.; Wen, S.; Wu, W.; and Huang, L. 2025 b . Tinyllava-video-r1: Towards smaller lmms for video reasoning. arXiv preprint arXiv:2504.09641

    2025 arXiv

  37. [45]

    Zhong, W.; Cui, R.; Guo, Y.; Liang, Y.; Lu, S.; Wang, Y.; Saied, A.; Chen, W.; and Duan, N. 2023. Agieval: A human-centric benchmark for evaluating foundation models. arXiv preprint arXiv:2304.06364

    2023 arXiv

  38. [46]

    Zhou, H.; Song, L.; Chen, J.; Zhou, Y.; Wang, G.; Yuan, J.; and Zhang, Q. 2021. Rethinking soft labels for knowledge distillation: A bias-variance tradeoff perspective. arXiv preprint arXiv:2102.00650

    2021 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.