Pith. sign in

REVIEW 4 major objections 5 minor 46 references

LLM math post-training can be improved by separating abstract strategy learning from concrete arithmetic: first fine-tune on number-free meta-thought traces, then reinforce with confidence-calibrated rewards on intermediate computations.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 06:45 UTC pith:OHUMAJQC

load-bearing objection The accuracy gains are internally consistent and the pipeline is plausible, but the abstraction story is confounded with target quality and the calibration evidence is circular. the 4 major comments →

arxiv 2601.21909 v2 pith:OHUMAJQC submitted 2026-01-29 cs.AI cs.CL

From Meta-Thought to Execution: Cognitively Aligned Post-Training for Generalizable and Reliable LLM Reasoning

Shaojie Wang , Liang Zhang This is my paper
classification cs.AI cs.CL
keywords mathematical reasoninglarge language modelspost-trainingchain-of-meta-thoughtconfidence calibrationreinforcement learningout-of-distribution generalizationtraining efficiency
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that the dominant post-training recipe for LLM math reasoning—imitate full step-by-step solutions, then reinforce correct final answers—limits generalization because it fuses abstract strategy with specific arithmetic. It proposes splitting training to mirror human cognition: Chain-of-Meta-Thought (CoMT) teaches the model to produce number-free reasoning skeletons (variable names, no calculations), and Confidence-Calibrated Reinforcement Learning (CCRL) rewards confident-correct intermediate computations while penalizing confident-wrong ones. On four base models and several benchmarks, the paper reports average gains of +2.19 points in-distribution and +4.63 points out-of-distribution over the standard pipeline, with roughly half the supervision tokens and 65-70% less training time. If right, the two-stage split improves accuracy, transfer, and training cost simultaneously.

Core claim

CoMT+CCRL is the paper's core claim: LLM math post-training should be split into meta-knowledge acquisition and task adaptation. During meta-knowledge acquisition, a teacher produces meta-thoughts—reasoning steps described in natural language with variable names and no concrete numbers—and the target model is SFT'd to imitate them. During task adaptation, PPO uses a combined reward: answer correctness plus a confidence term computed from the maximum token-level entropy over the 'computed numbers' the model emits, transformed to e^{-H_max}. Correct answers with confident intermediate arithmetic get extra reward; wrong answers with confident arithmetic get extra penalty. The paper reports that

What carries the argument

Two coupled mechanisms carry the argument. CoMT targets are abstract solution traces generated under an explicit 'use variable names, do not calculate numbers' instruction; they give the model a reusable reasoning skeleton without surface arithmetic to memorize. CCRL uses the model's own predictive entropy at each emitted computed-number token: H_max is the highest token-level entropy across those positions, i.e., the model's least confident computational step, and e^{-H_max} converts this to a confidence score in (0,1]. The total reward adds α·e^{-H_max} to a correct answer and subtracts β·e^{-H_max} from an incorrect one, so the training signal explicitly rewards being confident only when

Load-bearing premise

Everything in the RL stage rests on the assumption that the entropy of the model's token distribution at the moment it outputs a computed number is a faithful measure of whether that intermediate computation is correct; the paper does not validate this assumption against step-level correctness.

What would settle it

Collect held-out model trajectories, label each computed-number step correct or incorrect, bin steps by their e^{-H_max} confidence, and plot the resulting reliability curve; if high-confidence bins are not substantially more accurate than low-confidence bins (or if CCRL training only lowers reported scores without improving step accuracy), the confidence-calibration mechanism is not doing the claimed work.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • CoMT can replace standard CoT-SFT as the supervised stage: it yields better accuracy with roughly half the target tokens, so the same compute budget trains more or better data.
  • RL for math reasoning does not need an external process reward model: a token-level entropy signal on intermediate numbers provides a cheap calibration reward.
  • Strategy-level supervision transfers: out-of-distribution gains (+4.63 points) are larger than in-distribution gains (+2.19 points), suggesting the abstract traces are what generalize.
  • Models become less overconfident on errors: at confidence >0.5 the share of confident mistakes drops by 27-65% across base models, reducing the risk of silent wrong answers.
  • The two-stage recipe is compatible with existing PPO post-training infrastructure, so it can be adopted without replacing the optimization stack.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If entropy at computed-number tokens truly tracks stepwise correctness, CCRL is a parameter-free substitute for process reward models; the natural next test is to benchmark it against learned process supervision on the same trajectories.
  • The CoMT 'no numbers' instruction suggests a testable mechanism: model gains should shrink if the training data is replaced by complete solutions whose numbers are replaced by variables while keeping solution length identical—isolating abstraction from length.
  • The same two-stage template could extend beyond arithmetic to any multi-step reasoning with identifiable intermediate artifacts (symbolic algebra, code, or structured tool calls), provided a cheap confidence proxy for those artifacts exists.
  • A caution worth testing: because the overconfidence metric uses the same e^{-H_max} quantity that CCRL optimizes, independent verification of calibration (e.g., reliability diagrams) is needed before taking the confidence gains at face value.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a two-stage post-training framework for math reasoning. Stage 1, Chain-of-Meta-Thought (CoMT), performs SFT on teacher-generated 'meta-thought' trajectories that describe reasoning steps with variable names and no concrete numeric calculations. Stage 2, Confidence-Calibrated Reinforcement Learning (CCRL), runs PPO with a reward that adds a confidence term exp(-H_max) to outcome correctness, where H_max is the maximum token-level entropy over computed-number tokens. The authors evaluate LLaMA-3.1-8B, Qwen2.5-7B, Qwen3-4B, and Qwen3-8B on GSM8K/SVAMP (in-distribution) and AsDiv, SVAMP, MAWPS, TabMWP, GSM-Hard (out-of-distribution), reporting average gains of +2.19 and +4.63 over a CoT-SFT+RL baseline, together with 65-70% reductions in SFT training time and roughly 50% shorter supervision tokens.

Significance. If the empirical claims held, the paper would make a useful practical contribution: it offers a simple way to split post-training into strategy-level SFT and confidence-aware RL, with appealing efficiency gains and no architectural changes. The experimental breadth (four base models, multiple benchmarks, OOD and symbolic-problem evaluations) is a genuine strength. The core empirical claims, however, rest on an uncontrolled comparison (CoT-SFT uses unfiltered teacher rationales while CoMT does not), and the headline calibration evidence is partly entailed by the optimized reward. These are fixable within the manuscript's scope, but they are load-bearing and currently prevent the paper from supporting its central claims.

major comments (4)
  1. [Sections 3.1, 4.1; Tables 1-2] CoT-SFT is trained on complete LLaMA-3.1-70B CoT trajectories with no correctness filtering, while CoMT targets are generated under the prompt 'Do NOT calculate any specific numbers' and therefore contain no arithmetic tokens. The SFT comparison varies both abstraction and target noise: if the teacher makes arithmetic errors on a nontrivial fraction of solutions, CoT-SFT is trained on incorrect rationales/answers, whereas CoMT targets are immune to this error class by construction. This alone could explain the +3.91 ID and +7.35 OOD gains attributed to meta-knowledge. A control baseline using verified-correct CoT rationales (e.g., filtered to those yielding the ground-truth answer) and reporting teacher difficulty statistics is needed.
  2. [Section 4.2.4, Eq. 5] The overconfidence analysis is circular. It reports P(exp(-H_max) > c | incorrect), where exp(-H_max) is exactly the quantity CCRL's reward penalizes on incorrect trajectories (-beta exp(-H_max)). A reduction in this metric is partially entailed by optimizing the objective, not evidence of improved calibration. Independent metrics such as expected calibration error (ECE) or NLL against accuracy, ideally computed on held-out data and compared against outcome-only RL with the same confidence score, are needed before claiming that CCRL 'successfully calibrates model confidence.'
  3. [Section 3.2.2, Eqs. 2-5] The entire CCRL mechanism assumes that token-level entropy of the predictive distribution at computed-number positions is a valid confidence proxy for intermediate computational correctness. No validation of this proxy is reported: no step-level correctness annotations, no comparison of entropy on correct vs. incorrect computational steps, and no alternative confidence measure (e.g., self-consistency or verbalized confidence). If the proxy is weak, both the training signal and the calibration evaluation rest on an unvalidated surrogate. The authors should provide direct evidence that computed-token entropy tracks step correctness, or ablate CCRL under alternative confidence definitions.
  4. [Sections 4.2.1-4.2.3, Tables 1-2, Figure 2] Results are single-run with no error bars, seeds, or significance tests. Several deltas are small (ablation range +0.2% to +2.5%; Table 1, Qwen2.5-7B SVAMP shows CoMT+CCRL 90.10 vs. CoT-SFT+RL 90.90, a degradation). The headline +2.19 ID and +4.63 OOD averages may not be statistically robust. Reporting multiple seeds with confidence intervals and paired significance tests across benchmark/model sets is necessary to support the 'consistent across all models' wording.
minor comments (5)
  1. [Abstract/paper] The paper abstract reports '2.10% and 3.86% improvements' and 'ten benchmarks,' while the full text and tables report 2.19%, 4.63%, and eight benchmarks. These numbers should be reconciled.
  2. [Table 1] DeepSeek-Math and Qwen2.5-Math reference rows are repeated four times identically; this is likely a formatting artifact, but as printed it conflates per-model reference values. Consider reporting them as single reference lines.
  3. [Appendix B / Section 5] The '65-70% less training time' claim is only supported by SFT-stage timings (Table 3); the RL stage is not timed/reported. The abstract/conclusion claim about total post-training time should be relaxed or supplemented with RL-stage costs.
  4. [Section 4.1 / Table 2] Section 4.1 says the training sets for CoT-SFT+RL and CoMT+CCRL are the complete GSM8K and SVAMP training sets, while Table 2 says models are 'trained on GSM8K.' Clarify which dataset(s) each table uses so the notion of OOD is unambiguous.
  5. [Throughout] Minor naming/formatting issues: 'SV AMP' is used inconsistently (SVAMP), and the symbolic benchmark is called both GSM8K-Symbolic and GSM-Symbolic in Figures/Text.

Circularity Check

1 steps flagged

CCRL's overconfidence evidence is drawn from the same e^{-Hmax} term its reward optimizes; headline accuracy results remain independent.

specific steps
  1. fitted input called prediction [Section 3.2.3 Eq. (5) and Section 4.2.4 (Overconfidence Analysis, Figure 3)]
    "rconfidence(q, y) = ( α·e−Hmax if answer is correct; −β·e−Hmax if answer is incorrect ) (5) ... Following Section 3.2.3, for each incorrect prediction, we compute the entropy Ht for all computed numbers ... then use Hmax ... The confidence score is c=e−Hmax (Equation 5). We measure overconfidence rates at thresholds c>0.5,0.7,0.9 ... These results validate that CCRL successfully calibrates model confidence."

    The reward in Eq. (5) directly penalizes e^{-Hmax} on incorrect trajectories and rewards it on correct ones; PPO therefore optimizes the policy to lower e^{-Hmax} on errors. Section 4.2.4 then defines the same c=e^{-Hmax} as 'confidence' and reports P(high-confidence | incorrect) as an overconfidence rate, concluding CCRL 'successfully calibrates model confidence.' The measured reduction is entailed by the training objective rather than independent evidence of calibration: a model trained to minimize e^{-Hmax} on incorrect answers will, by construction, show fewer high-e^{-Hmax} errors. No ECE-style comparison of predicted confidence to empirical accuracy is reported, so the calibration claim rests on a metric that is the training signal itself.

full rationale

The central accuracy claims (Tables 1-2, Figures 2 and 4) are external benchmark comparisons; CoMT's SFT targets and CCRL's reward are clearly described inputs, and no load-bearing self-citation or uniqueness theorem is invoked. The efficiency results (token/time reduction) follow from the deliberately shorter number-free CoMT targets but are reported as design properties rather than independent predictions. The one genuine circularity is the overconfidence analysis: Section 4.2.4 evaluates calibration with the same e^{-Hmax} quantity that Eq. (5) uses as the confidence reward, so the reported 27-70% overconfidence reductions are partly forced by the objective. This affects the paper's calibration evidence, not the benchmark accuracy results, hence a partial circularity score of 6 rather than a higher score.

Axiom & Free-Parameter Ledger

6 free parameters · 6 axioms · 1 invented entities

The framework adds no physical entities; its load-bearing assumptions are domain assumptions about human cognition mapping to LLM training, about entropy-as-confidence, and about teacher meta-thought quality. Free parameters are concentrated in the reward design and RL configuration, none of which are given numeric values in the paper.

free parameters (6)
  • alpha (confidence reward scale, correct) = unspecified
    Eq. 5: weights confidence reward for correct answers relative to outcome reward; value never given; hand-tuned.
  • beta (confidence penalty scale, incorrect) = unspecified
    Eq. 5: controls penalty strength for confident errors; never stated.
  • r+, r- (outcome reward values) = unspecified
    Eq. 6: absolute outcome reward scale; never stated.
  • PPO/GAE hyperparameters (gamma, lambda, epsilon_clip, beta_KL, LR, epochs, rollouts) = unspecified
    Appendix A gives equations but no numeric values; 'same hyperparameters' claimed without listing them.
  • Teacher model choice and exact CoMT prompt = LLaMA-3.1-70B-Instruct
    CoMT quality depends on teacher output; the abstract claims robustness to teacher selection but no experiment varies it.
  • Computed-number token detection rule = numeric token not in N_q
    Defines which tokens get entropy scoring; BPE multi-token numbers make this ambiguous; rule never validated.
axioms (6)
  • domain assumption Human problem-solving decomposes into abstract-strategy acquisition then instance adaptation, and mirroring this in LLM post-training is the right inductive bias.
    Section 1 framing; motivates the entire design; no experiment isolates whether the cognitive analogy, rather than the specific training signal, causes the gains.
  • domain assumption Token-level entropy at computed-number positions measures computation confidence.
    Section 3.2.2, Eqs. 2-4; load-bearing for CCRL; unvalidated as a calibration measure (no ECE-style check).
  • domain assumption Teacher-generated variable-name meta-thoughts are correct abstract strategies.
    Section 3.1 data generation; quality of D_CoMT is never checked (no human or gold verification of meta-thoughts).
  • domain assumption Training under the CoMT prompt transfers to improved standard 'Let's think step by step' reasoning at inference.
    Section 3.1 trains under the CoMT prompt; Section 4.1 evaluates under the CoT prompt; the transfer mechanism is asserted, not tested.
  • domain assumption Exact final-number match is a sufficient correctness signal for RL and evaluation.
    Section 4.1 evaluation protocol; normalization for decimals/formatting is unspecified.
  • standard math PPO with GAE and KL penalty behaves as standard (Schulman et al. 2015, 2017).
    Appendix A; standard machinery, not the paper's contribution.
invented entities (1)
  • meta-thought / meta-knowledge no independent evidence
    purpose: New SFT training target: abstract, number-free reasoning plans intended to be the 'strategy' stage of reasoning.
    A post-hoc label for a training signal, not an observable entity; it has no falsifiable handle outside this paper's own benchmarks.

pith-pipeline@v1.3.0-alltime-deepseek · 18137 in / 28134 out tokens · 274009 ms · 2026-08-03T06:45:21.694916+00:00 · methodology

0 comments
read the original abstract

Current LLM post-training methods optimize complete reasoning trajectories through Supervised Fine-Tuning (SFT) followed by outcome-based Reinforcement Learning (RL). While effective, a closer examination reveals a fundamental gap: this approach does not align with how humans actually solve problems. Human cognition naturally decomposes problem-solving into two distinct stages: first acquiring abstract strategies (i.e., meta-knowledge) that generalize across problems, then adapting them to specific instances. In contrast, by treating complete trajectories as basic units, current methods are inherently problem-centric, entangling abstract strategies with problem-specific execution. To address this misalignment, we propose a cognitively-inspired framework that explicitly mirrors the two-stage human cognitive process. Specifically, Chain-of-Meta-Thought CoMT focuses supervised learning on abstract reasoning patterns without specific executions, enabling acquisition of generalizable strategies. Confidence-Calibrated Reinforcement Learning (CCRL) then optimizes task adaptation via confidence-aware rewards on intermediate steps, preventing overconfident errors from cascading and improving execution reliability. Experiments across four models and ten benchmarks show 2.10% and 3.86% improvements in-distribution and out-of-distribution respectively over standard methods, while remaining highly robust to variations in teacher model selection, optimization methods, and symbolic perturbations.

Figures

Figures reproduced from arXiv: 2601.21909 by Liang Zhang, Shaojie Wang.

Figure 1
Figure 1. Figure 1: Overview of our two-stage post-training framework with a concrete example from GSM8K. Stage 1 (Meta-Knowledge Acquisition): A teacher LLM generates abstract meta-thoughts excluding numerical calculations, which are used for CoMT super￾vised fine-tuning. Stage 2 (Task Adaptation): The CoMT-tuned model undergoes RL with rewards combining answer correctness and intermediate confidence scores (highlighted in t… view at source ↗
Figure 2
Figure 2. Figure 2: Ablation study comparing standard outcome-based Rein￾forcement Learning and our Confidence-Calibrated Reinforcement Learning (CCRL). Both methods are applied to CoMT-tuned mod￾els with. CCRL consistently outperforms standard RL by incor￾porating confidence-aware rewards on intermediate computations, with improvements ranging from +0.2% to +2.5% across models and datasets. 4.2.1. IN-DISTRIBUTION RESULTS [P… view at source ↗
Figure 4
Figure 4. Figure 4: Generalization evaluation on GSM-Symbolic benchmark. GSM-Symbolic tests genuine reasoning capabilities through sym￾bolic variations of mathematical problems. Models are trained on GSM8K and SVAMP, then evaluated on GSM-Symbolic. CoMT+CCRL consistently outperforms CoT-SFT+RL consistently across all models, suggesting that our framework enables more robust reasoning. derlying reasoning patterns—thus making i… view at source ↗
Figure 3
Figure 3. Figure 3: Overconfidence analysis on incorrect predictions. We measure the percentage of high-confidence errors at three confi￾dence thresholds (> 0.5, > 0.7, > 0.9). Confidence is calculated as described in Section 3.2.3. CoMT+CCRL consistently reduces overconfidence across all models and thresholds, demonstrating that CCRL successfully teaches models appropriate confidence calibration. overconfidence rates at thre… view at source ↗
Figure 5
Figure 5. Figure 5: compares our CoMT+CCRL trained Qwen2.5-7B model against larger base models (14B and 32B) across six benchmarks. Our enhanced 7B model achieves substantial improvements over its base version, with gains ranging from 4.6% to 15.4% across different benchmarks, and remark￾ably matches or exceeds the performance of models 2-4.5× larger. Notably, on four key benchmarks—AsDiv, SVAMP, MAWPS, and TabMWP—our CoMT+CC… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

46 extracted references · 21 linked inside Pith

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Large language models for mathematical reasoning: Progresses and challenges

    Ahn, J., Verma, R., Lou, R., Liu, D., Zhang, R., and Yin, W. Large language models for mathematical reasoning: Progresses and challenges. arXiv preprint arXiv:2402.00157, 2024

    Pith/arXiv arXiv 2024

  3. [3]

    T., Feltovich, P

    Chi, M. T., Feltovich, P. J., and Glaser, R. Categorization and representation of physics problems by experts and novices. Cognitive science, 5 0 (2): 0 121--152, 1981

    1981

  4. [4]

    V., Levine, S., and Ma, Y

    Chu, T., Zhai, Y., Yang, J., Tong, S., Xie, S., Schuurmans, D., Le, Q. V., Levine, S., and Ma, Y. Sft memorizes, rl generalizes: A comparative study of foundation model post-training. arXiv preprint arXiv:2501.17161, 2025

    Pith/arXiv arXiv 2025

  5. [5]

    Training verifiers to solve math word problems

    Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021

    Pith/arXiv arXiv 2021

  6. [6]

    Supervised reinforcement learning: From expert trajectories to step-wise reasoning

    Deng, Y., Hsu, I., Yan, J., Wang, Z., Han, R., Zhang, G., Chen, Y., Wang, W., Pfister, T., Lee, C.-Y., et al. Supervised reinforcement learning: From expert trajectories to step-wise reasoning. arXiv preprint arXiv:2510.25992, 2025

    arXiv 2025

  7. [7]

    Theory-based causal transfer: Integrating instance-level induction and abstract-level structure learning

    Edmonds, M., Ma, X., Qi, S., Zhu, Y., Lu, H., and Zhu, S.-C. Theory-based causal transfer: Integrating instance-level induction and abstract-level structure learning. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pp.\ 1283--1291, 2020

    2020

  8. [8]

    Pal: Program-aided language models

    Gao, L., Madaan, A., Zhou, S., Alon, U., Liu, P., Yang, Y., Callan, J., and Neubig, G. Pal: Program-aided language models. arXiv preprint arXiv:2211.10435, 2022

    Pith/arXiv arXiv 2022

  9. [9]

    Structure-mapping: A theoretical framework for analogy

    Gentner, D. Structure-mapping: A theoretical framework for analogy. Cognitive science, 7 0 (2): 0 155--170, 1983

    1983

  10. [10]

    Gick, M. L. and Holyoak, K. J. Schema induction and analogical transfer. Cognitive psychology, 15 0 (1): 0 1--38, 1983

    1983

  11. [11]

    The llama 3 herd of models

    Grattafiori, A., Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Vaughan, A., et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

    Pith/arXiv arXiv 2024

  12. [12]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Guo, D., Yang, D., Zhang, H., Song, J., Zhang, R., Xu, R., Zhu, Q., Ma, S., Wang, P., Bi, X., et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    Pith/arXiv arXiv 2025

  13. [13]

    Holyoak, K. J. and Thagard, P. Analogical mapping by constraint satisfaction. Cognitive science, 13 0 (3): 0 295--355, 1989

    1989

  14. [14]

    Compositional generalization through abstract representations in human and artificial neural networks

    Ito, T., Klinger, T., Schultz, D., Murray, J., Cole, M., and Rigotti, M. Compositional generalization through abstract representations in human and artificial neural networks. Advances in neural information processing systems, 35: 0 32225--32239, 2022

    2022

  15. [15]

    Kingma, D. P. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014

    Pith/arXiv arXiv 2014

  16. [16]

    Mawps: A math word problem repository

    Koncel-Kedziorski, R., Roy, S., Amini, A., Kushman, N., and Hajishirzi, H. Mawps: A math word problem repository. In Proceedings of the 2016 conference of the north american chapter of the association for computational linguistics: human language technologies, pp.\ 1152--1157, 2016

    2016

  17. [17]

    M., Cholakkal, H., Shah, M., Yang, M.-H., Torr, P

    Kumar, K., Ashraf, T., Thawakar, O., Anwer, R. M., Cholakkal, H., Shah, M., Yang, M.-H., Torr, P. H., Khan, F. S., and Khan, S. Llm post-training: A deep dive into reasoning large language models. arXiv preprint arXiv:2502.21321, 2025

    Pith/arXiv arXiv 2025

  18. [18]

    D., Cohen, J

    Kumar, S., Dasgupta, I., Daw, N. D., Cohen, J. D., and Griffiths, T. L. Disentangling abstraction from statistical pattern matching in human and machine learning. PLoS computational biology, 19 0 (8): 0 e1011316, 2023

    2023

  19. [19]

    Let's verify step by step

    Lightman, H., Kosaraju, V., Burda, Y., Edwards, H., Baker, B., Lee, T., Leike, J., Schulman, J., Sutskever, I., and Cobbe, K. Let's verify step by step. In The Twelfth International Conference on Learning Representations, 2023

    2023

  20. [20]

    N., Zhu, S.-C., Rajpurohit, T., Clark, P., and Kalyan, A

    Lu, P., Qiu, L., Chang, K.-W., Wu, Y. N., Zhu, S.-C., Rajpurohit, T., Clark, P., and Kalyan, A. Dynamic prompt learning via policy gradient for semi-structured mathematical reasoning. arXiv preprint arXiv:2209.14610, 2022

    Pith/arXiv arXiv 2022

  21. [21]

    Towards a unified view of large language model post-training

    Lv, X., Zuo, Y., Sun, Y., Liu, H., Wei, Y., Chen, Z., He, L., Zhu, X., Zhang, K., Wang, B., et al. Towards a unified view of large language model post-training. arXiv preprint arXiv:2509.04419, 2025

    arXiv 2025

  22. [22]

    W., and Behrens, T

    Mark, S., Moran, R., Parr, T., Kennerley, S. W., and Behrens, T. E. Transferring structural knowledge across cognitive maps in humans and models. Nature communications, 11 0 (1): 0 4783, 2020

    2020

  23. [23]

    A diverse corpus for evaluating and developing english math word problem solvers

    Miao, S.-Y., Liang, C.-C., and Su, K.-Y. A diverse corpus for evaluating and developing english math word problem solvers. In Proceedings of the 58th annual meeting of the Association for Computational Linguistics, pp.\ 975--984, 2020

    2020

  24. [24]

    Gsm-symbolic: Understanding the limitations of mathematical reasoning in large language models

    Mirzadeh, I., Alizadeh, K., Shahrokhi, H., Tuzel, O., Bengio, S., and Farajtabar, M. Gsm-symbolic: Understanding the limitations of mathematical reasoning in large language models. arXiv preprint arXiv:2410.05229, 2024

    Pith/arXiv arXiv 2024

  25. [25]

    Abstraction and analogy-making in artificial intelligence

    Mitchell, M. Abstraction and analogy-making in artificial intelligence. Annals of the New York Academy of Sciences, 1505 0 (1): 0 79--101, 2021

    2021

  26. [26]

    Training language models to follow instructions with human feedback

    Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35: 0 27730--27744, 2022

    2022

  27. [27]

    Plan-tuning: Post-training language models to learn step-by-step planning for complex problem solving

    Parmar, M., Goyal, P., Liu, X., Song, Y., Ling, M., Baral, C., Palangi, H., and Pfister, T. Plan-tuning: Post-training language models to learn step-by-step planning for complex problem solving. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pp.\ 21430--21444, 2025

    2025

  28. [28]

    Are nlp models really able to solve simple math word problems? arXiv preprint arXiv:2103.07191, 2021

    Patel, A., Bhattamishra, S., and Goyal, N. Are nlp models really able to solve simple math word problems? arXiv preprint arXiv:2103.07191, 2021

    Pith/arXiv arXiv 2021

  29. [29]

    High-dimensional continuous control using generalized advantage estimation

    Schulman, J., Moritz, P., Levine, S., Jordan, M., and Abbeel, P. High-dimensional continuous control using generalized advantage estimation. arXiv preprint arXiv:1506.02438, 2015

    Pith/arXiv arXiv 2015

  30. [30]

    Proximal policy optimization algorithms

    Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    Pith/arXiv arXiv 2017

  31. [31]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models

    Shao, Z., Wang, P., Zhu, Q., Xu, R., Song, J., Bi, X., Zhang, H., Zhang, M., Li, Y., Wu, Y., et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024

    Pith/arXiv arXiv 2024

  32. [32]

    Singley, M. K. and Anderson, J. R. The transfer of cognitive skill. Number 9. Harvard University Press, 1989

    1989

  33. [33]

    Cognitive load during problem solving: Effects on learning

    Sweller, J. Cognitive load during problem solving: Effects on learning. Cognitive science, 12 0 (2): 0 257--285, 1988

    1988

  34. [34]

    Team, Q. et al. Qwen2 technical report. arXiv preprint arXiv:2407.10671, 2 0 (3), 2024

    Pith/arXiv arXiv 2024

  35. [35]

    Solving math word problems with process-and outcome-based feedback

    Uesato, J., Kushman, N., Kumar, R., Song, F., Siegel, N., Wang, L., Creswell, A., Irving, G., and Higgins, I. Solving math word problems with process-and outcome-based feedback. arXiv preprint arXiv:2211.14275, 2022

    Pith/arXiv arXiv 2022

  36. [36]

    M., Lin, H.-c., and Ga s i \'c , M

    van Niekerk, C., Vukovic, R., Ruppik, B. M., Lin, H.-c., and Ga s i \'c , M. Post-training large language models via reinforcement learning from self-feedback. arXiv preprint arXiv:2507.21931, 2025

    Pith/arXiv arXiv 2025

  37. [37]

    X., Kurth-Nelson, Z., Kumaran, D., Tirumala, D., Soyer, H., Leibo, J

    Wang, J. X., Kurth-Nelson, Z., Kumaran, D., Tirumala, D., Soyer, H., Leibo, J. Z., Hassabis, D., and Botvinick, M. Prefrontal cortex as a meta-reinforcement learning system. Nature neuroscience, 21 0 (6): 0 860--868, 2018

    2018

  38. [38]

    A survey on large language models for mathematical reasoning

    Wang, P.-Y., Liu, T.-S., Wang, C., Wang, Y.-D., Yan, S., Jia, C.-X., Liu, X.-H., Chen, X.-W., Xu, J.-C., Li, Z., et al. A survey on large language models for mathematical reasoning. arXiv preprint arXiv:2506.08446, 2025

    Pith/arXiv arXiv 2025

  39. [39]

    Uft: Unifying fine-tuning of sft and rlhf/dpo/una through a generalized implicit reward function

    Wang, Z., Bi, B., Zhu, Z., Mao, X., Wang, J., and Wang, S. Uft: Unifying fine-tuning of sft and rlhf/dpo/una through a generalized implicit reward function. arXiv preprint arXiv:2410.21438, 2024

    Pith/arXiv arXiv 2024

  40. [40]

    V., Zhou, D., et al

    Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q. V., Zhou, D., et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35: 0 24824--24837, 2022

    2022

  41. [41]

    M., Meder, B., and Schulz, E

    Wu, C. M., Meder, B., and Schulz, E. Unifying principles of generalization: past, present, and future. Annual Review of Psychology, 76 0 (1): 0 275--302, 2025

    2025

  42. [42]

    Qwen2.5-math technical report: Toward mathematical expert model via self-improvement

    Yang, A., Zhang, B., Hui, B., Gao, B., Yu, B., Li, C., Liu, D., Tu, J., Zhou, J., Lin, J., Lu, K., Xue, M., Lin, R., Liu, T., Ren, X., and Zhang, Z. Qwen2.5-math technical report: Toward mathematical expert model via self-improvement. arXiv preprint arXiv:2409.12122, 2024

    Pith/arXiv arXiv 2024

  43. [43]

    Qwen3 technical report

    Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., Lv, C., et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025

    Pith/arXiv arXiv 2025

  44. [44]

    Star: Bootstrapping reasoning with reasoning

    Zelikman, E., Wu, Y., Mu, J., and Goodman, N. Star: Bootstrapping reasoning with reasoning. Advances in Neural Information Processing Systems, 35: 0 15476--15488, 2022

    2022

  45. [45]

    Groundedprm: Tree-guided and fidelity-aware process reward modeling for step-level reasoning

    Zhang, Y., Wu, Y., Zhang, H., Li, W., Chen, H., Wu, J., Li, G., Han, Z., and Tresp, V. Groundedprm: Tree-guided and fidelity-aware process reward modeling for step-level reasoning. arXiv preprint arXiv:2510.14942, 2025

    arXiv 2025

  46. [46]

    Y., Garvert, M

    Zheng, X. Y., Garvert, M. M., den Ouden, H. E., Horstman, L. I., Richter, D., and Cools, R. Abstract rule generalization for composing novel meaning recruits a frontoparietal control network. Imaging Neuroscience, 3: 0 IMAG--a, 2025

    2025