Pith. sign in

REVIEW 5 major objections 5 minor 50 references

SIGMA: Refining Large Language Model Reasoning via Sibling-Guided Monte Carlo Augmentation

T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read By turning discarded sibling branches of a Monte Carlo Tree Search into natural-language critique that revises the chosen reasoning path, the paper shows a 7B model trained on 30K samples reaches 54.9% on MATH, beating models trained on…

desk verdict A genuinely new idea for reusing MCTS siblings as symbolic feedback, but the 30K-vs-590K headline conflates data efficiency with teacher strength and needs missing controls before it lands. read the letter →

arxiv 2506.06470 v1 pith:EIRKVFCW submitted 2025-06-06 cs.AI cs.LG

Yanwei Ren , Haotian Zhang , Fuxiang Wu , Jiayan Qiu , Jiaxing Huang , Baosheng Yu , Liu Liu This is my paper
classification cs.AIcs.LG
keywords MonteCarloTreeSearchsiblingguidancetextualgradientdescentmathematicalreasoningdataefficiencychain-of-thoughtsynthesiscritiqueandrevisionsynthetictraining
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that the branches a Monte Carlo Tree Search rejects when picking a best reasoning path are not wasted computation but a source of free, locally attributable supervision. It introduces SIGMA, which at each step of the chosen path compares the selected node with its sibling nodes under the same parent, turns that comparison into a natural-language critique, and uses the critique to revise the step. Fine-tuning a 7B model on the resulting 15K-30K refined chains beats or matches models trained on 60K-590K samples on the MATH benchmark and five out-of-domain math suites. The point matters because it suggests data quality can be increased by re-using the internal structure of a single search tree, rather than scaling up generation or curation.

What carries the argument

The central object is the sibling-guidance pipeline: the sibling set $\mathcal{S}(p)$ of a chosen node $p$ (all children of the same parent except $p$), a critique LLM that acts as a symbolic gradient oracle producing $G$, and a revision LLM that applies one textual-gradient-descent update $\tilde{T}_p \leftarrow R_{\text{LLM}}(T_p, G)$ per step, sweeping depths $d=1,\dots,D$ — one pass of coordinate descent over the chain-of-thought. Its work is to convert the search tree's discarded branches into localized, step-level revision signals, so the refined chain preserves the MCTS-selected structure while repairing local weaknesses.

What would settle it

Sample a few hundred SIGMA-refined training chains, compare each revised step against the original step and the ground-truth answer, and count how often the revision introduces a false statement or changes the final answer; if a material fraction of chains are corrupted and removing them fails to restore the reported gains, the framework's data-quality claim is falsified.

Watch

Extended reading notes

Core claim

The central discovery is that sibling nodes under a shared parent carry contrastive signal that can be attributed to step-level content, because all candidates share the same partial reasoning context. SIGMA formalizes this as a symbolic loss $\mathcal{L}_{\text{text}} = \Phi\big(T_p, \{T_s \mid s \in \mathcal{S}(p)\}\big)$, a natural-language comparison between the chosen step and its siblings; a critique language model converts this into a gradient-like message $G = \mathcal{C}_{\text{LLM}}(T_p, \{T_s\})$; and a revision model performs a single textual-gradient-descent step per depth, sweeping coordinate-wise over the chain to produce a refined path $T^\dagger$. The paper reports that this refinement converts ordinary MCTS paths into training data strong enough that 30K SIGMA examples reach 54.9% on MATH, beating 590K-sample models, and that 15K examples already beat 30K-scale baselines across three 7B base models, with gains also on out-of-domain benchmarks.

Load-bearing premise

The load-bearing premise is that a general-purpose language model, given sibling comparisons, reliably rewrites each step to be better without corrupting it, and that no verification of the revised step or final answer is needed.

Editorial extensions

If this is right

  • Fine-tuning on 15K SIGMA-refined chains beats every 30K-scale baseline on all three base models, and the 30K dataset beats or matches 60K-scale methods, so data efficiency is improved by roughly 2-4x.
  • The refinement step requires no additional rollouts, reward models, or ground-truth labels beyond the MCTS tree and one critique/revision model pair, so it can be bolted onto existing data-generation pipelines.
  • Out-of-domain gains on advanced math suites (college, Olympiad, theorem-proving, and curriculum-style problems) indicate the refined chains transfer beyond the training distribution rather than overfitting the benchmark.
  • Scaling SIGMA from 30K to 60K samples continues to improve all three backbones, with the strongest math-specialized variant reaching 56.5% accuracy on MATH — the highest among every model compared in the paper.
  • The gap between 15K and 30K refined samples is much smaller than the gap between SIGMA-15K and 30K-scale baselines, which suggests the first 15K refined chains carry most of the benefit.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The sibling-comparison mechanism is not specific to mathematics: any search tree with alternative continuations (code repair, proof search, planning) could in principle supply the same contrastive signal, though the paper does not test these domains.
  • The 15K-to-30K result hints that the marginal value of each additional refined sample falls quickly; a scaling curve would tell whether 60K is near the useful ceiling for this form of supervision.
  • The critiques identify recurrent error patterns in rejected branches; an untested extension would convert those criticisms into explicit negative examples or preference pairs for direct preference optimization.
  • Because the critique and revision engines are a single strong external model, an open question is whether the data-quality gain survives when the student model itself must generate the feedback — i.e., whether the loop can be closed without external supervision.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. SIGMA is a data-synthesis method for mathematical reasoning. The pipeline first uses an MCTS search with Qwen2.5-Math-7B as the generator, then, instead of discarding the non-selected branches, it keeps up to two sibling nodes at each depth. GPT-4o-mini is used as a critique model to compare the selected step against its siblings and produce natural-language 'textual gradients', and then as a revision model to rewrite each step in the selected path. The refined chains form SIGMA-15K and SIGMA-30K training sets, which are used to fully fine-tune DeepSeekMath-7B, LLaMA3-8B, and Mistral-7B. The paper reports that DeepSeekMath-SIGMA-7B-30K reaches 54.9% on MATH with 30K samples, outperforming 590K-sample baselines on that benchmark, and that SIGMA-15K outperforms all 30K-scale baselines on the six-benchmark average. Ablations compare SIGMA with unrefined MCTS paths and with GPT-4o-mini black-box CoT.

Significance. The core idea is attractive and potentially useful: sibling nodes that are normally discarded during MCTS are converted into contrastive supervision, the method requires no additional rollouts or reward models, and it is model-agnostic. The paper also has clear strengths: it adopts the public DART-Math evaluation protocol, includes a detailed appendix with concrete before/after examples, and at least attempts a black-box teacher control. If the mechanism were properly isolated, the data-efficiency result would be a meaningful contribution to math data synthesis. However, as it stands, the evidence does not separate sibling-guided refinement from strong-teacher rewriting, and the headline claim rests on single-run, partially quoted comparisons with small margins.

major comments (5)
  1. [§4.3 / Abstract] The headline 30K-vs-590K comparison is confounded with teacher strength. Section 4.1 states that GPT-4o-mini-2024-07-18 is used as both the critique model and the revision model, so every step in SIGMA-15K/30K has been rewritten by a frontier teacher, whereas the 590K baselines such as RFT and DART-Math are generated by 7B models. The reported gains therefore do not establish that sibling-guided refinement, as opposed to strong-teacher rewriting, causes the data-efficiency improvement. A control that applies the same GPT-4o-mini critique-plus-revision pipeline to the same MCTS paths with sibling texts removed, or to ordinary non-search step sequences, is needed to support the mechanism claim in the title and abstract.
  2. [§4.4 / Table 2] The black-box CoT ablation, which is the main control for generator capability, is under-powered and under-specified. The average gains of SIGMA-15K over BLACKBOX-15K are only 1.4, 2.6, and 1.6 points for DeepSeekMath, LLaMA3, and Mistral, respectively, and several individual benchmarks favor the black-box baseline (DeepSeekMath College -1.2; LLaMA3 Olympiad -2.1; Mistral MATH -1.2 and Olympiad -3.4). All numbers appear to be single-run point estimates, and the 'standard prompt' used for the black-box CoT baseline is not given. The paper should report means and standard deviations over multiple seeds and provide the exact prompt and decoding settings.
  3. [§3.3 / Eq. (5)] The refinement step applies a single unverified textual-gradient update to every step of the selected path, and the refined chain is used directly as supervision. There is no check that the revised step is valid, that the final answer is unchanged, or that the rewrite preserves correctness. Since a frontier-model rewrite can corrupt an otherwise correct MCTS path, the paper should report the per-path answer-preservation and correctness rates before and after refinement, or add a verification step, in order to rule out contamination of the SIGMA training data.
  4. [§4.4] No ablation removes sibling texts from the critique prompt. The two reported controls, unrefined MCTS paths and black-box GPT-4o-mini CoT, cannot distinguish between the effect of sibling comparisons and the effect of any GPT-4o-mini-mediated rewriting of weak 7B paths. An additional condition that critiques each step from the selected step alone, using the same revision model and the same prompts apart from the sibling list, is required to attribute the gains to sibling guidance.
  5. [§1 / §4.3 / Table 1] The contribution statement that 'SIGMA-30K remains better or competitive with 60K-scale methods' is not supported for all backbones. In Table 1, Llama3-SIGMA-8B-30K averages 37.7 versus 39.0 for MathFusion-Llama3-8B-60K, and Mistral-SIGMA-7B-30K averages 34.9 versus 36.1 for MathFusion-Mistral-7B-60K. In addition, DeepSeekMath-SIGMA-7B-30K (48.2) is below DeepSeekMath-7B-DART-Math-590K (49.4) on the six-benchmark average, so the 30K-vs-590K claim holds on MATH (54.9 vs. 53.6) but not on the aggregate reported in Table 1. These claims should be qualified to the specific benchmarks and baselines where they hold.
minor comments (5)
  1. [§1, §3.2, §3.1] There are several typos that should be corrected: 'larage language models' in Section 1, 'that operats over natural language' in Section 3.2, and 'N c denotes the counts visits' in Section 3.1.
  2. [Tables 1 and 3] Tables 1 and 3 duplicate the same rows, but the identical DeepSeekMath-SIGMA-7B-15K result is listed as 47.0 average in Table 1 and 46.9 average in Table 2; the rounding inconsistency should be fixed.
  3. [Table captions] The arrow notation in Tables 1-3 is not explicitly defined in the captions; the text suggests the arrows are relative to MathFusion-30K, but this should be stated directly.
  4. [§4.4] The MCTS-15K baseline should specify whether the extracted best path is required to end in a verifiably correct answer or whether it is the highest Q-value path regardless of correctness; this affects the interpretation of the large MCTS-to-SIGMA gap.
  5. [§4.1 / §3.2] The critique and revision prompts are central to the method, but they are not included in the paper or appendix; for reproducibility, the exact prompt templates and, ideally, the code should be released.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: SIGMA's gains are empirical measurements against fixed external benchmarks with explicit controls; no claim reduces to its own input.

full rationale

I find no significant circularity. The central claim—that sibling-guided critique and revision improves MCTS-derived chain-of-thought data—is tested empirically. Table 2 compares SIGMA-15K against vanilla MCTS-15K (the same trees without refinement) and against black-box GPT-4o-mini CoT (the same generator model and query set, without sibling comparisons), and SIGMA wins on average across all three backbones. The headline MATH figure (54.92% at 30K samples) is produced by fine-tuning a 7B model on SIGMA-30K and evaluating under the fixed DART-Math protocol on the MATH test set; no parameter is fitted to that number, and the comparison against 590K baselines uses externally published results. Table 1 also reports four out-of-domain benchmarks (CollegeMath, DeepMind Mathematics, OlympiadBench-Math, TheoremQA) that were not used in training, providing an independent generalization check. Equations (3)-(5) define the symbolic loss, critique oracle, and revision operator; they are construction steps, not reductions of the measured outcome to an input. No fitted input is renamed as a prediction, no uniqueness theorem is imported from prior work, and the only self-citation (Mulberry in related work) is background, not load-bearing. The remaining concerns—that GPT-4o-mini's teacher strength is not fully ablated from sibling guidance and that MATH/GSM8K are used for both training and in-domain evaluation—are experimental confounds or standard-practice caveats, not circularity. The paper's own limitations section is similarly about capability and scope, not about a definitional dependency. I therefore assign a score of 0.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claim rests mainly on empirical pipeline choices rather than on equations. The most significant unvalidated assumptions are the reliability of the critique/revision oracle and the absence of answer verification after refinement.

free parameters (6)
  • UCT exploration constant c_p = not reported
    Controls the exploration-exploitation trade-off in Eq. (2); the paper does not state its value or sensitivity.
  • max tree depth D = 16
    Chosen for the MCTS search; affects the granularity of step-level refinement.
  • candidate completions n and samples k per node = n=3, k=5
    Determines how many sibling nodes exist; the method keeps up to two siblings.
  • decoding temperatures for MCTS datasets = 0.4 and 0.7
    Two MCTS datasets are generated at these temperatures and merged into the 30K set.
  • number of retained siblings = up to 2
    The critique prompt includes the selected node and up to two siblings; the choice is not ablated.
  • learning rate per base model = 5e-5 (DeepSeekMath), 4e-6 (Mistral), 1e-5 (LLaMA3)
    Tuned per model in Appendix B; standard fine-tuning hyperparameters.
assumptions (5)
  • standard math UCT selection and value update equations (Eq. 1-2) are correct for the MCTS procedure.
    Standard MCTS; the paper relies on convergence to a high-value path.
  • domain assumption Sibling nodes sharing a parent provide a locally attributable contrastive signal.
    The critique model compares steps under the same parent, assuming differences in content are due to step-level choices rather than upstream variation.
  • domain assumption Natural-language critiques from GPT-4o-mini act as reliable 'symbolic gradients' for improving reasoning steps.
    The revision model is assumed to produce improved text; no quantitative validation of critique quality is provided.
  • domain assumption Refined traces do not need answer verification before being used as training targets.
    The pipeline lacks a correctness filter; any errors introduced by the revision model become supervision signal.
  • ad hoc to paper GPT-4o-mini serves as both critique and revision oracle and is strong enough for the task.
    The method's benefits depend on this external model; the authors note this as a limitation in Section 5.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SIGMA: Refining Large Language Model Reasoning via Sibling-Guided Monte Carlo Augmentation." pith.science (2026). https://pith.science/paper/EIRKVFCW

@misc{pith2026250606470,
  author       = {Pith},
  title        = {Pith review of: SIGMA: Refining Large Language Model Reasoning via Sibling-Guided Monte Carlo Augmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EIRKVFCW}},
  note         = {Machine review of arXiv:2506.06470}
}
read the original abstract

Enhancing large language models by simply scaling up datasets has begun to yield diminishing returns, shifting the spotlight to data quality. Monte Carlo Tree Search (MCTS) has emerged as a powerful technique for generating high-quality chain-of-thought data, yet conventional approaches typically retain only the top-scoring trajectory from the search tree, discarding sibling nodes that often contain valuable partial insights, recurrent error patterns, and alternative reasoning strategies. This unconditional rejection of non-optimal reasoning branches may waste vast amounts of informative data in the whole search tree. We propose SIGMA (Sibling Guided Monte Carlo Augmentation), a novel framework that reintegrates these discarded sibling nodes to refine LLM reasoning. SIGMA forges semantic links among sibling nodes along each search path and applies a two-stage refinement: a critique model identifies overlooked strengths and weaknesses across the sibling set, and a revision model conducts text-based backpropagation to refine the top-scoring trajectory in light of this comparative feedback. By recovering and amplifying the underutilized but valuable signals from non-optimal reasoning branches, SIGMA substantially improves reasoning trajectories. On the challenging MATH benchmark, our SIGMA-tuned 7B model achieves 54.92% accuracy using only 30K samples, outperforming state-of-the-art models trained on 590K samples. This result highlights that our sibling-guided optimization not only significantly reduces data usage but also significantly boosts LLM reasoning.

Figures

Figures reproduced from arXiv: 2506.06470 by the authors.

Figure 1
Figure 1. An illustration of the reasoning performance of fully fine-tuned DeepSeekMath-7B models [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The main SIGMA framework for enhanced CoT data generation. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Visualization of the gradient computation process involving sibling nodes and the original [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: A textual example of the proposed SIGMA framework including three aspects: Original [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Performance comparison of models fine-tuned on [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]
Figure 6
Figure 6. Figure 6: Performance comparison of models fine-tuned on [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: Performance comparison of models fine-tuned on [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

50 extracted references · 26 canonical work pages

  1. [1]

    Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  2. [2]

    Self-RAG: Learning to re- trieve, generate, and critique through self-reflection

    Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. Self-RAG: Learning to re- trieve, generate, and critique through self-reflection. InThe Twelfth International Conference on Learning Representations, 2024

    work page 2024

  3. [3]

    Graph of thoughts: Solving elaborate problems with large language models

    Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Michal Podstawski, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Hubert Niewiadomski, Piotr Nyczyk, et al. Graph of thoughts: Solving elaborate problems with large language models. InProceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 17682–17690, 2024

    2024

  4. [4]

    Byrd, Robert Zinkov, and Nada Amin

    David Brandfonbrener, Simon Henniger, Sibi Raja, Tarun Prasad, Chloe R Loughridge, Federico Cas- sano, Sabrina Ruixin Hu, Jianang Yang, William E. Byrd, Robert Zinkov, and Nada Amin. VerMCTS: Synthesizing multi-step programs using a verifier, a large language model, and tree search. InThe 4th Workshop on Mathematical Reasoning and AI at NeurIPS’24, 2024

    work page 2024

  5. [5]

    Large language monkeys: Scaling inference compute with repeated sampling.arXiv preprint arXiv:2407.21787, 2024

    Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoc V Le, Christopher R ´e, and Azalia Mirhoseini. Large language monkeys: Scaling inference compute with repeated sampling.arXiv preprint arXiv:2407.21787, 2024

    arXiv 2024

  6. [6]

    Alphamath almost zero: Process supervision without process

    Guoxin Chen, Minpeng Liao, Chengxi Li, and Kai Fan. Alphamath almost zero: Process supervision without process. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    work page 2024

  7. [7]

    Divide-and-conquer meets consensus: Unleashing the power of functions in code generation

    Jingchang Chen, Hongxuan Tang, Zheng Chu, Qianglong Chen, Zekun Wang, Ming Liu, and Bing Qin. Divide-and-conquer meets consensus: Unleashing the power of functions in code generation. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    work page 2024

  8. [8]

    TheoremQA: A theorem-driven question answering dataset

    Wenhu Chen, Ming Yin, Max Ku, Pan Lu, Yixin Wan, Xueguang Ma, Jianyu Xu, Xinyi Wang, and Tony Xia. TheoremQA: A theorem-driven question answering dataset. InThe 2023 Conference on Empirical Methods in Natural Language Processing, 2023

    work page 2023

Show all 50 references
  1. [10]

    Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021

    arXiv

  2. [11]

    ToRA: A tool-integrated reasoning agent for mathematical problem solving

    Zhibin Gou, Zhihong Shao, Yeyun Gong, yelong shen, Yujiu Yang, Minlie Huang, Nan Duan, and Weizhu Chen. ToRA: A tool-integrated reasoning agent for mathematical problem solving. InThe Twelfth Inter- national Conference on Learning Representations, 2024

    2024

  3. [12]

    The llama 3 herd of models

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al- Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  4. [13]

    Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems

    Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Leng Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, Jie Liu, Lei Qi, Zhiyuan Liu, and Maosong Sun. Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scienti...

    2024

  5. [14]

    Measuring mathematical problem solving with the MATH dataset

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the MATH dataset. InThirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (...

    2021

  6. [15]

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, L ´elio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thoma...

    2023

  7. [16]

    Scaling laws for neural language models.arXiv preprint arXiv:2001.08361, 2020

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models.arXiv preprint arXiv:2001.08361, 2020

    2001 arXiv

  8. [17]

    Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, and Christopher Potts

    Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Sri Vard- hamanan A, Saiful Haq, Ashutosh Sharma, Thomas T. Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, and Christopher Potts. DSPy: Compiling declarative language model calls into state-...

    2024

  9. [18]

    Adam: A method for stochastic optimization.arXiv preprint arXiv:1412.6980, 2014

    Diederik P Kingma. Adam: A method for stochastic optimization.arXiv preprint arXiv:1412.6980, 2014

    2014 arXiv

  10. [19]

    Let’s verify step by step

    Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. InThe Twelfth International Conference on Learning Representations, 2024

    2024

  11. [20]

    Program induction by rationale generation: Learning to solve and explain algebraic word problems.arXiv preprint arXiv:1705.04146, 2017

    Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blunsom. Program induction by rationale generation: Learning to solve and explain algebraic word problems.arXiv preprint arXiv:1705.04146, 2017

    2017 arXiv

  12. [21]

    De- ductive verification of chain-of-thought reasoning

    Zhan Ling, Yunhao Fang, Xuanlin Li, Zhiao Huang, Mingu Lee, Roland Memisevic, and Hao Su. De- ductive verification of chain-of-thought reasoning. InThirty-seventh Conference on Neural Information Processing Systems, 2023

    2023

  13. [22]

    Augmenting math word problems via iter- ative question composing

    Haoxiong Liu, Yifan Zhang, Yifan Luo, and Andrew C Yao. Augmenting math word problems via iter- ative question composing. InProceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 24605–24613, 2025

    2025

  14. [23]

    Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct.arXiv preprint arXiv:2308.09583, 2023

    Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Jianguang Lou, Chongyang Tao, Xiubo Geng, Qingwei Lin, Shifeng Chen, and Dongmei Zhang. Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct.arXiv preprint arXiv:2308.09583, 2023

    2023 arXiv

  15. [24]

    Self-refine: Iterative refine- ment with self-feedback

    Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. Self-refine: Iterativ...

    2023

  16. [25]

    Mixed precision training

    Paulius Micikevicius, Sharan Narang, Jonah Alben, Gregory Diamos, Erich Elsen, David Garcia, Boris Ginsburg, Michael Houston, Oleksii Kuchaiev, Ganesh Venkatesh, et al. Mixed precision training. In International Conference on Learning Representations, 2018

    2018

  17. [26]

    Language model self-improvement by reinforcement learning contemplation

    Jing-Cheng Pang, Pengyuan Wang, Kaiyuan Li, Xiong-Hui Chen, Jiacheng Xu, Zongzhang Zhang, and Yang Yu. Language model self-improvement by reinforcement learning contemplation. InThe Twelfth International Conference on Learning Representations, 2024

    2024

  18. [27]

    Mathfusion: Enhancing mathematic problem-solving of llm through instruction fusion.arXiv preprint arXiv:2503.16212, 2025

    Qizhi Pei, Lijun Wu, Zhuoshi Pan, Yu Li, Honglin Lin, Chenlin Ming, Xin Gao, Conghui He, and Rui Yan. Mathfusion: Enhancing mathematic problem-solving of llm through instruction fusion.arXiv preprint arXiv:2503.16212, 2025

    2025 arXiv

  19. [28]

    Scaling language models: Methods, analysis & insights from training gopher.arXiv preprint arXiv:2112.11446, 2021

    Jack W Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, et al. Scaling language models: Methods, analysis & insights from training gopher.arXiv preprint arXiv:2112.11446, 2021

    2021 arXiv

  20. [29]

    Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters

    Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, and Yuxiong He. Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters. InProceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pages 35...

    2020

  21. [30]

    Analysing mathematical reasoning abilities of neural models

    David Saxton, Edward Grefenstette, Felix Hill, and Pushmeet Kohli. Analysing mathematical reasoning abilities of neural models. InInternational Conference on Learning Representations, 2019

    2019

  22. [31]

    Rewarding progress: Scaling automated process verifiers for LLM reasoning

    Amrith Setlur, Chirag Nagpal, Adam Fisch, Xinyang Geng, Jacob Eisenstein, Rishabh Agarwal, Alekh Agarwal, Jonathan Berant, and Aviral Kumar. Rewarding progress: Scaling automated process verifiers for LLM reasoning. InThe Thirteenth International Conference on Learning Represe...

    2025

  23. [32]

    Deepseekmath: Pushing the limits of mathematical reasoning in open lan- guage models.arXiv preprint arXiv:2402.03300, 2024

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open lan- guage models.arXiv preprint arXiv:2402.03300, 2024

    2024 arXiv

  24. [33]

    Mathscale: Scaling instruction tuning for mathematical reasoning

    Zhengyang Tang, Xingxing Zhang, Benyou Wang, and Furu Wei. Mathscale: Scaling instruction tuning for mathematical reasoning. InForty-first International Conference on Machine Learning, 2024

    2024

  25. [34]

    Deepdistill: Enhancing llm reasoning capabilities via large-scale difficulty-graded data training

    Xiaoyu Tian, Sitong Zhao, Haotian Wang, Shuaiting Chen, Yiping Peng, Yunjie Ji, Han Zhao, and Xian- gang Li. Deepdistill: Enhancing llm reasoning capabilities via large-scale difficulty-graded data training. arXiv preprint arXiv:2504.17565, 2025

    2025 arXiv

  26. [35]

    DART-math: Difficulty-aware rejection tuning for mathematical problem-solving

    Yuxuan Tong, Xiwen Zhang, Rui Wang, Ruidong Wu, and Junxian He. DART-math: Difficulty-aware rejection tuning for mathematical problem-solving. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    2024

  27. [36]

    Openmathinstruct-2: Accelerating AI for math with massive open-source instruction data

    Shubham Toshniwal, Wei Du, Ivan Moshkov, Branislav Kisacanin, Alexan Ayrapetyan, and Igor Git- man. Openmathinstruct-2: Accelerating AI for math with massive open-source instruction data. InThe Thirteenth International Conference on Learning Representations, 2025

    2025

  28. [37]

    Mathcoder: Seamless code integration in llms for enhanced mathemat- ical reasoning

    Ke Wang, Houxing Ren, Aojun Zhou, Zimu Lu, Sichun Luo, Weikang Shi, Renrui Zhang, Linqi Song, Mingjie Zhan, and Hongsheng Li. Mathcoder: Seamless code integration in llms for enhanced mathemat- ical reasoning. InThe Twelfth International Conference on Learning Representations, 2024

    2024

  29. [38]

    Math-shepherd: Verify and reinforce llms step-by-step without human annotations

    Peiyi Wang, Lei Li, Zhihong Shao, Runxin Xu, Damai Dai, Yifei Li, Deli Chen, Yu Wu, and Zhifang Sui. Math-shepherd: Verify and reinforce llms step-by-step without human annotations. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volu...

    2024

  30. [39]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc V Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. InAdvances in Neural Information Processing Systems, volume 35, pages 24824–24837. Curran As...

    2022

  31. [40]

    Monte carlo tree search boosts reasoning via iterative preference learning.arXiv preprint arXiv:2405.00451, 2024

    Yuxi Xie, Anirudh Goyal, Wenyue Zheng, Min-Yen Kan, Timothy P Lillicrap, Kenji Kawaguchi, and Michael Shieh. Monte carlo tree search boosts reasoning via iterative preference learning.arXiv preprint arXiv:2405.00451, 2024

    2024 arXiv

  32. [41]

    Mulberry: Empowering mllm with o1-like reasoning and reflection via collective monte carlo tree search.arXiv preprint arXiv:2412.18319, 2024

    Huanjin Yao, Jiaxing Huang, Wenhao Wu, Jingyi Zhang, Yibo Wang, Shunyu Liu, Yingjie Wang, Yuxin Song, Haocheng Feng, Li Shen, et al. Mulberry: Empowering mllm with o1-like reasoning and reflection via collective monte carlo tree search.arXiv preprint arXiv:2412.18319, 2024

    2024 arXiv

  33. [42]

    Optimizing chain-of-thought reasoners via gradient variance minimization in rejection sampling and rl.arXiv preprint arXiv:2505.02391, 2025

    Jiarui Yao, Yifan Hao, Hanning Zhang, Hanze Dong, Wei Xiong, Nan Jiang, and Tong Zhang. Optimizing chain-of-thought reasoners via gradient variance minimization in rejection sampling and rl.arXiv preprint arXiv:2505.02391, 2025

    2025 arXiv

  34. [43]

    Griffiths, Yuan Cao, and Karthik R Narasimhan

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik R Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. InThirty-seventh Conference on Neural Information Processing Systems, 2023

    2023

  35. [44]

    Metamath: Bootstrap your own mathematical questions for large language models

    Longhui Yu, Weisen Jiang, Han Shi, Jincheng YU, Zhengying Liu, Yu Zhang, James Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu. Metamath: Bootstrap your own mathematical questions for large language models. InThe Twelfth International Conference on Learning Representations, 2024

    2024

  36. [45]

    Scaling relationship on learning mathematical reasoning with large language models.arXiv preprint arXiv:2308.01825, 2023

    Zheng Yuan, Hongyi Yuan, Chengpeng Li, Guanting Dong, Keming Lu, Chuanqi Tan, Chang Zhou, and Jingren Zhou. Scaling relationship on learning mathematical reasoning with large language models.arXiv preprint arXiv:2308.01825, 2023

    2023 arXiv

  37. [46]

    Optimizing generative ai by backpropagating language model feedback.Nature, 639:609– 616, 2025

    Mert Yuksekgonul, Federico Bianchi, Joseph Boen, Sheng Liu, Pan Lu, Zhi Huang, Carlos Guestrin, and James Zou. Optimizing generative ai by backpropagating language model feedback.Nature, 639:609– 616, 2025

    2025

  38. [47]

    Rest-mcts*: Llm self-training via process reward guided tree search.Advances in Neural Information Processing Systems, 37:64735–64772, 2024

    Dan Zhang, Sining Zhoubian, Ziniu Hu, Yisong Yue, Yuxiao Dong, and Jie Tang. Rest-mcts*: Llm self-training via process reward guided tree search.Advances in Neural Information Processing Systems, 37:64735–64772, 2024

    2024

  39. [48]

    Generative verifiers: Reward modeling as next-token prediction

    Lunjun Zhang, Arian Hosseini, Hritik Bansal, Mehran Kazemi, Aviral Kumar, and Rishabh Agarwal. Generative verifiers: Reward modeling as next-token prediction. InThe Thirteenth International Confer- ence on Learning Representations, 2025

    2025

  40. [49]

    Chain of preference optimization: Improving chain-of-thought reasoning in LLMs

    Xuan Zhang, Chao Du, Tianyu Pang, Qian Liu, Wei Gao, and Min Lin. Chain of preference optimization: Improving chain-of-thought reasoning in LLMs. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. 12 A Extended Results of Our Method on the 60...

    2024

  41. [50]

    The computation sequence token length was fixed at 4096 to capture long range mathematical rea- soning

    optimizations, operating in mixed precision [25] (FP16) to maximize memory efficiency and throughput. The computation sequence token length was fixed at 4096 to capture long range mathematical rea- soning. We setper device train batch size=8and usedgradient accumulation steps=...

  42. [51]

    Step 2:Use the given altitude length to solve forx

    Thus, the legs arexandx √ 3, and the hypotenuse is2x. Step 2:Use the given altitude length to solve forx. We are told the altitude to the hypotenuse is 3 units, so: 3 = x √ 3 2 =⇒x= 3×2√ 3 = 6√ 3 . Rationalizing the denominator: x= 6√ 3 × √ 3√ 3 = 6 √ 3 3 = 2 √ 3. Step 3:Calcu...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.