Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

GHPO: Adaptive Guidance for Stable and Efficient LLM Reinforcement Learning

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Hint-guided reinforcement learning lifts math reasoning by roughly 5 percent.

desk verdict A practical training trick for RLVR reward sparsity, but the headline gains are compromised by train/eval overlap and an inconsistent objective. read the letter →

arxiv 2507.10628 v2 pith:BWCXFO5G submitted 2025-07-14 cs.LG cs.AI

Ziru Liu , Cheng Gong , Xinyu Fu , Yaofang Liu , Ran Chen , Shoubo Hu , Suiyun Zhang , Rui Liu
show 2 more authors
Qingfu Zhang Dandan Tu
This is my paper · ORCID
classification cs.LGcs.AI
keywords GHPOreinforcementlearningwithverifiablerewardsGRPOrewardsparsityadaptivepromptrefinementground-truthhintscurriculummathematicalreasoning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that reinforcement learning with verifiable rewards on math problems stalls when the training problems outpace the model: sampled answer groups come back all wrong, rewards are all zero, and the policy receives no gradient. It proposes Guided Hybrid Policy Optimization (GHPO), which detects those failed groups and appends a prefix of the ground-truth solution to the prompt as a hint. Harder queries receive longer hint prefixes as training progresses, while queries the model can already solve keep ordinary on-policy reinforcement learning. The paper reports that this adaptive schedule improves average accuracy by about 5% across six mathematics benchmarks compared with GRPO and curriculum-learning baselines, and that training becomes more stable, with smaller gradient norms.

What carries the argument

The core mechanism is a two-module training loop. A difficulty detector samples $G$ responses for a query and marks the query as hard when every response receives reward $0$. An adaptive prompt refiner then rebuilds the prompt as $q^* = q + \omega \cdot h_{f,q}$, where $h_{f,q}$ is the ground-truth solution trace and $\omega$ is a stage-based hint ratio chosen from $\{0.25, 0.5, 0.75\}$; the ratio increases only if the model keeps failing, so the same data point is reused with stronger guidance rather than discarded. The policy-gradient objective remains GRPO-style clipped advantage estimation, but the likelihood ratio is evaluated against the refined prompt, so difficult queries receive an imitation-learning update while manageable queries retain exploration-based reinforcement learning.

What would settle it

Run the same GHPO-versus-GRPO comparison on a newly written, publicly unavailable set of competition math problems and check whether the roughly 5% average gain shrinks; if it mostly disappears, the reported improvement is benchmark memorization rather than guided generalization. A second check: replace the ground-truth hint with a scrambled or incorrect prefix and see whether the gain vanishes, which would show the signal comes from the solution content rather than the added text.

Watch

Extended reading notes

Core claim

The central claim is that capacity-difficulty mismatch, not lack of data, is the main obstacle to RLVR training, and that exposing a failing policy to the beginning of a correct solution converts reward-sparse failures into usable imitation-learning signals. GHPO demonstrates this by showing that a 7B-parameter model trained with adaptive hint prompts outperforms the same model trained with plain GRPO, with a fixed-hint curriculum, or with a manually split curriculum, and that the gain persists when the base model is replaced by a stronger math-specialized model.

Load-bearing premise

The load-bearing premise is that for a problem the policy fails to solve, appending part of the correct solution improves out-of-distribution reasoning performance; the evaluation under-tests this because its benchmarks overlap with the data sources used to build the training set.

Editorial extensions

If this is right

  • Models too small to solve competition-level problems can still learn from those problems, because the hint turns an all-zero reward group into a valid gradient signal.
  • Training runs become more stable: the paper reports smaller gradient norms and steadier accuracy-reward growth for GHPO than for GRPO.
  • No training data is discarded: unlike filtering approaches that drop too-hard and too-easy prompts, GHPO keeps every query by escalating the hint ratio.
  • The gains appear across base models: both Qwen2.5-7B-Base and the stronger Qwen2.5-Math-7B improve over their GRPO counterparts.
  • The adaptive hint schedule doubles as a curriculum, so manual difficulty partitioning of the dataset is no longer required.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the overlap between the NuminaMath-S training set and benchmarks such as OlympiadBench and AMC is real and undeduplicated, part of the reported gain may come from memorizing near-duplicate problems; a held-out benchmark disjoint from training sources would settle how much of the roughly 5% is genuine generalization.
  • The same reward-sparsity argument should transfer to other verifiable domains, such as code generation or theorem proving, where partial reference solutions are available; the predicted effect is largest for lower-capacity policies.
  • The fixed three-stage hint schedule is a simple heuristic; a controller that sets the hint ratio from per-query accuracy or confidence could refine the idea further.
  • Because hints are extracted from ground-truth solutions, the method assumes solution traces are available at training time; if they are absent or expensive, a proxy difficulty signal would be needed to keep the adaptive guidance mechanism.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes GHPO (Guided Hybrid Policy Optimization), a reinforcement-learning-with-verifiable-rewards framework that adaptively injects partial ground-truth solution traces into prompts for problems the current policy cannot solve. For each prompt, GHPO samples a group of responses; if all responses are incorrect, it appends a fraction (0.25/0.5/0.75 by training stage) of the ground-truth solution and continues, otherwise it performs standard GRPO. The paper claims that this improves training stability and sample efficiency and reports an average accuracy gain of about 5% over GRPO and curriculum baselines on six mathematics benchmarks, with code released. The paper also presents training-dynamics analyses and an extension to Qwen2.5-Math-7B.

Significance. The core idea is timely and potentially valuable: adaptively converting reward-sparse difficult problems into guided imitation-learning problems is a plausible way to stabilize RLVR for small models. The paper has concrete strengths: the code is released, the reward is rule-based and verifiable, results are reported on two base models, and the training-dynamics monitoring is useful. However, the technical objective is not correctly specified as written, and the main empirical claims are compromised by construction because the training data overlap with the evaluation benchmarks. If these two problems were fixed, the contribution could be meaningful; in its current form the paper does not establish its central claim.

major comments (3)
  1. [§3.2, Eqs. (4)–(6)] The importance-weight update is not valid as written. Equation (4) defines the expectation with responses {o_i} sampled from π_{θ_old}(·|q), but Equation (6) defines the ratio r_{i,t}(θ) = π_θ(o_{i,t} | q^*, o_{i,<t}) / π_{θ_old}(o_{i,t} | q^*, o_{i,<t}) with q^* in both numerator and denominator. For a difficult query, q^* is the refined prompt (q + ω·h_{f,q}) and the sampled responses were generated before q^* was chosen. Under the stated sampling distribution, the correct importance ratio would be π_θ(o|q)/π_{θ_old}(o|q); replacing q by q^* in both terms does not correct for the sampling mismatch and introduces bias. If the intended algorithm instead re-samples responses from π_{θ_old}(·|q^*) after refinement, then Eq. (4) must sample from q^*, and it must also account for q^* being a random function of the first-stage group. The current formulation leaves the actual behavior policy underspecified, and this is load-bearing because the guided update is the central mechanism of GHPO.
  2. [§4.1 and §4.4, Tables 1–2] The evaluation is contaminated by construction. NuminaMath-S is built from math3to5 (drawn from MATH) plus additional problems sourced from OlympiadBench and AMC, while the evaluation suite includes MATH-500 (a subset of MATH), OlympiadBench, and AMC2023. The paper reports no deduplication or overlap statistics, so exact or near-duplicate test items can appear in the training data. The reported deltas on these benchmarks—for example AMC23 0.575 vs 0.525 in Table 2, or MATH-500 0.786 vs 0.752 in Table 1—may therefore reflect memorization rather than improved reasoning. This also undercuts the paper's own framing, since Assumption 1 is explicitly about out-of-distribution generalization but the main experiments do not establish OOD behavior. To support the headline claim, the authors need to filter training items that match the evaluation sets, report overlap counts (including near-duplicates), and either re-run the experiments or evaluate on disjoint held-out benchmarks with verified exclusion from every training source.
  3. [§3.1, Assumption 1] The validation of Assumption 1 is circular as presented. The paper states the assumption and then says it will 'demonstrate the effectiveness of this Assumption 1 through comprehensive experiment detailed in Section 4'; however, the Section 4 experiments compare the full GHPO method against GRPO and curriculum baselines, not the controlled effect of adding ground-truth hints to a fixed set of failing problems. The fixed-hint baseline (GRPO-CL-H) is a step in this direction, but it is coupled with curriculum learning and is evaluated on the contaminated test sets. A direct test would train on identical hard problems with and without hints at each fixed hint ratio and measure held-out, deduplicated accuracy. Until such a test is provided, the motivation for trace guidance remains an assumption rather than a demonstrated property.
minor comments (6)
  1. [§2.2, Eq. (3)] The sentence 'where πθ is the current policy and πθ is the reference model' defines both quantities with the same symbol; the reference model should be denoted (e.g., π_ref) and used consistently.
  2. [§3.4] The sentence 'As defined in Equation (6), for queries identified as difficult...' cites the wrong equation: Equation (6) defines the importance ratio, while the q^* construction and difficulty-detection condition are given in Equation (5) and §3.3.
  3. [§4.5 and Table 2] The fixed-hint baseline is named 'Qwen2.5-7B-GRPO-CL-H(0.5)' in Table 2 but 'Qwen2.5-7B-GHPO-CL-H0.5' in the text; the naming should be made consistent.
  4. [Abstract and §6] The claimed 'approximately 5%' average gain is not directly supported by the tables: Table 1 shows 0.442 vs 0.398 (4.4 percentage points), and Table 2 shows 0.442 vs 0.409 (3.3 points) for Qwen2.5-7B and 0.5076 vs 0.4728 (3.5 points) for Qwen2.5-Math-7B; the paper should state whether the 5% figure is relative or absolute and correct it.
  5. [§4.4] Reporting evaluation temperature as '0.0 or 1.0 (depending on the benchmark’s difficulty)' is not reproducible; a per-benchmark listing of sampling parameters is needed.
  6. [References] References [15] and [22] cite the same MATH dataset paper; they should be consolidated or clearly distinguished.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: GHPO's derivation chain is self-contained; the train/eval overlap is a validity caveat, not a circular step.

full rationale

The paper's derivation is not circular. GHPO modifies the GRPO objective by conditioning on a refined prompt q* (Eqs. 4-6); no fitted parameter is renamed as a prediction, no equation is defined in terms of the benchmark it claims to explain, and Assumption 1 is an explicit empirical hypothesis (Section 3.1) rather than a theorem derived from the method's own outputs. The claimed ~5% improvement is evaluated on benchmarks with no hints at inference time, so the result is not an identity. The main caveat lies in Section 4.1 versus Section 4.4: NuminaMath-S is built from 'additional problems sourced from OlympiadBench and AMC' and evaluation includes OlympiadBench and AMC2023, with no reported deduplication; this threatens the independence of those benchmark results, but it is a test-contamination/validity concern, not circularity, because the method's equations do not reduce to the evaluation and the overlap is not used to define the model's predictions. No load-bearing self-citation or imported uniqueness theorem appears in the derivation.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

GHPO introduces no new physical or conceptual entities; it is a training procedure. The central claim depends on three hand-set parameters (hint ratio schedule, stage count, cold-start length) and on two key assumptions: that partial ground-truth hints improve OOD generalization (Assumption 1) and that the training and evaluation sets are disjoint. A further ad hoc assumption is that the importance ratio as written in Eq (6) is valid, which is not satisfied by the stated sampling procedure.

free parameters (3)
  • hint_ratio_schedule = 0.25, 0.5, 0.75
    Manually chosen linear schedule for the proportion of ground-truth solution appended as a hint; no sensitivity analysis is reported.
  • num_stages = 3
    Maximum of three stages for increasing hint ratio; chosen by the authors without reported ablation.
  • cold_start_steps = 20
    Initial GRPO optimization steps before difficulty detection is enabled; selected without sensitivity analysis.
assumptions (4)
  • domain assumption Assumption 1: appending a partial ground-truth trace to a problem the base policy fails improves OOD generalization.
    Stated in Section 3.1 and then 'demonstrated' by the same experiments that evaluate GHPO; no independent proof.
  • ad hoc to paper The importance ratio in Eq (6) is a valid off-policy correction, which requires responses to be sampled from the refined prompt q*.
    Section 3.2 says responses are sampled from π_old(·|q), but the ratio conditions on q* in both numerator and denominator; no resampling step is described.
  • domain assumption Training and evaluation benchmark sets are disjoint.
    Training includes MATH, OlympiadBench, and AMC material while evaluation includes MATH-500, OlympiadBench, and AMC2023; no dedup is reported (Sections 4.1 and 4.4).
  • domain assumption The rule-based verifier correctly identifies correct and incorrect final answers.
    The method relies on binary verifiable rewards; no analysis of verifier noise is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GHPO: Adaptive Guidance for Stable and Efficient LLM Reinforcement Learning." pith.science (2026). https://pith.science/paper/BWCXFO5G

@misc{pith2026250710628,
  author       = {Pith},
  title        = {Pith review of: GHPO: Adaptive Guidance for Stable and Efficient LLM Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BWCXFO5G}},
  note         = {Machine review of arXiv:2507.10628}
}
read the original abstract

Reinforcement Learning with Verifiable Rewards (RLVR) has recently emerged as a powerful paradigm for facilitating the self-improvement of large language models (LLMs), particularly in the domain of complex reasoning tasks. However, prevailing on-policy RL methods often contend with significant training instability and inefficiency. This is primarily due to a capacity-difficulty mismatch, where the complexity of training data frequently outpaces the model's current capabilities, leading to critically sparse reward signals and stalled learning progress. This challenge is particularly acute for smaller, more resource-efficient LLMs. To overcome this, we introduce the Guided Hybrid Policy Optimization (GHPO), a novel difficulty-aware reinforcement learning framework. GHPO dynamically calibrates task difficulty by employing adaptive prompt refinement to provide targeted guidance. This unique approach adaptively balances direct imitation learning for problems currently beyond the model's reach with exploration-based reinforcement learning for more manageable tasks, effectively creating a smooth and optimized learning curriculum. Extensive experiments demonstrate that GHPO achieves an average performance gain of approximately 5% across six challenging mathematics benchmarks, consistently outperforming strong on-policy reinforcement learning and curriculum learning baselines. Further analysis confirms that our framework significantly enhances both training stability and final reasoning performance, thus offering a scalable and efficient solution for developing powerful and robust reasoning models.

Figures

Figures reproduced from arXiv: 2507.10628 by the authors.

Figure 1
Figure 1. Overall Performance of GHPO across several benchmarks on the Qwen2.5-7B base model. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. An illustration of the proposed GHPO framework. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Prompt template used in the proposed GHPO framework. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: An illustration of using the proposed GHPO for addressing a detected difficult problem with 50% hint [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: The proportion of problems detected to be difficult within a mini-batch. [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: The metric curves of format reward, accuracy reward, mean response length, and gradient norm of GRPO and [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Don't Tell the Answer, Truly Guide the Reasoning During RL Rollouts

    cs.LG 2025-10 conditional novelty 6.0 of 10

    HINT boosts LLM reasoning RL by injecting teacher-generated heuristic hints only on all-failed rollouts, keeping hints out of the policy-optimization prompt, and monitoring guidance quality with a new Affinity metric.

Reference graph

Works this paper leans on

29 extracted references · 9 canonical work pages · cited by 1 Pith paper

  1. [1]

    OpenAI o3 and o4-mini system card

    OpenAI. OpenAI o3 and o4-mini system card. https://openai.com/index/o3-o4-mini-system-card/, 2025

    work page 2025

  2. [2]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    arXiv 2025

  3. [3]

    Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi k1. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501.12599, 2025

    arXiv 2025

  4. [4]

    Chi, Quoc V

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V . Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS ’22, Red Hook, NY , USA, 2022. Curran Associates Inc

    work page 2022

  5. [5]

    Le, Sergey Levine, and Yi Ma

    Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V . Le, Sergey Levine, and Yi Ma. Sft memorizes, rl generalizes: A comparative study of foundation model post-training, 2025

    2025

  6. [6]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024

    arXiv 2024

  7. [7]

    Dapo: An open-source llm reinforcement learning system at scale

    Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, et al. Dapo: An open-source llm reinforcement learning system at scale. arXiv preprint arXiv:2503.14476, 2025

    arXiv 2025

  8. [8]

    Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Li Erran Li, Raluca Ada Popa, and Ion Stoica

    Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Y . Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Li Erran Li, Raluca Ada Popa, and Ion Stoica. Deepscaler: Surpassing o1-preview with a 1.5b model by scaling rl,

    work page

Show all 29 references
  1. [9]

    Learning to reason under off-policy guidance

    Jianhao Yan, Yafu Li, Zican Hu, Zhi Wang, Ganqu Cui, Xiaoye Qu, Yu Cheng, and Yue Zhang. Learning to reason under off-policy guidance. arXiv preprint arXiv:2504.14945, 2025

    2025 arXiv

  2. [10]

    Dynamic programming and statistical communication theory

    Richard Bellman and Robert Kalaba. Dynamic programming and statistical communication theory. Proceedings of the National Academy of Sciences of the United States of America, 43(8):749–751, 1957

    1957

  3. [11]

    Policy gradient methods for re- inforcement learning with function approximation

    Richard S Sutton, David McAllester, Satinder Singh, and Yishay Mansour. Policy gradient methods for re- inforcement learning with function approximation. Advances in neural information processing systems , 12, 1999. 13 GHPO

    1999

  4. [12]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  5. [13]

    Qwen2.5 technical report, 2025

    Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, ...

    2025

  6. [14]

    Numinamath

    Jia LI, Edward Beeching, Lewis Tunstall, Ben Lipkin, Roman Soletskyi, Shengyi Costa Huang, Kashif Rasul, Longhui Yu, Albert Jiang, Ziju Shen, Zihan Qin, Bin Dong, Li Zhou, Yann Fleureau, Guillaume Lample, and Stanislas Polu. Numinamath. [https://huggingface.co/AI-MO/NuminaMath...

    2024

  7. [15]

    Measuring mathematical problem solving with the math dataset

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. NeurIPS, 2021

    2021

  8. [16]

    Qwen2.5-math technical report: Toward mathematical expert model via self-improvement, 2024

    An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, Keming Lu, Mingfeng Xue, Runji Lin, Tianyu Liu, Xingzhang Ren, and Zhenru Zhang. Qwen2.5-math technical report: Toward mathematical expert model via se...

    2024

  9. [17]

    Open r1: A fully open reproduction of deepseek-r1, January 2025

    Hugging Face. Open r1: A fully open reproduction of deepseek-r1, January 2025

    2025

  10. [18]

    Trl: Transformer reinforcement learning

    Leandro von Werra, Younes Belkada, Lewis Tunstall, Edward Beeching, Tristan Thrush, Nathan Lambert, Shengyi Huang, Kashif Rasul, and Quentin Gallouédec. Trl: Transformer reinforcement learning. https: //github.com/huggingface/trl, 2020

    2020

  11. [19]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017

    2017 arXiv

  12. [20]

    Understanding r1-zero-like training: A critical perspective

    Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. Understanding r1-zero-like training: A critical perspective. arXiv preprint arXiv:2503.20783, 2025

    2025 arXiv

  13. [21]

    Lighteval: A lightweight framework for llm evaluation, 2023

    Nathan Habib, Clémentine Fourrier, Hynek Kydlíˇcek, Thomas Wolf, and Lewis Tunstall. Lighteval: A lightweight framework for llm evaluation, 2023

    2023

  14. [22]

    Measuring mathematical problem solving with the math dataset, 2021

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset, 2021

    2021

  15. [23]

    Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems, 2024

    Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Leng Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, Jie Liu, Lei Qi, Zhiyuan Liu, and Maosong Sun. Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scienti...

    2024

  16. [24]

    Solving quantitative reasoning problems with language models, 2022

    Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, Yuhuai Wu, Behnam Neyshabur, Guy Gur-Ari, and Vedant Misra. Solving quantitative reasoning problems with language models, 2022

    2022

  17. [25]

    David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. Gpqa: A graduate-level google-proof qa benchmark, 2023

    2023

  18. [26]

    Suttle, Mengdi Wang, Amrit Singh Bedi, and Dinesh Manocha

    Bhrij Patel, Souradip Chakraborty, Wesley A. Suttle, Mengdi Wang, Amrit Singh Bedi, and Dinesh Manocha. Aime: Ai system optimization via multiple llm evaluators, 2024

    2024

  19. [27]

    Simplerl-zoo: Investigating and taming zero reinforcement learning for open base models in the wild, 2025

    Weihao Zeng, Yuzhen Huang, Qian Liu, Wei Liu, Keqing He, Zejun Ma, and Junxian He. Simplerl-zoo: Investigating and taming zero reinforcement learning for open base models in the wild, 2025. URL https://arxiv. org/abs/2503.18892

    2025 arXiv

  20. [28]

    Vapo: Efficient and reliable reinforcement learning for advanced reasoning tasks

    Yu Yue, Yufeng Yuan, Qiying Yu, Xiaochen Zuo, Ruofei Zhu, Wenyuan Xu, Jiaze Chen, Chengyi Wang, TianTian Fan, Zhengyin Du, et al. Vapo: Efficient and reliable reinforcement learning for advanced reasoning tasks. arXiv preprint arXiv:2504.05118, 2025

    2025 arXiv

  21. [29]

    Demystifying long chain-of-thought reasoning in llms

    Edward Yeo, Yuxuan Tong, Morry Niu, Graham Neubig, and Xiang Yue. Demystifying long chain-of-thought reasoning in llms. arXiv preprint arXiv:2502.03373, 2025. 14 GHPO A Supplementary Case Table 3: The original problem and the ground-truth solution in the case study. Problem: C...

    2025 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.