Pith. sign in

REVIEW 4 major objections 5 minor 48 references

LEEPS: Latent-Guided Explore-Exploit Prompt Sampling for Efficient RLVR in Large Language Models

T0 review · 4 major / 5 minor · reviewed 2026-07-31 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A pre-rollout sampler that mixes known useful prompts with latent-neighbor exploration raises math RL scores without extra rollouts.

desk verdict Solid pre-rollout RLVR sampler with a coherent explore/exploit + latent-neighbor design; gains are real-looking but single-run and small, so treat the ranking claims as provisional. read the letter →

arxiv 2607.28077 v1 pith:VM3TJCKL submitted 2026-07-30 cs.CL

Shuang Liang , Haoyang Zhou , Yifan Gong , Guowei Wang , Xiting Wang This is my paper · ORCID
classification cs.CL
keywords RLVRpromptsamplingexplore-exploitlatentneighborsGRPOmathematicalreasoningpre-rolloutselectionnon-trivialratio
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

When language models learn from checkable answers, many prompts waste generation because every sample in a group is right or every sample is wrong, so the update gets no signal. Earlier pre-rollout selectors either keep recycling a small set of historically useful prompts or explore so widely that the batch fills with uninformative ones. This paper argues that you can keep a high share of informative prompts and still cover more of the training pool by splitting candidates into an exploit portfolio and an explore portfolio, then allocating the fixed rollout budget from their recent non-trivial rates, and by ranking uncertain prompts with a static nearest-neighbor estimate built from model hidden states and past neighbor outcomes. On two math-specialized model sizes the method posts the best average across six math benchmarks under matched rollout budgets, improves faster during training, transfers to three out-of-domain reasoning suites, and adds only about two seconds of online sampling cost per step.

What carries the argument

LEEPS: Adaptive Explore–Exploit Portfolio Allocation plus Latent-Guided Exploration. Candidates are split into an exploit set of previously non-trivial prompts (weighted by success-rate variance) and an explore set of unseen or zero-variance prompts; the explore fraction is chosen so the expected batch non-trivial ratio stays near a target; uncertain prompts are ranked by a count-decayed score from cosine neighbors in a fixed final-token hidden-state cache whose observed success rates estimate intermediate difficulty.

What would settle it

Replace latent-neighbor ranking with uniform sampling from the explore portfolio (or refresh/remove the static K-NN graph) and check whether the reported average gains on the six math benchmarks and the high non-trivial ratio disappear under the same rollout budget.

Watch

Extended reading notes

Core claim

Under matched rollout budgets and without any extra selection rollouts, LEEPS—the latent-guided explore–exploit prompt sampler—attains the highest average score on six mathematical reasoning benchmarks for both Qwen2.5-Math-1.5B and 7B, with relative gains of 2.6% and 3.7% over the strongest prior pre-rollout baseline, while also leading on three OOD general-reasoning benchmarks and adding roughly two seconds of online sampling overhead per training step.

Load-bearing premise

A fixed nearest-neighbor map of prompt hidden states, built once before training, keeps predicting which currently uncertain prompts will produce mixed right-and-wrong rollouts as the model changes.

Editorial extensions

If this is right

  • Pre-rollout RLVR can raise final math scores and training speed without paying the extra generation cost of post-rollout filtering.
  • Keeping separate exploit and explore portfolios with a target non-trivial ratio sustains both informative batches and broader prompt coverage than pure exploitation or pure exploration.
  • Static representation-space neighbors plus historical neighbor outcomes can steer exploration without online re-encoding or extra rollouts.
  • Gains under this sampler also show up on OOD general-reasoning suites, not only in-domain math.
  • Online selection overhead stays on the order of two seconds per step once the neighbor cache is built.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If latent locality of non-triviality holds more generally, the same portfolio-plus-neighbor pattern could apply to other verifiable domains such as code or formal proof without redesigning the reward.
  • Periodic light refresh of the neighbor graph might be needed when policy shift breaks the offline AUROC signal; that is a natural stress test the paper leaves open.
  • Curriculum and adaptive rollout-count methods could plug into the same explore–exploit quota rather than competing with it.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes LEEPS, a pre-rollout prompt sampler for RLVR/GRPO that reduces waste from zero-variance rollout groups. Candidates are split into an exploit portfolio (historically non-trivial prompts, weighted by Bernoulli variance â(1−â)) and an explore portfolio (unseen or latest zero-variance prompts). Rollout budget is allocated by an adaptive explore fraction ρ chosen so that a linear mixture of recent portfolio non-trivial ratios tracks a target τ. Exploration is further ranked by a count-decayed latent uncertainty score that estimates success rate from cosine K-NN neighbors in a fixed final-token hidden-state cache (no extra selection rollouts). On DAPO-Math-17K with Qwen2.5-Math-1.5B/7B, under matched rollout budgets, LEEPS reports the highest six-benchmark average versus GRPO, DS, MoPPS, and DPS (relative gains 2.6% and 3.7% over DPS), plus the highest average on three OOD suites, ablations of random explore / uniform exploit / no portfolios, Llama-3.2-3B results, sensitivity to τ/K, and ~2s per-step sampling overhead. Code is linked.

Significance. If the empirical ranking holds under matched budgets, LEEPS is a useful systems contribution to efficient RLVR: it targets the explore–exploit tension that the paper documents between DPS (high non-trivial ratio, narrow coverage) and MoPPS (broader coverage, lower early non-trivial ratio), without the extra generation cost of post-rollout filters like DS. Strengths include a clear problem formulation, matched-budget and rollout-normalized comparisons, training diagnostics (non-trivial and medium ratios, coverage CCDF), component ablations, a second backbone (Llama), OOD checks, overhead profiling, and public code. The method is heuristic rather than theoretically guaranteed, but that is appropriate for this empirical systems setting. The main significance risk is whether the reported 1–1.7 point overall-score margins and OOD tenths-of-a-point lifts survive multi-seed evaluation and fixed checkpoint protocols.

major comments (4)
  1. [Table 1; Fig. 5; Abstract] Table 1 and the abstract’s 2.6%/3.7% relative gains over DPS rest on single training trajectories per method/scale, with no multi-seed means, standard errors, or significance tests. GRPO with temperature-1 group rollouts and random Bcand draws is high-variance; Fig. 5 curves fluctuate by amounts comparable to or larger than the 0.99–1.70 absolute overall-score gaps. Without seed replication (or at least final-checkpoint multi-run reporting), the central ranking claim and the stated relative percentages are not established at the precision claimed.
  2. [Table 2; Table 5; Fig. 5] Several tables select favorable checkpoints rather than a pre-specified final or average-over-window metric: Table 5 explicitly takes “the checkpoint with the highest average score within the first 160 steps”; Table 2 uses “best overall scores within the model-specific selection ranges.” Combined with single runs, peak-picking can reorder methods when curves cross (Fig. 5). Please report a fixed protocol (e.g., last checkpoint, or mean of last K evals) for all methods, and apply the same protocol to ablations and Llama.
  3. [Table 3; Abstract; Conclusion] Table 3 OOD averages favor LEEPS by only +0.14 (1.5B) and +0.10 (7B) over the strongest baselines, with mixed per-benchmark wins/losses. The abstract and conclusion state that LEEPS “achieves the highest average score across the three evaluated OOD … benchmarks at both model scales” as a parallel headline to the in-domain gains. Those margins are well inside plausible run noise given the issues above; either multi-seed OOD results or substantially softer wording is needed so OOD is not presented as equally established.
  4. [Methods, Latent-Guided Exploration; Fig. 3; Eq. (9)] Latent-Guided Exploration (Eqs. 6–9; Fig. 3) uses a static pre-RL K-NN graph on fixed layers (22/26) and never refreshes representations as the policy changes. Fig. 3 is an offline AUROC on a held-out split of the same training pool, not a demonstration that neighbor non-triviality remains predictive late in training. The random-exploration ablation (Table 2, Fig. 6) supports some value of the score, but does not test representation staleness. A load-bearing check would be mid/late-training AUROC of âlat_p vs. realized non-triviality, or a cheap refresh ablation; without it, the claim that latent locality guides exploration “as the policy evolves” is only weakly supported.
minor comments (5)
  1. [Figure 1] Figure 1 right panel y-axis is log-scaled with uneven tick labels (0,1,10,20,50,100); a clearer log scale or explicit note would help read the coverage CCDF.
  2. [Preliminaries; Methods] Notation mixes a(p), âp, and âlat_p; a short symbol table in Preliminaries or Methods would reduce scanning cost.
  3. [Algorithm 1; Table 4] Algorithm 1 is helpful; briefly state how q_explore and q_exploit are estimated from the “adaptive window” of 3 (Table 4)—e.g., simple moving average of batch non-trivial indicators—so the ρ update is fully reproducible from the main text.
  4. [Related Work] Related Work cites concurrent/arXiv pre-rollout selectors (MoPPS, DPS) appropriately; ensure camera-ready citation keys and any venue updates are consistent.
  5. [Throughout] Minor prose spacing artifacts appear throughout (“improvesthereasoning”, “beforerolloutgeneration”); clean for camera-ready.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: empirical sampler whose gains are measured on held-out benchmarks, not derived from quantities defined as the fit target.

full rationale

LEEPS is an engineering/heuristic pre-rollout prompt sampler (exploit portfolio weighted by Bernoulli variance â(1−â), explore portfolio ranked by count-decayed latent-neighbor success-rate estimates, adaptive ρ targeting a preset non-trivial ratio τ). The load-bearing claims are empirical rankings under matched rollout budgets on six math benchmarks and three OOD suites (Tables 1–3, 5), plus ablations (Table 2) and training diagnostics (Figs. 1, 4–6). Nothing in the Methods equates a reported benchmark gain to a fitted input by construction: âlat_p is a neighbor-weighted average of previously observed group success rates used only to prioritize uncertain candidates; Fig. 3 is an offline AUROC diagnostic of that heuristic on a held-out split of the training pool, not a prediction of the main evaluation metrics; τ, ρ0, K, and layer choice are hyperparameters validated by sensitivity/ablation, not smuggled uniqueness theorems. There is no self-definitional loop, no fitted parameter renamed as the headline result, and no load-bearing self-citation chain. Self-contained against external benchmarks with a rule verifier; circularity score 0.

Assumptions & free parameters 5 free parameters · 4 assumptions · 2 invented entities

Load-bearing content is methodological and empirical, not axiomatic physics. The claim rests on GRPO/RLVR problem structure, the operational definitions of non-trivial groups and portfolios, several hand-set sampler hyperparameters, and the modeling assumption that static latent neighborhoods proxy future group variance. No new physical entities; free parameters are the usual ML knobs that affect allocation and ranking.

free parameters (5)
  • target non-trivial ratio τ = 0.90 (main); 0.80 (Llama)
    Online allocation solves for explore fraction ρ so estimated batch non-trivial ratio matches τ; main runs use 0.90 (0.80 on Llama). Sensitivity is shown but peak scores use the chosen default.
  • default exploration fraction ρ0 and clip range [ρmin, ρmax] = ρ0=0.15, [0.05, 0.95]
    Used when portfolio feedback is insufficient and to stabilize ρ; set to 0.15 and 0.05–0.95.
  • neighbor count K and cached neighbors = K=64, cache=1024
    Controls latent success-rate estimate; default K=64 with 1024 cached neighbors, justified by offline AUROC plateau not by nested CV on final scores.
  • representation layer index = 22 / 26 / 21 by model
    Chosen via offline non-trivial AUROC (L22 for 1.5B, L26 for 7B, L21 for Llama); fixed for all RL training.
  • cold-start steps Tcold, adaptive window, candidate ratio |Bcand|/B = Tcold=5, window=3, ratio=16
    Schedule and candidate enlargement hyperparameters that shape early exploration and selection pool size.
assumptions (4)
  • domain assumption GRPO group-normalized advantages vanish when all binary rewards in a group are identical (a(p)∈{0,1}), so zero-variance groups waste rollout budget.
    Preliminaries; standard under binary verifiable rewards and group-relative normalization as used in cited GRPO/RLVR work.
  • domain assumption Prompts with intermediate empirical success rates are more informative than very easy or very hard ones (Bernoulli variance weight â(1−â)).
    Exploit weighting and related-work citations (Bae et al., Chen et al., Qu et al.); treated as design prior, not proved.
  • ad hoc to paper Cosine neighborhoods in a fixed final-token hidden state space share enough rollout behavior that weighted neighbor success rates predict non-triviality of unseen/zero-variance prompts.
    Core of Latent-Guided Exploration; supported by offline AUROC (Fig. 3) and ablation vs random explore, not by a general theorem.
  • ad hoc to paper A single scalar target non-trivial ratio τ is a sufficient control objective for allocating explore vs exploit budget toward better final reasoning scores.
    Adaptive allocation section; τ is chosen and sensitivity-tested, but the mapping from τ to generalization is empirical.
invented entities (2)
  • LEEPS explore/exploit portfolios with adaptive ρ targeting τ
    purpose: Operational partition of candidates and budget split to keep batch non-trivial ratio high while continuing exploration.
    Method construct defined in Eqs. (3)–(7) and Algorithm 1; evaluated only via training metrics and downstream scores.
  • Latent-neighbor success estimate âlat_p and count-decayed explore weight
    purpose: Rank uncertain prompts without extra rollouts using static K-NN over hidden states.
    Defined in Eqs. (6)–(9); predictive power shown offline and via ablation, not as an external physical quantity.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LEEPS: Latent-Guided Explore-Exploit Prompt Sampling for Efficient RLVR in Large Language Models." pith.science (2026). https://pith.science/paper/VM3TJCKL

@misc{pith2026260728077,
  author       = {Pith},
  title        = {Pith review of: LEEPS: Latent-Guided Explore-Exploit Prompt Sampling for Efficient RLVR in Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VM3TJCKL}},
  note         = {Machine review of arXiv:2607.28077}
}
read the original abstract

Reinforcement learning with verifiable rewards (RLVR) improves the reasoning capabilities of large language models, but prompt groups with identical rollout rewards consume generation budget without effective learning signals. Pre-rollout prompt selection can reduce this waste by screening prompts before rollout generation. However, existing pre-rollout methods struggle to balance exploitation and exploration: repeatedly exploiting historically informative prompts can narrow training coverage, whereas broader exploration can lower the fraction of informative prompts. To address these limitations, we introduce LEEPS, a Latent-Guided Explore--Exploit Prompt Sampler that adaptively balances the reuse of previously observed informative prompts with continued exploration of uncertain ones. LEEPS partitions candidates into exploit and explore portfolios and adaptively allocates rollout budget according to their recent non-trivial ratios. It further uses representation-space neighbors and historical rollout outcomes to prioritize uncertain prompts likely to yield non-zero reward variance, thereby making exploration more targeted without additional rollouts. Across six mathematical reasoning benchmarks, LEEPS achieves the highest average score at both model scales, with relative gains of 2.6\% and 3.7\% over the strongest baseline for Qwen2.5-Math-1.5B and 7B, respectively, and generally improves faster during the training process. It also achieves the highest average score across the three evaluated OOD general-reasoning benchmarks at both model scales and adds only about 2 seconds of online sampling overhead per training step. Code is available at https://github.com/ShuangLiangX/LEEPS.

Figures

Figures reproduced from arXiv: 2607.28077 by the authors.

Figure 1
Figure 1. Training dynamics on Qwen2.5-Math-7B. Left: the [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of LEEPS. Candidate prompts are divided into exploit and explore portfolios, and the training budget is [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Layer-wise AUROC of the latent-neighbor score [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Training-sample characteristics for Qwen2.5-Math-1.5B and Qwen2.5-Math-7B. Panels (a)–(b) report the non-trivial [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Training progress for 1.5B and 7B models. Panels (a)–(b) show overall score versus training step, and Panels (c)–(d) [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Ablation diagnostics on Qwen2.5-Math-1.5B. [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Average per-step runtime over 20 profiled steps for [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: Additional results on Llama-3.2-3B-Instruct. Panels (a)–(b) show the overall score versus training step and cumulative [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: Sample efficiency of layer calibration. Pearson [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]
Figure 10
Figure 10. Figure 10: Sensitivity of non-trivial-prompt prediction AU [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]
Figure 11
Figure 11. Figure 11: Sensitivity of LEEPS to the target non-trivial ratio [PITH_FULL_IMAGE:figures/full_fig_p015_11.png]
Figure 12
Figure 12. Figure 12: Extended training-step comparison between GRPO and LEEPS. LEEPS is shown through the model-specific ranges [PITH_FULL_IMAGE:figures/full_fig_p015_12.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

48 extracted references · 20 linked inside Pith

  1. [1]

    arXiv preprint arXiv:2411.15124 , year=

    Tulu 3: Pushing frontiers in open language model post-training , author=. arXiv preprint arXiv:2411.15124 , year=

    arXiv

  2. [2]

    arXiv preprint arXiv:2506.14245 , year=

    Reinforcement learning with verifiable rewards implicitly incentivizes correct reasoning in base llms , author=. arXiv preprint arXiv:2506.14245 , year=

    arXiv

  3. [3]

    arXiv preprint arXiv:2502.14768 , year=

    Logic-rl: Unleashing llm reasoning with rule-based reinforcement learning , author=. arXiv preprint arXiv:2502.14768 , year=

    arXiv

  4. [4]

    Advances in Neural Information Processing Systems , volume=

    Open-reasoner-zero: An open source approach to scaling up reinforcement learning on the base model , author=. Advances in Neural Information Processing Systems , volume=

  5. [5]

    arXiv preprint arXiv:2211.14275 , year=

    Solving math word problems with process-and outcome-based feedback , author=. arXiv preprint arXiv:2211.14275 , year=

    arXiv

  6. [6]

    International Conference on Learning Representations , volume=

    Let's verify step by step , author=. International Conference on Learning Representations , volume=

  7. [7]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Math-shepherd: Verify and reinforce llms step-by-step without human annotations , author=. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  8. [8]

    5: Scaling reinforcement learning with llms , author=

    Kimi k1. 5: Scaling reinforcement learning with llms , author=. arXiv preprint arXiv:2501.12599 , year=

    arXiv

Show all 48 references
  1. [9]

    arXiv preprint arXiv:2503.17287 , year=

    Fastcurl: Curriculum reinforcement learning with stage-wise context scaling for efficient training r1-like reasoning models , author=. arXiv preprint arXiv:2503.17287 , year=

  2. [10]

    arXiv preprint arXiv:2506.06632 , year=

    Curriculum reinforcement learning from easy to hard tasks improves LLM reasoning , author=. arXiv preprint arXiv:2506.06632 , year=

  3. [11]

    Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Online difficulty filtering for reasoning oriented reinforcement learning , author=. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  4. [12]

    Advances in Neural Information Processing Systems , volume=

    Dapo: An open-source llm reinforcement learning system at scale , author=. Advances in Neural Information Processing Systems , volume=

  5. [13]

    Advances in Neural Information Processing Systems , volume=

    Improving data efficiency for llm reinforcement fine-tuning through difficulty-targeted online data selection and rollout replay , author=. Advances in Neural Information Processing Systems , volume=

  6. [14]

    arXiv preprint arXiv:2506.09016 , year=

    Speed-rl: Faster training of reasoning models via online curriculum learning , author=. arXiv preprint arXiv:2506.09016 , year=

  7. [15]

    Advances in Neural Information Processing Systems , volume=

    Act only when it pays: Efficient reinforcement learning for llm reasoning via selective rollouts , author=. Advances in Neural Information Processing Systems , volume=

  8. [16]

    Advances in Neural Information Processing Systems , volume=

    Optimizing chain-of-thought reasoners via gradient variance minimization in rejection sampling and rl , author=. Advances in Neural Information Processing Systems , volume=

  9. [17]

    arXiv preprint arXiv:2510.01037 , year=

    CurES: From Gradient Analysis to Efficient Curriculum Learning for Reasoning LLMs , author=. arXiv preprint arXiv:2510.01037 , year=

  10. [18]

    Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V

    Can prompt difficulty be online predicted for accelerating rl finetuning of reasoning models? , author=. Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 1 , pages=

  11. [19]

    arXiv preprint arXiv:2603.10887 , year=

    Dynamics-predictive sampling for active RL finetuning of large reasoning models , author=. arXiv preprint arXiv:2603.10887 , year=

  12. [20]

    arXiv preprint arXiv:2505.14970 , year=

    Self-evolving curriculum for llm reasoning , author=. arXiv preprint arXiv:2505.14970 , year=

  13. [21]

    arXiv preprint arXiv:2602.19208 , year=

    How to allocate, how to learn? dynamic rollout allocation and advantage modulation for policy optimization , author=. arXiv preprint arXiv:2602.19208 , year=

    arXiv

  14. [22]

    arXiv preprint arXiv:2504.05520 , year=

    Efficient reinforcement finetuning via adaptive curriculum learning , author=. arXiv preprint arXiv:2504.05520 , year=

    arXiv

  15. [23]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    Learning like humans: Advancing llm reasoning capabilities via adaptive difficulty curriculum learning and expert-guided self-reformulation , author=. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    2025

  16. [24]

    arXiv preprint arXiv:2510.01135 , year=

    Prompt curriculum learning for efficient llm post-training , author=. arXiv preprint arXiv:2510.01135 , year=

  17. [25]

    arXiv preprint arXiv:2504.13818 , year=

    Not all rollouts are useful: Down-sampling rollouts in llm reinforcement learning , author=. arXiv preprint arXiv:2504.13818 , year=

    arXiv

  18. [26]

    Advances in neural information processing systems , volume=

    Training language models to follow instructions with human feedback , author=. Advances in neural information processing systems , volume=

  19. [27]

    Advances in neural information processing systems , volume=

    Deep reinforcement learning from human preferences , author=. Advances in neural information processing systems , volume=

  20. [29]

    Advances in neural information processing systems , volume=

    Direct preference optimization: Your language model is secretly a reward model , author=. Advances in neural information processing systems , volume=

  21. [30]

    arXiv preprint arXiv:2501.12948 , year=

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning , author=. arXiv preprint arXiv:2501.12948 , year=

    arXiv

  22. [31]

    arXiv preprint arXiv:2402.03300 , year=

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models , author=. arXiv preprint arXiv:2402.03300 , year=

    arXiv

  23. [32]

    arXiv preprint arXiv:1707.06347 , year=

    Proximal policy optimization algorithms , author=. arXiv preprint arXiv:1707.06347 , year=

    arXiv

  24. [33]

    arXiv preprint arXiv:2204.05862 , year=

    Training a helpful and harmless assistant with reinforcement learning from human feedback , author=. arXiv preprint arXiv:2204.05862 , year=

    arXiv

  25. [34]

    arXiv preprint arXiv:2110.14168 , year=

    Training verifiers to solve math word problems , author=. arXiv preprint arXiv:2110.14168 , year=

    arXiv

  26. [35]

    5-coder technical report , author=

    Qwen2. 5-coder technical report , author=. arXiv preprint arXiv:2409.12186 , year=

    arXiv

  27. [36]

    arXiv preprint arXiv:2409.12122 , year=

    Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement , author=. arXiv preprint arXiv:2409.12122 , year=

    arXiv

  28. [37]

    arXiv preprint arXiv:2407.10671 , year=

    Qwen2 technical report , author=. arXiv preprint arXiv:2407.10671 , year=

    arXiv

  29. [38]

    Proceedings of the Twentieth European Conference on Computer Systems , pages=

    Hybridflow: A flexible and efficient rlhf framework , author=. Proceedings of the Twentieth European Conference on Computer Systems , pages=

  30. [39]

    Proceedings of the 29th symposium on operating systems principles , pages=

    Efficient memory management for large language model serving with pagedattention , author=. Proceedings of the 29th symposium on operating systems principles , pages=

  31. [40]

    arXiv preprint arXiv:2103.03874 , year=

    Measuring mathematical problem solving with the math dataset , author=. arXiv preprint arXiv:2103.03874 , year=

    arXiv

  32. [41]

    Advances in neural information processing systems , volume=

    Solving quantitative reasoning problems with language models , author=. Advances in neural information processing systems , volume=

  33. [42]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems , author=. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  34. [43]

    Hugging Face repository , volume=

    Numinamath: The largest public dataset in ai4maths with 860k pairs of competition math problems and solutions , author=. Hugging Face repository , volume=

  35. [44]

    arXiv preprint arXiv:2509.10625 , year=

    No answer needed: Predicting llm answer accuracy from question-only linear probes , author=. arXiv preprint arXiv:2509.10625 , year=

  36. [45]

    arXiv preprint arXiv:2406.01574 , year=

    Mmlu-pro: A more robust and challenging multi-task language understanding benchmark , author=. arXiv preprint arXiv:2406.01574 , year=

    arXiv

  37. [46]

    arXiv:1803.05457v1 , year=

    Peter Clark and Isaac Cowhey and Oren Etzioni and Tushar Khot and Ashish Sabharwal and Carissa Schoenick and Oyvind Tafjord , title=. arXiv:1803.05457v1 , year=

    arXiv

  38. [47]

    arXiv preprint arXiv:2311.12022 , year=

    Gpqa: A graduate-level google-proof q&a benchmark , author=. arXiv preprint arXiv:2311.12022 , year=

    arXiv

  39. [48]

    arXiv preprint arXiv:2509.24711 , year=

    On the Self-awareness of Large Reasoning Models' Capability Boundaries , author=. arXiv preprint arXiv:2509.24711 , year=

  40. [49]

    arXiv preprint arXiv:2510.26374 , year=

    BOTS: A Unified Framework for Bayesian Online Task Selection in LLM Reinforcement Finetuning , author=. arXiv preprint arXiv:2510.26374 , year=

Pith tools

Reviewed July 31, 2026 · model on record in the stance chip above.