Pith. sign in

REVIEW 2 major objections 6 minor 31 references

GRPO shrinks reasoning coverage by reinforcing paths the base model already favors; ReCo reweights both response and token updates to keep more correct paths alive.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

ReCo fixes GRPO’s collapse onto high-probability base-model answers by normalizing response weights by expected occurrence and scaling token updates by Bernoulli variance, improving large-k Pass@k on math reasoning.

T0 review reviewed 2026-07-30 challenge →

load-bearing objection Clean, usable fix for GRPO’s large-k Pass@k collapse: two concrete reweights, solid ablations, real but ordinary methods-paper gaps. the 2 major comments →

arxiv 2607.26862 v1 pith:BVSY6CPC submitted 2026-07-29 cs.LG cs.AI

ReCo: Reweighting GRPO Against Distributional Concentration

Junoh Park , Junseo Hwang , Wonguk Cho , Taesup Kim This is my paper
classification cs.LG cs.AI
keywords GRPORLVRPass@kreasoning diversityimportance ratiodistributional concentrationpolicy reweightingmathematical reasoning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Standard group-relative policy training for language models improves easy sampling success but often loses the base model's breadth of correct solutions when many samples are drawn. This paper argues that the loss is not mysterious: the update itself piles gradient mass on high-probability responses that appear often in each rollout group, and the usual importance ratio further hardens already-dominant token choices. ReCo counters both effects by downweighting responses in proportion to how often the old policy is expected to emit them, and by scaling token updates with a variance-style ratio that is large only when a decision is still open to alternatives. Across math reasoning benchmarks and several model sizes, the method matches ordinary group-relative training at small sample budgets and beats it at large ones, while preserving more diverse rollouts during training. A sympathetic reader cares because many post-training pipelines treat group-relative optimization as default; if its internal concentration mechanisms are the real bottleneck, a simple reweighting can recover coverage without adding a separate diversity objective.

Core claim

The Pass@k drop at large k under GRPO is tied to distributional concentration on correct answers the base model already assigns high probability. That concentration is produced by two update mechanisms: repeated sampling lets frequent responses dominate the group gradient, and the token importance ratio amplifies choices whose probability has already risen. Reweighting response contributions by inverse expected occurrence and replacing the importance ratio with a variance ratio p(1-p) restores broader coverage of correct reasoning paths while keeping small-k efficiency comparable to GRPO.

What carries the argument

ReCo (Reweighting GRPO Against Distributional Concentration): a two-level correction that multiplies each token gradient by a response weight 1/(G · length-normalized old-policy likelihood) and by a variance ratio pθ(1−pθ)/pold(1−pold), so frequent trajectories and saturated local decisions receive less update mass.

Load-bearing premise

The method treats length-normalized sequence likelihood under the old policy as a good enough stand-in for how often a reasoning path would dominate the group gradient, even though exact duplicate answers are rare in open-ended generation.

What would settle it

Train GRPO and ReCo with identical rollouts and rewards, then check whether correct answers under ReCo systematically sit lower in the base model's probability ranking (as in the paper's KDE plot) and whether Pass@64 falls back to GRPO levels if either the expected-occurrence response weight or the p(1-p) token ratio is removed.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • At large sampling budgets, policies trained with ReCo should beat GRPO on Pass@k while staying competitive at small k.
  • Training rollouts should stay closer to the base model's lexical and prefix diversity instead of collapsing as reward rises.
  • The same reweighting can be stacked on other GRPO variants (for example alternative clipping or token-selection rules) and still add gains.
  • The concentration problem and the fix are not math-only: code-generation Pass@k shows the same pattern of recovery at larger k.
  • Token-level Bernoulli variance should remain higher in early and middle response positions under ReCo than under GRPO.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If expected-count reweighting is the right response-level idea, sampling schemes that deliberately oversample rare high-reward paths could complement ReCo instead of fighting the same concentration after the fact.
  • The variance ratio is essentially a soft focus on unsaturated decision points; similar p(1-p) scaling may help any on-policy token update that currently uses raw probability ratios.
  • When RL post-training hurts a backbone at large k, diagnosing base-model probability mass of correct outputs may be a quicker failure check than only plotting Pass@k curves.
  • Adaptive schedules that strengthen response reweighting only after diversity metrics start falling could reduce the slower early reward climb the paper reports for ReCo.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper argues that GRPO’s known Pass@k degradation at large k stems from distributional concentration on responses already favored by the base model. It attributes this to two update mechanisms: (i) high-probability responses dominate group gradients via more frequent sampling, and (ii) the token importance ratio further amplifies tokens whose probability has already risen. ReCo counters both by normalizing each response’s contribution by a length-normalized expected-occurrence proxy (1/(G π̄_old)) and replacing the importance ratio with a variance ratio pθ(1−pθ)/pold(1−pold) that upweights non-saturated token decisions. Across Qwen2.5-Math-1.5B/7B and Llama-3.1-8B-Instruct on five math benchmarks (plus code transfer), ReCo matches GRPO at small k, improves Pass@64 and large-k curves, and better preserves rollout diversity, entropy, and path uniqueness; ablations and combinations with DAPO/80-20 support complementarity of the two corrections.

Significance. If the results hold under broader replication, this is a useful, low-overhead fix for a widely used RLVR algorithm rather than a new training paradigm. The contribution is practical and mechanistic: it ties a documented Pass@k pathology to concrete terms in the GRPO gradient and shows that reweighting those terms recovers large-k coverage without sacrificing small-k efficiency. Strengths include multi-backbone/multi-benchmark evaluation, component ablations (Table 3), external baselines (ASPO, DAPO, 80/20), training-dynamics and path-uniqueness analyses, code-domain transfer, and qualitative case studies. The work is incremental relative to the diversity-collapse literature but actionable for practitioners already on GRPO-style stacks.

major comments (2)
  1. [§3.2, Eqs. (2)–(5)] §3.2, Eqs. (2)–(5): The response-level story is framed as correcting repeated occurrence via E[Nq(o)]=G π_old(o|q), but open-ended generation almost never yields exact duplicates (as the paper notes). The implemented weight uses length-normalized likelihood as a soft proxy. Table 3 shows Response-only still helps and Token-only already beats GRPO, so the headline result does not rest solely on this proxy; still, the manuscript should either (a) quantify how well π̄_old ranks near-duplicate/semantically equivalent paths that actually share gradient mass, or (b) reframe the response term as likelihood-based inverse propensity rather than “expected count,” so the claimed mechanism matches the operator used in training.
  2. [Tables 1–3; Figures 2–3] Tables 1–3 and Figures 2–3 report point estimates without multi-seed variance or confidence intervals, while several claimed deltas are a few points (e.g., +2–4 Pass@64 on Qwen). Given stochastic rollouts and RL training noise, at least 2–3 seeds on the main 1.5B setting (or bootstrap CIs on Pass@k) are needed to establish that large-k gains and the Response/Token ablation ordering are stable, not run-specific.
minor comments (6)
  1. [Figure 1] Figure 1 KDE is only on AIME 2025 / 1.5B; a brief multi-benchmark or multi-model check would strengthen the concentration diagnosis that motivates the method.
  2. [§3.4] Clarify interaction of w_resp with the existing 1/|oi| length average and with advantage normalization; a short note on whether response weights are renormalized within the group would aid reimplementation.
  3. [§3.4] Stop-gradient on (1−pθ) in the clipped objective (§3.4) is important; state explicitly whether unclipped analysis gradients use the same stop-grad convention.
  4. [§5] Related work could more sharply contrast ReCo with entropy bonuses and other diversity regularizers beyond ASPO/DAPO/80-20, since the claim is “reweight the update” vs “add an external objective.”
  5. [Appendix B.4] Appendix code results (Table 7) are strong; consider promoting a short summary into the main experiments so the transfer claim is visible without the appendix.
  6. Minor polish: consistent notation for π̄ vs bar-pi; ensure all cited arXiv preprints that have venue versions are updated if applicable.

Circularity Check

0 steps flagged

No significant circularity: empirical reweighting method with independent held-out metrics; design formulas do not force Pass@k by construction.

full rationale

ReCo is an empirical RLVR methods paper. The load-bearing claims are (i) GRPO concentrates on base-model high-probability correct responses and (ii) two-level reweighting (expected-occurrence response weights plus variance-based token ratios) improves large-k Pass@k while staying comparable at small k. Mechanism statements in §3 are design motivations, not derivations that redefine the evaluation targets: w_resp and r_var are chosen scalars in the GRPO-style objective; Pass@k, Distinct-2, Self-BLEU, entropy, and path uniqueness are measured on held-out benchmarks and training rollouts after optimization. Ablations (Table 3) and external baselines (GRPO, ASPO, DAPO, 80/20) provide independent grounding rather than fitted-input-as-prediction. Citations are to prior GRPO/RLVR literature and diversity analyses by other groups; there is no self-citation uniqueness theorem or ansatz chain that forces the result. Minor definitional content (Eqs. 2–5, variance ratio) does not make the reported gains tautological. Honest finding: no circular reduction of the central claim to its inputs.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 1 invented entities

Load-bearing content is mostly standard RL/PPO practice plus modeling choices for open-ended LLM rollouts. No new physical entities. Free parameters are ordinary training hyperparameters and one ReCo-specific clip. Domain assumptions include verifiable binary rewards and Pass@k as a capacity/coverage proxy.

free parameters (3)
  • response-weight clip max (wresp ≤ 10) = 10
    Ad hoc stability cap on inverse expected-count weights; triggered in 1.7% of steps. Ablated in Table 5 but still a hand-chosen threshold the default method uses.
  • PPO/GRPO training hyperparameters (lr, G, ε, β_KL, batch sizes, epochs) = lr=1e-6, G=8, ε=0.2, β=0.001, etc.
    Shared with GRPO baseline; not fitted to the Pass@k claim but required for the reported runs (Appendix Table 4).
  • evaluation sampling (temperature 0.6, top-p 0.95, n for Pass@k estimator) = T=0.6, top-p=0.95, n=1024/128
    Chosen following prior work; affects measured Pass@k curves though held fixed across methods.
axioms (5)
  • domain assumption Group-relative advantages and PPO-style clipped importance-weighted policy gradients are valid RLVR updates for sequence-level binary rewards.
    Inherited from GRPO/PPO setup in Section 2; ReCo modifies weights inside this framework rather than re-deriving RL correctness.
  • ad hoc to paper Expected occurrence E[Nq(o)] = G πθold(o|q) can be proxied by length-normalized likelihood for open-ended responses where exact duplicates are rare.
    Section 3.2 explicitly substitutes ¯π for raw π to avoid length bias; this is a modeling choice, not a theorem.
  • ad hoc to paper Bernoulli variance p(1−p) is the right scalar for how much a token decision still affects future path diversity.
    Motivated in Section 3.3 and supported by ablation vs pθ/pold and pold/pθ, but not derived from a diversity objective.
  • domain assumption Large-k Pass@k (and related diversity metrics) measures retention/expansion of reasoning-path coverage relative to the base model.
    Framing taken from Yue et al. and related citations; underpins the paper’s interpretation of success.
  • standard math Standard arithmetic and expectation identities used to write GRPO/ReCo gradients.
    Sections 2–3.
invented entities (1)
  • ReCo (response-level expected-count weight + token-level variance ratio) no independent evidence
    purpose: Reallocate GRPO update mass away from frequent trajectories and saturated tokens.
    Named method introduced in Section 3; defined entirely operationally via wresp and rvar. Not a latent physical entity; independent_evidence is the empirical suite, not an external predicted observable.

reviewed 2026-07-30 · how reviews work

0 comments
Cite this review

Pith. "Pith review of ReCo: Reweighting GRPO Against Distributional Concentration." pith.science (2026). https://pith.science/paper/BVSY6CPC

@misc{pith2026260726862,
  author       = {Pith},
  title        = {Pith review of: ReCo: Reweighting GRPO Against Distributional Concentration},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BVSY6CPC}},
  note         = {Machine review of arXiv:2607.26862}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Group Relative Policy Optimization (GRPO) has become a standard reinforcement learning method for post-training language models. Recent work shows that GRPO can reduce the base model's reasoning capacity and underperform it in Pass@k when k is large, indicating reduced coverage of reasoning paths. We find that this reduction is associated with GRPO concentrating on responses that the base model already generates with high probability. We trace this concentration to two mechanisms in the GRPO update. At the response level, high-probability responses dominate the group gradient through repeated occurrence. At the token level, GRPO's importance ratio scales gradients, further reinforcing tokens that become more likely under the current policy. We propose ReCo, a reweighting method that addresses both effects. Response contributions are normalized by their expected occurrence within the rollout group, and the token-level importance ratio is replaced with a variance-based ratio that gives larger update scale to non-saturated decision points where alternative token choices remain plausible. Across Qwen2.5-Math-1.5B/7B and Llama-3.1-8B-Instruct on five mathematical reasoning benchmarks, ReCo improves Pass@k for large values of k and is comparable to GRPO for small values of k.

Figures

Figures reproduced from arXiv: 2607.26862 by Junoh Park, Junseo Hwang, Taesup Kim, Wonguk Cho.

Figure 1
Figure 1. Figure 1: Kernel density estimation (KDE) of base [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Pass@k scaling curves on five mathematical reasoning benchmarks for Qwen2.5-Math-7B. Each panel shows Base, GRPO, and ReCo (Ours) across different sampling budgets k. Distinct-2 (Li et al., 2016), and Self-BLEU (Pap￾ineni et al., 2002). This analysis examines whether the policy continues to produce diverse rollouts on the training prompts as reward improves [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Training dynamics of training reward, Distinct-2, and Self-BLEU over training steps on Qwen2.5-Math [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Mean token-level Bernoulli variance pt(1−pt) by relative position within each response on Qwen2.5- Math-1.5B. Error bars show standard error across re￾sponses. Higher values indicate that the sampled token has more room for alternative choices. toward more deterministic choices. The late seg￾ment, which corresponds to answer-confirmation steps, naturally saturates under both methods [PITH_FULL_IMAGE:figur… view at source ↗
Figure 5
Figure 5. Figure 5: Prompt format used for Qwen-based models. [PITH_FULL_IMAGE:figures/full_fig_p011_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Pass@k scaling curves on five mathematical reasoning benchmarks for Qwen2.5-Math-1.5B. Each panel shows Base, GRPO, and ReCo (Ours) across different sampling budgets k [PITH_FULL_IMAGE:figures/full_fig_p012_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Pass@k scaling curves on five mathematical reasoning benchmarks for Llama-3.1-8B-Instruct. Each panel shows Base, GRPO, and ReCo (Ours) across different sampling budgets k. ports that RL training on this backbone can un￾derperform the base model under Pass@k evalua￾tion (Yue et al., 2025; Zhu et al., 2025; Wang et al., 2025b; Yeo et al., 2025), GRPO falls well below the base model across all benchmarks. Re… view at source ↗
Figure 8
Figure 8. Figure 8: Reasoning path uniqueness across training [PITH_FULL_IMAGE:figures/full_fig_p012_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Case 1, GRPO rollouts. All 8 rollouts apply Apollonius’s theorem with minor textual variants. [PITH_FULL_IMAGE:figures/full_fig_p015_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Case 1, ReCo rollouts using Strategy A (Apollonius’s theorem). [PITH_FULL_IMAGE:figures/full_fig_p016_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Case 1, ReCo rollouts using Strategy B (median-length formula). This form is algebraically equivalent to [PITH_FULL_IMAGE:figures/full_fig_p017_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Case 1, ReCo rollouts using Strategy C (coordinate geometry). The rollout uses a symmetric placement [PITH_FULL_IMAGE:figures/full_fig_p018_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Case 2, GRPO correct rollouts. All 4 correct rollouts apply the remainder theorem. [PITH_FULL_IMAGE:figures/full_fig_p019_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Case 2, GRPO incorrect rollouts. All 4 incorrect rollouts attempt the same strategy and fail at execution. [PITH_FULL_IMAGE:figures/full_fig_p020_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Case 2, ReCo correct rollouts. The 6 correct rollouts span two distinct strategies. [PITH_FULL_IMAGE:figures/full_fig_p021_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: Case 2, ReCo incorrect rollouts. R2 fails at execution within the substitution path (Strategy B), and R7 [PITH_FULL_IMAGE:figures/full_fig_p021_16.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

31 extracted references · 7 linked inside Pith

  1. [1]

    arXiv preprint arXiv:2412.16720 , year=

    OpenAI o1 System Card , author=. arXiv preprint arXiv:2412.16720 , year=

    Pith/arXiv arXiv

  2. [2]

    arXiv preprint arXiv:2402.03300 , year=

    DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models , author=. arXiv preprint arXiv:2402.03300 , year=

    Pith/arXiv arXiv

  3. [3]

    arXiv preprint arXiv:2501.12948 , year=

    DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning , author=. arXiv preprint arXiv:2501.12948 , year=

    Pith/arXiv arXiv

  4. [4]

    arXiv preprint arXiv:2503.14476 , year=

    DAPO: An Open-Source LLM Reinforcement Learning System at Scale , author=. arXiv preprint arXiv:2503.14476 , year=

    Pith/arXiv arXiv

  5. [5]

    arXiv preprint arXiv:2503.20783 , year=

    Understanding R1-Zero-Like Training: A Critical Perspective , author=. arXiv preprint arXiv:2503.20783 , year=

    Pith/arXiv arXiv

  6. [6]

    arXiv preprint arXiv:2504.14286 , year=

    SRPO: A Cross-Domain Implementation of Large-Scale Reinforcement Learning on LLM , author=. arXiv preprint arXiv:2504.14286 , year=

    Pith/arXiv arXiv

  7. [7]

    arXiv preprint arXiv:2504.13837 , year=

    Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model? , author=. arXiv preprint arXiv:2504.13837 , year=

    Pith/arXiv arXiv

  8. [8]

    2026 , eprint=

    The Invisible Leash: Why RLVR May or May Not Escape Its Origin , author=. 2026 , eprint=

    2026

  9. [9]

    Scaling Self-Improving Foundation Models without Human Supervision , year=

    Assessing diversity collapse in reasoning , author=. Scaling Self-Improving Foundation Models without Human Supervision , year=

  10. [10]

    2025 , eprint=

    Jointly Reinforcing Diversity and Quality in Language Model Generations , author=. 2025 , eprint=

    2025

  11. [11]

    2025 , eprint=

    Diversity-Aware Policy Optimization for Large Language Model Reasoning , author=. 2025 , eprint=

    2025

  12. [12]

    arXiv preprint arXiv:2505.09655 , year=

    DRA-GRPO: Exploring Diversity-Aware Reward Adjustment for R1-Zero-Like Training of Large Language Models , author=. arXiv preprint arXiv:2505.09655 , year=

    arXiv

  13. [13]

    2025 , eprint=

    Rewarding the Unlikely: Lifting GRPO Beyond Distribution Sharpening , author=. 2025 , eprint=

    2025

  14. [14]

    arXiv preprint arXiv:2509.07430 , year=

    The Choice of Divergence: A Neglected Key to Mitigating Diversity Collapse in Reinforcement Learning with Verifiable Reward , author=. arXiv preprint arXiv:2509.07430 , year=

    arXiv

  15. [15]

    2024 , eprint=

    Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement , author=. 2024 , eprint=

    2024

  16. [16]

    2024 , eprint=

    The Llama 3 Herd of Models , author=. 2024 , eprint=

    2024

  17. [17]

    2025 , eprint=

    OctoThinker: Mid-training Incentivizes Reinforcement Learning Scaling , author=. 2025 , eprint=

    2025

  18. [18]

    2025 , eprint=

    Demystifying Long Chain-of-Thought Reasoning in LLMs , author=. 2025 , eprint=

    2025

  19. [19]

    2021 , eprint=

    Evaluating Large Language Models Trained on Code , author=. 2021 , eprint=

    2021

  20. [20]

    2021 , eprint=

    Measuring Mathematical Problem Solving With the MATH Dataset , author=. 2021 , eprint=

    2021

  21. [21]

    HybridFlow: A Flexible and Efficient RLHF Framework , url=

    Sheng, Guangming and Zhang, Chi and Ye, Zilingfeng and Wu, Xibin and Zhang, Wang and Zhang, Ru and Peng, Yanghua and Lin, Haibin and Wu, Chuan , year=. HybridFlow: A Flexible and Efficient RLHF Framework , url=. doi:10.1145/3689031.3696075 , booktitle=

    arXiv

  22. [22]

    2025 , eprint=

    The Surprising Effectiveness of Negative Reinforcement in LLM Reasoning , author=. 2025 , eprint=

    2025

  23. [23]

    ACL , year=

    Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems , author=. ACL , year=

  24. [24]

    B leu: a Method for Automatic Evaluation of Machine Translation

    Papineni, Kishore and Roukos, Salim and Ward, Todd and Zhu, Wei-Jing. B leu: a Method for Automatic Evaluation of Machine Translation. Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics. 2002. doi:10.3115/1073083.1073135

    arXiv 2002

  25. [25]

    2024 , eprint=

    Progress or Regress? Self-Improvement Reversal in Post-training , author=. 2024 , eprint=

    2024

  26. [26]

    2026 , eprint=

    When Importance Sampling Misallocates Credit: Asymmetric Ratios for Outcome-Supervised RL , author=. 2026 , eprint=

    2026

  27. [27]

    2025 , eprint=

    ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models , author=. 2025 , eprint=

    2025

  28. [28]

    2025 , eprint=

    SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild , author=. 2025 , eprint=

    2025

  29. [29]

    Advances in Neural Information Processing Systems , volume=

    The first few tokens are all you need: An efficient and effective unsupervised prefix fine-tuning method for reasoning models , author=. Advances in Neural Information Processing Systems , volume=

  30. [30]

    Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: human language technologies , pages=

    A diversity-promoting objective function for neural conversation models , author=. Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: human language technologies , pages=

    2016

  31. [31]

    2025 , eprint=

    Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoning , author=. 2025 , eprint=

    2025

This paper was first reviewed by grok-4.5 on July 30, 2026.