Pith. sign in

REVIEW 4 major objections 5 minor 28 references

Group entropy from existing GRPO samples can correct an entropy-dependent bias in per-group advantage normalization and improve multi-task LLM training.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 19:45 UTC pith:ASQSBGW2

load-bearing objection Useful, well-specified GRPO variant with honest math; the empirical claim 'consistently outperforms' is ahead of the evidence (single seeds, thin margins, no code). the 4 major comments →

arxiv 2607.16850 v1 pith:ASQSBGW2 submitted 2026-07-18 cs.CL

Group Entropy-Controlled Policy Optimization

Guangran Cheng , Chengqi Lyu , Songyang Gao , Wenwei Zhang , Kai Chen This is my paper
classification cs.CL
keywords group entropyGRPOadvantage shapingentropy controlmulti-task reinforcement learninglarge language model alignmentexploration-exploitation trade-off
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that in reinforcement learning of large language models on mixtures of heterogeneous tasks, GRPO's per-group reward standardization produces advantages that are not statistically comparable across groups: low-entropy groups get strong, concentrated gradient signals while high-entropy groups get weak, noisy ones. It proposes GEPO, a lightweight modification that uses the group entropy estimated from the K responses already sampled for each prompt to shape advantages asymmetrically — attenuating positive advantages in low-entropy groups and negative advantages in high-entropy groups, with thresholds that adapt from batch statistics over training. If this is right, a drop-in modification with zero extra sampling stabilizes training, preserves task-specific exploration, and improves average performance across thirteen benchmarks on two base models, without task annotations. The stakes: entropy control for LLM alignment can move from global or token-level regulation to group-level regulation, matching the granularity of GRPO's own advantage computation.

Core claim

The central claim is that group entropy — the average per-token negative log-likelihood of the K responses sampled for a prompt — is a reliable diagnostic of a task's exploration state, and that GRPO's within-group normalization converts entropy heterogeneity into a structural optimization bias. The paper supports this with two observations from a first rollout (different domains occupy different entropy regimes, and these regimes coincide with different advantage skewness profiles) and two propositions bounding the discrepancy between policy-weighted and uniform-reference reward distributions and advantages in terms of log N - H(x). On this basis GEPO applies an asymmetric advantage-scaling

What carries the argument

Group entropy ℋg(x), the per-token Monte Carlo estimate of a prompt's sequence-level entropy from the K responses GRPO already samples. It is the signal that makes the intervention task-aware without extra sampling. The intervention itself is a piecewise advantage multiplier: within a group, positive advantages are scaled by α_low when group entropy falls below an adaptive lower threshold, negative advantages by α_high when entropy exceeds an adaptive upper threshold, with α_high < α_low to avoid length collapse; thresholds are set from batch entropy mean and standard deviation and smoothed with an exponential moving average.

Load-bearing premise

The load-bearing premise is that the uniform distribution over responses is the correct reference frame for judging GRPO's advantages, so the gap between policy-weighted and uniform-standardized reward moments is a bias to be corrected — together with the first-rollout observation that high entropy coincides with low group accuracy.

What would settle it

Compute GRPO advantages under a reference measure that is not uniform (e.g., a task prior) and check whether the entropy-dependent bound in Proposition 2.2 still holds or flips; or run GEPO on a synthetic mixture with high-entropy, high-accuracy groups and observe whether attenuating negative advantages still helps or actually degrades performance.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • GRPO's entropy-dependent advantage bias can be mitigated without changing the sampling budget, so the same KL and clipping machinery of GRPO can be retained.
  • GEPO should provide larger gains on task mixtures with wide entropy spreads, and its adaptive thresholds should transfer to new base models without retuning.
  • Preserving differentiated per-task entropy regimes, rather than pushing all tasks toward one target, is a better objective for multi-task alignment.
  • Because the correction is applied on the normalized advantage, it composes with other advantage-shaping or clipping methods.
  • The ablations imply both directions of control matter and asymmetry is necessary: removing high-entropy control hurts most, and symmetric scaling worsens results.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A testable extension the paper leaves implicit: on a task mixture engineered to decouple entropy from accuracy (e.g., easy prompts with high per-token entropy), GEPO's attenuation rule could harm exploration if the low-accuracy/high-entropy coincidence is the real driver; the paper's own Appendix A suggests advantage skewness, not entropy per se, may be the operative variable.
  • The same reference-measure arbitrariness applies to any statistic computed per prompt group — reward variance, token-level covariance, or baseline values — so GEPO's shaping is one instance of a more general group-conditional correction that could be explored.
  • The online adaptive thresholds implicitly assume batch entropy statistics faithfully proxy the policy's true entropy; in a degenerate run the band could tighten as the policy collapses, so a long-horizon stress test would reveal whether the EMA smoothing alone is enough to prevent runaway contraction.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes GEPO, a lightweight extension of GRPO that uses per-prompt group entropy to shape advantages asymmetrically: positive advantages in low-entropy groups are attenuated, negative advantages in high-entropy groups are attenuated, with adaptive thresholds derived from batch entropy statistics and smoothed by EMA. The authors provide propositions (2.1 and 2.2) bounding the difference between policy-weighted and reference reward distributions in terms of (log N - H), and report experiments on Intern-S1-mini and Qwen3.5-9B across thirteen benchmarks, claiming consistent improvements over GRPO, AEPO, Clip-Cov, and KL-Cov, with best averages of 54.2 vs 51.5 and 71.2 vs 70.9.

Significance. If substantiated, GEPO would be an attractive drop-in modification with zero additional sampling cost and a plausible mechanism for balanced multi-task exploration. The appendix contains correct proof steps: the TV/Pinsker bound, the mean/variance/advantage-difference bounds, and the binary-reward skewness calculation; the appendix also honestly reconciles pooled versus single-group skewness. However, the theoretical motivation depends on an arbitrary reference measure, and the experimental evidence is currently too fragile to support the headline claim of 'consistently outperforms.' The idea is potentially significant but requires stronger empirical support and a clearer framing of the theoretical status.

major comments (4)
  1. [§3.2, Table 1] The central empirical claim, 'GEPO consistently outperforms GRPO and recent entropy-controlled methods,' rests on single-seed, best-checkpoint results with no error bars or statistical significance. The margins on Qwen3.5-9B are 0.3 points over Clip-Cov (71.2 vs 70.9) and 0.5 points over GRPO; these are within typical run-to-run variance for LLM RL. Please report means and standard deviations over at least three independent runs, define the best-checkpoint selection protocol, and state whether the average differences are statistically reliable.
  2. [§3.1, Table 1] Clip-Cov and KL-Cov reuse parameters from Qwen2.5-7B (reference [8], §3.1) rather than being tuned for the two base models, whereas GEPO uses its own fixed coefficients (α_high=0.2, α_low=0.5, β_high=0.3, β_low=0.2, γ=0.01, Eqs. 7–9). This confounds method comparison with hyperparameter tuning and can bias the comparison in GEPO's favor. Add a sensitivity analysis for these five coefficients and either tune baselines per base model or report the original parameters and justify their transfer.
  3. [§2.2, Prop. 2.2 and Appendix B] The 'entropy-dependent bias' is defined as the discrepancy between policy-weighted standardization and standardization under a reference measure ν_x; the paper takes ν_x to be uniform over the finite response space. This reference frame is arbitrary: any reference measure would yield an entropy-dependent bound, and GRPO's normalization is by construction based on policy-weighted moments. The paper does not justify why uniform-over-responses is the correct target. Without such a justification, the 'bias' framing is misleading and GEPO's shaping is an unproven heuristic shrinkage of the policy gradient. Please either justify the reference measure or reframe Propositions 2.1–2.2 as characterizing sensitivity to policy reweighting, not as an inherent bias.
  4. [§2.3, Obs. 2 and Eq. (7)] The asymmetric design (α_high < α_low) and the high-entropy intervention are motivated primarily by a single first-rollout observation (Figure 2) and by the undocumented claim that aggressive negative-advantage penalization in low-entropy groups triggers length collapse. This is a load-bearing design choice. Please provide evidence across training or across a range of α values, check response-length dynamics, and report at least a small sensitivity study of the α_high/α_low asymmetry.
minor comments (5)
  1. [§3.2] Typo: 'covaiance-aware constraints' and 'controlling covaiance' should be 'covariance'.
  2. [§3.2] Typo: 'Qwen3,5-9B' should be 'Qwen3.5-9B'.
  3. [Appendix B.1] The reference and policy reward CDFs are defined twice in consecutive paragraphs; keep a single definition.
  4. [Abstract / §2.2] The abstract says 'response groups of each task,' but group entropy is computed per prompt; task-level grouping is used only for visualization. Clarify this distinction to avoid implying task annotations are used in the method.
  5. [Figure 3 caption] The caption says 'Typical validation curves'; specify whether these are from one selected run, how the run was chosen, and whether the curves correspond to the same random seed as Table 1.

Circularity Check

0 steps flagged

No significant circularity; results are genuine measurements, with only a mild in-sample motivational flavor in the design observations.

full rationale

GEPO's central claims are empirical: the method's objective (Eq. 7) and adaptive thresholds (Eqs. 8-9) depend only on current-batch group entropy and fixed hand-set coefficients (alpha_high=0.2, alpha_low=0.5, beta_high=0.3, beta_low=0.2, gamma=0.01). No benchmark score or external evaluation number enters the method equations, so the Table 1 improvements are true measurements rather than values reconstructed from fitted inputs. The theoretical Propositions 2.1-2.2 are self-contained inequalities (Pinsker/TV bounds) that do not assume GEPO's conclusion; the choice of a uniform reference measure is a modeling assumption that can be criticized on correctness grounds, but it does not make the derivation circular by construction. Observations 1-2 in Section 2.2 are used to motivate the direction of advantage shaping, and they come from the first rollout on the same training mixture used later in evaluation; this is an in-sample motivation and a potential data-snooping / generalization concern, but it is not a fitted parameter renamed as a prediction, nor does it force the reported benchmark outcomes. There is no load-bearing self-citation chain: the entropy-control baselines [8], [19] and GRPO [18] are external prior work, and the citation to Intern-S1 [1] supplies only the base model, not the method's validity. Overall, the paper is self-contained against external benchmarks, and the mild overlap between the motivating observations and the evaluation distribution does not rise to circularity under the strict definition requiring a specific reduction of a prediction to its input.

Axiom & Free-Parameter Ledger

5 free parameters · 6 axioms · 0 invented entities

The method's central intervention depends on five hand-set coefficients. None are fit to the evaluation benchmarks, but none are sensitivity-tested either; all five are fixed across both base models in the reported runs. Group entropy itself is not an invented quantity—it is the standard Monte Carlo estimate of sequence entropy from the K=16 rollouts (Eq. 4). The crucial un-argued premise is the uniform reference measure in Props. 2.1-2.2, which converts the policy-vs-uniform gap into a 'bias'. No new physical or algorithmic entities are postulated.

free parameters (5)
  • α_high (high-entropy negative-advantage scale) = 0.2
    Multiplies (and shrinks) negative advantages when group entropy exceeds the upper threshold (§2.3, Eq. 7); hand-set; the strict inequality α_high<α_low is justified by a length-collapse fragility claim, not derived.
  • α_low (low-entropy positive-advantage scale) = 0.5
    Multiplies positive advantages in low-entropy groups to give a 'gentler nudge' against over-exploitation; hand-set (§2.3, Eq. 7).
  • β_high (upper entropy-threshold width) = 0.3
    Sets H_high = μ_H + β_high·σ_H in Eq. 8; hand-set; no sensitivity analysis is provided.
  • β_low (lower entropy-threshold width) = 0.2
    Sets H_low = μ_H − β_low·σ_H in Eq. 8; hand-set; no sensitivity analysis is provided.
  • γ (EMA smoothing coefficient) = 0.01
    Temporal smoothing of the thresholds in Eq. 9; hand-set; stated trade-off between responsiveness and stability.
axioms (6)
  • standard math Pinsker's inequality: TV ≤ sqrt(0.5·KL(π∥ν))
    Invoked directly in Proposition 2.1 (Eq. 5) and cited to Tsybakov [22]; used to convert the entropy gap into a total-variation bound.
  • ad hoc to paper Uniform over the finite response space is the correct reference frame ν_x for advantages
    Propositions 2.1-2.2 (Appendix B) define the 'bias' as the deviation of GRPO's policy-weighted standardization from ν_x-standardization, with ν_x taken uniform. No argument establishes uniform as the right target; other ν_x give different, still entropy-dependent bounds.
  • domain assumption Regularity: |μ_x(a)| ≤ M and σ_π, σ_ν ≥ σ_0
    Assumed in Proposition 2.2 (Appendix B.2) to obtain the constant C; standard but unstated in the main text.
  • domain assumption High-entropy groups coincide with low-accuracy groups in this mixture
    Observation 2 (§2.2) and Appendix A: the entire high-entropy intervention rests on the empirical association between high group entropy and low group accuracy, observed in one first GRPO rollout.
  • ad hoc to paper Aggressive negative-advantage penalization in the low-entropy regime triggers length collapse
    Empirical claim in §2.3 used to fix α_high < α_low; no supporting data or reference is provided.
  • standard math Group responses are i.i.d. samples from π_θ, so Eq. 4 is unbiased (variance O(1/K))
    Used in §2.2 to justify group entropy as an estimator of H(πθ(·|x)); standard Monte Carlo reasoning.

pith-pipeline@v1.3.0-alltime-deepseek · 14513 in / 22687 out tokens · 206903 ms · 2026-08-01T19:45:51.037351+00:00 · methodology

0 comments
read the original abstract

Entropy control has become an effective tool in reinforcement learning (RL) of large language models (LLMs), helping balance exploration-exploitation trade-off during alignment process. Such RL paradigm is often conducted on mixtures of heterogeneous tasks, which induce distinct entropy regimes under the same policy, making global or token-level entropy regulation insufficient to corresponding heterogeneous needs of exploration. This heterogeneity further makes GRPO-style normalized advantages induce an entropy-dependent bias, making advantage signals across prompt groups statistically non-comparable. To address this issue, we propose Group Entropy-Controlled Policy Optimization (GEPO), a lightweight extension to GRPO that uses group entropy, estimated from existing grouped samples to perform entropy-conditioned asymmetric advantage shaping. GEPO attenuates positive advantages in low-entropy groups to reduce over-exploitation, and negative advantages in high-entropy groups to preserve exploration, with adaptive thresholds derived from historical entropy statistics. Extensive experiments on two base models across thirteen benchmarks spanning mathematics, physics, science, code generation, and instruction following show that GEPO consistently outperforms GRPO and recent entropy-controlled methods, delivering balanced cross-task improvements while preserving task-specific exploration levels throughout training.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

28 extracted references · 6 linked inside Pith

  1. [1]

    Intern-s1: A scientific multimodal foundation model, 2025

    Lei Bai, Zhongrui Cai, Maosong Cao, Weihan Cao, Chiyu Chen, Haojiong Chen, Kai Chen, Pengcheng Chen, Ying Chen, Yongkang Chen, Yu Cheng, Yu Cheng, Pei Chu, Tao Chu, Erfei Cui, Ganqu Cui, Long Cui, Ziyun Cui, Nianchen Deng, Ning Ding, Nanqin Dong, Peijie Dong, Shihan Dou, Sinan Du, Haodong Duan, Caihua Fan, Ben Gao, Changjiang Gao, Jianfei Gao, Songyang Ga...

    2025

  2. [2]

    Unifying count-based exploration and intrinsic motivation.Advances in neural information processing systems, 29, 2016

    Marc Bellemare, Sriram Srinivasan, Georg Ostrovski, Tom Schaul, David Saxton, and Remi Munos. Unifying count-based exploration and intrinsic motivation.Advances in neural information processing systems, 29, 2016. 1

    2016

  3. [3]

    Galaz-Montoya, Yuhui Zhang, Yuchang Su, Disha Bhowmik, Zachary Coman, Sarina M

    James Burgess, Jeffrey J Nirschl, Laura Bravo-Sánchez, Alejandro Lozano, Sanket Rajan Gupte, Jesus G. Galaz-Montoya, Yuhui Zhang, Yuchang Su, Disha Bhowmik, Zachary Coman, Sarina M. Hasan, Alexandra Johannesson, William D. Leineweber, Malvika G Nair, Ridhi Yarlagadda, Connor Zuraski, Wah Chiu, Sarah Cohen, Jan N. Hansen, Manuel D Leonetti, Chad Liu, Emma ...

    2025

  4. [4]

    Minimax-m1: Scaling test-time compute efficiently with lightning attention

    Aili Chen, Aonian Li, Bangwei Gong, Binyang Jiang, Bo Fei, Bo Yang, Boji Shan, Changqing Yu, Chao Wang, Cheng Zhu, et al. Minimax-m1: Scaling test-time compute efficiently with lightning attention. arXiv preprint arXiv:2506.13585, 2025. 3.1

    Pith/arXiv arXiv 2025

  5. [5]

    Reasoning with exploration: An entropy perspective

    Daixuan Cheng, Shaohan Huang, Xuekai Zhu, Bo Dai, Xin Zhao, Zhenliang Zhang, and Furu Wei. Reasoning with exploration: An entropy perspective. InProceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 30377–30385, 2026. 1, 4

    2026

  6. [6]

    Lmdeploy: A toolkit for compressing, deploying, and serving llm.https: //github.com/InternLM/lmdeploy, 2023

    LMDeploy Contributors. Lmdeploy: A toolkit for compressing, deploying, and serving llm.https: //github.com/InternLM/lmdeploy, 2023. 3.1

    2023

  7. [7]

    Xtuner: A toolkit for efficiently fine-tuning llm

    XTuner Contributors. Xtuner: A toolkit for efficiently fine-tuning llm. https://github.com/ InternLM/xtuner, 2023. 3.1

    2023

  8. [8]

    The entropy mechanism of reinforcement learning for reasoning language models.arXiv preprint arXiv:2505.22617, 2025

    Ganqu Cui, Yuchen Zhang, Jiacheng Chen, Lifan Yuan, Zhi Wang, Yuxin Zuo, Haozhan Li, Yuchen Fan, Huayu Chen, Weize Chen, et al. The entropy mechanism of reinforcement learning for reasoning language models.arXiv preprint arXiv:2505.22617, 2025. 1, 3.1, 4

    Pith/arXiv arXiv 2025

  9. [9]

    Physics: Benchmarking foundation models on university-level physics problem solving, 2025

    Kaiyue Feng, Yilun Zhao, Yixin Liu, Tianyu Yang, Chen Zhao, John Sous, and Arman Cohan. Physics: Benchmarking foundation models on university-level physics problem solving, 2025. 3.1

    2025

  10. [10]

    Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor

    Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. InInternational conference on machine learning, pages 1861–1870. Pmlr, 2018. 4 11 Group Entropy-Controlled Policy Optimization

    2018

  11. [11]

    Open- reasoner-zero: An open source approach to scaling up reinforcement learning on the base model.arXiv preprint arXiv:2503.24290, 2025

    Jingcheng Hu, Yinmin Zhang, Qi Han, Daxin Jiang, Xiangyu Zhang, and Heung-Yeung Shum. Open- reasoner-zero: An open source approach to scaling up reinforcement learning on the base model.arXiv preprint arXiv:2503.24290, 2025. 1

    Pith/arXiv arXiv 2025

  12. [12]

    Openaio1systemcard.arXivpreprintarXiv:2412.16720,

    Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, AleksanderMadry, AlexBeutel, AlexCarney, etal. Openaio1systemcard.arXivpreprintarXiv:2412.16720,

    Pith/arXiv arXiv

  13. [13]

    Livecodebench: Holistic and contamination free evaluation of large language models for code, 2024

    Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar- Lezama, Koushik Sen, and Ion Stoica. Livecodebench: Holistic and contamination free evaluation of large language models for code, 2024. 3.1

    2024

  14. [14]

    Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts, 2024

    Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts, 2024. 3.1

    2024

  15. [15]

    Chartqapro: A more diverse and challenging benchmark for chart question answering, 2025

    Ahmed Masry, Mohammed Saidul Islam, Mahir Ahmed, Aayush Bajaj, Firoz Kabir, Aaryaman Kartha, Md Tahmid Rahman Laskar, Mizanur Rahman, Shadikur Rahman, Mehrad Shahmohammadi, Megh Thakkar, Md Rizwan Parvez, Enamul Hoque, and Shafiq Joty. Chartqapro: A more diverse and challenging benchmark for chart question answering, 2025. 3.1

    2025

  16. [16]

    Generalizing verifiable instruction following, 2025

    Valentina Pyatkin, Saumya Malik, Victoria Graf, Hamish Ivison, Shengyi Huang, Pradeep Dasigi, Nathan Lambert, and Hannaneh Hajishirzi. Generalizing verifiable instruction following, 2025. 3.1

    2025

  17. [17]

    David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. Gpqa: A graduate-level google-proof q&a benchmark, 2023. 3.1

    2023

  18. [18]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024. 1, 2.1

    Pith/arXiv arXiv 2024

  19. [19]

    Qwenlong-l1

    Weizhou Shen, Ziyi Yang, Chenliang Li, Zhiyuan Lu, Miao Peng, Huashan Sun, Yingcheng Shi, Shengyi Liao, Shaopeng Lai, Bo Zhang, et al. Qwenlong-l1. 5: Post-training recipe for long-context reasoning and memory management.arXiv preprint arXiv:2512.12967, 2025. 1, 3.1

    arXiv 2025

  20. [20]

    MIT press Cambridge, 1998

    Richard S Sutton, Andrew G Barto, et al.Reinforcement learning: An introduction, volume 1. MIT press Cambridge, 1998. 1

    1998

  21. [21]

    Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi k1. 5: Scaling reinforcement learning with llms.arXiv preprint arXiv:2501.12599, 2025. 1

    Pith/arXiv arXiv 2025

  22. [22]

    Tsybakov.Introduction to Nonparametric Estimation

    Alexandre B. Tsybakov.Introduction to Nonparametric Estimation. Springer Series in Statistics. Springer,

  23. [23]

    Dsdr: Dual-scale diversity regularization for exploration in llm reasoning.arXiv preprint arXiv:2602.19895, 2026

    Zhongwei Wan, Yun Shen, Zhihao Dou, Donghao Zhou, Yu Zhang, Xin Wang, Hui Shen, Jing Xiong, Chaofan Tao, Zixuan Zhong, et al. Dsdr: Dual-scale diversity regularization for exploration in llm reasoning.arXiv preprint arXiv:2602.19895, 2026. 4

    arXiv 2026

  24. [24]

    On the entropy dynamics in reinforcement fine-tuning of large language models.arXiv preprint arXiv:2602.03392, 2026

    Shumin Wang, Yuexiang Xie, Wenhao Zhang, Yuchang Sun, Yanxi Chen, Yaliang Li, and Yanyong Zhang. On the entropy dynamics in reinforcement fine-tuning of large language models.arXiv preprint arXiv:2602.03392, 2026. 4

    arXiv 2026

  25. [25]

    Cmphysbench: A benchmark for evaluating large language models in condensed matter physics, 2025

    Weida Wang, Dongchen Huang, Jiatong Li, Tengchao Yang, Ziyang Zheng, Di Zhang, Dong Han, Benteng Chen, Binzhao Luo, Zhiyu Liu, Kunling Liu, Zhiyuan Gao, Shiqi Geng, Wei Ma, Jiaming Su, Xin Li, Shuchen Pu, Yuhan Shui, Qianjia Cheng, Zhihao Dou, Dongfei Cui, Changyong He, Jin Zeng, Zeke Xie, Mao Su, Dongzhan Zhou, Yuqiang Li, Wanli Ouyang, Yunqi Cai, Xi Dai...

    2025

  26. [26]

    Entropic: Towards stable long-term training of llms via entropy stabilization with proportional-integral control.arXiv preprint arXiv:2511.15248, 2025

    Kai Yang, Xin Xu, Yangkun Chen, Weijie Liu, Jiafei Lyu, Zichuan Lin, Deheng Ye, and Saiyong Yang. Entropic: Towards stable long-term training of llms via entropy stabilization with proportional-integral control.arXiv preprint arXiv:2511.15248, 2025. 1, 4

    arXiv 2025

  27. [27]

    Mmmu-pro: A more robust multi-discipline multimodal understanding benchmark, 2025

    Xiang Yue, Tianyu Zheng, Yuansheng Ni, Yubo Wang, Kai Zhang, Shengbang Tong, Yuxuan Sun, Botao Yu, Ge Zhang, Huan Sun, Yu Su, Wenhu Chen, and Graham Neubig. Mmmu-pro: A more robust multi-discipline multimodal understanding benchmark, 2025. 3.1

    2025

  28. [28]

    Scientists’ first exam: Probing cognitive abilities of mllm via perception, understanding, and reasoning, 2025

    Yuhao Zhou, Yiheng Wang, Xuming He, Ao Shen, Ruoyao Xiao, Zhiwei Li, Qiantai Feng, Zijie Guo, Yuejin Yang, Hao Wu, Wenxuan Huang, Jiaqi Wei, Dan Si, Xiuqi Yao, Jia Bu, Haiwen Huang, Manning Wang, Tianfan Fu, Shixiang Tang, Ben Fei, Dongzhan Zhou, Fenghua Ling, Yan Lu, Siqi Sun, Chenhui Li, Guanjie Zheng, Jiancheng Lv, Wenlong Zhang, and Lei Bai. Scientist...

    2025