Pith. sign in

REVIEW 5 major objections 6 minor 1 cited by

Cultivating Helpful, Personalized, and Creative AI Tutors: A Framework for Pedagogical Alignment using Reinforcement Learning

T0 review · 5 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A three-dimensional reward model plus group-relative policy optimization shifts a large language model toward more helpful, personalized, and creativity-stimulating tutoring while preserving general capability.

desk verdict EduAlign is a plausible RL pipeline for steering tutors toward helpful, personalized, creative responses, but the reported gains rest mostly on LLM labels and LLM judges; the human-agreement numbers are too weak to carry the headline claim. read the letter →

arxiv 2507.20335 v1 pith:EEYERHPX submitted 2025-07-27 cs.LG cs.AI

Siyu Song , Wentao Liu , Ye Lu , Ruohua Zhang , Tao Liu , Jinze Lv , Xinyun Wang , Aimin Zhou
show 3 more authors
Fei Tan Bo Jiang Hao Hao
This is my paper · ORCID
classification cs.LGcs.AI
keywords largelanguagemodelsreinforcementlearningfromhumanfeedbackrewardGRPOAIineducationpersonalizedtutoringcreativitystimulationpedagogicalalignment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

EduAlign claims that a large language model can be steered toward better tutoring behavior—more helpful, more personalized, and more creativity-stimulating—without sacrificing general ability, by training a dedicated reward model and then fine-tuning the policy with reinforcement learning. The paper builds HPC-RM, a reward model that scores responses along the three dimensions of Helpfulness, Personalization, and Creativity, trained on 8k educational question-answer pairs annotated by human experts and by LLMs. HPC-RM's weighted scalar reward is used with Group Relative Policy Optimization (GRPO) to fine-tune Qwen2.5-72B-Instruct on 2k prompts. On 100 educational scenarios and three public benchmarks, the fine-tuned model scores higher on all three dimensions, while general benchmarks stay roughly flat. If correct, the contribution is a scalable three-dimensional reward recipe for pedagogical alignment that does not cost general competence.

What carries the argument

The central mechanism is HPC-RM, a generative reward model built on Qwen2.5-32B-Base and trained by supervised fine-tuning to output a score triple $(S_h, S_p, S_c)$ for a given question-answer pair. A weighted sum $R(x,y)=w_h S_h + w_p S_p + w_c S_c$ collapses the triple into one scalar reward, and GRPO optimizes the policy against that reward subject to a KL penalty that keeps the model close to its original distribution. The design intent is to give the policy a graded, multi-dimensional signal rather than a single correctness score, so that the model can learn trade-offs among being helpful, personalized, and creativity-stimulating.

What would settle it

Have a new panel of human educators independently score the same 100 pre- and post-training responses on the three dimensions; if their ratings do not show the reported advantage for the fine-tuned model, or if the LLM-annotated HPC-RM does not beat the human-annotated model on that shared human test set, the central claim collapses.

Watch

Extended reading notes

Core claim

The paper's central claim is that rewarding an LLM on three separate pedagogical dimensions—helpfulness (promoting positive, ethical, socially responsible content), personalization (adapting to a learner's profile and needs), and creativity (stimulating original thinking)—produces a measurably better AI tutor. HPC-RM is trained to output the score triple $(S_h,S_p,S_c)$, and the scalar reward $R(x,y)=w_h S_h + w_p S_p + w_c S_c$ is maximized under GRPO with a KL penalty to the original policy. The authors report significant gains across all three HPC dimensions on a curated 100-prompt educational set and on Edu-Values, PersonaMem, and MathTutorBench, with general benchmarks essentially unchanged. This is presented as evidence that multi-dimensional reward modeling captures pedagogical quality that single correctness-oriented rewards miss, and that RL fine-tuning can generalize these traits beyond the training distribution.

Load-bearing premise

The pipeline rests on the assumption that the LLM-generated scores used to train HPC-RM reflect genuine pedagogical quality as human educators would judge it, rather than the annotation model's stylistic preferences.

Editorial extensions

If this is right

  • Any open-weight chat model can be nudged toward pedagogical behavior by training a comparable reward model and running GRPO, without needing a new architecture.
  • Replacing the scoring rubric allows the same pipeline to target other educational goals, such as curiosity, metacognition, or social-emotional support.
  • Because general benchmarks barely move, the method addresses a common concern that RL specialization degrades general reasoning and instruction following.
  • The improvements on Edu-Values, PersonaMem, and MathTutorBench suggest the aligned behavior transfers to tasks and domains beyond the 8k training pairs.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct head-to-head on a shared human-labeled test set would clarify whether the LLM-generated annotations used to train HPC-RM capture human pedagogical judgment or only the annotation model's stylistic preferences; the paper's own numbers leave this open.
  • The same triple-reward setup could be extended to predict downstream student outcomes, such as learning gain, persistence, or self-reported understanding, turning the reward model from an expert-rating proxy into an outcome predictor.
  • The weighted-sum formulation invites per-learner or per-task reward weights, so the framework could adapt how much the tutor prioritizes creativity for an exploratory lesson versus helpful clarity for a struggling student.
  • The 0-2 scoring granularity and the single weighted scalar may obscure trade-offs between dimensions; a Pareto-style analysis of the three scores before aggregation would reveal whether gains in one dimension come at the expense of another.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes EduAlign, a two-stage framework to align LLMs with pedagogical goals. In the first stage, the authors curate 8k educational Q&A pairs and annotate them along three dimensions (Helpfulness, Personalization, Creativity) using both human experts and LLM-based automatic annotation, then train a multi-dimensional reward model HPC-RM (based on Qwen2.5-32B) to predict these scores. In the second stage, HPC-RM is used as the reward signal in GRPO to fine-tune Qwen2.5-72B-Instruct on 2k educational prompts. The authors evaluate the pre- and post-training models on a self-constructed 100-prompt set judged by external LLMs (Gemini-2.5-Pro, DeepSeek-V3, DeepSeek-R1), on three public benchmarks (Edu-Values, PersonaMem, MathTutorBench), and on general capability benchmarks. The paper reports improvements on HPC dimensions and stable general capabilities, concluding that RL with the multi-dimensional reward model yields pedagogically better and not less capable tutors.

Significance. If the central claim holds, EduAlign would be a practical and scalable recipe for injecting multi-dimensional pedagogical objectives into LLM alignment, going beyond the correctness/relevance focus of previous educational RLHF work. The development of a reward model explicitly targeting helpfulness, personalization, and creativity, and the release of HPC-RM, are useful contributions. The paper also makes a sensible methodological choice by using GRPO, which avoids critic training and stabilizes RL for LLMs. The public-benchmark evidence (Table 2) and general-capability checks (Figure 3) are welcome. However, the significance of the contributions is currently constrained by an unvalidated reward signal and by evaluation methodology that shares the same rubric as the training signal, so the headline improvement is not yet established as genuine pedagogical improvement rather than rubric mimicry.

major comments (5)
  1. [Table 1, Section 4.2.1] Table 1 shows that the HPC-RM trained on human annotations attains ACC=0.62 and Pearson=0.43 on the helpfulness dimension, while the LLM-annotation-trained RM that is used for RL is never evaluated against a human-labeled test set. Because the main claim of the paper is that RL improves pedagogical helpfulness, the validity of the reward signal is load-bearing. The text's dismissal of the human-annotated model's weaker performance as 'inconsistencies in annotation standards or subjectivity' is not sufficient; this is precisely the construct the framework aims to align, and the paper should either validate the LLM-annotated RM against a shared human test set or temper the claim in the Introduction that HPC-RM achieves 'highly consistent scores with human experts.'
  2. [Section 3.1, Figure 1, Section 4.2.2, Figure 2] There is a circularity concern: the external LLM judges in Figure 2 are given scoring criteria derived from the same rubric used to train HPC-RM and to create the LLM annotations (Figure 1). The policy is optimized to maximize HPC-RM scores, and the evaluation judges use essentially the same dimensions and criteria. Therefore the reported before/after gains in Figure 2 may reflect adherence to this shared rubric rather than genuine pedagogical improvement. To break this circularity, the paper should include human expert ratings of the pre- and post-training responses, or use an independent evaluation protocol with a different rubric, or explicitly discuss this as a limitation and provide corroborating human evidence.
  3. [Section 4.2.2, Figure 2] The 100-prompt evaluation set used for the main HPC improvement claims is self-constructed and not released, and Figure 2 reports no error bars, confidence intervals, or significance tests. The reader cannot assess whether the improvements are consistent across prompts or driven by a few outliers. The authors should release the evaluation set (or a representative subset) and report per-prompt variance with paired statistical tests.
  4. [Table 2, Section 4.2.2] The paper states that the fine-tuned model achieves 'state-of-the-art performance' on dedicated benchmarks, but Table 2 only compares the model before and after RL training, with no comparison against existing educational models or baselines. The improvements are also modest (e.g., Edu-Values 4.10 to 4.29, PersonaMem PF 56.54 to 58.06) and no significance tests are reported. The claim of state-of-the-art performance is unsupported and should be removed or substantiated with baselines and statistical testing.
  5. [Abstract, Section 5] The abstract and conclusion claim 'significantly improved alignment' and state that HPC-RM demonstrated 'strong consistency with human evaluations.' These statements are contradicted by Table 1, where the human-annotation-trained reward model has ACC=0.62 and Pearson=0.43 for helpfulness, and by the absence of human evaluation of the final RL-finetuned model. The claims should be scaled back to match the evidence, or additional human validation should be added.
minor comments (6)
  1. [Author affiliations] There is a typo in the affiliation: 'Shanghai Innavation Institute' should be 'Shanghai Innovation Institute.'
  2. [Section 3.1, Figure 1] The grading prompt in Figure 1 is entirely in Chinese, which is a usability issue for the international readership of an English-language paper; an English translation should be provided.
  3. [Equation (3)] The reward weights w_h, w_p, w_c are described as 'predefined' but their actual values are never reported, nor is any sensitivity analysis given. Please provide the exact weights used in the experiments.
  4. [Equation (4)] The KL regularization coefficient beta is not reported in the experimental setup. Please provide the value of beta and any tuning procedure.
  5. [Section 4.1, RL setup] The RL training details are sparse: only learning rate 5e-6 and one epoch are given, with no mention of batch size, number of GPUs, training time, or computational cost. These details are important for reproducibility.
  6. [References] Several references are incomplete, missing venue or year information (e.g., [10], [11], [24], [25]). Please standardize the bibliography.

Circularity Check

1 steps flagged · score 5.0 of 10

Internal HPC-RM evaluation is circular, but external benchmarks keep the central claim partially independent.

  1. fitted input called prediction [Section 4.1, 'Reinforcement Learning' paragraph, evaluation protocol]
    "To evaluate the effectiveness of RL training, we construct a representative evaluation set consisting of 100 real-world educational scenarios. We compare the pre-trained and RL-enhanced models by automatically scoring their outputs using HPC-RM, thereby quantifying performance improvements along the three target dimensions."

    The policy is trained by GRPO to maximize the scalar reward R(x,y)=w_h·S_h+w_p·S_p+w_c·S_c, where S_h, S_p, S_c are HPC-RM scores (Eqs. 2-3). The same HPC-RM is then used as the measurement instrument for the claimed improvement. Thus the improvement reported in this evaluation is a measure of how successfully the optimizer maximized its own training objective, not an independent assessment of pedagogical quality. Even on a disjoint prompt set, the evaluation is performed by the fitted model whose scores defined the training signal, so the 'prediction' of improvement is forced by the optimization setup rather than by an external criterion.

full rationale

The clearest circular step is the internal evaluation of RL effectiveness with HPC-RM itself. Because HPC-RM supplies the reward signal in Eq. (3) and Eq. (4), measuring pre/post improvement with HPC-RM reports how well the policy optimized its own training objective. This is a by-construction reduction: the evaluation instrument is the fitted model that defined the training signal. The circularity is partial, not total, because the paper also offers independent evidence: external LLM judges in Figure 2, public benchmarks (Edu-Values, PersonaMem, MathTutorBench) in Table 2, and general-capability benchmarks in Figure 3. Those are not fitted in this paper, although the external LLM judges share the same HPC rubric. A related validity gap, not itself a formal equation-level circularity, is that the LLM-annotation-trained HPC-RM used for RL is validated only against LLM annotations; the human-anchored RM shows lower agreement (ACC 0.62 on helpfulness in Table 1), and no human-labeled test is reported for the reward model actually used in RL. Overall, one supporting evaluation reduces by construction, but the central claim retains independent empirical content, so the circularity score is moderate.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim depends on three unreported reward-weight parameters, the KL coefficient, and a set of domain assumptions about the reliability of LLM judges, synthetic student profiles, and the sufficiency of the three HPC dimensions. No new physical or conceptual entities are posited.

free parameters (2)
  • Reward weights w_h, w_p, w_c = Not reported
    Equation (3) aggregates the three reward scores with a weighted sum. The paper says the weights are 'predefined' but never gives their values, which directly controls the RL objective and the final policy behavior.
  • KL regularization coefficient beta = Not reported
    Equation (4) includes a beta term penalizing divergence from the reference policy. Its value is never specified, and it materially affects the trade-off between reward maximization and policy stability.
assumptions (4)
  • domain assumption LLM-generated annotations are reliable proxies for human pedagogical judgments
    HPC-RM is trained primarily on LLM-generated scores. The paper's Table 1 shows a model trained on human annotations has much lower agreement, and no shared human-labeled test set is used to validate the LLM-annotated model's alignment with human judgment.
  • domain assumption Synthetic Q&A pairs and simulated student profiles are representative of real educational interactions
    The 8k dataset is generated by models using simulated students and teacher roles. Real student queries, error patterns, and emotional states may differ, so the reward signal may not transfer to actual tutoring contexts.
  • domain assumption External LLM judges (Gemini-2.5-Pro, DeepSeek-V3, DeepSeek-R1) provide valid scores of pedagogical quality
    The main evaluation in Figure 2 relies on these three models scoring outputs on 0-10 scales. Their agreement with human experts is not reported, so the validity of the reported gains depends on the unverified validity of these judges.
  • domain assumption The three dimensions (helpfulness, personalization, creativity) sufficiently capture pedagogical alignment
    The framework reduces pedagogical quality to these three rubric dimensions, which are expert-defined but not justified as a complete or orthogonal set. Other factors, such as factual correctness, clarity, or emotional support, are either folded into the dimensions or ignored.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Cultivating Helpful, Personalized, and Creative AI Tutors: A Framework for Pedagogical Alignment using Reinforcement Learning." pith.science (2026). https://pith.science/paper/EEYERHPX

@misc{pith2026250720335,
  author       = {Pith},
  title        = {Pith review of: Cultivating Helpful, Personalized, and Creative AI Tutors: A Framework for Pedagogical Alignment using Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EEYERHPX}},
  note         = {Machine review of arXiv:2507.20335}
}
read the original abstract

The integration of large language models (LLMs) into education presents unprecedented opportunities for scalable personalized learning. However, standard LLMs often function as generic information providers, lacking alignment with fundamental pedagogical principles such as helpfulness, student-centered personalization, and creativity cultivation. To bridge this gap, we propose EduAlign, a novel framework designed to guide LLMs toward becoming more effective and responsible educational assistants. EduAlign consists of two main stages. In the first stage, we curate a dataset of 8k educational interactions and annotate them-both manually and automatically-along three key educational dimensions: Helpfulness, Personalization, and Creativity (HPC). These annotations are used to train HPC-RM, a multi-dimensional reward model capable of accurately scoring LLM outputs according to these educational principles. We further evaluate the consistency and reliability of this reward model. In the second stage, we leverage HPC-RM as a reward signal to fine-tune a pre-trained LLM using Group Relative Policy Optimization (GRPO) on a set of 2k diverse prompts. We then assess the pre- and post-finetuning models on both educational and general-domain benchmarks across the three HPC dimensions. Experimental results demonstrate that the fine-tuned model exhibits significantly improved alignment with pedagogical helpfulness, personalization, and creativity stimulation. This study presents a scalable and effective approach to aligning LLMs with nuanced and desirable educational traits, paving the way for the development of more engaging, pedagogically aligned AI tutors.

Figures

Figures reproduced from arXiv: 2507.20335 by the authors.

Figure 1
Figure 1. Prompt template for assessing helpfulness, personalization, and creativity [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Result of HPC Evaluation As illustrated in [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. ELBench: A Multi-Dimensional Benchmark for Education-Facing Large Language Models

    cs.CL 2026-08 conditional novelty 6.0 of 10

    ELBench evaluates nine LLMs on capability, safety, teaching, and cultivation in one protocol, finding module profiles diverge, safety trades off with teaching, and all models share a style-over-fit judgment blind spot.

Reference graph

Works this paper leans on

45 extracted references · 23 canonical work pages · cited by 1 Pith paper

  1. [1]

    D. R. Thomas, C. Borchers, S. Bhushan, E. Gatz, S. Gupta, K. R. Koedinger, Llm-generated feedback sup- ports learning if learners choose to use it, arXiv preprint arXiv:2506.17006 (2025)

    work page Pith review arXiv 2025

  2. [2]

    Grassucci, G

    E. Grassucci, G. Grassucci, A. Uncini, D. Comminiello, Beyond answers: How llms can pursue strategic thinking in education, arXiv preprint arXiv:2504.04815 (2025)

    arXiv 2025

  3. [3]

    L. J. Jacobsen, K. E. Weber, The promises and pitfalls of large language models as feedback providers: A study of prompt engineering and the quality of ai-driven feedback, AI 6 (2) (2025) 35

    work page 2025

  4. [4]

    Seßler, A

    K. Seßler, A. Bewersdorff, C. Nerdel, E. Kasneci, To- wards adaptive feedback with ai: Comparing the feedback quality of llms and teachers on experimentation protocols, arXiv preprint arXiv:2502.12842 (2025)

    arXiv 2025

  5. [5]

    Sonkar, K

    S. Sonkar, K. Ni, S. Chaudhary, R. G. Baraniuk, Pedagog- ical alignment of large language models, arXiv preprint arXiv:2402.05000 (2024)

    arXiv 2024

  6. [6]

    Giannakos, R

    M. Giannakos, R. Azevedo, P. Brusilovsky, M. Cukurova, Y . Dimitriadis, D. Hernandez-Leo, S. Järvelä, M. Mavrikis, B. Rienties, The promise and challenges of generative ai in education, Behaviour & Information Technology (2024) 1–27

    work page 2024

  7. [7]

    Mulyani, M

    H. Mulyani, M. A. Istiaq, E. R. Shauki, F. Kurniati, H. Ar- linda, Transforming education: exploring the influence of generative ai on teaching performance, Cogent Education 12 (1) (2025) 2448066

    work page 2025

  8. [8]

    Scarlatos, D

    A. Scarlatos, D. Smith, S. Woodhead, A. Lan, Improv- ing the validity of automatically generated feedback via reinforcement learning, in: International Conference on Artificial Intelligence in Education, Springer, 2024, pp. 280–294

    work page 2024

Show all 45 references
  1. [9]

    Zhong, W

    J. Zhong, W. Shen, Y . Li, S. Gao, H. Lu, Y . Chen, Y . Zhang, W. Zhou, J. Gu, L. Zou, A comprehen- sive survey of reward models: Taxonomy, applications, challenges, and future, arXiv preprint arXiv:2504.12328 (2025)

    2025 arXiv

  2. [10]

    Choshen, L

    L. Choshen, L. Fox, Z. Aizenbud, O. Abend, On the weak- nesses of reinforcement learning for neural machine trans- lation, in: International Conference on Learning Repre- sentations

  3. [11]

    Engstrom, A

    L. Engstrom, A. Ilyas, S. Santurkar, D. Tsipras, F. Janoos, L. Rudolph, A. Madry, Implementation matters in deep policy gradients: A case study on ppo and trpo, in: Inter- national Conference on Learning Representations, 2020

    2020

  4. [12]

    Rafailov, A

    R. Rafailov, A. Sharma, E. Mitchell, C. D. Manning, S. Ermon, C. Finn, Direct preference optimization: Your language model is secretly a reward model, Advances in Neural Information Processing Systems 36 (2023) 53728– 53741

    2023

  5. [13]

    D. Guo, D. Yang, H. Zhang, J. Song, R. Zhang, R. Xu, Q. Zhu, S. Ma, P. Wang, X. Bi, et al., Deepseek-r1: In- centivizing reasoning capability in llms via reinforcement learning, arXiv preprint arXiv:2501.12948 (2025)

    2025 arXiv

  6. [14]

    Ouyang, J

    L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, J. Schulman, J. Hilton, F. Kelton, L. Miller, M. Simens, A. Askell, P. Welinder, P. F. Christiano, J. Leike, R. Lowe, Training language models to follow instructions with...

    2022

  7. [15]

    P. F. Christiano, J. Leike, T. Brown, M. Martic, S. Legg, D. Amodei, Deep reinforcement learning from human preferences, in: I. Guyon, U. V . Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, R. Garnett (Eds.), Advances in Neural Information Processing Sys- tems, V o...

    2017

  8. [16]

    Y . Bai, A. Jones, K. Ndousse, A. Askell, A. Chen, N. Das- Sarma, D. Drain, S. Fort, D. Ganguli, T. Henighan, et al., Training a helpful and harmless assistant with rein- forcement learning from human feedback, arXiv preprint arXiv:2204.05862 (2022)

    2022 arXiv

  9. [17]

    Z. Li, T. Xu, Y . Zhang, Z. Lin, Y . Yu, R. Sun, Z.-Q. Luo, Remax: a simple, effective, and efficient reinforcement learning method for aligning large language models, in: Proceedings of the 41st International Conference on Ma- chine Learning, 2024, pp. 29128–29163

    2024

  10. [18]

    H. Dong, W. Xiong, D. Goyal, Y . Zhang, W. Chow, R. Pan, S. Diao, J. Zhang, S. KaShun, T. Zhang, Raft: Re- ward ranked finetuning for generative foundation model alignment, Transactions on Machine Learning Research

  11. [19]

    Q. Yu, Z. Zhang, R. Zhu, Y . Yuan, X. Zuo, Y . Yue, W. Dai, T. Fan, G. Liu, L. Liu, et al., Dapo: An open-source llm reinforcement learning system at scale, arXiv preprint arXiv:2503.14476 (2025). 7

    2025 arXiv

  12. [20]

    Scarlatos, N

    A. Scarlatos, N. Liu, J. Lee, R. Baraniuk, A. Lan, Training llm-based tutors to improve student learning outcomes in dialogues, arXiv preprint arXiv:2503.06424 (2025)

    2025 arXiv

  13. [21]

    Dinucu-Jianu, J

    D. Dinucu-Jianu, J. Macina, N. Daheim, I. Hakimi, I. Gurevych, M. Sachan, From problem-solving to teach- ing problem-solving: Aligning llms with pedagogy using reinforcement learning, arXiv preprint arXiv:2505.15607 (2025)

    2025

  14. [22]

    Lamsiyah, A

    S. Lamsiyah, A. El Mahdaouy, A. Nourbakhsh, C. Schom- mer, Fine-tuning a large language model with reinforce- ment learning for educational question generation, in: In- ternational Conference on Artificial Intelligence in Edu- cation, Springer, 2024, pp. 424–438

    2024

  15. [23]

    Z. Cai, M. Cao, H. Chen, K. Chen, K. Chen, X. Chen, X. Chen, Z. Chen, Z. Chen, P. Chu, et al., Internlm2 tech- nical report, arXiv preprint arXiv:2403.17297 (2024)

    2024 arXiv

  16. [24]

    L. Yuan, G. Cui, H. Wang, N. Ding, X. Wang, B. Shan, Z. Liu, J. Deng, H. Chen, R. Xie, et al., Advancing llm reasoning generalists with preference trees, in: The Thir- teenth International Conference on Learning Representa- tions

  17. [25]

    R. Yang, R. Ding, Y . Lin, H. Zhang, T. Zhang, Regular- izing hidden states enables learning generalizable reward model for llms, in: The Thirty-eighth Annual Conference on Neural Information Processing Systems

  18. [26]

    Adler, N

    B. Adler, N. Agarwal, A. Aithal, D. H. Anh, P. Bhat- tacharya, A. Brundyn, J. Casper, B. Catanzaro, S. Clay, J. Cohen, et al., Nemotron-4 340b technical report, arXiv preprint arXiv:2406.11704 (2024)

    2024 arXiv

  19. [27]

    H. Wang, W. Xiong, T. Xie, H. Zhao, T. Zhang, Inter- pretable preferences via multi-objective reward modeling and mixture-of-experts, in: Findings of the Association for Computational Linguistics: EMNLP 2024, 2024, pp. 10582–10592

    2024

  20. [28]

    Zheng, W.-L

    L. Zheng, W.-L. Chiang, Y . Sheng, S. Zhuang, Z. Wu, Y . Zhuang, Z. Lin, Z. Li, D. Li, E. Xing, et al., Judg- ing llm-as-a-judge with mt-bench and chatbot arena, Ad- vances in Neural Information Processing Systems 36 (2023) 46595–46623

    2023

  21. [29]

    B. Gao, Z. Cai, R. Xu, P. Wang, C. Zheng, R. Lin, K. Lu, D. Liu, C. Zhou, W. Xiao, et al., Llm critics help catch bugs in mathematics: Towards a better mathemati- cal verifier with natural language feedback, arXiv preprint arXiv:2406.14024 (2024)

    2024 arXiv

  22. [30]

    Z. Ye, X. Li, Q. Li, Q. Ai, Y . Zhou, W. Shen, D. Yan, Y . Liu, Beyond scalar reward model: Learn- ing generative judge from preference data, arXiv preprint arXiv:2410.03742 (2024)

    2024 arXiv

  23. [31]

    Y . Zhao, R. Joshi, T. Liu, M. Khalman, M. Saleh, P. J. Liu, Slic-hf: Sequence likelihood calibration with human feedback, arXiv preprint arXiv:2305.10425 (2023)

    2023 arXiv

  24. [32]

    Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y . Li, Y . Wu, et al., Deepseekmath: Pushing the limits of mathematical reasoning in open language mod- els, arXiv preprint arXiv:2402.03300 (2024)

    2024 arXiv

  25. [33]

    Schulman, F

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, O. Klimov, Proximal policy optimization algorithms, arXiv preprint arXiv:1707.06347 (2017)

    2017 arXiv

  26. [34]

    A. Yang, B. Yang, B. Hui, B. Zheng, B. Yu, C. Zhou, C. Li, C. Li, D. Liu, F. Huang, G. Dong, H. Wei, H. Lin, J. Tang, J. Wang, J. Yang, J. Tu, J. Zhang, J. Ma, J. Xu, J. Zhou, J. Bai, J. He, J. Lin, K. Dang, K. Lu, K. Chen, K. Yang, M. Li, M. Xue, N. Ni, P. Zhang, P. Wang, R. ...

    2024 arXiv

  27. [35]

    Team, Qwen2.5: A party of foundation models (September 2024)

    Q. Team, Qwen2.5: A party of foundation models (September 2024). URLhttps://qwenlm.github.io/blog/qwen2.5/

    2024

  28. [36]

    Fawcett, An introduction to roc analysis, Pattern recog- nition letters 27 (8) (2006) 861–874

    T. Fawcett, An introduction to roc analysis, Pattern recog- nition letters 27 (8) (2006) 861–874

    2006

  29. [37]

    Cohen, Y

    I. Cohen, Y . Huang, J. Chen, J. Benesty, J. Benesty, J. Chen, Y . Huang, I. Cohen, Pearson correlation coeffi- cient, Noise reduction in speech processing (2009) 1–4

    2009

  30. [38]

    J. H. Zar, Spearman rank correlation, Encyclopedia of bio- statistics 7 (2005)

    2005

  31. [39]

    Abdi, The kendall rank correlation coefficient, Ency- clopedia of measurement and statistics 2 (2007) 508–510

    H. Abdi, The kendall rank correlation coefficient, Ency- clopedia of measurement and statistics 2 (2007) 508–510

    2007

  32. [40]

    Zhang, Y

    P. Zhang, Y . Zhang, B. Wang, L. Rong, P. Tiwari, J. Qin, Edu-values: Towards evaluating the chinese education values of large language models, in: Companion Proceed- ings of the ACM on Web Conference 2025, 2025, pp. 1519–1523

    2025

  33. [41]

    Jiang, Z

    B. Jiang, Z. Hao, Y .-M. Cho, B. Li, Y . Yuan, S. Chen, L. Ungar, C. J. Taylor, D. Roth, Know me, respond to me: Benchmarking llms for dynamic user profil- ing and personalized responses at scale, arXiv preprint arXiv:2504.14225 (2025)

    2025

  34. [42]

    Macina, N

    J. Macina, N. Daheim, I. Hakimi, M. Kapur, I. Gurevych, M. Sachan, Mathtutorbench: A benchmark for measuring open-ended pedagogical capabilities of llm tutors (2025). arXiv:2502.18940. URLhttps://arxiv.org/abs/2502.18940 8

    2025

  35. [43]

    Y . Wang, X. Ma, G. Zhang, Y . Ni, A. Chandra, S. Guo, W. Ren, A. Arulraj, X. He, Z. Jiang, et al., Mmlu-pro: A more robust and challenging multi-task language un- derstanding benchmark, arXiv preprint arXiv:2406.01574 (2024)

    2024 arXiv

  36. [44]

    Huang, Y

    Y . Huang, Y . Bai, Z. Zhu, J. Zhang, J. Zhang, T. Su, J. Liu, C. Lv, Y . Zhang, J. Lei, Y . Fu, M. Sun, J. He, C-eval: A multi-level multi-discipline chinese evaluation suite for foundation models, in: Advances in Neural Information Processing Systems, 2023

    2023

  37. [45]

    J. Zhou, T. Lu, S. Mishra, S. Brahma, S. Basu, Y . Luan, D. Zhou, L. Hou, Instruction-following evaluation for large language models (2023).arXiv:2311.07911. 9

    2023 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.