Pith. sign in

REVIEW 4 major objections 4 minor 8 cited by

Reinforcement Learning with Rubric Anchors

T0 review · 4 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Rubric-based rewards extend reinforcement learning from verifiable tasks to open-ended writing and reasoning, and a 30B-parameter model trained on about 5,000 samples beats a 671B-parameter model on open-ended benchmarks.

desk verdict The abstract makes a significant, plausible claim about rubric-based RLVR for open-ended tasks, but the supplied full text is corrupted, so this is an abstract-level note: send it to review, but only after the full text and reward-validity evidence are actually inspectable. read the letter →

arxiv 2508.12790 v1 pith:LLK7TYZX submitted 2025-08-18 cs.AI cs.CLcs.LG

Zenan Huang , Yihong Zhuang , Guoshan Lu , Zeyu Qin , Haokai Xu , Tianyu Zhao , Ru Peng , Jiaqi Hu
show 13 more authors
Zhanming Shen Xiaomeng Hu Xijun Gu Peiyi Tu Jiaxin Liu Wenyu Chen Yuzhuo Fu Zhiting Fan Yanmei Gu Yuanyuan Wang Zhengkai Yang Jianguo Li Junbo Zhao
This is my paper · ORCID
classification cs.AIcs.CLcs.LG
keywords RLVRrubric-basedrewardsopen-endedtaskslargelanguagemodelsrewarddesignstylecontrolhuman-LLMrubricconstruction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Reinforcement learning from verifiable rewards (RLVR) has worked for tasks with objectively checkable answers, such as code that passes tests or math with a right answer, but it has not transferred to open-ended writing and reasoning. The paper's central claim is that replacing verifiable signals with rubric-based automatic scoring, built at a scale of more than 10,000 rubrics, extends the same RL machinery to subjective tasks. On only 5,000-plus training samples, the resulting Qwen-30B-A3B model improves by 5.2% on open-ended benchmarks, especially humanities, and outperforms the 671B DeepSeek-V3 model by 2.4% while preserving general and reasoning abilities. Rubrics also act as anchors that steer style, producing less 'AI-like' and more human, expressive responses. If the claim holds, verifiability is no longer the bottleneck that confines RL training of language models.

What carries the argument

The load-bearing machinery is the rubric-based reward: a rubric is a set of structured criteria that a model can interpret, each marking out what a good subjective response contains, and the criteria are converted into an automatic score used as the RL reward. The paper ties this reward to 'rubric anchors', rubrics that steer not only quality but also style, so that training reinforces human-like phrasing rather than only factual correctness. The scale of over 10,000 rubrics, from human, LLM, and hybrid sources, is what turns a single scoring rubric into a general reward signal usable across diverse open-ended tasks.

What would settle it

Measure whether the model's gains on rubric scores coincide with gains in human preference judgments on held-out open-ended responses, and train a control model with shuffled rubric rewards; the central claim fails if benchmark gains appear without human preference gains or appear with shuffled rewards.

Watch

Extended reading notes

Core claim

The paper claims that rubrics, defined as structured, model-interpretable criteria for scoring subjective outputs, can serve as the reward in reinforcement learning for open-ended tasks, removing the need for a mechanically verifiable answer. It reports building the largest such rubric reward system to date, with over 10,000 rubrics sourced from humans, LLMs, or a hybrid of both, and a training framework that makes rubric-based RL stable. The resulting open-sourced model, Qwen-30B-A3B, trained on just over five thousand samples, gains 5.2% on open-ended benchmarks, especially humanities, outperforms the much larger DeepSeek-V3 671B model by 2.4%, and preserves general and reasoning performance. The paper further claims that rubrics can be used as 'anchors' for fine-grained stylistic control, reducing the generic AI-like tone and producing more human-like, expressive responses.

Load-bearing premise

The paper's central claim rests on the assumption that automated scoring with a rubric actually measures the quality human readers care about, rather than just matching the rubric's wording or the benchmark's answer key.

Editorial extensions

If this is right

  • Open-ended domains such as essay writing, summarization, and creative reasoning become trainable by the same RLVR recipe that worked for code and math.
  • A 30B-parameter model can outperform a 671B-parameter model on open-ended benchmarks, suggesting rubric-guided RL is a data- and compute-efficient route to strong subjective-task performance.
  • Because rubrics are interpretable, users can inspect and alter the criteria, making reward design debuggable rather than a black-box reward model.
  • Stylistic control can be achieved through the reward itself, so output tone is a training objective rather than a post-hoc prompt adjustment.
  • General and reasoning abilities are preserved, indicating that rubric rewards do not trade away core skills for benchmark gains.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: the most direct test of the mechanism is whether rubric-score gains transfer to human preference; if they do not, the reported benchmark gains may partly reflect overlap between rubric criteria and benchmark evaluation metrics.
  • Inference: the rubric-anchor idea suggests a general recipe: define any measurable stylistic or quality dimension as a rubric and optimize it, which could enable controllable RL for voice, tone, formatting, and safety.
  • Inference: rubric quality and coverage likely matter more than raw count, so comparing the same model trained on a curated subset versus a random subset of rubrics would isolate the value of scale.
  • Inference: hybrid human-LLM rubric construction points toward scalable RLVR where an LLM proposes criteria and humans validate a sample, rather than hand-writing every reward rule.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes extending Reinforcement Learning from Verifiable Rewards (RLVR) to open-ended tasks by replacing verifiable outcome checkers with rubric-based rewards. The authors report constructing, to their knowledge, the largest rubric reward system to date, with over 10,000 rubrics sourced from humans, LLMs, or hybrid human-LLM collaboration. They train an open-sourced Qwen-30B-A3B model and claim that with only 5K+ samples it improves by +5.2% on open-ended benchmarks (especially humanities), outperforms a 671B DeepSeek-V3 model by +2.4%, and provides fine-grained stylistic control that reduces 'AI-like' tone. The supplied full text is corrupted and largely unreadable, so the review is necessarily based on the abstract and on the fragments that can be recovered; no experimental details, tables, or derivations are verifiable from the submitted material.

Significance. If the claims hold, this is a meaningful contribution: it broadens RLVR to subjective, open-ended domains; it introduces a large-scale rubric resource and an open-sourced model; and it proposes a concrete mechanism for stylistic control via 'rubric anchors.' The open release of the model and the 10,000-rubric system is a tangible strength. However, the central quantitative claims (+5.2% and +2.4%) are asserted in the abstract without statistical support, baseline details, or an evaluation protocol, and the submitted manuscript text does not permit independent verification. The circularity risk between rubric-based training rewards and rubric-like benchmark evaluation is real and must be addressed with concrete evidence, such as correlation with human judgments. The paper is promising but not yet verifiable in its current form.

major comments (4)
  1. [Full text] The submitted manuscript text is corrupted and unreadable beyond the abstract; equations, tables, and any Sections 4-5 content cannot be inspected. This blocks verification of the training framework, the construction of the 10,000+ rubrics, the experimental setup, and the reported gains. Please provide a clean, decodable version before a full review can be completed.
  2. [Abstract] The central claims '+5.2% on open-ended benchmarks' and '+2.4% over 671B DeepSeek-V3' are presented without error bars, number of evaluation instances, number of seeds, or significance tests. Given that the training set is only 5K+ samples, it is important to report variance across seeds and to specify which benchmarks constitute 'open-ended' and 'humanities.'
  3. [Abstract (reward validity)] The load-bearing premise is that rubric-based automatic scoring reflects genuine output quality. The abstract does not provide evidence that rubric scores correlate with human preferences or with held-out quality judgments. Please report a rubric-vs-human correlation study on a held-out set, and include an independent human evaluation of the trained model's outputs to rule out the alternative explanation that the gain is driven by overlap between the training reward and the benchmark evaluator.
  4. [Abstract (data contamination)] With only 5K+ training samples and open-ended benchmarks, data leakage is a serious concern. The abstract does not state whether the training prompts, rubric instances, or their evaluator prompts overlap with the benchmark test sets. Please clarify the separation between training data, rubric construction data, and evaluation data, and report a contamination check.
minor comments (4)
  1. [Abstract] The phrase '5K+ samples' is ambiguous: specify whether this refers to training prompts, generated rollouts, or rubric instances.
  2. [Abstract] The statement 'rubrics serve as structured, model-interpretable criteria' would benefit from a precise definition of rubric structure and how the score is computed from the rubric components.
  3. [Abstract] The claim of 'more human-like, expressive responses' should be supported with a concrete evaluation rubric or an inter-annotator agreement measure, rather than relying on the authors' stylistic description.
  4. [Abstract] Please provide a breakdown of the 10,000+ rubrics by source (human, LLM, hybrid) and report quality-control measures used during rubric construction.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity demonstrated; the abstract-level reward/benchmark overlap concern is not supported by quotable evidence in the supplied text.

full rationale

The paper's derivation chain is: define rubric-based rewards for subjective outputs, train a policy with RLVR, and report gains on open-ended benchmarks. For this to be circular, the benchmark evaluation would need to be the same rubric-based scoring used as the training reward. The supplied abstract and the readable portions of the full text do not state or imply that the external benchmarks are scored with the training rubrics; no equation or procedure equates the benchmark metric to the reward. The only cleanly quotable text is the abstract, which defines rubrics as scoring criteria but does not claim the benchmarks use those rubrics. There is also evidence of a human-evaluation component in the body ('conduct a human study'), which would provide an external check. The concern raised by the reader, that open-ended benchmarks may share criteria with the rubrics, is a plausible correctness risk but not an exhibited reduction; flagging it as circularity would require speculation. Therefore no circular step is identified, and the score is 0.

Assumptions & free parameters 1 free parameters · 3 assumptions · 1 invented entities

The central claim rests on the rubric reward system being a valid training signal and on the benchmark evaluations being unbiased. The rubric system itself is not released, and the training details are absent from the abstract, leaving these as unverified assumptions.

free parameters (1)
  • RL training hyperparameters
    The abstract does not report them, yet they determine the result.
assumptions (3)
  • domain assumption Rubric-based automated scoring is a valid proxy for human judgment of open-ended output quality.
    The entire training signal depends on rubric scores indicating true quality; if rubrics miss key human preferences, RL optimizes the wrong objective.
  • domain assumption The open-ended benchmarks used for evaluation measure the intended capability and are not contaminated by the training rubric data.
    If benchmark prompts or rubrics overlap with training rubrics, the reported gains would be inflated.
  • domain assumption The base model and the 671B DeepSeek-V3 comparison are directly comparable under identical evaluation conditions.
    The headline +2.4% comparison requires matched settings, which cannot be verified from the abstract.
invented entities (1)
  • Rubric reward system (10,000+ rubrics)
    purpose: Used as the reward signal for RL training on open-ended tasks.
    The abstract does not state that the rubrics are released; only the trained model is open-sourced. Without the rubrics or scoring code, the reward signal cannot be independently applied or validated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Reinforcement Learning with Rubric Anchors." pith.science (2026). https://pith.science/paper/LLK7TYZX

@misc{pith2026250812790,
  author       = {Pith},
  title        = {Pith review of: Reinforcement Learning with Rubric Anchors},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LLK7TYZX}},
  note         = {Machine review of arXiv:2508.12790}
}
read the original abstract

Reinforcement Learning from Verifiable Rewards (RLVR) has emerged as a powerful paradigm for enhancing Large Language Models (LLMs), exemplified by the success of OpenAI's o-series. In RLVR, rewards are derived from verifiable signals-such as passing unit tests in code generation or matching correct answers in mathematical reasoning. While effective, this requirement largely confines RLVR to domains with automatically checkable outcomes. To overcome this, we extend the RLVR paradigm to open-ended tasks by integrating rubric-based rewards, where carefully designed rubrics serve as structured, model-interpretable criteria for automatic scoring of subjective outputs. We construct, to our knowledge, the largest rubric reward system to date, with over 10,000 rubrics from humans, LLMs, or a hybrid human-LLM collaboration. Implementing rubric-based RL is challenging; we tackle these issues with a clear framework and present an open-sourced Qwen-30B-A3B model with notable gains: 1) With only 5K+ samples, our system improves by +5.2% on open-ended benchmarks (especially humanities), outperforming a 671B DeepSeek-V3 model by +2.4%, while preserving general and reasoning abilities. 2) Our method provides fine-grained stylistic control, using rubrics as anchors to mitigate the "AI-like" tone and produce more human-like, expressive responses. We share key lessons in rubric construction, data selection, and training, and discuss limitations and future releases.

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 8 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Self-Preference Bias in Rubric-Based Evaluation of Large Language Models

    cs.CL 2026-04 unverdicted novelty 7.0 of 10

    Self-preference bias persists in rubric-based LLM evaluation even with fully objective, programmatically verifiable rubrics, and can shift subjective medical-chat scores by up to ~10 points.

  2. Rubric Dropout: A Simple Way to Mitigate Reward Hacking in Rubric-as-Reward RL

    cs.LG 2026-08 conditional novelty 6.0 of 10

    Randomly dropping a fraction of rubric criteria during GRPO training mitigates the out-of-distribution quality decay caused by optimizing a fixed rubric reward.

  3. LEAP: Lean Environment-Feedback via Adaptive Pruning for Code RL in GPU Kernel Generation

    cs.LG 2026-08 conditional novelty 6.0 of 10

    LEAP combines difficulty-conditioned pruning with pairwise rank-based rewards to improve multi-turn reinforcement learning for CUDA kernel generation.

  4. ARCO: Adaptive Rubrics with Co-Evolution for Multi-Step LLM-Based Agents

    cs.AI 2026-06 unverdicted novelty 6.0 of 10

    ARCO introduces a co-evolving rubric model with generation and scoring heads plus a trajectory decomposition constraint that improves exact-match scores on multi-hop QA tasks over outcome, rubric, and process reward b...

  5. Verbalizing LLM's Higher-order Uncertainty via Imprecise Probabilities

    cs.AI 2026-03 conditional novelty 6.0 of 10

    Prompting LLMs to report imprecise probability intervals (lower/upper confidence) instead of a single point value yields higher-order uncertainty scores that track prediction error and question ambiguity more coherent...

  6. RubricRL: Simple Generalizable Rewards for Text-to-Image Generation

    cs.CV 2025-11 conditional novelty 6.0 of 10

    Using an LLM to generate prompt-specific visual rubrics and grade each criterion independently gives a more interpretable reward that improves text-to-image model alignment beyond composite and learned scalar rewards.

  7. DeepTravel: An End-to-End Agentic Reinforcement Learning Framework for Autonomous Travel Planning Agents

    cs.AI 2025-09 conditional novelty 6.0 of 10

    An RL framework with a travel sandbox, verifier-based rewards, and failure replay produces a deployed travel-planning agent that outperforms frontier LLMs on internal benchmarks.

  8. Baichuan-M2: Scaling Medical Capability with Large Verifier System

    cs.LG 2025-09 conditional novelty 5.0 of 10

    Baichuan-M2, a 32B medical LLM trained with a patient simulator and a clinical rubric generator as RL verifiers, reports state-of-the-art HealthBench scores (60.1 overall, 34.7 hard), ahead of all open-source models.

Reference graph

Works this paper leans on

43 extracted references · 10 canonical work pages · cited by 8 Pith papers

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Healthbench: Evaluating large language models towards improved human health

    Rahul K Arora, Jason Wei, Rebecca Soskin Hicks, Preston Bowman, Joaquin Qui \ n onero-Candela, Foivos Tsimpourlas, Michael Sharman, Meghan Shah, Andrea Vallone, Alex Beutel, et al. Healthbench: Evaluating large language models towards improved human health. arXiv preprint arXiv:2505.08775, 2025

    arXiv 2025

  3. [3]

    Constitutional ai: Harmlessness from ai feedback

    Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. Constitutional ai: Harmlessness from ai feedback. arXiv preprint arXiv:2212.08073, 2022

    arXiv 2022

  4. [4]

    Tombench: Benchmarking theory of mind in large language models, 2024

    Zhuang Chen, Jincenzi Wu, Jinfeng Zhou, Bosi Wen, Guanqun Bi, Gongyao Jiang, Yaru Cao, Mengting Hu, Yunghwei Lai, Zexuan Xiong, and Minlie Huang. Tombench: Benchmarking theory of mind in large language models, 2024

    work page 2024

  5. [5]

    Gemini models, 2025

    Google DeepMind. Gemini models, 2025. URL https://deepmind.google/models/gemini/

    work page 2025

  6. [6]

    Deepseek-v3 technical report, 2025

    DeepSeek-AI, Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report, 2025. URL https://arxiv.org/abs/2412.19437

    arXiv 2025

  7. [7]

    Areal: A large-scale asynchronous reinforcement learning system for language reasoning, 2025

    Wei Fu, Jiaxuan Gao, Xujie Shen, Chen Zhu, Zhiyu Mei, Chuyi He, Shusheng Xu, Guo Wei, Jun Mei, Jiashu Wang, Tongkai Yang, Binhang Yuan, and Yi Wu. Areal: A large-scale asynchronous reinforcement learning system for language reasoning, 2025. URL https://arxiv.org/abs/2505.24298

    arXiv 2025

  8. [8]

    Deliberative alignment: Reasoning enables safer language models

    Melody Y Guan, Manas Joglekar, Eric Wallace, Saachi Jain, Boaz Barak, Alec Helyar, Rachel Dias, Andrea Vallone, Hongyu Ren, Jason Wei, et al. Deliberative alignment: Reasoning enables safer language models. arXiv preprint arXiv:2412.16339, 2024

    arXiv 2024

Show all 43 references
  1. [9]

    Rubrics as rewards: Reinforcement learning beyond verifiable domains

    Anisha Gunjal, Anthony Wang, Elaine Lau, Vaskar Nath, Bing Liu, and Sean Hendryx. Rubrics as rewards: Reinforcement learning beyond verifiable domains. arXiv preprint arXiv:2507.17746, 2025

    2025 arXiv

  2. [10]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    2025 arXiv

  3. [11]

    Measuring massive multitask language understanding, 2021 a

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding, 2021 a . URL https://arxiv.org/abs/2009.03300

    2021 arXiv

  4. [12]

    Measuring mathematical problem solving with the math dataset

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874, 2021 b

    2021 arXiv

  5. [13]

    Open r1: A fully open reproduction of deepseek-r1, January 2025

    Hugging Face . Open r1: A fully open reproduction of deepseek-r1, January 2025. URL https://github.com/huggingface/open-r1

    2025

  6. [14]

    Livecodebench: Holistic and contamination free evaluation of large language models for code, 2024

    Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. Livecodebench: Holistic and contamination free evaluation of large language models for code, 2024. URL https://arxiv.org/abs/2403.07974

    2024 arXiv

  7. [15]

    How many instructions can llms follow at once?, 2025

    Daniel Jaroslawicz, Brendan Whiting, Parth Shah, and Karime Maamari. How many instructions can llms follow at once?, 2025. URL https://arxiv.org/abs/2507.11538

    2025 arXiv

  8. [16]

    Search-r1: Training llms to reason and leverage search engines with reinforcement learning

    Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. Search-r1: Training llms to reason and leverage search engines with reinforcement learning. arXiv preprint arXiv:2503.09516, 2025

    2025 arXiv

  9. [17]

    Tulu 3: Pushing frontiers in open language model post-training

    Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, et al. Tulu 3: Pushing frontiers in open language model post-training. arXiv preprint arXiv:2411.15124, 2024

    2024 arXiv

  10. [18]

    Omni-think: Scaling cross-domain generalization in llms via multi-task rl with hybrid rewards

    Derek Li, Jiaming Zhou, Amirreza Kazemi, Qianyi Sun, Abbas Ghaddar, Mohammad Ali Alomrani, Liheng Ma, Yu Luo, Dong Li, Feng Wen, et al. Omni-think: Scaling cross-domain generalization in llms via multi-task rl with hybrid rewards. arXiv preprint arXiv:2507.14783, 2025

    2025

  11. [19]

    Deepcoder: A fully open-source 14b coder at o3-mini level, 2025 a

    Michael Luo, Sijun Tan, Roy Huang, Ameen Patel, Alpay Ariyak, Qingyang Wu, Xiaoxiang Shi, Rachel Xin, Colin Cai, Maurice Weber, Ce Zhang, Li Erran Li, Raluca Ada Popa, and Ion Stoica. Deepcoder: A fully open-source 14b coder at o3-mini level, 2025 a . Notion Blog

    2025

  12. [20]

    Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Li Erran Li, Raluca Ada Popa, and Ion Stoica

    Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Y. Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Li Erran Li, Raluca Ada Popa, and Ion Stoica. Deepscaler: Surpassing o1-preview with a 1.5b model by scaling rl, 2025 b . Notion Blog

    2025

  13. [21]

    Aime 2024

    Math-AI. Aime 2024. https://huggingface.co/datasets/math-ai/aime24, 2024. URL https://huggingface.co/datasets/math-ai/aime24

    2024

  14. [22]

    Aime 2025

    Math-AI. Aime 2025. https://huggingface.co/datasets/math-ai/aime25, 2025

    2025

  15. [23]

    Llm critics help catch llm bugs, 2024

    Nat McAleese, Rai Michael Pokorny, Juan Felipe Ceron Uribe, Evgenia Nitishinskaya, Maja Trebacz, and Jan Leike. Llm critics help catch llm bugs, 2024. URL https://arxiv.org/abs/2407.00215

    2024 arXiv

  16. [24]

    Rule based rewards for language model safety

    Tong Mu, Alec Helyar, Johannes Heidecke, Joshua Achiam, Andrea Vallone, Ian Kivlichan, Molly Lin, Alex Beutel, John Schulman, and Lilian Weng. Rule based rewards for language model safety. Advances in Neural Information Processing Systems, 37: 0 108877--108901, 2024

    2024

  17. [25]

    Learning to reason with llms, 2024

    OpenAI. Learning to reason with llms, 2024. URL https://openai.com/index/learning-to-reason-with-llms/

    2024

  18. [26]

    Introducing openai o3 and o4-mini, 2025

    OpenAI. Introducing openai o3 and o4-mini, 2025. URL https://openai.com/index/introducing-o3-and-o4-mini/

    2025

  19. [27]

    Samuel J. Paech. Eq-bench: An emotional intelligence benchmark for large language models, 2024. URL https://arxiv.org/abs/2312.06281

    2024 arXiv

  20. [28]

    Siva Reddy, Danqi Chen, and Christopher D. Manning. Coqa: A conversational question answering challenge, 2019. URL https://arxiv.org/abs/1808.07042

    2019 arXiv

  21. [29]

    David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. Gpqa: A graduate-level google-proof q&a benchmark, 2023. URL https://arxiv.org/abs/2311.12022

    2023 arXiv

  22. [30]

    Socialiqa: Commonsense reasoning about social interactions, 2019

    Maarten Sap, Hannah Rashkin, Derek Chen, Ronan LeBras, and Yejin Choi. Socialiqa: Commonsense reasoning about social interactions, 2019. URL https://arxiv.org/abs/1904.09728

    2019 arXiv

  23. [31]

    Self-critiquing models for assisting human evaluators, 2022

    William Saunders, Catherine Yeh, Jeff Wu, Steven Bills, Long Ouyang, Jonathan Ward, and Jan Leike. Self-critiquing models for assisting human evaluators, 2022. URL https://arxiv.org/abs/2206.05802

    2022 arXiv

  24. [32]

    A simple and effective approach to the story cloze test, 2018

    Siddarth Srinivasan, Richa Arora, and Mark Riedl. A simple and effective approach to the story cloze test, 2018. URL https://arxiv.org/abs/1803.05547

    2018 arXiv

  25. [33]

    Salmon: Self-alignment with instructable reward models

    Zhiqing Sun, Yikang Shen, Hongxin Zhang, Qinhong Zhou, Zhenfang Chen, David Daniel Cox, Yiming Yang, and Chuang Gan. Salmon: Self-alignment with instructable reward models. In The Twelfth International Conference on Learning Representations, 2023

    2023

  26. [34]

    Glm-4.5: Agentic, reasoning, and coding (arc) foundation models, 2025 a

    GLM-4.5 Team, Aohan Zeng, Xin Lv, Qinkai Zheng, Zhenyu Hou, Bin Chen, Chengxing Xie, Cunxiang Wang, Da Yin, Hao Zeng, Jiajie Zhang, et al. Glm-4.5: Agentic, reasoning, and coding (arc) foundation models, 2025 a . URL https://arxiv.org/abs/2508.06471

    2025 arXiv

  27. [35]

    Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi k1. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501.12599, 2025 b

    2025 arXiv

  28. [36]

    Checklists are better than reward models for aligning language models

    Vijay Viswanathan, Yanchao Sun, Shuang Ma, Xiang Kong, Meng Cao, Graham Neubig, and Tongshuang Wu. Checklists are better than reward models for aligning language models. arXiv preprint arXiv:2507.18624, 2025

    2025

  29. [37]

    Safety reasoning with guidelines

    Haoyu Wang, Zeyu Qin, Li Shen, Xueqian Wang, Dacheng Tao, and Minhao Cheng. Safety reasoning with guidelines. In Forty-second International Conference on Machine Learning, 2025

    2025

  30. [38]

    Writingbench: A comprehensive benchmark for generative writing

    Yuning Wu, Jiahao Mei, Ming Yan, Chenliang Li, Shaopeng Lai, Yuran Ren, Zijia Wang, Ji Zhang, Mengyue Wu, Qin Jin, et al. Writingbench: A comprehensive benchmark for generative writing. arXiv preprint arXiv:2503.05244, 2025

    2025

  31. [39]

    Logic-rl: Unleashing llm reasoning with rule-based reinforcement learning

    Tian Xie, Zitian Gao, Qingnan Ren, Haoming Luo, Yuqian Hong, Bryan Dai, Joey Zhou, Kai Qiu, Zhirong Wu, and Chong Luo. Logic-rl: Unleashing llm reasoning with rule-based reinforcement learning. arXiv preprint arXiv:2502.14768, 2025

    2025 arXiv

  32. [40]

    Qwen3 technical report

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025

    2025 arXiv

  33. [41]

    Hanjie, Runzhe Yang, and Karthik Narasimhan

    Shunyu Yao, Howard Chen, Austin W. Hanjie, Runzhe Yang, and Karthik Narasimhan. Collie: Systematic construction of constrained text generation tasks, 2023. URL https://arxiv.org/abs/2307.08689

    2023 arXiv

  34. [42]

    Hellaswag: Can a machine really finish your sentence?, 2019

    Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can a machine really finish your sentence?, 2019. URL https://arxiv.org/abs/1905.07830

    2019 arXiv

  35. [43]

    Instruction-following evaluation for large language models, 2023

    Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. Instruction-following evaluation for large language models, 2023. URL https://arxiv.org/abs/2311.07911

    2023 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.