Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecycle

Dingkang Yang; Hongsheng Li; Jun Liu; Keliang Liu; Lihua Zhang; Peng Zhai; Weijie Yin; Yang Liu; Yuchi Wang; Ziyun Qian

REVIEW 5 major objections 5 minor 3 cited by

Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecycle

T0 review · 5 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This survey systematically maps how reinforcement learning operates across the full large-language-model lifecycle—pre-training, alignment fine-tuning, and reinforced reasoning—and argues that Reinforcement Learning with Verifiable Rewards

desk verdict Broad, current survey with a genuinely useful lifecycle organization, but a wrong GRPO equation and citation errors make it unreliable as-is; needs major revision before it earns a cite. read the letter →

arxiv 2509.16679 v1 pith:VFSZIZUG submitted 2025-09-20 cs.CL

Keliang Liu , Dingkang Yang , Ziyun Qian , Weijie Yin , Yuchi Wang , Hongsheng Li , Jun Liu , Peng Zhai

show 2 more authors

Yang Liu Lihua Zhang

This is my paper · ORCID

classification cs.CL

keywords reinforcementlearninglargelanguagemodelswithverifiablerewardsRLHFGRPOreasoningalignmentLLMlifecycle

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is a survey that tries to establish a complete, lifecycle-wide picture of how reinforcement learning (RL) is used to improve large language models (LLMs). It claims that RL now plays a role in every major phase of model development—pre-training, alignment with human preferences, and the newer 'reinforced reasoning' phase that powers state-of-the-art mathematical and coding performance. The survey's central focus is Reinforcement Learning with Verifiable Rewards (RLVR), where a model is rewarded by automated checks such as unit tests or math-answer verification rather than by human judgments. It also consolidates datasets, benchmarks, and open-source training frameworks into a single reference. A reader would care because the paper offers a practical map of when and how to apply RL, and it surfaces the unresolved question of whether RL truly creates new reasoning abilities or merely amplifies what pre-training already encoded.

What carries the argument

The load-bearing organizational device is the lifecycle taxonomy, which divides RL-for-LLM work into pre-training, alignment, and reinforced reasoning, with a separate branch for RLVR. The central algorithmic object is RLVR—reinforcement learning with verifiable rewards—where the reward signal is produced by an automated oracle (unit tests, math answer keys, formal verification) rather than by a learned human-preference model. The methodological anchor within RLVR is GRPO (group relative policy optimization), which samples several responses per prompt, uses the group-average reward as a baseline in place of a trained value network, and updates the policy to favor above-average responses. The

What would settle it

Take one of the base models listed in Table 1 and train it with the same data and compute but no RL—for example, supervised fine-tuning on the RL-generated correct rollouts—then evaluate on AIME2024. If the non-RL run reproduces the reported gain, the survey's central attribution of the improvement to RL is empirically falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is that reinforcement learning has become a unifying training mechanism across the LLM lifecycle, and that the most consequential recent variant is Reinforcement Learning with Verifiable Rewards (RLVR). In RLVR, reward comes from an objective, automatically checkable source—passing unit tests, matching a known math answer, or satisfying a formal specification—rather than from a learned model of human preference. The paper argues this paradigm, implemented by algorithms in the GRPO family, is what drove the sharp jumps in mathematical and coding reasoning seen in recent models. To support this, it organizes the field into a five-branch taxonomy—pre-training RL, align

Load-bearing premise

The load-bearing premise is that the large benchmark gains in Table 1 are actually caused by reinforcement learning and not by simultaneous changes in data, compute, or supervised fine-tuning; if that attribution fails, the survey's central story loses its foundation.

Editorial extensions

If this is right

For tasks with automated correctness checks (math, code, formal logic), RLVR is the current high-leverage training recipe, so new work in those domains should start from the GRPO-style family.
For alignment, classic RLHF and preference-optimization methods remain the relevant toolkit; the paper's organization implies alignment and reasoning are distinct lifecycle phases needing different reward setups.
The consolidated benchmark list means researchers can evaluate an RL method across a standard set: math competitions, live code, general knowledge, logic, and agentic multi-turn tasks.
The open-source framework collection makes large-scale RL experimentation reproducible enough for practitioners to run without building infrastructure from scratch.
The cited controversy over whether RLVR expands reasoning beyond the base model implies that reported gains should be interpreted with care, especially at small sample sizes (pass@k).

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: the reward-design dimensions the paper surfaces (source, density, dynamics, landscape) read like a checklist for diagnosing failures in RLVR; if a run stagnates, identifying which dimension is shallow—e.g., outcome-only rewards on a long-horizon task—points to the likely fix.
Editorial inference: if the cited evidence that RLVR mostly amplifies base-model sampling holds, then the next bottleneck is not more compute but diversity-preserving exploration; methods that manage entropy or generate new tasks should see disproportionate gains.
Editorial inference: the same lifecycle taxonomy could be turned into a benchmark suite that tests one model across pre-training RL, alignment, and reasoning RL under matched compute, which would directly quantify how much each stage contributes.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

You asked for a read on arXiv:2509.16679. The short version: it's a broad, current survey of RL for LLMs with a sensible lifecycle organization, but it has a load-bearing technical error in the GRPO equation and some citation sloppiness that make it unreliable as a reference until fixed.

What's genuinely useful: the lifecycle framing (pre-training, alignment, reinforced reasoning) is a reasonable way to organize the space, and the compilation of 2024-2025 RLVR work, datasets, benchmarks, and open-source frameworks is convenient. Figure 1's five-branch taxonomy gives a quick map, and the reference list is rich. If you need a starting point for what's happening in reasoning RL, this covers a lot of ground.

The soft spots are real. The stress-test note is right: Equation (5) in §2.1 uses max where GRPO uses mean. The surrounding text even says "intra-group average reward" and defines A_i = R_i - mean(R_group), so the equation contradicts the prose. If you implement from the equation, the best response gets advantage zero and everyone else gets negative advantage, so no response is positively reinforced. That's not cosmetic; for a survey whose selling point is being an accurate reference, this is a serious defect.

The citation issues the reader flagged hold up too. Reference [56] is used for both APPS and MATH, while Table 3 lists APPS with [32] (which is actually StepCoder). And §3.2 attributes the foundational RLHF paper to Christiano et al. [129], but [129] is Ouyang et al.'s InstructGPT. These look mechanical, but in a reference work they corrode trust.

The novelty claim of being the first full-lifecycle survey is asserted rather than demonstrated; Table 2 compares a handful of surveys, mostly from 2024, and the dimensions are coarse. Several 2025 surveys overlap heavily.

That said, the core organizational idea holds up. With the GRPO equation fixed and citations cleaned, this could become a legitimate reference.

Who it's for: practitioners and newcomers wanting a map of the current RL-for-LLM landscape. It doesn't answer a scientific question, but it saves time. Recommendation: send to peer review with expectation of major revision. I wouldn't cite it in its current form.

Referee Report

5 major / 5 minor

Summary. The manuscript is a survey of reinforcement learning applied to large language models across the full LLM training lifecycle: pre-training, alignment, and reasoning, with particular emphasis on Reinforcement Learning with Verifiable Rewards (RLVR). It proposes a taxonomy, reviews algorithmic foundations (REINFORCE, PPO, GRPO), surveys recent RLVR-based methods for reasoning, multimodal reasoning, adaptive thinking, agents, and internal feedback, and consolidates datasets, benchmarks, and open-source training frameworks. The central claim is that this lifecycle-wide synthesis, together with the consolidated resources, provides a practical and comprehensive roadmap for researchers and practitioners working on RL-enhanced LLMs. The paper does not present new experiments or derivations; its contribution is compilation and organization.

Significance. If accurate, the survey is a genuinely useful resource: it covers a fast-moving 2024-2025 literature, organizes methods by lifecycle stage, and includes tables of benchmarks and frameworks that practitioners could use as a starting point. It also surfaces important controversies, such as whether RLVR truly expands reasoning or merely re-weights pretrained behavior, and it discusses training-stability and reward-design issues. However, the value of a survey of this type rests entirely on technical correctness and faithful citation. The manuscript currently contains a load-bearing algorithmic error in the GRPO formula, citation mismatches in the datasets section, and a mis-attributed foundational reference. These issues weaken confidence in the survey's reliability as a reference, although they are local and correctable rather than fatal to the overall structure.

major comments (5)

[§2.1, Eq. (5)] The GRPO advantage formula is misstated as \hat A_{i,t} = (r_i - max_j r_j)/std. The surrounding prose says GRPO uses the intra-group average reward as baseline, and the original DeepSeekMath paper defines A_i = (r_i - mean)/std. With max subtraction, the best response in the group receives zero advantage and all other responses receive negative advantage, so no response is positively reinforced. This is not a cosmetic typo; it inverts the learning signal. Since GRPO is a backbone of the RLVR discussion, this equation must be corrected and the notation (r_i vs R_i) harmonized.
[§5.2, Table 3, ref. [56]] Reference [56] (Hendrycks et al., APPS) is used for both APPS and MATH: the text says 'APPS [56]' and later 'MATH [56] introduced 12,500...'. Table 3 lists APPS with citation [32], which is the StepCoder paper, not APPS. This makes the consolidated resource table unreliable. Please map APPS to the correct APPS reference, MATH to the Hendrycks MATH dataset reference, and align Table 3 with the text.
[§3.2, first paragraph] The sentence 'Christiano et al. [129] established a foundational paradigm for modern LLM alignment' cites reference [129], which is Ouyang et al. (InstructGPT/RLHF). Christiano et al.'s deep RL from human preferences is a different paper and does not appear in the reference list. Please correct the attribution and add the appropriate reference, or rephrase to cite Ouyang et al. directly.
[§5.2, SWE-bench Verified] The text states that SWE-bench Verified contains 50,000 instances collected from 128 GitHub repositories. SWE-bench Verified is a human-validated subset of 500 instances; the 50,000-instance figure corresponds to the synthetic SWE-smith data in reference [207], not to SWE-bench Verified. This is a factual error in a section whose purpose is to consolidate benchmark resources. Please verify all dataset sizes and descriptions.
[Table 1] Table 1 attributes the full performance gains (e.g., DeepSeek-R1 vs DeepSeek-V3, +40.6 on AIME2024) to RL, but DeepSeek-R1 includes a cold-start SFT stage and other pipeline changes, so the comparison is not an RL-only ablation. DeepSeek-R1-Zero is a cleaner RL-only comparison. Please add a caveat or decompose the contributions; otherwise the motivating claim overstates the causal effect of RL.

minor comments (5)

[§4.2 and §4.3] Two distinct methods, 'SRPO [226]' (cross-domain LLM RL) and 'SRPO [164]' (reflection-aware multimodal RL), share the same acronym and are both discussed in the survey. Please disambiguate, e.g., 'cross-domain SRPO' and 'multimodal SRPO', to avoid confusion.
[Table 3] The Math row lists OlympiadBench twice. Remove the duplicate entry.
[§7.1.4] There is a malformed sentence: 'violating user intent or ethical normswang [41, 121, 122, 168]'. A word or punctuation is missing after 'norms'.
[§3.3] The phrase 'Wang, Fu, and Miao et al. [41, 121, 122, 168]' is an awkward grouping; either list the author names separately or write 'several studies'.
[§4.6] Kang et al. [77] is described twice with nearly identical sentences in the same subsection. Consolidate the two descriptions.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper is a literature survey whose claims are supported by external citations and compilation, not by equations or fitted parameters that reduce to its own inputs.

full rationale

This paper is a survey, not a derivation or prediction pipeline. Its central claims—lifecycle coverage, RLVR emphasis, consolidated datasets/benchmarks, and tool summaries—are supported by citing external works and by organizing existing taxonomies. There are no fitted parameters renamed as predictions, no derived quantities equivalent to their assumptions by construction, and no load-bearing uniqueness theorem imported from the authors' prior work. The handful of self-citations (e.g., refs. [20,21,89,111,205,206], which include author-group members) are incidental in-text mentions of the group's own prior papers and are not used to justify the survey's organizational framework or its central conclusions. The skeptical note about Equation (5) misstating GRPO's advantage baseline (max instead of mean) is a technical correctness issue, not a circularity issue: it does not make the survey's claims equivalent to its inputs, nor does it show that any result is assumed in order to be derived. Under the hard rules requiring a quoted reduction or a fitted-input-called-prediction to flag circularity, none of the enumerated patterns is present. The appropriate finding is no significant circularity, score 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

As a survey, the paper adds no free parameters or invented entities. It does rely on the accuracy of external model reports and the representativeness of its literature selection, and it adopts an interpretive stance that RLVR is the pivotal driver of reasoning advances.

assumptions (3)

domain assumption Reported benchmark scores in Tables 1 and 4 are accurate transcriptions of the cited model reports.
The survey's motivation and comparisons rely on these numbers, and no independent re-evaluation is provided. If the numbers are wrong, the central narrative about RL-driven gains weakens.
domain assumption The cited papers are represented faithfully and the taxonomy branch assignments match the methods' primary contributions.
The survey organizes roughly two hundred works into categories. Its value depends on correct categorization and attribution, which the visible citation errors call into question.
domain assumption The 2024-2025 reasoning gains in models such as DeepSeek-R1 and OpenAI o1 are primarily attributable to RL or RLVR rather than to SFT, data scaling, or inference-time compute.
This interpretive premise shapes the entire survey. It is presented as a given rather than derived or critically tested, and the survey itself later cites work questioning whether RLVR adds new reasoning capacity.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecycle." pith.science (2026). https://pith.science/paper/VFSZIZUG

@misc{pith2026250916679,
  author       = {Pith},
  title        = {Pith review of: Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecycle},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VFSZIZUG}},
  note         = {Machine review of arXiv:2509.16679}
}

read the original abstract

In recent years, training methods centered on Reinforcement Learning (RL) have markedly enhanced the reasoning and alignment performance of Large Language Models (LLMs), particularly in understanding human intents, following user instructions, and bolstering inferential strength. Although existing surveys offer overviews of RL augmented LLMs, their scope is often limited, failing to provide a comprehensive summary of how RL operates across the full lifecycle of LLMs. We systematically review the theoretical and practical advancements whereby RL empowers LLMs, especially Reinforcement Learning with Verifiable Rewards (RLVR). First, we briefly introduce the basic theory of RL. Second, we thoroughly detail application strategies for RL across various phases of the LLM lifecycle, including pre-training, alignment fine-tuning, and reinforced reasoning. In particular, we emphasize that RL methods in the reinforced reasoning phase serve as a pivotal driving force for advancing model reasoning to its limits. Next, we collate existing datasets and evaluation benchmarks currently used for RL fine-tuning, spanning human-annotated datasets, AI-assisted preference data, and program-verification-style corpora. Subsequently, we review the mainstream open-source tools and training frameworks available, providing clear practical references for subsequent research. Finally, we analyse the future challenges and trends in the field of RL-enhanced LLMs. This survey aims to present researchers and practitioners with the latest developments and frontier trends at the intersection of RL and LLMs, with the goal of fostering the evolution of LLMs that are more intelligent, generalizable, and secure.

Figures

Figures reproduced from arXiv: 2509.16679 by the authors.

**Figure 1.** A taxonomy of RL enhanced LLMs. This figure presents a taxonomy of the key stages and resources involved in creating [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗

**Figure 2.** Key components in RL-enhanced LLMs. This figure illustrates the key components and their interactions within the lifecycle [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

**Figure 3.** Technical architecture of the RLVR methods. It depicts the overall workflow of the RLVR and expands on the design methods for [PITH_FULL_IMAGE:figures/full_fig_p015_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Spectral Origins of the Self-Correction Blind Spot in Autoregressive Generation
cs.LG 2026-07 conditional novelty 5.5 of 10

Self-correction blind spots in residual-stream autoregressive models arise iff the product of attention Jacobians has spectral radius ≥1, with a sharp marker threshold and RL coupling condition derived from that radius.
Switch-Reasoner: Learn When to Think in Multitask Mixtures via Reinforcement Learning
cs.CV 2026-07 conditional novelty 5.0 of 10

A GRPO framework that treats thinking as a tool call and uses dual-level regulation so multimodal models learn when to reason versus answer directly.
EMO-R3: Reflective Reinforcement Learning for Emotional Reasoning in Multimodal Large Language Models
cs.AI 2026-02 conditional novelty 5.0 of 10

EMO-R3, which combines a three-step emotional reasoning prompt with a reward for the model agreeing with its own image–emotion judgments, raises visual emotion-recognition accuracy by about one point over plain GRPO.

Reference graph

Works this paper leans on

263 extracted references · 153 linked inside Pith · cited by 3 Pith papers

[56]

Dan Hendrycks, Steven Basart, Saurav Kadavath, Mantas Mazeika, Akul Arora, Ethan Guo, Collin Burns, Samir Puranik, Horace He, Dawn Song, and Jacob Steinhardt. 2021. Measuring Coding Challenge Competence With APPS. InProceedings of the Neural Information Pro- cessing Systems Track on Datasets and Benchmarks, Vol. 1. https://datasets-benchmarks-proceedings....

2021
[32]

Shihan Dou, Yan Liu, Haoxiang Jia, Enyu Zhou, Limao Xiong, Junjie Shan, Caishuang Huang, Xiao Wang, Xiaoran Fan, Zhiheng Xi, Yuhao Zhou, Tao Ji, Rui Zheng, Qi Zhang, Tao Gui, and Xuanjing Huang. 2024. StepCoder: Improving Code Generation with Reinforcement Learning from Compiler Feedback. InProceedings of the 62nd Annual Meeting of the Association for Com...

doi:10.18653/v1/2024.a 2024
[129]

Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al. 2015. Human-level control through deep reinforcement learning.nature518, 7540 (2015), 529–533

2015
[207]

Wei Xiong, Hanze Dong, Chenlu Ye, Ziqi Wang, Han Zhong, Heng Ji, Nan Jiang, and Tong Zhang. 2023. Iterative preference learning from human feedback: Bridging theory and practice for rlhf under kl-constraint.arXiv preprint arXiv:2312.11456(2023)

arXiv 2023
[1]

NeMo RL: A Scalable and Efficient Post-Training Library

2025. NeMo RL: A Scalable and Efficient Post-Training Library. https://github.com/NVIDIA-NeMo/RL. GitHub repository

2025
[2]

Pranjal Aggarwal and Sean Welleck. 2025. L1: Controlling how long a reasoning model thinks with reinforcement learning.arXiv preprint arXiv:2503.04697(2025)

arXiv 2025
[3]

Anthropic. 2025. Claude Sonnet 4. https://www.anthropic.com/claude/sonnet

2025
[4]

Amanda Askell, Yuntao Bai, Anna Chen, Dawn Drain, Deep Ganguli, Tom Henighan, Andy Jones, Nicholas Joseph, Ben Mann, Nova DasSarma, et al. 2021. A general language assistant as a laboratory for alignment.arXiv preprint arXiv:2112.00861(2021)

arXiv 2021

Show all 263 references

[5]

Mohammad Gheshlaghi Azar, Zhaohan Daniel Guo, Bilal Piot, Remi Munos, Mark Rowland, Michal Valko, and Daniele Calandriello. 2024. A general theoretical paradigm to understand learning from human preferences. InInternational Conference on Artificial Intelligence and Statistics....

2024
[6]

Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. 2022. Training a helpful and harmless assistant with reinforcement learning from human feedback.arXiv preprint arXiv:2204.05862(2022)

2022 arXiv
[7]

Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. 2022. Constitutional ai: Harmlessness from ai feedback.arXiv preprint arXiv:2212.08073(2022)

2022 arXiv
[8]

Dibyanayan Bandyopadhyay, Soham Bhattacharjee, and Asif Ekbal. 2025. Thinking machines: A survey of llm based reasoning strategies.arXiv preprint arXiv:2503.10814(2025)

2025 arXiv
[9]

Do, Yan Xu, and Pascale Fung

Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, Quyet V. Do, Yan Xu, and Pascale Fung. 2023. A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactiv...

2023
[10]

2025.𝜏2-Bench: Evaluating Conversational Agents in a Dual-Control Environment.arXiv preprint arXiv:2506.07982(2025)

Victor Barres, Honghua Dong, Soham Ray, Xujie Si, and Karthik Narasimhan. 2025.𝜏2-Bench: Evaluating Conversational Agents in a Dual-Control Environment.arXiv preprint arXiv:2506.07982(2025)

2025 arXiv
[11]

Emily M Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. On the dangers of stochastic parrots: Can language models be too big?. InProceedings of the 2021 ACM conference on fairness, accountability, and transparency. 610–623

2021
[12]

Maciej Besta, Julia Barth, Eric Schreiber, Ales Kubicek, Afonso Catarino, Robert Gerstenberger, Piotr Nyczyk, Patrick Iff, Yueling Li, Sam Houliston, et al. 2025. Reasoning language models: A blueprint.arXiv preprint arXiv:2501.11223(2025)

2025 arXiv
[13]

Paul C Bogdan, Uzay Macar, Neel Nanda, and Arthur Conmy. 2025. Thought Anchors: Which LLM Reasoning Steps Matter?arXiv preprint arXiv:2506.19143(2025)

2025
[14]

Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. 2021. On the opportunities and risks of foundation models.arXiv preprint arXiv:2108.07258(2021)

2021 arXiv
[15]

Yuji Cao, Huan Zhao, Yuheng Cheng, Ting Shu, Yue Chen, Guolong Liu, Gaoqi Liang, Junhua Zhao, Jinyue Yan, and Yun Li. 2024. Survey on large language model-enhanced reinforcement learning: Concept, taxonomy, and methods.IEEE Transactions on Neural Networks and Learning Systems (2024)

2024
[16]

Shreyas Chaudhari, Pranjal Aggarwal, Vishvak Murahari, Tanmay Rajpurohit, Ashwin Kalyan, Karthik Narasimhan, Ameet Deshpande, and Bruno Castro da Silva. 2024. Rlhf deciphered: A critical analysis of reinforcement learning from human feedback for llms.Comput. Surveys(2024)

2024
[17]

Aili Chen, Aonian Li, Bangwei Gong, Binyang Jiang, Bo Fei, Bo Yang, Boji Shan, Changqing Yu, Chao Wang, Cheng Zhu, et al. 2025. MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attention.arXiv preprint arXiv:2506.13585(2025)

2025 arXiv
[18]

Chen Chen, Xinlong Hao, Weiwen Liu, Xu Huang, Xingshan Zeng, Shuai Yu, Dexun Li, Shuai Wang, Weinan Gan, Yuefeng Huang, et al. 2025. ACEBench: Who Wins the Match Point in Tool Learning?arXiv e-prints(2025), arXiv–2501

2025
[19]

Hardy Chen, Haoqin Tu, Fali Wang, Hui Liu, Xianfeng Tang, Xinya Du, Yuyin Zhou, and Cihang Xie. 2025. Sft or rl? an early investigation into training r1-like reasoning large vision-language models.arXiv preprint arXiv:2504.11468(2025)

2025 arXiv
[20]

Jiawei Chen, Dingkang Yang, Yue Jiang, Mingcheng Li, Jinjie Wei, Xiaolu Hou, and Lihua Zhang. 2024. Efficiency in Focus: LayerNorm as a Catalyst for Fine-tuning Medical Visual Language Models. InProceedings of the 32nd ACM International Conference on Multimedia. 3122–3130

2024
[21]

Jiawei Chen, Dingkang Yang, Tong Wu, Yue Jiang, Xiaolu Hou, Mingcheng Li, Shunli Wang, Dongling Xiao, Ke Li, and Lihua Zhang. 2024. Detecting and evaluating medical hallucinations in large vision language models.arXiv preprint arXiv:2406.10185(2024). Manuscript submitted to AC...

2024 arXiv
[22]

Wenhu Chen, Ming Yin, Max Ku, Pan Lu, Yixin Wan, Xueguang Ma, Jianyu Xu, Xinyi Wang, and Tony Xia. 2023. TheoremQA: A Theorem- driven Question Answering Dataset. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. Association for Computati...

2023 doi
[23]

Xiusi Chen, Gaotang Li, Ziqi Wang, Bowen Jin, Cheng Qian, Yu Wang, Hongru Wang, Yu Zhang, Denghui Zhang, Tong Zhang, et al. 2025. Rm-r1: Reward modeling as reasoning.arXiv preprint arXiv:2505.02387(2025)

2025
[24]

Zuyao Chen, Jinlin Wu, Zhen Lei, Marc Pollefeys, and Chang Wen Chen. 2025. Compile scene graphs with reinforcement learning.arXiv preprint arXiv:2504.13617(2025)

2025 arXiv
[25]

Zhoujun Cheng, Shibo Hao, Tianyang Liu, Fan Zhou, Yutao Xie, Feng Yao, Yuexin Bian, Yonghao Zhuang, Nilabjo Dey, Yuheng Zha, et al. 2025. Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspective.arXiv preprint arXiv:2506.14965(2025)

2025 arXiv
[26]

Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. 2021. Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168(2021)

2021 arXiv
[27]

Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic c...

2025 arXiv
[28]

Ganqu Cui, Yuchen Zhang, Jiacheng Chen, Lifan Yuan, Zhi Wang, Yuxin Zuo, Haozhan Li, Yuchen Fan, Huayu Chen, Weize Chen, et al. 2025. The entropy mechanism of reinforcement learning for reasoning language models.arXiv preprint arXiv:2505.22617(2025)

2025 arXiv
[29]

Huilin Deng, Ding Zou, Rui Ma, Hongchen Luo, Yang Cao, and Yu Kang. 2025. Boosting the generalization and reasoning of vision language models with curriculum reinforcement learning.arXiv preprint arXiv:2503.07065(2025)

2025 arXiv
[30]

Guanting Dong, Yifei Chen, Xiaoxi Li, Jiajie Jin, Hongjin Qian, Yutao Zhu, Hangyu Mao, Guorui Zhou, Zhicheng Dou, and Ji-Rong Wen. 2025. Tool-Star: Empowering LLM-Brained Multi-Tool Reasoner via Reinforcement Learning.arXiv preprint arXiv:2505.16410(2025)

2025 arXiv
[31]

Qingxiu Dong, Li Dong, Yao Tang, Tianzhu Ye, Yutao Sun, Zhifang Sui, and Furu Wei. 2025. Reinforcement Pre-Training.arXiv preprint arXiv:2506.08007(2025)

2025 arXiv
[33]

Xinrun Du, Yifan Yao, Kaijing Ma, Bingli Wang, Tianyu Zheng, King Zhu, Minghao Liu, Yiming Liang, Xiaolong Jin, Zhenlin Wei, et al . 2025. Supergpqa: Scaling llm evaluation across 285 graduate disciplines.arXiv preprint arXiv:2502.14739(2025)

2025 arXiv
[34]

Kawin Ethayarajh, Winnie Xu, Niklas Muennighoff, Dan Jurafsky, and Douwe Kiela. 2024. Model alignment as prospect theoretic optimization. In Forty-first International Conference on Machine Learning

2024
[35]

Kaixuan Fan, Kaituo Feng, Haoming Lyu, Dongzhan Zhou, and Xiangyu Yue. 2025. SophiaVL-R1: Reinforcing MLLMs Reasoning with Thinking Reward.arXiv preprint arXiv:2505.17018(2025)

2025
[36]

Gongfan Fang, Xinyin Ma, and Xinchao Wang. 2025. Thinkless: Llm learns when to think.arXiv preprint arXiv:2505.13379(2025)

2025 arXiv
[37]

Jiazhan Feng, Shijue Huang, Xingwei Qu, Ge Zhang, Yujia Qin, Baoquan Zhong, Chengquan Jiang, Jinxin Chi, and Wanjun Zhong. 2025. Retool: Reinforcement learning for strategic tool use in llms.arXiv preprint arXiv:2504.11536(2025)

2025 arXiv
[38]

Kaituo Feng, Kaixiong Gong, Bohao Li, Zonghao Guo, Yibing Wang, Tianshuo Peng, Junfei Wu, Xiaoying Zhang, Benyou Wang, and Xiangyu Yue
[39]

Lang Feng, Zhenghai Xue, Tingcong Liu, and Bo An. 2025. Group-in-group policy optimization for llm agent training.arXiv preprint arXiv:2505.10978 (2025)

2025 arXiv
[40]

Simon Frieder, Luca Pinchetti, Ryan-Rhys Griffiths, Tommaso Salvatori, Thomas Lukasiewicz, Philipp Petersen, and Julius Berner. 2023. Mathematical capabilities of chatgpt.Advances in neural information processing systems36 (2023), 27699–27744

2023
[41]

Jiayi Fu, Xuandong Zhao, Chengyuan Yao, Heng Wang, Qi Han, and Yanghua Xiao. 2025. Reward shaping to mitigate reward hacking in rlhf.arXiv preprint arXiv:2502.18770(2025)

2025
[42]

Wei Fu, Jiaxuan Gao, Xujie Shen, Chen Zhu, Zhiyu Mei, Chuyi He, Shusheng Xu, Guo Wei, Jun Mei, Jiashu Wang, et al. 2025. AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoning.arXiv preprint arXiv:2505.24298(2025)

2025 arXiv
[43]

Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A. Smith. 2020. RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models. InFindings of the Association for Computational Linguistics: EMNLP 2020. Association for Computational Linguistics,...

2020 doi
[44]

Aryo Pradipta Gema, Joshua Ong Jun Leang, Giwon Hong, Alessio Devoto, Alberto Carlo Maria Mancino, Rohit Saxena, Xuanli He, Yu Zhao, Xiaotang Du, Mohammad Reza Ghasemi Madani, Claire Barale, Robert McHardy, Joshua Harris, Jean Kaddour, Emile Van Krieken, and Pasquale Minervini...

2025
[45]

Dibya Ghosh and Sergey Levine. 2025. Visual Pre-Training on Unlabeled Images using Reinforcement Learning.arXiv preprint arXiv:2506.11967 (2025). Manuscript submitted to ACM 28 Keliang Liuet al

2025 arXiv
[46]

Anna Goldie, Azalia Mirhoseini, Hao Zhou, Irene Cai, and Christopher D Manning. 2025. Synthetic data generation & multi-step rl for reasoning & tool use.arXiv preprint arXiv:2504.04736(2025)

2025 arXiv
[47]

Yandong Guan, Xilin Wang, Xingxi Ming, Jing Zhang, Dong Xu, and Qian Yu. 2025. CAD-Coder: Text-to-CAD Generation with Chain-of-Thought and Geometric Reward.arXiv preprint arXiv:2505.19713(2025)

2025 arXiv
[48]

Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al . 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948(2025)

2025 arXiv
[49]

Jiaxin Guo, Zewen Chi, Li Dong, Qingxiu Dong, Xun Wu, Shaohan Huang, and Furu Wei. 2025. Reward reasoning model.arXiv preprint arXiv:2505.14674(2025)

2025 arXiv
[50]

Yiduo Guo, Zhen Guo, Chuanwei Huang, Zi-Ang Wang, Zekai Zhang, Haofei Yu, Huishuai Zhang, and Yikang Shen. 2025. Synthetic Data RL: Task Definition Is All You Need.arXiv preprint arXiv:2505.17063(2025)

2025 arXiv
[51]

Qianyue Hao, Sibo Li, Jian Yuan, and Yong Li. 2025. Rl of thoughts: Navigating llm reasoning with inference-time reinforcement learning.arXiv preprint arXiv:2505.14140(2025)

2025
[52]

Hado Hasselt. 2010. Double Q-learning.Advances in neural information processing systems23 (2010)

2010
[53]

Alexander Havrilla, Maksym Zhuravinskyi, Duy Phung, Aman Tiwari, Jonathan Tow, Stella Biderman, Quentin Anthony, and Louis Castricato
[54]

Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, Jie Liu, Lei Qi, Zhiyuan Liu, and Maosong Sun. 2024. OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scient...

2024 doi
[55]

Shenghua He, Tian Xia, Xuan Zhou, and Hui Wei. 2025. Response-Level Rewards Are All You Need for Online Reinforcement Learning in LLMs: A Mathematical Perspective.arXiv preprint arXiv:2506.02553(2025)

2025 arXiv
[57]

Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. Measuring Massive Multitask Language Understanding. InICLR. OpenReview.net

2021
[58]

Jiwoo Hong, Noah Lee, and James Thorne. 2024. ORPO: Monolithic Preference Optimization without Reference Model. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Miami, Florida, USA, 11170–11189...

2024 doi
[59]

Wenyi Hong, Wenmeng Yu, Xiaotao Gu, Guo Wang, Guobing Gan, Haomiao Tang, Jiale Cheng, Ji Qi, Junhui Ji, Lihang Pan, et al. 2025. GLM-4.1 V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning.arXiv preprint arXiv:2507.01006(2025)

2025 arXiv
[60]

Zhenyu Hou, Ziniu Hu, Yujiang Li, Rui Lu, Jie Tang, and Yuxiao Dong. 2025. TreeRL: LLM Reinforcement Learning with On-Policy Tree Search. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computa...

2025 doi
[61]

Jian Hu. 2025. Reinforce++: A simple and efficient approach for aligning large language models.arXiv preprint arXiv:2501.03262(2025)

2025 arXiv
[62]

Jian Hu, Xibin Wu, Zilin Zhu, Weixun Wang, Dehao Zhang, Yu Cao, et al. 2024. Openrlhf: An easy-to-use, scalable and high-performance rlhf framework.arXiv preprint arXiv:2405.11143(2024)

2024 arXiv
[63]

Jingcheng Hu, Yinmin Zhang, Qi Han, Daxin Jiang, Xiangyu Zhang, and Heung-Yeung Shum. 2025. Open-reasoner-zero: An open source approach to scaling up reinforcement learning on the base model.arXiv preprint arXiv:2503.24290(2025)

2025 arXiv
[64]

Yang Hu, Xingyu Zhang, Xueji Fang, Zhiyang Chen, Xiao Wang, Huatian Zhang, and Guojun Qi. 2025. SLOT: Sample-specific Language Model Optimization at Test-time.arXiv preprint arXiv:2505.12392(2025)

2025 arXiv
[65]

Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Yu, Xinying Song, and Denny Zhou. 2024. Large Language Models Cannot Self-Correct Reasoning Yet. InInternational Conference on Representation Learning, Vol. 2024. 32808–32824. https://proceedings.iclr.cc/ pape...

2024
[66]

Qihan Huang, Weilong Dai, Jinlong Liu, Wanggui He, Hao Jiang, Mingli Song, Jingyuan Chen, Chang Yao, and Jie Song. 2025. Boosting mllm reasoning with text-debiased hint-grpo.arXiv preprint arXiv:2503.23905(2025)

2025 arXiv
[67]

Ting Huang, Zeyu Zhang, and Hao Tang. 2025. 3D-R1: Enhancing Reasoning in 3D VLMs for Unified Scene Understanding.arXiv preprint arXiv:2507.23478(2025)

2025 arXiv
[68]

Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Zhe Xu, Yao Hu, and Shaohui Lin. 2025. Vision-r1: Incentivizing reasoning capability in multimodal large language models.arXiv preprint arXiv:2503.06749(2025)

2025 arXiv
[69]

Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. 2024. Gpt-4o system card.arXiv preprint arXiv:2410.21276(2024)

2024 arXiv
[70]

Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. 2024. Openai o1 system card.arXiv preprint arXiv:2412.16720(2024). Manuscript submitted to ACM RL Meets LLMs: A Survey of Advanceme...

2024 arXiv
[71]

Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. 2024. Livecodebench: Holistic and contamination free evaluation of large language models for code.arXiv preprint arXiv:2403.07974(2024)

2024 arXiv
[72]

Miaomiao Ji, Yanqiu Wu, Zhibin Wu, Shoujin Wang, Jian Yang, Mark Dras, and Usman Naseem. 2025. A survey on progress in llm alignment from the perspective of reward design.arXiv preprint arXiv:2505.02666(2025)

2025 arXiv
[73]

Dongzhi Jiang, Ziyu Guo, Renrui Zhang, Zhuofan Zong, Hao Li, Le Zhuo, Shilin Yan, Pheng-Ann Heng, and Hongsheng Li. 2025. T2i-r1: Reinforcing image generation with collaborative semantic-level and token-level cot.arXiv preprint arXiv:2505.00703(2025)

2025 arXiv
[74]

Lingjie Jiang, Xun Wu, Shaohan Huang, Qingxiu Dong, Zewen Chi, Li Dong, Xingxing Zhang, Tengchao Lv, Lei Cui, and Furu Wei. 2025. Think only when you need with large hybrid-reasoning models.arXiv preprint arXiv:2505.14631(2025)

2025 arXiv
[75]

Ruili Jiang, Kehai Chen, Xuefeng Bai, Zhixuan He, Juntao Li, Muyun Yang, Tiejun Zhao, Liqiang Nie, and Min Zhang. 2024. A survey on human preference learning for large language models.arXiv preprint arXiv:2406.11191(2024)

2024 arXiv
[76]

Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. 2025. Search-r1: Training llms to reason and leverage search engines with reinforcement learning.arXiv preprint arXiv:2503.09516(2025)

2025 arXiv
[77]

Zhewei Kang, Xuandong Zhao, and Dawn Song. 2025. Scalable best-of-n selection for large language models via self-certainty.arXiv preprint arXiv:2502.18581(2025)

2025
[78]

Timo Kaufmann, Paul Weng, Viktor Bengs, and Eyke Hüllermeier. 2024. A Survey of Reinforcement Learning from Human Feedback. arXiv:2312.14925 [cs.LG] https://arxiv.org/abs/2312.14925

2024
[79]

Mehran Kazemi, Bahare Fatemi, Hritik Bansal, John Palowitch, Chrysovalantis Anastasiou, Sanket Vaibhav Mehta, Lalit K Jain, Virginia Aglietti, Disha Jindal, Peter Chen, et al. 2025. Big-bench extra hard.arXiv preprint arXiv:2502.19187(2025)

2025 arXiv
[80]

Zixuan Ke, Fangkai Jiao, Yifei Ming, Xuan-Phi Nguyen, Austin Xu, Do Xuan Long, Minzhi Li, Chengwei Qin, Peifeng Wang, Silvio Savarese, et al
[81]

Zachary Kenton, Tom Everitt, Laura Weidinger, Iason Gabriel, Vladimir Mikulik, and Geoffrey Irving. 2021. Alignment of language agents.arXiv preprint arXiv:2103.14659(2021)

2021 arXiv
[82]

Vijay Konda and John Tsitsiklis. 1999. Actor-critic algorithms.Advances in neural information processing systems12 (1999)

1999
[83]

A survey of frontiers in llm reasoning: Inference scaling, learning to reason, and agentic systems.arXiv preprint arXiv:2504.09037(2025)

2025
[84]

Suhas Kotha, Jacob Mitchell Springer, and Aditi Raghunathan. 2024. Understanding Catastrophic Forgetting in Language Models via Implicit Inference. InICLR. OpenReview.net

2024
[85]

Komal Kumar, Tajamul Ashraf, Omkar Thawakar, Rao Muhammad Anwer, Hisham Cholakkal, Mubarak Shah, Ming-Hsuan Yang, Phillip HS Torr, Fahad Shahbaz Khan, and Salman Khan. 2025. Llm post-training: A deep dive into reasoning large language models.arXiv preprint arXiv:2502.21321 (2025)

2025 arXiv
[86]

Tomasz Korbak, Hady Elsahar, Germán Kruszewski, and Marc Dymetman. 2022. On reinforcement learning and distribution matching for fine-tuning language models with no catastrophic forgetting.Advances in Neural Information Processing Systems35 (2022), 16203–16220

2022
[87]

Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, et al. 2024. Tulu 3: Pushing frontiers in open language model post-training.arXiv preprint arXiv:2411.15124(2024)

2024 arXiv
[88]

Harrison Lee, Samrat Phatale, Hassan Mansoor, Kellie Ren Lu, Thomas Mesnard, Johan Ferret, Colton Bishop, Ethan Hall, Victor Carbune, and Abhinav Rastogi. 2023. Rlaif: Scaling reinforcement learning from human feedback with ai feedback. (2023)

2023
[89]

Minae Kwon, Sang Michael Xie, Kalesha Bullard, and Dorsa Sadigh. 2023. Reward Design with Language Models. InICLR. OpenReview.net

2023
[90]

Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, et al. 2022. Solving quantitative reasoning problems with language models.Advances in neural information processing system...

2022
[91]

Ming Li, Jike Zhong, Shitian Zhao, Yuxiang Lai, Haoquan Zhang, Wang Bill Zhu, and Kaipeng Zhang. 2025. Think or not think: A study of explicit thinking in rule-based visual reinforcement fine-tuning.arXiv preprint arXiv:2503.16188(2025)

2025
[92]

Yuxuan Lei, Dingkang Yang, Zhaoyu Chen, Jiawei Chen, Peng Zhai, and Lihua Zhang. 2025. Large Vision-Language Models as Emotion Recognizers in Context Awareness. InAsian Conference on Machine Learning. PMLR, 111–126

2025
[93]

Tianle Li, Wei-Lin Chiang, Evan Frick, Lisa Dunlap, Tianhao Wu, Banghua Zhu, Joseph E Gonzalez, and Ion Stoica. 2024. From crowdsourced data to high-quality benchmarks: Arena-hard and benchbuilder pipeline.arXiv preprint arXiv:2406.11939(2024)

2024 arXiv
[94]

Xuefeng Li, Haoyang Zou, and Pengfei Liu. 2025. Torl: Scaling tool-integrated rl.arXiv preprint arXiv:2503.23383(2025)

2025 arXiv
[95]

Pengyi Li, Matvey Skripkin, Alexander Zubrey, Andrey Kuznetsov, and Ivan Oseledets. 2025. Confidence Is All You Need: Few-Shot RL Fine-Tuning of Language Models.arXiv preprint arXiv:2506.06395(2025)

2025 arXiv
[96]

Zhuoling Li, Xiaogang Xu, Zhenhua Xu, Ser-Nam Lim, and Hengshuang Zhao. 2025. LARM: Large Auto-Regressive Model for Long-Horizon Embodied Intelligence. InForty-second International Conference on Machine Learning. https://openreview.net/forum?id=zcx7jqUZg5

2025
[97]

Eric Liang, Richard Liaw, Robert Nishihara, Philipp Moritz, Roy Fox, Ken Goldberg, Joseph Gonzalez, Michael Jordan, and Ion Stoica. 2018. RLlib: Abstractions for distributed reinforcement learning. InInternational conference on machine learning. PMLR, 3053–3062

2018
[98]

Yi-Chen Li, Tian Xu, Yang Yu, Xuqin Zhang, Xiong-Hui Chen, Zhongxiang Ling, Ningjing Chao, Lei Yuan, and Zhi-Hua Zhou. 2025. Generalist Reward Models: Found Inside Large Language Models.arXiv preprint arXiv:2506.23235(2025)

2025 arXiv
[99]

Xiao Liang, Zhong-Zhi Li, Yeyun Gong, Yang Wang, Hengyuan Zhang, Yelong Shen, Ying Nian Wu, and Weizhu Chen. 2025. SwS: Self-aware Weakness-driven Problem Synthesis in Reinforcement Learning for LLM Reasoning.arXiv preprint arXiv:2506.08989(2025)

2025 arXiv
[100]

Zhenyi Liao, Qingsong Xie, Yanhao Zhang, Zijian Kong, Haonan Lu, Zhenyu Yang, and Zhijie Deng. 2025. Improved visual-spatial reasoning via r1-zero-like training.arXiv preprint arXiv:2504.00883(2025)

2025 arXiv
[101]

Guannan Liang and Qianqian Tong. 2025. LLM-Powered AI Agent Systems and Their Applications in Industry.arXiv preprint arXiv:2505.16120 (2025). Manuscript submitted to ACM 30 Keliang Liuet al

2025 arXiv
[102]

Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al
[103]

Bo Liu, Leon Guertler, Simon Yu, Zichen Liu, Penghui Qi, Daniel Balcells, Mickel Liu, Cheston Tan, Weiyan Shi, Min Lin, et al. 2025. SPIRAL: Self-Play on Zero-Sum Games Incentivizes Reasoning via Multi-Agent Multi-Turn Reinforcement Learning.arXiv preprint arXiv:2506.24119(2025)

2025
[104]

Bill Yuchen Lin, Ronan Le Bras, Kyle Richardson, Ashish Sabharwal, Radha Poovendran, Peter Clark, and Yejin Choi. 2025. ZebraLogic: On the Scaling Limits of LLMs for Logical Reasoning. InForty-second International Conference on Machine Learning. https://openreview.net/forum?id...

2025
[105]

Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. 2023. Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation.Advances in Neural Information Processing Systems36 (2023), 21558–21572

2023
[106]

Liyuan Liu, Feng Yao, Dinghuai Zhang, Chengyu Dong, Jingbo Shang, and Jianfeng Gao. 2025. FlashRL: 8Bit Rollouts, Full Power RL. https: //fengyao.notion.site/flash-rl

2025
[107]

Mingjie Liu, Shizhe Diao, Ximing Lu, Jian Hu, Xin Dong, Yejin Choi, Jan Kautz, and Yi Dong. 2025. Prorl: Prolonged reinforcement learning expands reasoning boundaries in large language models.arXiv preprint arXiv:2505.24864(2025)

2025 arXiv
[108]

Chengzhi Liu, Zhongxing Xu, Qingyue Wei, Juncheng Wu, James Zou, Xin Eric Wang, Yuyin Zhou, and Sheng Liu. 2025. More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning Models.arXiv preprint arXiv:2505.21523(2025)

2025 arXiv
[109]

Xiao Liu, Xuanyu Lei, Shengyuan Wang, Yue Huang, Andrew Feng, Bosi Wen, Jiale Cheng, Pei Ke, Yifan Xu, Weng Lam Tam, Xiaohan Zhang, Lichao Sun, Xiaotao Gu, Hongning Wang, Jing Zhang, Minlie Huang, Yuxiao Dong, and Jie Tang. 2024. AlignBench: Benchmarking Chinese Alignment of L...

2024 doi
[110]

Yihao Liu, Shuocheng Li, Lang Cao, Yuhang Xie, Mengyu Zhou, Haoyu Dong, Xiaojun Ma, Shi Han, and Dongmei Zhang. 2025. SuperRL: Reinforcement Learning with Supervision to Boost Language Model Reasoning.arXiv preprint arXiv:2506.01096(2025)

2025 arXiv
[111]

Yizhou Liu, Jingwei Wei, Zizhi Chen, Minghao Han, Xukun Zhang, Keliang Liu, and Lihua Zhang. 2025. Breaking Reward Collapse: Adaptive Reinforcement for Open-ended Medical Reasoning with Enhanced Semantic Discrimination.arXiv preprint arXiv:2508.12957(2025)

2025
[112]

Qianchu Liu, Sheng Zhang, Guanghui Qin, Timothy Ossowski, Yu Gu, Ying Jin, Sid Kiblawi, Sam Preston, Mu Wei, Paul Vozila, et al . 2025. X-reasoner: Towards generalizable reasoning across modalities and domains.arXiv preprint arXiv:2505.03981(2025)

2025 arXiv
[113]

Ziyu Liu, Zeyi Sun, Yuhang Zang, Xiaoyi Dong, Yuhang Cao, Haodong Duan, Dahua Lin, and Jiaqi Wang. 2025. Visual-rft: Visual reinforcement fine-tuning.arXiv preprint arXiv:2503.01785(2025)

2025 arXiv
[114]

Zijun Liu, Peiyi Wang, Runxin Xu, Shirong Ma, Chong Ruan, Peng Li, Yang Liu, and Yu Wu. 2025. Inference-time scaling for generalist reward modeling.arXiv preprint arXiv:2504.02495(2025)

2025
[115]

Lin Long, Yichen He, Wentao Ye, Yiyuan Pan, Yuan Lin, Hang Li, Junbo Zhao, and Wei Li. 2025. Seeing, Listening, Remembering, and Reasoning: A Multimodal Agent with Long-Term Memory.arXiv preprint arXiv:2508.09736(2025)

2025
[116]

Zichen Liu, Changyu Chen, Wenjun Li, Tianyu Pang, Chao Du, and Min Lin. 2025. There may not be aha moment in r1-zero-like training—a pilot study

2025
[117]

Haoran Luo, Guanting Chen, Qika Lin, Yikai Guo, Fangzhi Xu, Zemin Kuang, Meina Song, Xiaobao Wu, Yifan Zhu, Luu Anh Tuan, et al. 2025. Graph-R1: Towards Agentic GraphRAG Framework via End-to-end Reinforcement Learning.arXiv preprint arXiv:2507.21892(2025)

2025 arXiv
[118]

Wenjie Ma, Jingxuan He, Charlie Snell, Tyler Griggs, Sewon Min, and Matei Zaharia. 2025. Reasoning models can be effective without thinking. arXiv preprint arXiv:2504.09858(2025)

2025 arXiv
[119]

Xinyu Ma, Ziyang Ding, Zhicong Luo, Chi Chen, Zonghao Guo, Derek F Wong, Xiaoyi Feng, and Maosong Sun. 2025. Deepperception: Advancing r1-like cognitive visual perception in mllms for knowledge-intensive visual grounding.arXiv preprint arXiv:2503.12797(2025)

2025
[120]

Chenwei Lou, Zewei Sun, Xinnian Liang, Meng Qu, Wei Shen, Wenqi Wang, Yuntao Li, Qingping Yang, and Shuangzhi Wu. 2025. AdaCoT: Pareto-Optimal Adaptive Chain-of-Thought Triggering via Reinforcement Learning.arXiv preprint arXiv:2505.11896(2025)

2025 arXiv
[121]

Yuchun Miao, Sen Zhang, Liang Ding, Rong Bao, Lefei Zhang, and Dacheng Tao. 2024. Inform: Mitigating reward hacking in rlhf via information- theoretic reward modeling.Advances in Neural Information Processing Systems37 (2024), 134387–134429

2024
[122]

Yuchun Miao, Sen Zhang, Liang Ding, Yuqi Zhang, Lefei Zhang, and Dacheng Tao. 2025. The energy loss phenomenon in rlhf: A new perspective on mitigating reward hacking.arXiv preprint arXiv:2501.19358(2025)

2025 arXiv
[123]

Iman Mirzadeh, Keivan Alizadeh, Hooman Shahrokhi, Oncel Tuzel, Samy Bengio, and Mehrdad Farajtabar. 2025. GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models. InICLR. OpenReview.net

2025
[124]

Yecheng Jason Ma, William Liang, Guanzhi Wang, De-An Huang, Osbert Bastani, Dinesh Jayaraman, Yuke Zhu, Linxi Fan, and Anima Anandkumar
[125]

Eureka: Human-level reward design via coding large language models.arXiv preprint arXiv:2310.12931(2023)

2023 arXiv
[126]

OpenAI. 2022. Introducing ChatGPT. https://openai.com/blog/chatgpt. Manuscript submitted to ACM RL Meets LLMs: A Survey of Advancements and Applications Across the LLM Lifecycle 31

2022
[127]

2025.OpenAI o3 and o4 -mini System Card

OpenAI. 2025.OpenAI o3 and o4 -mini System Card. Technical Report. OpenAI. https://cdn.openai.com/pdf/2221c875-02dc-4789-800b-e7758f3722c1/ o3-and-o4-mini-system-card.pdf System Card officially released by OpenAI on April 16, 2025

2025
[128]

Kun Ouyang, Yuanxin Liu, Haoning Wu, Yi Liu, Hao Zhou, Jie Zhou, Fandong Meng, and Xu Sun. 2025. SpaceR: Reinforcing MLLMs in Video Spatial Reasoning.arXiv preprint arXiv:2504.01805(2025)

2025 arXiv
[130]

Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. 2025. s1: Simple test-time scaling.arXiv preprint arXiv:2501.19393(2025)

2025 arXiv
[131]

Arka Pal, Deep Karkhanis, Samuel Dooley, Manley Roberts, Siddartha Naidu, and Colin White. 2024. Smaug: Fixing failure modes of preference optimisation with dpo-positive.arXiv preprint arXiv:2402.13228(2024)

2024 arXiv
[132]

Feng Peiyuan, Yichen He, Guanhua Huang, Yuan Lin, Hanchong Zhang, Yuchen Zhang, and Hang Li. 2024. Agile: A novel reinforcement learning framework of llm agents.Advances in Neural Information Processing Systems37 (2024), 5244–5284

2024
[133]

Long Phan, Alice Gatti, Ziwen Han, Nathaniel Li, Josephina Hu, Hugh Zhang, Chen Bo Calvin Zhang, Mohamed Shaaban, John Ling, Sean Shi, et al. 2025. Humanity’s last exam.arXiv preprint arXiv:2501.14249(2025)

2025 arXiv
[134]

Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback.Advances in neural information processing systems35 (...

2022
[135]

Samuel J Paech. 2025. EQ-Bench Creative Writing Benchmark v3. https://github.com/EQ-bench/creative-writing-bench

2025
[136]

Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, ...

2025 arXiv
[137]

Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model.Advances in neural information processing systems36 (2023), 53728–53741

2023
[138]

Rajkumar Ramamurthy, Prithviraj Ammanabrolu, Kianté Brantley, Jack Hessel, Rafet Sifa, Christian Bauckhage, Hannaneh Hajishirzi, and Yejin Choi. 2023. Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Langua...

2023
[139]

Moschoula Pternea, Prerna Singh, Abir Chakraborty, Yagna Oruganti, Mirco Milletari, Sayli Bapat, and Kebei Jiang. 2024. The rl/llm taxonomy tree: Reviewing synergies between reinforcement learning and large language models.Journal of Artificial Intelligence Research80 (2024), ...

2024
[140]

Zhangyang Qi, Zhixiong Zhang, Yizhou Yu, Jiaqi Wang, and Hengshuang Zhao. 2025. VLN-R1: Vision-Language Navigation via Reinforcement Fine-Tuning.arXiv preprint arXiv:2506.17221(2025)

2025 arXiv
[141]

1994.On-line Q-learning using connectionist systems

Gavin A Rummery and Mahesan Niranjan. 1994.On-line Q-learning using connectionist systems. Vol. 37. University of Cambridge, Department of Engineering Cambridge, UK

1994
[142]

Soumya Rani Samineni, Durgesh Kalwar, Karthik Valmeekam, Kaya Stechly, and Subbarao Kambhampati. 2025. RL in Name Only? Analyzing the Structural Assumptions in RL post-training for LLMs.arXiv preprint arXiv:2505.13697(2025)

2025
[143]

John Schulman, Sergey Levine, Pieter Abbeel, Michael Jordan, and Philipp Moritz. 2015. Trust region policy optimization. InInternational conference on machine learning. PMLR, 1889–1897

2015
[144]

Abhinav Rastogi, Albert Q Jiang, Andy Lo, Gabrielle Berrada, Guillaume Lample, Jason Rute, Joep Barmentlo, Karmesh Yadav, Kartik Khandelwal, Khyathi Raghavi Chandu, et al. 2025. Magistral.arXiv preprint arXiv:2506.10910(2025)

2025 arXiv
[145]

David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. 2024. Gpqa: A graduate-level google-proof q&a benchmark. InFirst Conference on Language Modeling

2024
[146]

Chenyang Shao, Xinyang Liu, Yutang Lin, Fengli Xu, and Yong Li. 2025. Route-and-Reason: Scaling Large Language Model Reasoning with Reinforced Model Router.arXiv preprint arXiv:2506.05901(2025)

2025
[147]

Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al . 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300(2024)

2024 arXiv
[148]

Haozhan Shen, Peng Liu, Jingcheng Li, Chunxin Fang, Yibo Ma, Jiajia Liao, Qiaoli Shen, Zilun Zhang, Kangjia Zhao, Qianqian Zhang, et al. 2025. Vlm-r1: A stable and generalizable r1-style large vision-language model.arXiv preprint arXiv:2504.07615(2025)

2025 arXiv
[149]

John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347(2017)

2017 arXiv
[150]

Darsh J Shah, Peter Rushton, Somanshu Singla, Mohit Parmar, Kurt Smith, Yash Vanjani, Ashish Vaswani, Adarsh Chaluvaraju, Andrew Hojel, Andrew Ma, et al. 2025. Rethinking reflection in pre-training.arXiv preprint arXiv:2504.04022(2025)

2025 arXiv
[151]

Parshin Shojaee, Iman Mirzadeh, Keivan Alizadeh, Maxwell Horton, Samy Bengio, and Mehrdad Farajtabar. 2025. The illusion of thinking: Understanding the strengths and limitations of reasoning models via the lens of problem complexity.arXiv preprint arXiv:2506.06941(2025)

2025 arXiv
[152]

Ved Sirdeshmukh, Kaustubh Deshpande, Johannes Mols, Lifeng Jin, Ed-Yeremai Cardona, Dean Lee, Jeremy Kritz, Willow Primack, Summer Yue, and Chen Xing. 2025. Multichallenge: A realistic multi-turn conversation evaluation benchmark challenging to frontier llms.arXiv preprint Man...

2025 arXiv
[153]

Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adri Garriga-Alonso, et al. 2023. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models.Transactions on...

2023
[154]

Hui Shen, Taiqiang Wu, Qi Han, Yunta Hsieh, Jizhou Wang, Yuyue Zhang, Yuxin Cheng, Zijian Hao, Yuansheng Ni, Xin Wang, et al. 2025. PhyX: Does Your Model Have the" Wits" for Physical Reasoning?arXiv preprint arXiv:2505.15929(2025)

2025 arXiv
[155]

Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. 2025. Hybridflow: A flexible and efficient rlhf framework. InProceedings of the Twentieth European Conference on Computer Systems. 1279–1297

2025
[156]

Richard S Sutton, David McAllester, Satinder Singh, and Yishay Mansour. 1999. Policy gradient methods for reinforcement learning with function approximation.Advances in neural information processing systems12 (1999)

1999
[157]

Gokul Swamy, Christoph Dann, Rahul Kidambi, Zhiwei Steven Wu, and Alekh Agarwal. 2024. A minimaximalist approach to reinforcement learning from human feedback.arXiv preprint arXiv:2401.04056(2024)

2024 arXiv
[158]

Alex Tamkin, Miles Brundage, Jack Clark, and Deep Ganguli. 2021. Understanding the capabilities, limitations, and societal impact of large language models.arXiv preprint arXiv:2102.02503(2021)

2021 arXiv
[159]

Saksham Sahai Srivastava and Vaneet Aggarwal. 2025. A Technical Survey of Reinforcement Learning Techniques for Large Language Models. arXiv preprint arXiv:2507.04136(2025)

2025 arXiv
[160]

Zhaochen Su, Linjie Li, Mingyang Song, Yunzhuo Hao, Zhengyuan Yang, Jun Zhang, Guanjie Chen, Jiawei Gu, Juntao Li, Xiaoye Qu, et al. 2025. Openthinkimg: Learning to think with images via visual tool reinforcement learning.arXiv preprint arXiv:2505.08617(2025)

2025 arXiv
[161]

Shulin Tian, Ruiqi Wang, Hongming Guo, Penghao Wu, Yuhao Dong, Xiuying Wang, Jingkang Yang, Hao Zhang, Hongyuan Zhu, and Ziwei Liu
[162]

Carel van Niekerk, Renato Vukovic, Benjamin Matthias Ruppik, Hsien chin Lin, and Milica Gašić. 2025. Post-Training Large Language Models via Reinforcement Learning from Self-Feedback. arXiv:2507.21931 [cs.CL] https://arxiv.org/abs/2507.21931

2025 arXiv
[163]

Leandro von Werra, Younes Belkada, Lewis Tunstall, Edward Beeching, Tristan Thrush, Nathan Lambert, Shengyi Huang, Kashif Rasul, and Quentin Gallouédec. 2020. TRL: Transformer Reinforcement Learning. https://github.com/huggingface/trl

2020
[164]

Zhen Tan, Jun Yan, I Hsu, Rujun Han, Zifeng Wang, Long T Le, Yiwen Song, Yanfei Chen, Hamid Palangi, George Lee, et al. 2025. In prospect and retrospect: Reflective memory management for long-term personalized dialogue agents.arXiv preprint arXiv:2503.08026(2025)

2025 arXiv
[165]

Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al
[166]

5: Scaling reinforcement learning with llms.arXiv preprint arXiv:2501.12599(2025)

Kimi k1. 5: Scaling reinforcement learning with llms.arXiv preprint arXiv:2501.12599(2025)

2025 arXiv
[167]

Jiacong Wang, Zijiang Kang, Haochen Wang, Haiyong Jiang, Jiawen Li, Bohong Wu, Ya Wang, Jiao Ran, Xiao Liang, Chao Feng, et al. 2025. VGR: Visual Grounded Reasoning.arXiv preprint arXiv:2506.11991(2025)

2025 arXiv
[168]

Ego-R1: Chain-of-Tool-Thought for Ultra-Long Egocentric Video Reasoning.arXiv preprint arXiv:2506.13654(2025)

2025 arXiv
[169]

Shuhe Wang, Shengyu Zhang, Jie Zhang, Runyi Hu, Xiaoya Li, Tianwei Zhang, Jiwei Li, Fei Wu, Guoyin Wang, and Eduard Hovy. 2024. Reinforcement learning enhanced llms: A survey.arXiv preprint arXiv:2412.10400(2024)

2024 arXiv
[170]

Tevin Wang and Chenyan Xiong. 2025. AutoRule: Reasoning Chain-of-thought Extracted Rule-based Rewards Improve Preference Learning.arXiv preprint arXiv:2506.15651(2025)

2025 arXiv
[171]

Zhongwei Wan, Zhihao Dou, Che Liu, Yu Zhang, Dongfei Cui, Qinjian Zhao, Hui Shen, Jing Xiong, Yi Xin, Yifan Jiang, et al. 2025. Srpo: Enhancing multimodal llm reasoning via reflection-aware reinforcement learning.arXiv preprint arXiv:2506.01713(2025)

2025
[172]

Hanlin Wang, Chak Tou Leong, Jiashuo Wang, Jian Wang, and Wenjie Li. 2025. SPA-RL: Reinforcing LLM Agents via Stepwise Progress Attribution. arXiv preprint arXiv:2505.20732(2025)

2025 arXiv
[173]

Haiming Wang, Mert Unsal, Xiaohan Lin, Mantas Baksys, Junqi Liu, Marco Dos Santos, Flood Sung, Marina Vinyes, Zhenzhe Ying, Zekai Zhu, et al
[174]

Kimina-prover preview: Towards large formal reasoning models with reinforcement learning.arXiv preprint arXiv:2504.11354(2025)

2025 arXiv
[175]

Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, et al
[176]

Shenzhi Wang, Le Yu, Chang Gao, Chujie Zheng, Shixuan Liu, Rui Lu, Kai Dang, Xionghui Chen, Jianxin Yang, Zhenru Zhang, et al. 2025. Beyond the 80/20 rule: High-entropy minority tokens drive effective reinforcement learning for llm reasoning.arXiv preprint arXiv:2506.01939(2025)

2025 arXiv
[177]

Yunhao Wang, Yuhao Zhang, Tinghao Yu, Can Xu, Feng Zhang, and Fengzong Lian. 2025. Adaptive Deep Reasoning: Triggering Deep Thinking When Needed.arXiv preprint arXiv:2505.20101(2025)

2025 arXiv
[178]

Zhichao Wang, Bin Bi, Shiva Kumar Pentyala, Kiran Ramnath, Sougata Chaudhuri, Shubham Mehrotra, Xiang-Bo Mao, Sitaram Asur, et al. 2024. A comprehensive survey of llm alignment techniques: Rlhf, rlaif, ppo, dpo and more.arXiv preprint arXiv:2407.16216(2024). Manuscript submitt...

2024 arXiv
[179]

Weixun Wang, Shaopan Xiong, Gengru Chen, Wei Gao, Sheng Guo, Yancheng He, Ju Huang, Jiaheng Liu, Zhendong Li, Xiaoyang Li, et al. 2025. Reinforcement Learning Optimization for Large-Scale Learning: An Efficient and User-Friendly Scaling Library.arXiv preprint arXiv:2506.06122 (2025)

2025 arXiv
[180]

Xubin Wang, Jianfei Wu, Yichen Yuan, Deyu Cai, Mingzhe Li, and Weijia Jia. 2024. Demonstration selection for in-context learning via reinforcement learning.arXiv preprint arXiv:2412.03966(2024)

2024 arXiv
[181]

Xiyao Wang, Zhengyuan Yang, Chao Feng, Yongyuan Liang, Yuhang Zhou, Xiaoyu Liu, Ziyi Zang, Ming Li, Chung-Ching Lin, Kevin Lin, et al
[182]

ViCrit: A Verifiable Reinforcement Learning Proxy Task for Visual Perception in VLMs.arXiv preprint arXiv:2506.10128(2025)

2025 arXiv
[183]

Yibin Wang, Zhimin Li, Yuhang Zang, Chunyu Wang, Qinglin Lu, Cheng Jin, and Jiaqi Wang. 2025. Unified multimodal chain-of-thought reward model through reinforcement fine-tuning.arXiv preprint arXiv:2505.03318(2025)

2025
[184]

Jason Wei, Nguyen Karina, Hyung Won Chung, Yunxin Joy Jiao, Spencer Papay, Amelia Glaese, John Schulman, and William Fedus. 2024. Measuring short-form factuality in large language models.arXiv preprint arXiv:2411.04368(2024)

2024 arXiv
[185]

Mmlu-pro: A more robust and challenging multi-task language understanding benchmark.Advances in Neural Information Processing Systems 37 (2024), 95266–95290

2024
[186]

Yiming Wang, Pei Zhang, Jialong Tang, Haoran Wei, Baosong Yang, Rui Wang, Chenshu Sun, Feitong Sun, Jiran Zhang, Junxuan Wu, et al. 2025. Polymath: Evaluating mathematical reasoning in multilingual contexts.arXiv preprint arXiv:2504.18428(2025)

2025
[187]

Ronald J Williams. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning.Machine learning8, 3 (1992), 229–256

1992
[188]

Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, et al. 2020. Transformers: State-of-the-art natural language processing. InProceedings of the 2020 conference on empirical methods ...

2020
[189]

Zhexu Wang, Yiping Liu, Yejie Wang, Wenyang He, Bofei Gao, Muxi Diao, Yanxu Chen, Kelin Fu, Flood Sung, Zhilin Yang, et al. 2025. OJBench: A Competition Level Code Benchmark For Large Language Models.arXiv preprint arXiv:2506.16395(2025)

2025
[190]

Zihan Wang, Kangrui Wang, Qineng Wang, Pingyue Zhang, Linjie Li, Zhengyuan Yang, Xing Jin, Kefan Yu, Minh Nhat Nguyen, Licheng Liu, et al
[191]

Ragen: Understanding self-evolution in llm agents via multi-turn reinforcement learning.arXiv preprint arXiv:2504.20073(2025)

2025 arXiv
[192]

Zengzhi Wang, Fan Zhou, Xuefeng Li, and Pengfei Liu. 2025. Octothinker: Mid-training incentivizes reinforcement learning scaling.arXiv preprint arXiv:2506.20512(2025)

2025 arXiv
[193]

Zhixin Wang, Tianyi Zhou, Liming Liu, Ao Li, Jiarui Hu, Dian Yang, Jinlong Hou, Siyuan Feng, Yuan Cheng, and Yuan Qi. 2025. DistFlow: A Fully Distributed RL Framework for Scalable and Efficient LLM Post-Training.arXiv preprint arXiv:2507.13833(2025)

2025 arXiv
[194]

Christopher JCH Watkins and Peter Dayan. 1992. Q-learning.Machine learning8, 3 (1992), 279–292

1992
[195]

Tianbao Xie, Siheng Zhao, Chen Henry Wu, Yitao Liu, Qian Luo, Victor Zhong, Yanchao Yang, and Tao Yu. 2024. Text2Reward: Reward Shaping with Language Models for Reinforcement Learning. InICLR. OpenReview.net

2024
[196]

Laura Weidinger, John Mellor, Maribeth Rauh, Conor Griffin, Jonathan Uesato, Po-Sen Huang, Myra Cheng, Mia Glaese, Borja Balle, Atoosa Kasirzadeh, et al. 2021. Ethical and social risks of harm from language models.arXiv preprint arXiv:2112.04359(2021)

2021 arXiv
[197]

Colin White et al. 2025. LiveBench: A Challenging, Contamination-Free LLM Benchmark. InThe Thirteenth International Conference on Learning Representations

2025
[198]

Hongling Xu, Qi Zhu, Heyuan Deng, Jinpeng Li, Lu Hou, Yasheng Wang, Lifeng Shang, Ruifeng Xu, and Fei Mi. 2025. KDRL: Post-Training Reasoning LLMs via Unified Knowledge Distillation and Reinforcement Learning.arXiv preprint arXiv:2506.02208(2025)

2025 arXiv
[199]

Weiye Xu, Jiahao Wang, Weiyun Wang, Zhe Chen, Wengang Zhou, Aijun Yang, Lewei Lu, Houqiang Li, Xiaohua Wang, Xizhou Zhu, et al. 2025. Visulogic: A benchmark for evaluating visual reasoning in multi-modal large language models.arXiv preprint arXiv:2504.15279(2025)

2025 arXiv
[200]

Bo Wu, Sid Wang, Yunhao Tang, Jia Ding, Eryk Helenowski, Liang Tan, Tengyu Xu, Tushar Gowda, Zhengxing Chen, Chen Zhu, et al . 2025. Llamarl: A distributed asynchronous reinforcement learning framework for efficient large-scale llm trainin.arXiv preprint arXiv:2505.24034(2025)

2025 arXiv
[201]

Fang Wu, Weihao Xuan, Ximing Lu, Zaid Harchaoui, and Yejin Choi. 2025. The Invisible Leash: Why RLVR May Not Escape Its Origin.arXiv preprint arXiv:2507.14843(2025)

2025
[202]

Haoyuan Wu, Xueyi Chen, Rui Ming, Jilong Gao, Shoubo Hu, Zhuolun He, and Bei Yu. 2025. ToTRL: Unlock LLM Tree-of-Thoughts Reasoning Potential through Puzzles Solving.arXiv preprint arXiv:2505.12717(2025)

2025
[203]

Junkang Wu, Yuexiang Xie, Zhengyi Yang, Jiancan Wu, Jinyang Gao, Bolin Ding, Xiang Wang, and Xiangnan He. 2024. \beta-DPO: Direct Preference Optimization with Dynamic \beta. InAdvances in Neural Information Processing Systems, A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. P...

2024
[204]

Mingqi Wu, Zhihao Zhang, Qiaole Dong, Zhiheng Xi, Jun Zhao, Senjie Jin, Xiaoran Fan, Yuhao Zhou, Yanwei Fu, Qin Liu, et al. 2025. Reasoning or Memorization? Unreliable Results of Reinforcement Learning Due to Data Contamination.arXiv preprint arXiv:2507.10532(2025)

2025
[205]

xAI. 2025. Grok 3 Beta — The Age of Reasoning Agents. https://x.ai/news/grok-3. Accessed: 2025-08-25

2025
[206]

Dingkang Yang, Dongling Xiao, Jinjie Wei, Mingcheng Li, Zhaoyu Chen, Ke Li, and Lihua Zhang. 2025. Improving factuality in large language models via decoding-time hallucinatory and truthful comparators. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 39. ...

2025
[208]

Fengli Xu, Qianyue Hao, Zefang Zong, Jingwei Wang, Yunke Zhang, Jingyi Wang, Xiaochong Lan, Jiahui Gong, Tianjian Ouyang, Fanjin Meng, et al. 2025. Towards large reasoning models: A survey of reinforced reasoning with large language models.arXiv preprint arXiv:2501.09686(2025)

2025 arXiv
[209]

Zhicheng Yang, Zhijiang Guo, Yinya Huang, Xiaodan Liang, Yiwei Wang, and Jing Tang. 2025. TreeRPO: Tree Relative Policy Optimization.arXiv preprint arXiv:2506.05183(2025)

2025
[210]

Feng Yao, Liyuan Liu, Dinghuai Zhang, Chengyu Dong, Jingbo Shang, and Jianfeng Gao. 2025. Your Efficient RL Framework Secretly Brings You Off-Policy RL Training. https://fengyao.notion.site/off-policy-rl

2025
[211]

Yi Xu, Chengzu Li, Han Zhou, Xingchen Wan, Caiqi Zhang, Anna Korhonen, and Ivan Vulić. 2025. Visual Planning: Let’s Think Only with Images. arXiv preprint arXiv:2505.11409(2025)

2025
[212]

Zeyue Xue, Jie Wu, Yu Gao, Fangyuan Kong, Lingting Zhu, Mengzhao Chen, Zhiheng Liu, Wei Liu, Qiushan Guo, Weilin Huang, et al . 2025. DanceGRPO: Unleashing GRPO on Visual Generation.arXiv preprint arXiv:2505.07818(2025)

2025 arXiv
[213]

Jianhao Yan, Yafu Li, Zican Hu, Zhi Wang, Ganqu Cui, Xiaoye Qu, Yu Cheng, and Yue Zhang. 2025. Learning to reason under off-policy guidance. arXiv preprint arXiv:2504.14945(2025)

2025 arXiv
[214]

Sikuan Yan, Xiufeng Yang, Zuchao Huang, Ercong Nie, Zifeng Ding, Zonggen Li, Xiaowen Ma, Hinrich Schütze, Volker Tresp, and Yunpu Ma
[215]

Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learning.arXiv preprint arXiv:2508.19828(2025)

2025 arXiv
[216]

An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. 2025. Qwen3 technical report.arXiv preprint arXiv:2505.09388(2025)

2025 arXiv
[217]

Dingkang Yang, Jinjie Wei, Dongling Xiao, Shunli Wang, Tong Wu, Gang Li, Mingcheng Li, Shuaibing Wang, Jiawei Chen, Yue Jiang, et al. 2024. Pediatricsgpt: Large language models as chinese medical assistants for pediatric applications.Advances in Neural Information Processing S...

2024
[218]

Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Shiji Song, and Gao Huang. 2025. Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model?arXiv preprint arXiv:2504.13837(2025)

2025 arXiv
[219]

John Yang, Kilian Leret, Carlos E Jimenez, Alexander Wettig, Kabir Khandpur, Yanzhe Zhang, Binyuan Hui, Ofir Press, Ludwig Schmidt, and Diyi Yang. 2025. Swe-smith: Scaling data for software engineering agents.arXiv preprint arXiv:2504.21798(2025)

2025 arXiv
[220]

Yi Yang, Xiaoxuan He, Hongkun Pan, Xiyan Jiang, Yan Deng, Xingtao Yang, Haoyu Lu, Dacheng Yin, Fengyun Rao, Minfeng Zhu, et al . 2025. R1-onevision: Advancing generalized multimodal reasoning through cross-modal formalization.arXiv preprint arXiv:2503.10615(2025)

2025 arXiv
[221]

Junyu Zhang, Runpei Dong, Han Wang, Xuying Ning, Haoran Geng, Peihao Li, Xialin He, Yutong Bai, Jitendra Malik, Saurabh Gupta, et al. 2025. AlphaOne: Reasoning Models Thinking Slow and Fast at Test Time.arXiv preprint arXiv:2505.24863(2025)

2025 arXiv
[222]

Jingyi Zhang, Jiaxing Huang, Huanjin Yao, Shunyu Liu, Xikun Zhang, Shijian Lu, and Dacheng Tao. 2025. R1-vl: Learning to reason with multimodal large language models via step-wise group relative policy optimization.arXiv preprint arXiv:2503.12937(2025)

2025 arXiv
[223]

Huanjin Yao, Jiaxing Huang, Yawen Qiu, Michael K Chen, Wenzheng Liu, Wei Zhang, Wenjie Zeng, Xikun Zhang, Jingyi Zhang, Yuxin Song, et al
[224]

MMReason: An Open-Ended Multi-Modal Multi-Step Reasoning Benchmark for MLLMs Toward AGI.arXiv preprint arXiv:2506.23563(2025)

2025 arXiv
[225]

Zhewei Yao, Reza Yazdani Aminabadi, Olatunji Ruwase, Samyam Rajbhandari, Xiaoxia Wu, Ammar Ahmad Awan, Jeff Rasley, Minjia Zhang, Conglong Li, Connor Holmes, et al. 2023. Deepspeed-chat: Easy, fast and affordable rlhf training of chatgpt-like models at all scales.arXiv preprin...

2023 arXiv
[226]

Jingyang Yi, Jiazheng Wang, and Sida Li. 2025. Shorterbetter: Guiding reasoning models to find optimal inference length for efficient reasoning. arXiv preprint arXiv:2504.21370(2025)

2025
[227]

Yang You. 2023. “Colossalchat: An open-source solution for cloning chatgpt with a complete rlhf pipeline

2023
[228]

Hongli Yu, Tinghong Chen, Jiangtao Feng, Jiangjie Chen, Weinan Dai, Qiying Yu, Ya-Qin Zhang, Wei-Ying Ma, Jingjing Liu, Mingxuan Wang, et al
[229]

MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agent.arXiv preprint arXiv:2507.02259(2025)

2025 arXiv
[230]

Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, et al. 2025. Dapo: An open-source llm reinforcement learning system at scale.arXiv preprint arXiv:2503.14476(2025)

2025 arXiv
[231]

Zhuohao Yu, Jiali Zeng, Weizheng Gu, Yidong Wang, Jindong Wang, Fandong Meng, Jie Zhou, Yue Zhang, Shikun Zhang, and Wei Ye. 2025. RewardAnything: Generalizable Principle-Following Reward Models.arXiv preprint arXiv:2506.03637(2025)

2025 arXiv
[232]

Andrew Zhao, Yiran Wu, Yang Yue, Tong Wu, Quentin Xu, Matthieu Lin, Shenzhi Wang, Qingyun Wu, Zilong Zheng, and Gao Huang. 2025. Absolute zero: Reinforced self-play reasoning with zero data.arXiv preprint arXiv:2505.03335(2025). Manuscript submitted to ACM RL Meets LLMs: A Sur...

2025 arXiv
[233]

Kaiwen Zha, Zhengqi Gao, Maohao Shen, Zhang-Wei Hong, Duane S Boning, and Dina Katabi. 2025. RL Tango: Reinforcing Generator and Verifier Together for Language Reasoning.arXiv preprint arXiv:2505.15034(2025)

2025
[234]

Yufei Zhan, Ziheng Wu, Yousong Zhu, Rongkun Xue, Ruipu Luo, Zhenghao Chen, Can Zhang, Yifan Li, Zhentao He, Zheming Yang, et al. 2025. GThinker: Towards General Multimodal Reasoning via Cue-Guided Rethinking.arXiv preprint arXiv:2506.01078(2025)

2025 arXiv
[235]

Rosie Zhao, Alexandru Meterez, Sham Kakade, Cengiz Pehlevan, Samy Jelassi, and Eran Malach. 2025. Echo chamber: Rl post-training amplifies behaviors learned in pretraining.arXiv preprint arXiv:2504.07912(2025)

2025 arXiv
[236]

Xuandong Zhao, Zhewei Kang, Aosong Feng, Sergey Levine, and Dawn Song. 2025. Learning to reason without external rewards.arXiv preprint arXiv:2505.19590(2025)

2025 arXiv
[237]

Jiajie Zhang, Nianyi Lin, Lei Hou, Ling Feng, and Juanzi Li. 2025. Adaptthink: Reasoning models can learn when to think.arXiv preprint arXiv:2505.13417(2025)

2025 arXiv
[238]

Kaiyan Zhang, Yuxin Zuo, Bingxiang He, Youbang Sun, Runze Liu, Che Jiang, Yuchen Fan, Kai Tian, Guoli Jia, Pengfei Li, et al. 2025. A Survey of Reinforcement Learning for Large Reasoning Models.arXiv preprint arXiv:2509.08827(2025)

2025 arXiv
[239]

Xiaoyun Zhang, Jingqing Ruan, Xing Ma, Yawen Zhu, Haodong Zhao, Hao Li, Jiansong Chen, Ke Zeng, and Xunliang Cai. 2025. When to continue thinking: Adaptive thinking mode switching for efficient reasoning.arXiv preprint arXiv:2505.15400(2025)

2025 arXiv
[240]

Xiaojiang Zhang, Jinghui Wang, Zifei Cheng, Wenhao Zhuang, Zheng Lin, Minglei Zhang, Shaojie Wang, Yinghan Cui, Chao Wang, Junyi Peng, et al. 2025. Srpo: A cross-domain implementation of large-scale reinforcement learning on llm.arXiv preprint arXiv:2504.14286(2025)

2025 arXiv
[241]

Xingjian Zhang, Siwei Wen, Wenjun Wu, and Lei Huang. 2025. Tinyllava-video-r1: Towards smaller lmms for video reasoning.arXiv preprint arXiv:2504.09641(2025)

2025 arXiv
[242]

Aha Moments

Xiaoqing Zhang, Huabin Zheng, Ang Lv, Yuhan Liu, Zirui Song, Xiuying Chen, Rui Yan, and Flood Sung. 2025. Divide-Fuse-Conquer: Eliciting" Aha Moments" in Multi-Scenario Games.arXiv preprint arXiv:2505.16401(2025)

2025 arXiv
[243]

Yudi Zhang, Lu Wang, Meng Fang, Yali Du, Chenghua Huang, Jun Wang, Qingwei Lin, Mykola Pechenizkiy, Dongmei Zhang, Saravan Rajmohan, et al. 2025. Distill Not Only Data but Also Rewards: Can Smaller Language Models Surpass Larger Ones?arXiv preprint arXiv:2502.19557(2025)

2025 arXiv
[244]

Yimeng Zhang, Tian Wang, Jiri Gesi, Ziyi Wang, Yuxuan Lu, Jiacheng Lin, Sinong Zhan, Vianne Gao, Ruochen Jiao, Junze Liu, Kun Qian, Yuxin Tang, Ran Xue, Houyu Zhang, Qingjun Cui, Yufan Guo, and Dakuo Wang. 2025. Shop-R1: Rewarding LLMs to Simulate Human Behavior in Online Shop...

2025
[245]

Yanzhi Zhang, Zhaoxi Zhang, Haoxiang Guan, Yilin Cheng, Yitong Duan, Chen Wang, Yue Wang, Shuxin Zheng, and Jiyan He. 2025. No Free Lunch: Rethinking Internal Feedback for LLM Reasoning.arXiv preprint arXiv:2506.17219(2025)

2025 arXiv
[246]

Liyun Zhu, Qixiang Chen, Xi Shen, and Xiaodong Cun. 2025. VAU-R1: Advancing Video Anomaly Understanding via Reinforcement Fine-Tuning. arXiv preprint arXiv:2505.23504(2025)

2025 arXiv
[247]

Baining Zhao, Ziyou Wang, Jianjie Fang, Chen Gao, Fanhang Man, Jinqiang Cui, Xin Wang, Xinlei Chen, Yong Li, and Wenwu Zhu. 2025. Embodied-R: Collaborative Framework for Activating Embodied Spatial Reasoning in Foundation Models via Reinforcement Learning.arXiv preprint arXiv:...

2025 arXiv
[248]

Jian Zhao, Runze Liu, Kaiyan Zhang, Zhimu Zhou, Junqi Gao, Dong Li, Jiafei Lyu, Zhouyi Qian, Biqing Qi, Xiu Li, et al. 2025. Genprm: Scaling test-time compute of process reward models via generative reasoning.arXiv preprint arXiv:2504.00891(2025)

2025 arXiv
[249]

Yuxin Zuo, Kaiyan Zhang, Li Sheng, Shang Qu, Ganqu Cui, Xuekai Zhu, Haozhan Li, Yuchen Zhang, Xinwei Long, Ermo Hua, et al. 2025. Ttrl: Test-time reinforcement learning.arXiv preprint arXiv:2504.16084(2025). Manuscript submitted to ACM

2025 arXiv
[251]

Chujie Zheng, Shixuan Liu, Mingze Li, Xiong-Hui Chen, Bowen Yu, Chang Gao, Kai Dang, Yuqiong Liu, Rui Men, An Yang, et al. 2025. Group Sequence Policy Optimization.arXiv preprint arXiv:2507.18071(2025)

2025 arXiv
[252]

Thinking with Images

Ziwei Zheng, Michael Yang, Jack Hong, Chenxiao Zhao, Guohai Xu, Le Yang, Chao Shen, and Xing Yu. 2025. DeepEyes: Incentivizing" Thinking with Images" via Reinforcement Learning.arXiv preprint arXiv:2505.14362(2025)

2025 arXiv
[253]

Jialun Zhong, Wei Shen, Yanzeng Li, Songyang Gao, Hua Lu, Yicheng Chen, Yang Zhang, Wei Zhou, Jinjie Gu, and Lei Zou. 2025. A comprehensive survey of reward models: Taxonomy, applications, challenges, and future.arXiv preprint arXiv:2504.12328(2025)

2025 arXiv
[254]

Guanghao Zhou, Panjia Qiu, Cen Chen, Jie Wang, Zheming Yang, Jian Xu, and Minghui Qiu. 2025. Reinforced mllm: A survey on rl-based reasoning in multimodal large language models.arXiv preprint arXiv:2504.21277(2025)

2025 arXiv
[255]

Aha Moment

Hengguang Zhou, Xirui Li, Ruochen Wang, Minhao Cheng, Tianyi Zhou, and Cho-Jui Hsieh. 2025. R1-Zero’s" Aha Moment" in Visual Reasoning on a 2B Non-SFT Model.arXiv preprint arXiv:2503.05132(2025)

2025 arXiv
[256]

Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. 2023. Instruction-following evaluation for large language models.arXiv preprint arXiv:2311.07911(2023)

2023 arXiv
[257]

Xiangxin Zhou, Zichen Liu, Anya Sims, Haonan Wang, Tianyu Pang, Chongxuan Li, Liang Wang, Min Lin, and Chao Du. 2025. Reinforcing General Reasoning without Verifiers.arXiv preprint arXiv:2505.21493(2025)

2025 arXiv
[258]

Jason Zhu and Hongyu Li. 2025. Towards Concise and Adaptive Thinking in Large Reasoning Models: A Survey.arXiv preprint arXiv:2507.09662 (2025)

2025 arXiv
[259]

Ke Zhu, Yu Wang, Jiangjiang Liu, Qunyi Xie, Shanshan Liu, and Gang Zhang. 2025. On Data Synthesis and Post-training for Visual Abstract Reasoning.arXiv preprint arXiv:2504.01324(2025)

2025 arXiv
[261]

Qin Zhu, Fei Huang, Runyu Peng, Keming Lu, Bowen Yu, Qinyuan Cheng, Xipeng Qiu, Xuanjing Huang, and Junyang Lin. 2025. AutoLogi: Automated generation of logic puzzles for evaluating reasoning abilities of large language models.arXiv preprint arXiv:2502.16906(2025)

2025 arXiv
[262]

Xinyu Zhu, Mengzhou Xia, Zhepei Wei, Wei-Lin Chen, Danqi Chen, and Yu Meng. 2025. The surprising effectiveness of negative reinforcement in LLM reasoning.arXiv preprint arXiv:2506.01347(2025)

2025
[2023]

InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing

trlX: A framework for large scale reinforcement learning from human feedback. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 8578–8595

2023
[2024]

Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437(2024)

2024 arXiv
[2025]

Video-r1: Reinforcing video reasoning in mllms.arXiv preprint arXiv:2503.21776(2025)

2025 arXiv

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.

[1] [56]

Dan Hendrycks, Steven Basart, Saurav Kadavath, Mantas Mazeika, Akul Arora, Ethan Guo, Collin Burns, Samir Puranik, Horace He, Dawn Song, and Jacob Steinhardt. 2021. Measuring Coding Challenge Competence With APPS. InProceedings of the Neural Information Pro- cessing Systems Track on Datasets and Benchmarks, Vol. 1. https://datasets-benchmarks-proceedings....

2021

[2] [32]

Shihan Dou, Yan Liu, Haoxiang Jia, Enyu Zhou, Limao Xiong, Junjie Shan, Caishuang Huang, Xiao Wang, Xiaoran Fan, Zhiheng Xi, Yuhao Zhou, Tao Ji, Rui Zheng, Qi Zhang, Tao Gui, and Xuanjing Huang. 2024. StepCoder: Improving Code Generation with Reinforcement Learning from Compiler Feedback. InProceedings of the 62nd Annual Meeting of the Association for Com...

doi:10.18653/v1/2024.a 2024

[3] [129]

Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al. 2015. Human-level control through deep reinforcement learning.nature518, 7540 (2015), 529–533

2015

[4] [207]

Wei Xiong, Hanze Dong, Chenlu Ye, Ziqi Wang, Han Zhong, Heng Ji, Nan Jiang, and Tong Zhang. 2023. Iterative preference learning from human feedback: Bridging theory and practice for rlhf under kl-constraint.arXiv preprint arXiv:2312.11456(2023)

arXiv 2023

[5] [1]

NeMo RL: A Scalable and Efficient Post-Training Library

2025. NeMo RL: A Scalable and Efficient Post-Training Library. https://github.com/NVIDIA-NeMo/RL. GitHub repository

2025

[6] [2]

Pranjal Aggarwal and Sean Welleck. 2025. L1: Controlling how long a reasoning model thinks with reinforcement learning.arXiv preprint arXiv:2503.04697(2025)

arXiv 2025

[7] [3]

Anthropic. 2025. Claude Sonnet 4. https://www.anthropic.com/claude/sonnet

2025

[8] [4]

Amanda Askell, Yuntao Bai, Anna Chen, Dawn Drain, Deep Ganguli, Tom Henighan, Andy Jones, Nicholas Joseph, Ben Mann, Nova DasSarma, et al. 2021. A general language assistant as a laboratory for alignment.arXiv preprint arXiv:2112.00861(2021)

arXiv 2021