REVIEW 19 cited by
Agent-RLVR: Training Software Engineering Agents via Guidance and Environment Rewards
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Agent-RLVR: Training Software Engineering Agents via Guidance and Environment Rewards
read the original abstract
Reinforcement Learning from Verifiable Rewards (RLVR) has been widely adopted as the de facto method for enhancing the reasoning capabilities of large language models and has demonstrated notable success in verifiable domains like math and competitive programming tasks. However, the efficacy of RLVR diminishes significantly when applied to agentic environments. These settings, characterized by multi-step, complex problem solving, lead to high failure rates even for frontier LLMs, as the reward landscape is too sparse for effective model training via conventional RLVR. In this work, we introduce Agent-RLVR, a framework that makes RLVR effective in challenging agentic settings, with an initial focus on software engineering tasks. Inspired by human pedagogy, Agent-RLVR introduces agent guidance, a mechanism that actively steers the agent towards successful trajectories by leveraging diverse informational cues. These cues, ranging from high-level strategic plans to dynamic feedback on the agent's errors and environmental interactions, emulate a teacher's guidance, enabling the agent to navigate difficult solution spaces and promotes active self-improvement via additional environment exploration. In the Agent-RLVR training loop, agents first attempt to solve tasks to produce initial trajectories, which are then validated by unit tests and supplemented with agent guidance. Agents then reattempt with guidance, and the agent policy is updated with RLVR based on the rewards of these guided trajectories. Agent-RLVR elevates the pass@1 performance of Qwen-2.5-72B-Instruct from 9.4% to 22.4% on SWE-Bench Verified. We find that our guidance-augmented RLVR data is additionally useful for test-time reward model training, shown by further boosting pass@1 to 27.8%. Agent-RLVR lays the groundwork for training agents with RLVR in complex, real-world environments where conventional RL methods struggle.
Forward citations
Cited by 19 Pith papers
-
SENTINEL: Failure-Driven Reinforcement Learning for Training Tool-Using Language Model Agents
SENTINEL generates targeted tasks from model failures in a Controller-Proposer-Solver loop, raising Pass^1 from 66.4 to 74.9 on Tau2-Bench Retail and outperforming standard RL.
-
Revisiting Reinforcement Learning with Verifiable Rewards from a Contrastive Perspective
ConSPO improves RLVR training by aligning rollout scores with generation likelihoods via length-normalized log-probabilities and applying a group-wise InfoNCE contrastive loss with a scheduled margin, outperforming GR...
-
Masked Diffusion Language Models are Strong and Steerable Text-Based World Models for Agentic RL
Masked diffusion language models, not larger autoregressive LLMs, are the better building block for text-based world models in agentic RL, improving rollout fidelity, diversity, and downstream task success.
-
Generate, Filter, Control, Replay: A Comprehensive Survey of Rollout Strategies for LLM Reinforcement Learning
This survey introduces the Generate-Filter-Control-Replay (GFCR) taxonomy to structure rollout pipelines for RL-based post-training of reasoning LLMs.
-
Agentic-DPO: From Imitation to Agentic Policy Optimization on Expert Trajectories
Offline DPO on expert states with one-step student negatives and schema-preserving augmentation improves LLM agents beyond SFT and can match online GRPO without environment interaction.
-
AgenticRL: Self-Refining Agentic Reinforcement Learning for Vision-Conditioned UAV Navigation
AgenticRL deploys a multimodal GPT agent in a closed-loop process to autonomously design and refine reward functions for PPO-trained vision-conditioned UAV navigation policies, reporting 71% policy improvement and 91%...
-
ReSkill: Reconciling Skill Creation with Policy Optimization in Agentic RL
ReSkill is an RL-in-the-loop framework that embeds assertion-driven skill creation, within-group sampling, and Thompson Sampling into GRPO to reconcile skill evolution with policy learning, outperforming prior methods...
-
Revisiting Reinforcement Learning with Verifiable Rewards from a Contrastive Perspective
ConSPO is a new contrastive sequence-level policy optimization method that addresses GRPO limitations via length-normalized log-probability scores and InfoNCE-style objectives, outperforming baselines on reasoning benchmarks.
-
Revisiting Reinforcement Learning with Verifiable Rewards from a Contrastive Perspective
ConSPO introduces a contrastive sequence-level policy optimization that aligns rollout scores with generation likelihoods via length-normalized log-probabilities and an InfoNCE-style group contrast with curriculum mar...
-
Fate of Secondary Droplets Produced by High-speed Raindrops Interacting with a Liquid Pool
Secondary droplet size distribution from raindrop-liquid pool impacts scales as N_d(r_s) ∝ r_s^{-5/2} and collapses onto one curve when normalized by surface tension and raindrop diameter.
-
SERA: Soft-Verified Efficient Repository Agents
SERA trains competitive coding agents via supervised finetuning on synthetic trajectories verified only by patch-overlap recall, cutting data-generation cost by roughly two orders of magnitude.
-
SWE-IF: Aligning Code Evaluation with Human Preference
A blend of functional correctness and instruction-following predicts human preference for code LLMs better than pass@k alone.
-
SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?
SWE-Bench Pro is a new benchmark with 1,865 long-horizon tasks from 41 repositories designed to evaluate AI agents on realistic enterprise-level software engineering problems beyond prior benchmarks.
-
Trading Human Curation for Synthetic Augmentation in RLVR
Gated synthetic augmentations of a 10-task human base substitute for ~87 extra human RLVR tasks on aggregate held-out pass@1, with cost-adjusted trade rate ρ_cost in [1.4×, 11.6×].
-
SWE-Shepherd: Advancing PRMs for Reinforcing Code Agents
SWE-Shepherd trains a lightweight PRM on SWE-Bench trajectories to score intermediate actions and guide code agents, showing gains in efficiency and action quality on SWE-Bench Verified.
-
Fate of Secondary Droplets Produced by High-speed Raindrops Interacting with a Liquid Pool
Secondary droplets from 7 m/s raindrop impacts on a liquid pool obey N_d(r_s) ∝ r_s^{-5/2}, collapsing under a new scaling that also depends on surface tension and raindrop diameter.
-
ODYSSE: Episode-wise Policy Optimization for Personalized Agentic Reasoning
An episode-level variant of GRPO (ESPO) improves personalized GUI-agent reasoning on the 102-episode SmartSpot benchmark, outperforming step-wise and outcome-only training baselines.
-
Beyond Next-Token Prediction: An RLVR Proof of Concept for Tool-Use Agents on Atlassian Workflows
RLVR training on five synthetic Atlassian API environments raises average tool-use reward for Qwen models from 0.35-0.92 to 0.95-1.00 on four non-degenerate scenarios.
-
Trading Human Curation for Synthetic Augmentation in RLVR
Gated synthetic augmentations can substitute for additional human-authored RLVR tasks at a cost-adjusted trade rate of 1.4x-11.6x while retaining held-out generalization on ten benchmarks spanning code, instruction fo...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.