REVIEW 7 cited by
Process Supervision-Guided Policy Optimization for Code Generation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Reinforcement learning (RL) with unit test feedback has enhanced large language models' (LLMs) code generation, but relies on sparse rewards provided only after complete code evaluation, limiting learning efficiency and incremental improvements. When generated code fails all unit tests, no learning signal is received, hindering progress on complex tasks. To address this, we propose a Process Reward Model (PRM) that delivers dense, line-level feedback on code correctness during generation, mimicking human code refinement and providing immediate guidance. We explore various strategies for training PRMs and integrating them into the RL framework, finding that using PRMs both as dense rewards and for value function initialization significantly boosts performance. Our experimental results also highlight the effectiveness of PRMs in enhancing RL-driven code generation, especially for long-horizon scenarios.
Forward citations
Cited by 7 Pith papers
-
Evaluating Judges as Evaluators: The JETTS Benchmark of LLM-as-Judges as Test-Time Scaling Evaluators
JETTS, a new benchmark, shows LLM-as-judges are competitive in response reranking, worse than process reward models in beam search, and ineffective as critique providers for refinement.
-
Beyond Binary: Turning Partial Success into Dense Verifiable Rewards for Reinforcement Learning in Code Generation
VeRPO turns partial test-suite success into dense, difficulty-weighted rewards for RL code generation, beating GRPO by up to 8.8 pass@1 points at negligible cost.
-
Reasoning Through Execution: Unifying Process and Outcome Rewards for Code Generation
A tree-based, inference-only framework combining execution metrics with LLM self-critique improves code generation correctness and efficiency across models and benchmarks.
-
Improving LLM-Generated Code Quality with GRPO
Adding a hand-built code-quality score to the GRPO reward makes a 3B model produce code that blind raters prefer, but evidence rests on one model and a synthetic benchmark.
-
CRScore++: Reinforcement Learning with Verifiable Tool and AI Feedback for Code Review
A two-stage SFT+DPO pipeline using linter and code-smell tool outputs plus an LLM judge improves generated code review comments and appears to transfer from Python to Java and JavaScript, though the judge-based evalua...
-
CRPE: Expanding The Reasoning Capability of Large Language Model for Code Generation
A three-stage training pipeline (synthetic problems, multi-agent chain-of-thought data, step-DPO self-improvement) raises LiveCodeBench pass@1 for Qwen2.5-Coder bases to 21.88 (7B) and 35.08 (32B).
-
Process-Supervised Reinforcement Learning for Code Generation
A mutation/refactoring, compile, and execute pipeline auto-generates line-level process supervision that improves reinforcement learning for code generation over outcome-only supervision.
Discussion (0). Continue with ORCID to comment.