Pith. sign in

REVIEW 4 major objections 5 minor 24 references

A staged reward — progress feedback for failing programs, baseline-relative efficiency for correct ones — lifts Qwen3-32B on PerfCodeBench from 11.1% to 54.6% correct-and-runnable rate and from 8.1% to 38.6% relative efficiency.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 10:46 UTC pith:XGB3Q2TA

load-bearing objection Genuinely new reward design for code RL, but the headline efficiency numbers rest on the authors' own benchmark and single-run timing—worth reviewing, with revision. the 4 major comments →

arxiv 2607.27271 v1 pith:XGB3Q2TA submitted 2026-07-29 cs.LG cs.SE

RLPF: Reinforcement Learning from Performance Feedback for Code Generation

Huihao Jing , Haozhe Cui , Wenbin Hu , Shaojin Chen , Haochen Shi , Changxuan Fan , Yuxuan Liu , Hanyu Yang
show 4 more authors
Sirui Zhang Ziyi Chen Haoran Li Yangqiu Song
This is my paper
classification cs.LG cs.SE
keywords reinforcement learningcode generationperformance feedbackexecution feedbackreward shapingrelative efficiencyruntime optimizationGRPO
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper asks whether code generators can be trained to prefer faster correct implementations, not merely to pass tests, and proposes RLPF, a reinforcement-learning reward that stages execution outcomes. Programs that fail are ranked by how far they advance through the pipeline (extraction, compilation, execution, oracle check), while correct programs are ranked by how much of the gap between the task's baseline and its expert reference they close. The authors argue this staging is needed because raw runtime is a fragile signal: it is meaningful only after a program is correct, its scale differs across tasks, and it gives no guidance when most sampled programs do not compile or run. Fine-tuning Qwen3-32B on PerfCodeBench raises correct-and-runnable solutions from 11.1% to 54.6% and relative efficiency from 8.1% to 38.6%; a correctness-only baseline reaches similar correctness but leaves a much larger share of correct programs slower than baseline. The paper concludes that code agents can be trained not only to pass tests but to optimize the programs they write.

Core claim

The central claim is that execution outcomes can be converted into a dense, task-comparable reward for reinforcement learning on code. RLPF splits outcomes into two regimes: failed rollouts get a shaping reward ordered by execution progress (no extractable program, no compilation, no successful run, wrong output), so the policy receives signal before anything is correct; correct rollouts get a reward anchored to each task's baseline and expert reference, measuring the fraction of the baseline-to-reference runtime gap the candidate closes. A strict gap between the worst correct and the best failed reward keeps correctness as the boundary. As evidence, the authors contrast a correctness-only b

What carries the argument

The reward staircase. RLPF orders every rollout outcome on one line: no extractable program < no compilation < no successful run < wrong output < correct < faster-than-baseline < reference-or-better. The failure-mode shaping ladder assigns -0.05, 0.00, +0.05, +0.10 to the four pre-correct states, giving GRPO an ordering even when nothing is correct. CGRE (correctness-gated relative efficiency) measures the fraction of the task's baseline-to-reference runtime gap a correct candidate closes, normalized per task. Correct programs get max{s(WO)+ε, CGRE + αF·FBR + αR·RBR}, with ε keeping correctness the boundary. The staircase turns group-relative optimization into an ordering over execution outc

Load-bearing premise

The load-bearing premise is that each task's baseline and expert reference, together with median wall-clock runtime, define a meaningful, stable scale for efficiency across heterogeneous languages and machines; the paper itself notes in its limitations that timing and test suites remain imperfect for systems tasks, so if references are noisy or runtime measurements unstable, the CGRE reward and the reported gains can mislead even when training behaves as described.

What would settle it

Re-run the trained model on the PerfCodeBench test split under a stricter timing protocol — more repetitions, high-resolution timers, fixed hardware — and check whether the efficiency margin over the correctness-only baseline (CGRE 38.6 vs 29.0) survives; if it collapses, the speed claim is partly a measurement artifact. Second, retrain without the EMA-peak checkpoint rule: since the full reward declines after its peak, headline gains that evaporate at the final checkpoint would show the results depend on the selection rule rather than the reward design.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Efficiency becomes a first-class training objective: the same run that lifts correctness also changes the model's preferences among correct programs, so passing tests and preferring speed are optimized together rather than as separate stages.
  • A LoRA adapter on a frozen base model can recover a large share of the gap to much stronger open-weight systems on a hard systems benchmark, raising correct-and-runnable rate from 11.1% to 54.6% and relative efficiency from 8.1% to 38.6%.
  • Correctness-only rewards are the limiting factor: a binary reward baseline matches RLPF's correctness but leaves more than twice the share of slow correct programs (36.6% vs 15.0%).
  • Verified model-generated references can substitute for curated expert references with only a partial loss (50.3% vs 54.6% correct-and-runnable), extending the method to settings where expert implementations are unavailable.
  • The learned speed preference transfers modestly beyond the training benchmark: 3.9% geometric-mean execution-time improvement and a 57.9% win rate on EffiBench-X, which the paper notes does not establish statistical significance.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Generalizing the recipe: any RL setting with sparse, late-arriving feedback could use the same structure — an ordinal ladder over failure states plus a task-normalized continuous reward at the top — applied to other performance axes such as memory use, energy, or binary size.
  • The paper's own training curves show the full reward peaks mid-training and declines toward the final step, so the reported results depend on the EMA-peak checkpoint rule; a natural stress test is whether the gains survive always taking the last checkpoint or a fixed step budget.
  • CGRE is reference-relative, not absolute: the paper's case studies show a 1.27× speedup scoring 1.00 CGRE while a 1.01× speedup scores 0.01, depending on how far the expert reference sits from the baseline. Teams adopting the metric should audit reference quality before interpreting CGRE deltas as absolute speedups.
  • The transition analysis shows RLPF loses correctness on 20 tasks the correctness-only baseline solves — a speed-for-correctness tradeoff the paper reports but does not study in depth; an adaptive reward weight holding the correctness floor high early in training is a concrete extension worth testing.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes RLPF, a reinforcement-learning reward for code generation that replaces a binary correctness signal with a staged reward. Failed rollouts are ordered by execution progress (NX, NC, NR, WO), while correct rollouts are scored by CGRE, a normalized measure of how much of the gap between a provided baseline and an expert reference the candidate closes. The method is instantiated by fine-tuning Qwen3-32B with GRPO and LoRA on the PerfCodeBench training split; the authors report that correct-and-runnable rate rises from 11.1% to 54.6% and CGRE from 8.1% to 38.6% on the test split. The paper also reports ablations, a model-generated-reference variant, case studies, and a transfer experiment to EffiBench-X.

Significance. If the reported results hold, this is a useful step beyond correctness-only code RL: it demonstrates that a normalized, correctness-gated efficiency reward can change the preference of a code model among correct programs. The ablation structure is informative, the release of code and model weights is a clear strength, and the transfer experiment, while modest, provides some evidence that the learned behavior is not entirely tied to the training harness. However, the headline efficiency gains rest on runtime measurements whose noise properties are not quantified, and the main in-distribution evaluation uses the same normalized quantity (CGRE) that is optimized during training on a benchmark co-constructed by the same group. These concerns materially affect the strength of the central claim as stated.

major comments (4)
  1. [§Problem Setup, Eq. (3); §Training Setup; §Limitations] The headline CGRE gain (8.1% to 38.6%) depends on wall-clock runtime as both reward and evaluation signal. During training, each rollout is executed once; during evaluation, the median of K=3 runs is used. The number of repetitions used for the baseline runtime T_b and reference runtime T_r is not reported. Since CGRE divides by (T_b − T_r), tasks with small baseline-to-reference gaps are hypersensitive to single-run timing noise. The paper's Limitations paragraph acknowledges that timing and test suites are imperfect, but no robustness analysis quantifies this. Please report (i) the distribution of T_b − T_r across tasks, (ii) how many timing repetitions were used for T_b and T_r, and (iii) a stability analysis, e.g., evaluating with different numbers of repeats or recomputing rewards under perturbed timings. Without this, the efficiency improvement could be substantially inflated by me
  2. [§Experiment Results, Table 1; §Baseline and Ablation Setup; §Reward Baselines and Component Ablations] All models are trained once and all checkpoints are selected at the EMA peak of the training reward curve. No seed variance or error bars are reported. The paper itself states that point estimates 'cannot establish component necessity' and that the EffiBench-X transfer 'does not establish statistical significance.' This is a particular concern for the small differences in Table 2 (e.g., RLPF +3.9% vs. w/o FBR −0.3%) and for the component-ablation comparisons in Table 1. Please provide multiple seeds (or at least task-level bootstrap confidence intervals) and report final-checkpoint or non-peak results, so that checkpoint selection and training noise are not confounded with reward design.
  3. [§Metrics; §Problem Setup; §Experiment Setup] The reward for correct rollouts is exactly the headline evaluation metric CGRE (Eq. 3), and both training and in-distribution evaluation use PerfCodeBench, which was constructed by overlapping authors (Jing et al. 2026). This makes the in-distribution improvement partly self-consistent: the model is optimizing the precise quantity used to report success. The EffiBench-X experiment partly addresses this, but its gain is modest (+3.9%). Please discuss the risk of overfitting to the benchmark's specific baseline/reference pairs and, if possible, report results on an independently constructed efficiency benchmark or with a held-out harness. Even a careful qualitative analysis of which optimization patterns transfer would help separate true behavioral change from benchmark-specific reward hacking.
  4. [§Training Setup; §Limitations] Several hand-set constants are load-bearing for the reward: α_F=0.16, α_R=0.10, the failure-mode rewards s(σ), the correctness gap ε=0.05, and the GRPO group size G=8. No sensitivity analysis is reported, and the paper's Limitations notes that RLPF requires a meaningful baseline-reference gap. A small sweep over α_F and α_R, or over the failure-mode reward ordering, would clarify whether the reported gains are robust or concentrated at a single hyperparameter point. This also matters for the claim that the full composite reward is 'more reliable' than its parts.
minor comments (5)
  1. [Abstract] There are several missing spaces in the abstract (e.g., 'Itismeaningfulonlyafteraprogramiscorrect'), which makes the text hard to read in the provided version. Please fix the formatting.
  2. [§Training Setup] The timing protocol is under-specified: please state the number of repetitions for T_b and T_r, whether the harness runs on the same machine as training, and whether any warm-up or outlier filtering is applied.
  3. [§Baseline and Ablation Setup] The text says the reward baselines and ablations are trained with 'the same seed,' but it never specifies the seed or reports whether any seed was varied. Please state the seed explicitly and, ideally, provide results for one more seed for the main variants.
  4. [Appendix, Table 12] The transition analysis is useful, but the column header 'Reference' is ambiguous. Clarify which models are the reference models and that transitions are always measured on the same 306 test tasks.
  5. [§Metrics] The definition of 'CGRE≥0.8' says it is computed over all test tasks, which is clear, but the main text sometimes discusses CGRE as if it were a conditional metric. Please consistently distinguish the conditional average over correct programs from the unconditional metric used in Table 1.

Circularity Check

2 steps flagged

PerfCodeBench CGRE is the training reward itself; primary evaluation is on an overlapping-authors benchmark, but EffiBench-X gives partial independent grounding.

specific steps
  1. self definitional [Success-Mode Reward (Eqs. 3–4); Metrics; Abstract]
    "CGRE_i = C_i · clip( (T_i^b − T_i^c)/(T_i^b − T_i^r), 0, 1 ) ... The success-mode reward is R^succ_i = CGRE_i + α_F FBR_i + α_R RBR_i. (4) ... For PerfCodeBench, we report CRR, FBR, RBR, CGRE, and CGRE≥0.8."

    The headline PerfCodeBench result 'relative efficiency from 8.1% to 38.6%' is measured by CGRE, and Eq. (4) makes CGRE the dominant term of the training reward. The same executable harness computes both the reward used in GRPO and the reported test metric, so the in-distribution CGRE gain mainly shows that the policy increased the exact scalar it was optimized against. This is not an independent test that the model prefers faster code; it is a report of the optimized objective. Partial independence comes only from the EffiBench-X transfer and from reward-variant comparisons, which still use the same CGRE metric on the same benchmark.

  2. self citation load bearing [Experiment Setup, Benchmarks and Evaluation Protocol]
    "We use PerfCodeBench (Jing et al. 2026) for both training and in-distribution evaluation. ... It matches the setting targeted by RLPF: each task asks the model to produce a drop-in implementation, which is then checked for correctness and compared with a baseline and an expert reference for efficiency."

    PerfCodeBench is a companion benchmark by the same first author and overlapping co-authors, and its baseline/expert-reference structure is exactly the input that defines RLPF's CGRE reward. Using it as the primary in-distribution test imports the benchmark's design assumptions rather than subjecting them to an external check. The paper's only independent evidence is the EffiBench-X transfer, which it describes as modest; thus the main headline result is partly self-referential.

full rationale

The staged-reward and GRPO derivation is self-contained, and the paper honestly exposes the reward equations. The principal circularity is metric/reward overlap: the PerfCodeBench CGRE evaluation metric is the same normalized quantity that Eq. (4) optimizes, so the headline 'relative efficiency' gain on that benchmark is partially by construction. In addition, the primary benchmark is an overlapping-authors artifact whose baseline/reference design is the same setup RLPF assumes, weakening the independence of the in-distribution evaluation. The claim is not wholly circular: the policy could fail to improve, the test split is family-disjoint, RLVR and runtime-only baselines share the same evaluation protocol, and EffiBench-X provides an external transfer check with a different protocol. The Limitations acknowledge timing and oracle imperfections but do not quantify their effect on the single-run training-time reward. On balance, score 4.

Axiom & Free-Parameter Ledger

8 free parameters · 5 axioms · 0 invented entities

The central claim rests on hand-set reward constants, the quality of PerfCodeBench's baselines/references/oracles, and the stability of runtime measurements. No new physical or model entities are introduced.

free parameters (8)
  • alpha_F = 0.16
    Hand-chosen weight for the faster-than-baseline bonus in Eq. (4); no sensitivity analysis reported.
  • alpha_R = 0.10
    Hand-chosen weight for the reference-or-better bonus in Eq. (4); no sensitivity analysis.
  • failure-mode rewards s(σ) = -0.05, 0.00, +0.05, +0.10
    Manually assigned reward values in Eq. (5) to encode the assumed NX<NC<NR<WO order.
  • epsilon correctness gap = 0.05
    Manually chosen gap between worst correct reward and best failed reward in Eq. (6).
  • GRPO group size G = 8
    Samples per prompt; affects variance of group-relative advantages (Training Setup).
  • LoRA rank/alpha = rank 32, alpha 64
    Adapter hyperparameters used for all runs; part of the training recipe.
  • EMA checkpoint selection alpha = 0.99
    Checkpoint chosen at EMA peak of training reward (Checkpoint Selection, Table 9); this selection rule affects final point estimates.
  • max completion length = 768 tokens
    Cap on rollout length; if too short, correct solutions may be truncated.
axioms (5)
  • domain assumption Median wall-clock runtime is a stable, comparable proxy for code efficiency across tasks and languages.
    Used implicitly in Eqs. (1)-(3) and evaluation; Limitations concedes timing and benchmark suites are imperfect and cites Le et al. 2026.
  • domain assumption Each task's baseline and expert reference define a meaningful optimization gap that candidate programs should be measured against.
    Problem Setup states every task provides a baseline and expert reference; if these are weak or misaligned, CGRE normalizes by an arbitrary gap.
  • domain assumption Passing the oracle tests indicates functional correctness for the task.
    All rewards and metrics gate on the correctness oracle; Limitations notes benchmark test suites are imperfect for heterogeneous systems tasks.
  • ad hoc to paper The ordering NX<NC<NR<WO is a useful learning signal for climbing toward correctness.
    Eq. (5) assigns monotonically increasing rewards to execution states; this is a designer choice with no theoretical or empirical proof.
  • standard math GRPO's group-relative normalization provides a valid optimization signal in this setting.
    The method builds on GRPO (Shao et al. 2024); standard theory assumed.

pith-pipeline@v1.3.0-daily-deepseek · 18149 in / 13072 out tokens · 129463 ms · 2026-08-01T10:46:36.123659+00:00 · methodology

0 comments
read the original abstract

Code models are increasingly trained with execution feedback, but most training signals still stop at correctness. This leaves an important gap for systems code: two programs can pass the same tests while differing greatly in runtime. We study how to train code agents to prefer faster correct implementations, rather than treating efficiency only as an evaluation metric. The key difficulty is that runtime is a fragile reward. It is meaningful only after a program is correct, varies across tasks, and gives little guidance when most sampled programs fail to compile or run. We propose \textbf{RLPF}, reinforcement learning from performance feedback, which turns execution outcomes into a staged reward. Failed programs are ordered by execution progress, while correct programs are ranked by their relative improvement from the baseline toward the expert reference. This gives useful feedback before correctness and performance-sensitive feedback after correctness. Fine-tuning Qwen3-32B with RLPF on PerfCodeBench raises correct-and-runnable solutions from $11.1\%$ to $54.6\%$ and improves relative efficiency from $8.1\%$ to $38.6\%$. The trained model becomes competitive with stronger open-weight systems, and its optimization behavior transfers modestly to EffiBench-X. Additional studies show that model-generated references provide useful but weaker supervision, and that the full composite reward is more reliable than correctness-only or runtime-only baselines. These results suggest that code agents can be trained not only to pass tests, but also to optimize the programs they write.

Figures

Figures reproduced from arXiv: 2607.27271 by Changxuan Fan, Hanyu Yang, Haochen Shi, Haoran Li, Haozhe Cui, Huihao Jing, Shaojin Chen, Sirui Zhang, Wenbin Hu, Yangqiu Song, Yuxuan Liu, Ziyi Chen.

Figure 1
Figure 1. Figure 1: From correctness-only rewards to correctness-gated performance feedback. RLVR rewards correctness alone and cannot distinguish fast correct programs from slow correct programs. A naive speedup reward uses runtime improvement as the main signal, but provides little feedback before correctness. RLPF decomposes execution outcomes into an ordered reward structure: failed rollouts receive staged shaping feedbac… view at source ↗
Figure 2
Figure 2. Figure 2: Training dynamics under each reward’s native [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

24 extracted references · 19 linked inside Pith

  1. [2]

    Dou, Z.; Zhao, Q.; Wan, Z.; Xia, X.; and Biswas, S

    Evaluating Large Language Models Trained on Code.arXiv preprint arXiv:2107.03374. Dou, Z.; Zhao, Q.; Wan, Z.; Xia, X.; and Biswas, S

    Pith/arXiv arXiv

  2. [5]

    Hendrycks,D.;Basart,S.;Kadavath,S.;Mazeika,M.;Arora, A.; Guo, E.; Burns, C.; Puranik, S.; He, H.; Song, D.; and Steinhardt, J

    EffiPair: Improving the EfficiencyofLLM-generatedCodewithRelativeContrastive Feedback.arXiv preprint arXiv:2604.05137. Hendrycks,D.;Basart,S.;Kadavath,S.;Mazeika,M.;Arora, A.; Guo, E.; Burns, C.; Puranik, S.; He, H.; Song, D.; and Steinhardt, J

    Pith/arXiv arXiv

  3. [6]

    InAdvances in Neural Information Pro- cessing Systems

    Measuring Coding Challenge Compe- tence with APPS. InAdvances in Neural Information Pro- cessing Systems. Hu,E.J.;Shen,Y.;Wallis,P.;Allen-Zhu,Z.;Li,Y.;Wang,S.; Wang,L.;andChen,W.2022. LoRA:Low-RankAdaptation of Large Language Models. InInternational Conference on Learning Representations. Huang,D.;Qing,Y.;Shang,W.;Cui,H.;andZhang,J.2024. EffiBench: Benchm...

    2022

  4. [8]

    arXiv:2605.15222

    PerfCodeBench: Benchmarking LLMs for System-Level High-Performance Code Optimiza- tion. arXiv:2605.15222. Kumarappan,A.;Golnari,P.A.;Wen,W.;Liu,X.;Ryan,G.; Sun, Y.; Fu, S.; and Nallipogu, E

    Pith/arXiv arXiv

  5. [9]

    Le, H.; Wang, Y.; Gotmare, A

    DevBench: A Real- istic, Developer-Informed Benchmark for Code Generation Models.arXiv preprint arXiv:2601.11895. Le, H.; Wang, Y.; Gotmare, A. D.; Savarese, S.; and Hoi, S. C. H

    Pith/arXiv arXiv

  6. [11]

    Li, X.; Wang, A.; Wang, G.; Li, J.; and Shum, C

    Re- thinking Code Performance Benchmarks for LLMs.arXiv preprint arXiv:2607.07619. Li, X.; Wang, A.; Wang, G.; Li, J.; and Shum, C

    Pith/arXiv arXiv

  7. [12]

    Liu, J.; Xia, C

    CUDA-L1: Improving CUDA Optimization via Contrastive Reinforcement Learning.arXiv preprint arXiv:2507.14111. Liu, J.; Xia, C. S.; Wang, Y.; and Zhang, L. 2023a. Is Your CodeGeneratedbyChatGPTReallyCorrect?RigorousEval- uationofLargeLanguageModelsforCodeGeneration.arXiv preprint arXiv:2305.01210. Liu, J.; Xie, S.; Wang, J.; Wei, Y.; Ding, Y.; and Zhang, L

    Pith/arXiv arXiv

  8. [13]

    InConference on Language Modeling

    Evaluating Language Models for Efficient Code Generation. InConference on Language Modeling. Liu,J.;Zhu,Y.;Xiao,K.;Fu,Q.;Han,X.;Yang,W.;andYe, D. 2023b. RLTF: Reinforcement Learning from Unit Test Feedback.arXiv preprint arXiv:2307.04349. Mikasa, R.; Hayashi, S.-i.; Mukunoki, D.; Hoshino, T.; and Katagiri, T

    Pith/arXiv arXiv

  9. [14]

    OpenAI.2026

    Improving HPC Code Generation Capability of LLMs via Online Reinforcement Learning with Real-Machine Benchmark Rewards.arXiv preprint arXiv:2602.12049. OpenAI.2026. Codex. https://openai.com/codex/. Accessed July 27,

    arXiv 2026

  10. [15]

    Ouyang, A.; Guo, S.; Arora, S.; Zhang, A

    MaxCode: A Max- Reward Reinforcement Learning Framework for Automated Code Optimization.arXiv preprint arXiv:2601.05475. Ouyang, A.; Guo, S.; Arora, S.; Zhang, A. L.; Hu, W.; Ré, C.;andMirhoseini,A.2025. KernelBench:CanLLMsWrite Efficient GPU Kernels?arXiv preprint arXiv:2502.10517. Peng, Y.; Gotmare, A. D.; Lyu, M.; Xiong, C.; Savarese, S.; and Sahoo, D

    arXiv 2025

  11. [16]

    Peng, Y.; Wan, J.; Li, Y.; and Ren, X

    PerfCodeGen: Improving Performance of LLM Generated Code with Execution Feedback.arXiv preprint arXiv:2412.03578. Peng, Y.; Wan, J.; Li, Y.; and Ren, X

    Pith/arXiv arXiv

  12. [17]

    Qiu,R.;Zeng,W.W.;Ezick,J.;Lott,C.;andTong,H.2025

    EffiBench-X: A Multi-Language Benchmark for Measuring Efficiency of LLM-Generated Code.CoRR, abs/2505.13004. Qiu,R.;Zeng,W.W.;Ezick,J.;Lott,C.;andTong,H.2025. How Efficient is LLM-Generated Code? A Rigorous and High-Standard Benchmark. InInternational Conference on Learning Representations. Schulman, J.; Wolski, F.; Dhariwal, P.; Radford, A.; and Klimov,O...

    Pith/arXiv arXiv 2025

  13. [18]

    Shen, B.; Zhang, J.; Chen, T.; Zan, D.; Geng, B.; Fu, A.; Zeng,M.;Yu,A.;Ji,J.;Zhao,J.;Guo,Y.;andWang,Q.2023

    DeepSeekMath: Pushing the Limits of Mathemati- cal Reasoning in Open Language Models.arXiv preprint arXiv:2402.03300. Shen, B.; Zhang, J.; Chen, T.; Zan, D.; Geng, B.; Fu, A.; Zeng,M.;Yu,A.;Ji,J.;Zhao,J.;Guo,Y.;andWang,Q.2023. PanGu-Coder2: Boosting Large Language Models for Code with Ranking Feedback.arXiv preprint arXiv:2307.14936. Shypula, A.; Madaan, ...

    Pith/arXiv arXiv 2023

  14. [20]

    Wang, X.; Li, B.; Song, Y.; Xu, F

    Improving Small Language Models for Code Generation with Reinforce- ment Learning from Verification Feedback.arXiv preprint arXiv:2605.30478. Wang, X.; Li, B.; Song, Y.; Xu, F. F.; Tang, X.; Zhuge, M.; Pan,J.;Song,Y.;Li,B.;Singh,J.;Tran,H.H.;Li,F.;Ma,R.; Zheng, M.; Qian, B.; Shao, Y.; Muennighoff, N.; Zhang, Y.; Hui, B.; Lin, J.; et al

    Pith/arXiv arXiv

  15. [21]

    OpenHands: An Open Platform for AI Software Developers as Generalist Agents. InICLR. OpenReview.net. Wei, A.; Suresh, T.; Tan, H.; Xu, Y.; Singh, G.; Wang, K.; andAiken,A.2025. SuperCoder:AssemblyProgramSuper- optimization with Large Language Models.arXiv preprint arXiv:2505.11480. Wu, J.; Li, H.; Zhang, X.; Guo, J.; Luo, J.; Liu, S.; Huang, Y.; Chu, R.; ...

    arXiv 2025

  16. [22]

    Qwen3 Technical Report.arXiv preprint arXiv:2505.09388. Language Train Val Test Total C++ 869 55 163 1087 C 114 16 32 162 CUDA 93 16 16 125 Go 112 16 32 160 Java 128 16 32 176 Python 97 16 31 144 Total 1413 135 306 1854 Table 3: Language distribution of PerfCodeBench. Yang, J.; Jimenez, C. E.; Wettig, A.; Lieret, K.; Yao, S.; Narasimhan, K.; and Press, O

    Pith/arXiv arXiv

  17. [23]

    Zhu, X.; Zhou, X.; Zhu, B.; Hu, H.; Du, M.; Zhang, H.; Wang, H.; and Guo, Z

    Large Language Model-Powered Evolu- tionary Code Optimization on a Phylogenetic Tree.arXiv preprint arXiv:2601.14523. Zhu, X.; Zhou, X.; Zhu, B.; Hu, H.; Du, M.; Zhang, H.; Wang, H.; and Guo, Z

    arXiv

  18. [24]

    arXiv preprint arXiv:2602.17684

    CodeScaler: Scaling Code LLM Training and Test-Time Inference via Reward Models. arXiv preprint arXiv:2602.17684. Zhuo,T.Y.;Vu,M.C.;Chim,J.;Hu,H.;Yu,W.;Widyasari, R.; Yusuf, I. N. B.; Zhan, H.; He, J.; Paul, I.; Brunner, S.; Gong, C.; Hoang, J.; Zebaze, A. R.; Hong, X.; Li, W.; Kad- dour, J.; Xu, M.; Zhang, Z.; Yadav, P.; et al

    Pith/arXiv arXiv

  19. [2021]

    arXiv preprint arXiv:2108.07732

    Program Synthesis with Large Language Models. arXiv preprint arXiv:2108.07732. Chen, M.; Tworek, J.; Jun, H.; Yuan, Q.; Pinto, H. P. d. O.; Kaplan, J.; Edwards, H.; Burda, Y.; Joseph, N.; Brockman, G.; Ray, A.; Puri, R.; Krueger, G.; Petrov, M.; Khlaaf, H.; Sastry,G.;Mishkin,P.;Chan,B.;Gray,S.;Ryder,N.;Pavlov, M.; Power, A.; Kaiser, L.; Bavarian, M.; Wint...

    Pith/arXiv arXiv

  20. [2022]

    CodeRL: Mastering Code Generation through PretrainedModelsandDeepReinforcementLearning.arXiv preprint arXiv:2207.01780. Le, N. M.; Xu, Y.; Wang, Z.; and Chen, T.-H. P

    Pith/arXiv arXiv

  21. [2023]

    Skopin, E.; and Kotelnikov, E

    Learning Performance-Improving Code Edits.arXiv preprint arXiv:2302.07867. Skopin, E.; and Kotelnikov, E

    Pith/arXiv arXiv

  22. [2024]

    Guo, D.; Yang, D.; Zhang, H.; et al

    RLEF: Ground- ing Code LLMs in Execution Feedback with Reinforcement Learning.arXiv preprint arXiv:2410.02089. Guo, D.; Yang, D.; Zhang, H.; et al

    Pith/arXiv arXiv

  23. [2025]

    Jimenez, C

    CodeRL+: Improving Code Generation via Reinforcement with Execution Semantics Alignment.arXiv preprint arXiv:2510.18471. Jimenez, C. E.; Yang, J.; Wettig, A.; Yao, S.; Pei, K.; Press, O.; and Narasimhan, K. R

    Pith/arXiv arXiv

  24. [2026]

    Du,M.;Luu,A.T.;Ji,B.;Liu,Q.;andNg,S.-K.2024

    CoRe-Code:CollaborativeReinforcementLearningforCode Generation.arXiv preprint arXiv:2605.24812. Du,M.;Luu,A.T.;Ji,B.;Liu,Q.;andNg,S.-K.2024. Mer- cury: A Code Efficiency Benchmark for Code Large Lan- guage Models.arXiv preprint arXiv:2402.07844. Gehring, J.; Zheng, K.; Copet, J.; Mella, V.; Carbonneaux, Q.; Cohen, T.; and Synnaeve, G

    Pith/arXiv arXiv 2024