Pith.

Computation and Language · arXiv:2609.03955 v1 · submitted 3 Sep 2026 · 57 references

Two-Stage Reinforcement Learning for Sound and Adversarial Test Generation in Code LLMs

Jiacheng Xu, Wentao Zhang, Zhiyi Lyu, Fuxiang Zhang, Chaojie Wang, Yang Liu, Bo An

The paper says

“Reinforcement learning (RL) has substantially advanced code generation with large language models (LLMs) through executable feedback.”

more of the abstract

The feedback for coding problems mainly comes from specific test cases, where high-quality test cases are often scarce since they should be both sound and discriminative. We thus turn to study the auto-generation of test cases using the learned model. We find this is naturally an adversarial RL problem: the model is expected to generate effective test cases as counterexamples, depending on the solver's current failure modes. We propose Test Cases Scaling (TCS), a two-stage RL framework for effective test generation. Both stages train a test generator from a rolling policy-aligned buffer: Stage 1 generates tests consistent with the reference solution, and Stage 2 restricts the buffer to current failure modes and learns counterexample tests. Across TACO and LiveCodeBench, TCS improves both pass@1 and inference-time answer selection according to generated tests. We find the learned test generator also enables effective selection among other LLM outputs.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 8 Sep 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

The passes disagreed. Reader: holds up. Skeptic: holds up only with conditions. Referee: major revision.

Novel two-stage RL for test generation, but the theoretical reliability bound assumes test inputs satisfy problem constraints, which the training rewards do not enforce—so the formal guarantee may not hold. This line is wrong

1 major objection · 6 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 57 references say.

The paper says, at Section 3.4 (Proposition 1) and Equation (3), Equation (4), buffer admission (Section 3.2.1)

The review did not quote this passage.

Pith says: Objection 1 of 1 · major

The theoretical bound (Proposition 1) explicitly assumes that 'every generated test input satisfies the input constraints of P.' However, the reward functions R_t1 (Eq. 3) and R_t2 (Eq. 4) score a test solely based on execution output against C* (plus a non-copy check) and buffer admission checks only executability and incorrectness.

more of objection 1

Nothing in the training pipeline verifies that the generated input respects the problem's constraints (e.g., bounds on integers, non-negativity, specific format). A test with a constraint violation can still receive reward 1 if C* happens to produce the predicted output; R_t2 additionally rewards runtime errors, which invalid stress inputs can easily trigger. At inference, self-generated tests feed Eq. (5) with no C* and no constraint check. The paper reports no invalid-input rate on TACO or LiveCodeBench, so it is unknown whether the empirical selection gains partly stem from a test distribution that violates the theory's core assumption. This is a load-bearing gap between the reward specification and the formal guarantee. The authors should either (a) add a constraint-verification step to the reward or buffer admission, or (b) empirically show that the fraction of invalid inputs is negligible and that the bound's qualitative prediction still holds. Without this, the paper's 'soundness' claim and the theoretical justification for inference-time selection are not fully supported.

6 minor objections

The paper says, at Section 4.2, Table 1

Pith says: minor objection 1 of 6

The caption says 'Pass@1' but the table reports pass@1 values under various selection methods (BoN with reward model or self-generated tests). The pass@1 row for each model is actually the base pass@1 without selection. Clarify the column headers: '+ Reward Model' and '+ Self-Generated Test Cases' are Best-of-N results, not pass@1. This could confuse readers.

The paper says, at Section 4.4, Figure 4(b)

Pith says: minor objection 2 of 6

The accuracy numbers for R1-Distill-1.5B (2.1%) seem extraordinarily low. Is this a typo? If the base model cannot predict any test output correctly, how can self-generated tests be useful? Please verify and correct.

show 4 more

The paper says, at Section 3.2.1, Buffer size T_b

Pith says: minor objection 3 of 6

The rolling buffer size T_b (number of recent training steps) is not reported in the main text or appendix. Please specify this hyperparameter, as it affects the diversity and recency of failure modes seen by the verifier.

The paper says, at Section 3.2.2, Equation (3)

Pith says: minor objection 4 of 6

The non-copy condition (I_g, O_g) ∉ T_example is mentioned, but it is unclear whether exact duplication of example test inputs is rare. The paper should report how often this filter fires during training (e.g., what fraction of test generations are penalized for copying).

The paper says, at Section 4.2, Appendix Table 4

Pith says: minor objection 5 of 6

Table 4 shows that Test-RL (test-only RL) improves selection but not pass@1, while Code-RL (code-only RL) improves pass@1 but not selection. This is a nice ablation, but the paper should also report whether the test generator from Test-RL produces tests that are sound (constrained-valid) at a comparable rate to TCS. This would help isolate whether the RL objective alone or the staged training drives input validity.

The paper says, at Appendix A.2, Transition threshold

Pith says: minor objection 6 of 6

The Stage 1→2 transition is a hard switch based on test-generation accuracy reaching ~0.75. The paper should justify this threshold (e.g., learning curve knee) and show sensitivity to it. Could small variations significantly change the results?

What would settle it

Sample a held-out set (about 200 TACO-validation and 100 LiveCodeBench problems). For each, run the final TCS-7B test generator under the exact inference protocol (M = 1 test per candidate). Parse the generated JSON tests and implement lightweight constraint validators from each problem statement (input bounds, value ranges, etc.). Count the fraction of generated tests whose input violates the stated constraints, separately for tests that C* passes. Then rerun Best-of-N selection after discarding invalid tests; if the invalid fraction is non-negligible or the gains shrink materially, the reward does not enforce the assumption used in Proposition 1, and TCS should add an explicit validity check or penalty.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 8 Sep 2026

  1. Reader holds up confidence moderate
  2. Skeptic partly agrees: says it holds up only with conditions 3 min 24 s · 23,970 tokens read, 22,300 written No mechanism enforces that generated test inputs satisfy problem constraints, although the reliability bound explicitly assumes they do.
  3. Editor wrote the verdict above 17 s · 24,657 tokens read, 1,961 written
  4. Referee major revision: 1 major objection, 6 minor 18 s · 24,154 tokens read, 1,839 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

Hey [colleague],

This paper proposes TCS, a two-stage reinforcement learning framework that jointly trains a code LLM to both solve problems and generate test cases. Stage 1 trains the verifier to produce tests that are sound w.r.t. a ground-truth solution C*. Stage 2 trains it to generate candidate-conditioned counterexample tests. The key novelty is the staged curriculum with a policy-aligned buffer that tracks the solver's current failure modes. This is a real contribution: training a verifier that learns to target the model's weaknesses online, rather than from offline data.

The experiments are solid. TCS consistently improves pass@1 and inference-time selection on TACO and LiveCodeBench for both 1.5B and 7B models. Ablations show that both stages matter, joint training beats decoupled training, and self-generated tests beat reward-model ranking on some settings. The paper is clearly written and the reproducibility seems good (code and checkpoints released). The theoretical bound in Sec. 3.4 is a nice addition, though not surprising: exponential reliability when the margin δ−α is positive.

Now the soft spots. The stress-test concern about input validity is legitimate. Proposition 1 (and Appendix B) explicitly assumes that every generated test input satisfies the problem's input constraints. But the training rewards (Eqs. 3 and 4) score tests solely by execution against C* (and optionally failing C_wrong). There is no reward signal or buffer filter for whether the generated input actually respects constraints like bounds, non-negative integers, etc. So the bound may not apply to the actual test generator—its empirical performance does not necessarily come from the theoretical margin. The paper does not report invalid-input rates, so we cannot assess how often the assumption is violated. This weakens the claim that TCS's gains are attributable to soundness in the formal sense. It is not a paper-killer, but it is a gap between the formal guarantee and the reward specification.

Other limitations: the reliance on ground-truth solutions during training (acknowledged), the heuristic hard transition threshold, and the fact that the method only generates one test per inference call. These are reasonable constraints given the setting.

Overall: this is a well-motivated, empirically strong paper. The stress-test input-validity gap should be addressed in revision—either by adding constraint checking to the reward, measuring violation rates, or relaxing the theory. The paper deserves a serious referee.

I would bring this to reading group and would cite it if I work on code verification. Recommendation: accept conditional on addressing the constraint validity issue.

The strongest objection

Proposition 1 (Sec. 3.4) and Appendix B.1 assume that every generated test input satisfies the constraints of P. For the bound to justify inference-time selection, the TCS generator must therefore produce valid inputs. However, the training rewards R_t1 (Eq. 3) and R_t2 (Eq. 4) score a test solely by Exec(C*, I) = O plus a non-copy check, and buffer admission checks only executability and incorrectness. Nothing in the reward, the buffer, or the inference-time pipeline verifies input validity. A test with n outside the stated bounds, negative values where positives are required, or other constraint violations can receive reward 1 whenever C* happens to emit the predicted output; R_t2 additionally rewards runtime errors and timeouts, which invalid stress inputs can easily trigger. At inference, self-generated tests feed Eq. (5) with no C* and no constraint check. The paper reports no invalid-input rate on TACO or LiveCodeBench, so the empirical selection gains may partly come from a distribution that violates the theory's central assumption and weakens the paper's 'soundness' claim. This is a load-bearing gap between the reward specification and the formal guarantee.

The paper's central claim is well supported empirically: the two-stage RL setup, policy-aligned buffer, matched-baseline ablations, and monotonic improvements with more tests per candidate all point in the same direction. The reader's identified weakest assumption, access to C* during training, is real but is explicitly scoped and shared by all compared methods; it limits applicability without undermining the comparison. The more load-bearing issue is a mismatch between the formal guarantee and the training objective. Proposition 1 depends on inputs satisfying problem constraints, yet neither reward function nor the buffer verifies validity, and Stage 2 even rewards robustness failures such as timeouts. This opens a concrete reward-hacking channel that could produce 'counterexamples' that are not valid tests, inflating both training reward and inference-time selection in a way the theory does not cover. Because the paper does not measure invalid-test frequency, the soundness of the deployed verifier is unverified. A direct validity audit on held-out problems would settle the issue; if invalid tests are rare, the current verdict stands, but until then a conditional acceptance asking for this measurement is the honest recommendation.

The referee's report

If the claims hold, TCS provides a principled method for improving code generation via learned test generation, with theoretical grounding and strong empirical results across two benchmarks and two model sizes. The staged curriculum (soundness then adversariality) and the policy-aligned buffer are novel contributions. The paper's approach to inference-time scaling using self-generated tests is timely and practically relevant. However, the significance is partially contingent on whether the theoretical guarantee's central assumption—that generated test inputs satisfy problem constraints—actually holds in the implementation, which is not verified.

Pith's reading of the paper (a machine's summary, not the authors')

The paper argues that effective test generation for code LLMs requires two properties: soundness (tests pass the correct solution) and candidate-conditioned adversariality (tests expose failures in incorrect solutions). The authors propose Test Cases Scaling (TCS), a two-stage RL training process. Stage 1 trains the model to produce sound tests by rewarding consistency with a ground-truth solution. Stage 2 shifts the focus to counterexample tests by restricting training to the solver's current incorrect outputs and rewarding tests that pass the correct solution but fail the incorrect one. Jointly training the solver and verifier with a rolling policy-aligned buffer yields improvements in both pass@1 and inference-time answer selection on TACO and LiveCodeBench. The paper also derives an exponential reliability bound showing that test-based selection becomes more reliable as the soundness error decreases and the counterexample rate increases.

What carries the argument. The central mechanism is the two-stage reinforcement learning pipeline using Group Relative Policy Optimization (GRPO), with stage-specific rewards and a rolling policy-aligned buffer. Stage 1 uses a soundness reward ($R_t^{1}$) that awards 1 if the generated test passes the ground-truth solution and is not an exact copy of example tests. Stage 2 uses an adversarial reward ($R_t^{2}$) that awards 1 only if the test passes the ground-truth solution and fails an incorrect candidate from the buffer. The buffer stores the solver's outputs from recent training steps, keeping the verifier aligned with current failure modes.

Core claim. The paper discovers that a staged curriculum—first teaching a test generator to produce sound tests against a ground-truth solution, then teaching it to produce candidate-conditioned counterexamples against the solver's current failure modes—substantially outperforms joint supervised fine-tuning, code-only RL, and test-only RL in improving both code generation accuracy and inference-time selection with self-generated tests. The rolling policy-aligned buffer, which dynamically tracks the solver's evolving errors, is essential for this curriculum.

The premise it rests on. The method assumes access to a ground-truth solution C* during training to verify the soundness of generated tests, limiting the approach to post-training scenarios where such verified solutions are available.

If the paper is right (Pith's inferences, not the authors')

If the TCS framework is broadly adopted, code LLMs can become self-improving by generating their own test cases, reducing reliance on expensive human-written or curated test suites.

The staged approach (soundness first, then adversariality) provides a template for training verification abilities in other domains where correctness can be checked against a reference, such as mathematical reasoning or formal verification.

The exponential reliability bound (Proposition 1) suggests that even a modest increase in the counterexample rate δ or decrease in soundness error α can dramatically improve inference-time selection, providing a theoretical target for future verifier training.

Since TCS outperforms an offline joint-SFT baseline that uses the same data, the result implies that online, policy-aligned training is crucial for learning effective test generation.

The learned verifier transfers to improve selection among outputs from stronger external LLMs, suggesting that the test-generation capability generalizes beyond self-play.

The method's reliance on a ground-truth solution C* during training could be relaxed by using model-based consistency checks or self-consistency across multiple generated solutions, extending the approach to settings where verified solutions are unavailable.

The core idea of staged adversarial training with a rolling buffer might be applicable to other sequence-generation tasks where a reference is available and the goal is to produce discriminative counterexamples (e.g., question generation for education, test-suite generation for software engineering).

The paper implicitly suggests that inference-time scaling (more tests per candidate) is beneficial, but the practical token cost of generating many tests per candidate may be high; future work could explore amortizing test generation across candidates or using smaller test generators.

The hard switch between stages could be replaced by a soft transition that dynamically mixes the two rewards based on the verifier's current soundness, potentially smoothing learning and reducing the need for a predefined threshold.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Reinforcement learning (RL) has substantially advanced code generation with large language models (LLMs) through executable feedback. The feedback for coding problems mainly comes from specific test cases, where high-quality test cases are often scarce since they should be both sound and discriminative. We thus turn to study the auto-generation of test cases using the learned model. We find this is naturally an adversarial RL problem: the model is expected to generate effective test cases as counterexamples, depending on the solver's current failure modes. We propose Test Cases Scaling (TCS), a two-stage RL framework for effective test generation. Both stages train a test generator from a rolling policy-aligned buffer: Stage 1 generates tests consistent with the reference solution, and Stage 2 restricts the buffer to current failure modes and learns counterexample tests. Across TACO and LiveCodeBench, TCS improves both pass@1 and inference-time answer selection according to generated tests. We find the learned test generator also enables effective selection among other LLM outputs.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-09-23

The authors' main claim, as the classifier picked it out 1.00

“Across TACO and LiveCodeBench, TCS improves both pass@1 and inference-time answer selection according to generated tests.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.04 · states limitation 0.09 · human participants 0.03 · controlled comparison 0.27 · randomized assignment 0.02 · states quantitative result 0.09 · compares with prior results 0.41 · mentions code or data availability 0.03

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Reinforcement learning (RL) has substantially advanced code generation with large language models (LLMs) through executable feedback. background 1.00 0.05 0.03 asserted 1.00
The feedback for coding problems mainly comes from specific test cases, where high-quality test cases are often scarce since they should be both sound and discriminative. gap 0.58 0.11 0.02 asserted 0.98
We thus turn to study the auto-generation of test cases using the learned model. contribution 0.80 0.35 0.03 asserted 0.81
We find this is naturally an adversarial RL problem: the model is expected to generate effective test cases as counterexamples, depending on the solver's current failure modes. interpretation 0.40 0.90 0.07 asserted 0.99
We propose Test Cases Scaling (TCS), a two-stage RL framework for effective test generation. contribution 0.84 0.87 0.13 asserted 0.99
Both stages train a test generator from a rolling policy-aligned buffer: Stage 1 generates tests consistent with the reference solution, and Stage 2 restricts the buffer to current failure modes and learns counterexample tests. method 0.99 0.93 0.10 asserted 1.00
Across TACO and LiveCodeBench, TCS improves both pass@1 and inference-time answer selection according to generated tests. result 1.00 0.94 0.58 asserted 1.00
We find the learned test generator also enables effective selection among other LLM outputs. result 0.95 0.95 0.16 asserted 1.00

What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.97

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-09-23: 1 request, 6,181 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (7)
Figure 1 of the paper
Figure 1. TCS enables adversarial test case generation
Figure 2 of the paper
Figure 2. Overview of the two-stage training process in Test Cases Scaling (TCS).
Figure 3 of the paper
Figure 3. Comparison of different inference-time scal
Figure 4 of the paper
Figure 4. (a) Pass@1 (blue) and BoN (N=8) (green/orange) performance selected by self-generated or TCS-7B
Figure 5 of the paper
Figure 5. Comparison of different inference-time scaling methods for the TCS-fine-tuned 7B model on Live
Figure 6 of the paper
Figure 6. A generated test case that receives a reward in training stage two.
Figure 7 of the paper
Figure 7. A generated test case that can identify an error in the generated code, even though it passes all the public

Figures are the authors', reproduced from the paper.

References (57)
  1. [1] Zheng Cai, Maosong Cao, Haojiong Chen, Kai Chen, Keyu Chen, Xin Chen, Xun Chen, Zehui Chen, Zhi Chen, Pei Chu, et al. 2024. InternLM2 technical report. arXiv preprint arXiv:2403.17297 on PithPith's reviewarXiv
  2. [2] Bei Chen, Fengji Zhang, Anh Nguyen, Daoguang Zan, Zeqi Lin, Jian - Guang Lou, and Weizhu Chen. 2023. CodeT : Code generation with generated tests. In International Conference on Learning Representations (ICLR), Kigali, Rwanda. OpenReview.net on Pith
  3. [3] Lichang Chen, Chen Zhu, Jiuhai Chen, Davit Soselia, Tianyi Zhou, Tom Goldstein, Heng Huang, Mohammad Shoeybi, and Bryan Catanzaro. 2024. ODIN: disentangled reward mitigates hacking in RLHF . In Proceedings of the 41st International Conference on Machine Learning (ICML), volume... on Pith
  4. [4] Yihong Dong, Xue Jiang, Jiaru Qian, Tian Wang, Kechi Zhang, Zhi Jin, and Ge Li. 2025. A survey on code generation with LLM -based agents. arXiv preprint arXiv:2508.00083 on PithPith's reviewarXiv
  5. [6] Jiayi Fu, Xuandong Zhao, Chengyuan Yao, Heng Wang, Qi Han, and Yanghua Xiao. 2025. Reward shaping to mitigate reward hacking in RLHF . arXiv preprint arXiv:2502.18770 on PithPith's reviewarXiv
  6. [8] Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Yu Wu, YK Li, et al. 2024. DeepSeek-Coder : When the large language model meets programming--the rise of code intelligence. arXiv preprint arXiv:2401.14196 on PithPith's reviewarXiv
  7. [10] Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, Kai Dang, et al. 2024. Qwen2.5-Coder technical report. arXiv preprint arXiv:2409.12186 on PithPith's reviewarXiv
  8. [11] Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. 2025. LiveCodeBench : Holistic and contamination free evaluation of large language models for code. In International Conference on Learning Repr... on Pith
  9. [14] Junlong Li, Daya Guo, Dejian Yang, Runxin Xu, Yu Wu, and Junxian He. 2025. CodeIO : Condensing reasoning patterns via code input-output prediction. In Proceedings of the 42nd International Conference on Machine Learning (ICML), volume 267 of Proceedings of Machine Learning Res... on Pith
  10. [15] Kefan Li and Yuan Yuan. 2024. Large language models as test case generators: Performance evaluation and enhancement. arXiv preprint arXiv:2404.13340 on PithPith's reviewarXiv
  11. [16] Rongao Li, Jie Fu, Bo-Wen Zhang, Tao Huang, Zhihong Sun, Chen Lyu, Guang Liu, Zhi Jin, and Ge Li. 2023. TACO : Topics in algorithmic code generation dataset. arXiv preprint arXiv:2312.14852 on PithPith's reviewarXiv
  12. [17] Jonathan Light, Yue Wu, Yiyou Sun, Wenchao Yu, Yanchi Liu, Xujiang Zhao, Ziniu Hu, Haifeng Chen, and Wei Cheng. 2025. SFS : Smarter code space search improves LLM inference scaling. In International Conference on Learning Representations (ICLR), Singapore on Pith
  13. [20] Michael Luo, Sijun Tan, Roy Huang, Ameen Patel, Alpay Ariyak, Qingyang Wu, Xiaoxiang Shi, Rachel Xin, Colin Cai, Maurice Weber, Ce Zhang, Li Erran Li, Raluca Ada Popa, and Ion Stoica. 2025. https://www.together.ai/blog/deepcoder DeepCoder : A fully open-source 14B coder at O3-... on Pith
  14. [22] Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, et al. 2023. Code Llama : Open foundation models for code. arXiv preprint arXiv:2308.12950 on PithPith's reviewarXiv
  15. [23] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. 2024. DeepSeekMath : Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300 on PithPith's reviewarXiv
  16. [26] Parshin Shojaee, Aneesh Jain, Sindhu Tipirneni, and Chandan K. Reddy. 2023. Execution-based code generation using deep reinforcement learning. Transactions on Machine Learning Research on Pith
  17. [27] Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. 2025. Scaling LLM test-time compute optimally can be more effective than scaling parameters for reasoning. In International Conference on Learning Representations (ICLR), Singapore on Pith
  18. [30] Leandro von Werra, Younes Belkada, Lewis Tunstall, Edward Beeching, Tristan Thrush, Nathan Lambert, Shengyi Huang, Kashif Rasul, and Quentin Gallouédec. 2020. https://github.com/huggingface/trl TRL : Transformers reinforcement learning . GitHub repository on Pith
  19. [31] Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. Self-consistency improves chain of thought reasoning in language models. In International Conference on Learning Representations (ICLR), Kigali, Rwanda. Op... on Pith
  20. [33] Yangzhen Wu, Zhiqing Sun, Shanda Li, Sean Welleck, and Yiming Yang. 2025. Inference scaling laws: An empirical analysis of compute-optimal inference for LLM problem-solving. In International Conference on Learning Representations (ICLR), Singapore on Pith
show 37 more
  1. [35] Zhuohao Yu, Weizheng Gu, Yidong Wang, Xingru Jiang, Zhengran Zeng, Jindong Wang, Wei Ye, and Shikun Zhang. 2025 b . Reasoning through execution: Unifying process and outcome rewards for code generation. In Proceedings of the 42nd International Conference on Machine Learning (I... on Pith
  2. [37] Yihong Dong and Xue Jiang and Jiaru Qian and Tian Wang and Kechi Zhang and Zhi Jin and Ge Li , title =. 2025 , journal = on Pith
  3. [38] Bei Chen and Fengji Zhang and Anh Nguyen and Daoguang Zan and Zeqi Lin and Jian. International Conference on Learning Representations (ICLR) , address = on Pith
  4. [39] Kefan Li and Yuan Yuan , title =. 2024 , journal = on Pith
  5. [40] Dimensions in Program Synthesis , author =. Proceedings of the 12th International ACM SIGPLAN Symposium on Principles and Practice of Declarative Programming (PPDP) , pages =. doi:10.1145/1836089.1836091 , year = on PitharXiv
  6. [41] Roziere, Baptiste and Gehring, Jonas and Gloeckle, Fabian and Sootla, Sten and Gat, Itai and Tan, Xiaoqing Ellen and Adi, Yossi and Liu, Jingyu and Sauvestre, Romain and Remez, Tal and others , journal = on Pith
  7. [42] Execution-Based Code Generation Using Deep Reinforcement Learning , author =. Transactions on Machine Learning Research , year = on Pith
  8. [43] Le, Hung and Wang, Yue and Gotmare, Akhilesh Deepak and Savarese, Silvio and Hoi, Steven Chu Hong , booktitle =. doi:10.52202/068431-1549 , year = on Pithdoi:10.52202/068431-1549
  9. [44] Luo, Michael and Tan, Sijun and Huang, Roy and Patel, Ameen and Ariyak, Alpay and Wu, Qingyang and Shi, Xiaoxiang and Xin, Rachel and Cai, Colin and Weber, Maurice and Zhang, Ce and Li, Li Erran and Popa, Raluca Ada and Stoica, Ion , year = on Pith
  10. [45] Lichang Chen and Chen Zhu and Jiuhai Chen and Davit Soselia and Tianyi Zhou and Tom Goldstein and Heng Huang and Mohammad Shoeybi and Bryan Catanzaro , title =. Proceedings of the 41st International Conference on Machine Learning (ICML) , series = on Pith
  11. [46] Fu, Jiayi and Zhao, Xuandong and Yao, Chengyuan and Wang, Heng and Han, Qi and Xiao, Yanghua , journal =. Reward Shaping to Mitigate Reward Hacking in on Pith
  12. [47] Yu, Qiying and Zhang, Zheng and Zhu, Ruofei and Yuan, Yufeng and Zuo, Xiaochen and Yue, Yu and Dai, Weinan and Fan, Tiantian and Liu, Gaohong and Liu, Juncai and others , booktitle =. doi:10.52202/085713-3775 , year = on Pithdoi:10.52202/085713-3775
  13. [48] Learning to Solve and Verify: A Self-Play Framework for Code and Test Generation , author =. arXiv preprint arXiv:2502.14948 , year = on PitharXiv
  14. [49] Hui, Binyuan and Yang, Jian and Cui, Zeyu and Yang, Jiaxi and Liu, Dayiheng and Zhang, Lei and Liu, Tianyu and Zhang, Jiajun and Yu, Bowen and Lu, Keming and Dang, Kai and others , journal = on Pith
  15. [50] Shao, Zhihong and Wang, Peiyi and Zhu, Qihao and Xu, Runxin and Song, Junxiao and Bi, Xiao and Zhang, Haowei and Zhang, Mingchuan and Li, YK and Wu, Y and others , journal = on Pith
  16. [51] Li, Rongao and Fu, Jie and Zhang, Bo-Wen and Huang, Tao and Sun, Zhihong and Lyu, Chen and Liu, Guang and Jin, Zhi and Li, Ge , journal = on Pith
  17. [52] Jain, Naman and Han, King and Gu, Alex and Li, Wen-Ding and Yan, Fanjia and Zhang, Tianjun and Wang, Sida and Solar-Lezama, Armando and Sen, Koushik and Stoica, Ion , booktitle = on Pith
  18. [53] Cai, Zheng and Cao, Maosong and Chen, Haojiong and Chen, Kai and Chen, Keyu and Chen, Xin and Chen, Xun and Chen, Zehui and Chen, Zhi and Chu, Pei and others , journal = on Pith
  19. [54] Jiang, Nan and Li, Xiaopeng and Wang, Shiqi and Zhou, Qiang and Hossain, Soneya Binta and Ray, Baishakhi and Kumar, Varun and Ma, Xiaofei and Deoras, Anoop , booktitle =. doi:10.52202/079017-1120 , year = on Pithdoi:10.52202/079017-1120
  20. [55] Wannita Takerngsaksiri and Rujikorn Charakorn and Chakkrit Tantithamthavorn and Yuan. Journal of Systems and Software , volume =. doi:10.1016/j.jss.2025.112381 , year = on PitharXiv
  21. [56] Light, Jonathan and Wu, Yue and Sun, Yiyou and Yu, Wenchao and Liu, Yanchi and Zhao, Xujiang and Hu, Ziniu and Chen, Haifeng and Cheng, Wei , booktitle = on Pith
  22. [57] Tang, Hao and Hu, Keya and Zhou, Jin Peng and Zhong, Sicheng and Zheng, Wei-Long and Si, Xujie and Ellis, Kevin , booktitle =. Code repair with. doi:10.52202/079017-3746 , year = on Pithdoi:10.52202/079017-3746
  23. [58] Fuxiang Zhang and Jiacheng Xu and Chaojie Wang and Ce Cui and Yang Liu and Bo An , title =. Advances in Neural Information Processing Systems (NeurIPS) , volume =. doi:10.52202/085713-0169 , year = on Pithdoi:10.52202/085713-0169
  24. [59] Xuezhi Wang and Jason Wei and Dale Schuurmans and Quoc V. Le and Ed H. Chi and Sharan Narang and Aakanksha Chowdhery and Denny Zhou , title =. International Conference on Learning Representations (ICLR) , address = on Pith
  25. [60] Reflexion: Language agents with verbal reinforcement learning , author=. Advances in Neural Information Processing Systems (NeurIPS) , volume=. doi:10.52202/075280-0377 , year= on Pithdoi:10.52202/075280-0377
  26. [61] Reasoning Through Execution: Unifying Process and Outcome Rewards for Code Generation , author =. Proceedings of the 42nd International Conference on Machine Learning (ICML) , series = on Pith
  27. [62] Guo, Daya and Zhu, Qihao and Yang, Dejian and Xie, Zhenda and Dong, Kai and Zhang, Wentao and Chen, Guanting and Bi, Xiao and Wu, Yu and Li, YK and others , journal= on Pith
  28. [63] Fan, Lishui and Liu, Zhongxin and Wang, Haoye and Bao, Lingfeng and Xia, Xin and Li, Shanping , booktitle =. doi:10.1109/ASE63991.2025.00114 , year = on PitharXiv
  29. [64] Li, Junlong and Guo, Daya and Yang, Dejian and Xu, Runxin and Wu, Yu and He, Junxian , booktitle= on Pith
  30. [65] Inference-time scaling for generalist reward modeling , author=. arXiv preprint arXiv:2504.02495 , year= on PitharXiv
  31. [66] Snell, Charlie and Lee, Jaehoon and Xu, Kelvin and Kumar, Aviral , booktitle=. Scaling on Pith
  32. [67] Wu, Yangzhen and Sun, Zhiqing and Li, Shanda and Welleck, Sean and Yang, Yiming , booktitle=. Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for on Pith
  33. [68] Guangming Sheng and Chi Zhang and Zilingfeng Ye and Xibin Wu and Wang Zhang and Ru Zhang and Yanghua Peng and Haibin Lin and Chuan Wu , booktitle =. doi:10.1145/3689031.3696075 , year = on PitharXiv
  34. [69] Jujie He and Jiacai Liu and Chris Yuhao Liu and Rui Yan and Chaojie Wang and Peng Cheng and Xiaoyu Zhang and Fuxiang Zhang and Jiacheng Xu and Wei Shen and Siyuan Li and Liang Zeng and Tianwen Wei and Cheng Cheng and Bo An and Yang Liu and Yahui Zhou , title =. arXiv preprint ... on PithPith's reviewarXiv
  35. [70] Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallouédec , title =. 2020 , howpublished = on Pith
  36. [71] Yinjie Wang and Ling Yang and Ye Tian and Ke Shen and Mengdi Wang , title =. Advances in Neural Information Processing Systems (NeurIPS) , volume =. doi:10.52202/085713-4809 , year = on Pithdoi:10.52202/085713-4809
  37. [72] Zeyao Ma and Xiaokang Zhang and Jing Zhang and Jifan Yu and Sijia Luo and Jie Tang , title =. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (ACL) , pages =. doi:10.18653/v1/2025.acl-long.343 , year = on Pithdoi:10.18653/v1/2025.acl-long.343
Cited by 3 Pith papers

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment