REVIEW 2 major objections 5 minor 31 references
Unit-test-only self-play for code repair drifts to unrealistic bugs; a small reference set of real bugs anchors generation and restores cross-source gains.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · grok-4.5
2026-07-12 01:53 UTC pith:RJOKFTVY
load-bearing objection Solid empirical package: open-ended generator–fixer self-play, a controlled multi-source repair bench, and dual anchoring that actually reduces the drift they document. the 2 major comments →
Anchored Self-Play for Code Repair
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Generator–fixer self-play that relies only on unit-test pass/fail can co-evolve a hard curriculum, but the generator drifts toward valid yet unrealistic bugs and the fixer loses robustness on human-authored bugs. Anchoring the same loop to a small mixed reference set—via embedding similarity of code diffs for generation plus reference-bug mixing for the fixer—stabilizes training and yields the best fix rates across human, human-edited-LM, and LM bug sources on BugSourceBench.
What carries the argument
Anchored Self-Play (ASP): a shared policy alternates generator and fixer roles under GRPO, with the generator’s difficulty-shaped unit-test reward plus a centered k-NN embedding-similarity term to a reference bug pool, and with a fraction of fixer episodes replaced by those reference bugs.
Load-bearing premise
That how close a generated bug’s edit looks to a finite reference pool in a frozen code-embedding space is a good enough proxy for “realistic” that it can stop harmful drift without killing the open-ended curriculum.
What would settle it
Train ASP with the same reference pool and then evaluate on held-out human bugs whose edit embeddings lie far from that pool; if fix rate on those distant human bugs falls back to or below plain self-play while synthetic fix rate stays high, the embedding anchor is not carrying realism.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies open-ended generator–fixer self-play for code repair: a single LM is trained with GRPO to introduce unit-test-failing bugs into correct programs and to repair them, using unit-test outcomes as rewards. It shows that unit-test-only self-play co-evolves difficulty but drifts toward unrealistic bugs, improving synthetic repair while degrading on human-originated bugs. To measure this, the authors introduce BugSourceBench, which holds tasks, specs, and tests fixed while varying only bug source (human, human-edited LM, Qwen-7B, gpt-oss-20b). They propose Anchored Self-Play (ASP), which anchors generation with a centered voyage-code-3 (or CodeBERT) embedding-similarity reward on reference-to-bug diffs and mixes a small multi-source reference pool into fixer training. On held-out BugSourceBench tasks, ASP improves average fix rate over standard self-play by +7.0 pp / 24% relative, with gains on both LM- and human-originated sources, and transfers to DebugBench.
Significance. If the results hold, the paper makes three concrete contributions of clear value to automated program repair and LM post-training: (i) a controlled multi-source repair benchmark that isolates bug-source shift rather than confounding task difficulty; (ii) a documented failure mode of unit-test-only open-ended self-play (distribution drift toward hard but unrealistic bugs); and (iii) a practical, sample-efficient anchoring recipe that recovers cross-source generalization without abandoning open-ended generation. Strengths include extensive ablations (mix vs. similarity, reference composition/size, embeddings/k-NN, shared vs. decoupled weights, alternate base model), pass@k, semantic-type breakdowns, test-time fixer transfer to larger coders, and released code. The work is empirical rather than theoretical, but the evaluation design is unusually careful for this area.
major comments (2)
- The central empirical claim is well supported by Figures 3–5 and Tables 1a–1b, 9–13, but the paper should more carefully bound what ‘realism’ means. Section 4.5.2 treats k-NN cosine similarity of voyage-code-3 (or CodeBERT) unified-diff embeddings to a 900-bug training reference pool as a soft realism signal. Table 1b shows that reference-pool composition steers which sources improve, so the method is better described as source-anchoring than as recovering an intrinsic notion of realistic bugs. A short discussion of this scope (and of residual risk when deployment bugs fall outside the reference neighborhood) would strengthen the claim without changing the results.
- BugSourceBench construction (Section 3 / A.2) is a load-bearing contribution, yet human and human-edited-LM bugs come from a small annotator process (two annotators, 1–4 localized edits) with limited inter-annotator or style-diversity analysis. Because the main human-originated gains are smaller than LM-source gains (+1.3 pp Human, +5.5 pp Human-Edited LM vs. larger LM gains in Figure 3), the paper should report annotator agreement / edit-type statistics or a sensitivity check so readers can judge how much the human splits represent broader developer mistakes rather than a narrow edit style.
minor comments (5)
- Figure 2 and Figure 4 are central to the drift narrative; ensure axis labels, checkpoint definitions, and whether curves are single-run or multi-seed are fully specified in the captions.
- Notation for role-conditioned policies π_G / π_F and shared weights is clear in Section 4, but the main text should state earlier that generator and fixer share parameters by default (currently deferred to setup / Table 11).
- Table 3 and Section A.4 usefully separate repair from codegen; a one-sentence pointer in the main evaluation section would help readers who skip the appendix.
- Hyperparameters (λ=0.20, ρ band [0.25,0.75], pmix=20%, k=5, β=0.99) are listed in B.1; a compact main-text table or footnote would improve reproducibility for readers of the body only.
- Minor polish: consistent hyphenation of ‘self-play’ / ‘bug-source’, and ensure all appendix table references (e.g., Tables 9–13) are cross-linked from the main ablations paragraph.
Circularity Check
No significant circularity: empirical method + held-out multi-source evaluation; ASP is not success-by-construction on the reference pool.
full rationale
This is a standard empirical ML paper. Generator–fixer self-play and Anchored Self-Play are defined via unit-test rewards (r_F = v(x,y); r_G_base from fix-rate band ρ) plus optional embedding-similarity shaping and reference mixing (Section 4). The central claim is not a first-principles derivation but a measured improvement: on held-out BugSourceBench task IDs (127 tasks shared across sources, disjoint from the 900-task train set and from the 900-bug reference pool), ASP raises average fix rate over unit-test-only self-play by +7.0 pp / 24% relative (Figure 3, Section 6), with supporting dynamics, ablations, and external DebugBench transfer (Tables 1, 9–13; Figure 4–5). The reference pool is used only as training signal (λ-clipped centered sim reward; p_mix fixer mixing); evaluation metrics are fix rates on held-out bugs, not similarity to the reference set. No equation defines the reported fix rate in terms of the training objective by construction; no parameter is fitted to the test splits and renamed a prediction; no uniqueness theorem or load-bearing ansatz is imported via self-citation. Minor reuse of BigCodeBench-derived tasks for both train and benchmark construction is ordinary dataset practice, not circular reduction. Score 0.
Axiom & Free-Parameter Ledger
free parameters (5)
- similarity reward weight λ =
0.20
- difficulty band [ρℓ, ρh] =
[0.25, 0.75], α=0.2
- reference mix probability pmix =
0.20
- k-NN pooling k and EMA baseline β =
k=5, β=0.99
- GRPO rollout counts G, K and learning rate =
G=4, K=4, lr=1e-6
axioms (5)
- domain assumption Unit-test pass/fail is a sufficient hard reward for both bug validity and repair correctness in this setting.
- ad hoc to paper Cosine similarity of voyage-code-3 (or CodeBERT) embeddings of reference-to-bug diffs tracks ‘realistic’ bug style well enough to reduce harmful drift.
- domain assumption A small mixed reference pool drawn from the same BugSourceBench construction process is an adequate anchor for target sources.
- standard math GRPO with shared generator/fixer weights and group-normalized advantages is a valid optimizer for the dual-role policy.
- domain assumption BugSourceBench sources (human edits, human-edited LM, Qwen-7B, gpt-oss-20b errors) are representative enough of LM-assisted programming bugs for cross-source claims.
invented entities (3)
-
BugSourceBench
independent evidence
-
Anchored Self-Play (ASP)
no independent evidence
-
Generator–fixer self-play for open-ended code repair
no independent evidence
read the original abstract
Code repair is an important capability for language models (LMs): given a buggy program and unit tests, an LM must produce a fixed program that passes the tests. Because code repair data is limited, we aim to scale supervision by using an LM to generate bug--fix tasks. We propose __generator--fixer self-play__, in which a single model is trained with reinforcement learning to generate bugs and fix them. As the fixer improves, the generator adapts to produce more difficult bugs, yielding an automatic curriculum. To test whether this curriculum generalizes, we introduce BugSourceBench, a repair benchmark spanning realistic bug sources: bugs in human-written code, LM-generated code, and human-edited LM-generated code. On BugSourceBench, we find that self-play drifts toward difficult but unrealistic bugs, improving on synthetic bugs but degrading on human-authored ones. We propose Anchored Self-Play (ASP), which anchors self-play with a small reference set by adding a code-embedding similarity reward for generation and mixing reference bugs into fixer training. Across bug sources, ASP achieves the best fix rates, improving average fix rate over standard self-play by $+24\%$ relative / $+7.0$ pp absolute, with gains on bugs from both LMs and humans.
Figures
Reference graph
Works this paper leans on
-
[1]
cc/paper/2021/hash/ea96efc03b9a050d895110db8c4af057-Abstract.html
URL https://proceedings.neurips. cc/paper/2021/hash/ea96efc03b9a050d895110db8c4af057-Abstract.html. Austin, J., Odena, A., Nye, M., Bosma, M., Michalewski, H., Dohan, D., Jiang, E., Cai, C., Terry, M., Le, Q., et al. Program synthesis with large language models. InarXiv preprint arXiv:2108.07732,
Pith/arXiv arXiv 2021
-
[2]
Self-questioning language models.arXiv preprint arXiv:2508.03682,
Chen, L., Prabhudesai, M., Fragkiadaki, K., Liu, H., and Pathak, D. Self-questioning language models.arXiv preprint arXiv:2508.03682,
-
[3]
Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. d. O., Kaplan, J., Edwards, H., Burda, Y ., Joseph, N., Brockman, G., et al. Evaluating large language models trained on code. InarXiv preprint arXiv:2107.03374,
-
[4]
B., and Poshyvanyk, D
Chen, Z., Tufano, M., Pantiuchina, J., Watson, C., Jiang, G. B., and Poshyvanyk, D. Sequencer: Sequence-to-sequence learning for end-to-end program repair. InProceedings of the 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE),
2019
-
[5]
Chen, Z., Deng, Y ., Yuan, H., Ji, K., and Gu, Q. Self-play fine-tuning converts weak language models to strong language models.arXiv preprint arXiv:2401.01335,
-
[6]
Cheng, J., Liu, X., Wang, C., Gu, X., Lu, Y ., Zhang, D., Dong, Y ., Tang, J., Wang, H., and Huang, M. Spar: Self-play with tree-search refinement to improve instruction-following in large language models.arXiv preprint arXiv:2412.11605,
-
[7]
URLhttps://arxiv.org/abs/2012.09830. Dong, K. and Ma, T. Stp: Self-play llm theorem provers with iterative conjecturing and proving
arXiv 2012
-
[8]
URL https: //arxiv.org/abs/2502.00212. Dou, S., Jia, H., Wu, S., Zheng, H., Wu, M., Tao, Y ., Zhang, M., Chai, M., Fan, J., Xi, Z., Zheng, R., Wu, Y ., Wen, M., Gui, T., Zhang, Q., Qiu, X., and Huang, X. What’s wrong with your code generated by large language models? an extensive study,
-
[9]
Forrest, S., Nguyen, T., Weimer, W., and Le Goues, C
URLhttps://arxiv.org/abs/2407.06153. Forrest, S., Nguyen, T., Weimer, W., and Le Goues, C. A genetic programming approach to automated software repair. In Proceedings of the 11th Annual Conference on Genetic and Evolutionary Computation (GECCO),
-
[10]
R-zero: Self-evolving reasoning llm from zero data.arXiv preprint arXiv:2508.05004,
Huang, C., Yu, W., Wang, X., Zhang, H., Li, Z., Li, R., Huang, J., Mi, H., and Yu, D. R-zero: Self-evolving reasoning llm from zero data.arXiv preprint arXiv:2508.05004,
-
[11]
Qwen2.5-Coder technical report.arXiv preprint arXiv:2409.12186,
Hui, B., Yang, J., Cui, Z., Yang, J., Liu, D., Zhang, L., Liu, T., Zhang, J., Yu, B., Lu, K., et al. Qwen2.5-Coder technical report.arXiv preprint arXiv:2409.12186,
-
[12]
URL https://arxiv.org/ abs/2403.07974. Jimenez, C. E., Yang, J., Wettig, A., Yao, S., Pei, K., Press, O., and Narasimhan, K. Swe-bench: Can language models resolve real-world github issues?arXiv preprint arXiv:2310.06770,
-
[13]
20 Anchored Self-Play for Code Repair Just, R., Jalali, D., and Ernst, M. D. Defects4j: A database of existing faults to enable controlled testing studies for java programs. InProceedings of the 2014 international symposium on software testing and analysis, pp. 437–440,
2014
-
[14]
URLhttps://doi.org/10.5281/zenodo.14751899. Kuba, J. G., Gu, M., Ma, Q., Tian, Y ., Mohan, V ., and Chen, J. Language self-play for data-free training.arXiv preprint arXiv:2509.07414,
-
[15]
Quixbugs: A multi-lingual program repair benchmark set based on the quixey challenge
Lin, D., Koppel, J., Chen, A., and Solar-Lezama, A. Quixbugs: A multi-lingual program repair benchmark set based on the quixey challenge. InProceedings Companion of the 2017 ACM SIGPLAN international conference on systems, programming, languages, and applications: software for humanity, pp. 55–56,
2017
-
[16]
Lin, Z., Shen, S., Shang, J., Weston, J., and Nie, Y . Learning to solve and verify: A self-play framework for code and test generation.arXiv preprint arXiv:2502.14948,
-
[17]
Spice: Self-play in corpus environments improves reasoning.arXiv preprint arXiv:2510.24684,
Liu, B., Jin, C., Kim, S., Yuan, W., Zhao, W., Kulikov, I., Li, X., Sukhbaatar, S., Lanchantin, J., and Weston, J. Spice: Self-play in corpus environments improves reasoning.arXiv preprint arXiv:2510.24684,
-
[18]
URL https://arxiv.org/abs/2305.01210. arXiv:2305.01210. Long, F. and Rinard, M. Automatic patch generation by learning correct code. InProceedings of the 43rd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL),
-
[19]
URL https://arxiv.org/abs/1901
doi: 10.1109/SANER.2019.8667991. URL https://arxiv.org/abs/1901. 06024. Oliva, G. A., Rajbahadur, G. K., Bhatia, A., Zhang, H., Chen, Y ., Chen, Z., Leung, A., Lin, D., Chen, B., and Hassan, A. E. Spice: An automated swe-bench labeling pipeline for issue clarity, test coverage, and effort estimation.arXiv preprint arXiv:2507.09108,
-
[20]
URLhttps://arxiv.org/abs/2203.01302. Pham, M. V ., Phan, H. N., Phan, H. N., Chi, C. L., Nguyen, T. N., and Bui, N. D. Swe-synth: Synthesizing verifiable bug-fix data to enable large language models in resolving real-world bugs.arXiv preprint arXiv:2504.14757,
-
[21]
Pourcel, J., Colas, C., Molinaro, G., Oudeyer, P.-Y ., and Teodorescu, L
URLhttps://arxiv.org/abs/2407.00695. Pourcel, J., Colas, C., Molinaro, G., Oudeyer, P.-Y ., and Teodorescu, L. Aces: Generating a diversity of challenging programming puzzles with autotelic generative models.Advances in Neural Information Processing Systems, 37: 67627–67662,
-
[22]
doi: 10.18653/v1/2022.acl-long.230
Association for Computational Linguistics. doi: 10.18653/v1/2022.acl-long.230. URL https://aclanthology. org/2022.acl-long.230/. Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al. Mastering the game of go without human knowledge.nature, 550(7676):354–359,
-
[23]
Teodorescu, L., Colas, C., Bowers, M., Carta, T., and Oudeyer, P.-Y
URL https://arxiv.org/abs/ 2510.19898. Teodorescu, L., Colas, C., Bowers, M., Carta, T., and Oudeyer, P.-Y . Codeplay: Autotelic learning through collaborative self-play in programming environments. InIMOL 2023: Intrinsically Motivated Open-ended Learning Workshop at NeurIPS 2023,
arXiv 2023
-
[24]
Wang, Y ., Yang, L., Tian, Y ., Shen, K., and Wang, M
URLhttps://arxiv.org/abs/2401.04621. Wang, Y ., Yang, L., Tian, Y ., Shen, K., and Wang, M. Cure: Co-evolving coders and unit testers via reinforcement learning. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems,
-
[25]
Wei, Y ., Sun, Z., McMilin, E., Gehring, J., Zhang, D., Synnaeve, G., Fried, D., Zhang, L., and Wang, S. Toward training superintelligent software agents through self-play swe-rl.arXiv preprint arXiv:2512.18552,
-
[26]
Wilf, A., Aggarwal, P., Parno, B., Fried, D., Morency, L.-P., Liang, P. P., and Welleck, S. Propose, solve, verify: Self-play through formal verification.arXiv preprint arXiv:2512.18160,
-
[27]
E., Wettig, A., Khandpur, K., Zhang, Y ., Hui, B., Press, O., Schmidt, L., and Yang, D
Yang, J., Leret, K., Jimenez, C. E., Wettig, A., Khandpur, K., Zhang, Y ., Hui, B., Press, O., Schmidt, L., and Yang, D. Swe-smith: Scaling data for software engineering agents.arXiv preprint arXiv:2504.21798, 2025a. Yang, W., Wang, H., Liu, Z., Li, X., Yan, Y ., Wang, S., Gu, Y ., Yu, M., Liu, Z., and Yu, G. Coast: Enhancing the code debugging ability of...
-
[28]
Association for Computing Machinery. ISBN 9781450394758. doi: 10.1145/3551349. 3556926. URLhttps://doi.org/10.1145/3551349.3556926. Yu, W., Liang, Z., Huang, C., Panaganti, K., Fang, T., Mi, H., and Yu, D. Guided self-evolving llms with minimal human supervision.arXiv preprint arXiv:2512.02472,
-
[29]
Absolute zero: Reinforced self-play reasoning with zero data.arXiv preprint arXiv:2505.03335,
Zhao, A., Wu, Y ., Yue, Y ., Wu, T., Xu, Q., Lin, M., Wang, S., Wu, Q., Zheng, Z., and Huang, G. Absolute zero: Reinforced self-play reasoning with zero data.arXiv preprint arXiv:2505.03335,
-
[30]
Zhuo, T. Y ., Vu, M. C., Chim, J., Hu, H., Yu, W., Widyasari, R., Yusuf, I. N. B., Zhan, H., He, J., Paul, I., et al. Bigcodebench: Benchmarking code generation with diverse function calls and complex instructions.arXiv preprint arXiv:2406.15877,
-
[31]
ISSN 1049-331X. doi: 10.1145/3631972. URLhttps://doi.org/10.1145/3631972. 22
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.