{"id":"adf284b2-dcf1-4fc7-8084-9eea22ce49f8","arxiv_id":"2607.03523","paper_version":1,"verdict":"ACCEPT","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"low","formal_verification":"none","parameter_count":5,"one_line_summary":"Anchored Self-Play uses a small reference bug set and code-embedding similarity to stop generator–fixer self-play from drifting into unrealistic bugs, raising average fix rate by 7.0 points on BugSourceBench.","lead":"Language models can train themselves to fix code by inventing bugs and repairing them, but pure self-play drifts into unrealistic bugs. Anchored Self-Play keeps the curriculum realistic with a small reference set and embedding similarity, improving repair across human and model-made bugs.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"No significant objection identified","rationale":"I agree with the reader that the embedding-as-realism proxy is the softest modeling choice (Section 4.5.2; λ=0.20, k=5, 900 mixed reference bugs). I disagree that this is a load-bearing threat to the strongest claim. The paper does not claim embeddings define realism in general; it claims that anchoring self-play with a small reference set via similarity shaping plus reference mixing improves multi-source repair relative to unit-test-only self-play. That is directly measured. Ablations show both components help and that the method is not tied to one embedder; dynamics show standard self-play regresses on human-originated bugs while ASP does not. Within the paper's stated scope (function-level Python, unit-test feedback, BigCodeBench-derived tasks), the evidence is adequate for ACCEPT. No adjustment is warranted.","tokens_in":22866,"tokens_out":567,"duration_ms":4665,"concrete_test":"Re-run the main ASP vs. Self-Play comparison on the 127-task BugSourceBench test split with the CodeBERT ASP variant already reported in Table 8/9, and also with reference mixing only (no similarity reward) at the same compute; if average fix rate remains within ~1–2 pp of the voyage-code-3 ASP result and still beats Self-Play by ≥5 pp, the embedding-proxy concern does not overturn the headline claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The reader's weakest assumption (voyage-code-3 k-NN similarity of unified diffs as a soft realism signal) is a real modeling bet, but it is not load-bearing for the central claim in a way that undermines the paper. The claim is empirical and scoped: on BugSourceBench, ASP improves average fix rate over unit-test-only self-play by +7.0 pp, with gains on both LM and human-originated sources. That claim is supported by the main comparison (Figure 3), training dynamics (Figure 4), component ablations (Table 1a), reference-pool composition (Table 1b), reference-size scaling (Figure 5), embedding/k-NN robustness (Table 9, including CodeBERT), shared vs. decoupled weights (Table 11), alternate base model (Table 12), and external DebugBench transfer (Table 13). Even if embeddings imperfectly track \"realism,\" the paper shows that the combined anchoring recipe reduces the observed drift and improves held-out multi-source fix rates under the stated protocol. No internal inconsistency or untested leap is required for the strongest claim to hold.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper studies open-ended generator–fixer self-play for code repair: a single LM is trained with GRPO to introduce unit-test-failing bugs into correct programs and to repair them, using unit-test outcomes as rewards. It shows that unit-test-only self-play co-evolves difficulty but drifts toward unrealistic bugs, improving synthetic repair while degrading on human-originated bugs. To measure this, the authors introduce BugSourceBench, which holds tasks, specs, and tests fixed while varying only bug source (human, human-edited LM, Qwen-7B, gpt-oss-20b). They propose Anchored Self-Play (ASP), which anchors generation with a centered voyage-code-3 (or CodeBERT) embedding-similarity reward on reference-to-bug diffs and mixes a small multi-source reference pool into fixer training. On held-out BugSourceBench tasks, ASP improves average fix rate over standard self-play by +7.0 pp / 24% relative, with gains on both LM- and human-originated sources, and transfers to DebugBench.","tokens_in":23201,"tokens_out":1165,"duration_ms":9398,"significance":"If the results hold, the paper makes three concrete contributions of clear value to automated program repair and LM post-training: (i) a controlled multi-source repair benchmark that isolates bug-source shift rather than confounding task difficulty; (ii) a documented failure mode of unit-test-only open-ended self-play (distribution drift toward hard but unrealistic bugs); and (iii) a practical, sample-efficient anchoring recipe that recovers cross-source generalization without abandoning open-ended generation. Strengths include extensive ablations (mix vs. similarity, reference composition/size, embeddings/k-NN, shared vs. decoupled weights, alternate base model), pass@k, semantic-type breakdowns, test-time fixer transfer to larger coders, and released code. The work is empirical rather than theoretical, but the evaluation design is unusually careful for this area.","major_comments":[{"comment":"The central empirical claim is well supported by Figures 3–5 and Tables 1a–1b, 9–13, but the paper should more carefully bound what ‘realism’ means. Section 4.5.2 treats k-NN cosine similarity of voyage-code-3 (or CodeBERT) unified-diff embeddings to a 900-bug training reference pool as a soft realism signal. Table 1b shows that reference-pool composition steers which sources improve, so the method is better described as source-anchoring than as recovering an intrinsic notion of realistic bugs. A short discussion of this scope (and of residual risk when deployment bugs fall outside the reference neighborhood) would strengthen the claim without changing the results.","section":null},{"comment":"BugSourceBench construction (Section 3 / A.2) is a load-bearing contribution, yet human and human-edited-LM bugs come from a small annotator process (two annotators, 1–4 localized edits) with limited inter-annotator or style-diversity analysis. Because the main human-originated gains are smaller than LM-source gains (+1.3 pp Human, +5.5 pp Human-Edited LM vs. larger LM gains in Figure 3), the paper should report annotator agreement / edit-type statistics or a sensitivity check so readers can judge how much the human splits represent broader developer mistakes rather than a narrow edit style.","section":null}],"minor_comments":[{"comment":"Figure 2 and Figure 4 are central to the drift narrative; ensure axis labels, checkpoint definitions, and whether curves are single-run or multi-seed are fully specified in the captions.","section":null},{"comment":"Notation for role-conditioned policies π_G / π_F and shared weights is clear in Section 4, but the main text should state earlier that generator and fixer share parameters by default (currently deferred to setup / Table 11).","section":null},{"comment":"Table 3 and Section A.4 usefully separate repair from codegen; a one-sentence pointer in the main evaluation section would help readers who skip the appendix.","section":null},{"comment":"Hyperparameters (λ=0.20, ρ band [0.25,0.75], pmix=20%, k=5, β=0.99) are listed in B.1; a compact main-text table or footnote would improve reproducibility for readers of the body only.","section":null},{"comment":"Minor polish: consistent hyphenation of ‘self-play’ / ‘bug-source’, and ensure all appendix table references (e.g., Tables 9–13) are cross-linked from the main ablations paragraph.","section":null}],"recommendation":"minor_revision","confidential_remarks":"I agree with the reader/skeptic that the embedding-realism assumption is a modeling bet rather than a circularity or load-bearing error: the paper’s claim is empirical and multi-source, and the ablations (including CodeBERT and reference-composition steering) make the bet testable. Minor revision is appropriate; the work is above the bar for a solid empirical ML/SE contribution once the realism-scope language and human-annotation characterization are tightened. Fit for a serious ML venue is good; pure SE venues may want more repository-level discussion, which the limitations already acknowledge."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The useful takeaway is simple: unit-test-only self-play for open-ended bug generation drifts toward hard-but-unrealistic edits, and a small reference pool plus embedding-similarity shaping plus reference mixing largely fixes that on their held-out multi-source bench.\n\nWhat is new is the combination, not any single piece. They train one policy as both generator and fixer with GRPO, reward valid intermediate-difficulty bugs, then show co-evolution on synthetic data while fix rates later fall on human-originated bugs. BugSourceBench holds tasks and tests fixed and only swaps the bug source (human, human-edited LM, Qwen-7B, gpt-oss-20b). Anchored Self-Play adds a centered voyage-code-3 k-NN similarity term on diffs and mixes reference bugs into fixer training only. Main result: +7.0 pp average over standard self-play, gains on both LM and human sources, with component ablations, pool composition, size scaling, CodeBERT swap, shared vs decoupled weights, another base model, pass@k, and DebugBench transfer. Code is released. That is a clean, reproducible systems result for function-level Python repair.\n\nSoft spots are real but proportionate. The realism signal is still a frozen embedding neighborhood over a finite mixed pool; if those neighborhoods miss deployment style, you steer wrong. Scope is deliberately not SWE-bench: no multi-file localization, builds, or long-horizon tools. Free parameters (lambda, mix rate, difficulty band, k) are tuned but ablated enough that the central claim does not hang on one magic setting. Citations cover APR, synthetic bugs, and self-play curricula without obvious gaps or padding.\n\nThis is for people building coding agents or synthetic repair data who already care about distribution shift. Math is standard RL; data and protocol are careful. I would send it to peer review and would bring it to reading group. Worth engaging if you work in this area.","headline":"Solid empirical package: open-ended generator–fixer self-play, a controlled multi-source repair bench, and dual anchoring that actually reduces the drift they document.","tokens_in":23805,"tokens_out":491,"would_cite":true,"duration_ms":5098,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Unit-test-only self-play for code repair drifts to unrealistic bugs; a small reference set of real bugs anchors generation and restores cross-source gains.","keywords":["code repair","self-play","language models","synthetic data","distribution drift","unit tests","BugSourceBench","anchored self-play"],"falsifier":"Train ASP with the same reference pool and then evaluate on held-out human bugs whose edit embeddings lie far from that pool; if fix rate on those distant human bugs falls back to or below plain self-play while synthetic fix rate stays high, the embedding anchor is not carrying realism.","tokens_in":23767,"feed_emoji":"🐛","tokens_out":678,"duration_ms":5492,"temperature":0.7,"pith_summary":"Language models can repair buggy code when unit tests tell them whether a fix works, but real repair examples are scarce. This paper trains one model both to introduce bugs into correct programs and to fix those bugs, so that as the fixer improves the generator invents harder failures and builds an automatic curriculum. Without any realism signal, that curriculum drifts: the generator invents difficult test-failing edits that do not look like human or model mistakes, so repair performance rises on synthetic bugs and falls on human-authored ones. Anchored Self-Play (ASP) keeps a small reference pool of real bugs and uses two anchors—an embedding-similarity reward that steers the generator toward reference-like edits, and mixing of reference bugs into fixer training. On BugSourceBench, which holds the programming task fixed while varying only the bug source, ASP raises average fix rate over plain self-play by 7 percentage points (24% relative) and improves both LM-generated and human-originated bugs.","feed_headline":"Self-play code repair drifts; a small real-bug set stops it","feed_subtitle":"Anchoring generation and fixer training lifts fix rates on human and LM bugs by 7 points","key_machinery":"Anchored Self-Play (ASP): a shared policy alternates generator and fixer roles under GRPO, with the generator’s difficulty-shaped unit-test reward plus a centered k-NN embedding-similarity term to a reference bug pool, and with a fraction of fixer episodes replaced by those reference bugs.","core_discovery":"Generator–fixer self-play that relies only on unit-test pass/fail can co-evolve a hard curriculum, but the generator drifts toward valid yet unrealistic bugs and the fixer loses robustness on human-authored bugs. Anchoring the same loop to a small mixed reference set—via embedding similarity of code diffs for generation plus reference-bug mixing for the fixer—stabilizes training and yields the best fix rates across human, human-edited-LM, and LM bug sources on BugSourceBench.","pith_inferences":[],"forward_implications":[],"fun_headline_variants":["Self-play code repair drifts to unrealistic bugs; anchors fix it","Anchored self-play stops drift, lifts fix rates 7 points on real bugs","Generator-fixer loop needs real-bug anchors for robust code repair","ASP anchors self-play, best fix rates across human and LM bugs","Small reference set prevents self-play drift in code repair LMs"],"cache_read_input_tokens":16512,"weakest_assumption_plain":"That how close a generated bug’s edit looks to a finite reference pool in a frozen code-embedding space is a good enough proxy for “realistic” that it can stop harmful drift without killing the open-ended curriculum.","fun_headline_variants_meta":{"raw":{"variants":["Self-play code repair drifts to unrealistic bugs; anchors fix it","Anchored self-play stops drift, lifts fix rates 7 points on real bugs","Generator-fixer loop needs real-bug anchors for robust code repair","ASP anchors self-play, best fix rates across human and LM bugs","Small reference set prevents self-play drift in code repair LMs"]},"model":"grok-4.5","effort":"low","cost_usd":0.005266,"raw_usage":{"total_tokens":1477,"prompt_tokens":803,"num_sources_used":0,"completion_tokens":98,"cost_in_usd_ticks":52660000,"prompt_tokens_details":{"text_tokens":803,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":576,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":803,"tokens_out":98,"duration_ms":4430,"temperature":1.0,"reasoning_tokens":576,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-12T01:53:08.094964+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Train ASP with the same reference pool and then evaluate on held-out human bugs whose edit embeddings lie far from that pool; if fix rate on those distant human bugs falls back to or below plain self-play while synthetic fix rate stays high, the embedding anchor is not carrying realism.","supporting_citations":[],"review_version":1}