{"id":"0bcae66f-6bbb-4f09-8251-333a1f709d64","arxiv_id":"2502.09183","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A code model can improve its one-shot code generation by iteratively training on its own best solutions and external critiques of its weak solutions, using only SFT and 80K instructions.","lead":"RefineCoder introduces a training loop where a code model generates its own candidate solutions, an external judge scores them, and the model is fine-tuned on its best solutions plus critiques of its weaker ones. The authors report steady gains across code benchmarks after three rounds using only 80K instructions and a simple SFT objective, beating same-size models trained by direct teacher distillation.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The dataset-update gate in Eq. 10 relies on pairwise LLM-as-a-Judge decisions that are only indirectly validated; if the judge systematically favors self-generated code, the reported gains could reflect judge bias rather than genuine code-quality improvement.","rationale":"The reader's weakest_assumption correctly identifies the scoring system as the linchpin of ACR. Without a reliable winner decision, every downstream data update in Eq. 10 is suspect: single-turn updates may enshrine mediocre self-generated code, and two-turn critique updates may pair a 'loser' that is actually better than the reference, teaching the model to degrade. The paper's Section 5.2 validation is indirect: it shows that the composite score separates best from worst sampled code on HumanEval+ and MBPP+ for the final model, but does not test the pairwise self-vs-teacher decision, does not evaluate on the training distribution, and does not study the judges used during earlier iterations. The executor (Eq. 4) only checks whether code runs, not whether it is correct, so correctness hinges on the LLM judge. This is more load-bearing than the continuity issue raised by the non-monotone MultiPL-E results (Table 5): an overstated 'continuous' claim can be corrected by rephrasing, but an unreliable judge would invalidate the core mechanism. The paper does provide supporting ablations (Figure 3) and a TLI leakage analysis, so the concern is not that the results are fraudulent; rather, the judge's reliability is an unverified precondition. My proposed test is a direct pairwise accuracy check on the training distribution; if it passes, the method's mechanism is supported, and if it fails, the conditional verdict should be reconsidered. Therefore I recommend keeping the reader's CONDITIONAL verdict unchanged pending this check.","tokens_in":19157,"tokens_out":12064,"duration_ms":121968,"concrete_test":"Sample 200 instructions from the 80K D0. For each, run M0 to sample 7 responses, compute yself via Eqs. 1-5, and record the PairJudge decision (Eqs. 6-8). Independently create hidden test cases for each instruction (e.g., by prompting GPT-4o to generate unit tests, then filtering by executing on y0 and a reference solution) and execute yself and y0 to determine the true winner. Compare the judge's winner with the test-based winner. If pairwise agreement is below 85% or if the judge prefers self-generated code in more than 55% of disagreements, the gating in Eq. 10 is unreliable; rerunning the pipeline with an oracle (test-based) gate would show whether the reported improvements persist.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (Section 1: 'By iteratively applying the ACR process, the model achieves continuous improvement in its code generation capabilities') depends entirely on Eqs. 5-8 correctly determining whether the best self-generated response beats the teacher response. The selective critique strategy (Eq. 10) then converts that binary decision into either single-turn self-generated data or two-turn critique data. If the judge is noisy or biased, the pipeline will preferentially retain self-generated code that the judge likes but that is not actually better, and the model can drift toward judge-preferred styles. Section 5.2 attempts to validate the scoring system only on HumanEval+ and MBPP+ with the final Iter3 model, measuring best-vs-worst pass rates and an error rate (Table 4); it does not measure pairwise decision accuracy on the training distribution, does not report judge agreement or failure cases, and does not test the models used at intermediate iterations (M0, M1, M2). The executor (Eq. 4) only checks executability, not correctness, so it cannot catch the judge's preference for plausible-but-wrong code. A biased judge is therefore the weakest link: if it is wrong even 10-15% of the time, the iterative SFT will accumulate mislabeled winners and the 'continuous improvement' claim would not generalize beyond the reported runs.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes Adaptive Critique Refinement (ACR), an iterative SFT pipeline for code LLMs. At each iteration the current model samples n responses per instruction; a composite scoring system (LLM-as-a-judge plus a code executor) selects the best self-generated response and compares it pairwise with the existing teacher response. Depending on the winner, the dataset is updated to either a single-turn (x, y_w) example or a two-turn critique example (x, y_l, c, y_w), and the next model is fine-tuned on the updated dataset from the original base model. Applied for three iterations to DeepSeek-Coder-6.7B-Base and Qwen2.5-Coder-7B-Base, the method yields RefineCoder-DS-6.7B and RefineCoder-QW-7B. The paper reports monotone gains on LiveCodeBench, BigCodeBench(-hard), HumanEval(+), and MBPP(+), claims better or comparable results than same-size baselines with only 80K data, and includes additional analyses of data leakage, scoring effectiveness, ablations, multilingual OOD transfer, and refinement with external feedback.","tokens_in":19443,"tokens_out":4967,"duration_ms":48625,"significance":"The contribution is potentially significant: if the conclusions hold, a code model can improve itself through iterative SFT on self-generated responses and teacher-generated critiques, without DPO/RLHF and without directly imitating teacher code, while using only 80K instructions. The paper has several commendable elements: the constructed dataset is checked for contamination with the TLI metric, the two data-construction branches are ablated, an OOD multilingual benchmark is included, and an appendix honestly documents diminishing returns. The method is simple and the pipeline is transparent. However, the evidence is currently limited by single-run evaluations, largely copied baseline numbers, and an indirect validation of the judge that gates the entire data update; these issues need to be addressed before the claimed continuous improvement can be fully accepted.","major_comments":[{"comment":"The dataset update rule in Eq. (10) is gated entirely by the pairwise judge decision in Eqs. (6)–(8), yet §5.2 validates the composite scoring system only by comparing the highest- and lowest-scoring self-generated responses on HumanEval+ and MBPP+ for the final Iter3 model. This does not measure pairwise decision accuracy on the training distribution, does not report judge agreement, and does not cover the M0, M1, and M2 models that actually generated the training data. Since the executor in Eq. (4) only checks executability rather than correctness, a judge that is wrong on even a modest fraction of pairwise comparisons will mislabel winners and misdirect the iterative SFT. The paper should provide pairwise accuracy against unit-test ground truth on a held-out subset of the training questions, or an explicit analysis of judge errors and their propagation across rounds; without this the central claim rests on an unvalidated oracle.","section":"§5.2 and Eqs. (6)–(10)"},{"comment":"All reported improvements come from a single pass with no variance, confidence intervals, or multiple seeds, and the paper states that baseline results are prioritized from leaderboards or original papers rather than re-evaluated. The headline increments on Table 1 (2.7 and 3.1 average points) and Table 2 (2.7 and 4.5 average points) are small enough that prompt-format differences, sampling temperature, and judge stochasticity could shift them. The paper should report at least three seeds or bootstrap intervals over generated samples, and should re-evaluate the principal baselines under the same evaluation harness. This is load-bearing because the 'less data while outperforming baselines' conclusion in §4.5 assumes the baseline numbers are commensurable with the paper's own evaluation settings.","section":"§4.4 and Tables 1–2"},{"comment":"The MultiPL-E results for RefineCoder-DS-6.7B are not monotone: the average pass@1 goes 57.2 (Iter0), 58.2 (Iter1), 57.5 (Iter2), 57.8 (Iter3). Since Section 1 and Section 3.1 claim continuous improvement from iterative ACR, this OOD result contradicts the claim as stated. The claim should either be restricted to the in-distribution benchmarks where monotonicity is observed, or the paper should explain the non-monotonicity and provide error bars to show whether the fluctuations are meaningful.","section":"§5.4, Table 5"},{"comment":"Baseline fairness is a load-bearing concern. The paper takes baseline numbers from leaderboards or original papers when available, and one baseline (Qwen2.5-Coder-7B-Instruct) is listed only as using 'Millions' of data, without an exact count. Because HumanEval(+) and MBPP(+) results are highly sensitive to the exact prompting harness and sampling procedure, the comparison 'outperforms baselines of the same size using only 80K data' is stronger than the evidence currently supports. I would like to see the main baseline numbers reproduced locally with the same evaluation code, together with an explicit statement of which numbers are taken from which source and what differences remain.","section":"§4.4 and §4.5"}],"minor_comments":[{"comment":"The y-axis labels in Figure 3 are garbled unicode sequences; the ablation curves are much harder to read than they should be, and the text '(without 3.1 module or without 3.2 module)' is not a substitute for readable axis labels.","section":"Figure 3"},{"comment":"There are several typos: 'provied' should be 'provided' in the pointwise judge prompt, and 'Similarity' should be 'Similarly' in Section 4.5.","section":"Figure 8 and Section 4.5"},{"comment":"The criterion weights alpha_j in Eq. (2) are never specified; if they are all equal, state this explicitly, and if not, give the values used in the experiments.","section":"Section 3.2"},{"comment":"The pairwise judge prompt allows a 'tie' verdict, but Eqs. (7)–(8) in Section 3.2 only define outcomes for strictly greater scores; the paper should state how ties are resolved.","section":"Appendix A"},{"comment":"No link to code, data, prompts, or model weights is provided. Given that the method's value depends on reproducing the dataset construction and judging pipeline, releasing these artifacts is important for the paper's utility.","section":"Availability"}],"recommendation":"major_revision","confidential_remarks":"The reader's report and the skeptic's concern align with my own reading: the core idea is sound and the paper is worth pursuing, but the judge validation and the single-run, largely copied baseline evaluation need to be strengthened before the central claims are fully supported. I would send this back for major revision rather than reject; there is no indication of any issue beyond what is stated in the report."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is worth taking seriously. ACR is a genuine new recipe: it iteratively fine-tunes a code model on data it constructs from its own sampled responses, using a composite judge (pointwise plus pairwise LLM-as-a-Judge with an executor check) to decide whether self-generated code beats the teacher response, and then either keeps the self-generated code as single-turn data or uses the teacher response as a reference to produce two-turn critique data. That selective data construction is distinct from CodeLutra, Self-Boosting, and Critique Fine-Tuning, and the SFT-only training loop is clean. The main result - consistent gains over three iterations on LiveCodeBench and BigCodeBench-hard for both DeepSeek-Coder and Qwen2.5-Coder - supports the central claim, and the leakage analysis is a real plus: their 80K dataset has low TLI, which explains why they trail on HumanEval/MBPP without hand-waving. The ablations are sensible, and the diminishing-returns appendix is honest.\n\nThe soft spots are mostly about verification, not about the core idea. No code, data, or models are released, which is a real problem for a paper whose value is in the iterative pipeline. Numbers are single-pass with no variance, and most baselines are lifted from leaderboards rather than re-evaluated. The judge validation in Section 5.2 is indirect: best-vs-worst pass rates on the final models and an error rate, but no pairwise decision accuracy on the training distribution, no judge agreement, and nothing from intermediate iterations. The stress-test concern about a biased judge favoring self-generated code is legitimate, though not fatal - the paper's own Table 4 shows the judge at least separates best from worst by 8-16 points, which is suggestive even if not conclusive. I would want direct pairwise accuracy before trusting the Eq. 10 gate fully. Also, MultiPL-E is non-monotone for the DS model (57.2->58.2->57.5->57.8), so the \"continuous improvement\" claim is slightly overstated in the multilingual setting. That is a minor overstatement, not a load-bearing flaw.\n\nWho is this for? People working on code instruction tuning or data-centric self-improvement of LLMs. It deserves a serious referee: I would send it out, with the clear expectation that the authors either release artifacts or add much stronger judge-validation and variance reporting. The method is plausible and the within-subfield contribution is solid; it just needs to be made verifiable.","headline":"A credible SFT-only iterative refinement method for code LLMs with a real empirical payoff, but the judge-dependent data gate is validated only indirectly and no artifacts are released.","tokens_in":706,"tokens_out":876,"would_cite":true,"duration_ms":29563,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Iteratively applying Adaptive Critique Refinement makes code LLMs continuously improve without imitating a teacher.","keywords":["adaptive critique refinement","code generation","self-refinement","LLM-as-a-judge","LLM-as-a-critic","iterative fine-tuning","supervised fine-tuning","code LLM"],"falsifier":"Take the 80K dataset, run one ACR iteration with the declared winners from the original judge, and on a held-out subset have hidden unit tests decide the true winner. If a nontrivial fraction of declared winners fails while the declared loser passes, or if swapping the judge for the hidden-test oracle removes the pass@1 gain, the claimed improvement depends on judge accuracy rather than on self-refinement per se.","tokens_in":18969,"feed_emoji":"🧑‍💻","tokens_out":5834,"duration_ms":46114,"temperature":0.7,"pith_summary":"This paper argues that code LLMs can improve themselves through a loop of generating code, scoring it, and training on the better of self-generated or teacher-written responses plus external critiques, rather than by imitating teacher outputs. The proposed Adaptive Critique Refinement (ACR) replaces the usual distillation pipeline with a student-like process: the model tries the problem, compares its best attempt against the teacher response, and either keeps its own solution or learns from a critique of its weaker solution. The authors build the RefineCoder series on DeepSeek-Coder-6.7B and Qwen2.5-Coder-7B using only an 80K Python instruction set, and report that three iterations raise average pass@1 on LiveCodeBench and BigCodeBench-hard from 14.2 to 16.9 (DS) and from 18.6 to 21.7 (QW), while also improving HumanEval/MBPP/BigCodeBench averages. The claim matters because it suggests that with a reliable judge and critic, a fixed amount of instruction data can be reused across rounds to extract improvement from the student's own samples instead of buying new teacher-generated data.","feed_headline":"Self-critique loop lifts code models by 3.1 points","feed_subtitle":"Three rounds of critique refinement improve hard-benchmark pass@1 using only 80K samples and plain supervised fine-tuning.","key_machinery":"The carrying mechanism is the ACR dataset update rule (Equation 10), gated by a composite scoring system. The composite score for each response combines a normalized weighted sum of point-wise judge scores over five criteria with a binary executability check from a code executor, and a pairwise judge plus executor score decides the winner between the best self-generated response and the teacher response. Depending on which side wins, the original sample $(x,y)$ is replaced either by the self-generated response $(x,y_w)$ or by a two-turn critique sample $(x, y_{l1}, c, y_w)$ generated by an LLM-as-a-Critic that explains the weakness and the better solution. Training on the resulting dataset with a plain supervised fine-tuning loss is what encodes the refinement into the weights, so inference-time calls are not needed for the gain.","core_discovery":"The paper's central claim is that iterative application of Adaptive Critique Refinement yields continuous, measurable improvement in one-shot code generation. In each round, the current model samples several code responses per instruction, a composite scoring system (multi-faceted LLM-as-a-Judge plus a code executor) selects the best self-generated response, a pairwise judge decides whether it beats the original teacher-generated response, and a selective critique step either promotes the self-generated winner as new single-turn data or generates a two-turn critique sample in which a critic explains why the weaker self-generated code failed. The updated dataset, still 80K samples, is used to SFT the base model from scratch for the next round. The paper reports that the largest jump comes in the first iteration, that gains continue but shrink over three iterations, and that the final models beat same-size baselines on the harder, leakage-analyzed benchmarks using less data. The authors also show the loop improves multilingual generalization despite Python-only training.","pith_inferences":["Going beyond the paper, the same gated winner-selection loop should transfer to other reasoning domains, such as mathematics or structured logic, whenever a composite judge can be built; the paper notes the idea is general but does not test it.","The fixed 80K dataset means each iteration's ceiling is set by instruction diversity; a natural extension is to inject fresh instructions or hard negatives each round to slow the reported diminishing returns after iteration three.","The paper's validation of the judge is indirect; a stronger test would be to compare judge-chosen winners against hidden-test execution on a sample of the 80K instructions, and to publish judge agreement statistics.","If judge noise is low enough, the method implies a data-efficiency argument for self-improvement: teacher data is a seed, not a ceiling, and the marginal value of additional teacher-generated samples may drop once the self-refinement loop is in place."],"forward_implications":["Three refinement rounds improve RefineCoder-DS-6.7B average pass@1 on LiveCodeBench and BigCodeBench-hard by 2.7 points and RefineCoder-QW-7B by 3.1 points, with the largest single gain at iteration one.","The 80K instruction set is reused every round, so the method improves capacity without expanding data or distilling new teacher responses.","Because only SFT is used, ACR is orthogonal to preference-based self-improvement and to inference-time debugging methods, and can be combined with either.","The critique data, not just the self-generated winner, carries part of the gain: ablating the second-turn critique data hurts more than removing the selective critiquing module.","The gains transfer to out-of-distribution languages on MultiPL-E even though the instruction set is Python-only."],"supporting_citations":[{"why":"Supplies DeepSeek-Coder-6.7B-Base, one of the two base models for the RefineCoder series.","marker":"(Guo et al., 2024)"},{"why":"Supplies Qwen2.5-Coder-7B-Base and the Qwen2.5-32B-Instruct model used as Judge and Critic.","marker":"(Hui et al., 2024)"},{"why":"SelfCodeAlign is one of the two strategies used to build the contamination-free 80K initial SFT set D0.","marker":"(Wei et al., 2024a)"},{"why":"WizardCoder's Evol-Instruct is the other strategy for creating the initial 80K Python instruction set.","marker":"(Luo et al., 2024)"},{"why":"Critique fine-tuning is the direct conceptual precedent for learning to critique rather than imitate.","marker":"(Wang et al., 2025)"},{"why":"CodeLutra is the prior iterative refinement approach that ACR contrasts with; it depends on golden labels, while ACR uses SFT only.","marker":"(Tao et al., 2024)"},{"why":"Self-Debugging is the inference-time refinement baseline that ACR distinguishes itself from by improving intrinsic one-shot ability.","marker":"(Chen et al., 2024)"},{"why":"OpenCodeInterpreter is a same-size baseline and provides the feedback evaluation prompts used in Section 5.5.","marker":"(Zheng et al., 2024a)"}],"fun_headline_variants":["Iterative self-critique lifts code LLMs by 3.1 points","RefineCoder: self-critique beats teacher with less data","Three refinement rounds add 3.1 points to code pass@1","Code LLMs improve by critiquing own outputs iteratively"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the LLM-as-a-Judge combined with the code executor reliably tells which response is truly better; if the judge is biased or noisy, the loop keeps training on mediocre self-generated code and critique data whose declared winner is not actually better.","fun_headline_variants_meta":{"raw":{"variants":["Iterative self-critique lifts code LLMs by 3.1 points","RefineCoder: self-critique beats teacher with less data","Three refinement rounds add 3.1 points to code pass@1","Code LLMs improve by critiquing own outputs iteratively"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00039,"raw_usage":{"total_tokens":2042,"prompt_tokens":922,"completion_tokens":1120,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":538,"completion_tokens_details":{"reasoning_tokens":1041}},"tokens_in":538,"tokens_out":1120,"duration_ms":9360,"temperature":1.0,"reasoning_tokens":1041,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T22:21:31.184750+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the 80K dataset, run one ACR iteration with the declared winners from the original judge, and on a held-out subset have hidden unit tests decide the true winner. If a nontrivial fraction of declared winners fails while the declared loser passes, or if swapping the judge for the hidden-test oracle removes the pass@1 gain, the claimed improvement depends on judge accuracy rather than on self-refinement per se.","supporting_citations":[],"review_version":1}