{"id":"2d8ef4c8-f353-4adb-a9b8-a67dd04b4d62","arxiv_id":"2505.06267","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"AKD combines adversarially sampled synthetic exercises with Direct Preference Optimization to fine-tune small code models, but its reported gains over standard fine-tuning are not supported by its own tables.","lead":"This paper introduces AKD, a training method that uses a large teacher model to generate coding exercises, a small student model to attempt them, and preference learning to push the student toward the teacher. A generalist reader might care because AKD aims to produce small, cheap coding assistants without large human-annotated datasets.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central outperformance claim is contradicted by the paper's own Table 5, where AKD and SFT both score 38% on HumanEval; the only DPO win is a single unseeded 3-point gain, so the evidence does not support the conclusion.","rationale":"The reader's verdict is REJECT, and I agree with that verdict, but my strongest reason is different from the stated weakest assumption. The reader flagged unverified teacher solutions as the key risk; that is a real threat to training-signal quality, but it is not the most decisive problem because the central claim is an empirical superiority claim. The paper's own tables contradict that claim: Table 5 gives AKD and SFT identical HumanEval scores, and Table 2 shows negative MBPP deltas in most configurations. A method can tolerate noisy teacher labels and still improve; it cannot tolerate its own evidence showing parity. I also note the loss in Section 3 is not the standard DPO loss as written (it lacks the log-ratio of policy and reference probabilities), and the reference model is never specified, which is a separate methodological concern. However, even granting the loss as intended, the empirical case fails. The format-alignment issue is important because the only benchmark with consistent gains (HumanEval) is also the format of the synthetic training data; this makes the positive results hard to interpret. A concrete rerun with seeds, variances, and all four benchmarks, plus a dataset-overlap check, would settle whether the claimed advantage is real or an artifact of format and chance. If the rerun shows no significant edge, the correct verdict is REJECT; if it shows a robust edge on all four benchmarks, the paper would need substantial revision (full method details, code, data) but the core empirical claim could be revived. Given the current text, REJECT remains appropriate.","tokens_in":17879,"tokens_out":8180,"duration_ms":82717,"concrete_test":"Re-run the Table 5 comparison (AKD vs SFT on AKD-generated data vs SFT on APPS) with the Qwen2.5-Coder-7B / Llama-3.2-1B pair for at least 5 seeds, reporting mean and standard deviation on HumanEval, HumanEval+, MBPP, and MBPP+. If AKD is not significantly better than the SFT-on-AKD-dataset baseline on the two MBPP variants as well as on HumanEval, the 'outperforms standard fine-tuning' claim is not supported. As a second check, compute the overlap or format similarity between the generated synthetic exercises and HumanEval problems (e.g., n-gram overlap of docstrings/signatures); if overlap is high, the HumanEval gains are plausibly format leakage rather than general capability.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing problem is not a missing baseline but a contradiction inside the paper. Section 5 concludes that AKD 'outperform[s] traditional methods such as standard fine-tuning and Direct Preference Optimization,' yet Table 5 reports AKD at 38% HumanEval, identical to Self-Supervised Fine-Tuning on the AKD dataset and to SFT on APPS (38%). On the broader comparison in Table 2, three of five configurations show negative deltas on MBPP or MBPP+, so the 'meaningful performance improvements' are not consistent across benchmarks. The only positive result over a DPO baseline is a single 3-point gain in Table 6, with no standard errors, no multiple seeds, and no description of the DPO baseline's construction. In addition, the caption to Table 2 states the synthetic training data is 'mainly in the HumanEval format,' which is also the benchmark where AKD's gains concentrate; the natural-language MBPP results are the weak or negative ones. This pattern makes it impossible to tell whether the claimed improvement reflects the adversarial curriculum or simply train/eval format alignment. The abstract's reliability and security claims are never measured by any security test. The central assertion therefore rests on one unreplicated, format-aligned comparison, and it is contradicted by the paper's own parity result.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Adversarial Knowledge Distillation (AKD), a fine-tuning framework for code LLMs in which a teacher model generates exercises and reference solutions, a student model generates attempt solutions, and a DPO-style loss is optimized over adversarially selected curricula. The selection is driven by margin rewards between teacher and student solutions, and three prompting strategies (incremental, opposite, deceptive) generate new exercises. Experiments compare AKD with self-supervised fine-tuning and a DPO baseline on HumanEval and MBPP for several teacher-student pairs, and also report a speculative decoding study. The central claim is that AKD yields meaningful performance improvements with limited data and outperforms standard fine-tuning and DPO.","tokens_in":18146,"tokens_out":6678,"duration_ms":65136,"significance":"If the empirical claims were reliable, AKD would be a useful contribution to automated curriculum generation for code alignment, combining self-play-style adversarial sampling with preference optimization and knowledge distillation in a resource-limited setting. The paper's strengths include concrete prompt templates in the appendix, experiments across multiple model families (1B-8B), and an honest report of a null result for speculative decoding. However, the significance is substantially limited by the absence of controlled, repeated experiments, by internal numerical contradictions, and by the fact that the headline outperformance claim is contradicted by the paper's own Table 5.","major_comments":[{"comment":"The conclusion that AKD \"outperform[s] traditional methods such as standard fine-tuning and Direct Preference Optimization\" is directly contradicted by Table 5, where AKD, SFT on the AKD dataset, and SFT on APPS all report 38% accuracy on HumanEval. This is parity, not outperformance. Moreover, Table 2 reports deltas relative to the initial student model rather than against a controlled SFT baseline, so no evidence for superiority over SFT is actually presented in the paper.","section":"Section 5 and Table 5"},{"comment":"The only evidence for an advantage over DPO is Table 6, which reports a single 3-point difference (while the text says \"4 percentage point improvement\") between a DPO baseline and AKD. The table gives no number of seeds, no standard errors, and no description of how the DPO baseline was constructed, so the claimed advantage over DPO cannot be assessed and may easily be noise.","section":"Table 6 and Section 4.3"},{"comment":"The same teacher-student pair, Qwen2.5-Coder-7B / Llama-3.2-1B, is reported with 34.7% on HumanEval in Table 2 and with 38% in Table 5, and the dataset sizes are given as 0.7M tokens and 1.6k samples, respectively. No explanation is provided for these inconsistent numbers, which erodes confidence in the reliability of the reported evaluations.","section":"Tables 2 and 5"},{"comment":"The loss function defined before the \"Adversarial Step\" is -log(exp(R(c,x))/(exp(R(c,x))+exp(R(r,x)))), which is a binary classification loss over reward estimates, not the DPO objective of Rafailov et al. that optimizes a policy ratio relative to a reference policy. The method is therefore not DPO as named, and the comparison against a \"DPO baseline\" in Table 6 is not a controlled DPO ablation.","section":"Section 3"},{"comment":"The teacher's solutions are labeled \"chosen\" without any test-execution or compiler validation, and the exercises produced by the three adversarial strategies are also not verified for correctness. In addition, the abstract's claims about improving \"security\" and \"reliability\" are never measured: no security benchmark, adversarial robustness test, or correctness-by-execution metric appears in the experiments.","section":"Sections 3 and 4"},{"comment":"Section 3 states that the appendix provides \"the exact prompts used for dataset generation, the parameters for each prompting strategy, and the iterative training schedule,\" but Appendix 6 contains only prompt templates. The values of Delta D and theta from the Incremental and Opposite approaches are not specified, and the iterative scheduling beyond \"Repetitions 5\" in Table 3 is not detailed, so the proposed method cannot be reproduced from the paper as written.","section":"Section 3 and Appendix 6"}],"minor_comments":[{"comment":"The caption's statement that \"constant improvements are seen on HumanEval\" is too strong given the negative HumanEval delta of -2.5 in the Llama3.1-8B / Llama-3.2-1B row, and the broader claim of constant improvement is further weakened by negative MBPP or MBPP+ deltas in three of five rows.","section":"Table 2 caption"},{"comment":"The subsection on speculative decoding is titled \"Speculative Decoding with Teacher-Student Framework\" and promises an evaluation of speed and accuracy, but it reports no quantitative results and only discusses why the experiment did not show improvements.","section":"Section 4.4"},{"comment":"The conclusion says the synthetic datasets contain \"approximately 1400 samples,\" while Table 5 reports a dataset size of 1.6k samples; these numbers should be reconciled.","section":"Conclusion"},{"comment":"There are numerous typos and grammatical errors, including \"exercices\", \"datasaets\", \"we have introduce\", \"This approach can improves\", and \"As a consequence to the expensive cost\". These should be corrected in any revision.","section":"Throughout"}],"recommendation":"reject","confidential_remarks":"The core contribution is presented as an outperformance result, but the paper's own tables show parity with SFT and provide only a single unseeded 3-point comparison with DPO. Combined with the numerical inconsistency between Tables 2 and 5 and the incorrect use of the DPO loss name, the evidence does not support the central claim. A resubmission with controlled multi-seed baselines, corrected loss formalism, and execution-based validation of the synthetic data could be reconsidered, but the current manuscript is not publishable in its present form."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Fair read: the idea is genuinely new — DPO with adversarial sampling of prompts based on margin rewards, plus three prompt-evolution strategies. The appendix prompt templates are concrete and the three strategies are clear. That is the useful part. The authors also deserve credit for reporting the speculative decoding experiment as a failure rather than burying it.\n\nThe empirical case is where it falls apart. The conclusion says AKD outperforms standard fine-tuning and DPO, but Table 5 shows AKD and SFT both at 38% on HumanEval — that is parity, not outperformance. Table 2 gives deltas against the initial model, not against controlled baselines, and several MBPP deltas are negative. The only direct DPO comparison is a single 3-point gain in Table 6, with no seeds, no error bars, and no description of the DPO baseline. That is not enough to support the central claim.\n\nThere are two more problems. The teacher's solutions are taken as 'chosen' without any execution check; if the teacher generates wrong code, the preference signal is garbage. And the abstract promises reliability and security improvements, but no security evaluation appears anywhere. Also, the generated data is mainly in HumanEval format, which is exactly where the gains show up; the MBPP numbers are weak or negative, so the effect could be train/eval format alignment rather than the adversarial curriculum.\n\nWho gets value from this? Someone designing synthetic-data curriculum methods for code LLMs will find the approach worth thinking about. Practitioners should not rely on the numbers. I'd read the paper for the idea and the prompts, not for the evidence.\n\nRecommendation: it deserves a serious referee — the method is coherent and the failure modes are fixable — but I would not accept it as is. If I were the editor, I'd send it out and expect the reviewers to ask for real baselines, seeds, and a security test, or a scaled-down claim.","headline":"A plausible DPO-curriculum idea that the experiments do not actually support; the paper's own Table 5 shows parity with SFT, so the central claim is unproven.","tokens_in":18688,"tokens_out":2266,"would_cite":false,"duration_ms":21958,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A training loop that turns a large language model into an adversary can align small code models on roughly 1,400 synthetic exercises.","keywords":["adversarial knowledge distillation","direct preference optimization","synthetic curricula","code generation","language model alignment","knowledge distillation","curriculum learning"],"falsifier":"Run the AKD loop exactly as specified, then execute every teacher-generated 'chosen' solution against the exercise's own test cases or compile it; if a material fraction fail, the preference signal is corrupted. Separately, replace the margin-based softmax sampler with a random or fixed-order sampler over the same synthetic exercises: if the random curriculum reaches the same HumanEval accuracy, the adversarial ordering is not what drives the gains.","tokens_in":17627,"feed_emoji":"🤖","tokens_out":6812,"duration_ms":67443,"temperature":0.7,"pith_summary":"This paper proposes Adversarial Knowledge Distillation (AKD), a training loop in which a large teacher model writes coding exercises and solutions, a small student model attempts them, and the student's failures determine which exercises appear next. The authors claim this adversarially ordered curriculum, trained with a direct preference optimization loss, yields meaningful accuracy gains on code-generation benchmarks using only about 1,400 synthetic samples, without a large human-annotated dataset. If correct, this matters because data scarcity and compute cost, not just model size, are the main bottlenecks for improving code language models: a smaller model could be aligned cheaply from a larger one, and the curriculum can be regenerated as the student improves. The reported gains are uneven across model pairs and benchmarks, and the speculative-decoding experiments showed no improvement, but the central claim stands on the HumanEval results and the comparison against a non-adversarial DPO baseline.","feed_headline":"Adversarial teacher loop beats plain DPO for code model alignment","feed_subtitle":"Margin-ranked synthetic exercises lift small models' pass rates without big human-curated datasets.","key_machinery":"The engine of the method is the margin reward $M = R(c) - R(r)$, the gap between the teacher's reward for a chosen solution and the student's reward for a rejected one, which is converted into a sampling probability $P(p) = \\exp(-M(p)) / \\sum_{p' \\in B} \\exp(-M(p'))$ over the batch of prompts. A small margin means the student cannot distinguish the better solution, so that exercise is given higher weight in the next adversarial dataset. This margin-ranked softmax is what turns open-ended synthetic data generation into a curriculum that continually targets the student's current weak points.","core_discovery":"The central claim is that a teacher-student pair engaged in an adversarial game can align a small code model to the teacher's distribution using only synthetic data. The teacher generates coding exercises and solutions that are labeled 'chosen,' while the student's attempted solutions are labeled 'rejected'; a direct preference optimization loss then pushes the student toward the teacher's answers. The adversarial part comes from the margin reward, the difference between the rewards assigned to chosen and rejected answers, and a softmax over negative margins that makes low-margin exercises more likely to be sampled in the next training round. Three prompting strategies, incremental, opposite, and deceptive, generate new exercises that target the student's weaknesses. The authors report that this pipeline improves student pass rates over standard fine-tuning and over unordered DPO, and that it matches a handcrafted 5,000-sample benchmark with a 1,600-sample synthetic dataset.","pith_inferences":["If the margin signal is computable from any reward proxy, the same loop could be ported to domains with automatic correctness signals, such as math proofs, formal specifications, or database queries, where a compiler or verifier would replace the teacher's subjective labels.","The gains concentrate on HumanEval-style function-completion exercises rather than MBPP-style natural-language problems, so a testable extension is to generate exercises in the natural-language format and check whether the adversarial-vs-DPO gap persists there.","Since the paper's own conclusion names compiler feedback as future work, adding execution-based filtering of 'chosen' solutions would either strengthen AKD by cleaning the preference labels or change its behavior when the teacher is wrong.","The absence of speculative-decoding gains within same-family teacher-student pairs suggests a natural next test: cross-family pairs that share a tokenizer, which could turn AKD into a practical factory for building assistant models for faster inference."],"forward_implications":["Small models could be aligned to larger teachers with datasets of roughly 1.4k exercise-solution pairs, sharply reducing the need for human annotations.","Adversarial curriculum sampling can beat presenting the same preference data in random order, so the ordering of exercises is itself a training signal.","The teacher-student margin can serve as an automatic indicator of where a student is weak, guiding both data generation and data selection without human intervention.","On the HumanEval benchmark, AKD matched the performance of fine-tuning on a handcrafted 5,000-sample dataset while using about 1,600 samples, suggesting synthetic data can substitute for curated benchmarks.","The same framework did not improve speculative decoding for same-family teacher-student pairs, so the benefits of AKD are real but not automatic across all uses of teacher-student setups."],"supporting_citations":[{"why":"Supplies the direct preference optimization loss that converts teacher/student solution pairs into a preference-learning objective.","marker":"(Rafailov et al., 2023)"},{"why":"Provides the HumanEval benchmark, the primary code-generation evaluation used for the accuracy comparisons.","marker":"(Chen et al., 2021)"},{"why":"Provides the MBPP benchmark, the second code-generation evaluation used in the experiments.","marker":"(Austin et al., 2021)"},{"why":"Supplies the EvalPlus framework with expanded test suites, used to measure robustness beyond the base benchmark cases.","marker":"(Liu et al., 2023)"},{"why":"Establishes the knowledge-distillation idea that AKD adapts to an adversarial fine-tuning setting.","marker":"(Hinton et al., 2015)"},{"why":"Contributes the automatic-curriculum and self-play mechanism that motivates the adversarial exercise generation.","marker":"(Sukhbaatar et al., 2018)"},{"why":"Demonstrates asymmetric self-play for automatic goal discovery, the inspiration for having the teacher propose progressively harder exercises.","marker":"(OpenAI et al., 2021)"},{"why":"Defines speculative decoding, the application setting tested in Section 4.4 to evaluate teacher-student pairs.","marker":"(Leviathan et al., 2023)"}],"fun_headline_variants":["Adversarial teacher-student loop boosts code model alignment","Synthetic adversarial exercises match handcrafted code data","AKD: Adversarial distillation trumps plain DPO for code"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the teacher model's generated solutions, labeled 'chosen' for preference training, are actually correct and aligned with expert human standards, since the paper applies no compiler or test-execution check to verify them.","fun_headline_variants_meta":{"raw":{"variants":["Adversarial teacher-student loop boosts code model alignment","Synthetic adversarial exercises match handcrafted code data","AKD: Adversarial distillation trumps plain DPO for code"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000207,"raw_usage":{"total_tokens":1383,"prompt_tokens":912,"completion_tokens":471,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":528,"completion_tokens_details":{"reasoning_tokens":416}},"tokens_in":528,"tokens_out":471,"duration_ms":5117,"temperature":1.0,"reasoning_tokens":416,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T00:00:16.395090+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the AKD loop exactly as specified, then execute every teacher-generated 'chosen' solution against the exercise's own test cases or compile it; if a material fraction fail, the preference signal is corrupted. Separately, replace the margin-based softmax sampler with a random or fixed-order sampler over the same synthetic exercises: if the random curriculum reaches the same HumanEval accuracy, the adversarial ordering is not what drives the gains.","supporting_citations":[{"cited_title":"D., and Finn, C","cited_arxiv_id":null,"evidence_quote":"Supplies the direct preference optimization loss that converts teacher/student solution pairs into a preference-learning objective."},{"cited_title":"Intrinsic motivation and automatic curricula via asymmetric self-play, 2018","cited_arxiv_id":null,"evidence_quote":"Contributes the automatic-curriculum and self-play mechanism that motivates the adversarial exercise generation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Demonstrates asymmetric self-play for automatic goal discovery, the inspiration for having the teacher propose progressively harder exercises."}],"review_version":1}