{"id":"92145b97-90f5-4039-96bb-a3990bbc43dc","arxiv_id":"2509.03537","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Adversarial teacher-student RL on computationally equivalent narrative rewrites of coding problems yields modest but consistent pass@1 gains on competitive programming benchmarks.","lead":"AR2 trains a teacher LLM to rewrite simple coding problems into story-rich disguised versions while a student LLM learns to solve them through an adversarial reinforcement learning loop. The paper reports small but consistent accuracy gains over standard RL on three competitive programming benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"AR2's benchmark gains are not yet attributable to adversarial abstraction training because the RL baseline is not matched on data distribution; no ablation isolates the adversarial loop.","rationale":"The paper's strongest claim is causal: AR2's pass@1 gains demonstrate that adversarial abstraction training improves generalization. For that inference, 'Base Model w/ RL' must be a matched control. From the text, the RL baseline is described only as 'standard reinforcement learning' (Sec. 3.2), and Table 2 lists hyperparameters for a 'baseline solver' without stating what data it sees. If the baseline trains on the original 300 kernel problems while AR2 trains on additional teacher-generated rewrites, then AR2 has strictly more and harder training data; the gains could be due to distributional coverage alone. This is a control problem, not an internal contradiction. The missing ablation is concrete and addressable: train on the same generated corpus without the adversarial loop. The GPT-o3 oracle concern is real and documented in Appendix C, but even a perfect oracle would not fix the missing control. I therefore agree with the reader's weakest-assumption identification and see no reason to change the CONDITIONAL verdict.","tokens_in":10088,"tokens_out":4093,"duration_ms":51483,"concrete_test":"Collect all teacher-generated problems actually used to train the AR2 student. Train a new student from the same base model with the standard RL algorithm on this exact fixed corpus—same prompts, same reward components, same GRPO steps, same hyperparameters—but with no teacher updates and no adversarial student-failure reward. Also include the original 300 kernel problems as a second baseline. Report pass@1 on AtCoder, HumanEval, and LiveCodeBench with bootstrap 95% CIs over problems and at least 5 seeds. If the fixed-corpus student matches AR2 within the CI, the adversarial loop is not load-bearing; if AR2 exceeds it beyond the CI, the abstraction explanation is supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central causal claim—that AR2 improves generalization specifically through adversarial abstraction training—requires that 'Base Model w/ RL' differ from AR2 only by the presence of the teacher/adversarial loop. The paper never specifies the RL baseline's training distribution (Sec. 3.2; Table 2 gives only hyper-parameters), and no ablation holds data quantity, text difficulty, or diversity fixed while removing the adversarial teacher update. The observed deltas over the RL baseline are small (AtCoder +1.215, HumanEval +0.819, LiveCodeBench +0.696 pass@1), and no error bars are reported for n=128 sampling, so even the existence of a real improvement over a matched baseline is unverified. The most parsimonious alternative is that training on a larger, more diverse set of narrative rewrites—not the adversarial loop or abstraction—produces the gains. Appendix C's documented semantic drift makes the GPT-o3 equivalence reward an additional reliability risk, but it is secondary to the missing control.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes AR2, a teacher–student adversarial reinforcement learning framework for improving abstract reasoning in code-generating LLMs. A teacher model rewrites simple algorithmic 'kernel' problems into narrative-rich, supposedly computationally equivalent versions; a student model is trained with GRPO to solve the rewritten problems by recovering the underlying computational kernel. The original kernel test cases are reused as the reward signal, exploiting computational equivalence. The authors evaluate the student (Qwen 2.5 7B Coder) on AtCoder, HumanEval, and LiveCodeBench, reporting pass@1 gains over the base model and over a base model trained with standard RL. The paper also releases the generated problem dataset and includes appendices on hyperparameters, reward dynamics, and examples of teacher-generated transformations, including acknowledged semantic-drift failures.","tokens_in":10376,"tokens_out":3378,"duration_ms":41372,"significance":"If the central causal claim were established, the contribution would be meaningful: a scalable, automated adversarial loop that generates computationally equivalent narrative problems while reusing original test cases for reward computation would provide a low-cost way to train abstraction and cross-context generalization in code LLMs. The framework is concrete, the dataset release is a useful community resource, and the cross-language result (C++-trained student improving on Python LiveCodeBench) is suggestive. However, the current evidence does not support the paper's strongest claim that the improvements are specifically due to adversarial abstraction training. The comparison against a standard RL baseline is not controlled, no statistical uncertainty is reported, and the equivalence oracle is not validated despite documented failures. These are not merely presentation issues; they directly affect the interpretability and reliability of every reported benchmark gain.","major_comments":[{"comment":"The central claim that AR2 improves generalization through adversarial abstraction training requires that the 'Base Model w/ RL' baseline be matched to AR2 except for the adversarial teacher loop. The paper never specifies what data the RL baseline is trained on: original kernel problems only, or the same narrative-rich rewrites without the adversarial teacher update? Table 2 lists only hyperparameters. In addition, no ablation holds data quantity, text difficulty, and problem diversity fixed while removing the adversarial loop. With Table 1 deltas of only +1.215 (AtCoder), +0.819 (HumanEval), and +0.696 (LiveCodeBench) pass@1, the most parsimonious explanation—that the gains come from exposure to a different or larger distribution of harder/diverse narrative rewrites—is not excluded. This is the load-bearing missing control for the paper's main conclusion.","section":"Section 3.2, Table 1; Section 3.1; Table 2"},{"comment":"No error bars, confidence intervals, or multiple-seed runs are reported for any of the three benchmarks. Pass@1 is computed from n=128 samples at temperature 0.2; the variance of this estimator is non-negligible for the reported differences (all under 1.3 points). For example, a difference of 0.8 on HumanEval with 164 problems and 128 samples may be within sampling noise. Without repeated runs or a proper variance-aware estimator, the paper has not established that AR2 improves over the RL baseline at all. The wording 'substantially outperforms' in Section 3.2 is disproportionate to the observed deltas.","section":"Section 3.2, Table 1"},{"comment":"The entire teacher reward (and therefore the adversarial training signal) is gated by R_eqv, which is assigned by GPT-o3. Appendix C explicitly documents multiple flawed transformations where the rewritten problem is 'not fully equivalent' to the original kernel, and states that 'semantic drift' occurs. The paper does not measure the oracle's agreement with human judgment, does not report how often R_eqv is correct, and does not filter or account for the documented failures. Because R_eqv is a precondition for R_dvg, R_nvt, and R_adv, oracle errors directly corrupt the teacher's training signal and can reward non-equivalent rewrites. The appendix's own acknowledgment that 'not all transformations preserve computational equivalence' undercuts the paper's assumption that original test cases can be safely reused for all generated problems. This needs either a validated oracle, a measured er","section":"Section 2.4 and Appendix C"}],"minor_comments":[{"comment":"The paper states 'Our contributions are threefold' but then lists four bullet points. Adjust the wording or merge the first two.","section":"Section 1"},{"comment":"The format tags should be printed as <think>...</think> and <answer>...</answer>; the current '⟨think⟩...⟨think⟩' notation is confusing and may be a LaTeX rendering issue.","section":"Section 2.3"},{"comment":"The GRPO objective is typeset incorrectly: 'Eq∼P(D) ,{o_i}...' is unreadable. Use standard notation with a distribution over prompts and group samples, and define all symbols in the text.","section":"Section 2.2, Eq. (1)"},{"comment":"The description of the RL baseline is incomplete even at the level of what is trained: is it the student model trained with GRPO on the original 300 kernel problems? How many GRPO steps, and which reward functions? At minimum, state the training distribution explicitly.","section":"Section 3.1 / Table 2"},{"comment":"The claim of 'emerging cross-language reasoning' is based on a single benchmark with no contamination analysis. HumanEval and LiveCodeBench Python problems may share surface forms with the teacher's rephrasings. Please add a contamination/overlap check or temper the claim.","section":"Section 3.2"},{"comment":"The references and template contain placeholders ('Conference’17', 'Washington, DC, USA', 'https://doi.org/10.1145/nnnnnnn.nnnnnnn') and an incomplete reference [12] beginning 'Qwen, :'. The manuscript should be formatted for the intended venue before publication.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The stress-test concern is well placed. The reported benchmark gains are not yet attributable to adversarial abstraction training because the RL baseline is not matched on data distribution and no ablation isolates the adversarial loop. In my view this is a fixable problem within the scope of a revision: add a controlled baseline trained on the same narrative-rich data without the teacher/adversarial update, report multiple seeds or variance-aware pass@1 intervals, and validate or filter the equivalence oracle. The paper's own Appendix C acknowledges a limitation that the main text does not fully address. I therefore recommend major revision rather than rejection or acceptance at this stage."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth a look if you work on RL for code LLMs. The core design is genuinely new as far as I can tell: a teacher rewrites kernel problems into narrative-rich but computationally equivalent versions, original test cases are reused as reward, and a student learns to extract the kernel. That's a clever way to make an adversarial loop stable and automated, and the dataset release is a real resource.\n\nThe paper is honest about its limitations, more than most: Appendix C documents flawed transformations and semantic drift, and the authors acknowledge the diversity/fidelity tension in reward design. That is good faith reporting.\n\nThe soft spots are the ones you'd expect. The key comparison is Base Model w/ RL vs Base Model w/ AR2, but the RL baseline is not described in terms of its training distribution—so it might be getting less diverse or less difficult data. The gains are tiny (about +1.2 AtCoder, +0.8 HumanEval, +0.7 LiveCodeBench pass@1) and there are no error bars or multiple seeds, so even the existence of an improvement over a matched baseline is uncertain. More importantly, no ablation holds data quantity, difficulty, or diversity fixed while removing the adversarial loop. The most parsimonious explanation for the gains is that the student simply trained on harder, more varied prompts, not that adversarial abstraction training did anything special. The GPT-o3 equivalence oracle adds risk; Appendix C shows it misses non-equivalent rewrites.\n\nI don't think these are fatal flaws. The idea is reusable and the direction is plausible. But the 'abstraction as a key skill' conclusion is overreach on the current evidence. The paper needs a proper matched baseline, error bars, and ideally an ablation that swaps adversarial teacher updates for a fixed pool of narrative rewrites.\n\nWho is this for? People doing post-training for code models, particularly on data generation / curriculum. It deserves a serious referee—it's a legitimate, clearly written proposal with a public dataset, and the empirical claims are fixable with more controlled experiments. I'd accept it for review, but I'd push hard for the controlled comparison.","headline":"AR2's training recipe is neat and the paper is readable, but the benchmark gains are small and the causal claim about abstraction is not yet supported by the controls.","tokens_in":15,"tokens_out":2126,"would_cite":true,"duration_ms":44593,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Training LLMs to strip narrative-rich programming problems down to their computational kernel improves coding accuracy on held-out benchmarks.","keywords":["abstraction","adversarial reinforcement learning","code generation","teacher-student training","computational equivalence","problem rewriting","LLM generalization","competitive programming"],"falsifier":"Train a control student on the exact same set of narrative rewrites produced by the final AR2 teacher, with the teacher's adversarial reward removed and the same total number of training steps; if its pass@1 on AtCoder, HumanEval, and LiveCodeBench matches the AR2 student, the gain comes from the rewritten data itself, not from the adversarial loop.","tokens_in":10029,"feed_emoji":"🧩","tokens_out":5855,"duration_ms":64994,"temperature":0.7,"pith_summary":"AR2 is a teacher–student reinforcement learning setup aimed at making coding LLMs better at abstraction. A teacher model rewrites simple algorithmic problems into long, narrative-dense versions that are computationally equivalent—same input-output behavior, different surface story—and a student model is trained, via reinforcement learning, to solve the rewritten versions by recovering the underlying kernel. The teacher is rewarded for producing diverse, novel, challenging rewrites; the student is rewarded for format, compilability, and passing the original test cases. The paper reports that a 7-billion-parameter student trained this way beats both its base model and a standard-RL-trained baseline on AtCoder, HumanEval, and LiveCodeBench, including solving Python problems despite being trained on C++. The authors take this as evidence that explicit abstraction training improves generalization rather than surface pattern matching.","feed_headline":"Rewriting puzzles adversarially lifts LLM coding accuracy","feed_subtitle":"A teacher-student loop trains coding models to solve unseen problems by extracting the hidden computational kernel.","key_machinery":"The load-bearing object is the computationally equivalent narrative rewrite. Each kernel problem is a simple algorithmic task with test cases; the teacher rewrites it into a richer story (for example, arrays become graphs and subarrays become connected subgraphs) while promising identical input-output behavior. Because equivalence holds, the original test cases can judge the student's code directly, giving the student a clean training signal and the teacher a clean way to know if a rewrite is faithful. Training uses group-relative policy optimization in a modified form (no value model, no standardization, no KL term) to update both models, with the teacher's reward combining equivalence veri","core_discovery":"The central claim is that abstraction can be trained directly through an adversarial RL loop, and that doing so transfers to unseen programming challenges. The mechanism is computational equivalence: every teacher rewrite must preserve the original problem's logic, so the original hidden test cases remain valid rewards. This lets the authors run a stable, fully automated teacher–student arms race—the teacher tries to disguise the kernel, the student tries to recover it—and the student's improved pass rates on fresh benchmarks are read as evidence that it has learned to abstract, not just to memorize formats.","pith_inferences":["I would not yet attribute the pass@1 gains to abstraction: the reported experiments compare AR2 as a whole against base and standard-RL models, so the adversarial loop is confounded with the extra diversity and difficulty of the rewritten problem set. A control with fixed non-adversarial rewrites would isolate the mechanism.","The paper's own appendix documents teacher rewrites that drift semantically and still pass the equivalence check; a dedicated equivalence verifier or a sample audit of the oracle would strengthen the reward signal and would be a direct follow-up.","Because the diversity reward is based on surface text similarity, the teacher may be narrowing the space of rewrites over time (the appendix shows a tendency toward graph-flavored phrasing); a semantic or structural diversity measure could broaden what the student practices on.","Given the reported cross-language transfer, a cheap extension would be to test the same loop on mathematical word problems where the underlying computation can be preserved exactly, to see whether the abstraction-training effect generalizes beyond code."],"forward_implications":["If the reported gains reproduce, explicit abstraction training becomes a viable complement to standard RL for code generation, with visible gains on held-out competitive programming tasks.","Because the student was trained in C++ yet improved on Python evaluation, the method points to surface-agnostic transfer: reasoning about the kernel, not the language, carries across.","Computational equivalence makes reward computation cheap and stable, so the approach can scale to larger problem corpora without needing a human-written test suite for every rewritten problem.","The same teacher–student loop with preserved test cases can be ported to other domains where a problem statement can be dressed up without changing its underlying computation."],"supporting_citations":[{"why":"Supplies the group-relative policy-gradient objective that updates both teacher and student.","marker":"[13]"},{"why":"Supplies the modified GRPO formulation (unstandardized advantage, no KL term) adopted by the paper for memory-efficient training.","marker":"[10]"},{"why":"The instruction-tuned base model from which the teacher is initialized.","marker":"[8]"},{"why":"The code-tuned base model from which the student is initialized.","marker":"[12]"},{"why":"Provides the AtCoder and LiveCodeBench problems used to measure held-out competitive-programming performance.","marker":"[9]"},{"why":"Provides HumanEval, the third held-out benchmark for general coding ability.","marker":"[1]"}],"fun_headline_variants":["Adversarial rewrites train LLMs to extract core logic","Teacher-student RL hides kernels, then tests LLM abstraction","New RL method boosts LLM coding by disguising problem logic","Train LLM abstraction with adversarial story-to-kernel puzzles","Disguise the kernel, find it: RL improves LLM coding"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The claim that abstraction training causes the improvement assumes the standard RL baseline differs from AR2 only in the adversarial narrative-rewriting loop; the experiments do not isolate that factor.","fun_headline_variants_meta":{"raw":{"variants":["Adversarial rewrites train LLMs to extract core logic","Teacher-student RL hides kernels, then tests LLM abstraction","New RL method boosts LLM coding by disguising problem logic","Train LLM abstraction with adversarial story-to-kernel puzzles","Disguise the kernel, find it: RL improves LLM coding"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000203,"raw_usage":{"total_tokens":1181,"prompt_tokens":664,"completion_tokens":517,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":408,"completion_tokens_details":{"reasoning_tokens":444}},"tokens_in":408,"tokens_out":517,"duration_ms":6592,"temperature":1.0,"reasoning_tokens":444,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T15:13:22.396243+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a control student on the exact same set of narrative rewrites produced by the final AR2 teacher, with the teacher's adversarial reward removed and the same total number of training steps; if its pass@1 on AtCoder, HumanEval, and LiveCodeBench matches the AR2 student, the gain comes from the rewritten data itself, not from the adversarial loop.","supporting_citations":[],"review_version":1}