AR$^2$: Adversarial Reinforcement Learning for Abstract Reasoning in Large Language Models

Cheng-Kai Yeh; Chung-Hung Kuo; Hen-Hsen Huang; Hsing-Wang Lee

REVIEW 3 major objections 6 minor 18 references

AR$^2$: Adversarial Reinforcement Learning for Abstract Reasoning in Large Language Models

T0 review · 3 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Training LLMs to strip narrative-rich programming problems down to their computational kernel improves coding accuracy on held-out benchmarks.

desk verdict AR2's training recipe is neat and the paper is readable, but the benchmark gains are small and the causal claim about abstraction is not yet supported by the controls. read the letter →

arxiv 2509.03537 v1 pith:MG37DAWE submitted 2025-08-27 cs.CL cs.AIcs.LG

Cheng-Kai Yeh , Hsing-Wang Lee , Chung-Hung Kuo , Hen-Hsen Huang This is my paper

classification cs.CLcs.AIcs.LG

keywords abstractionadversarialreinforcementlearningcodegenerationteacher-studenttrainingcomputationalequivalenceproblemrewritingLLMgeneralizationcompetitiveprogramming

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

AR2 is a teacher–student reinforcement learning setup aimed at making coding LLMs better at abstraction. A teacher model rewrites simple algorithmic problems into long, narrative-dense versions that are computationally equivalent—same input-output behavior, different surface story—and a student model is trained, via reinforcement learning, to solve the rewritten versions by recovering the underlying kernel. The teacher is rewarded for producing diverse, novel, challenging rewrites; the student is rewarded for format, compilability, and passing the original test cases. The paper reports that a 7-billion-parameter student trained this way beats both its base model and a standard-RL-trained baseline on AtCoder, HumanEval, and LiveCodeBench, including solving Python problems despite being trained on C++. The authors take this as evidence that explicit abstraction training improves generalization rather than surface pattern matching.

What carries the argument

The load-bearing object is the computationally equivalent narrative rewrite. Each kernel problem is a simple algorithmic task with test cases; the teacher rewrites it into a richer story (for example, arrays become graphs and subarrays become connected subgraphs) while promising identical input-output behavior. Because equivalence holds, the original test cases can judge the student's code directly, giving the student a clean training signal and the teacher a clean way to know if a rewrite is faithful. Training uses group-relative policy optimization in a modified form (no value model, no standardization, no KL term) to update both models, with the teacher's reward combining equivalence veri

What would settle it

Train a control student on the exact same set of narrative rewrites produced by the final AR2 teacher, with the teacher's adversarial reward removed and the same total number of training steps; if its pass@1 on AtCoder, HumanEval, and LiveCodeBench matches the AR2 student, the gain comes from the rewritten data itself, not from the adversarial loop.

Watch

Extended reading notes

Core claim

The central claim is that abstraction can be trained directly through an adversarial RL loop, and that doing so transfers to unseen programming challenges. The mechanism is computational equivalence: every teacher rewrite must preserve the original problem's logic, so the original hidden test cases remain valid rewards. This lets the authors run a stable, fully automated teacher–student arms race—the teacher tries to disguise the kernel, the student tries to recover it—and the student's improved pass rates on fresh benchmarks are read as evidence that it has learned to abstract, not just to memorize formats.

Load-bearing premise

The claim that abstraction training causes the improvement assumes the standard RL baseline differs from AR2 only in the adversarial narrative-rewriting loop; the experiments do not isolate that factor.

Editorial extensions

If this is right

If the reported gains reproduce, explicit abstraction training becomes a viable complement to standard RL for code generation, with visible gains on held-out competitive programming tasks.
Because the student was trained in C++ yet improved on Python evaluation, the method points to surface-agnostic transfer: reasoning about the kernel, not the language, carries across.
Computational equivalence makes reward computation cheap and stable, so the approach can scale to larger problem corpora without needing a human-written test suite for every rewritten problem.
The same teacher–student loop with preserved test cases can be ported to other domains where a problem statement can be dressed up without changing its underlying computation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

I would not yet attribute the pass@1 gains to abstraction: the reported experiments compare AR2 as a whole against base and standard-RL models, so the adversarial loop is confounded with the extra diversity and difficulty of the rewritten problem set. A control with fixed non-adversarial rewrites would isolate the mechanism.
The paper's own appendix documents teacher rewrites that drift semantically and still pass the equivalence check; a dedicated equivalence verifier or a sample audit of the oracle would strengthen the reward signal and would be a direct follow-up.
Because the diversity reward is based on surface text similarity, the teacher may be narrowing the space of rewrites over time (the appendix shows a tendency toward graph-flavored phrasing); a semantic or structural diversity measure could broaden what the student practices on.
Given the reported cross-language transfer, a cheap extension would be to test the same loop on mathematical word problems where the underlying computation can be preserved exactly, to see whether the abstraction-training effect generalizes beyond code.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Worth a look if you work on RL for code LLMs. The core design is genuinely new as far as I can tell: a teacher rewrites kernel problems into narrative-rich but computationally equivalent versions, original test cases are reused as reward, and a student learns to extract the kernel. That's a clever way to make an adversarial loop stable and automated, and the dataset release is a real resource.

The paper is honest about its limitations, more than most: Appendix C documents flawed transformations and semantic drift, and the authors acknowledge the diversity/fidelity tension in reward design. That is good faith reporting.

The soft spots are the ones you'd expect. The key comparison is Base Model w/ RL vs Base Model w/ AR2, but the RL baseline is not described in terms of its training distribution—so it might be getting less diverse or less difficult data. The gains are tiny (about +1.2 AtCoder, +0.8 HumanEval, +0.7 LiveCodeBench pass@1) and there are no error bars or multiple seeds, so even the existence of an improvement over a matched baseline is uncertain. More importantly, no ablation holds data quantity, difficulty, or diversity fixed while removing the adversarial loop. The most parsimonious explanation for the gains is that the student simply trained on harder, more varied prompts, not that adversarial abstraction training did anything special. The GPT-o3 equivalence oracle adds risk; Appendix C shows it misses non-equivalent rewrites.

I don't think these are fatal flaws. The idea is reusable and the direction is plausible. But the 'abstraction as a key skill' conclusion is overreach on the current evidence. The paper needs a proper matched baseline, error bars, and ideally an ablation that swaps adversarial teacher updates for a fixed pool of narrative rewrites.

Who is this for? People doing post-training for code models, particularly on data generation / curriculum. It deserves a serious referee—it's a legitimate, clearly written proposal with a public dataset, and the empirical claims are fixable with more controlled experiments. I'd accept it for review, but I'd push hard for the controlled comparison.

Referee Report

3 major / 6 minor

Summary. The paper proposes AR2, a teacher–student adversarial reinforcement learning framework for improving abstract reasoning in code-generating LLMs. A teacher model rewrites simple algorithmic 'kernel' problems into narrative-rich, supposedly computationally equivalent versions; a student model is trained with GRPO to solve the rewritten problems by recovering the underlying computational kernel. The original kernel test cases are reused as the reward signal, exploiting computational equivalence. The authors evaluate the student (Qwen 2.5 7B Coder) on AtCoder, HumanEval, and LiveCodeBench, reporting pass@1 gains over the base model and over a base model trained with standard RL. The paper also releases the generated problem dataset and includes appendices on hyperparameters, reward dynamics, and examples of teacher-generated transformations, including acknowledged semantic-drift failures.

Significance. If the central causal claim were established, the contribution would be meaningful: a scalable, automated adversarial loop that generates computationally equivalent narrative problems while reusing original test cases for reward computation would provide a low-cost way to train abstraction and cross-context generalization in code LLMs. The framework is concrete, the dataset release is a useful community resource, and the cross-language result (C++-trained student improving on Python LiveCodeBench) is suggestive. However, the current evidence does not support the paper's strongest claim that the improvements are specifically due to adversarial abstraction training. The comparison against a standard RL baseline is not controlled, no statistical uncertainty is reported, and the equivalence oracle is not validated despite documented failures. These are not merely presentation issues; they directly affect the interpretability and reliability of every reported benchmark gain.

major comments (3)

[Section 3.2, Table 1; Section 3.1; Table 2] The central claim that AR2 improves generalization through adversarial abstraction training requires that the 'Base Model w/ RL' baseline be matched to AR2 except for the adversarial teacher loop. The paper never specifies what data the RL baseline is trained on: original kernel problems only, or the same narrative-rich rewrites without the adversarial teacher update? Table 2 lists only hyperparameters. In addition, no ablation holds data quantity, text difficulty, and problem diversity fixed while removing the adversarial loop. With Table 1 deltas of only +1.215 (AtCoder), +0.819 (HumanEval), and +0.696 (LiveCodeBench) pass@1, the most parsimonious explanation—that the gains come from exposure to a different or larger distribution of harder/diverse narrative rewrites—is not excluded. This is the load-bearing missing control for the paper's main conclusion.
[Section 3.2, Table 1] No error bars, confidence intervals, or multiple-seed runs are reported for any of the three benchmarks. Pass@1 is computed from n=128 samples at temperature 0.2; the variance of this estimator is non-negligible for the reported differences (all under 1.3 points). For example, a difference of 0.8 on HumanEval with 164 problems and 128 samples may be within sampling noise. Without repeated runs or a proper variance-aware estimator, the paper has not established that AR2 improves over the RL baseline at all. The wording 'substantially outperforms' in Section 3.2 is disproportionate to the observed deltas.
[Section 2.4 and Appendix C] The entire teacher reward (and therefore the adversarial training signal) is gated by R_eqv, which is assigned by GPT-o3. Appendix C explicitly documents multiple flawed transformations where the rewritten problem is 'not fully equivalent' to the original kernel, and states that 'semantic drift' occurs. The paper does not measure the oracle's agreement with human judgment, does not report how often R_eqv is correct, and does not filter or account for the documented failures. Because R_eqv is a precondition for R_dvg, R_nvt, and R_adv, oracle errors directly corrupt the teacher's training signal and can reward non-equivalent rewrites. The appendix's own acknowledgment that 'not all transformations preserve computational equivalence' undercuts the paper's assumption that original test cases can be safely reused for all generated problems. This needs either a validated oracle, a measured er

minor comments (6)

[Section 1] The paper states 'Our contributions are threefold' but then lists four bullet points. Adjust the wording or merge the first two.
[Section 2.3] The format tags should be printed as <think>...</think> and <answer>...</answer>; the current '⟨think⟩...⟨think⟩' notation is confusing and may be a LaTeX rendering issue.
[Section 2.2, Eq. (1)] The GRPO objective is typeset incorrectly: 'Eq∼P(D) ,{o_i}...' is unreadable. Use standard notation with a distribution over prompts and group samples, and define all symbols in the text.
[Section 3.1 / Table 2] The description of the RL baseline is incomplete even at the level of what is trained: is it the student model trained with GRPO on the original 300 kernel problems? How many GRPO steps, and which reward functions? At minimum, state the training distribution explicitly.
[Section 3.2] The claim of 'emerging cross-language reasoning' is based on a single benchmark with no contamination analysis. HumanEval and LiveCodeBench Python problems may share surface forms with the teacher's rephrasings. Please add a contamination/overlap check or temper the claim.
[General] The references and template contain placeholders ('Conference’17', 'Washington, DC, USA', 'https://doi.org/10.1145/nnnnnnn.nnnnnnn') and an incomplete reference [12] beginning 'Qwen, :'. The manuscript should be formatted for the intended venue before publication.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: AR2's training and evaluation are self-contained; the RL-baseline confound is an experimental-design concern, not a circular dependency.

full rationale

The paper does not present a mathematical derivation or fitted constant that its conclusions reduce to. The student model is trained on LeetCode kernel problems and narrative rewrites, with rewards computed from the original kernel test cases; evaluation is performed on AtCoder, HumanEval, and LiveCodeBench, which are external benchmarks not used to set reward weights or hyperparameters. The teacher's equivalence verification uses GPT-o3 as an oracle, which is an accuracy assumption rather than a definitional identity. No self-citation is load-bearing: all cited methods (GRPO, Qwen, LiveCodeBench, etc.) are external prior work. The absence of a matched RL baseline is a legitimate experimental-control weakness, but it does not make the claimed improvement equivalent to the training signal by construction. Appendix C's documented semantic drift is a reliability limitation, not a circular step. Therefore no circularity is present.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim rests mainly on unvalidated tooling assumptions (GPT-o3 as equivalence oracle, automated test-case generation), hand-chosen hyperparameters, and the absence of a controlled ablation for the abstraction component. No fitted constants or invented entities are introduced, but the empirical comparison does not rule out distribution shift as the cause of the gains.

free parameters (5)

Learning rate = 1e-6
Fixed for teacher and student; no sensitivity analysis or schedule reported (Section 3.1).
GRPO steps per adversarial iteration = teacher 40, student 100
Chosen by hand; no ablation showing these counts are sufficient for stable convergence (Section 3.1).
Reward coefficients = student: format 1/-1, compile 2/-2, accuracy 3/-3; teacher: format and equivalence 1/0, diversity and novelty as 1 minus
Integer weights are chosen without sensitivity analysis, and no alternative reward balances are tested (Sections 2.3, 2.4).
KL regularization and advantage standardization = beta=0, no standardization, no KL term
Removed to save GPU memory following Liu et al.; no ablation shows this does not affect stability or the final gains (Section 2.2).
Candidate samples per prompt = student 24, teacher 21, evaluation 128
Chosen for memory and protocol; no analysis of how group size affects GRPO or pass@1 estimates (Appendix A).

assumptions (4)

domain assumption GPT-o3 verification of computational equivalence is reliable enough to serve as the teacher reward oracle.
Section 2.4 uses GPT-o3 verdicts as the equivalence reward, but the paper reports no accuracy, false positive rate, or human validation of this oracle. Appendix C shows the teacher can produce non-equivalent rewrites.
domain assumption Automatically generated test-case generators and reference solutions correctly encode the intended kernel problem after the GPT-o3-based validation.
Section 3.1 filters 400 LeetCode problems to 300 using automated generation and validation, with no human audit of the remaining triplets.
domain assumption Removing the KL term and using unstandardized advantage preserves GRPO training stability and does not itself cause the reported gains.
Section 2.2 modifies GRPO to save memory, but no controlled comparison with standard GRPO is reported.
ad hoc to paper The improvement over the standard RL baseline is attributable to abstraction-oriented narrative rewrites rather than to differences in data distribution or training length.
The paper asserts this causal story in Sections 3.2 and 4, but Table 1 only compares against an underspecified RL baseline and no ablation isolates the adversarial narrative component.

how reviews work

0 comments

Cite this review

Pith. "Pith review of AR$^2$: Adversarial Reinforcement Learning for Abstract Reasoning in Large Language Models." pith.science (2026). https://pith.science/paper/MG37DAWE

@misc{pith2026250903537,
  author       = {Pith},
  title        = {Pith review of: AR$^2$: Adversarial Reinforcement Learning for Abstract Reasoning in Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MG37DAWE}},
  note         = {Machine review of arXiv:2509.03537}
}

abstract

Abstraction--the ability to recognize and distill essential computational patterns from complex problem statements--is a foundational skill in computer science, critical both for human problem-solvers and coding-oriented large language models (LLMs). Despite recent advances in training LLMs for code generation using reinforcement learning (RL), most existing approaches focus primarily on superficial pattern recognition, overlooking explicit training for abstraction. In this study, we propose AR$^2$ (Adversarial Reinforcement Learning for Abstract Reasoning), a novel framework explicitly designed to enhance the abstraction abilities of LLMs. AR$^2$ employs a teacher model to transform kernel problems into narrative-rich, challenging descriptions without changing their fundamental logic. Simultaneously, a student coding model is trained to solve these complex narrative problems by extracting their underlying computational kernels. Experimental results demonstrate that AR$^2$ substantially improves the student model's accuracy on previously unseen, challenging programming tasks, underscoring abstraction as a key skill for enhancing LLM generalization.

Figures

Figures reproduced from arXiv: 2509.03537 by the authors.

**Figure 2.** Overview of our AR2 framework for enhancing abstract reasoning in LLMs via adversarial reinforcement learning. A Problem Giver (teacher) rewrites a simple kernel problem into a computationally equivalent, narrative-rich version, preserving the original test cases for direct and efficient reward computation. The Problem Solver (student) extracts the underlying abstraction and generates a solution, which is evaluate… view at source ↗

**Figure 3.** Teacher model reward curve during AR2 training. Convergence reflects the balance between generating challenging problems and student adaptation. B Adversarial Reward Dynamics Understanding the reward dynamics of teacher and student models provides insight into how AR2 fosters abstraction learning. These reward trends correspond directly to the reward functions defined in Section 2.3 and Section 2.4. By tracking the… view at source ↗

Figures from the paper (2 more)

**Figure 5.** Figure 5: An example of teacher evolution: a kernel prob [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]

**Figure 6.** Figure 6: Examples of flawed transformations. Semantic drift [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

18 extracted references · 4 canonical work pages

[1]

Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian...

arXiv 2021
[2]

Le, Sergey Levine, and Yi Ma

Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V. Le, Sergey Levine, and Yi Ma. 2025. SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training. arXiv:2501.17161 [cs.AI] https://arxiv.org/abs/2501.17161

arXiv 2025
[3]

DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai D...

arXiv 2025
[4]

Shihan Dou, Yan Liu, Haoxiang Jia, Limao Xiong, Enyu Zhou, Wei Shen, Junjie Shan, Caishuang Huang, Xiao Wang, Xiaoran Fan, et al. 2024. Stepcoder: Improve code generation with reinforcement learning from compiler feedback. arXiv preprint arXiv:2402.01391 (2024)

arXiv 2024
[5]

Ahmed El-Kishky, Alexander Wei, Andre Saraiva, Borys Minaiev, Daniel Selsam, David Dohan, Francis Song, Hunter Lightman, Ignasi Clavera, Jakub Pachocki, et al. 2025. Competitive programming with large reasoning models.arXiv preprint arXiv:2502.06807 (2025)

arXiv 2025
[6]

Patrick Haluptzok, Matthew Bowers, and Adam Tauman Kalai. 2022. Generating Programming Puzzles to Train Language Models. Work- shop. https://www.microsoft.com/en-us/research/publication/generating- programming-puzzles-to-train-language-models/

work page 2022
[7]

Patrick Haluptzok, Matthew Bowers, and Adam Tauman Kalai. 2022. Language models can teach themselves to program better. arXiv preprint arXiv:2207.14502 (2022)

arXiv 2022
[8]

Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Kai Dang, et al. 2024. Qwen2. 5-Coder Technical Report. arXiv preprint arXiv:2409.12186 (2024)

arXiv 2024

Show all 18 references

[9]

Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. 2024. Live- CodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code. arXiv preprint arXiv:2403.07974 (2024)

2024 arXiv
[10]

Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. 2025. Understanding r1-zero-like training: A critical perspective. arXiv preprint arXiv:2503.20783 (2025)

2025 arXiv
[11]

Shanghaoran Quan, Jiaxi Yang, Bowen Yu, Bo Zheng, Dayiheng Liu, An Yang, Xu- ancheng Ren, Bofei Gao, Yibo Miao, Yunlong Feng, Zekun Wang, Jian Yang, Zeyu Cui, Yang Fan, Yichang Zhang, Binyuan Hui, and Junyang Lin. 2025. CodeElo: Benchmarking Competition-level Code Generation o...

2025 arXiv
[12]

Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, ...

2024 arXiv
[13]

Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. DeepSeek- Math: Pushing the Limits of Mathematical Reasoning in Open Language Models. arXiv:2402.03300 [cs.CL] https://arxiv.org/abs/2402.03300

2024 arXiv
[14]

Parshin Shojaee, Aneesh Jain, Sindhu Tipirneni, and Chandan K Reddy. 2023. Execution-based code generation using deep reinforcement learning. arXiv preprint arXiv:2301.13816 (2023)

2023 arXiv
[15]

Sutton and Andrew G

Richard S. Sutton and Andrew G. Barto. 2018. Reinforcement Learning: An Intro- duction. A Bradford Book, Cambridge, MA, USA

2018
[16]

abcdefg”, then valid words are “faced

Shuhe Wang, Shengyu Zhang, Jie Zhang, Runyi Hu, Xiaoya Li, Tianwei Zhang, Jiwei Li, Fei Wu, Guoyin Wang, and Eduard Hovy. 2025. Reinforcement Learning Enhanced LLMs: A Survey. arXiv:2412.10400 [cs.CL] https://arxiv.org/abs/2412. 10400 A Hyper-parameter Setup Table 2 summarizes...

2025 arXiv
[17]

the word is a subset of the puzzle’s characters)

In the original problem, a word is valid if it (a) contains the first letter of the puzzle and (b) every letter in the word is found in the puzzle (i.e. the word is a subset of the puzzle’s characters)
[18]

includes the first character of the puzzle and contains all the characters from the puzzle

In the rewritten problem, the validity condition is stated as “includes the first character of the puzzle and contains all the characters from the puzzle. ” This erroneously implies that a valid word must include every single letter present in the puzzle, which is not the same...

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.

[1] [1]

Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian...

arXiv 2021

[2] [2]

Le, Sergey Levine, and Yi Ma

Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V. Le, Sergey Levine, and Yi Ma. 2025. SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training. arXiv:2501.17161 [cs.AI] https://arxiv.org/abs/2501.17161

arXiv 2025

[3] [3]

DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai D...

arXiv 2025

[4] [4]

Shihan Dou, Yan Liu, Haoxiang Jia, Limao Xiong, Enyu Zhou, Wei Shen, Junjie Shan, Caishuang Huang, Xiao Wang, Xiaoran Fan, et al. 2024. Stepcoder: Improve code generation with reinforcement learning from compiler feedback. arXiv preprint arXiv:2402.01391 (2024)

arXiv 2024

[5] [5]

Ahmed El-Kishky, Alexander Wei, Andre Saraiva, Borys Minaiev, Daniel Selsam, David Dohan, Francis Song, Hunter Lightman, Ignasi Clavera, Jakub Pachocki, et al. 2025. Competitive programming with large reasoning models.arXiv preprint arXiv:2502.06807 (2025)

arXiv 2025

[6] [6]

Patrick Haluptzok, Matthew Bowers, and Adam Tauman Kalai. 2022. Generating Programming Puzzles to Train Language Models. Work- shop. https://www.microsoft.com/en-us/research/publication/generating- programming-puzzles-to-train-language-models/

work page 2022

[7] [7]

Patrick Haluptzok, Matthew Bowers, and Adam Tauman Kalai. 2022. Language models can teach themselves to program better. arXiv preprint arXiv:2207.14502 (2022)

arXiv 2022

[8] [8]

Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Kai Dang, et al. 2024. Qwen2. 5-Coder Technical Report. arXiv preprint arXiv:2409.12186 (2024)

arXiv 2024