Multi-turn RL with Structural and Performance Aware Rewards for CUDA Kernel Generation

Ali Jannesari; Nesreen K. Ahmed; Quazi Ishtiaque Mahmud

REVIEW 4 major objections 5 minor 26 references

Multi-turn RL with Structural and Performance Aware Rewards for CUDA Kernel Generation

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read CudaPerf argues that combining an offline-learned structural code score with execution rewards makes RL-generated CUDA kernels both more correct and faster, with 89% correctness and 11.02x geomean speedup reported on its C-to-CUDA…

desk verdict CudaPerf's composite reward idea is genuinely new and plausibly useful, but the paper's evidential base needs a ranker-generalization check and released code before the big numbers can be trusted. read the letter →

arxiv 2607.20908 v1 pith:PWE6XCK4 submitted 2026-07-23 cs.LG cs.AI

Quazi Ishtiaque Mahmud , Nesreen K. Ahmed , Ali Jannesari This is my paper · ORCID

classification cs.LGcs.AI

keywords CUDAkernelgenerationreinforcementlearningwithverifiablerewardsstructuralcodememorycoalescingpairwiserankingGRPOCtotranslationPyTorch

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to show that reinforcement learning for CUDA kernel generation should reward program structure, not just measured outcomes. Its framework, CudaPerf, adds an offline-learned structural score over source features such as memory coalescing, occupancy, arithmetic intensity, and synchronization counts to a verifiable execution reward based on compilation, test pass rate, and speedup. The policy is trained with group-relative policy optimization and up to five refinement turns that feed compiler errors, failed test cases, measured speedup, and structural feature values back into the prompt. The central empirical claim is that both reward components matter: ablations report that removing the structural reward drops correctness by 7–9 percentage points and geomean speedup by 1.72–2.92X across three benchmarks, and removing the verifiable reward drops them further. If this is right, RL-based code generation can be guided by interpretable performance features in addition to execution feedback, which matters because execution feedback alone is sparse and expensive.

What carries the argument

The load-bearing mechanism is the structural reward $R_{\mathrm{str}}(y) = s_{\psi}(\hat{\phi}(y))$, produced by an offline pairwise ranker. The ranker is an MLP trained with a logistic pairwise ranking loss to assign higher scores to CUDA variants drawn from the top measured-speedup quartile than from the bottom quartile, using an 11-dimensional normalized feature vector $\phi(y) = [\mathrm{coal}, \mathrm{ai}, \mathrm{occ}, \mathrm{div}, \mathrm{xfer}, \mathrm{atomics}, \mathrm{sync}, \mathrm{kernels}, \mathrm{tpb}, \mathrm{gmem}, \mathrm{ops}]$. These features approximate memory coalescing, arithmetic intensity, occupancy, control divergence, data transfer overhead, atomic and synchronization counts, kernel count, threads per block, global memory accesses, and operation counts. During RL this score is added to the verifiable reward $R_{\mathrm{ver}}(y,x)$, which applies a compile penalty, a correctness penalty proportional to failed tests, a log-scaled and clipped speedup reward, and a runtime variance penalty. The combined reward is normalized across each group of 16 generated candidates and fed into the group-relative policy objective; the structural term is what provides a dense, execution-free signal that can steer the policy before a correct, fast kernel is found.

What would settle it

Take a held-out set of CUDA kernels produced by a different generator or written by hand, measure their actual speedup, and compute the rank correlation between $R_{\mathrm{str}}$ and measured speedup; if the correlation is not clearly positive, the structural proxy is not doing the claimed work. A complementary check is to retrain CudaPerf with the ranker's pairwise labels randomized or inverted and see whether the reported gains in Table 3 survive.

Watch

Extended reading notes

Core claim

The central discovery, stated on the paper's own terms, is that a composite reward $R_{\mathrm{ver}}(y,x) + R_{\mathrm{str}}(y)$ drives a Qwen-3-based policy to generate CUDA kernels that are both correct more often and faster than baselines that use only execution rewards or only structural rewards. On the paper's C-to-CUDA evaluation split it reports 89% correctness and 11.02X geomean speedup; on BabelTower, 94% and 9.02X; and on KernelBench, 93% and 6.41X, with the corresponding comparisons to strong baselines given in Table 1. The ablation in Table 3 shows each reward component contributes: with structural reward only, correctness is 75–81% and speedup 3.7–6.9X; with verifiable reward only, 81–87% and 4.3–9.3X; with both, 89–94% and 6.41–11.02X. The paper interprets this as evidence that the model learns both what works and why it works.

Load-bearing premise

The load-bearing premise is that the 11 hand-crafted source-level features and the ranker trained on them are a reliable enough proxy for runtime performance on kernels outside the Qwen-3-generated training distribution; if that proxy fails, the structural reward can push the policy toward code that looks optimized rather than code that runs fast, and the paper's own Section 5 acknowledges the approach is limited by execution cost and by the model and reward design.

Editorial extensions

If this is right

Removing either reward component degrades every benchmark: without structural rewards correctness falls 7–9 points and geomean speedups fall 1.72–2.92X, while without verifiable rewards the drops are larger, so both signals are load-bearing in the reported system.
The same two-stage recipe—offline ranker, then online RL with composite reward—is applied successfully to both C-to-CUDA and PyTorch-to-CUDA translation, suggesting the approach is not tied to a single translation direction.
Larger base policy models yield stronger results: scaling the Qwen-3 policy from 8B to 32B raises correctness and geomean speedup on all three evaluation sets, consistent with the claim that a stronger prior helps RL for kernel generation.
Because the ranker is trained once offline and the structural reward is execution-free, the method turns a sparse outcome signal into a dense per-candidate score during RL, which is what the ablation attributes the extra gains to.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A direct test of the paper's proxy assumption would be to measure the rank correlation between $R_{\mathrm{str}}$ and actual runtime speedup on kernels written by a different generator or by hand; positive correlation would support transfer, while near-zero correlation would suggest the structural reward is overfit to the training distribution.
The 11-feature vector omits several known GPU bottlenecks, including shared-memory bank conflicts, cache reuse, and tensor-core utilization; adding such features could widen the reported speedup gains on memory- and tensor-heavy workloads.
The paper's Section 5 limitations note that execution-based training is expensive and that iterative refinement is bounded by the model and reward design; a natural extension is to anneal the structural reward's weight as the verifiable reward becomes informative, reducing both cost and reward-hacking risk.
The prompt construction sorts few-shot examples by measured speedup, so an evaluation on a benchmark whose distribution differs strongly from the training pool would test whether the gains reflect learned optimization principles rather than familiar example patterns.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Two things to know. First, the core idea is new enough: pairing an offline pairwise ranker over static CUDA features with execution-based GRPO does not appear in the cited prior work, which sticks to correctness and speedup rewards. Second, the paper is currently a good idea with incomplete verification. The central claim—that adding a structural reward to the verifiable reward helps both correctness and speed—is plausible and the ablations show it directionally, but the evidence as reported does not pin down whether the gain comes from the CUDA-specific structure or just from a denser reward.

What is good: the recipe is concrete and reproducible in principle. The feature extractor is described in detail in the appendix, the MLP ranker is small and cheap, the RL setup follows standard GRPO, and the evaluation covers C-to-CUDA and PyTorch-to-CUDA with several strong baselines. The ablation on reward components (Table 3) and the pass@1/3/5 breakdown in the appendix are the right things to report, and the results are consistent in direction across all three benchmarks. The related-work coverage is adequate and the limitations paragraph is honest about training cost and the model's ceiling.

Soft spots, in rough order of severity. No code or data are released, so none of the numbers can be checked. The structural ranker is never validated: no rank correlation with measured speedups, no held-out accuracy, no seed variance. Since Rstr is a fitted surrogate of Rver, the "white-box reasoning" claim in Section 2.2.1 is overstated. BabelTower is used both to train the ranker and as an evaluation benchmark, and the paper does not say how the eval set was excluded from training. Also, the headline results are pass@5 with no error bars; Table 4 shows that at pass@1 the differences shrink, and on BabelTower the baseline QiMeng-MuPa is actually ahead in pass@1 correctness (82 vs 76). One minor thing: the abstract's "5X" is the additive speedup difference (11.02 - 6.2), so it is consistent with Table 1, just awkwardly phrased.

I do not think any of this sinks the paper. The stress-test worry about reward hacking is real but unproven; the ablation gains could come from generic dense reward, but they could also come from structure, and the paper as written cannot distinguish the two.

Who this is for: researchers working on RLVR for code generation and GPU kernel synthesis. It deserves a serious referee, provided the first requests are for code, data, and a ranker-generalization analysis. I would send it out rather than desk reject.

Referee Report

4 major / 5 minor

Summary. The paper proposes CudaPerf, a two-stage RL framework for CUDA kernel generation. In an offline stage, an MLP ranker is trained with a pairwise logistic loss on measured speedups to produce a structural reward Rstr(y) from hand-crafted source-level features (coalescing, arithmetic intensity, occupancy, divergence, synchronization, etc.). In an online stage, GRPO optimizes a composite reward combining this structural reward with an execution-based verifiable reward Rver(y,x) that includes compilation, correctness, speedup, and stability terms. The method also uses multi-turn refinement with execution feedback and structural feature values. The paper reports large gains in correctness and speedup over strong baselines on a new C-to-CUDA dataset, BabelTower, and KernelBench, plus ablations showing both reward components contribute.

Significance. The core idea is timely and potentially valuable: injecting performance-relevant structural priors into RLVR for kernel generation, rather than relying solely on sparse execution outcomes, could improve both sample efficiency and final performance. The curated dataset of 2.9k C-to-CUDA and 1k PyTorch-to-CUDA programs with multiple CUDA implementations is a useful resource if released, and the systematic model-size study in Table 2 is informative. However, the empirical claims are not yet rigorously supported: all tables report single runs without variance or significance testing, the offline ranker is never validated, and the BabelTower evaluation appears contaminated by ranker training data. If these issues are addressed, the paper would be a solid contribution to the RL-for-code-generation literature.

major comments (4)

[Section 3 (Training offline ranker) and Table 1 (BabelTower row)] The offline ranker is trained using BabelTower data, as stated in Section 3: 'We use existing dataset like BabelTower ... to some extent'. CudaPerf is then evaluated on the BabelTower evaluation set in Table 1 (94% correctness, 9.02X speedup). This is a training/evaluation overlap that directly contaminates the BabelTower results and can inflate the reported improvements over Qwen-3-32B. The paper must either train the ranker without BabelTower or evaluate on a genuinely held-out set with a clear data split; the current presentation does not support the BabelTower claims.
[Section 2.1, Eq. (1), Appendix A.13, Table 3] The structural reward Rstr is a fitted MLP trained on pairwise preferences derived from measured speedups. No validation of this ranker is reported: no held-out accuracy, no rank correlation with measured speedups, no comparison with a linear baseline. Because Table 3 attributes large gains (1.72X-2.92X speedup and 7-9% correctness) to the presence of Rstr, the results are equally consistent with a generic dense-reward effect that improves RL credit assignment. Please report ranker validation on held-out tasks, and add a control condition where the structural reward is replaced by a reward-matched dense signal that does not encode CUDA structure, to support the interpretation that structure, not merely density, drives the gains.
[Section 3 (Metrics) and Tables 1-4] The pass@5 protocol is not defined. The text states only that correctness means passing all test cases and performance is the geomean speedup, but it does not specify how the 5 candidates are sampled, how correctness is aggregated across them (e.g., any-of-5 vs majority), or which candidate's speedup enters the geomean (e.g., fastest correct candidate vs average over correct candidates). Without this definition, the headline speedups (11.02X, 9.02X, 6.41X) and all comparisons to baselines are not reproducible. Please specify the protocol and, ideally, report per-task distributions or confidence intervals.
[Section 2.1 vs Appendix A.11] The structural reward is defined inconsistently: Section 2.1 says Rstr = sψ(φ(y)) with an MLP containing hidden layers, while Appendix A.11 gives Rstr(y) = w^T (φ(y)-μ)/σ, a linear form. This discrepancy must be resolved because it affects the reproducibility of the reward and the claim that the ranker is nonlinear. Also, the fixed thresholds in Appendix A.4, A.7, and A.10 and the α weights in Section 2.2.1 are set empirically without sensitivity analysis; given that Table 3's central ablation depends on the calibration of these values, at least a brief robustness check is needed.

minor comments (5)

[Abstract and Section 1] The abstract reports 'up to 5X & 3.32X improvements', while the introduction's contribution list says 'up to 5× and 6× speedups over strong baselines such as Qwen-3-32B and CUDA-Agent'; these numbers are inconsistent and should be reconciled.
[Section 2.2.1, Verifiable Reward] In the performance-reward definition, 'TCPU(y)' is listed twice; the second term should presumably be 'TGPU(y)'.
[Abstract and Section 3 (Prompt Construction)] There are several typos: 'Arithmatic' should be 'Arithmetic', and 'perforamnce' should be 'performance'.
[Appendix A.16] The text refers to 'Table A.16', but the appendix table numbering is not consistent with the text reference; please correct the cross-reference.
[Section 2.2.1] The phrase 'white-box reasoning over program performance' is an overstatement, since Rstr is a learned MLP on hand-crafted features rather than an interpretable performance model; consider softening this claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the structural reward is an openly fitted surrogate of measured speedup, and the paper's remaining concerns are validation/leakage gaps, not definitional circularity.

full rationale

CudaPerf's derivation chain is an empirical pipeline, not a formal derivation from first principles. The structural reward Rstr (Eq. 1) is transparently trained by pairwise logistic loss on measured speedups (Sec. 2.1), so it is a fitted surrogate of execution performance rather than an independent structural model; this makes the paper's 'white-box reasoning' wording an overstatement, and the absence of any held-out ranker validation (e.g., rank correlation with measured speedups) means the Table 3 gains cannot be attributed to structural generalization. However, none of this is circularity under the strict definition: no equation defines an output variable in terms of the quantity it is used to predict, no fitted parameter is relabeled as a prediction of a distinct quantity, and no load-bearing premise is justified only by a self-citation. The self-citations (CodeRosetta, OptiML) appear only as baselines or related work and are not load-bearing. The BabelTower evaluation is a potential data-leakage concern because BabelTower is also named as ranker training data (Sec. 3, 'Training offline ranker'), but the paper does not state whether the evaluation set overlaps; leakage is a correctness risk, not circularity. Section 5's limitation that CudaPerf 'remains limited by the model and reward design' is consistent with the surrogate-reward interpretation. The central claim—that adding a learned structural surrogate to verifiable execution rewards improves correctness and speedup—is an empirical ablation result that could have failed and is not forced by construction.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The method introduces no new physical or ontological entities, but it does introduce several hand-set reward weights and a learned surrogate Rstr that is fitted to measured speedups. The central empirical claim therefore rests on the validity of the static feature proxy, on the candidate-generation distribution created by Qwen-3-32B plus filtering, and on hardware-specific execution measurement. The structural reward is a fitted predictor rather than a first-principles performance model.

free parameters (5)

alpha_comp, alpha_wrong, alpha_speed, alpha_cv = 3.0, 2.0, 1.0, 0.5
Hand-set reward weights in Section 2.2.1. They determine the balance between compilation penalty, correctness penalty, speedup reward, and variance penalty, so the headline results depend on them.
smin and smax speedup clipping bounds = -0.5 and 4.0
Bounds on log2 speedup to stabilize training; chosen empirically in Section 2.2.1 and directly shape the magnitude of Rver.
Structural feature thresholds = Coalescing map stride 1->1.0, 2->0.4, 4->0.1, >4->-0.5, 0->0.2; occupancy proxy 128-256->1.0, 64-128 or 256-512->0.4…
Hand-designed mappings from source code to feature scores in Appendix A.4-A.7. These ad hoc thresholds directly determine Rstr and are not derived from data.
Dataset filtering threshold = At least 5 valid CUDA candidates per program and at least one variant with >=1.5X speedup
Section 3 defines the ranker training distribution using this filter. It may bias the ranker and policy toward programs that are easy to optimize with the Qwen-3-32B candidate generator.
RL sampling and prompt hyperparameters = 16 candidates, k=5, N=20, Dk=50, temperatures [0.2, 0.7], 60 epochs
Chosen empirically in Section 3. They affect exploration, few-shot context quality, and training cost, but are not theoretically derived.

assumptions (5)

domain assumption Static source-level features are reliable proxies for CUDA runtime performance and generalize beyond the ranker training distribution.
Appendix A.1-A.11 builds Rstr from 11 hand-crafted source features. The entire structural-reward contribution rests on this proxy being predictive across kernels, hardware, and compiler versions.
domain assumption The Qwen-3-32B-generated candidate pool with the >=1.5X filter spans the space of reasonable CUDA optimizations for the target tasks.
Section 3 describes generating at least 10 CUDA variants per program and filtering. The ranker and policy only see kernels in this stylistic distribution, so generalization to expert-written kernels is assumed without direct evidence.
domain assumption Measured speedup on a single pinned A100 GPU is the correct optimization target and is stable across runs.
Section 3's execution harness takes the median of 5 runs on one A100 GPU. The paper assumes this hardware-specific measurement is a sound reward for kernels that may be deployed elsewhere.
standard math GRPO with beta_DKL=0 and QLoRA 4-bit fine-tuning preserves the base model's code-generation competence while optimizing reward.
The policy update in Eq. 3 follows DeepSeek-R1 and the beta_DKL=0 choice follows Kevin [Baronio et al. 2025]. This is a standard but load-bearing training assumption.
ad hoc to paper BabelTower evaluation remains unbiased even though BabelTower data were used in offline ranker training.
Section 3 states the offline ranker uses BabelTower and CUDA Agent data, while Section 3.1 uses BabelTower as an evaluation benchmark. The paper never states or justifies this assumed independence.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Multi-turn RL with Structural and Performance Aware Rewards for CUDA Kernel Generation." pith.science (2026). https://pith.science/paper/PWE6XCK4

@misc{pith2026260720908,
  author       = {Pith},
  title        = {Pith review of: Multi-turn RL with Structural and Performance Aware Rewards for CUDA Kernel Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PWE6XCK4}},
  note         = {Machine review of arXiv:2607.20908}
}

read the original abstract

Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a powerful technique to enhance the reasoning capacity of LLMs for optimized code generation. However, existing RLVR approaches primarily rely on outcome-based signals such as correctness and speedup, overlooking performance-critical structural properties of programs that are essential for generating optimized code. In this work, we propose CudaPerf, a reflective RL framework that incorporates both verifiable execution rewards and structural code-aware rewards derived from parallelization features (e.g., memory coalescing, occupancy, Arithmatic Intensity, and synchronization patterns). CudaPerf operates in two stages: (1) an offline pairwise ranking module that learns to distinguish strong and weak program candidates via contrastive comparisons, and (2) an online RL training phase that jointly optimizes for correctness, performance, and structural efficiency through a unified reward signal. To further enhance learning, CudaPerf utilizes iterative refinement using execution feedback enabling progressive improvement of generated candidates. We also introduce a dataset comprising 2.9k C to CUDA and 1k PyTorch to CUDA programs, each paired with diverse input configurations and multiple CUDA implementations encompassing diverse optimization strategies. CudaPerf is evaluated across multiple benchmarks comprising both C to CUDA and PyTorch to CUDA transformations. Empirical findings suggest that CudaPerf significantly outperforms strong baselines, including Qwen-3-32B (for C to CUDA) and CUDA Agent (for PyTorch to CUDA) by achieving up to 5X & 3.32X improvements in speedup, and 17% & 7% improvements in correctness, respectively.

Figures

Figures reproduced from arXiv: 2607.20908 by the authors.

**Figure 1.** CUDAPERF workflow. The framework generates multiple CUDA candidates from sequential programs, evaluates them using execution-based verifiable rewards and structure-aware performance features, and iteratively refines the policy using reflection-driven feedback and reinforcement learning updates. details of the feature extraction techniques is described in Appendix A.1. The feature vector looks like following, ϕ(y) =… view at source ↗

**Figure 2.** Reward Progression over epochs for C-to-CUDA translation. (CUDAPERF Dataset) Dataset. For RL Training on this task we use the curated dataset of 2903 C-to-CUDA programs. We keep around 10% of the dataset (290 samples) for evaluation. In addition to that, we also evaluate on the BabelTower [Wen et al., 2022] evaluation set of 233 pairs of C and CUDA programs. Baselines. As baselines we compare with state-ofthe-art o… view at source ↗

**Figure 3.** Reward Progression over epochs for PyTorch-to-CUDA translation. (KernelBench Dataset) Dataset. We use the curated dataset which contains 1013 PyTorch-CUDA programs for RL training on this task. For evaluation we use the KernelBench Ouyang et al. [2025] dataset which contains 250 PyTorch programs consisting of mostly ML workloads across three different levels. Baselines. We choose Qwen2.5-Coder-32B [Hui et al., 20… view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Epoch vs loss curve for Pairwise Ranking MLP model [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

26 extracted references · 6 canonical work pages

[2]

Accessed: 2026-05-07. C. Baronio, P. Marsella, B. Pan, S. Guo, and S. Alberti. Kevin: Multi-turn rl for generating cuda kernels.arXiv preprint arXiv:2507.11948,

arXiv 2026
[4]

M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. D. O. Pinto, J. Kaplan, H. Edwards, Y . Burda, N. Joseph, G. Brockman, et al. Evaluating large language models trained on code.arXiv preprint arXiv:2107.03374,

arXiv
[8]

W. W. Fung, I. Sham, G. Yuan, and T. M. Aamodt. Dynamic warp formation and scheduling for efficient gpu control flow. In40th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO 2007), pages 407–420. IEEE,

work page 2007
[13]

C. Ke, R. Zhang, S. Wang, L. Ding, G. Li, Y . Wen, S. Zhang, R. Xu, J. Qin, J. Guo, et al. Qimeng-mupa: Mutual-supervised learning for sequential-to-parallel code translation.arXiv preprint arXiv:2506.11153,

work page arXiv
[14]

R. T. Lange, Q. Sun, A. Prasad, M. Faldor, Y . Tang, and D. Ha. Towards robust agentic cuda kernel benchmarking, verification, and optimization.arXiv preprint arXiv:2509.14279,

arXiv
[15]

10 X. Li, X. Sun, A. Wang, J. Li, and C. Shum. Cuda-l1: Improving cuda optimization via contrastive reinforcement learning.arXiv preprint arXiv:2507.14111,

arXiv
[16]

G. Liao, H. Qin, Y . Wang, A. Golden, M. Kuchnik, Y . Yetim, J. J. Ang, C. Fu, Y . He, S. Hsia, et al. Kernelevolve: Scaling agentic kernel coding for heterogeneous ai accelerators at meta.arXiv preprint arXiv:2512.23236,

arXiv
[17]

Nijkamp, B

E. Nijkamp, B. Pang, H. Hayashi, L. Tu, H. Wang, Y . Zhou, S. Savarese, and C. Xiong. Codegen: An open large language model for code with multi-turn program synthesis.arXiv preprint arXiv:2203.13474,

arXiv

Show all 26 references

[18]

Accessed: 2026-05-07. A. Ouyang, S. Guo, S. Arora, A. L. Zhang, W. Hu, C. Ré, and A. Mirhoseini. Kernelbench: Can llms write efficient gpu kernels?arXiv preprint arXiv:2502.10517,

2026 arXiv
[20]

URLhttps://arxiv.org/abs/2505.09388. B. Roziere, J. Gehring, F. Gloeckle, S. Sootla, I. Gat, X. E. Tan, Y . Adi, J. Liu, R. Sauvestre, T. Remez, et al. Code llama: Open foundation models for code.arXiv preprint arXiv:2308.12950,

arXiv
[21]

Shypula, A

A. Shypula, A. Madaan, Y . Zeng, U. Alon, J. Gardner, M. Hashemi, G. Neubig, P. Ranganathan, O. Bastani, and A. Yazdanbakhsh. Learning performance-improving code edits.arXiv preprint arXiv:2302.07867,

arXiv
[23]

Y . Wang, Q. Yang, Z. Zeng, L. Ren, L. Liu, B. Peng, H. Cheng, X. He, K. Wang, J. Gao, et al. Reinforcement learning for reasoning in large language models with one training example.arXiv preprint arXiv:2504.20571, 2025a. Z. Wang, K. Wang, Q. Wang, P. Zhang, L. Li, Z. Yang, X....

arXiv
[24]

Y . Zhou, A. Zanette, J. Pan, S. Levine, and A. Kumar. Archer: Training language model agents via hierarchical multi-turn rl, 2024.URL https://arxiv. org/abs/2402.19446,

2024 arXiv
[25]

Y . Zhu, D. Huang, H. Lyu, X. Zhang, C. Li, W. Shi, Y . Wu, J. Mu, J. Wang, Y . Zhao, et al. Qimeng-codev-r1: Reasoning-enhanced verilog generation.arXiv preprint arXiv:2505.24183,
[26]

These features are integrated into the reinforcement learning reward to guide the model toward efficient parallel implementations

11 A Appendix A.1 Structural Feature Extraction for Performance-Aware RL To enable performance-aware reasoning without expensive execution, we design a structural feature extractor that maps a candidate CUDA programy into a compact feature representation capturing GPU-relevant...

2026
[2007]

Gehring, K

J. Gehring, K. Zheng, J. Copet, V . Mella, Q. Carbonneaux, T. Cohen, and G. Synnaeve. Rlef: Grounding code llms in execution feedback with reinforcement learning.arXiv preprint arXiv:2410.02089,

arXiv
[2009]

B. Hui, J. Yang, Z. Cui, J. Yang, D. Liu, L. Zhang, T. Liu, J. Zhang, B. Yu, K. Dang, et al. Qwen2. 5-coder technical report.arXiv preprint arXiv:2409.12186,

arXiv
[2010]

Waghjale, V

S. Waghjale, V . Veerendranath, Z. Wang, and D. Fried. Ecco: Can we improve model-generated code efficiency without sacrificing functional correctness? InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 15362–15376,

2024
[2012]

W. Dai, H. Wu, Q. Yu, H.-a. Gao, J. Li, C. Jiang, W. Lou, Y . Song, H. Yu, J. Chen, et al. Cuda agent: Large-scale agentic rl for high-performance cuda kernel generation.arXiv preprint arXiv:2602.24286,
[2013]

R. Puri, D. S. Kung, G. Janssen, W. Zhang, G. Domeniconi, V . Zolotov, J. Dolby, J. Chen, M. Choudhury, L. Decker, et al. Codenet: A large-scale ai for code dataset for learning a diversity of coding tasks.arXiv preprint arXiv:2105.12655,

arXiv
[2014]

D. Guo, D. Yang, H. Zhang, J. Song, P. Wang, Q. Zhu, R. Xu, R. Zhang, S. Ma, X. Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948,

arXiv
[2021]

Cobbe, V

K. Cobbe, V . Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, et al. Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168,

arXiv
[2023]

J. Du, Q. Yuan, Z. Zhang, Y . Yi, J. Hu, W. Chen, Y . Zhu, Q. Zheng, W. Zou, X. Chang, et al. Akg kernel agent: A multi-agent framework for cross-platform kernel synthesis.arXiv preprint arXiv:2512.23424,
[2024]

Goldie, A

A. Goldie, A. Mirhoseini, H. Zhou, I. Cai, and C. D. Manning. Synthetic data generation & multi-step rl for reasoning & tool use.arXiv preprint arXiv:2504.04736,

arXiv
[2025]

Bhattacharjee, H

A. Bhattacharjee, H. Ping, S. V . Le, P. Bogdan, N. K. Ahmed, and A. Jannesari. Optiml: An end-to-end framework for program synthesis and cuda kernel optimization.arXiv preprint arXiv:2602.12305,
[2026]

Accessed: 2026-05-07. Atcoder. Atcoder.https://atcoder.jp/.,

2026

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [2]

Accessed: 2026-05-07. C. Baronio, P. Marsella, B. Pan, S. Guo, and S. Alberti. Kevin: Multi-turn rl for generating cuda kernels.arXiv preprint arXiv:2507.11948,

arXiv 2026

[2] [4]

M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. D. O. Pinto, J. Kaplan, H. Edwards, Y . Burda, N. Joseph, G. Brockman, et al. Evaluating large language models trained on code.arXiv preprint arXiv:2107.03374,

arXiv

[3] [8]

W. W. Fung, I. Sham, G. Yuan, and T. M. Aamodt. Dynamic warp formation and scheduling for efficient gpu control flow. In40th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO 2007), pages 407–420. IEEE,

work page 2007

[4] [13]

C. Ke, R. Zhang, S. Wang, L. Ding, G. Li, Y . Wen, S. Zhang, R. Xu, J. Qin, J. Guo, et al. Qimeng-mupa: Mutual-supervised learning for sequential-to-parallel code translation.arXiv preprint arXiv:2506.11153,

work page arXiv

[5] [14]

R. T. Lange, Q. Sun, A. Prasad, M. Faldor, Y . Tang, and D. Ha. Towards robust agentic cuda kernel benchmarking, verification, and optimization.arXiv preprint arXiv:2509.14279,

arXiv

[6] [15]

10 X. Li, X. Sun, A. Wang, J. Li, and C. Shum. Cuda-l1: Improving cuda optimization via contrastive reinforcement learning.arXiv preprint arXiv:2507.14111,

arXiv

[7] [16]

G. Liao, H. Qin, Y . Wang, A. Golden, M. Kuchnik, Y . Yetim, J. J. Ang, C. Fu, Y . He, S. Hsia, et al. Kernelevolve: Scaling agentic kernel coding for heterogeneous ai accelerators at meta.arXiv preprint arXiv:2512.23236,

arXiv

[8] [17]

Nijkamp, B

E. Nijkamp, B. Pang, H. Hayashi, L. Tu, H. Wang, Y . Zhou, S. Savarese, and C. Xiong. Codegen: An open large language model for code with multi-turn program synthesis.arXiv preprint arXiv:2203.13474,

arXiv