Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

A two-stage reinforcement scheme turns diffusion LLMs into state-of-the-art CUDA kernel generators, matching a specialist model on a fraction of the training budget.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 23:59 UTC pith:PFWGUHJP

load-bearing objection A genuine contribution to dLLM-based kernel generation with a sensible RL curriculum, but the robust-check protocol is underspecified and the SOTA claim overreaches. the 3 major comments →

arxiv 2602.11715 v2 pith:PFWGUHJP submitted 2026-02-12 cs.LG cs.CL

DICE: Diffusion Large Language Models Excel at Generating CUDA Kernels

Haolei Bai , Lingcheng Kong , Xueyi Chen , Jianmian Wang , Zhiqiang Tao , Huan Wang This is my paper
classification cs.LG cs.CL
keywords CUDA kernelsdiffusion language modelsreinforcement learningkernel infillingsupervised fine-tuningdata efficiencyGPU kernel generationdeceptive outputs
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that diffusion large language models, which generate text by denoising masked blocks rather than predicting left-to-right, are well suited to CUDA kernel generation when trained in the right order. It builds a curated dataset of high-performance kernels and trains three model sizes with a two-phase reinforcement curriculum: first infill the core logic of a kernel inside a fixed scaffold, then generate the whole kernel end-to-end. On a standard 250-task kernel benchmark, the resulting models outperform comparable autoregressive and diffusion baselines, and the 8B model matches a CUDA-specialized autoregressive model that used roughly fourteen times more reinforcement samples. If the claim holds, kernel generation is less a matter of model scale than of structured data and a progressive training curriculum.

Core claim

The paper's central claim is that blockwise diffusion decoding is not a handicap for CUDA kernel generation but can be leveraged into an advantage when the model is trained in a targeted way. The authors show that masking out only the inner CUDA logic of a kernel, leaving the import/glue prefix and the module-wrapper suffix fixed, and training the model to infill that core before attempting full generation, yields a family of models with Level 1/2/3 execution correctness of 24/6/10 at 1.7B, 27/24/12 at 4B, and 40/39/16 at 8B. The 8B model thereby roughly matches a CUDA-specialized autoregressive model that used many more training samples and a much larger generation budget. The paper also re

What carries the argument

BiC-RL, a bi-phase curated reinforcement learning framework built on a block-diffusion base model. Phase one is CUDA kernel infilling: a kernel is decomposed into prefix, core, and suffix, with only the core generated by the model, making it structurally impossible to bypass the kernel while still passing a compile-based check. Phase two is end-to-end kernel generation from a PyTorch reference. The accompanying CuKe dataset is filtered to entries with at least a 2.0x measured speedup over PyTorch and augmented with whole-model structures; rewards are granted only when generated kernels compile and run correctly. The infilling scaffold is the mechanism that suppresses deceptive behavior and y

Load-bearing premise

The comparison hinges on an unspecified 'robust check' that removes deceptive outputs; if that filter is applied inconsistently or too permissively, the reported ranking over baselines, especially the near-total collapse of a masked-diffusion baseline, could be wrong.

What would settle it

Re-run the published evaluation with a fully specified, deterministic robust-check script that requires the generated custom kernel to be invoked in the forward pass, then inspect every case where DICE-8B passes Levels 2/3 after filtering; if many of those pass only under looser criteria, or if the masked-diffusion baseline regains most of its Level 2/3 scores under identical criteria, the state-of-the-art claim is unsupported.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Kernel-generation reinforcement learning can be made substantially more data-efficient: roughly five thousand RL samples suffice to match a specialist trained on tens of thousands.
  • At the 1.7B scale, diffusion models trained this way produce correct kernels where comparable autoregressive and general-purpose diffusion models produce none, pointing to training paradigm rather than parameter count.
  • Deceptive outputs are widespread in current kernel evaluations; excluding them can change a masked-diffusion baseline's Level 3 score from 44% to 0%, so benchmark rankings need a standard, transparent filter.
  • The infill-first curriculum converges more stably than one-shot reinforcement learning, with higher initial correctness in the later generation stage.
  • Filtering supervised data to verified high-speedup kernels (6,303 samples) outperforms a larger, less selective set (8,920 samples) on the same base model, suggesting a data-quality-over-volume law.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The infilling scaffold could be standardized and reused as a public kernel template, letting any code model be trained and evaluated without deceptive shortcuts, and potentially transferring to other accelerator dialects like Triton.
  • Because the robust check is the gate that changes rankings, releasing its exact criteria would allow an independent audit and would likely make baseline scores more conservative.
  • The same bi-phase curriculum — infill the core, then generate the whole — applies to other structured artifacts with stable skeletons, such as shell scripts, test suites, or configuration files, where skipping hard content while passing superficial checks is common.
  • To isolate the architectural contribution, a further test would hold the RL budget and prompt format fixed and compare this diffusion model with an autoregressive model under the same infill-first curriculum; the current evidence mixes architecture and curriculum effects.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces DICE, a family of diffusion large language models (1.7B, 4B, 8B) specialized for CUDA kernel generation, together with a curated SFT dataset (CuKe) and a two-stage RL framework (BiC-RL: kernel infilling followed by end-to-end kernel generation). The models are initialized from SDAR and trained using compilation-and-correctness rewards. Evaluation on KernelBench Levels 1–3 reports execution correctness and speedup metrics against a broad set of autoregressive and diffusion LLMs, including the CUDA-specialized cudaLLM-8B. The paper claims state-of-the-art performance and improved data efficiency, and ablation tables attribute gains to each training stage.

Significance. If the reported results hold, the contribution is significant: it is, to my knowledge, the first specialized diffusion LLM for CUDA kernel generation, and the bi-phase RL curriculum plus curated dataset are plausible and useful design choices. The paper has real strengths: evaluation is against the external KernelBench benchmark, the reward is compilation-and-correctness rather than the benchmark metric itself, and the ablations in Tables 4–5 support the incremental contribution of SFT, kernel infilling, and the CuKe dataset. The data-efficiency comparison with cudaLLM in Table 7 is also informative. However, the headline numbers depend on a 'robust check' whose protocol is never specified, and on an internal arithmetic inconsistency in the CuKe construction. These issues block independent verification and must be resolved before the claims can be accepted.

major comments (3)
  1. [§4.1.2, §5.2, Table 6] The robust-check protocol is unspecified but load-bearing. Section 4.1.2 names three deceptive-behavior categories and Appendix D gives examples, but the actual algorithm, deterministic criteria, prompt, or annotation protocol are not given. Tables 1–3 appear to report robust-checked numbers, since DICE-8B in Table 6 is 40/39/16 with the check and Table 1 lists the same values; DiffuCoder similarly matches the checked row. The check changes Level 3 correctness by 44 points for DiffuCoder and 18 points for DICE, so the ranking and the cudaLLM comparison are sensitive to it. The paper must specify the check precisely, state whether it was applied uniformly to every model in Tables 1–3 (including commercial models), and release the script/annotations to permit reproduction.
  2. [§3] The CuKe dataset construction is arithmetically inconsistent. The text states that the 2.0× speedup filter yields 1,425 pairs, and the structural-diversity path yields 36 verified instances. The next paragraph says 'Combining the samples from these two diverse dimensions results in a final, high-quality training set of 6,303 samples.' 1,425 + 36 = 1,461, not 6,303. No explanation is given for where the remaining ~4,842 samples come from, despite Table 5 listing ConCuR as 4,842 samples and CuKe as 6,303. This discrepancy must be clarified because the data-efficiency argument depends on the exact composition and provenance of CuKe.
  3. [Abstract and §5.2 / Appendix C] The abstract's claim that 'DICE significantly outperforms both autoregressive and diffusion LLMs of comparable scale' is not supported by the paper's own data. Table 7 shows cudaLLM-8B at 46/34/6 versus DICE-8B at 40/39/16: cudaLLM is better on Level 1, DICE is better on Levels 2 and 3. Section 5.2 itself describes the comparison as 'comparable results with cudaLLM.' The phrasing should be corrected to match the evidence, e.g., 'matches or improves on specialized AR baselines at a fraction of the training budget,' unless a formal aggregate significance test is provided.
minor comments (5)
  1. [Table 7] The RL column shows '71.996' — presumably a typo for '71,996'. Please fix.
  2. [Figure 4 caption / §5.2] The text says 'with the comparative results illustrated in Figure 4. In this table, correctness denotes...' — 'In this table' should be 'In this figure.'
  3. [Tables 1–3] The model name 'Trado-Instruct' appears in Tables 1–2 while the related work and references use 'TraDo.' Please standardize the notation.
  4. [Impact Statement] The Impact Statement says 'by releasing our whole pipeline,' but no code, data, or model weights appear to be linked beyond the project page. If the release is planned, give a URL or anonymous repository; otherwise revise the statement. This is also relevant to the reproducibility concerns in the major comments.
  5. [§5.1 / Appendix B.3] The evaluation details give specific settings for many dLLMs but do not report hyperparameters such as temperature or block size for commercial models (Gemini-3-Pro, Claude-Sonnet-4, Mercury-Coder-Small). Please clarify whether these were used with default settings and, if so, state that explicitly.

Circularity Check

0 steps flagged

No load-bearing circularity; central evaluation is on the external KernelBench benchmark; a minor same-group dataset citation (ConCuR) is used as a building block, and the unspecified robust-check protocol is a reproducibility concern rather than a circularity.

full rationale

The paper's central claim—that DICE outperforms comparable AR and diffusion LLMs on CUDA kernel generation—is tested on the external KernelBench benchmark, with functional correctness determined by compiling and executing generated kernels against PyTorch references. The RL reward is compile-and-correctness, which is aligned with the benchmark metric but is a standard training objective, not a fitted parameter renamed as a prediction. The CuKe dataset is constructed on top of ConCuR (Kong et al., 2025), a same-group prior dataset, and the method builds on TraceRL and SDAR; these self-citations/prior works are used as building blocks, not as evidence for the headline SOTA claim. The only significant caveat is that Tables 1–3 appear to report robust-checked numbers (Table 6 shows DICE-8B as 40/39/16 under robust check vs 40/43/34 without, and DiffuCoder as 1/0/0 vs 12/46/44), while the robust-check algorithm is never specified. This makes the evaluation difficult to reproduce and could affect the cudaLLM comparison, but it is an evaluation-protocol/verifiability issue, not a derivation that reduces to its own inputs by construction. Therefore, the circularity score is low: 2, reflecting only a minor, non-load-bearing self-citation (ConCuR).

Axiom & Free-Parameter Ledger

2 free parameters · 3 axioms · 0 invented entities

The paper does not derive any mathematical quantity or introduce new physical entities. The main hand-chosen parameters are the data-filtering thresholds in CuKe; the evaluation relies on the external KernelBench benchmark and the authors' robust-check procedure as an assumed oracle. The SDAR backbone and TraceRL are external, though partly self-cited.

free parameters (2)
  • Speedup threshold for CuKe filtering = 2.0x
    Hand-chosen threshold to decide which ConCuR samples count as 'high-performance'. The paper reports that only 86% of cudaLLM2 examples had speedup >1x and calls near-1x speedups unreliable, so 2.0x is chosen as a robust filter, but no sensitivity analysis is given.
  • Structural sample retention criterion = confirmed speedup after up to 5 runs (36 of 291 retained)
    Mercury Coder is used to generate 291 structural samples; only 36 that demonstrated 'confirmed speedup' are kept. The retention rule is environment-dependent and not specified beyond 'confirmed speedup'.
axioms (3)
  • domain assumption KernelBench's raw functional correctness metric is unreliable and the robust check fixes it.
    The paper argues that standard evaluation scripts erroneously count 'deceptive' outputs as correct (Section 4.1.2), so the robust-checked numbers are presented as the truth. This assumes the robust check itself is a valid and consistent oracle, but its implementation is not described.
  • ad hoc to paper The prefix/suffix scaffold fully captures the software stack around a CUDA kernel.
    The infilling stage trains the model only on the core C++ body given a fixed prefix and suffix (Figure 3). This assumes that all CUDA kernels can be decomposed in this way and that mastery transfers to end-to-end generation, which may not hold for kernels with unusual integration requirements.
  • domain assumption SDAR is an appropriate backbone for kernel generation.
    All DICE models are built on SDAR (Cheng et al., 2025). The paper does not compare against other dLLM backbones with the same CuKe + BiC-RL pipeline, so the reported gains could be partly attributed to SDAR rather than to the new framework.

pith-pipeline@v1.3.0-alltime-deepseek · 19219 in / 10019 out tokens · 94140 ms · 2026-08-02T23:59:20.638536+00:00 · methodology

0 comments
read the original abstract

Diffusion large language models (dLLMs) have emerged as a compelling alternative to autoregressive (AR) LLMs, owing to their capacity for parallel token generation. This paradigm is particularly well-suited for code generation, where holistic structural planning and non-sequential refinement are critical. Despite this potential, tailoring dLLMs for CUDA kernel generation remains challenging, obstructed not only by the high specialization but also by the severe lack of high-quality training data. To address these challenges, we construct CuKe, an augmented supervised fine-tuning dataset optimized for high-performance CUDA kernels. On top of it, we propose a bi-phase curated reinforcement learning (BiC-RL) framework consisting of a CUDA kernel infilling stage and an end-to-end CUDA kernel generation stage. Leveraging this training framework, we introduce DICE, a series of diffusion large language models designed for CUDA kernel generation, spanning three parameter scales, 1.7B, 4B, and 8B. Extensive experiments on KernelBench demonstrate that DICE significantly outperforms both autoregressive and diffusion LLMs of comparable scale, establishing a new state-of-the-art for CUDA kernel generation.

Figures

Figures reproduced from arXiv: 2602.11715 by Haolei Bai, Huan Wang, Jianmian Wang, Lingcheng Kong, Xueyi Chen, Zhiqiang Tao.

Figure 1
Figure 1. Figure 1: Overview of DICE. The framework enhances CUDA kernel generation robustness in dLLMs by leveraging TraceRL. This hierarchical approach integrates: (1) Bi-phase Curated Reinforcement Learning framework, a progressive RL training strategy that consists of kernel infilling and end-to-end kernel generation stages to ensure functional correctness and high performance of generated CUDA kernels, and (2) Data Sched… view at source ↗
Figure 2
Figure 2. Figure 2: The inference paradigm of diffusion large language models. Left Part: The sequence is divided into several blocks, where the block length equals four in this figure. The block diffusion mechanism enables models to generate autoregressively between blocks, while parallel discrete decoding within blocks. All the KV cache from previous blocks will be reused. Right Part: An actual step-by-step generation traje… view at source ↗
Figure 3
Figure 3. Figure 3: Our defined CUDA kernel components: the prefix, the suffix, and the core implementation, which is a C++ snippet. 4.1.2. BIC-RL To further extend the capabilities of DICE in CUDA kernel generation and mitigate the deceptive behavior, we propose a bi-phase curated reinforcement learning framework built upon TraceRL (Wang et al., 2025). This framework operates in two dimensions. In the data dimension, we leve… view at source ↗
Figure 4
Figure 4. Figure 4: RL training trajectory comparison of BiC-RL framework and baseline RL on 8B model. 5.2. Experimental Results Main Results on KernelBench. Tables 1, 2, and 3 sum￾marize the performance of our DICE models across the 8B, 4B, and 1.7B scales. These results validate the effectiveness of our method, as our models consistently demonstrate the state-of-the-art performance against both autoregressive and diffusion … view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Adaptive Causal Alignment for High-Confidence Adversarial Training

    cs.CV 2026-06 unverdicted novelty 6.0

    HICAT applies a Measure-Debias-Align pipeline with a Learnable Background-Bias Estimator and Foreground Logit Orthogonal Enhancement loss to reduce spurious context reliance in high-confidence adversarial training on ...

Reference graph

Works this paper leans on

34 extracted references · 18 linked inside Pith · cited by 1 Pith paper

  1. [1]

    L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al

    Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774,

    Pith/arXiv arXiv

  2. [4]

    Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities.arXiv preprint arXiv:2507.06261,

    Comanici, G., Bieber, E., Schaekermann, M., Pasupat, I., Sachdeva, N., Dhillon, I., Blistein, M., Ram, O., Zhang, D., Rosen, E., et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities.arXiv preprint arXiv:2507.06261,

    Pith/arXiv arXiv

  3. [5]

    Fan, C., Heng, W., Li, B., Liu, S., Song, Y ., Su, J., Qu, X., Shen, K., and Wei, W

    URL https://blog.google/technology/ google-deepmind/gemini-diffusion. Fan, C., Heng, W., Li, B., Liu, S., Song, Y ., Su, J., Qu, X., Shen, K., and Wei, W. Stable-diffcoder: Pushing the frontier of code diffusion large language model.arXiv preprint arXiv:2601.15892,

    arXiv

  4. [7]

    Gong, S., Zhang, R., Zheng, H., Gu, J., Jaitly, N., Kong, L., and Zhang, Y

    URLhttps: //huggingface.co/facebook/KernelLLM. Gong, S., Zhang, R., Zheng, H., Gu, J., Jaitly, N., Kong, L., and Zhang, Y . Diffucoder: Understanding and improving masked diffusion models for code generation.arXiv preprint arXiv:2506.20639,

    Pith/arXiv arXiv

  5. [8]

    The llama 3 herd of models.arXiv preprint arXiv:2407.21783,

    Grattafiori, A., Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Vaughan, A., et al. The llama 3 herd of models.arXiv preprint arXiv:2407.21783,

    Pith/arXiv arXiv

  6. [10]

    Deepseek-r1: In- centivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948,

    Guo, D., Yang, D., Zhang, H., Song, J., Zhang, R., Xu, R., Zhu, Q., Ma, S., Wang, P., Bi, X., et al. Deepseek-r1: In- centivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948,

    Pith/arXiv arXiv

  7. [11]

    Hui, B., Yang, J., Cui, Z., Yang, J., Liu, D., Zhang, L., Liu, T., Zhang, J., Yu, B., Lu, K., et al. Qwen2. 5-coder technical report.arXiv preprint arXiv:2409.12186,

    Pith/arXiv arXiv

  8. [12]

    Concur: Con- ciseness makes state-of-the-art kernel generation.arXiv preprint arXiv:2510.07356,

    Kong, L., Wei, J., Shen, H., and Wang, H. Concur: Con- ciseness makes state-of-the-art kernel generation.arXiv preprint arXiv:2510.07356,

    arXiv

  9. [13]

    Mercury: Ultra-fast language models based on diffusion

    Labs, I., Khanna, S., Kharbanda, S., Li, S., Varma, H., Wang, E., Birnbaum, S., Luo, Z., Miraoui, Y ., Palrecha, A., et al. Mercury: Ultra-fast language models based on diffusion. arXiv preprint arXiv:2506.17298,

    Pith/arXiv arXiv

  10. [14]

    Beyond autoregression: An empirical study of diffusion large language models for code generation.arXiv preprint arXiv:2509.11252, 2025a

    Li, C., Zhang, Y ., Li, J., Cai, L., and Li, G. Beyond autoregression: An empirical study of diffusion large language models for code generation.arXiv preprint arXiv:2509.11252, 2025a. Li, J., Li, S., Gao, Z., Shi, Q., Li, Y ., Wang, Z., Huang, J., WangHaojie, W., Wang, J., Han, X., et al. Triton- bench: Benchmarking large language model capabilities for ...

    arXiv

  11. [15]

    Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437,

    Liu, A., Feng, B., Xue, B., Wang, B., Wu, B., Lu, C., Zhao, C., Deng, C., Zhang, C., Ruan, C., et al. Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437,

    Pith/arXiv arXiv

  12. [16]

    Wedlm: Reconciling diffusion language models with standard causal atten- tion for fast inference.arXiv preprint arXiv:2512.22737,

    Liu, A., He, M., Zeng, S., Zhang, S., Zhang, L., Wu, C., Jia, W., Liu, Y ., Zhou, X., and Zhou, J. Wedlm: Reconciling diffusion language models with standard causal atten- tion for fast inference.arXiv preprint arXiv:2512.22737,

    arXiv

  13. [17]

    Ni, J., Liu, Q., Dou, L., Du, C., Wang, Z., Yan, H., Pang, T., and Shieh, M. Q. Diffusion language models are super data learners.arXiv preprint arXiv:2511.03276, 2025a. Ni, J., Liu, Q., Du, C., Dou, L., Yan, H., Wang, Z., Pang, T., and Shieh, M. Q. Training optimal large diffusion lan- guage models.arXiv preprint arXiv:2510.03280, 2025b. Nie, S., Zhu, F....

    arXiv

  14. [18]

    Prabhudesai, M., Wu, M., Zadeh, A., Fragkiadaki, K., and Pathak, D

    URL https://github.com/pengzhangzhi/ Open-dLLM. Prabhudesai, M., Wu, M., Zadeh, A., Fragkiadaki, K., and Pathak, D. Diffusion beats autoregressive in data- constrained settings.arXiv preprint arXiv:2507.15857,

    arXiv

  15. [19]

    Seed-coder: Let the code model curate data for itself.arXiv preprint arXiv:2506.03524,

    Seed, B., Zhang, Y ., Su, J., Sun, Y ., Xi, C., Xiao, X., Zheng, S., Zhang, A., Liu, K., Zan, D., et al. Seed-coder: Let the code model curate data for itself.arXiv preprint arXiv:2506.03524,

    Pith/arXiv arXiv

  16. [20]

    Seed diffusion: A large-scale diffusion language model with high-speed inference.arXiv preprint arXiv:2508.02193,

    Song, Y ., Zhang, Z., Luo, C., Gao, P., Xia, F., Luo, H., Li, Z., Yang, Y ., Yu, H., Qu, X., et al. Seed diffusion: A large-scale diffusion language model with high-speed inference.arXiv preprint arXiv:2508.02193,

    Pith/arXiv arXiv

  17. [21]

    G., Hardin, C., Bhupatiraju, S., Hussenot, L., Mesnard, T., Shahri- ari, B., Ram ´e, A., et al

    Team, G., Riviere, M., Pathak, S., Sessa, P. G., Hardin, C., Bhupatiraju, S., Hussenot, L., Mesnard, T., Shahri- ari, B., Ram ´e, A., et al. Gemma 2: Improving open language models at a practical size.arXiv preprint arXiv:2408.00118,

    Pith/arXiv arXiv

  18. [22]

    Revolutionizing reinforcement learning framework for diffusion large language models.arXiv preprint arXiv:2509.06949,

    Wang, Y ., Yang, L., Li, B., Tian, Y ., Shen, K., and Wang, M. Revolutionizing reinforcement learning framework for diffusion large language models.arXiv preprint arXiv:2509.06949,

    Pith/arXiv arXiv

  19. [23]

    Astra: A multi- agent system for gpu kernel performance optimization

    Wei, A., Sun, T., Seenichamy, Y ., Song, H., Ouyang, A., Mirhoseini, A., Wang, K., and Aiken, A. Astra: A multi- agent system for gpu kernel performance optimization. InNeurIPS 2025 Fourth Workshop on Deep Learning for Code,

    2025

  20. [24]

    Tritonrl: Training llms to think and code triton without cheating.arXiv preprint arXiv:2510.17891,

    Woo, J., Zhu, S., Nie, A., Jia, Z., Wang, Y ., and Park, Y . Tritonrl: Training llms to think and code triton without cheating.arXiv preprint arXiv:2510.17891,

    arXiv

  21. [25]

    Teaching language models to critique via reinforcement learning

    Xie, Z., Chen, L., Mao, W., Xu, J., Kong, L., et al. Teaching language models to critique via reinforcement learning. arXiv preprint arXiv:2502.03492, 2025a. Xie, Z., Ye, J., Zheng, L., Gao, J., Dong, J., Wu, Z., Zhao, X., Gong, S., Jiang, X., Li, Z., et al. Dream-coder 7b: An open diffusion language model for code.arXiv preprint arXiv:2509.01142, 2025b. ...

    arXiv

  22. [26]

    J., Li, Y ., Ren, L., Shen, Y ., et al

    Xu, H., Peng, B., Awadalla, H., Chen, D., Chen, Y .-C., Gao, M., Kim, Y . J., Li, Y ., Ren, L., Shen, Y ., et al. Phi-4-mini- reasoning: Exploring the limits of small reasoning lan- guage models in math.arXiv preprint arXiv:2504.21233,

    Pith/arXiv arXiv

  23. [27]

    Qwen3 technical report.arXiv preprint arXiv:2505.09388,

    Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., Lv, C., et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388,

    Pith/arXiv arXiv

  24. [28]

    Dream 7b: Diffusion large language models.arXiv preprint arXiv:2508.15487,

    Ye, J., Xie, Z., Zheng, L., Gao, J., Wu, Z., Jiang, X., Li, Z., and Kong, L. Dream 7b: Diffusion large language models.arXiv preprint arXiv:2508.15487,

    Pith/arXiv arXiv

  25. [29]

    Cudaforge: An agent framework with hardware feedback for cuda kernel optimization.arXiv preprint arXiv:2511.01884,

    Zhang, Z., Wang, R., Li, S., Luo, Y ., Hong, M., and Ding, C. Cudaforge: An agent framework with hardware feedback for cuda kernel optimization.arXiv preprint arXiv:2511.01884,

    arXiv

  26. [30]

    Llada 1.5: Variance- reduced preference optimization for large language diffu- sion models.arXiv preprint arXiv:2505.19223,

    10 DICE: Diffusion Large Language Models Excel at Generating CUDA Kernels Zhu, F., Wang, R., Nie, S., Zhang, X., Wu, C., Hu, J., Zhou, J., Chen, J., Lin, Y ., Wen, J.-R., et al. Llada 1.5: Variance- reduced preference optimization for large language diffu- sion models.arXiv preprint arXiv:2505.19223,

    Pith/arXiv arXiv

  27. [31]

    One-shot Prompt Templates The one-shot prompt for the kernel infilling RL stage is illustrated as follows

    11 DICE: Diffusion Large Language Models Excel at Generating CUDA Kernels A. One-shot Prompt Templates The one-shot prompt for the kernel infilling RL stage is illustrated as follows. We incorporate a comprehensive contextual demonstration to guide the model in generating the core C++ code while maintaining logic alignment with the provided prefix and suf...

    2025

  28. [32]

    As mentioned in Section 5.1, we conduct supervised fine-tuning on our curated CuKe dataset for three epochs using 8 NVIDIA A100 GPUS, with a learning rate of1.0×10 −5

    as our base models, which contain models across three parameter scales (1.7B, 4B, and 8B). As mentioned in Section 5.1, we conduct supervised fine-tuning on our curated CuKe dataset for three epochs using 8 NVIDIA A100 GPUS, with a learning rate of1.0×10 −5. B.2. RL Sampling Details During the reinforcement learning stage, we employ a block size of 4 and ...

    2025

  29. [33]

    The overall steps are equal to the generation sequence length

    and LLaDA-1.5 (Zhu et al., 2025), we set a maximum of 1,024 generated tokens, with a block size of 32 and a temperature of 0.0. The overall steps are equal to the generation sequence length. • For Dream-v0-Instruct (Ye et al.,

    2025

  30. [34]

    The overall steps are equal to the generation sequence length

    and Dream-Coder-v0-Instruct (Xie et al., 2025b), we set a maximum of 1,024 generated tokens, with temperatures of 0.2 and 0.1. The overall steps are equal to the generation sequence length. • For DiffuCoder-Instruct (Gong et al., 2025), we set a maximum of 1,024 generated tokens, with a temperature of 0.3. The overall steps are equal to the generation seq...

    2025

  31. [2023]

    T., Yang, Z., Qi, Z., Han, J., Sahoo, S

    Arriola, M., Gokaslan, A., Chiu, J. T., Yang, Z., Qi, Z., Han, J., Sahoo, S. S., and Kuleshov, V . Block diffusion: Inter- polating between autoregressive and diffusion language models.arXiv preprint arXiv:2503.09573,

    Pith/arXiv arXiv

  32. [2024]

    Deepseek-coder: When the large language model meets programming–the rise of code intelligence.arXiv preprint arXiv:2401.14196,

    Guo, D., Zhu, Q., Yang, D., Xie, Z., Dong, K., Zhang, W., Chen, G., Bi, X., Wu, Y ., Li, Y ., et al. Deepseek-coder: When the large language model meets programming–the rise of code intelligence.arXiv preprint arXiv:2401.14196,

    Pith/arXiv arXiv

  33. [2025]

    Sdar: A syn- ergistic diffusion-autoregression paradigm for scalable sequence generation.arXiv preprint arXiv:2510.06303,

    Cheng, S., Bian, Y ., Liu, D., Zhang, L., Yao, Q., Tian, Z., Wang, W., Guo, Q., Chen, K., Qi, B., et al. Sdar: A syn- ergistic diffusion-autoregression paradigm for scalable sequence generation.arXiv preprint arXiv:2510.06303,

    arXiv

  34. [2026]

    Unifying autore- gressive and diffusion-based sequence generation.arXiv preprint arXiv:2504.06416,

    Fathi, N., Scholak, T., and No ¨el, P.-A. Unifying autore- gressive and diffusion-based sequence generation.arXiv preprint arXiv:2504.06416,

    arXiv