REVIEW 14 cited by
CUDA-L1: Improving CUDA Optimization via Contrastive Reinforcement Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
CUDA-L1: Improving CUDA Optimization via Contrastive Reinforcement Learning
read the original abstract
The exponential growth in demand for GPU computing resources has created an urgent need for automated CUDA optimization strategies. While recent advances in LLMs show promise for code generation, current SOTA models achieve low success rates in improving CUDA speed. In this paper, we introduce CUDA-L1, an automated reinforcement learning framework for CUDA optimization that employs a novel contrastive RL algorithm. CUDA-L1 achieves significant performance improvements on the CUDA optimization task: trained on A100, it delivers an average speedup of x3.12 with a median speedup of x1.42 against default baselines over across all 250 CUDA kernels of KernelBench, with peak speedups reaching x120. In addition to the default baseline provided by KernelBench, CUDA-L1 demonstrates x2.77 over Torch Compile, x2.88 over Torch Compile with reduce overhead, x2.81 over CUDA Graph implementations, and x7.72 over cuDNN libraries. Furthermore, the model also demonstrates portability across different GPU architectures. Beyond these benchmark results, CUDA-L1 demonstrates several properties: it 1) discovers a variety of CUDA optimization techniques and learns to combine them strategically to achieve optimal performance; 2) uncovers fundamental principles of CUDA optimization, such as the multiplicative nature of optimizations; 3) identifies non-obvious performance bottlenecks and rejects seemingly beneficial optimizations that actually harm performance. The capabilities demonstrate that, RL can transform an initially poor-performing LLM into an effective CUDA optimizer through speedup-based reward signals alone, without human expertise or domain knowledge. This paradigm opens possibilities for automated optimization of CUDA operations, and holds promise to substantially promote GPU efficiency and alleviate the rising pressure on GPU computing resources. Project: deepreinforce-ai.github.io/cudal1_blog
Forward citations
Cited by 14 Pith papers
-
FastKernels: Benchmarking GPU Kernel Generation in Production
FastKernels is a production-aligned benchmark covering 96.2% of HuggingFace Transformers that reveals state-of-the-art kernel agents deliver at most 0.94x aggregate speedup.
-
PassNet: Scaling Large Language Models for Graph Compiler Pass Generation
PassNet provides a dataset of 18K graphs and PassBench for LLM-generated compiler passes, with fine-tuned models achieving 2.67x gains on long-tail tasks where TorchInductor underperforms.
-
Step-TP: A Grounded, Step-Level Dataset with Chain-of-Thought Reasoning for LLM-Guided Tensor Program Optimization
Step-TP is a dataset providing grounded, atomic step-level IR transitions and CoT supervision to enable reliable multi-step LLM-guided tensor program optimization instead of end-to-end imitation.
-
RLPF: Reinforcement Learning from Performance Feedback for Code Generation
RLPF's staged performance reward lifts Qwen3-32B on PerfCodeBench from 11.1% to 54.6% correct-and-runnable and from 8.1% to 38.6% relative efficiency.
-
Hawk: Harnessing Hardware-Aware Knowledge for High-Performance NPU Kernel Generation
Hawk raises NPU kernel generation accuracy from 49.4% to 80% and yields up to 2.2× speedups by retrieving and distilling structured hardware-aware knowledge without any model training.
-
Hawk: Harnessing Hardware-Aware Knowledge for High-Performance NPU Kernel Generation
Hawk is a training-free framework that boosts NPU kernel generation accuracy to 80% and achieves up to 2.2x speedup via hardware-aware knowledge synthesis, 2D retrieval, and effect-driven distillation.
-
SpecGen: Accelerating Agentic Kernel Optimization with Speculative Generation
SpecGen introduces speculative generation to fork non-reasoning kernel candidates during LLM reasoning traces, enabling early termination and parallel profiling to reduce end-to-end optimization time on H200 GPUs.
-
Learning When to Optimize: Verified Optimization Skills from Expert GPU-Kernel Lineages
KLineage derives verified optimization skills from backward lineages of expert GPU kernels to guide LLM agents toward higher-quality and more efficient kernels than memory-based baselines.
-
Kernel Foundry: A Diagnosis-driven Evolutionary Kernel Optimizer with Multi-Experts
Kernel Foundry improves GPU kernel correctness and performance on KernelBench via expert-guided evolutionary search with diagnostic feedback and an experience library, reaching 100% correctness on Level 2.
-
AdaExplore: Failure-Driven Adaptation and Diversity-Preserving Search for Efficient Kernel Generation
AdaExplore improves correctness and speed of Triton kernel generation by converting recurring failures into a memory of rules and organizing search as a tree that mixes local refinements with larger regenerations, yie...
-
GrandCode: Achieving Grandmaster Level in Competitive Programming via Agentic Reinforcement Learning
GrandCode is the first AI system to consistently beat all human participants and place first in live Codeforces competitive programming contests.
-
Kernel-Smith: A Unified Recipe for Evolutionary Kernel Optimization
Kernel-Smith combines evolutionary search with RL post-training to generate optimized GPU kernels, achieving SOTA speedups on KernelBench that beat Gemini-3.0-pro and Claude-4.6-opus on NVIDIA Triton and generalize to...
-
Compiler-Grounded Hierarchical Diagnosis for LLM-Based Triton Kernel Optimization
A hierarchical, compiler-grounded LLM optimization loop reports 4.35x geometric-mean Triton kernel speedups on 37 Ascend NPU benchmark kernels, but with no causal ablation yet.
-
AscendOptimizer: Episodic Agent for Ascend NPU Operator Optimization
AscendOptimizer combines kernel rewinding for reusable experience with evolutionary search on hardware feedback to optimize Ascend NPU operators, delivering 1.21x geometric-mean speedup and faster performance on 53.47...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.