REVIEW 8 cited by
CUDA-LLM: LLMs Can Write Efficient CUDA Kernels
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
CUDA-LLM: LLMs Can Write Efficient CUDA Kernels
read the original abstract
Large Language Models (LLMs) have demonstrated strong capabilities in general-purpose code generation. However, generating the code which is deeply hardware-specific, architecture-aware, and performance-critical, especially for massively parallel GPUs, remains a complex challenge. In this work, we explore the use of LLMs for the automated generation and optimization of CUDA programs, with the goal of producing high-performance GPU kernels that fully exploit the underlying hardware. To address this challenge, we propose a novel framework called \textbf{Feature Search and Reinforcement (FSR)}. FSR jointly optimizes compilation and functional correctness, as well as the runtime performance, which are validated through extensive and diverse test cases, and measured by actual kernel execution latency on the target GPU, respectively. This approach enables LLMs not only to generate syntactically and semantically correct CUDA code but also to iteratively refine it for efficiency, tailored to the characteristics of the GPU architecture. We evaluate FSR on representative CUDA kernels, covering AI workloads and computational intensive algorithms. Our results show that LLMs augmented with FSR consistently guarantee correctness rates. Meanwhile, the automatically generated kernels can outperform general human-written code by a factor of up to 179$\times$ in execution speeds. These findings highlight the potential of combining LLMs with performance reinforcement to automate GPU programming for hardware-specific, architecture-sensitive, and performance-critical applications.
Forward citations
Cited by 8 Pith papers
-
From Custom-Fit to Portable: Bridging the Gap Between Synthesized and Engineered GPU Query Execution
LLM-synthesized GPU query kernels outperform engineered engines by 7.4x, but a portable SYCL engine with lifted optimizations closes the gap to 1.27x, suggesting engineering remains preferable on GPUs.
-
CODA: Rewriting Transformer Blocks as GEMM-Epilogue Programs
CODA re-expresses most non-attention Transformer computations as GEMM-plus-epilogue programs using a constrained set of composable primitives to keep intermediate results on-chip and cut global memory traffic.
-
CODA: Rewriting Transformer Blocks as GEMM-Epilogue Programs
CODA re-expresses non-attention Transformer computations as GEMM epilogues using composable primitives for scaling, reductions, and transformations.
-
LLM-Based Porting of Optimized C++ to CUDA Through Deoptimization and Reoptimization
Deopt-Reopt workflow for LLM-based C++ to CUDA porting shows mixed performance gains over direct translation depending on kernel, model, and success rate, with no universal benefit.
-
KForge: LLM-Driven Cross-Platform Kernel Generation for AI Accelerators
KForge uses dual LLM agents for cross-platform kernel generation, reporting 2.12% throughput gain on NVIDIA B200 vs TensorRT-LLM and 5.13x geometric mean speedup on Intel Arc B580 vs PyTorch on 37 workloads.
-
Sol Video Inference Engine: Agent-Native Full-Stack Acceleration Framework for Efficient Video Generation
Sol Video Inference Engine uses parallel skill agents to optimize cache, sparse attention, token pruning, quantization, and kernel fusion, delivering over 2x end-to-end acceleration with near-lossless quality on three...
-
Generated, Parallel, Scalable? A Study of Agentic AI-Generated Julia Code on Supercomputers
Empirical study of agentic LLM generation of parallel Julia code finds reliable execution only at small scales with recurring failures in task dependencies and scheduling at larger scales.
-
HTAM: Hierarchical Transition-Attended Memory for Operator Optimization
HTAM builds a Hierarchical Transition Graph to organize coarse global directions and detailed local strategies for guiding LLM-based CUDA kernel optimization, improving results on KernelBench.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.