Large Language Models for Compiler Optimization
read the original abstract
We explore the novel application of Large Language Models to code optimization. We present a 7B-parameter transformer model trained from scratch to optimize LLVM assembly for code size. The model takes as input unoptimized assembly and outputs a list of compiler options to best optimize the program. Crucially, during training, we ask the model to predict the instruction counts before and after optimization, and the optimized code itself. These auxiliary learning tasks significantly improve the optimization performance of the model and improve the model's depth of understanding. We evaluate on a large suite of test programs. Our approach achieves a 3.0% improvement in reducing instruction counts over the compiler, outperforming two state-of-the-art baselines that require thousands of compilations. Furthermore, the model shows surprisingly strong code reasoning abilities, generating compilable code 91% of the time and perfectly emulating the output of the compiler 70% of the time.
This paper has not been read by Pith yet.
Forward citations
Cited by 8 Pith papers
-
LLM Translation of Compiler Intermediate Representation
IRIS-14B is the first LLM trained explicitly for GIMPLE-to-LLVM IR translation and outperforms much larger models by up to 44 percentage points on real-world C code.
-
Source-Free Detection and Impact Analysis of Compiler Optimization Problems in Mobile Applications
OptDetect identifies low-optimization native libraries in Android apps with 81.9% real-world accuracy, finding the issue in 30.5% of libraries across 91.7% of top apps, with fixes yielding 10-63% CPU reduction.
-
TensorBench: Benchmarking Coding Agents on a Compiler-Based Tensor Framework
TensorBench is a new benchmark of 199 tasks on a tensor framework used to evaluate seven coding agents, yielding pass rates from 22.1% to 64.8% with low inter-agent agreement.
-
Step-TP: A Grounded, Step-Level Dataset with Chain-of-Thought Reasoning for LLM-Guided Tensor Program Optimization
Step-TP is a dataset providing grounded, atomic step-level IR transitions and CoT supervision to enable reliable multi-step LLM-guided tensor program optimization instead of end-to-end imitation.
-
JETO-Bench: A Reproducible Benchmark for Execution Time Improvement Patches in Java
JETO-Mine is a reusable three-phase pipeline that mines 1.8 million Java commits to produce JETO-Bench containing 91 verified executable ETIPs, on which OpenHands succeeds at 14.3%.
-
AgentCompile: An LLM-Guided Compiler for Direct CUDA Inference
AgentCompile is an LLM-guided CUDA compiler that reports 4-5.7x speedups over PyTorch eager for small transformer models by treating LLM suggestions as search metadata while relying on compiler validation and measured...
-
InCoder-32B-Thinking: Industrial Code World Model for Thinking
InCoder-32B-Thinking uses error-feedback synthesized thinking traces and a code world model to reach top open-source scores on general and industrial code benchmarks including 81.3% on LiveCodeBench and 84.0% on CAD-Coder.
-
Modeling Pathology-Like Behavioral Patterns in Language Models Through Behavioral Fine-Tuning
Fine-tuning LLMs on structured tasks inspired by maladaptive behaviors produces stable, context-general shifts in next-token distributions and response tendencies consistent with altered behavioral priors.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.