{"id":"978611ca-582f-421c-bb11-0e6d0c1f0cc6","arxiv_id":"2501.08071","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"CuAsmRL uses deep reinforcement learning to reorder GPU SASS instructions and reports average 9% throughput improvement on six LLM kernels on an NVIDIA A100.","lead":"This paper trains a reinforcement learning agent to reorder GPU assembly (SASS) instructions, automatically finding faster schedules for LLM CUDA kernels on NVIDIA A100 GPUs. The technique, CuAsmRL, reports up to 26% and on average 9% throughput gains on top of the Triton compiler's -O3 output, and is being released as open source.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline speedups lack statistical validation: selection of the best of thousands of noisy reward measurements biases the reported 9% average, and no error bars or seed variance are shown.","rationale":"I read the paper as claiming an empirical result: CuAsmRL improves existing Triton kernels by 9% on average, and this improvement is transparent. What has to be true: (1) the measured speedups are real, not artifacts of measurement or selection; (2) optimized schedules preserve semantics; (3) the integration is actually drop-in. The artifact and SASS-level analysis are real evidence, and the paper honestly acknowledges the correctness limitation in Section 7. The weakest link, in my view, is (1): without independent validation measurements, the magnitude of the headline is unsecured. The paper's statement that measurement standard deviation is within 1% refers to reproducibility of two measurements, not to the bias from selecting the best of many noisy evaluations during RL training. The reader's weakest assumption was correctness of the dependency masks; that is a genuine and explicitly admitted limitation, and a broad differential-testing suite could test it, but it does not directly undermine the measured speedups if outputs are verified. Statistical validation is the more decisive check for the central numerical claim. If the held-out test shows the speedups persist, the paper is accept-shaped; if not, the central claim fails. Therefore I recommend keeping the CONDITIONAL verdict, with the added condition of held-out statistical validation.","tokens_in":19275,"tokens_out":7856,"duration_ms":83998,"concrete_test":"Re-run the full CuAsmRL pipeline on the six kernels with, say, 10 random seeds. Use a strict train/validation split: the RL reward and the choice of best cubin use one measurement set (e.g., 100-iteration averages with L2 clears), and the final reported speedup uses a held-out set of 30 independent runs per seed. Report per-seed held-out speedups, means, and 95% bootstrap CIs, and test whether each kernel's held-out speedup is significantly greater than 0 against the same held-out Triton baseline. If the mean held-out gains are not significant, or if the held-out gains are systematically smaller than the in-search best, the headline 9% average is not established.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing claim is the measured speedup of CuAsmRL over Triton (up to 26%, mean 9%, Fig. 6). The paper reports no error bars, confidence intervals, or number of RL training seeds for Figure 6. This matters because the reward is a noisy execution-time measurement (average of 100 iterations, §3.6) and the deployed cubin is the best schedule encountered during the assembly game (§4.2: 'best optimized cubin found throughout the assembly game'). Selecting an argmax over thousands of noisy evaluations inflates the apparent improvement; the final 5-run benchmark (§5.3) uses the same measurement procedure and is not independent of the search. For kernels with only 2% reported gains, the entire effect could be selection bias or run-to-run noise. Separately, Section 7 requires manual verification of optimized kernels, so the 'transparent' part of the central claim is overstated until correctness of the reordering masks is demonstrated beyond the six kernels.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"CuAsmRL formulates GPU SASS instruction scheduling as a reinforcement-learning 'assembly game' in which an agent repeatedly swaps memory instructions with adjacent instructions, subject to dependency masks derived from register, barrier, stall-count, and hand-coded 'additional' dependencies. The reward is measured kernel throughput on an A100 GPU. The approach is integrated into OpenAI Triton: it disassembles the compiled cubin, trains an agent to reorder the SASS schedule, and deploys the best schedule found during training. On six LLM-relevant kernels (fused feed-forward, GEMM with LeakyReLU, batch matrix multiply, flash-attention, softmax, and rmsnorm), the paper reports up to 26% and on average 9% throughput improvement over Triton, and it analyzes the discovered reorderings and their effect on memory throughput.","tokens_in":19419,"tokens_out":5848,"duration_ms":54991,"significance":"If the measured speedups hold, CuAsmRL is a practical drop-in SASS-to-SASS optimizer that extends a widely used compiler framework (Triton) with low-level scheduling improvements. The artifact is publicly available on Zenodo/GitHub, the paper reports training statistics, and the method for inferring stall counts from existing schedules is a useful contribution. The paper also provides a concrete case study (HMMA/LDGSTS reordering and the .reuse-flag interaction) that could inform future compiler optimizations. However, the significance is tempered by the absence of statistical validation for the headline speedups and by the manual-verification caveat for correctness, which together make the strength of the central claim uncertain.","major_comments":[{"comment":"The central speedup claim lacks statistical validation. The reported numbers come from selecting the 'best optimized cubin found throughout the assembly game' (Section 4.2) using a reward signal that is itself a noisy execution-time measurement (Section 3.6: average of 100 iterations, with standard deviation typically within 1% of two measurements). Selecting an argmax over thousands of noisy evaluations biases the apparent improvement upward. The final benchmark in Section 5.1 uses the same 100-iteration measurement procedure and is not independent of the search. The paper should report per-kernel mean and standard deviation across multiple RL training seeds, include error bars or confidence intervals in Figure 6, and validate the chosen schedule with an independent measurement protocol (e.g., different iteration counts, clock sources, or re-measurement after the fact). For kernels with reported gains near 2%, the effect may be entirely within run-to-run noise; without these numbers the headline 'up to 26%, on average 9%' is not yet established.","section":"Section 5.3, Figure 6, Section 4.2"},{"comment":"The claim that CuAsmRL optimizes kernels 'transparently' (Abstract, Section 1) is contradicted by the paper's own limitation statement that 'users are required to manually verify the optimized kernels as in §5.7.' The correctness of every reordering depends on the completeness of the hand-coded 'additional dependencies' (Section 3.5) and the stall-count table (Section 4.3), and the paper provides no formal verification or exhaustive testing; probabilistic testing with randomized inputs is a sanity check, not a proof. The authors should either (a) weaken the transparency claim to 'source-transparent' with a clear statement that correctness requires per-kernel verification, or (b) provide stronger evidence of correctness, such as systematic output comparison on many random inputs for each of the six kernels, a description of the verification performed in §5.7, and an explicit discussion of which SASS hazard classes are covered by the additional-dependency heuristics.","section":"Section 7, Section 3.5, Section 4.3"}],"minor_comments":[{"comment":"There is a missing space in 'we observe7% improvement'; it should read 'we observe 7% improvement'.","section":"Section 5.7.1"},{"comment":"Reference [46] appears corrupted: 'Rico Zhang, Biao an/cudad Sennrich' is not a valid citation; please correct the author and title.","section":"References"},{"comment":"The pseudocode for masking stall counts is difficult to evaluate: the semantics of 'is_user(inst_to_check, cur)' are not defined, and the loop breaks at the first encountered user without considering multiple users or producers in the opposite direction. Please clarify why this is sufficient for safe reordering.","section":"Section 3.5, Algorithm 1"},{"comment":"The Cutlass baseline is described as using the 'default configuration' and is reported as 10x slower than Triton. This comparison is not informative unless the exact configuration is specified; consider reporting results with a tuned configuration or removing the baseline.","section":"Section 5.1"},{"comment":"The hyperparameter sensitivity analysis (Figure 8) is shown only for one kernel (fused GEMM with LeakyReLU). Please state whether similar behavior was observed for the other kernels or restrict the robustness claim accordingly.","section":"Section 5.5"},{"comment":"The statement that the standard deviation of two measurements is typically within 1% would be more useful if the actual per-kernel variances were reported alongside Figure 6.","section":"Section 3.6"},{"comment":"The example SASS instruction appears to be missing from the rendered text: after 'A typical SASS instruction is shown below' there is a blank line rather than a code listing.","section":"Section 2.3"}],"recommendation":"major_revision","confidential_remarks":"This is a solid systems contribution with a commendable artifact. The main revisions needed are statistical rigor (error bars, seeds, independent validation) and reconciling the 'transparent' claim with the manual-verification requirement. These are addressable within the scope of the paper, so I do not recommend rejection. The paper fits CGO's scope well."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"CuAsmRL is the first RL-based SASS schedule optimizer I know of, and it ships a real artifact. The assembly-game formulation—train PPO to swap memory instructions past neighbors within basic blocks, with dependency masks built from stall-count microbenchmarks and hand-coded heuristics—is a sensible way to automate what MaxAs/TuringAs did by hand. The integration into Triton (intercept the cubin, disassemble, reorder, reassemble) is clean, and the measured gains (up to 26%, mean 9% on six LLM kernels on A100) are accompanied by a breakdown that attributes part of the gain to the .reuse operand-cache flag. The paper also reports training statistics and provides code and data on Zenodo/GitHub. That is more reproducible than most CGO submissions.\n\nThe soft spots are real but not disqualifying. First, Figure 6 has no error bars or confidence intervals. The reward is the average of 100 timed iterations, the agent keeps the best cubin found over ~15k steps, and the final benchmark is a 5-run average of the same noisy measurement. Selecting an argmax over a noisy search inflates the apparent speedup; for kernels with 2% reported gains, the whole effect could be selection bias or run-to-run variance. The paper should report per-kernel variance, ideally across RL seeds. Second, the 'transparent' claim is overstated. The paper's own Section 7 says users must manually verify optimized kernels because correctness rests on hand-coded dependency rules and a small stall-count table. That is fine as a scoped claim—'transparent to kernel source, but requires validation of numerical correctness'—but the abstract currently promises more. Third, the evaluation is narrow: one GPU architecture, six LLM kernels. That's appropriate for a first paper, but it limits how far the 9% average can be generalized.\n\nI don't see a load-bearing flaw. The speedups are direct measurements, not predictions from fitted parameters, and the authors are honest about the limits of their dependency analysis. The paper deserves a serious referee; I would send it out with a request for error bars, a scoped transparency claim, and ideally one or two non-LLM kernels to test generality.","headline":"First RL-based SASS reorderer with a real artifact and credible measured gains; the headline speedup needs error bars and the 'transparent' claim needs scoping, but this deserves a serious referee.","tokens_in":19957,"tokens_out":2572,"would_cite":true,"duration_ms":25836,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"An RL agent reorders GPU assembly to speed up LLM kernels by up to 26 percent.","keywords":["GPU instruction scheduling","SASS","reinforcement learning","CUDA kernels","Triton compiler","large language models","PPO","Ampere GPU"],"falsifier":"Take a kernel from outside the LLM domain, apply the one-line CuAsmRL integration on an A100, and compare its output against the original on randomized inputs; if any optimized cubin returns wrong results, the completeness of the dependency masks is refuted, and if training never yields a faster schedule on such kernels, the claim of general applicability is weakened.","tokens_in":19029,"feed_emoji":"⚡","tokens_out":7718,"duration_ms":75605,"temperature":0.7,"pith_summary":"CuAsmRL claims that a deep reinforcement-learning agent can find instruction schedules for NVIDIA GPUs that beat the -O3 schedule produced by NVIDIA's own compiler, without touching kernel source code. The agent plays an assembly game: starting from an already-optimized SASS binary, it repeatedly swaps a memory load/store instruction with its neighbor, runs the mutated kernel on an A100 GPU, and uses the measured runtime as reward. Integrated into the Triton compiler as a SASS-to-SASS pass, CuAsmRL improved throughput on six representative LLM kernels by up to 26 percent and on average about 9 percent. The significance is that specialized CUDA kernels are already heavily hand-tuned, so an automatic layer that still finds headroom suggests instruction scheduling is a general, under-exploited optimization surface.","feed_headline":"RL agent reorders GPU assembly, speeding LLM kernels up to 26%","feed_subtitle":"A SASS-level optimizer beat ptxas -O3 on all six Triton kernels tested, averaging 9% faster on A100.","key_machinery":"The engine is an assembly game, a Markov decision process whose states are embeddings of SASS instruction streams, whose actions are swaps of a memory instruction with the instruction above or below, and whose reward is the percentage runtime improvement measured against the initial -O3 schedule. Correctness is enforced by action masking: a candidate swap is killed if it violates a register dependency, a barrier dependency, or a stall-count dependency. Stall counts come from a microbenchmarked table for common fixed-latency integer instructions and from a static analysis pass that infers them from the already-valid original schedule; memory instructions whose dependencies cannot be resolved are denylisted and never moved. The policy is a CNN encoder over the embedded instruction matrix followed by an MLP, trained with proximal policy optimization.","core_discovery":"On its own terms, the paper's central claim is that SASS instruction scheduling is a substantial and still-open performance surface for GPU kernels, and that a reinforcement-learning agent can automate the rescheduling human experts currently do by hand. Starting from the best schedule ptxas produces at -O3, CuAsmRL restricts its actions to moving memory load/store instructions such as LDG, LDGSTS, and STG one position up or down inside a basic block, with legal moves filtered by dependency masks for registers, barrier codes, and stall counts. The reward is the measured reduction in kernel runtime, and the agent is trained with PPO to maximize cumulative reward. In evaluation on six kernels used by LLMs, including fused feed-forward, batch matrix multiplication, flash attention, fused GEMM with LeakyReLU, softmax, and RMSNorm, the optimized schedules outperform Triton on every kernel, up to 26 percent and on average 9 percent. The paper also claims the learned moves are interpretable: one reordering of HMMA before LDGSTS in fused GEMM contributes 7 percent and is tied to operand-cache reuse.","pith_inferences":["The paper does not claim the speedups transfer to other GPU architectures; the stall-count table is A100-specific, so porting would require re-benchmarking before the same gains can be expected on newer hardware.","The authors leave implicit that the roughly five-hour per-kernel training cost could be amortized by pre-training a policy across many kernels, turning the optimizer into a default compiler pass rather than an offline per-kernel search.","A practical deployment consequence the paper does not spell out: transparent applies to source code, not to correctness, so an automated equivalence check would be needed before the optimized cubin could ship without human review."],"forward_implications":["If the measured 9 percent average speedup holds, users of Triton-compiled LLM kernels can gain a transparent speedup on A100 simply by switching to CuAsmRL's decorator, with no source-level changes.","The discovered moves, such as placing HMMA before LDGSTS and moving LDGSTS ahead of a predicated-off LDS, can be read as concrete scheduling rules for ptxas or other SASS-level passes.","Because optimization happens after code generation, the approach applies to any cubin that can be disassembled and reassembled, not only to kernels written in Triton.","The comparison showing CuAsmRL matching or beating CuBLAS and FlashAttention-2 baselines on several kernels implies that schedule-level optimization can close part of the gap between compiler-generated and hand-written GPU code."],"supporting_citations":[{"why":"Shows manual SASS rescheduling via trial-and-error, the human practice CuAsmRL replaces and automates.","marker":"[12]"},{"why":"Provides SASS instruction decoding and profiling-guided scheduling techniques that motivate the automated approach.","marker":"[45]"},{"why":"Supplies the open-source SASS assembler that CuAsmRL uses to reassemble mutated SASS into a cubin.","marker":"[4]"},{"why":"Defines the Triton compiler pipeline that CuAsmRL reuses and intercepts for SASS-to-SASS optimization.","marker":"[40]"},{"why":"Provides the proximal policy optimization algorithm used to train the RL agent.","marker":"[35]"},{"why":"Supplies the GPU instruction-latency microbenchmarking methodology that CuAsmRL adapts to build its stall-count table.","marker":"[13]"}],"fun_headline_variants":["RL agent reschedules GPU assembly, speeding kernels up to 26%","Deep RL automates GPU SASS scheduling for 26% faster kernels","RL plays an assembly game to speed LLM kernels by 26%","CuAsmRL: RL-optimized GPU assembly cuts kernel time 26%","Reinforcement learning reschedules SASS for 26% faster GPUs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reordering is safe only if the hand-coded dependency rules and the measured stall-count table cover every SASS hazard that exists; if any hazard is missing, an allowed swap can silently change the kernel's results, and the paper's own Section 7 asks users to verify optimized kernels manually.","fun_headline_variants_meta":{"raw":{"variants":["RL agent reschedules GPU assembly, speeding kernels up to 26%","Deep RL automates GPU SASS scheduling for 26% faster kernels","RL plays an assembly game to speed LLM kernels by 26%","CuAsmRL: RL-optimized GPU assembly cuts kernel time 26%","Reinforcement learning reschedules SASS for 26% faster GPUs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000724,"raw_usage":{"total_tokens":3283,"prompt_tokens":1020,"completion_tokens":2263,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":636,"completion_tokens_details":{"reasoning_tokens":2163}},"tokens_in":636,"tokens_out":2263,"duration_ms":15124,"temperature":1.0,"reasoning_tokens":2163,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:29:02.314862+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a kernel from outside the LLM domain, apply the one-line CuAsmRL integration on an A100, and compare its output against the original on randomized inputs; if any optimized cubin returns wrong results, the completeness of the dependency masks is refuted, and if training never yields a faster schedule on such kernels, the claim of general applicability is weakened.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Shows manual SASS rescheduling via trial-and-error, the human practice CuAsmRL replaces and automates."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the open-source SASS assembler that CuAsmRL uses to reassemble mutated SASS into a cubin."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the GPU instruction-latency microbenchmarking methodology that CuAsmRL adapts to build its stall-count table."}],"review_version":1}