{"id":"e90e0f00-1677-4a77-a9ba-abc7f651c782","arxiv_id":"2506.15701","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"A two-stage SFT+RL pipeline trains LLM agents to call compiler-analysis tools and select pass sequences, achieving 8.46% mean IR instruction reduction, but the tool's own contribution is not controlled.","lead":"Compiler-R1 trains small LLMs with reinforcement learning to choose LLVM optimization pass sequences, reporting an average 8.46% reduction in IR instruction count over the -Oz baseline across seven benchmarks. The results are difficult to attribute to the RL-trained model because the agent can invoke a built-in search tool, find_best_pass_sequence, which may be performing the actual optimization.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Tool confound undermines attribution: find_best_pass_sequence alone may account for the reported 8.46% gain; no tool-only baseline is reported.","rationale":"The reader's weakest assumption is correct and load-bearing. The paper has real strengths: it releases code and data, uses a sensible two-stage SFT+RL pipeline, and includes ablations showing that RL improves over SFT-only direct prediction and that SFT is needed for protocol compliance. But those ablations never isolate the search tool. Because the same graph-guided search that generated training labels is callable by the agent during RL and evaluation, the framework can achieve high OverOz simply by learning to invoke an existing optimizer. This is not an internal inconsistency, but it is a decisive experimental gap for the stated contribution, which is about RL-trained LLM abilities. The missing baseline is cheap and would settle the question. Until it is reported, the central claim is unsupported; I agree with the reader's REJECT verdict.","tokens_in":15455,"tokens_out":5027,"duration_ms":59773,"concrete_test":"Run find_best_pass_sequence alone on the 335 test programs used in Table 1, under the same evaluation protocol (default -Oz fallback on failures), and report average OverOz% against GRPO-7B's 8.46%. In the same run, instrument every Compiler-R1 trajectory to record whether the final <answer> sequence is exactly the output of a find_best_pass_sequence call or a self-generated sequence, and report the per-source average OverOz. If FBPS-alone reaches or approaches 8.46%, or if tool-returned answers dominate the successful trajectories, the RL-trained LLM's learned contribution is not demonstrated. A complementary ablation that disables FBPS during RL evaluation would show the model's standalone optimization quality.","verdict_should_be":"REJECT","load_bearing_attack":"Compiler-R1's headline result is an 8.46% average OverOz gain by GRPO-7B (Table 1). The paper's central claim is that this gain comes from an RL-trained LLM agent that learns compiler-pass interactions. In the evaluation protocol, however, the agent is explicitly allowed to call find_best_pass_sequence (FBPS), a guided search tool, whenever its own initial sequence fails to beat -Oz (Sec. 3.2.1 prompt; Appendix D). FBPS is also the mechanism used to produce the 'optimal' PassSeqopt(P) that generated the SFT labels (Sec. 3.1), so it is the very search process the RL model is supposedly learning to replace or outperform. No experiment reports what FBPS alone achieves on the 335 test programs, and the ablations do not separate tool-returned sequences from self-generated ones. Section 4.3 even states that GRPO-3B obtains 5.12% OverOz by directly invoking FBPS and bypassing instrcount, which shows that a policy whose main action is to call the tool can produce strong results without learned pass reasoning. Consequently, the 8.46% figure may be almost entirely attributable to the search tool, not to the RL-trained LLM. The central attribution is unsupported without a tool-only baseline and a provenance breakdown of final answers.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"Compiler-R1 proposes a two-stage supervised fine-tuning (SFT) plus reinforcement learning (RL) framework for LLM-based compiler auto-tuning. The authors construct a reasoning dataset by using a graph-guided search to select optimal pass sequences, then train Qwen2.5 models to interact with two tools: instrcount and find_best_pass_sequence (FBPS). On seven benchmark suites, the GRPO-7B variant is reported to achieve an average 8.46% IR instruction count reduction over opt -Oz, outperforming SFT-only models and several traditional autotuners. The paper argues that RL enables the agent to learn pass interactions and generalize to unseen programs; however, the agent is allowed to invoke FBPS, the same search tool used to generate the SFT labels, and no standalone tool baseline is reported.","tokens_in":15729,"tokens_out":4699,"duration_ms":49407,"significance":"If the attribution were established, this would be a useful contribution: it combines LLM reasoning with compiler-environment tools and demonstrates an end-to-end RL pipeline with public code and datasets, deterministic evaluation protocols, and several ablations. The two-stage SFT+RL design and the outcome-based reward are sensible. The significance is currently contingent, however, because the experiments do not separate what the learned policy contributes from what the FBPS search tool contributes; the headline 8.46% figure may be largely a property of the tool rather than of RL-trained LLM reasoning.","major_comments":[{"comment":"The agent is instructed to call find_best_pass_sequence whenever the initial candidate sequence gives OverOz ≤ 0, and this tool is the same graph-guided search used in §3.1 to generate PassSeqopt(P). The paper never reports what find_best_pass_sequence alone achieves on the 335 test programs. Section 4.3 explicitly states that GRPO-3B obtains 5.12% OverOz by directly invoking find_best_pass_sequence and bypassing instrcount, which shows that the tool alone can produce strong results. Without a tool-only baseline and a per-answer provenance breakdown, the 8.46% improvement in Table 1 cannot be attributed to RL-trained LLM reasoning.","section":"§3.2.1, §4.3, Appendix D"},{"comment":"The SFT labels are the output of the same graph-guided search that the agent may invoke during RL and evaluation, so the model can succeed by reproducing the tool's output verbatim without learning any pass interaction. The paper does not report the fraction of final answers that are tool-returned sequences, nor does it ablate tool availability. I request an experiment that disables find_best_pass_sequence during RL/evaluation, or alternatively a separate measurement of the policy's self-generated sequences, to show that RL improves the policy beyond imitating the search tool.","section":"§3.1, §3.2.2, §4.2"},{"comment":"Table 2 reports success rates for two repetition penalties, and GRPO-7B's success rate changes from 51.92% at penalty 1.05 to 96.71% at penalty 1.10. The paper does not state which repetition penalty and other hyperparameters are used for the headline GRPO-7B result in Table 1, making the main result non-reproducible as written. This configuration must be specified.","section":"Table 1, Table 2"}],"minor_comments":[{"comment":"The text says 'average OverOrig gain of 8.46%,' but Table 1 and Eq. (2) define the 8.46% figure as OverOz; OverOrig is defined in Eq. (4) relative to the original unoptimized program, not to -Oz. Please correct the terminology.","section":"§4.5"},{"comment":"Equation (1) uses Apply(P, B) and Apply(P, A, B) without defining the order of application for the pair; the surrounding text explains that A is applied before B, but the notation should be made explicit in the equation or its caption.","section":"§3.1"},{"comment":"The observation that GRPO-3B 'circumvents protocol checks' by directly invoking find_best_pass_sequence is important and should be quantified: report how often each model's final answer comes from FBPS rather than from a self-generated sequence.","section":"§4.3"},{"comment":"The table label 'Avg. Max OverOrig' is not defined; the text describes 'reporting the best OverOrig among them,' so please clarify whether the table reports the mean over test programs of the maximum over N samples.","section":"§4.4, Table 3"}],"recommendation":"major_revision","confidential_remarks":"The tool confound is the central issue: the paper's headline result is not attributable to RL until a tool-only baseline and a provenance breakdown are provided. Given that the code and datasets are public, these experiments are feasible and should be required before reconsideration."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You asked about Compiler-R1. The short version: it's a well-put-together system paper with a clean two-stage SFT+RL pipeline for LLM-based compiler auto-tuning, but the headline number is not attributable to the RL component because the agent is allowed to call a search tool (find_best_pass_sequence, FBPS) that is also the source of the SFT labels, and the paper never measures what FBPS alone does on the test benchmarks. That's a load-bearing omission.\n\nWhat's new and good: the paper constructs a curated reasoning dataset by extracting synergistic pass pairs from a guided search and generating thought-action-feedback trajectories, then initializes an LLM with SFT on 800 samples and applies rule-based RL (GRPO/PPO/RPP) with a composite reward. The evaluation is fairly broad: seven benchmarks, several model sizes, and comparisons to classical autotuners (OpenTuner, GA, RIO, etc.) and SFT-only baselines. The runtime advantage over traditional autotuners is real and practically relevant. The code and data are public, which is creditable.\n\nThe soft spot is serious. Section 3.2.1 makes FBPS part of the SFT protocol: the model is trained to call it whenever its own sequence fails to beat -Oz. Section 3.2.2 keeps that tool available during RL and evaluation. In fact, Section 4.3 mentions that GRPO-3B gets 5.12% OverOz largely by invoking FBPS directly and skipping the instrcount validation. Since FBPS was used to generate the \"optimal\" sequences in the training data, the RL model may simply be learning to emit the tool call. The paper never reports a tool-only baseline (run FBPS on the 335 test programs with no LLM), nor does it break down final answers by whether they came from the model's own reasoning or from FBPS. Without that control, the 8.46% figure for GRPO-7B cannot be attributed to the RL-trained policy. The central claim in the abstract is unsupported.\n\nIs this fatal? For the current claim, yes. But the flaw is easy to fix: add a tool-only control and a provenance analysis. If FBPS alone gets 8% and the LLM adds nothing, the RL contribution is near zero. If the LLM's own sequences account for a meaningful share, the paper becomes interesting. Either way, the community would benefit from knowing.\n\nThe paper is well structured, the writing is clear, and the dataset and code are shared. It's above the desk-reject threshold, in my view. I would send it to referees, but with the explicit requirement that they address the attribution question. My own verdict would be major revision with a chance of acceptance if the control confirms the RL contribution; if not, the system is still a useful combination, but the framing has to change.\n\nI'd bring it to reading group—the issue is exactly the kind we like to argue about. I wouldn't cite the headline number until the control is reported.","headline":"Solid system paper, but the headline RL gain is unproven without a tool-only baseline; the FBPS confound is load-bearing.","tokens_in":16264,"tokens_out":4598,"would_cite":false,"duration_ms":39399,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Compiler-R1 shows that an LLM trained with reinforcement learning and tool access finds compiler pass sequences that reduce IR instruction counts by an average of 8.46% over the opt -Oz baseline across seven benchmark suites.","keywords":["compiler auto-tuning","LLM agent","reinforcement learning","LLVM pass ordering","IR instruction count","tool-augmented reasoning","two-stage training","outcome-based reward"],"falsifier":"Run find_best_pass_sequence alone on the same 335 test programs and compare its average OverOz with GRPO-7B's 8.46%; if the standalone tool matches or exceeds that figure, or if removing the tool from the agent's available actions during RL collapses performance toward the SFT-only baseline near 4.66%, then the paper's attribution of the gain to RL training is contradicted.","tokens_in":15272,"feed_emoji":"⚙️","tokens_out":14187,"duration_ms":128278,"temperature":0.7,"pith_summary":"Compiler auto-tuning today is caught between slow search-based autotuners and LLMs that imitate fixed examples without checking results. This paper argues that an LLM agent trained with reinforcement learning can do better by interacting with the compiler: querying pass effects, receiving feedback, and refining its strategy. Compiler-R1 first learns a tool-use protocol from curated reasoning traces, then optimizes a reward tied to actual IR instruction-count reduction. Across seven benchmark suites, the strongest variant reports an average 8.46% reduction in IR instruction count relative to opt -Oz, while supervised-only baselines reach at most 4.66% even after 40 sampling attempts. If the attribution holds, RL-trained agents would offer a practical and much faster route to compiler optimization.","feed_headline":"LLM agent trained with RL cuts compiler instructions 8.46%","feed_subtitle":"Tool-augmented agent beats opt -Oz across seven suites and rivals traditional autotuners in a fraction of the time","key_machinery":"The central mechanism is a closed loop between an LLM agent and the compiler environment, held together by two tools. The agent emits structured traces with <think>, <tool_call>, and <answer> markers; one tool, instrcount, returns a candidate sequence's improvement over -Oz, and the other, find_best_pass_sequence, is a guided search invoked when the candidate underperforms. On the data side, the paper builds a global graph of synergistic pass pairs (A,B), where B alone reduces instruction count and applying A before B reduces it further, then samples candidate sequences from that graph and keeps the best against -Oz as the supervised label. During RL, the reward is a weighted sum of a format reward for protocol-valid traces and an answer reward proportional to instruction-count reduction over the unoptimized program. This loop is what lets the model verify its guesses, fall back to search, and receive dense scalar feedback for policy learning.","core_discovery":"The paper's central claim is that reinforcement learning over a tool-augmented agent loop, rather than supervised imitation alone, is what makes LLM-based compiler auto-tuning work. The headline evidence is GRPO-7B reaching 8.46% average OverOz—the percentage IR instruction-count reduction relative to opt -Oz—across seven test suites, with a 96.71% task success rate, while the best SFT-only model reaches 4.66% after 40 inference attempts and the best traditional autotuner in the comparison reaches 6.09% but takes far longer. The authors attribute the result to the two-stage design: supervised fine-tuning provides a reliable thought–tool–answer protocol, and outcome-based RL lets the agent discover better sequences through trial and error. They further claim that neither stage alone suffices and that compact statistical features represent programs almost as well as raw LLVM IR for this task.","pith_inferences":["The paper does not measure how good find_best_pass_sequence is on its own; since the same guided search produced the SFT labels and stays available during RL and evaluation, a direct head-to-head would settle whether the RL policy adds optimization skill or simply learns when to call a strong oracle.","The reward is based on IR instruction count rather than wall-clock runtime, so the reported reductions may not translate one-to-one into faster executable code; an end-to-end hardware benchmark would test that translation.","All candidate sequences originate from the synergy-graph random walks, so the agent's discoveries are bounded by that search landscape; comparing RL-found sequences against graph-generated candidates would show whether genuinely new pass interactions are being learned.","The same tool-augmented RL recipe could carry over to other compiler objectives such as code size, compile time, or energy, and to any optimization domain with a cheap verifier; that would be a natural next test of generality."],"forward_implications":["GRPO-7B achieves an average 8.46% IR instruction-count reduction over opt -Oz across the seven evaluated suites in about 26 seconds per program.","RL-trained interactive agents outperform supervised-only direct predictors (best 4.66% at 40 attempts) with a single environment-driven trajectory, indicating that feedback matters more than brute-force sampling.","Compiler-R1 matches or beats traditional autotuners in optimization quality while using a fraction of their wall-clock time, making it a candidate for practical deployment.","The two-stage SFT-then-RL design is necessary: ablations show neither SFT alone nor RL alone yields reliable interaction or competitive performance.","Compact statistical program features perform about as well as raw LLVM IR as model input, so the approach can run under tight context limits."],"supporting_citations":[{"why":"Supplies the compiler environment, action space, and datasets used for training and evaluation.","marker":"[10]"},{"why":"Provides the statistical feature representation used as model input and is also a strong RL baseline.","marker":"[15]"},{"why":"Contributes the empirical tuning workflow and synergistic-search strategy behind the training-data construction.","marker":"[24]"},{"why":"Establishes the rule-based RL training paradigm that the two-stage pipeline extends.","marker":"[11]"},{"why":"Inspires the coreset-based sampling used to diversify the training set.","marker":"[21]"},{"why":"Serves as a leading traditional autotuner baseline in the comparison.","marker":"[1]"},{"why":"Is one of the on-policy RL algorithms applied in the second training stage.","marker":"[25]"},{"why":"Represents the prior LLM-based pass-order approach that the paper argues lacks environment interaction.","marker":"[8]"}],"fun_headline_variants":["RL-driven LLM agent cuts compiler instructions 8.46%","First RL framework for compiler auto-tuning: 8.46% cut","Agentic compiler tuning: RL beats SFT and traditional autotuners","Reinforcement learning improves LLM compiler tuning by 8.46%","RL-trained LLM agent outperforms opt -Oz by 8.46% on average"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that the instruction-count reductions come from the RL-trained agent's learned policy rather than from the embedded find_best_pass_sequence search tool, whose standalone performance is never reported even though the same guided search produced the SFT training labels.","fun_headline_variants_meta":{"raw":{"variants":["RL-driven LLM agent cuts compiler instructions 8.46%","First RL framework for compiler auto-tuning: 8.46% cut","Agentic compiler tuning: RL beats SFT and traditional autotuners","Reinforcement learning improves LLM compiler tuning by 8.46%","RL-trained LLM agent outperforms opt -Oz by 8.46% on average"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000245,"raw_usage":{"total_tokens":1513,"prompt_tokens":903,"completion_tokens":610,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":519,"completion_tokens_details":{"reasoning_tokens":509}},"tokens_in":519,"tokens_out":610,"duration_ms":5716,"temperature":1.0,"reasoning_tokens":509,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T12:35:17.877701+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run find_best_pass_sequence alone on the same 335 test programs and compare its average OverOz with GRPO-7B's 8.46%; if the standalone tool matches or exceeds that figure, or if removing the tool from the agent's available actions during RL collapses performance toward the SFT-only baseline near 4.66%, then the paper's attribution of the gain to RL training is contradicted.","supporting_citations":[{"cited_title":"Compilergym: Robust, performant compiler optimization environments for ai research","cited_arxiv_id":null,"evidence_quote":"Supplies the compiler environment, action space, and datasets used for training and evaluation."},{"cited_title":"Autophase: Juggling hls phase orderings in random forests with deep reinforcement learning","cited_arxiv_id":null,"evidence_quote":"Provides the statistical feature representation used as model input and is also a strong RL baseline."},{"cited_title":"Towards efficient compiler auto-tuning: Leveraging synergistic search spaces","cited_arxiv_id":null,"evidence_quote":"Contributes the empirical tuning workflow and synergistic-search strategy behind the training-data construction."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes the rule-based RL training paradigm that the two-stage pipeline extends."},{"cited_title":"Learning compiler pass orders using coreset and normalized value prediction","cited_arxiv_id":null,"evidence_quote":"Inspires the coreset-based sampling used to diversify the training set."},{"cited_title":"Opentuner: An extensible framework for program autotuning","cited_arxiv_id":null,"evidence_quote":"Serves as a leading traditional autotuner baseline in the comparison."}],"review_version":1}