{"id":"68214d9e-96d6-42bf-9bb1-83cfdb943d76","arxiv_id":"2411.18873","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A search-based GPU compiler that optimizes for energy as well as speed can cut kernel energy by up to about 22% with little latency change.","lead":"This paper describes a compiler that searches for GPU kernels that use less electricity, not just run faster, and reports up to 21.69% lower energy use on some deep learning operators. It matters because GPU power is a growing cost in AI data centers, and this is a software-only way to reduce it.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported RTX 4090 MV baseline energy implies 585 W average power, above the GPU's physical limit; measurement/unit error undermines cross-platform energy claims.","rationale":"The paper's central claim—'up to 21.69% reduced energy while maintaining low latency'—stands or falls on the correctness of the energy measurements. The RTX 4090 MV result in Table 3 implies 585 W average power for the baseline, which is physically impossible on that GPU; this is the kind of concrete red flag that puts the entire measurement pipeline in question. If the pipeline can produce a value above the hardware limit, then the A100 numbers, the 7.47% average reduction, and the energy cost model training data are all vulnerable to the same sampling/unit/overhead artifacts described in Section 4.4. The proposed concrete test would settle this by comparing the NVML-based method against a direct energy counter or wattmeter on the headline cases. If the reductions survive re-measurement with confidence intervals excluding zero, the central claim is credible despite the typo; if not, the paper needs substantial revision. Credit is due for the system design, the dynamic model-updating strategy, and the case study (Section 8), but these do not compensate for a physically impossible measurement in the main evaluation. This reinforces the reader's CONDITIONAL verdict: the paper should not be accepted until the measurement data are corroborated.","tokens_in":11966,"tokens_out":10550,"duration_ms":99689,"concrete_test":"Re-measure the Table 3 MV operator on RTX 4090 and the Table 2 MM1 operator on A100 using NVML's nvmlDeviceGetTotalEnergyConsumption (an energy counter rather than sampled power) or an external high-frequency wattmeter, with at least 10 trials per configuration. Report per-run energy as mean +/- std and compute the 95% confidence interval for each energy reduction. If the MV baseline does not reproduce near 6.9 mJ, or if the MM1 21.69% reduction's confidence interval includes 0%, the quantitative central claim is not supported by the evidence. Additionally, verify that the repeated-execution loop has no idle gaps that would make average-power times latency differ from true per-kernel energy.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"Table 3 reports the MV baseline on RTX 4090 as 6.909 mJ energy at 0.0118 ms latency. This implies an average power of 6.909e-3 J / 1.18e-5 s = 585 W, which exceeds the RTX 4090's ~450 W board limit. A sustained average over thousands of iterations cannot exceed the thermal/power limit if NVML readings are correct; the same table gives 263 W for the proposed MV kernel, so only the baseline is implausible. This is not a mere rounding issue: unless the energy/latency/unit entry is corrected, the cross-platform claim in Section 7.2 ('even higher on the RTX 4090', 53.13% reduction for MV) rests on an impossible measurement. Since every energy reduction and the energy cost model training data (Section 5.4) come from the same NVML pipeline described in Section 4.4, an internal inconsistency in one reported value casts doubt on the accuracy of the whole measurement procedure. The paper also reports single measurements with no confidence intervals, yet several Table 2 reductions are only 1.5-3%, a range that could be measurement noise at NVML's 30-50 Hz sampling rate.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an energy-aware search-based kernel generation framework for GPU tensor compilers. The method builds on Ansor's genetic-algorithm search, adding an energy objective: in each generation it first selects the fastest kernels, then ranks them by predicted energy from an XGBoost cost model, and finally measures a dynamic subset using NVML to update the model. The authors report up to 21.69% energy reduction on an NVIDIA A100 (average 7.47%), up to 53.13% on an RTX 4090, latency comparable to Ansor, and further comparison against cuBLAS. A case study attributes the savings to reduced static and memory-access energy due to smaller grid sizes and larger thread blocks.","tokens_in":12314,"tokens_out":5970,"duration_ms":51569,"significance":"If the reported energy reductions are accurate, this is a genuinely useful step toward making energy efficiency a first-class objective in tensor compilation. The paper identifies a real latency-power trade-off in kernel search, provides a fast machine-learning energy cost model, and introduces a dynamic measurement strategy that plausibly reduces search time. The evaluation against Ansor and cuBLAS is appropriate, and the A100 numbers are internally consistent. However, the absence of code, the lack of statistical repeatability, and a physically impossible baseline measurement in the RTX 4090 table currently limit confidence in the headline claims.","major_comments":[{"comment":"The RTX 4090 MV baseline is internally inconsistent and physically implausible: 6.909 mJ at 0.0118 ms implies an average power of 585 W, which exceeds the RTX 4090's ~450 W board power limit. The CONV baseline likewise implies 468 W. The proposed MV kernel gives 263 W, which is plausible. Because the 53.13% MV energy reduction and the statement that reductions are 'even higher on the RTX 4090' rest on this measurement, the NVML measurement pipeline described in Section 4.4 is called into question. Please re-measure these baselines and correct the table and derived percentages.","section":"Section 7.2, Table 3"},{"comment":"No error bars, standard deviations, confidence intervals, or repeated runs are reported. Several A100 reductions (for example MM2 4.57%, MM3 3.08%, MV1 2.92%, MV2 1.71%, CONV3 1.54%) are small enough to be comparable to NVML sampling noise at 30–50 Hz for millisecond-scale kernels. The paper should report variance over multiple independent searches and ideally a paired statistical test across operator shapes to establish that the reductions are not measurement artifacts.","section":"Section 7, Tables 2 and 3"},{"comment":"The energy cost model is claimed to be accurate, but the only evidence is a scatter plot with no quantitative error metric (R², MAPE, Spearman correlation) and no comparison to a trivial baseline predictor. Additionally, the weighted loss in Eq. (1) is introduced without an ablation showing it outperforms unweighted MSE. Since the dynamic updating strategy in Algorithm 1 relies on prediction-error thresholds, the paper should quantify the model's accuracy and report sensitivity of the final search quality to the threshold μ.","section":"Section 5.4 and Figure 4"},{"comment":"The claim that the cost-model-based search is 'nearly twice as fast' as NVML-only search is demonstrated only for a specific adjustment of μ that halves the number of measurements; this is a hyperparameter choice, not a parameter-free property of the method. The paper should provide a sensitivity analysis over μ and k, and should show that the final kernel energy and latency are not degraded relative to the NVML-only search. The prediction error is called 'SNR' in Algorithm 1 but the definition of this signal-to-noise ratio is never given.","section":"Section 7.4 and Algorithm 1"}],"minor_comments":[{"comment":"The word 'vaule' should be 'value', and the undefined term 'SNR' should either be defined formally or replaced with a concrete error metric.","section":"Algorithm 1"},{"comment":"The caption says 'The kernel generate by our method' and should be 'generated'; also 'Ansor's' would be clearer.","section":"Figure 2 caption"},{"comment":"The phrase 'An prediction error' should be 'A prediction error'.","section":"Section 6.4"},{"comment":"The description of 'normalized measured energy' and 'normalized predicted energy' is vague; please specify the normalization procedure so the plot can be interpreted.","section":"Section 7.3"},{"comment":"The time costs are given in hours (7.3h, 6.5h, etc.) but the number of search rounds and the number of kernels measured per round are not specified; please state the experimental protocol.","section":"Section 7.4"},{"comment":"The case study attributes the energy difference to static and memory-access energy, but no quantitative power or counter data is shown to support this decomposition; consider reporting measured power breakdowns or an ablation over grid/block sizes.","section":"Section 8"},{"comment":"The claim of being the 'first search-based energy-aware GPU kernel generation framework' is strong given prior work such as reference [19] on energy-aware tile size selection; please qualify the novelty statement.","section":"Section 1"}],"recommendation":"major_revision","confidential_remarks":"The RTX 4090 baseline inconsistency (6.909 mJ / 0.0118 ms = 585 W) looks like a unit or measurement error rather than a fundamental flaw in the methodology. However, because it appears in a headline result, the authors must correct it and ideally present repeated measurements before the paper can be accepted. The lack of code and the absence of error bars will also need attention. This is a fit for a systems/compilers venue, but the experimental rigor is not yet at the standard expected."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper's core idea is sound and worth engaging: add energy as an explicit objective in Ansor-style genetic search, backed by an XGBoost energy cost model and an adaptive measurement budget. That combination is new in the cited literature, and the case study in Section 8 gives a concrete, plausible explanation for why one kernel can use less energy than another at similar latency. The A100 results are internally consistent and show a meaningful 7.5% average energy reduction with negligible latency impact.\n\nThe soft spots are real and one is serious. Table 3 reports the MV baseline on RTX 4090 as 6.909 mJ at 0.0118 ms. That implies 585 W average power, which exceeds the RTX 4090's board limit of about 450 W. This is not a rounding issue; the entry is impossible. Since every energy number and the cost-model training data come from the same NVML pipeline, this internal inconsistency undermines the credibility of the measurement methodology, not just a single cell. The paper also gives no error bars, and several Table 2 reductions are only 1.5-3%, which could easily be noise at NVML's 30-50 Hz sampling rate on millisecond kernels even with thousands of repetitions. Add a small operator set, only two GPUs, no code or data release, and comparisons restricted to Ansor and cuBLAS, and you have an evaluation that is too thin for strong claims.\n\nStill, the central mechanism is principled. The paper does not claim to model energy from first principles; it measures what it can and uses the cost model only to accelerate search. The latency-focused selection within each generation is a sensible way to avoid trading away performance. The A100 numbers pass a sanity check (e.g., 239 W and 184 W for the MM1 kernels), so I do not think this is fabricated. The likely story is a misreported unit or a bad measurement on one platform.\n\nThis paper deserves a serious referee because the problem is important and the approach is novel. But it needs major revision: correct or explain Table 3, add confidence intervals, report the exact measurement protocol, and ideally release artifacts. As it stands, I would not cite it in my own work, but I would want to see a fixed version.\n\nMy recommendation: send it to peer review with a clear request for additional measurements and reproducibility. The idea should not be buried over a fixable data error.","headline":"A useful and novel combination for energy-aware GPU kernel search, but the evaluation has a physically impossible power reading in Table 3 that the authors must fix.","tokens_in":12766,"tokens_out":2224,"would_cite":false,"duration_ms":22631,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Search finds GPU kernels that use up to 21.69% less energy.","keywords":["GPU kernel generation","energy efficiency","search-based compilation","genetic algorithm","energy cost model","XGBoost","deep learning compilers","power measurement"],"falsifier":"Measure the same searched kernels from Table 2 with an independent high-bandwidth power trace, such as a current probe on the GPU power rail, and compare the energy ratios to the reported values; if the 21.69% reduction for MM1 does not reproduce under independent measurement, the reported savings are artifacts of NVML's 30-50 Hz sampling.","tokens_in":11808,"feed_emoji":"⚡","tokens_out":8753,"duration_ms":70469,"temperature":0.7,"pith_summary":"This paper claims that the energy a GPU kernel consumes can be cut substantially without slowing it down, if the search that generates kernels treats energy as a goal equal to speed. The authors show that kernels with nearly identical latency can differ a lot in energy, so among the fastest kernels there are lower-power versions worth choosing. To keep the search practical, they use a machine-learning model that predicts a kernel's energy from its loop structure and memory-access pattern, and they refresh that model during the search so that only a few candidate kernels need direct power measurement. Across operators from transformer and convolution models, their approach reduces kernel energy by up to 21.69% (7.47% on average) on an NVIDIA A100 and by more than 50% on some RTX 4090 cases, with latency staying about the same or even improving. The practical payoff is that data centers could lower both compute and cooling energy simply by choosing better kernel implementations.","feed_headline":"Search finds GPU kernels that use up to 21.69% less energy","feed_subtitle":"A genetic search that scores kernels by energy as well as speed avoids the usual latency trade-off.","key_machinery":"The central mechanism is a genetic algorithm with a two-stage fitness evaluation, plus a learned, dynamically updated energy cost model. In each search round, the algorithm measures kernel latency, keeps the $M$ fastest kernels, then uses the cost model to rank those by predicted energy and keeps a fraction of them. The cost model is an XGBoost regressor trained on high-level kernel features (floating-point and integer operation counts, loop depth and structure, vectorization, and cache-access counts) with a weighted squared-error loss that up-weights low-energy kernels. A parameter $k$ controls how many top-ranked kernels are actually measured with NVML power sampling each round: when the model's prediction error is below a threshold, $k$ shrinks and fewer measurements are needed; when the model drifts, $k$ grows. This online updating lets the search keep accuracy while cutting the main time cost, which is physical power measurement.","core_discovery":"The paper's central claim is that energy efficiency can be inserted into the kernel-generation search itself, rather than treated as a post-hoc hardware or workload-level fix. Building on a genetic search over tensor-program schedules, the authors select, in each generation, the fastest kernels first and then, among those, the ones with the lowest predicted energy. The key empirical discovery is that this two-stage selection finds kernels with clearly lower energy than a latency-only search while keeping latency essentially unchanged; in some cases the energy reduction even comes with slight latency improvement. The authors attribute the effect to structural differences in grid and block size: a smaller grid leaves more streaming multiprocessors idle, cutting static energy, and larger blocks increase data reuse, cutting memory-access energy. They report up to 21.69% energy reduction on an A100 GPU and up to 53.13% on an RTX 4090, against the Ansor baseline, with average latency changing by less than two percent.","pith_inferences":["The two-stage selection (fastest first, then most energy-efficient) could be applied to other search-based compilers beyond Ansor, and the energy cost model could be retrained on any hardware with an energy API, so the approach may generalize to CPUs, FPGAs, or future GPU generations beyond the two tested.","The reported 53.13% energy reduction for the MV operator on RTX 4090 suggests that memory-bound operators, common in large language model inference, might be the biggest beneficiaries; this is not a claim the paper makes explicitly.","The grid-size/block-size mechanism implies that deliberately under-utilizing a GPU (leaving SMs idle) can save energy when the workload is small; a testable extension is to benchmark whether a single search round can also optimize power-capping settings together with kernel structure.","Since the energy cost model is trained on a small set of operators, its predictions could drift on unseen shapes; the paper's dynamic updating strategy mitigates this, but a stronger test would be to evaluate the energy model's ranking accuracy (not just the final kernel) on a held-out operator family."],"forward_implications":["Energy becomes a first-class objective in tensor program search, so any workload compiled with such a scheduler can inherit energy savings without requiring new hardware or changes to the model.","Because the method keeps latency close to the latency-optimal kernel, it can be dropped into existing serving or training stacks as a drop-in kernel replacement.","The case study shows that static energy depends on how many SMs are active; choosing smaller grids for small batch sizes could become a general compiler heuristic.","The dynamic cost-model updating strategy reduces the number of on-device measurements, making energy-aware compilation practical at the scale of thousands of kernels."],"supporting_citations":[{"why":"supplies the genetic search space, the Ansor baseline, and the latency cost model structure that the paper extends with energy selection.","marker":"[30]"},{"why":"provides the XGBoost tree-boosting algorithm used to build the energy cost model.","marker":"[10]"},{"why":"supplies the constant/static/dynamic power decomposition and the memory-access energy share that motivate the model features and the case study.","marker":"[21]"},{"why":"is the NVIDIA Management Library API used for all ground-truth power and energy measurements.","marker":"[5]"},{"why":"is cited to justify preheating the GPU before measurements, making energy readings repeatable despite temperature effects.","marker":"[18]"}],"fun_headline_variants":["Energy-aware kernel search cuts GPU power up to 21.69%","Compiler search yields GPU kernels using 21.69% less energy","Search-based kernel generation trims GPU energy 21.69%","GPU kernel search slashes energy by up to 21.69%","Two-stage kernel search finds low-energy GPU kernels, 21.69% savings"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"All reported energy reductions and the training data for the cost model depend on NVML power sampling at 30-50 Hz, combined with preheating and thousands of repeated kernel executions, being accurate and repeatable enough to capture millisecond-scale kernel energy.","fun_headline_variants_meta":{"raw":{"variants":["Energy-aware kernel search cuts GPU power up to 21.69%","Compiler search yields GPU kernels using 21.69% less energy","Search-based kernel generation trims GPU energy 21.69%","GPU kernel search slashes energy by up to 21.69%","Two-stage kernel search finds low-energy GPU kernels, 21.69% savings"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00034,"raw_usage":{"total_tokens":1838,"prompt_tokens":871,"completion_tokens":967,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":487,"completion_tokens_details":{"reasoning_tokens":885}},"tokens_in":487,"tokens_out":967,"duration_ms":6977,"temperature":1.0,"reasoning_tokens":885,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T10:47:42.537352+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the same searched kernels from Table 2 with an independent high-bandwidth power trace, such as a current probe on the GPU power rail, and compare the energy ratios to the reported values; if the 21.69% reduction for MM1 does not reproduce under independent measurement, the reported savings are artifacts of NVML's 30-50 Hz sampling.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"supplies the constant/static/dynamic power decomposition and the memory-access energy share that motivate the model features and the case study."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"is the NVIDIA Management Library API used for all ground-truth power and energy measurements."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"is cited to justify preheating the GPU before measurements, making energy readings repeatable despite temperature effects."}],"review_version":1}