{"id":"bb36790a-da1e-4e44-8800-2872b1df72c8","arxiv_id":"2505.21758","paper_version":2,"verdict":"REJECT","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":1,"one_line_summary":"Task-specific power caps can shift the energy-time trade-off for GPU kernels in LSMS, but the paper's headline savings of 151% to 200% are artifacts of summing per-kernel percentages incorrectly.","lead":"The paper measures how capping the power of an NVIDIA GH200 chip affects the energy use and runtime of individual GPU tasks in a materials-science code, and tests two metrics for picking the best cap per task. It claims large energy savings for task-level power capping, but the headline figures come from an invalid way of adding up per-task percentages.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 151%/200% energy-savings headline is an unweighted sum of per-task percentages, which cannot exceed 100%; a correct energy-weighted aggregation gives roughly 7–11% savings.","rationale":"The reader's weakest assumption correctly identifies the fatal flaw: summing per-task percentage changes to estimate whole-application savings is arithmetically invalid. My independent reading of Section 4 confirms that the 151% and 200% figures come from exactly this unweighted sum, and a proper energy-weighted aggregation using the paper's own tables gives savings an order of magnitude smaller. The paper does contain real measurements and useful qualitative insights about GH200 power capping, which I credit, but the central quantitative claim and the generalized conclusion are unsupported. Since the reader's verdict is already REJECT and my analysis reinforces it, the verdict should remain unchanged.","tokens_in":9743,"tokens_out":3085,"duration_ms":29599,"concrete_test":"Recompute whole-application savings using the paper's own Table 1 and Table 2 values: for each GPU task, compute ΔE_k = (percent energy reduction_k) × (baseline task energy_k from Table 1), sum ΔE_k over tasks, and divide by the reported total application energy of 93,574.7 J; repeat for runtime using baseline task times and Table 2 runtime increases. If this yields roughly 7–11% energy savings and about 6% runtime increase instead of 151% and 90%, the claimed aggregation is invalid and the conclusion must be revised.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4's central quantitative claim ('~151% energy savings... ~200% energy reduction') is produced by summing per-GPU-task percentage reductions in Table 2. This is arithmetically invalid: each task's percentage is relative to that task's own baseline energy, so the percentages are not commensurable and cannot be added across tasks. Because total energy is a sum of task energies, whole-application savings from capping these tasks can never exceed 100%. The paper's own caveat that these are 'simple aggregations... ideal scenarios with no overhead' does not fix the weighting error; the error persists even with zero overhead. Using Table 1 baseline task energies and Table 2 percentage changes, the correctly weighted whole-application energy saving is roughly 11% of the profiled GPU-task energy (about 7% of the reported 93,574.7 J total), with a runtime increase of about 6% for the speedup-energy-delay selections. Thus the headline conclusion—'kernel-level power-capping can yield greater energy savings than application-wide tuning'—is not supported by the reported data. The per-kernel measurements and qualitative memory-bound versus compute-bound observations are plausible, but the paper's central quantitative contribution rests on this invalid aggregation.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper reports an empirical study of power-capping on an NVIDIA GH200 superchip for the LSMS exascale application. It profiles individual GPU kernels and GPU-idle phases under global superchip power caps from 200 W to 1000 W, evaluates two decision metrics (speedup-energy-delay and Euclidean distance of normalized energy/runtime), selects a per-task 'optimal' power cap for each metric, and reports percentage energy reductions and runtime increases relative to the 1000 W baseline. The central quantitative claim is that aggregating per-task percentage changes yields roughly 151% energy savings at about 90% runtime increase for speedup-energy-delay, and roughly 200% energy reduction at about 203% runtime increase for the Euclidean metric. The paper concludes that kernel-level power-capping can yield greater energy savings than application-wide tuning.","tokens_in":9882,"tokens_out":8839,"duration_ms":77745,"significance":"If the headline result were correct, this would be a practically important demonstration that fine-grained, per-GPU-task power capping on integrated CPU-GPU architectures can produce very large energy savings. The measurement methodology is a strength: the authors use Score-P, PAPI, and NVML to collect 5 ms power samples at GPU-task granularity, and the qualitative separation between compute-bound and memory-bound kernels is a useful characterization. However, the central quantitative claim is arithmetically invalid: the reported percentages are obtained by summing per-task percentage changes rather than weighting by each task's energy, and the resulting 'savings' exceed 100% of the total energy. Recomputing with Table 1 baseline energies gives roughly 11-14% energy savings on the profiled GPU-task energy. The contribution is therefore a preliminary, qualitatively informative case study, not the large efficiency gain claimed in the abstract and conclusions.","major_comments":[{"comment":"The headline '~151% energy savings' and '~200% energy reduction' are unweighted sums of the per-task percentage changes in Table 2. For example, the SED energy column sums to 0.85+22.92+31.40+20.61+10.05+9.02+9.10+46.58 = 150.53, and the ED column sums to about 200.09. Because each percentage is relative to that task's own baseline energy, the percentages are not commensurable and cannot be added across tasks; total energy is the sum of task energies, so savings can never exceed 100% of that total. Using the baseline energies in Table 1, the correctly weighted savings are about 6.8 kJ out of 63.6 kJ of profiled GPU-task energy (~11%) with about a 6% weighted runtime increase for SED, and about 8.7 kJ (~14%) with about a 14% weighted runtime increase for ED; as fractions of the reported 93,574.7 J total, these are even smaller (~7-9%). The sentence 'These numbers are simple aggregations ... ideal scenarios with no overhead' acknowledges an idealization but does not address the weighting error, which persists at zero overhead.","section":"Section 4, Table 2"},{"comment":"The 'optimal' power cap for each task is selected by minimizing or maximizing the chosen metric over the same measured power-cap sweep, and Table 2 then reports the energy/runtime difference between that selected setting and the 1000 W baseline within the same dataset. This is an in-sample selection of the optimum, so the reported percentage changes are best-case values for the measured data rather than expected savings under deployment. With only three runs per setting and no confidence intervals, cross-validation, or a holdout run at the selected caps, the claim in Section 4 that these results 'can be achieved' is overstated; the paper should either reframe the results as in-sample best-case observations or add validation to support predictive claims.","section":"Section 3.2 and Table 2"},{"comment":"The conclusion that 'kernel-level power-capping can yield greater energy savings than application-wide tuning' is not supported by the reported experiments. The paper does not compare against any application-wide power-capping tuning experiment, and the experiments apply a single global power cap for an entire run (Section 3 states 'We executed the entire application at every superchip power limit'), not dynamic per-task capping during a single execution. The claimed superiority of fine-grained capping is therefore an extrapolation, not a measured result. A direct comparison, or at minimum a clear statement that this is a hypothesis for future work, is needed.","section":"Section 4 and Section 5"}],"minor_comments":[{"comment":"Reference [1] contains a formatting error: 'In: 20Haidar,15 Sixth International' should be 'In: 2015 Sixth International Green and Sustainable Computing Conference'.","section":"References"},{"comment":"The text says min-max normalization 'yielded values in the interval (0,1)', but the minimum and maximum values can be attained, so the interval should be [0,1].","section":"Section 3.2"},{"comment":"The column headers 'Energy (%)' and 'Runtime (%)' are ambiguous; the caption should state explicitly that the energy values are reductions (positive percentages) and the runtime values are increases.","section":"Table 2"},{"comment":"The phrase 'dynamic power-cap adjustments' suggests that power caps were changed during execution, but the experiments apply a static cap for the whole run; consider rewording to 'potential dynamic adjustments' or describing the planned implementation more precisely.","section":"Abstract and Section 1"}],"recommendation":"reject","confidential_remarks":"The manuscript is a reasonable workshop-level empirical study with interesting measurement infrastructure, but the central quantitative claim is invalid as presented and the main conclusion is unsupported. I would be open to a substantially revised version that corrects the aggregation (weighting by task energy), adds validation of the selected power caps, and either adds an application-wide comparison or explicitly limits the claims to in-sample observations."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe useful part of this paper is the measurements: per-GPU-kernel energy and runtime on an NVIDIA GH200 across nine power caps for LSMS, a real exascale code. The per-kernel characterization—compute-bound vs memory-bound kernels reacting differently, and GPU idle phases favoring low caps—looks plausible and is worth knowing. The two metrics (speedup-energy-delay and Euclidean distance of normalized energy/runtime) are standard; the adaptation to per-kernel power-cap selection on GH200 is a legitimate small step.\n\nBut do not repeat the headline numbers. The ~151% and ~200% 'energy savings' come from summing per-task percentage reductions in Table 2. That is arithmetically wrong: each percentage is relative to a different baseline, they are not commensurable, and savings cannot exceed 100%. The paper's own caveat that these are 'simple aggregations... with no overhead' does not save it; the error is in the weighting, not the overhead. Using Table 1 energies as weights gives roughly 11% savings at about 6% runtime increase for the speedup-energy-delay picks. That is a modest result, not a dramatic one. The same problem affects the conclusion that kernel-level capping beats application-wide tuning.\n\nThere are two more weaknesses. The optimal power caps are selected on the same data used to evaluate them, so the savings are an in-sample fit; no held-out runs or error bars beyond three-run averages. And there is no code or data, which makes the numbers hard to check.\n\nNone of this kills the empirical contribution. The data and the qualitative findings are real, and the methodology is easy to fix: recompute the aggregation with energy weighting, add variability, and present the savings as percentages of total profiled energy. If the authors do that, the paper becomes a modest, usable energy-tuning case study.\n\nI would give this a serious referee: the dataset is genuinely new, the flaw is a defined arithmetic error rather than a suspicious measurement, and the fix is straightforward. A reviewer could send it back for major revision with a clear demand for corrected aggregation. It is not a desk reject.","headline":"Genuinely useful GH200 per-kernel power data, but the 151–200% savings headline is an arithmetic artifact of summing per-task percentages.","tokens_in":10503,"tokens_out":2583,"would_cite":false,"duration_ms":21849,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Per-GPU-kernel power caps chosen by energy-runtime metrics can cut a scientific application's energy use more sharply than one chip-wide cap.","keywords":["power capping","energy efficiency","HPC","GH200","performance metrics","automatic power steering","exascale applications","LSMS"],"falsifier":"A reader can falsify the headline totals by re-aggregating the paper's own Table 2 values: weight each task's energy reduction by its share of total baseline energy and each runtime increase by its share of baseline runtime. That calculation gives roughly 11% energy savings at about 6% runtime increase, far below the reported 151%/90% and 200%/203%; an experiment that applies the proposed per-task cap schedule on the same GH200 node and records superchip-level energy would show which aggregation matches reality.","tokens_in":9454,"feed_emoji":"⚡","tokens_out":14009,"duration_ms":135334,"temperature":0.7,"pith_summary":"This paper tries to show that the way a power cap is chosen matters as much as the cap itself. Running the LSMS materials-science code on a GH200 superchip, the authors measure energy and runtime separately for each GPU kernel and for the phases when the GPU sits idle. They then use two decision rules—the speedup-energy-delay product and the Euclidean distance between normalized energy and runtime—to pick the best chip-level power limit for each task. Their central claim is that applying these per-task caps would save considerably more energy than a single application-wide cap, because compute-bound kernels need high power while memory-bound kernels and idle phases can be capped low at little runtime cost. They report roughly 151% energy savings at a 90% runtime increase for one metric and roughly 200% energy reduction at a 203% runtime increase for the other.","feed_headline":"Capping each GPU kernel's power beats one chip-wide cap","feed_subtitle":"Two metric rules pick an energy-saving power cap per GPU task, saving more than a single chip-wide cap.","key_machinery":"The load-bearing machinery is a per-task scoring rule evaluated across a grid of power-cap settings. For each GPU task and each cap level, the paper computes either the speedup-energy-delay ratio $r_i = \\frac{\\mathrm{runtime}_1 \\cdot \\mathrm{energy}_1}{\\mathrm{runtime}_i \\cdot \\mathrm{energy}_i}$ (higher is better) or the Euclidean distance $d_i = \\sqrt{n_{\\mathrm{energy},i}^2 + n_{\\mathrm{runtime},i}^2}$, where each $n$ term is the min-max normalized energy or runtime of that task across all caps (lower is better). The recommended cap is the one that optimizes the chosen score, and the GH200's automatic power steering reallocates unused power between CPU and GPU, letting the chip-level cap act as a task-specific constraint.","core_discovery":"The paper's central finding is that power-capping decisions should be made per GPU task, not once for the whole application. Profiling the LSMS run on a GH200 node, the authors separate the dominant compute-bound dense matrix-multiply kernel, a memory-bound matrix-construction kernel, several LU-factorization and triangular-solve kernels, and the GPU compute idle phases between self-consistent-field iterations. At nine chip-level power caps from 200 W to the 1000 W default, the speedup-energy-delay rule peaks at 300–600 W for most tasks and at 900 W for the largest dense matrix-multiply kernel, while the Euclidean-distance rule settles most tasks at 400 W and the memory-bound kernel at 300 W. The authors conclude that kernel-level capping—for example, holding the GPU idle phase at 200–300 W—yields greater energy savings than application-wide tuning, with the exact trade-off depending on which metric the user trusts.","pith_inferences":["A correct energy-weighted aggregation of the paper's own per-task numbers yields roughly 11% energy savings at about 6% runtime increase, so the reported 151% and 200% figures are best read as indicators of per-task headroom rather than physical savings.","The two metrics' divergent recommendations imply a natural 'policy dial': speedup-energy-delay favors preserving runtime, while Euclidean distance favors deeper energy cuts, and a production system could expose this choice per job.","The same offline analysis could be turned into an online controller that computes the Euclidean distance from rolling energy and runtime samples and adjusts the chip-level cap during kernel execution.","The approach should transfer to other CPU-GPU superchips and to idle periods caused by communication, where capping the accelerator during host-side work is nearly free."],"forward_implications":["Applying the metric-selected caps task by task would give greater energy savings than any single application-wide cap, with the exact gain depending on the chosen metric.","GPU compute idle phases are the cheapest energy to reclaim: the measurements show roughly 40–47% energy reduction at 1–9% runtime increase.","Memory-bound kernels tolerate low caps well, saving about 23% energy for an 11% runtime increase, while compute-bound dense matrix-multiply kernels save little energy but keep runtime essentially unchanged.","The method needs only per-task energy and runtime measurements, so the same metric-based cap selection can be applied to other GPU-accelerated HPC applications without changing their source code."],"supporting_citations":[{"why":"Defines the speedup-energy-delay metric that scores each power-cap setting.","marker":"[13]"},{"why":"Supplies the Euclidean-distance-of-normalized-energy/runtime method adapted here to GPU tasks.","marker":"[15]"},{"why":"Provides the Global Criterion multi-objective justification for the Euclidean-distance decision rule.","marker":"[14]"},{"why":"Documents the GH200 power-monitoring interface used to isolate GPU power by subtracting CPU power from superchip power.","marker":"[4]"},{"why":"Supplies the run-time measurement infrastructure that records per-task energy and runtime.","marker":"[10]"},{"why":"Adds the custom power-measurement component that reads superchip and CPU power on the GH200.","marker":"[9]"},{"why":"Establishes power capping with composite energy-delay metrics for scientific applications, the baseline this work extends.","marker":"[8]"},{"why":"Shows how GPU power capping trades energy against performance under DVFS on modern GPUs.","marker":"[11]"},{"why":"Describes the LSMS code and its GPU-accelerated kernels that are profiled here.","marker":"[6]"}],"fun_headline_variants":["Per-kernel power caps beat a single chip-wide cap","Task-wise power capping beats a single cap for HPC","For HPC, per-GPU-task power caps save more energy","Kernel-level power caps beat app-wide cap on GH200"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that per-GPU-task percentage changes in energy and runtime can be summed to estimate whole-application savings, and that switching each task to its optimal cap costs no additional time or energy.","fun_headline_variants_meta":{"raw":{"variants":["Per-kernel power caps beat a single chip-wide cap","Task-wise power capping beats a single cap for HPC","For HPC, per-GPU-task power caps save more energy","Kernel-level power caps beat app-wide cap on GH200"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000488,"raw_usage":{"total_tokens":2381,"prompt_tokens":902,"completion_tokens":1479,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":518,"completion_tokens_details":{"reasoning_tokens":1408}},"tokens_in":518,"tokens_out":1479,"duration_ms":12283,"temperature":1.0,"reasoning_tokens":1408,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T13:24:15.348489+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A reader can falsify the headline totals by re-aggregating the paper's own Table 2 values: weight each task's energy reduction by its share of total baseline energy and each runtime increase by its share of baseline runtime. That calculation gives roughly 11% energy savings at about 6% runtime increase, far below the reported 151%/90% and 200%/203%; an experiment that applies the proposed per-task cap schedule on the same GH200 node and records superchip-level energy would show which aggregation matches reality.","supporting_citations":[{"cited_title":"https://developer.nvidia.com/blog/energy-efficiency- in-high-performance-computing-balancing-speed-and-sustainability/ (2023), ac- cessed: 2024-10-23","cited_arxiv_id":null,"evidence_quote":"Defines the speedup-energy-delay metric that scores each power-cap setting."},{"cited_title":"In: 2022 IEEE 13th Interna- tional Green and Sustainable Computing Conference (IGSC)","cited_arxiv_id":null,"evidence_quote":"Supplies the Euclidean-distance-of-normalized-energy/runtime method adapted here to GPU tasks."},{"cited_title":"https://docs.nvidia.com/grace-perf-tuning-guide/power-thermals.html (2024), ac- cessed: 2025-02-28","cited_arxiv_id":null,"evidence_quote":"Documents the GH200 power-monitoring interface used to isolate GPU power by subtracting CPU power from superchip power."},{"cited_title":"In: Brunst, H., Müller, M.S., Nagel, W.E., Resch, M.M","cited_arxiv_id":null,"evidence_quote":"Supplies the run-time measurement infrastructure that records per-task energy and runtime."},{"cited_title":"Concurrency and Computation: Practice and Experience31(6), e4485 (2019)","cited_arxiv_id":null,"evidence_quote":"Establishes power capping with composite energy-delay metrics for scientific applications, the baseline this work extends."},{"cited_title":"In: Wyrzykowski, R., Deelman, E., Dongarra, J., Karczewski, K","cited_arxiv_id":null,"evidence_quote":"Shows how GPU power capping trades energy against performance under DVFS on modern GPUs."},{"cited_title":"Computer Physics Commu- nications211, 2–7 (2017), high Performance Computing for Advanced Modeling and Simulation of Materials","cited_arxiv_id":null,"evidence_quote":"Describes the LSMS code and its GPU-accelerated kernels that are profiled here."}],"review_version":1}