{"id":"698d3fd6-b7ee-46a7-956d-f9887b8cb0e2","arxiv_id":"2505.14796","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A year of Polaris telemetry shows GPU power is overprovisioned, with jobs averaging only 400W per node and half of that going to idle power.","lead":"This paper analyzes a year of logs and telemetry from Argonne's Polaris supercomputer to see how its GPUs use power. It finds that GPUs mostly sit idle or run at low power, so big energy savings could come from reducing idle power and managing power at the job level rather than per GPU.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Observation 1's 'half idle' bound rests on an unmeasured 50 W/GPU spec value, and the paper never documents the power sensor behind the 400 W average—either gap can break the central claim.","rationale":"After reading the paper carefully, the strongest practical claim is Observation 1, and the paper's 'opportunities' section depends on it. The reader's weakest assumption correctly identifies that the DCGM utilization metric is a coarse 'any kernel active' flag and that 5-second sampling can miss short power events; I agree that is a limitation. However, the paper explicitly defines GPU_load as NVML's 'percent of time any kernel executing,' so the utilization observations are internally coherent as statements about kernel-active time. The more serious gap is that the power side of the central claim is never grounded: the sampled-metrics table omits power entirely, and the idle baseline is taken from a vendor specification rather than from the telemetry. This means the two numbers that make Observation 1 actionable—400 W total and 200 W idle—cannot be checked or reproduced. That is a correctness risk at the center of the paper, not a demonstrated falsehood. It is addressable by documentation plus a controlled measurement, so it supports a conditional verdict rather than rejection. I would keep the reader's CONDITIONAL verdict; the condition is that the authors must identify the power sensor and validate the idle baseline. My concern overlaps with the reader's but is not identical, so agreement is partial.","tokens_in":14223,"tokens_out":6808,"duration_ms":59223,"concrete_test":"Inspect the released GitHub dataset and code (promised in Section 3 but not yet linked) and locate the exact source of total_power_mean. Then run a controlled idle benchmark on a Polaris node: with no user job active, record DCGM power per GPU for at least one hour and compare the observed idle floor to the assumed 50 W/GPU. Recompute the idle fraction of Observation 1 using the measured idle power and the documented power metric. If the measured idle floor is below 50 W/GPU, or if the power metric is not per-GPU DCGM power, Observation 1 must be revised.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Observation 1 makes two quantitative assertions: average node GPU power is ~400 W, and at least 200 W is idle. The second assertion is not a measured result. Section 2.1 gives 'idles at just over 50 W' as a static spec, and no later section reports measured idle phases from the telemetry. If production idle power (with driver, memory-clock, and P-state behavior) is below 50 W/GPU, the 'at least 200 W' lower bound is false; if it is above, the number is accidentally conservative but still unsupported. The first assertion is equally under-documented. Section 2.2 says the study considers three telemetry metrics—GPU_load, GPU_mem_util, GPU_mem_alloc (Table 3)—none of which is power, yet Table 4 defines total_power_mean and total_energy and Section 4 plots mean power. The paper does not state whether the 400 W figure comes from DCGM per-GPU power, node PDU power, or another source, nor how GPU-only power was isolated. Without the power metric schema and a measured idle baseline, the headline insight and the P-state and provisioning recommendations are not verifiable. The reader's concern about 5-second NVML utilization is real for the utilization claims, but it is secondary: even perfect utilization metrics would not validate the idle-power split unless the power data themselves are identified and the idle baseline measured.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper presents a data co-analysis pipeline for one year of Polaris supercomputer PBS job logs and GPU telemetry (DCGM-sampled every five seconds). The pipeline preprocesses 3.2 billion telemetry rows into per-job summaries, reducing data volume by 94%, and computes resource-imbalance (RI) coefficients for GPU utilization, memory utilization, and memory allocation. On the basis of these summaries, the authors report that node-level GPU power averages about 400 W with at least 200 W idle (Observation 1), that GPU utilization is low (75% of jobs use GPUs for at most half their runtime), that memory utilization is even lower, and that memory allocation is weakly correlated with power. They propose job-level power management, software P-state control, and memory-aware optimization as practical opportunities.","tokens_in":14601,"tokens_out":4083,"duration_ms":34920,"significance":"If substantiated, Observation 1 identifies a concrete and sizable source of avoidable energy on a production leadership-class system, and the proposed job-level power-management approach is plausible given the reported spatial consistency of utilization across nodes within jobs. The paper's strengths are its descriptive and reproducible design: no parameters are fitted, the RI methodology is adopted from prior work rather than invented for this study, and the authors commit to releasing both the per-job summary data and the ~500-line analysis tool, which would allow independent verification of the pipeline. The main limitation is that the headline power claims rest on metric provenance and an idle-power baseline that are not documented in the manuscript.","major_comments":[{"comment":"The paper defines total_power_mean and total_energy in Table 4 and plots mean node GPU power in Figure 4, but it never states the source of the power values, such as whether they come from DCGM per-GPU power readings, node-level PDU measurements, or HPCM-collected fields, nor how a GPU-only component is extracted from node-level power. Because Observation 1's 400 W average is the central empirical claim, this provenance gap must be closed by stating the exact field name, measurement chain, sampling interval, and any calibration, and by documenting the missing power metric in Table 3.","section":"§2.2, Tables 3–4"},{"comment":"The 'at least 200 W idle' part of Observation 1 is not a measured result: Section 2.1 gives 'idles at just over 50 W' as a static specification, and no later section reports measured idle power from telemetry or from a dedicated idle-phase experiment. The direction of the lower bound depends on an unverified assumption about production idle behavior. The authors should report the measured idle-power distribution, for example the lowest decile of per-GPU power during job runs or a dedicated idle-node measurement, and state the uncertainty around the 50 W/GPU value.","section":"§2.1 and §4"},{"comment":"The five-second NVML GPU_load definition, which measures the percent of time any kernel is executing, cannot distinguish a GPU that is fully occupied by a low-occupancy kernel from one running near peak throughput, and power spikes shorter than the sample interval are not captured. This does not by itself invalidate the idle-power argument, but it weakens Observations 3 and 5 and the correlation-based power discussions; a robustness check against a higher-frequency or power-based activity measure is needed.","section":"§2.2, §5.1"},{"comment":"All quantitative observations, such as '75% of jobs', '90% of jobs', and 'at least 200 W', are reported as exact statements without uncertainty or sensitivity bounds. Since the underlying telemetry has known sampling and definitional noise, the authors should provide error bars, bootstrap intervals, or a sensitivity analysis over the RI thresholds and the >2% GPU-count threshold used to define gpu_count.","section":"§3.2, §4–§7"}],"minor_comments":[{"comment":"Typo: 'disperate' should be 'disparate', and 'datesets' in §2.2 should be 'datasets'.","section":"§3.1"},{"comment":"Typo: 'max TPD' should be 'max TDP'.","section":"§4"},{"comment":"The RI category boundaries [0, 0.2], (0.2, 0.6], and (0.6, 1] are presented without a citation or justification; since these thresholds directly determine the percentages in Observations 4, 6, and 9, the source of the thresholds should be stated explicitly.","section":"§3.2"},{"comment":"The GitHub link is given as '[link]'; the reproducibility claim in the abstract cannot be evaluated until the actual repository and dataset DOIs are provided.","section":"§3"},{"comment":"The units for total_energy are given as 'Kilojoules' with abbreviation 'KJ'; use consistent SI notation, i.e., 'kJ'.","section":"Table 4"},{"comment":"The CDFs would be more informative with the number of jobs per class annotated in the figure or in an accompanying table, given the large differences in class sizes.","section":"Figures 6 and 9"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a straightforward observational workload study. I did not find evidence of circularity: the RI thresholds are adopted from prior work and no parameters are fitted. My main concern is that the headline idle-power figure is built from a specification-sheet value and an undocumented power metric; both are fixable in revision. I would also ask the editor to require that the GitHub dataset link be resolved before final acceptance, since the paper's reproducibility claim depends on it."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a solid descriptive case study of one machine, and the pipeline is genuinely useful, but the central \"half of GPU power is idle\" claim is not actually measured — it is a calculation from a datasheet. The authors need to give us the power metric schema and a measured idle baseline, plus the promised code/data, before the headline result can be checked.\n\nWhat's new: the Polaris-specific numbers (mean node GPU power around 400 W, 75% of jobs at half or less GPU utilization, ~90% of jobs with low memory utilization) and the full co-analysis workflow on a year of DCGM + PBS data. The data processing is described carefully: 3.2 billion telemetry rows down to a per-job summary, compression, checkpointing, and RI coefficients adopted from prior work without introducing new free parameters. That part is reproducible in principle, and the paper is readable.\n\nThe soft spots are real, and one is load-bearing. Observation 1 says \"at least 200 W of the 400 W average is idle power.\" But on my reading the 50 W idle figure is a static spec from Section 2.1, and the paper never documents a measured idle phase, nor does it state which physical sensor produced the 400 W average (DCGM per-GPU? node PDU? something else?). Table 3 lists only GPU_load, GPU_mem_util, and GPU_mem_alloc as sampled metrics; total_power_mean appears in Table 4 but its source isn't specified. Without that, the idle-power split is an unverifiable inference, and it is the hook for the P-state and provisioning recommendations. The NVML utilization caveat the reader raises is legitimate but secondary — even perfect utilization data wouldn't fix the power attribution. The 94% condensation claim is also not validated; \"preserving essential insights\" is asserted, not shown. And the code/data are promised, not released.\n\nNone of this is fatal. The RI thresholds are adopted from prior work, there's no fitted-parameter circularity, and the descriptive observations are consistent with the stated methods. It's just a case where the most interesting number needs a better audit trail.\n\nHPC operations people and anyone doing GPU telemetry co-analysis will get something from this, and it deserves a serious referee once the artifacts and power metric documentation are in. I'd send it out, but with a clear request for the measured idle baseline and the metric source.","headline":"Solid case study with a useful pipeline, but the headline idle-power claim is a datasheet calculation, not a measurement — and the paper needs the power metric source and artifacts before that claim can be trusted.","tokens_in":15018,"tokens_out":2083,"would_cite":false,"duration_ms":18804,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A year of Polaris telemetry shows that typical GPU power draw is dominated by idle power, and most jobs barely use their GPUs.","keywords":["system logs","power efficiency","GPU usage","workload characteristics","GPU telemetry","idle power","job-level power management","resource imbalance"],"falsifier":"Take a representative sample of jobs on the same hardware and log GPU power both through the standard five-second DCGM path and through a high-frequency power monitor, for example 1 ms node PDU sampling. If the high-frequency trace shows frequent power excursions above the reported ~400 W per node, or if jobs with near-100% reported GPU utilization show long low-power periods, the paper's central picture of idle-dominated, underutilized GPU power would be contradicted.","tokens_in":14019,"feed_emoji":"⚡","tokens_out":7463,"duration_ms":63681,"temperature":0.7,"pith_summary":"The paper sets out to show that a year's worth of job scheduler logs and GPU telemetry from a production supercomputer can be co-analyzed into a compact dataset that reveals where GPU power actually goes, and that on the Polaris system the answer is mostly idle power. It reports that typical jobs draw about 400 W of node-wide GPU power, equal to the thermal design power of one GPU, with at least 200 W going simply to keep the four GPUs powered on. It further reports that about three-quarters of jobs use their GPUs for half or less of their runtime and that nearly all jobs access GPU memory only briefly. From these patterns it argues that the most practical savings would come from reducing idle power, applying power management at the job level, and optimizing active memory behavior rather than memory allocation. The contribution is a reproducible pipeline that reduces data volume by 94% while preserving these system-level insights.","feed_headline":"Half of Polaris GPU power is idle, log analysis finds","feed_subtitle":"Jobs draw only ~400 W per node and use GPUs less than half the time; P-state controls could save much of that.","key_machinery":"The central machinery is a data co-analysis pipeline that aligns PBS scheduler job records with five-second GPU telemetry samples by timestamp and then aggregates the joined data into per-job summaries. On top of that it computes resource imbalance (RI) coefficients, where temporal RI measures how much a resource fluctuates within a node over a job and spatial RI measures how consistent usage is across nodes, for GPU utilization, memory utilization, and memory allocation. These RI coefficients are what make the job-level power-management recommendation possible: they show spatial consistency that justifies sampling one node and applying one policy across the job. Pearson correlations among derived metrics then link power to utilization and memory access rather than to allocation.","core_discovery":"On its own terms, the paper's central claim is that a year of production telemetry from Polaris reveals a system-wide GPU power problem with a clear shape: node-level GPU power averages about 400 W, one GPU's TDP, and at least 200 W of that is fixed idle draw from keeping four A100 GPUs powered on. GPU utilization is low, with about 75% of jobs using their GPUs for half or less of the runtime; GPU memory utilization is lower still, with nearly 90% of jobs accessing memory less than a quarter of the time; and GPU memory allocation correlates almost not at all with power. Because temporal variability lives inside nodes while spatial behavior is consistent across a job's nodes, the paper argues that power management can be applied at the job level and sampled from one node, and that the largest wins would come from reducing idle power, for example by giving A100s software-controlled P-states, and from optimizing active memory access rather than memory capacity.","pith_inferences":["A testable extension would be to re-run the pipeline with sub-second power sampling on a sample of jobs; if short power spikes are frequent, the reported 400 W average may understate true peak demand and change provisioning advice.","Because GPU utilization as defined here counts any executing kernel, a job that keeps one low-occupancy kernel resident will look 100% utilized while drawing little power, so the paper's underutilization conclusions may actually understate how little work typical jobs do, and high utilization readings should be checked against power.","The spatial-consistency result implies a per-node sampling architecture, but bursty temporal behavior suggests power-management state transitions must be fast relative to job phases; profiling phase lengths at finer time scales would tell whether P-state transitions pay off.","The same co-analysis could be applied to other A100-based systems to see whether the idle-power dominance is a property of the GPU generation or of Polaris's scheduling and workload mix."],"forward_implications":["If the 200 W of idle power per node is real, enabling software-controlled P-states on A100-class GPUs could remove a large share of Polaris's GPU power draw without touching active workloads.","Because spatial RI is low for utilization and memory, power-saving strategies can be applied uniformly at the job level and monitored from a single node, which keeps implementation cost low.","Since nearly 90% of jobs touch GPU memory less than a quarter of the time, memory clock gating or right-sizing GPU memory capacity offers a low-risk power optimization.","GPU memory allocation is nearly uncorrelated with power, so capacity planning and power optimization can proceed independently, and utilization-aware strategies are the lever.","The released open-source analysis tool and condensed dataset make the same 94%-compression workflow directly transferable to other supercomputers."],"supporting_citations":[{"why":"Defines the NVML metrics (GPU utilization, memory utilization, allocation) that every observation in the paper is built from.","marker":"[24]"},{"why":"Establishes Polaris's TOP500 context and the system and GPU specifications used for TDP and idle-power baselines.","marker":"[33]"},{"why":"Introduces the resource-imbalance (RI) coefficient that the paper adapts to measure temporal and spatial consistency.","marker":"[28]"},{"why":"Extends the RI method to a production HPC system and provides the utilization-analysis baseline this study builds on.","marker":"[19]"},{"why":"Provides the production-workload power analysis that this paper contrasts with its system-log-driven approach.","marker":"[37]"},{"why":"Documents software-driven power management techniques (DVFS and power capping) that the paper proposes applying at scale.","marker":"[16]"},{"why":"Supplies the memory-access power cost result used to explain why memory utilization correlates with instantaneous power.","marker":"[3]"},{"why":"Defines GPU P-states, the mechanism the paper identifies as missing on A100s and as the main idle-power reduction lever.","marker":"[23]"}],"fun_headline_variants":["Polaris GPUs idle half the time, wasting 200W per node","Telemetry shows how to cut GPU power waste on Polaris","Job-level power strategies could reduce Polaris energy use","Argonne's Polaris wastes half its GPU power on idle","Polaris GPU utilization low, power controls can help"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central observations depend on the premise that five-second DCGM/NVML metrics, especially GPU utilization defined as the percent of time any kernel is executing, faithfully capture the activity that actually drives GPU power; a GPU that is always busy but at low occupancy will read as 100% utilized while drawing far less than peak power, and power spikes shorter than the sampling window are invisible.","fun_headline_variants_meta":{"raw":{"variants":["Polaris GPUs idle half the time, wasting 200W per node","Telemetry shows how to cut GPU power waste on Polaris","Job-level power strategies could reduce Polaris energy use","Argonne's Polaris wastes half its GPU power on idle","Polaris GPU utilization low, power controls can help"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000324,"raw_usage":{"total_tokens":1790,"prompt_tokens":890,"completion_tokens":900,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":506,"completion_tokens_details":{"reasoning_tokens":812}},"tokens_in":506,"tokens_out":900,"duration_ms":15581,"temperature":1.0,"reasoning_tokens":812,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T15:28:42.551161+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a representative sample of jobs on the same hardware and log GPU power both through the standard five-second DCGM path and through a high-frequency power monitor, for example 1 ms node PDU sampling. If the high-frequency trace shows frequent power excursions above the reported ~400 W per node, or if jobs with near-100% reported GPU utilization show long low-power periods, the paper's central picture of idle-dominated, underutilized GPU power would be contradicted.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the NVML metrics (GPU utilization, memory utilization, allocation) that every observation in the paper is built from."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes Polaris's TOP500 context and the system and GPU specifications used for TDP and idle-power baselines."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the resource-imbalance (RI) coefficient that the paper adapts to measure temporal and spatial consistency."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Extends the RI method to a production HPC system and provides the utilization-analysis baseline this study builds on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the production-workload power analysis that this paper contrasts with its system-log-driven approach."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the memory-access power cost result used to explain why memory utilization correlates with instantaneous power."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines GPU P-states, the mechanism the paper identifies as missing on A100s and as the main idle-power reduction lever."}],"review_version":1}