{"id":"3f87b5ae-5736-4b6a-b09a-a8fcda4cc9d1","arxiv_id":"2611.17206","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"PACER uses offline-profiled frequency caps and a digital power estimator to reduce GPU power excursions, improving LLM inference performance by up to 12%.","lead":"This paper characterizes power excursions (EDP_p) in modern GPUs and shows they are especially frequent and large in large language model workloads. It introduces hardware and firmware mechanisms to manage these excursions, improving inference performance by 3-12% on AMD Instinct MI300X GPUs.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek/deepseek-v4-flash","headline":"The offline-derived p90 frequency caps are never validated against online per-kernel effective-frequency distributions, even though Section 3.3 makes effective frequency context-dependent; this transfer is the least-secure link in the 3–12% claim.","rationale":"Good-faith read: the end-to-end measurements are clean — a fixed-seed request stream identical for baseline and PACER, on production MI300X hardware, with major limitations stated explicitly (b98/b99 for power granularity, b319 for diminishing returns, b354 for profiling cost). The offline results (Figure 10) support the 3–4% improvement range and the near-oracle gap, and Table 4's per-bucket gains are uniformly nonnegative. The concern raised here is therefore not about whether the measured improvements occurred, but about whether the headline range transfers: the mechanism-level link between Stage 1's offline p90 caps and the online conditions is supported only by aggregate outcomes. The paper's own Section 3.3 makes effective frequency context-dependent, so that link is structurally unsecured, and the gap between online p50 gains and offline per-bucket gains suggests load-dependent amplification that is neither decomposed nor characterized. A stronger objection is not warranted because nothing contradicts the measured end-to-end A/B results, and the flatness of Figure 12 plus the nonnegative Table 4 cells indicate the heuristic is not knife-edged. The reader's weakest assumption (offline profiling representativeness) is in the same territory; this version differs by pinning the failure mode to the Section 3.3 context-dependence and by adding the online-versus-offline gain discrepancy and the missing online EDP_p evidence as concrete symptoms. The recommendation stays UNCHANGED: the reader's CONDITIONAL verdict already conditions on broader evaluation, and this analysis reinforces condition (1) rather than changing the verdict. The absence of a released artifact additionally limits independent verification, but that is a reproducibility limitation, not a soundness threat.","tokens_in":33987,"tokens_out":24981,"duration_ms":219296,"concrete_test":"Rerun the online deployment (LLaMA3.1-405B, TP=8, 0.2 QPS, same fixed-seed LongBench stream) with Stage 1's tracing enabled during the uncapped baseline, and recompute Eq. 1 per bucket from the in-situ per-kernel effective-frequency distributions, comparing the results to Table 4. Then A/B serve with the offline caps versus the in-situ recomputed caps. If any recomputed cap differs from Table 4 by more than ~150 MHz, if serving with the in-situ caps improves p50 TTFT by more than 3 percentage points over the Table 4 caps, or if PACER's online runs still show >1.3×TDP events at rates comparable to the baseline, the offline-profiling transfer fails and the 3–12% claim is not robust to the online context change.","verdict_should_be":"UNCHANGED","load_bearing_attack":"PACER's central claim is a transfer claim: caps chosen by offline profiling (Section 4.2.1, Eq. 1) must remain near-optimal when deployed online. The weakest link is that the per-kernel p90 effective frequencies are measured in the uncapped baseline's specific DPM and power state, while Section 3.3 asserts that \"the same GEMM computations may experience different power fluctuations and EDP_p, depending on what kernel precedes it, and how it interacts with the DPM\" — effective frequency is context-dependent. Online deployment changes exactly those conditions: TP=8 introduces RCCL kernels into the profiled composition (Figure 13), Poisson load sustains back-to-back prefill near TDP, the EWMA of Eq. 2 mixes buckets, and chunked prefill inserts unprofiled boundaries. The paper validates the caps only through aggregate TTFT (Figure 14); it reports no online per-kernel frequency distributions, no online EDP_p counts under the caps, and no cap-sensitivity sweep. Two observed gaps make the concern concrete: online p50 gains (9–12% for LLaMA3.1-405B) run above the dominant bucket's offline gain (7.9% for 8k–16k tokens in Table 4) at every load, and the \"within 3% of an oracle\" bound is established only offline (Figure 10), not where the 12% headline figure arises. The acknowledged mitigations — b354 (one-time profiling cost), the uniformly nonnegative per-bucket gains in Table 4, and the flat kernel-duration curves in Figure 12 — bound the cost of modest mis-calibration, but they do not establish that the p90 statistic itself survives the context change.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek/deepseek-v4-flash","summary":"The paper presents PACER, a system for managing GPU power excursions (EDP_p) in ML workloads, particularly LLM inference. It first characterizes EDP_p across diverse workloads, finding that ML workloads exhibit frequent, high-magnitude power overshoots. Then it proposes two management mechanisms: (1) a hardware digital power estimator (DPE) with boost management that clamps power at configurable thresholds, and (2) a firmware frequency capping solution (PACER) that uses offline profiling to determine per-bucket frequency caps and memory clock states, deployed online via a lookup table. Evaluation on AMD Instinct MI300X GPUs shows 3-12% TTFT improvement over baseline power management, within 3% of an oracle, for dense LLMs like LLaMA3.1-405B, with smaller gains for MoE models like DeepSeek-V3.2.","tokens_in":34427,"tokens_out":1459,"duration_ms":49351,"significance":"If the results hold, this work provides a practical, deployable solution to a real problem in GPU-accelerated ML serving: power excursions causing performance variability and throttling. The characterization of EDP_p across workloads is valuable. The firmware-only approach (PACER) requires no hardware changes and is demonstrated on production hardware, making it potentially impactful for hyperscalers. The paper is well-structured and the claims are supported by experiments on a relevant platform.","major_comments":[{"comment":"The central transfer claim—that offline p90 frequency caps remain near-optimal online—is not directly validated. The paper reports aggregate TTFT improvements (Figure 14) but does not present online per-kernel effective-frequency distributions, online EDP_p counts under PACER, or a sensitivity sweep around the chosen caps. Without such evidence, the reader cannot assess whether the caps are actually optimal or merely non-harmful. The authors acknowledge that effective frequency is context-dependent (§3.3) and that diminishing returns occur for long inputs (§6.2.2, sentence b319), but these acknowledgments do not substitute for a direct validation of the transfer. I recommend adding either (a) a comparison of online per-kernel p90 frequencies against the offline-derived caps, or (b) a cap-sensitivity study showing that small deviations from the chosen cap do not yield better TTFT.","section":"§4.2.1, Eq. (1); §6.2.2, Table 4; §6.2.2, Figure 14","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The online evaluation is limited to a single request trace (LongBench, 500 requests) and a narrow QPS range (0.1–0.3). The authors justify the QPS bound by saturation analysis (§5.2), but the trace size is small and the workload diversity is limited. The paper would benefit from evaluation on additional traces (e.g., ShareGPT, Azure LLM traces) and at higher loads to demonstrate robustness. The current results may not generalize to production traffic patterns with different input-length distributions or burstiness.","section":"§6.2.2, Figure 14; §5.2","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The oracle comparison (Figure 10) is performed only for single-GPU offline inference, not for the multi-GPU online serving scenario where the headline 12% improvement is reported. The claim 'within 3% of an oracular solution' is therefore established only offline. The online oracle would need to account for multi-GPU synchronization, RCCL overhead, and dynamic request mixing. The authors should either provide an online oracle estimate or qualify the claim to reflect the offline-only validation.","section":"§4.2.1, Eq. (1); §6.2.1, Figure 10","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The selection of top-K kernels (K such that they constitute >50% of runtime) is heuristic and may miss kernels that are infrequent but critical for EDP_p. The paper does not report the sensitivity of the predicted cap to the choice of K. A robustness analysis (e.g., varying K from 3 to all kernels) would strengthen the methodology.","section":"§4.2.1, Eq. (1); §6.2.2, Table 4","authors_acknowledge":null,"authors_acknowledge_p":null}],"minor_comments":[{"comment":"The caption of Figure 1b refers to 'CU Frequency (1ms)' and 'CU Frequency (50ms)', but the legend in the figure uses 'CU Frequency (a)' and 'CU Frequency (b)'. Please align the legend with the caption.","section":"§1, Figure 1b","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The CDF in Figure 3 includes 'LLaMA2 fine-tuning' in the text but the legend lists only inference workloads. Clarify whether fine-tuning was included and, if so, report its configuration.","section":"§3.1, Figure 3","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The EWMA smoothing factor α is set to 0.3 for all experiments. The paper should discuss the sensitivity of results to this choice or justify why 0.3 is appropriate.","section":"§4.2.1, Eq. (2)","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The table lists 'TTFT SLO' for online models but does not define how T_mean is computed. Specify whether it is the mean TTFT over all requests in the trace or over a subset.","section":"§5.1, Table 3","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The y-axis label 'TTFT Improv (%)' is ambiguous. Clarify that it is percentage improvement over baseline (higher is better).","section":"§6.2.2, Figure 14","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The related work section mentions Kakolyris et al. [45] but does not discuss how PACER's approach differs in terms of objective (performance vs. energy) and phase (prefill vs. decode). A brief comparison would help position the contribution.","section":"§7","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The OCP specification table lists '1×TDP Indefinitely SW'. It would be helpful to note that this is a steady-state limit, not a peak.","section":"§2, Table 1a","authors_acknowledge":null,"authors_acknowledge_p":null}],"recommendation":"major_revision"},"author_rebuttal":null,"desk_editor":{"model":"deepseek/deepseek-v4-flash","letter":"This paper is worth reading if you care about GPU power management for LLM serving, but the headline 3–12% improvement is narrower than it sounds. The main contribution is a systematic characterization of EDP_p (power excursions above TDP) across ML, HPC, and graph workloads on AMD MI300X, plus a two-stage firmware solution (PACER) that caps GPU frequency based on offline profiling of per-kernel p90 effective frequencies.\n\nWhat is actually new: the characterization is the first I have seen that quantifies how often and how high these excursions go for modern LLM inference, and the observation that low-magnitude EDP_p (up to ~1.3×TDP) is beneficial for computational sprinting while higher excursions trigger throttling that hurts performance. The PACER algorithm itself is a straightforward application of frequency capping, but the offline profiling to pick per-bucket caps based on kernel composition is a practical engineering contribution. The hardware DPE for sub-microsecond power estimation is also a real piece of work, though it is evaluated only indirectly.\n\nWhat the paper does well: the experimental methodology is solid—three trials, comparisons against oracle and realistic baselines, explicit discussion of the 1 ms power measurement limitation, and a clear breakdown of why MoE models (DeepSeek) benefit less. The authors are honest about the offline profiling cost being a one-time hyperscaler expense. The kernel composition analysis in Figure 13 and the bell-curve behavior of TTFT improvement with input length are genuinely informative.\n\nSoft spots: the stress-test concern about the transfer from offline p90 frequencies to online deployment is real and not fully addressed. The paper validates caps only through aggregate TTFT; it does not report online per-kernel frequency distributions or EDP_p counts under the caps. The \"within 3% of an oracle\" claim is established offline (Figure 10), not in the online setting where the 12% headline comes from. The online evaluation is limited to three models, one QPS bound (0.3), and no code release. These are acknowledged limitations, but they mean the central transfer claim is load-bearing and not stress-tested. The DPE estimation error (up to 12% NRMSE) is noted but its impact on the clamping threshold is not explored.\n\nWho this is for: systems researchers working on GPU power management, especially at hyperscalers. The characterization alone is a useful reference. The PACER algorithm is practical but not theoretically deep.\n\nRecommendation: this deserves peer review. The characterization is reproducible in principle (though no artifacts), the methodology is sound, and the limitations are stated. A good referee would ask for online per-kernel validation and a sensitivity analysis on the profiling assumptions, but the paper is not fatally flawed.","headline":"Solid characterization of GPU power excursions with a practical frequency-capping solution; the 3-12% improvement is real but the offline-to-online transfer is the weakest link.","tokens_in":35000,"tokens_out":661,"duration_ms":9688},"pith_extraction":null,"created_at":"2026-11-13T09:19:16.925761+00:00","model_set":{"reader":"deepseek/deepseek-v4-flash"},"pipeline_version":"pith-pipeline@v1.4.1-daily-deepseek-or-medium","input_kind":null,"input_chars":null,"input_total_chars":null,"passes_missing":null,"published_standing":null,"published_confidence":null,"objection_audit":null,"run_record":null,"basis":{"kind":"not_recorded","source":null,"chars_given":null,"chars_total":null},"receipt":null,"falsifier":null,"supporting_citations":[],"lane_accuracy":{"status":"not yet measured","lane":{"model":"deepseek/deepseek-v4-flash","input_kind":null},"url":"https://pith.science/accuracy"},"review_version":1}