{"id":"c28e0d17-f8a2-47ee-8e1d-732fa0e10ab7","arxiv_id":"2608.03880","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"MFU is a valid linear predictor of GPU power in compute-bound LLM training, with per-(GPU, dtype, batch) fits reaching 1% error, matching measurement noise.","lead":"This paper tests whether MFU, a software-calculated measure of how much of a GPU's peak compute an AI training run actually uses, can predict how much power the GPU draws. It finds a linear MFU-to-power relationship that holds across six different GPUs, as long as the training is compute-bound, and that splitting the fit by GPU, precision, and batch size cuts prediction error to about one percent.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"GPU power telemetry ground truth validated on only 2 of 6 GPUs; unvalidated sensors could bias all fitted slopes and noise-floor claims.","rationale":"The reader's weakest_assumption identifies precisely the most load-bearing concern: the power telemetry ground truth is validated for only two of six GPUs, yet all fits and error claims depend on it. This is more fundamental than the L4 exception or the malformed repository URL, because if the sensor is biased on any unvalidated device, the empirical basis for 'fits every GPU' and the noise-floor comparison collapses. The paper itself acknowledges the limitation only in passing (Appendix A validates two devices), and its citation of Yang et al. [19]—which documents NVIDIA sensor inaccuracies—signals that this is a known risk. The concrete external-validation test would directly settle whether the concern lands; until then, CONDITIONAL is the appropriate verdict, and no change to the reader's verdict is needed.","tokens_in":12056,"tokens_out":6009,"duration_ms":57294,"concrete_test":"Run an external wall-power meter (e.g., a calibrated AC power analyzer) on the MI210 and at least one unvalidated NVIDIA GPU (e.g., A100) for a representative subset of the sweep: 3 model families × 2 dtypes × 4 batch sizes (excluding batch 1), with 3 repeats each. Record GPU-reported and wall power simultaneously. If the GPU-reported vs wall-power relationship deviates from a stable linear fit (R²<0.99 or slope not constant across the power range) on any device, the ground-truth assumption fails for that device and the fitted MFU-power model and noise-floor comparison must be recalibrated. For the MI210, additionally check whether the 0.20% repeat SD is an artifact of ROCm's telemetry resolution or smoothing rather than true power stability.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that a linear MFU-based power model fits every GPU rests on treating NVML/ROCm-reported power as ground truth. Section II-C states this explicitly, and Appendix A validates it only for the L40 and Quadro RTX 5000 (R²>0.99 vs external wall power). The remaining devices—A100, L4, RTX 4070 Ti, and MI210—are never checked. If those sensors have a systematic offset or nonlinearity (NVIDIA sensor accuracy varies by model; AMD telemetry on MI210 shows an unusually quiet 0.20% cross-repeat SD vs ≥0.35% on NVIDIA), every fitted slope, MAPE, and the noise-floor comparison in Table III is built on an unvalidated signal. In particular, the MI210's low repeat noise could make the conditioned residual appear 'matching the measurement-noise floor' when the sensor itself is smoothing or quantizing power. The paper cites Yang et al. [19] on built-in sensor inaccuracy but does not apply its findings to its own devices. Without external validation on the other four GPUs, the claim that the ~1% MAPE is at the data limit is not established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper evaluates whether Model FLOPs Utilization (MFU), a software-defined analytical metric, can serve as a portable predictor of GPU power for LLM training simulation. The authors benchmark nearly 3000 single-device training runs across six GPUs (A100, L40, L4, Quadro RTX 5000, RTX 4070 Ti, MI210), three model families, three precisions, seven batch sizes, and two context lengths, with four repetitions per configuration. They report that per-GPU linear MFU-power fits achieve MAPE 3.5--13.7% and R² 0.37--0.84, and that conditioning on (GPU, dtype, batch size) reduces MAPE to about 1%, which they argue is at the cross-repeat measurement-noise floor on NVIDIA devices. They also document a memory-bound batch-1 regime where MFU under-predicts power, and they provide public code/data and an appendix validating internal telemetry against external wall power for two devices.","tokens_in":12295,"tokens_out":5617,"duration_ms":57912,"significance":"If the central claims hold, this is a useful result: MFU-based power modeling is directly integrable into simulators such as Vidur without vendor-specific hardware counters. The strengths of the paper are its controlled experimental protocol, the large number of runs, repeated measures, bootstrap confidence intervals, comparison against a noise floor, and public release of the benchmarking pipeline. The cross-vendor comparison (including the MI210 case where GPU Utilization is binary) is valuable. The paper is honest about several limitations: inference and decode are out of scope, distributed-training communication power is not modeled, and fused attention kernels are disabled. However, the central 'fits every GPU' and 'at the data limit' claims are currently supported only by on-board telemetry that is externally validated for two of the six GPUs, and the noise-floor comparison lacks uncertainty quantification. These issues are load-bearing and need to be addressed before the strong claims can be accepted.","major_comments":[{"comment":"The manuscript uses NVML/ROCm-reported power as the ground-truth signal for all six devices, but Appendix A validates this signal against external wall power only for the L40 and Quadro RTX 5000. The remaining four devices (A100, L4, RTX 4070 Ti, MI210) are never checked. This is not a minor omission: every fitted slope, MAPE, and the noise-floor comparison in Table III are built on this unvalidated signal. The MI210's cross-repeat SD of 0.20% (versus ≥0.35% on all NVIDIA devices) suggests possible sensor smoothing or quantization; if so, the 'residual matches the measurement noise' conclusion could be an artifact of an overly smooth sensor. Please either validate the telemetry on the remaining GPUs or substantially soften the claims to state that the noise-floor comparison applies only to the two externally validated devices.","section":"§II-C, Appendix A, Table III"},{"comment":"The abstract and takeaway claim that a linear MFU-based power model 'fits every tested GPU.' Table II reports R²=0.37 for the L4, which the text explains by the device's narrow power range (64 W idle floor, 72 W TDP, Section IV-A(c)). The low R² is then offset by the L4's low MAPE (3.5%), because there is little power variation to explain. This means 'fit' is being assessed by absolute error, not explained variance, and the L4 is at best a qualified exception. The manuscript should either state a single criterion for what constitutes a 'fit' (e.g., MAPE below a threshold) or explicitly present the L4 as an exception to the blanket claim.","section":"Abstract, §IV-A, Table II"},{"comment":"The claim that the conditioned model is 'statistically indistinguishable' from the cross-repeat noise floor on the RTX 4070 Ti and L4 is not supported by inferential statistics. Table III reports point estimates of the per-cell residual SD and the repeat-noise SD, but no confidence intervals, significance tests, or agreement metrics are given. With only four repeats per configuration and six observations per (GPU, dtype, batch) cell, these SD estimates are themselves noisy, and the bootstrap used for MAPE is not applied to the residual SD comparison. Please provide bootstrap CIs or an equivalent uncertainty analysis for the SD ratio, or replace 'statistically indistinguishable' with the weaker statement that the residual is of the same order of magnitude as the repeat noise.","section":"§III-C, §IV-B(c), Table III"}],"minor_comments":[{"comment":"The formula for C_req uses notation like 'L_num', 'H_num', 'Q_dim', 'T_seq' that is not fully defined in the text. Please define each symbol or point to the referenced source more explicitly. A typo check is also needed: 'N_um' appears to be a leftover fragment.","section":"§II-B"},{"comment":"The slope values (1.21 W/% on MI210 to 4.49 W/% on A100) are useful, but could be presented in a small table or annotated figure for easier cross-device comparison.","section":"§IV-A(a)"},{"comment":"The validation figure shows a linear relationship between GPU-reported power and external node-level power, but the text does not state whether the external measurement is node-level or GPU-only, nor the sampling alignment between the two signals. Please clarify the measurement setup and the meaning of the constant offset.","section":"Appendix A"},{"comment":"The paper says 'almost 3000 runs.' Given the exclusions (Quadro RTX 5000 lacks bf16, L4 has one excluded cell), it would be more precise to state the exact number of completed runs or the number of configurations, so readers can verify the denominator in the error statistics.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for MASCOTS and the public data/code are a strength. The main concern is not novelty or internal consistency but the dependence of the central quantitative claims on telemetry that is only partially validated. I would be willing to accept after the authors either validate the remaining four GPUs or explicitly restrict the 'every GPU' and 'at the data limit' claims to the validated subset. I also recommend the editor ask the authors to provide uncertainty intervals for the residual-SD-to-noise-floor comparison, as the current Table III is too terse to support 'statistically indistinguishable.'"},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First, the positive: this is a well-executed empirical study. The protocol is careful—repeated measures, bootstrap CIs, a noise-floor comparison, and honest reporting of the L4's low R². The main finding—that a linear MFU-power model fits compute-bound LLM training on every device tested, once you condition per (GPU, dtype, batch)—is supported by the data. The per-cell MAPE of ~1% on NVIDIA hardware being indistinguishable from repeat noise is a solid result. It closes a real gap: prior work [13] proposed MFU as a power proxy but didn't validate it; this paper does, across six GPUs and two vendors. That is a genuine contribution for energy-aware simulation.\n\nThe main soft spot is the ground-truth question. The paper uses vendor telemetry as the reference power signal and validates that against external wall power on only two of six GPUs (L40 and Quadro RTX 5000). The other four—including the A100 and MI210—are unvalidated. If those sensors have offsets or non-linearities, the fitted slopes and the noise-floor claim are built on that unvalidated signal. The MI210's cross-repeat SD of 0.20% vs ≥0.35% on NVIDIA is a hint that the AMD sensor may be smoothing or quantizing. This matters because the paper's strongest claim—that the conditioned residual is at the data limit—rests on the noise floor being real. I'd like to see either external validation on more devices or a clear statement that the model predicts the telemetry signal, not physical power, which is what a simulator would consume anyway. That's a moderate fix, not a fatal flaw.\n\nMinor issues: the repository URL has a space in it, which will break the link; check when it's in camera-ready. Also, the batch-1 memory-bound issue is correctly identified and labeled as future work, so no complaint there.\n\nOverall, this is a solid paper for the systems/simulation community. The writing is clear and the analysis is honest. It deserves a serious referee and will be useful to anyone building energy-aware simulators. I'd cite it. My verdict would be conditional on the telemetry validation and the link fix.","headline":"Solid, careful empirical study showing a per-(GPU, dtype, batch) linear MFU-power fit reaches the noise floor on NVIDIA GPUs; the main weakness is unvalidated telemetry on four of six devices.","tokens_in":12823,"tokens_out":2474,"would_cite":true,"duration_ms":22431,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Linear MFU model tracks GPU power in LLM training within ~1% error","keywords":["Model FLOPs Utilization","GPU power modeling","LLM training simulation","energy-aware computing","NVML","ROCm","compute-bound workloads","power telemetry"],"falsifier":"Take one of the unvalidated GPUs (A100, L4, RTX 4070 Ti, or MI210), run the same (dtype, batch size) sweep while logging both vendor-reported power and an external wall-power meter, and check whether the per-cell linear MFU fit still lands within about 1% MAPE of the external measurement. A systematic offset or nonlinearity between internal and external readings would show up as a slope error or a residual floor well above the repeat-noise level, falsifying the portability claim.","tokens_in":11902,"feed_emoji":"⚡","tokens_out":5769,"duration_ms":50619,"temperature":0.7,"pith_summary":"This paper tries to establish that Model FLOPs Utilization (MFU)—a software-calculable ratio of achieved throughput to a GPU's theoretical peak FLOPs—can stand in for hardware power telemetry when simulating LLM training. Across nearly 3,000 single-GPU training runs on six NVIDIA and AMD accelerators, a simple linear fit of MFU to vendor-reported power holds in compute-bound workloads, provided each (GPU, numerical precision, batch size) combination gets its own slope and intercept. That conditioning drops mean absolute percentage error from roughly 10% to about 1%, statistically indistinguishable from the noise between repeated runs on NVIDIA devices. The practical payoff would be that energy-aware simulators can predict power from an analytic metric they already compute, without reading vendor-specific counters.","feed_headline":"MFU predicts GPU power in LLM training within ~1% error","feed_subtitle":"Per-GPU calibration with Model FLOPs Utilization lets simulators predict energy use without hardware counters.","key_machinery":"MFU, defined as (tokens per second × required FLOPs per token) divided by the hardware's per-precision peak FLOPS, is the central object: it is software-calculable, architecture-relative, and already produced by training and inference simulators. The argument's workhorse is ordinary least-squares regression of GPU-reported power on empirical MFU, with the key design choice being a separate slope and intercept per (GPU, numerical precision, batch size) cell. That conditioning, rather than any nonlinear model, is what takes error from roughly 10% to about 1% and down to the repeat-measurement noise floor.","core_discovery":"The paper's central claim is that MFU—the ratio of achieved tokens-per-second times required FLOPs per token to a GPU's theoretical peak FLOPs—predicts GPU power draw linearly across compute-bound LLM training workloads on every accelerator tested. Nearly 3,000 controlled single-GPU runs spanning six devices (NVIDIA A100, L40, L4, Quadro RTX 5000, RTX 4070 Ti, AMD MI210), three model families, three precisions, seven batch sizes, and two context lengths support this. A single per-GPU linear fit yields roughly 10% mean absolute percentage error; conditioning the fit on (GPU, dtype, batch size) cuts error to about 1%, statistically indistinguishable from the cross-repeat measurement-noise floo","pith_inferences":["The same per-(dtype, batch) conditioning trick might transfer to LLM prefill phases, which are more compute-bound than decode, giving inference simulators a cheap power proxy beyond the training regime the paper measured.","Because the paper disabled fused attention kernels, the fitted slopes reflect eager attention; production kernels may shift slopes, but since calibration absorbs implementation variance, the linear model class likely survives—a testable prediction.","The ~1% error sitting at the measurement-noise floor implies that further model refinements inside a calibrated cell cannot help; gains must come from expanding coverage (more cells, a memory-bound signal) rather than more complex regressors.","Extending external wall-power validation to all six devices would test the weakest link: any hidden nonlinearity in A100, L4, RTX 4070 Ti, or MI210 vendor telemetry would change fitted slopes and could undermine the portability claim."],"forward_implications":["Simulators that already compute MFU (e.g., Vidur) can add per-GPU-calibrated power estimates without any hardware counters.","Supporting a new accelerator requires only a one-shot calibration sweep per (GPU, dtype, batch size) cell, after which power follows from MFU alone.","Energy-aware what-if comparisons for cluster scheduling and parallelization strategies become possible at roughly 1% per-cell accuracy in compute-bound training.","The model class transfers across vendors and precision modes, but slopes do not: each device needs its own fit because MFU is normalized by peak FLOPs.","Low-arithmetic-intensity workloads (batch 1, decode-style inference) fall outside the model's validity and need a complementary memory-bandwidth signal."],"supporting_citations":[{"why":"Defines MFU and the per-token FLOP formula; supplies the predictor the paper validates.","marker":"[11]"},{"why":"CalFLOPS profiler supplies the empirical per-token FLOP counts used to compute MFU.","marker":"[17]"},{"why":"Vidur is the simulator that already computes MFU and would consume the power model.","marker":"[12]"},{"why":"Prior MFU-based power model for Vidur that was never validated against hardware; this paper closes that gap.","marker":"[13]"},{"why":"Documents the NVML/ROCm telemetry interfaces used as the ground-truth power signal.","marker":"[18]"},{"why":"Provides the 1:2 forward-to-backward FLOP ratio used to convert forward-pass FLOPs into training-step FLOPs.","marker":"[22]"},{"why":"Assesses accuracy of NVIDIA's built-in power sensor, supporting the choice to trust GPU-reported power and motivating the Appendix A validation.","marker":"[19]"}],"fun_headline_variants":["MFU predicts GPU power within 1% for compute-bound LLM training","Six GPUs, one trick: MFU predicts power within 1% for LLM runs","Energy-aware LLM sims: MFU cuts power prediction error to ~1%","No hardware counters? MFU predicts GPU power within 1%","MFU: a software-defined proxy for GPU power in LLM training"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The entire calibration chain trusts that the GPU's own reported power reading (NVML or ROCm) is an accurate measure of true power draw; the paper verifies this against an external meter on only two of the six GPUs, so a biased or nonlinear sensor on any other device would silently corrupt every fitted line.","fun_headline_variants_meta":{"raw":{"variants":["MFU predicts GPU power within 1% for compute-bound LLM training","Six GPUs, one trick: MFU predicts power within 1% for LLM runs","Energy-aware LLM sims: MFU cuts power prediction error to ~1%","No hardware counters? MFU predicts GPU power within 1%","MFU: a software-defined proxy for GPU power in LLM training"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000793,"raw_usage":{"total_tokens":3317,"prompt_tokens":720,"completion_tokens":2597,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":464,"completion_tokens_details":{"reasoning_tokens":2493}},"tokens_in":464,"tokens_out":2597,"duration_ms":17216,"temperature":1.0,"reasoning_tokens":2493,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T10:27:46.016793+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take one of the unvalidated GPUs (A100, L4, RTX 4070 Ti, or MI210), run the same (dtype, batch size) sweep while logging both vendor-reported power and an external wall-power meter, and check whether the per-cell linear MFU fit still lands within about 1% MAPE of the external measurement. A systematic offset or nonlinearity between internal and external readings would show up as a slope error or a residual floor well above the repeat-noise level, falsifying the portability claim.","supporting_citations":[{"cited_title":"PaLM: scaling language modeling with pathways,","cited_arxiv_id":null,"evidence_quote":"Defines MFU and the per-token FLOP formula; supplies the predictor the paper validates."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"CalFLOPS profiler supplies the empirical per-token FLOP counts used to compute MFU."},{"cited_title":"Vidur: A large-scale simulation frame- work for LLM inference,","cited_arxiv_id":null,"evidence_quote":"Vidur is the simulator that already computes MFU and would consume the power model."},{"cited_title":"Quantifying the energy consumption and carbon emissions of LLM inference via simulations,","cited_arxiv_id":null,"evidence_quote":"Prior MFU-based power model for Vidur that was never validated against hardware; this paper closes that gap."},{"cited_title":"Monitoring and characterizing GPU usage,","cited_arxiv_id":null,"evidence_quote":"Documents the NVML/ROCm telemetry interfaces used as the ground-truth power signal."},{"cited_title":"Accurate and convenient energy measurements for GPUs: A detailed study of NVIDIA GPU’s built-in power sensor,","cited_arxiv_id":null,"evidence_quote":"Assesses accuracy of NVIDIA's built-in power sensor, supporting the choice to trust GPU-reported power and motivating the Appendix A validation."}],"review_version":1}