{"id":"e87ae97a-8456-4c09-835b-791e5dbc29d1","arxiv_id":"2607.26571","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"An analytical, calibration-based estimator of GPU inference energy for LLMs splits energy into compute, parameter access, KV-cache writes, and attention reads, but validates against the same data it was fitted on.","lead":"This paper builds a formula-based tool to estimate how much electricity an LLM consumes on an H100 GPU while answering a request, splitting the cost into computation and memory movement. The tool is meant for comparing models or workloads when power meters are unavailable; its headline accuracy claims are weakened because the formulas were tuned and tested on the same measurements.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Validation is in-sample: the 5–27% agreement in Table 4 is against the same Caravaca et al. dataset used to calibrate γ0, β, s_attn(N), and η(N) in Section 3.5; no held-out or independent evidence supports the predictive accuracy claim.","rationale":"The paper's strongest claim is that the estimator matches measured LLM inference energy within 5–27%. The evidence for this is Table 4, which compares the estimator's predictions to Caravaca et al. [23]. But Section 3.5 calibrates the model's free parameters (γ0, β, s_attn(N), η(N)) against those same measurements. This is a textbook in-sample validation. With six parameters and only four data points, achieving a reasonable fit is not evidence of predictive power. The paper itself notes in Section 5 that recalibration is needed for other hardware/engines/batching regimes, which effectively concedes that the 5–27% is not a general accuracy bound.\n\nI also noticed an independent red flag: Table 3's request energies do not follow the simplified estimator's own Eq. (17) from the listed token energies. For example, the 70B row lists E_in/token = 262.1 mJ and E_out/token = 218.4 mJ; with T_in=T_out=500, Eq. (17) gives E_request = (500×262.1 + 500×218.4) mJ = 240,250 mJ = 0.0667 Wh, but the table reports 0.1707 Wh. The same inconsistency appears in other rows. This means the 'analytical energy' column in Table 4—the basis for the reported 5–27% agreement—cannot be reproduced from the paper's own equations, and no code/data is provided to resolve it.\n\nThe analytical skeleton (FLOP accounting, memory decomposition) is reasonable and the authors are transparent about the model's limitations. However, the central empirical claim is not established by the evidence presented. A leave-one-out cross-validation or an independent H100 dataset would directly test whether the calibration generalizes. Until then, REJECT is the appropriate verdict.","tokens_in":15719,"tokens_out":5734,"duration_ms":67952,"concrete_test":"Leave-one-out cross-validation on the four Table 4 entries: for each of the four models, re-fit γ0, β, s_attn(N), η(N) using the other three Caravaca et al. measurements, then predict the held-out model's request energy at T_in=500, T_out=500. Compute relative error per fold. If the mean held-out relative error substantially exceeds the reported in-sample 5–27% (e.g., >40%), the claimed agreement is an in-sample artifact. Report the held-out errors and the fitted parameters.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—Eq. (12) matches measured LLM inference energy within 5–27% (Section 4.5, Table 4)—rests on comparing the estimator to Caravaca et al. [23], the very study used for calibration in Section 3.5. The calibration fits six free parameters (γ0, β, the amplitude and exponent of s_attn(N), and the amplitude and exponent of η(N)) by 'minimizing the deviation between model predictions and measurement-based reported energy values' (Section 3.5). Against only four measured points (Table 4), a close in-sample fit is to be expected; it says nothing about predictive accuracy for other models, serving engines, batching policies, or GPU generations. The paper explicitly concedes in Section 5 that the factors 'should be recalibrated for other hardware platforms, serving engines, or batching regimes,' so the unqualified 5–27% statement is not a general property. Additionally, Table 3 is internally inconsistent: for Llama 3.3 70B, Eq. (17) with T_in=T_out=500 yields 500×262.1 mJ + 500×218.4 mJ = 0.0667 Wh, not the reported 0.1707 Wh. The same discrepancy appears in other rows, making the provenance of the 'analytical energy' column in Table 4 ambiguous. Because no code or data ships, the reader cannot trace how the validation numbers were produced. Thus the headline accuracy claim is unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a semi-analytical, GPU-level energy estimator for LLM inference on H100-class hardware. It separates prefill from decode, decomposes request energy into tensor-core compute and HBM memory movement, and introduces calibrated factors for parameter access, KV-cache read overhead, and memory inefficiency. The estimator is instantiated with hardware coefficients from the literature, calibrated against the measurement study of Caravaca et al. [23], and then compared with that same study in Table 4, reporting agreement within 5–27%. A simplified parameter-only estimator is also presented and used to produce per-token and per-request energy values for a model inventory.","tokens_in":16068,"tokens_out":6720,"duration_ms":72618,"significance":"If the claimed 5–27% accuracy were supported by independent evidence, the estimator would be a useful design-time tool: it would let analysts compare LLM inference energy across models and workloads without power instrumentation, and the decomposition into compute, parameter access, KV-cache writes, and attention reads would provide actionable insight into energy bottlenecks. The analytical scaffold is clear and the paper explicitly scopes the estimate to accelerator-side energy. However, the central validation is in-sample, and the numerical results contain internal inconsistencies that make the headline agreement unverifiable. Without code or data and with no held-out comparison, the paper does not currently establish predictive accuracy.","major_comments":[{"comment":"The validation is circular. Section 3.5 states that γ0, β, s_attn(N), and η(N) are calibrated by minimizing deviation against the measurement-based energies of Caravaca et al. [23]. Section 4.5 then reports agreement against the same [23] values as evidence of accuracy. With four data points and six fitted parameters, the 5–27% errors are in-sample fit residuals, not out-of-sample predictions. The paper itself concedes in Section 5 that these factors should be recalibrated for other hardware platforms, serving engines, or batching regimes, so the unqualified agreement claim in Table 4 is not a general property. An independent validation set, leave-one-out calibration, or a clear reframing as a calibration study is needed.","section":"§3.5 and §4.5, Table 4"},{"comment":"The request energies in Table 3 do not follow from Eq. (17). For Llama 3.3 70B, Table 3 gives E_in/token = 262.08 mJ and E_out/token = 218.40 mJ. Eq. (17) with T_in = T_out = 500 gives 500 × (0.26208 + 0.21840) J = 240.24 J = 0.06673 Wh, not the reported 0.170747 Wh. Similar inconsistencies appear in other rows (e.g., 32B: 0.03051 Wh by Eq. (17) vs 0.052721 Wh reported). Since Table 4 uses the same analytical-energy column, the provenance of the 'analytical energy' values used for validation is ambiguous and must be corrected or clearly explained.","section":"§4.4, Eq. (17), Table 3"},{"comment":"The measured-energy column is matched to models only by parameter count (8B, 24B, 70B, 72B). The underlying model architectures, inference engines, batching policies, and measurement conditions from [23] are not specified. Because request energy depends strongly on architecture and serving configuration, comparing only on nominal size is insufficient to interpret the reported errors. The authors should state which exact models are compared and justify that the comparison is meaningful under the stated workload.","section":"§4.5, Table 4"}],"minor_comments":[{"comment":"The piecewise prefill multiplier M(T_in) is introduced as a fixed model parameter, but no derivation or calibration source is given. Please clarify whether these values were fit to data or chosen heuristically.","section":"§3.4, Eq. (23)"},{"comment":"The calibrated values for s_attn(N) and η(N) are reported without uncertainties or sensitivity analysis. Given the small calibration set, reporting confidence intervals or a robustness check would strengthen the presentation.","section":"Table 2"},{"comment":"The notation Bits′_attn is easy to confuse with the un-scaled Bits_attn. Consider using a different symbol, such as Bits_attn^scaled.","section":"§2.3 and Supplementary B"},{"comment":"No code or data availability statement is provided. Since the estimator is deterministic and the calibration is central to the results, releasing the implementation and the exact [23] values used would substantially aid reproducibility.","section":"General"},{"comment":"Some rows, such as DeepSeek-Coder V2 (16B) showing lower request energy than Qwen3 (8B), are explained by architectural differences in the text, but a footnote or column describing the architecture actually used would help avoid apparent anomalies.","section":"Table 3"}],"recommendation":"reject","confidential_remarks":"The paper has a useful analytical scaffold, but the central accuracy claim is not supported: the validation is in-sample, and Table 3's request energies contradict Eq. (17), which casts doubt on the Table 4 numbers. These are load-bearing issues, not presentation fixes. I would not recommend acceptance in the current form."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know about arXiv:2607.26571. The headline — \"matches measured energy within 5–27%\" — is not a validated prediction. The parameters γ0, β, s_attn(N), and η(N) are fit to Caravaca et al. in Section 3.5, and Section 4.5 reports agreement against that same study. Four data points, six free parameters: that's an in-sample residual, and not a tight one (23–27% on three of the four rows). Second, the request-energy column in Table 3 doesn't follow from Eq. 17 given the listed token energies (70B row: 0.171 Wh reported vs 0.067 Wh by the stated formula), and in Table 4 the 72B row gets the identical analytical value to the 70B row. No code or data ships, so you can't trace it.\n\nWhat's genuinely good: the analytical skeleton. Splitting prefill from decode and decomposing energy into compute, parameter access, KV-cache writes, and attention reads is the right structure, and the K=6 FLOP accounting plus the Antepara hardware coefficients are properly cited standard pieces. The scaling figures show qualitatively correct behavior — linear compute, quadratic attention reads. The simplified E = αKN estimator is a decent design-time yardstick. The paper is also honest about scope: it excludes CPU, PUE, and cooling, and concedes in Section 5 that the factors need recalibration for other hardware, engines, or batching regimes.\n\nThe soft spot is the epistemic framing. Section 4.5 presents the 5–27% agreement as a comparison; given Section 3.5, it's a calibration fit. The fix is straightforward: reframe as fit residuals, or validate on a held-out study. Table 3's mixed columns is a transparency bug in a paper whose selling point is transparency — the request-level column appears to use the full estimator while the token columns use the simplified one. That needs to be stated or corrected.\n\nThis deserves a serious referee: the framework is useful and the flaws are repairable, but as written the empirical claims are not supportable. Who gets value: green-coding teams and people doing design-time model comparison who will treat this as a scaffold and recalibrate on their own measurements. Cite it for the decomposition, not for the numbers.","headline":"Useful analytical scaffold for LLM inference energy, but the 5–27% agreement is an in-sample calibration fit against the same four-point dataset, and Table 3's request energies don't reconcile with the stated equations.","tokens_in":16651,"tokens_out":6868,"would_cite":false,"duration_ms":70996,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A token- and parameter-based formula estimates LLM GPU inference energy within about 5–27% of measured values, without power instrumentation.","keywords":["LLM inference energy","GPU energy estimation","analytical energy model","H100","tensor-core FLOPs","HBM memory traffic","prefill/decode decomposition","energy-aware AI"],"falsifier":"Take the paper's fixed factors and run the estimator against an independent measurement dataset from a different inference engine, batching regime, or GPU generation; if prediction errors systematically fall outside the claimed 5–27% range, the calibrated factors do not transfer and the central claim fails.","tokens_in":15535,"feed_emoji":"⚡","tokens_out":2670,"duration_ms":31142,"temperature":0.7,"pith_summary":"This paper tries to establish that request-level GPU energy for LLM inference can be estimated analytically from token counts, model size, and architectural details, without runtime power instrumentation. It combines standard FLOP-per-token accounting with calibrated memory-traffic factors and hardware-specific energy coefficients, separating prompt prefill from autoregressive decoding. If correct, engineers and researchers can compare models, workloads, and green-coding interventions on energy without needing telemetry or wall-plug meters. The estimate is reported to match one measurement-based study within roughly 5–27% for H100-class FP16/BF16 inference, with the caveat that calibration and comparison use the same measurement source.","feed_headline":"LLM GPU energy estimated within 5-27% without power meters","feed_subtitle":"A token-count formula separates prefill and decode to compare models on energy without hardware telemetry.","key_machinery":"The load-bearing object is the additive energy identity E_request = α_TC(C_pre + C_dec) + Bits_total·e_HBM·η(N), where α_TC is energy per tensor-core FLOP, e_HBM is energy per transferred HBM bit, and η(N) is a calibrated memory-inefficiency multiplier. The memory traffic Bits_total is further decomposed into parameter-access traffic b_w·N·γ(N), KV-cache write traffic 2·b_kv·d_model·n_l·T_out, and scaled attention-read traffic that grows quadratically with output length. The calibrated factors γ(N), s_attn(N), and η(N) absorb the gap between ideal FLOP/bit accounting and real inference engines, and this identity is what converts workload descriptions into energy estimates.","core_discovery":"The central claim is that total GPU energy per inference request equals tensor-core compute energy plus HBM memory-movement energy: E_request = α_TC(C_pre + C_dec) + Bits_total · e_HBM · η(N). The compute term scales linearly with parameter count and token counts; the memory term decomposes into parameter access, KV-cache writes, and attention-related KV-cache reads, with calibrated scaling factors for imperfect parameter reuse, attention-read overhead, and HBM inefficiency. Instantiated with H100 coefficients and fitted factors, the estimator reproduces measured per-request energies within about 5–27% for the compared model sizes. The paper is explicit that these are GPU-level estimates, no","pith_inferences":["The honest reading of the 5–27% agreement is that it is in-sample: the factors were fitted to the same measurement study used for comparison. The method's predictive claim will only be established by applying the fixed factors to an independent dataset from a different engine, batching policy, or GPU generation.","The same additive compute-plus-memory structure likely transfers to other accelerators by swapping α_TC and e_HBM, but the calibrated factors γ(N), s_attn(N), and η(N) would need refitting; this is a concrete, testable extension.","The simplified parameter-only estimator ignores attention and KV-cache memory costs, so it will systematically understate energy for long-context or memory-bound workloads; the architecture-aware form should be preferred whenever layer and hidden-dimension data are available.","A natural next step is to plug measured serving traces into the same equations to produce deployment-specific energy estimates, effectively turning the estimator into a lightweight monitoring layer where telemetry is unavailable."],"forward_implications":["Models and workloads can be compared on GPU energy using only parameter counts and token counts, without runtime instrumentation.","The decomposition into compute, parameter-access, KV-cache write, and attention-read energy shows where optimization levers bite: shorter outputs, prompt compression, KV-cache quantization, and batching.","Attention-related KV-cache reads grow quadratically with generated length and become a dominant cost in long generations, so the estimator predicts super-linear energy growth for long outputs.","The simplified parameter-only estimator gives a first-order per-token energy for model inventories, enabling quick comparative rankings before detailed architecture data is gathered.","The estimates are explicitly accelerator-side; datacenter, cooling, and system overheads are outside the stated scope."],"fun_headline_variants":["LLM GPU energy from token counts: 5-27% accuracy without meters","Prefill vs decode: token-based formula estimates LLM energy on H100s","Estimate LLM inference energy on H100 GPUs from tokens alone","Watt-hours per token: analytical model for LLM GPU energy","Token-to-watt: LLM GPU energy within 27%"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that the calibration factors fitted to one measurement study generalize to other inference engines, batching policies, and GPU generations; if they do not, the reported 5–27% agreement is in-sample fit error rather than predictive accuracy.","fun_headline_variants_meta":{"raw":{"variants":["LLM GPU energy from token counts: 5-27% accuracy without meters","Prefill vs decode: token-based formula estimates LLM energy on H100s","Estimate LLM inference energy on H100 GPUs from tokens alone","Watt-hours per token: analytical model for LLM GPU energy","Token-to-watt: LLM GPU energy within 27%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001223,"raw_usage":{"total_tokens":4872,"prompt_tokens":757,"completion_tokens":4115,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":501,"completion_tokens_details":{"reasoning_tokens":4017}},"tokens_in":501,"tokens_out":4115,"duration_ms":30423,"temperature":1.0,"reasoning_tokens":4017,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T13:04:12.176421+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the paper's fixed factors and run the estimator against an independent measurement dataset from a different inference engine, batching regime, or GPU generation; if prediction errors systematically fall outside the claimed 5–27% range, the calibrated factors do not transfer and the central claim fails.","supporting_citations":[],"review_version":1}