Pith. sign in

REVIEW 3 major objections 5 minor 44 references

A token- and parameter-based formula estimates LLM GPU inference energy within about 5–27% of measured values, without power instrumentation.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 13:04 UTC pith:JEBUVC5P

load-bearing objection Useful analytical scaffold for LLM inference energy, but the 5–27% agreement is an in-sample calibration fit against the same four-point dataset, and Table 3's request energies don't reconcile with the stated equations. the 3 major comments →

arxiv 2607.26571 v1 pith:JEBUVC5P submitted 2026-07-29 cs.LG cs.SE

From Tokens to Watt-hours: Analytical Energy Estimation for LLM Inference on Modern GPUs

Tina Vartziotis , Rodopi Kosteli , Elli Vartziotis , George Dasoulas , Michael Keckeisen , Konstantinos Skianis , Sotirios Kotsopoulos , Francesca Dominici This is my paper
classification cs.LG cs.SE
keywords LLM inference energyGPU energy estimationanalytical energy modelH100tensor-core FLOPsHBM memory trafficprefill/decode decompositionenergy-aware AI
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to establish that request-level GPU energy for LLM inference can be estimated analytically from token counts, model size, and architectural details, without runtime power instrumentation. It combines standard FLOP-per-token accounting with calibrated memory-traffic factors and hardware-specific energy coefficients, separating prompt prefill from autoregressive decoding. If correct, engineers and researchers can compare models, workloads, and green-coding interventions on energy without needing telemetry or wall-plug meters. The estimate is reported to match one measurement-based study within roughly 5–27% for H100-class FP16/BF16 inference, with the caveat that calibration and comparison use the same measurement source.

Core claim

The central claim is that total GPU energy per inference request equals tensor-core compute energy plus HBM memory-movement energy: E_request = α_TC(C_pre + C_dec) + Bits_total · e_HBM · η(N). The compute term scales linearly with parameter count and token counts; the memory term decomposes into parameter access, KV-cache writes, and attention-related KV-cache reads, with calibrated scaling factors for imperfect parameter reuse, attention-read overhead, and HBM inefficiency. Instantiated with H100 coefficients and fitted factors, the estimator reproduces measured per-request energies within about 5–27% for the compared model sizes. The paper is explicit that these are GPU-level estimates, no

What carries the argument

The load-bearing object is the additive energy identity E_request = α_TC(C_pre + C_dec) + Bits_total·e_HBM·η(N), where α_TC is energy per tensor-core FLOP, e_HBM is energy per transferred HBM bit, and η(N) is a calibrated memory-inefficiency multiplier. The memory traffic Bits_total is further decomposed into parameter-access traffic b_w·N·γ(N), KV-cache write traffic 2·b_kv·d_model·n_l·T_out, and scaled attention-read traffic that grows quadratically with output length. The calibrated factors γ(N), s_attn(N), and η(N) absorb the gap between ideal FLOP/bit accounting and real inference engines, and this identity is what converts workload descriptions into energy estimates.

Load-bearing premise

The load-bearing premise is that the calibration factors fitted to one measurement study generalize to other inference engines, batching policies, and GPU generations; if they do not, the reported 5–27% agreement is in-sample fit error rather than predictive accuracy.

What would settle it

Take the paper's fixed factors and run the estimator against an independent measurement dataset from a different inference engine, batching regime, or GPU generation; if prediction errors systematically fall outside the claimed 5–27% range, the calibrated factors do not transfer and the central claim fails.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Models and workloads can be compared on GPU energy using only parameter counts and token counts, without runtime instrumentation.
  • The decomposition into compute, parameter-access, KV-cache write, and attention-read energy shows where optimization levers bite: shorter outputs, prompt compression, KV-cache quantization, and batching.
  • Attention-related KV-cache reads grow quadratically with generated length and become a dominant cost in long generations, so the estimator predicts super-linear energy growth for long outputs.
  • The simplified parameter-only estimator gives a first-order per-token energy for model inventories, enabling quick comparative rankings before detailed architecture data is gathered.
  • The estimates are explicitly accelerator-side; datacenter, cooling, and system overheads are outside the stated scope.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The honest reading of the 5–27% agreement is that it is in-sample: the factors were fitted to the same measurement study used for comparison. The method's predictive claim will only be established by applying the fixed factors to an independent dataset from a different engine, batching policy, or GPU generation.
  • The same additive compute-plus-memory structure likely transfers to other accelerators by swapping α_TC and e_HBM, but the calibrated factors γ(N), s_attn(N), and η(N) would need refitting; this is a concrete, testable extension.
  • The simplified parameter-only estimator ignores attention and KV-cache memory costs, so it will systematically understate energy for long-context or memory-bound workloads; the architecture-aware form should be preferred whenever layer and hidden-dimension data are available.
  • A natural next step is to plug measured serving traces into the same equations to produce deployment-specific energy estimates, effectively turning the estimator into a lightweight monitoring layer where telemetry is unavailable.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a semi-analytical, GPU-level energy estimator for LLM inference on H100-class hardware. It separates prefill from decode, decomposes request energy into tensor-core compute and HBM memory movement, and introduces calibrated factors for parameter access, KV-cache read overhead, and memory inefficiency. The estimator is instantiated with hardware coefficients from the literature, calibrated against the measurement study of Caravaca et al. [23], and then compared with that same study in Table 4, reporting agreement within 5–27%. A simplified parameter-only estimator is also presented and used to produce per-token and per-request energy values for a model inventory.

Significance. If the claimed 5–27% accuracy were supported by independent evidence, the estimator would be a useful design-time tool: it would let analysts compare LLM inference energy across models and workloads without power instrumentation, and the decomposition into compute, parameter access, KV-cache writes, and attention reads would provide actionable insight into energy bottlenecks. The analytical scaffold is clear and the paper explicitly scopes the estimate to accelerator-side energy. However, the central validation is in-sample, and the numerical results contain internal inconsistencies that make the headline agreement unverifiable. Without code or data and with no held-out comparison, the paper does not currently establish predictive accuracy.

major comments (3)
  1. [§3.5 and §4.5, Table 4] The validation is circular. Section 3.5 states that γ0, β, s_attn(N), and η(N) are calibrated by minimizing deviation against the measurement-based energies of Caravaca et al. [23]. Section 4.5 then reports agreement against the same [23] values as evidence of accuracy. With four data points and six fitted parameters, the 5–27% errors are in-sample fit residuals, not out-of-sample predictions. The paper itself concedes in Section 5 that these factors should be recalibrated for other hardware platforms, serving engines, or batching regimes, so the unqualified agreement claim in Table 4 is not a general property. An independent validation set, leave-one-out calibration, or a clear reframing as a calibration study is needed.
  2. [§4.4, Eq. (17), Table 3] The request energies in Table 3 do not follow from Eq. (17). For Llama 3.3 70B, Table 3 gives E_in/token = 262.08 mJ and E_out/token = 218.40 mJ. Eq. (17) with T_in = T_out = 500 gives 500 × (0.26208 + 0.21840) J = 240.24 J = 0.06673 Wh, not the reported 0.170747 Wh. Similar inconsistencies appear in other rows (e.g., 32B: 0.03051 Wh by Eq. (17) vs 0.052721 Wh reported). Since Table 4 uses the same analytical-energy column, the provenance of the 'analytical energy' values used for validation is ambiguous and must be corrected or clearly explained.
  3. [§4.5, Table 4] The measured-energy column is matched to models only by parameter count (8B, 24B, 70B, 72B). The underlying model architectures, inference engines, batching policies, and measurement conditions from [23] are not specified. Because request energy depends strongly on architecture and serving configuration, comparing only on nominal size is insufficient to interpret the reported errors. The authors should state which exact models are compared and justify that the comparison is meaningful under the stated workload.
minor comments (5)
  1. [§3.4, Eq. (23)] The piecewise prefill multiplier M(T_in) is introduced as a fixed model parameter, but no derivation or calibration source is given. Please clarify whether these values were fit to data or chosen heuristically.
  2. [Table 2] The calibrated values for s_attn(N) and η(N) are reported without uncertainties or sensitivity analysis. Given the small calibration set, reporting confidence intervals or a robustness check would strengthen the presentation.
  3. [§2.3 and Supplementary B] The notation Bits′_attn is easy to confuse with the un-scaled Bits_attn. Consider using a different symbol, such as Bits_attn^scaled.
  4. [General] No code or data availability statement is provided. Since the estimator is deterministic and the calibration is central to the results, releasing the implementation and the exact [23] values used would substantially aid reproducibility.
  5. [Table 3] Some rows, such as DeepSeek-Coder V2 (16B) showing lower request energy than Qwen3 (8B), are explained by architectural differences in the text, but a footnote or column describing the architecture actually used would help avoid apparent anomalies.

Circularity Check

1 steps flagged

In-sample validation: Table 4's 5–27% agreement is against the same Caravaca et al. [23] measurements that were used in §3.5 to fit γ0, β, s_attn(N), and η(N).

specific steps
  1. fitted input called prediction [Section 3.5 (Calibration Procedure) and Section 4.5 / Table 4; Eq. (12), Eq. (20)-(22), Eq. (39)-(44)]
    "We estimate the model parameters using a data-driven calibration procedure based on reported energy measurements for LLM inference [23]. ... The parameters of 𝑠attn(𝑁) (Equation 22) and 𝜂(𝑁) (Equation 11) are calibrated by minimizing the deviation between model predictions and measurement-based reported energy values across the evaluated models. ... Finally, we compare the analytical estimates against the measurement-based study of Caravaca et al. [23]."

    The calibration in §3.5 fixes the free constants (γ0, β, and the amplitude/exponent of s_attn(N) and η(N)) by fitting to [23]'s measured energies. These constants enter the headline estimator Eq. (12) through Bitsparams and Bits'attn. Table 4 then validates Eq. (12) against the same [23] measurements and reports 5–27% agreement. The comparison is thus in-sample: the reported errors are residuals of the fit, not evidence of out-of-sample predictive accuracy. The text even concedes in §3.5 that the calibration uses 'a small number of calibration parameters and limited data points.'

full rationale

The analytic skeleton of the model is not circular: the FLOP accounting (Eqs. 7-9), HBM traffic decomposition (Eqs. 20-22), and hardware coefficients α_TC/e_HBM from [21] are derived from external transformer and microarchitectural facts. The circularity is isolated to the validation protocol: §3.5 calibrates the memory-scaling factors against Caravaca et al. [23], and §4.5/Table 4 reports agreement against that same dataset. Therefore the '5–27%' headline is an in-sample fit quality measure. Section 5's admission that the factors 'should be recalibrated for other hardware platforms, serving engines, or batching regimes' reinforces that the fitted values are not portable. No load-bearing self-citation was found: author-own works [11,15] are contextual green-AI citations, and the empirical basis is external ([21], [23]). Separately, as a correctness (not circularity) concern, Table 3's request-energy column is internally inconsistent with Eq. (17) and its own per-token columns (e.g., Llama 3.3 70B: 500×262.08 mJ + 500×218.40 mJ ≈ 0.0667 Wh, not 0.170747 Wh), making the provenance of the 'analytical energy' column in Table 4 ambiguous. That ambiguity lowers confidence in the validation numbers but is not a circularity mechanism in itself.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 0 invented entities

The central estimate rests on six fitted/chosen parameters, all derived from the same external measurement study used for validation. The analytic skeleton is standard FLOP and memory accounting; the empirical content is entirely inside the fitted factors.

free parameters (6)
  • γ0 = 0.10
    Baseline parameter-access factor at reference model size; fitted in §3.5 against [23].
  • β = 0.8
    Power-law exponent for degradation of parameter reuse with model size; fitted in §3.5 against [23].
  • s_attn coefficients = 1.5, exponent 0.9
    Attention scaling factor s_attn(N)=1+1.5(N/N0)^0.9; calibrated to KV-cache read overhead against [23].
  • η coefficients = 0.8, exponent 0.8
    Global memory-inefficiency factor η(N)=1+0.8(N/N0)^0.8; calibrated to HBM overhead against [23].
  • M(T_in) prefill multiplier buckets = 1.2, 1.8, 3.0, 4.0
    Piecewise multiplier for simplified estimator, Eq. 23; chosen by input-length bucket with no calibration shown.
  • N0 reference model size = 24B
    Reference size used in all scaling factors; chosen by hand.
axioms (5)
  • domain assumption K=6 FLOPs per parameter per token for dense transformer forward inference.
    Used in Eqs. 7, 15, 31–34; taken from scaling-law literature [19,20] and treated as universal for the evaluated models.
  • domain assumption GPU energy is the sum of tensor-core FLOP energy and HBM movement energy, with coefficients α_TC and e_HBM from Antepara et al. [21].
    Eqs. 4–6, 12; other GPU components (e.g., SRAM, control logic, tensor-parallel communication) are not modeled.
  • domain assumption Weights and KV-cache entries use 16-bit precision (FP16/BF16).
    Eq. 18 and Section 3.1; quantized inference is explicitly out of scope.
  • domain assumption Attention memory traffic during decode grows as T_out*T_in + T_out(T_out−1)/2.
    Eqs. 22, 25, 42; standard quadratic self-attention counting, ignoring GQA/MQA and kernel optimizations.
  • domain assumption Measured energies in Caravaca et al. [23] are accurate and directly comparable across the four model sizes used for calibration and validation.
    This is the sole empirical reference for both the Table 2 factors and the Table 4 comparison.

pith-pipeline@v1.3.0-daily-deepseek · 15345 in / 16945 out tokens · 153761 ms · 2026-08-01T13:04:12.176421+00:00 · methodology

0 comments
read the original abstract

The operational energy consumption of large language model (LLM) inference is becoming an increasingly important component of the environmental footprint of deployed AI systems. However, direct measurement of inference energy often requires hardware telemetry, power instrumentation, or infrastructure-specific monitoring, limiting its applicability in comparative studies, early-stage system design, and sustainability reporting. This report presents an analytically structured, empirically calibrated, GPU-level methodology for estimating LLM inference energy on NVIDIA H100-class accelerators without direct runtime measurement. The proposed estimator combines parameter-scaled transformer FLOP accounting, calibrated memory-traffic factors, and hardware-specific energy coefficients for FP16/BF16 tensor-core computation and high-bandwidth-memory movement. It explicitly separates prompt prefill from autoregressive decoding, enabling energy estimates for input tokens, output tokens, and complete inference requests. The methodology further decomposes total energy into compute, parameter-access, key-value-cache write, and attention-read components, allowing the scaling behavior with model size, context length, and generated-token count to be analyzed. The resulting estimates are not intended to replace physical power measurements; rather, they provide transparent, reproducible, and assumption-explicit approximations suitable for model comparison, green-coding analysis, and design-time evaluation of LLM inference workloads.

Figures

Figures reproduced from arXiv: 2607.26571 by Elli Vartziotis, Francesca Dominici, George Dasoulas, Konstantinos Skianis, Michael Keckeisen, Rodopi Kosteli, Sotirios Kotsopoulos, Tina Vartziotis.

Figure 1
Figure 1. Figure 1: Energy breakdown as a function of generated out￾put length 𝑇out for a 32B-parameter model with 𝑇in = 100 input tokens. Compute and KV-cache write costs scale almost linearly with 𝑇out, while attention-related KV-cache reads ex￾hibit quadratic growth and become increasingly significant at larger output lengths [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Energy per request as a function of model size for a fixed workload of 500 input tokens and 500 output tokens. Total energy, compute energy, and memory energy are shown on a log–log scale [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

44 extracted references · 7 linked inside Pith

  1. [1]

    Strubell, A

    E. Strubell, A. Ganesh, A. McCallum, Energy and policy considerations for deep learning in nlp, in: Proceedings of the 57th Annual Meeting of the Asso- ciation for Computational Linguistics, Association for Computational Linguistics, Florence, Italy, 2019, pp. 3645–3650. URL: https://aclanthology.org/P19-1355/. doi:10.18653/v1/P19-1355

    doi:10.18653/v1/p19-1355 2019

  2. [2]

    Schwartz, J

    R. Schwartz, J. Dodge, N. A. Smith, O. Etzioni, Green ai, Communications of the ACM 63 (2020) 54–63. URL: https://doi.org/10.1145/3381831. doi:10.1145/ 3381831

    doi:10.1145/3381831 2020

  3. [3]

    Patterson, J

    D. Patterson, J. Gonzalez, Q. V. Le, C. Liang, L.-M. Munguia, D. Rothchild, D. So, M. Texier, J. Dean, Carbon emissions and large neural network training, 2021. URL: https://arxiv.org/abs/2104.10350. arXiv:2104.10350

    Pith/arXiv arXiv 2021

  4. [4]

    Henderson, J

    P. Henderson, J. Hu, J. Romoff, E. Brunskill, D. Jurafsky, J. Pineau, Towards the systematic reporting of the energy and carbon footprints of machine learning, Journal of Machine Learning Research 21 (2020) 1–43. URL: http://jmlr.org/papers/v21/20-312.html

    2020

  5. [5]

    Lacoste, A

    A. Lacoste, A. Luccioni, V. Schmidt, T. Dan- dres, Quantifying the carbon emissions of machine learning, 2019. URL: https://arxiv.org/abs/1910.09700. arXiv:1910.09700

    Pith/arXiv arXiv 2019

  6. [6]

    Lannelongue, J

    L. Lannelongue, J. Grealey, M. Inouye, Green al- gorithms: Quantifying the carbon footprint of com- putation, Advanced Science 8 (2021) 2100707. URL: https://doi.org/10.1002/advs.202100707. doi:10.1002/ advs.202100707

    doi:10.1002/advs.202100707 2021

  7. [7]

    Betello, V

    F. Betello, V. Vineis, A. Purificato, G. Tolomei, F. Sil- vestri, One search fits all: Pareto-optimal eco-friendly model selection, arXiv preprint arXiv:2505.01468 (2025).arXiv:2505.01468

    Pith/arXiv arXiv 2025

  8. [8]

    E. Lim, Z. Pan, Y. Zhou, Characterizing the carbon impact of llm inference, Final course project report for 15-642: Machine Learning Systems (2024). Inference characterization

    2024

  9. [9]

    Fernandez, C

    J. Fernandez, C. Na, V. Tiwari, Y. Bisk, S. Luccioni, E. Strubell, Energy considerations of large language model inference and efficiency optimizations, in: Pro- ceedings of the 63rd Annual Meeting of the Asso- ciation for Computational Linguistics, Association for Computational Linguistics, 2025. URL: https:// aclanthology.org/2025.acl-long.1563/

    2025

  10. [10]

    Z. Fu, F. Chen, S. Zhou, H. Li, L. Jiang, Llmco2: Ad- vancing accurate carbon footprint prediction for llm in- ferences, 2024. URL: https://arxiv.org/abs/2410.02950. arXiv:2410.02950

    Pith/arXiv arXiv 2024

  11. [11]

    Vartziotis, M

    T. Vartziotis, M. Schmidt, G. Dasoulas, I. Dellatolas, S. Attademo, V. D. Le, A. Wiechmann, T. Hoffmann, M. Keckeisen, S. Kotsopoulos, Carbon footprint eval- uation of code generation through llm as a service, in: A. C. Kulzer, H.-C. Reuss, A. Wagner (Eds.), 2024 Stuttgart International Symposium on Automotive and Engine Technology, Springer Fachmedien Wi...

    2024

  12. [12]

    Reports average data-center PUE of approximately 1.56

    Uptime Institute, Uptime Institute Global Data Cen- ter Survey 2024, Technical Report, Uptime Institute Intelligence, 2024. Reports average data-center PUE of approximately 1.56

    2024

  13. [13]

    arXiv:2410.12032

    MLCommons, MLPerf Power: Benchmarking the En- ergy Efficiency of Machine Learning Systems, 2024. arXiv:2410.12032

    Pith/arXiv arXiv 2024

  14. [14]

    nvidia.com/en-us-hopper-architecture/ nvidia-tensor-core-gpu-datasheet, 2023

    NVIDIA, NVIDIA H100 Tensor Core GPU Datasheet, https://resources. nvidia.com/en-us-hopper-architecture/ nvidia-tensor-core-gpu-datasheet, 2023. Accessed 2026-05-24

    2023

  15. [15]

    Vartziotis, I

    T. Vartziotis, I. Dellatolas, G. Dasoulas, M. Schmidt, F. Schneider, T. Hoffmann, S. Kotsopoulos, M. Keck- eisen, Learn to code sustainably: An empirical study on green code generation, in: Proceedings of the 1st International Workshop on Large Language Mod- els for Code, LLM4Code ’24, Association for Comput- ing Machinery, New York, NY, USA, 2024, p. 30–...

    arXiv 2024

  16. [16]

    Samsi, D

    S. Samsi, D. Zhao, J. McDonald, B. Li, A. Michaleas, M. Jones, W. Bergeron, J. Kepner, D. Tiwari, V. Gade- pally, From words to watts: Benchmarking the energy costs of large language model inference, in: 2023 IEEE High Performance Extreme Com- puting Conference (HPEC), IEEE, ????, pp. 1–9. URL: https://ieeexplore.ieee.org/document/10363447/. doi:10.1109/H...

    arXiv 2023

  17. [17]

    Luccioni, Y

    S. Luccioni, Y. Jernite, E. Strubell, Power hungry processing: Watts driving the cost of AI deploy- ment?, in: The 2024 ACM Conference on Fairness Ac- countability and Transparency, ACM, ????, pp. 85–99. URL: https://dl.acm.org/doi/10.1145/3630106.3658542. doi:10.1145/3630106.3658542

    arXiv 2024

  18. [18]

    B. Li, Y. Jiang, V. Gadepally, D. Tiwari, Sprout: Green generative ai with carbon-efficient llm inference, in: Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, 2024, pp. 21799–21813. URL: https://aclanthology.org/2024.emnlp-main.1215/

    2024

  19. [19]

    Kaplan, S

    J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, D. Amodei, Scaling laws for neural language models, 2020. URL: https://arxiv.org/abs/2001.08361. arXiv:2001.08361

    Pith/arXiv arXiv 2020

  20. [20]

    Hoffmann, S

    J. Hoffmann, S. Borgeaud, A. Mensch, E. Buchatskaya, T. Cai, E. Rutherford, D. de Las Casas, L. A. Hendricks, J. Welbl, A. Clark, T. Hennigan, E. Noland, K. Millican, G. van den Driessche, B. Damoc, A. Guy, S. Osindero, K. Simonyan, E. Elsen, J. W. Rae, O. Vinyals, L. Sifre, Training compute-optimal large language models, in: Advances in Neural Informatio...

    2022

  21. [21]

    Antepara, Z

    O. Antepara, Z. Zhao, B. Austin, N. Ding, L. Oliker, N. J. Wright, S. Williams, Benchmark-driven models for energy analysis and attribution of gpu-accelerated supercomputing, in: Proceedings of the Interna- tional Conference for High Performance Computing, Networking, Storage and Analysis, SC ’25, Associa- tion for Computing Machinery, St. Louis, MO, USA,

  22. [22]

    W. Kwon, Z. Li, S. Zhuang, Y. Sheng, L. Zheng, C. H. Yu, J. E. Gonzalez, H. Zhang, I. Stoica, Efficient memory management for large language model serving with pagedattention, in: Proceedings of the 29th ACM Symposium on Operating Systems Principles (SOSP), ACM, 2023. doi:10.1145/3600006.3613165

    arXiv 2023

  23. [23]

    Caravaca, Á

    F. Caravaca, Á. Cuevas, R. Cuevas, From prompts to power: Measuring the energy footprint of llm in- ference, 2025. URL: https://arxiv.org/abs/2511.05597. arXiv:2511.05597

    arXiv 2025

  24. [24]

    Accessed 2026-05-24

    MLCommons, MLPerf Inference: Power Measure- ment Documentation, https://docs.mlcommons.org/ inference/power/, 2024. Accessed 2026-05-24

    2024

  25. [25]

    T. Dao, D. Y. Fu, S. Ermon, A. Rudra, C. Ré, Flashatten- tion: Fast and memory-efficient exact attention with io-awareness, in: Advances in Neural Information Processing Systems, volume 35, 2022. URL: https:// proceedings.neurips.cc/paper_files/paper/2022/hash/ 67d57c32e20fd0a7a302cb81d36e40d5-Abstract-Conference. html

    2022

  26. [26]

    URL: https://docs.nvidia.com/nemo/megatron-bridge/ 0.2.0/models/vlm/ministral3.html

    NVIDIA, Ministral 3 model documentation, 2024. URL: https://docs.nvidia.com/nemo/megatron-bridge/ 0.2.0/models/vlm/ministral3.html

    2024

  27. [27]

    URL: https: //huggingface.co/google/embeddinggemma-300m

    Google, Embeddinggemma-300m, 2024. URL: https: //huggingface.co/google/embeddinggemma-300m

    2024

  28. [28]

    URL: https://www.aimodels.fyi/models/huggingFace/ mxbai-embed-large-v1-mixedbread-ai

    Mixedbread AI, mxbai-embed-large-v1, 2024. URL: https://www.aimodels.fyi/models/huggingFace/ mxbai-embed-large-v1-mixedbread-ai

    2024

  29. [29]

    URL: https: //huggingface.co/Qwen/Qwen3-Embedding-0.6B

    Alibaba, Qwen3 embedding 0.6b, 2024. URL: https: //huggingface.co/Qwen/Qwen3-Embedding-0.6B

    2024

  30. [30]

    URL: https://huggingface.co/Qwen/Qwen3-32B

    Alibaba, Qwen3-32b model configuration, 2024. URL: https://huggingface.co/Qwen/Qwen3-32B

    2024

  31. [31]

    URL: https:// huggingface.co/ibm-granite/granite-3.3-8b-instruct

    IBM, Granite 3.3 8b instruct, 2024. URL: https:// huggingface.co/ibm-granite/granite-3.3-8b-instruct

    2024

  32. [32]

    URL: https:// huggingface.co/ibm-granite/granite-3.2-2b-instruct

    IBM, Granite 3.2 2b instruct, 2024. URL: https:// huggingface.co/ibm-granite/granite-3.2-2b-instruct

    2024

  33. [33]

    DeepSeek AI, Deepseek coder v2 lite instruct,

  34. [34]

    DeepSeek AI, Deepseek r1 distill qwen 32b,

  35. [35]

    URL: https://huggingface.co/Qwen/Qwen3-8B

    Alibaba, Qwen3 8b model configuration, 2024. URL: https://huggingface.co/Qwen/Qwen3-8B

    2024

  36. [36]

    URL: https://huggingface.co/Qwen/Qwen3-32B

    Alibaba, Qwen3 32b model configuration, 2024. URL: https://huggingface.co/Qwen/Qwen3-32B

    2024

  37. [37]

    URL: https://huggingface.co/deepseek-ai/ DeepSeek-R1-Distill-Qwen-32B

  38. [38]

    URL: https: //arxiv.org/html/2409.12186v2

    Alibaba, Qwen2.5 technical report, 2024. URL: https: //arxiv.org/html/2409.12186v2

    Pith/arXiv arXiv 2024

  39. [39]

    URL: https:// unsloth.ai/blog/llama3-3

    Unsloth AI, Llama 3.3 overview, 2024. URL: https:// unsloth.ai/blog/llama3-3

    2024

  40. [40]

    URL: https://huggingface.co/Qwen/Qwen3-1.7B

    Alibaba, Qwen3 1.7b model configuration, 2024. URL: https://huggingface.co/Qwen/Qwen3-1.7B

    2024

  41. [41]

    URL: https://huggingface.co/openai/gpt-oss-20b

    OpenAI, Gpt-oss 20b model configuration, 2024. URL: https://huggingface.co/openai/gpt-oss-20b. Supplementary Material A. Derivation of the Compute Model This section provides the architecture-aware compute terms used by the estimator. The main paper reports the compact parameter-scaled form, while the full attention correction terms are given here for rep...

    2024

  42. [43]

    URL: https://huggingface.co/openai/gpt-oss-120b

    OpenAI, Gpt-oss 120b model configuration, 2024. URL: https://huggingface.co/openai/gpt-oss-120b

    2024

  43. [2024]

    URL: https://huggingface.co/deepseek-ai/ DeepSeek-Coder-V2-Lite-Instruct

  44. [2025]

    doi:10.1145/3712285.3759815

    URL: https://doi.org/10.1145/3712285.3759815. doi:10.1145/3712285.3759815

    arXiv