Benchmarking Energy Efficiency of Large Language Models Using vLLM

K. Pronk; Q. Zhao

REVIEW 5 major objections 5 minor 1 cited by

Benchmarking Energy Efficiency of Large Language Models Using vLLM

T0 review · 5 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Per-request LLM energy plateaus at 100 concurrent requests

desk verdict Useful vLLM-era energy measurements, but the architecture-null claim needs token-normalized controls before it can stand. read the letter →

arxiv 2509.08867 v1 pith:SZYHUT33 submitted 2025-09-10 cs.SE cs.AI

K. Pronk , Q. Zhao This is my paper · ORCID

classification cs.SEcs.AI

keywords largelanguagemodelsinferenceenergyefficiencyvLLMservingbenchmarkingPythiamodelfamilyGPUmeasurementconcurrentrequestload

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that energy-efficiency benchmarks for large language models should measure inference the way production systems actually serve it, and it demonstrates a benchmark built on the vLLM serving backend. It claims three regularities: GPU energy per request falls as simultaneous request volume rises and plateaus at about 100 requests; within the Pythia model family, energy per request grows close to linearly with parameter count; and among models near 3 billion parameters, architecture has no significant effect on energy per request. The practical point is that developers can estimate per-request energy cost from request load and model size, and that serving technology, not just model design, determines real-world efficiency. This matters because most earlier measurements came from lab setups that omit production serving optimizations.

What carries the argument

The load-bearing machinery is the benchmark itself: the vLLM serving backend, which optimizes throughput through paged attention (flexible, non-contiguous storage of key-value caches), dynamic batching, speculative decoding, and GPU acceleration; CodeCarbon, which reads GPU power sensors every 15 seconds to estimate per-component energy; and the experimental design that sends all requests concurrently without rate limiting, after 200 warm-up requests. The plateau at 100 requests is the signature of amortization: fixed per-request overheads shrink as the backend fills GPU batches until utilization saturates. The near-linear size scaling and the architecture null result are both read off this same per-request energy metric at the plateau load.

What would settle it

Run the same four roughly 3B models and the Pythia family at 100 concurrent requests with vLLM's optimization stack disabled or with a different serving backend; if architecture-level energy differences reappear or the parameter-size curve changes shape, the reported plateau, near-linear scaling, and architecture null result are properties of vLLM, not of the models.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that a production-style serving setup changes the energy-efficiency picture that laboratory benchmarks had reported. Using vLLM with all requests fired at once after warm-up, the benchmark finds that per-request GPU energy decreases with concurrency until it stabilizes at 100 simultaneous requests for both Pythia models and the four roughly 3B-parameter models. Within the Pythia family, per-request energy increases close to linearly with parameter count, with one exception: the 410M and 1B models consume nearly the same energy, which the paper attributes to the 1B model having fewer layers (16 versus 24), so layer count rather than parameter count is the limiting factor. Across four models of comparable size but different architectures, energy per request differs only marginally, in contrast to an earlier study that found a 47% spread. The paper reads the flat architecture result as likely an effect of vLLM's optimizations, while leaving the mechanism open.

Load-bearing premise

The load-bearing premise is that vLLM's automatic batching, speculative decoding, and GPU acceleration treat every model and architecture equally, so the measured patterns describe the models rather than the serving backend.

Editorial extensions

If this is right

Energy-efficiency comparisons between models are only meaningful at or above the plateau load; below 100 concurrent requests the measured cost reflects load rather than model efficiency.
For same-architecture families, per-request energy is roughly predictable from parameter count, with layer count as a secondary factor as seen in the 410M/1B pair.
At roughly 3 billion parameters, architecture choice carries little energy penalty under vLLM, so developers can choose among comparable models on accuracy and features without major energy trade-offs.
Increasing request concurrency up to the plateau reduces per-request GPU energy, making batching and queue management a practical lever for cutting inference energy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If vLLM's optimizations are what erase architectural efficiency differences, then energy rankings from offline, non-serving benchmarks may not transfer to production; production-aware benchmarks like this one are the relevant comparison for deployed systems.
The 410M/1B exception suggests the true driver of inference energy is the number of sequential layer computations, not raw parameter count; testing more same-family models with controlled layer counts would sharpen the scaling law.
Measuring energy per request rather than per token creates an incentive for concise outputs, since a model that answers in fewer tokens looks cheaper; pairing the metric with output quality or token-normalized cost would separate brevity gains from efficiency gains.
Since the paper only tested burst requests sent at once, varying arrival rates would reveal whether the plateau and architecture null result hold under realistic interleaved traffic; that is a direct next experiment.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Punchline: this is a genuinely useful measurement study, and the parts that matter most (vLLM changes the energy-vs-concurrency curve; plateau around 100 requests) are probably robust. The architecture comparison is the weak link: it is presented as a model-level null result but the metric is joules per request, not per token, and the models are poorly matched (2.8B vs 3B, base vs instruction-tuned, different tokenizers). On the plus side, the paper ships real observations not in the prior literature: per-request GPU energy under vLLM as a function of concurrent load across the Pythia family, the plateau at 100 requests, and the size-scaling curve with the 410M/1B anomaly. The methodology is transparent enough to reproduce, the authors pick a sensible serving backend, and they openly flag the core vLLM-vs-model validity threat. That last point matters: they know the confound and name it, which is more honest than most benchmark papers. The soft spots are real but not fatal to the whole paper. No error bars on Figures 1-3; the paper reports 10 repeats only for Figure 4. The workload is burst-only (all requests at once), so claiming production realism is a stretch. Energy per request is not normalized by generated tokens, and the paper does not report max_tokens or output lengths. The stress-test worry lands: J/request can look equal across architectures while J/token differs by a lot, and since this is exactly where the paper disagrees with Argerich et al., that claim does not test what it says. The Section V.A defense of J/request as an application-level metric is reasonable for a product lens, but the paper cannot have it both ways: either the null is about user-perceived serving efficiency (fine, but then do not call it a model-efficiency result), or it is about model efficiency, in which case token normalization is required. The size-scaling claim is mostly safe because the Pythia family shares a tokenizer and similar generation behavior, though the 410M outlier gets a plausible layer-count explanation from prior work. Citation pattern is fine; the comparison to Argerich et al. is external, and the self-citations are not inflated. Who this is for: practitioners choosing models under a vLLM workload, and people building the next generation of inference-energy benchmarks. A serious referee should see it, mainly to force the authors to either add token-level reporting or reframe the architecture claim. Recommendation to the editor: send to peer review, with a request for the token-normalized analysis and error bars; do not desk reject.

Referee Report

5 major / 5 minor

Summary. The paper introduces the "LLM Efficiency Benchmark," a measurement framework that uses the vLLM serving backend to measure GPU energy consumption of LLM inference under conditions meant to approximate production serving. Experiments vary concurrent request volume (5 to 5000 requests), model size across the Pythia family (70M to 6.9B parameters), and architecture across four models near 3B parameters (Pythia 2.8B, Dolly V2 2.8B, BLOOM 3B, RedPajama 2.8B). Energy is measured with CodeCarbon and reported as joules per request. The main claims are: per-request GPU energy decreases with concurrent request volume and plateaus at 100 requests; within the Pythia family energy per request grows close to linearly with parameter count; and model architecture has no significant effect on energy per request for models near 3B parameters. The results are compared against an earlier study by Argerich et al. that used the Transformers library and measured energy per token.

Significance. If the central claims are correct, the benchmark would be a useful step toward evaluating LLM inference efficiency in serving-oriented conditions rather than in isolated lab scripts, and the comparison with vLLM would be informative for practitioners. The paper is a direct measurement study with no fitted parameters or derived prediction, which is a strength in terms of transparency; the comparison to Argerich et al. is an external benchmark. The most valuable contribution--the platform-level observation that vLLM's batching behavior changes the energy profile relative to single-request Transformers inference--is plausible and worth pursuing. However, the architecture-null claim and the size-scaling claim are currently undersupported because the primary metric is not normalized by generated tokens, variance information is largely absent, and the compared models differ in size and tuning status. The paper's own validity section acknowledges the model-versus-backend threat and the tokenizer argument, but the conclusions are nevertheless stated as model-level results.

major comments (5)

[§IV.C, Figure 4, and §V.A] The architecture comparison is based on energy per request without any control for the number of generated tokens. HellaSwag completions are not length-controlled, and the paper does not report max_tokens, sampling parameters, or per-model output lengths. Since the models use different tokenizers and may produce different numbers of tokens, equal J/request is compatible with substantial differences in J/token. The Section V.A defense of J/request is an application-level argument, but the paper then interprets the null result as a model-efficiency statement and contrasts it with Argerich et al.'s 47% J/token effect. This comparison is not currently testing what it claims to test; token-level normalization or at least reported output-length statistics is needed.
[§IV.A, Figures 1-2, and §IV.B, Figure 3] Most of the reported results lack error bars, repetition counts, and any statistical test. Only Figure 4 states that each model was tested 10 times; Figures 1-3 appear to be based on unreported numbers of repeated runs. The plateau at 100 requests and the 'close to linear' relationship in Figure 3 are asserted from point estimates. The paper should report the number of repetitions per configuration, standard deviations or confidence intervals, and, for the linearity claim, a regression or residual analysis. The 410M/1B deviation noted in Section IV.B makes the linearity claim particularly sensitive to measurement uncertainty.
[§III.A and §IV.A] CodeCarbon is described as reading power values every 15 seconds. For small request counts such as 5 or 10 and small models like Pythia 70M, the total run duration may be comparable to or shorter than the sampling interval, which would make per-request energy estimates coarse or biased. The paper should report run durations, the number of power samples collected per run, and an assessment of whether the 15-second sampling interval is adequate for the shortest runs.
[Table II and §IV.C] The architecture comparison does not hold model size constant: BLOOM is 3B while the other three are 2.8B. In addition, Dolly V2 is an instruction-tuned model and the exact RedPajama checkpoint (base, instruction-tuned, or chat) is not specified. These differences confound architecture with model size and training objective. The paper should either match parameters and tuning status more carefully or explicitly treat these factors as limitations in the interpretation of the null result.
[§IV.B and §V] The claim that energy per request grows 'close to linear' with parameter count is not supported by a quantitative model. A linear fit with confidence intervals, or a comparison of linear versus alternative scaling models, should be provided. Additionally, the discussion attributes the 410M/1B deviation to layer count, but no sensitivity analysis or direct test of the layer-count explanation is presented; this explanation should be framed as a hypothesis rather than a conclusion.

minor comments (5)

[Throughout] The manuscript contains numerous typos and grammatical issues, including 'Alumni' in the author affiliation, 'preformed' for 'performed', 'Wether' for 'Whether', 'an growing' in the abstract, and 'scenario's' for 'scenarios'. A thorough copyedit is needed.
[§IV.A] The sentence 'the 6.9 billion parameter version of Pythia shows a stabilized energy consumption per request at 40 parameters' appears to contain a typo; it should presumably refer to a request-count value, not 40 parameters.
[§III.B] No link to the benchmark code or data is provided. Given that the paper introduces a 'benchmark', making the code and raw measurements available would substantially aid reproducibility.
[§III.C] The paper does not specify the exact model revisions used for Dolly V2 and RedPajama, nor the HuggingFace identifiers. This should be added so that the experiments can be reproduced.
[§II.C] The statement that 'existing research shows no correlation between parameter amount and energy efficiency' is followed by the claim that there is a sub-linear correlation within the same architecture; these statements should be reconciled and both should be cited precisely.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper is a direct measurement study whose results are observed quantities, not derived predictions.

full rationale

The paper contains no fitted parameters, no derived predictions, and no load-bearing self-citations. Its central results—per-request GPU energy versus concurrency, versus Pythia parameter count, and versus architecture—are direct measurements reported from the benchmark runs, so there is no equation in which an input is defined in terms of an output or in which a fitted quantity is renamed as a prediction. The selection of 100 requests for the size and architecture comparisons is a data-driven operating-point choice made after observing the plateau, but the paper does not present that selection as a prediction, and the comparisons at 100 requests are still measured quantities. The only external comparisons are to Argerich et al. and to the Pythia model documentation, neither of which is a self-citation by the present authors. The paper itself flags a genuine validity threat in Section V.A ('the question can be posed if the tests measure the efficiency of models, or the efficiency of vLLM'), but that is an acknowledged limitation about what the benchmark measures, not a circular reduction of the results to their inputs. Likewise, the use of energy per request rather than per token is a metric choice with stated justifications; it may raise questions about whether the architecture-null result holds at the token level, but that is a measurement-validity concern, not circularity. The derivation chain, if any, is simply observation-to-conclusion, and the conclusions are not presupposed by the benchmark design. Therefore the circularity score is 0.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The central claims rest on measurement assumptions rather than fitted parameters. No numbers are fitted to data; the main dependencies are the validity of CodeCarbon sampling, the energy-per-request metric, the representativeness of the burst workload, and the absence of vLLM-specific confounds.

assumptions (4)

domain assumption CodeCarbon's 15-second sampling of GPU sensors yields accurate energy estimates for runs of varying duration.
Section III.A: CodeCarbon reads sensors every 15 seconds; short runs with few requests may be shorter than the sampling interval, so per-request energy could be inaccurate. The paper does not validate against a power meter.
domain assumption Energy per request, without token normalization, is a valid efficiency metric across tokenizers and output lengths.
Section V.A defends this metric, but if models produce different numbers of tokens per request, joules per request conflates output length with efficiency.
domain assumption vLLM's dynamic batching and scheduling do not differentially distort energy measurements across models and architectures.
Section V.A acknowledges that tests may measure the efficiency of vLLM rather than the model; the central architecture comparison depends on this not being true.
domain assumption The burst workload, with all requests sent at once, represents realistic production conditions.
Section III.C sets 'no request rate', meaning requests arrive simultaneously; production workloads usually have arrival rates, so generalizing to production is an assumption.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Benchmarking Energy Efficiency of Large Language Models Using vLLM." pith.science (2026). https://pith.science/paper/SZYHUT33

@misc{pith2026250908867,
  author       = {Pith},
  title        = {Pith review of: Benchmarking Energy Efficiency of Large Language Models Using vLLM},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SZYHUT33}},
  note         = {Machine review of arXiv:2509.08867}
}

read the original abstract

The prevalence of Large Language Models (LLMs) is having an growing impact on the climate due to the substantial energy required for their deployment and use. To create awareness for developers who are implementing LLMs in their products, there is a strong need to collect more information about the energy efficiency of LLMs. While existing research has evaluated the energy efficiency of various models, these benchmarks often fall short of representing realistic production scenarios. In this paper, we introduce the LLM Efficiency Benchmark, designed to simulate real-world usage conditions. Our benchmark utilizes vLLM, a high-throughput, production-ready LLM serving backend that optimizes model performance and efficiency. We examine how factors such as model size, architecture, and concurrent request volume affect inference energy efficiency. Our findings demonstrate that it is possible to create energy efficiency benchmarks that better reflect practical deployment conditions, providing valuable insights for developers aiming to build more sustainable AI systems.

Figures

Figures reproduced from arXiv: 2509.08867 by the authors.

**Figure 2.** Inference energy cost per request on different model architectures. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 4.** Inference energy costs per 100 requests on different 3B parameter [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Lights, Camera, Carbon: Architectural Scaling Laws for Video Generation Energy Consumption
cs.MM 2026-07 conditional novelty 6.5 of 10

Energy of text-to-video diffusion models is predicted from architectural first principles and observable generation parameters with under 3% MAPE, without needing weights or model size.

Reference graph

Works this paper leans on

24 extracted references · 22 canonical work pages · cited by 1 Pith paper

[1]

Measuring and improving the energy efficiency of large language models inference

Mauricio Fadel Argerich and Marta Pati ˜no-Mart´ınez. Measuring and improving the energy efficiency of large language models inference. IEEE Access, 12:80194–80207, 2024

work page 2024
[2]

Smith, Nicole DeCario, and Will Buchanan

Jesse Dodge, Taylor Prewitt, Remi Tachet des Combes, Erika Odmark, Roy Schwartz, Emma Strubell, Alexandra Sasha Luccioni, Noah A. Smith, Nicole DeCario, and Will Buchanan. Measuring the carbon intensity of ai in cloud instances. InProceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’22, page 1877–1894, New York, NY ...

work page 2022
[3]

Towards the systematic reporting of the energy and carbon footprints of machine learning.J

Peter Henderson, Jieru Hu, Joshua Romoff, Emma Brunskill, Dan Jurafsky, and Joelle Pineau. Towards the systematic reporting of the energy and carbon footprints of machine learning.J. Mach. Learn. Res., 21(1), January 2020

work page 2020
[4]

Gonzalez, Hao Zhang, and Ion Stoica

Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention, 2023

2023
[5]

AI and Compute

Danny Hernandez Dario Amodei. AI and Compute. https://openai.com/ index/ai-and-compute/, 2024

work page 2024
[6]

Energy and policy considerations for modern deep learning research.Proceedings of the AAAI Conference on Artificial Intelligence, 34(09):13693–13696, April 2020

Emma Strubell, Ananya Ganesh, and Andrew McCallum. Energy and policy considerations for modern deep learning research.Proceedings of the AAAI Conference on Artificial Intelligence, 34(09):13693–13696, April 2020

work page 2020
[7]

Estimating the carbon footprint of bloom, a 176b parameter language model, 2022

Alexandra Sasha Luccioni, Sylvain Viguier, and Anne-Laure Ligozat. Estimating the carbon footprint of bloom, a 176b parameter language model, 2022

work page 2022
[8]

https: //developer.nvidia.com/system-management-interface

NVIDIA System Management Interface — developer.nvidia.com. https: //developer.nvidia.com/system-management-interface

work page

Show all 24 references

[9]

https://ss64

powermetrics Man Page - macOS - SS64.com — ss64.com. https://ss64. com/mac/powermetrics.html
[10]

Friedler, and Jonathan P

Kadan Lottick, Silvia Susai, Sorelle A. Friedler, and Jonathan P. Wilson. Energy usage reports: Environmental awareness as part of algorithmic accountability, 2019

2019
[11]

Eco2ai: carbon emissions tracking of machine learning models as the first step towards sustainable ai, 2022

Semen Budennyy, Vladimir Lazarev, Nikita Zakharenko, Alexey Ko- rovin, Olga Plosskaya, Denis Dimitrov, Vladimir Arkhipkin, Ivan Os- eledets, Ivan Barsola, Ilya Egorov, Aleksandra Kosterina, and Leonid Zhukov. Eco2ai: carbon emissions tracking of machine learning models as the ...

2022
[12]

Quantifying the carbon emissions of machine learning, 2019

Alexandre Lacoste, Alexandra Luccioni, Victor Schmidt, and Thomas Dandres. Quantifying the carbon emissions of machine learning, 2019

2019
[13]

Exploring the carbon footprint of hugging face’s ml models: A repository mining study

Joel Casta ˜no, Silverio Mart ´ınez-Fern´andez, Xavier Franch, and Justus Bogner. Exploring the carbon footprint of hugging face’s ml models: A repository mining study. In2023 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), pages 1–12, 2023

2023
[14]

Gemma Tokenizer — lunary.ai

Lunary. Gemma Tokenizer — lunary.ai. https://lunary.ai/ gemma2-tokenizer, 2024

2024
[15]

GPT Tokenizer

OpenAI. GPT Tokenizer. https://platform.openai.com/tokenizer
[16]

Hellaswag: Can a machine really finish your sentence?, 2019

Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can a machine really finish your sentence?, 2019

2019
[17]

Pythia: A suite for analyzing large language models across training and scaling, 2023

Stella Biderman, Hailey Schoelkopf, Quentin Anthony, Herbie Bradley, Kyle O’Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Puro- hit, USVSN Sai Prashanth, Edward Raff, Aviya Skowron, Lintang Sutawika, and Oskar van der Wal. Pythia: A suite for analyzing large language mo...

2023
[18]

https://www.databricks.com/blog/2023/04/12/ dolly-first-open-commercially-viable-instruction-tuned-llm

Free Dolly: Introducing the World’s First Truly Open Instruction-Tuned LLM — databricks.com. https://www.databricks.com/blog/2023/04/12/ dolly-first-open-commercially-viable-instruction-tuned-llm

2023
[19]

Rush, Stella Biderman, Albert Webson, Pawan Sasanka Ammanamanchi, Thomas Wang, Beno ˆıt Sagot, Niklas Muennighoff, Albert Villanova del Moral, Olatunji Ruwase, and Rachel Bawden

BigScience Workshop, :, Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ili ´c, Daniel Hesslow, Roman Castagn ´e, Alexan- dra Sasha Luccioni, Franc ¸ois Yvon, Matthias Gall ´e, Jonathan Tow, Alexander M. Rush, Stella Biderman, Albert Webson, Pawan Sasanka A...

2023
[20]

https://www

Releasing 3B and 7B RedPajama-INCITE family of models including base, instruction-tuned & chat models — together.ai. https://www. together.ai/blog/redpajama-models-v1
[21]

Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, R ´emi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Ma...

2020
[22]

Open llm leaderboard v2

Cl ´ementine Fourrier, Nathan Habib, Alina Lozovskaya, Konrad Szafer, and Thomas Wolf. Open llm leaderboard v2. https://huggingface.co/ spaces/open-llm-leaderboard/open llm leaderboard, 2024

2024
[23]

https://github.com/ huggingface/text-generation-inference

GitHub - huggingface/text-generation-inference: Large Language Model Text Generation Inference — github.com. https://github.com/ huggingface/text-generation-inference
[24]

This repository contains the open source components of TensorRT

GitHub - NVIDIA/TensorRT: NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT. — github.com. https://github.com/NVIDIA/TensorRT

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

Measuring and improving the energy efficiency of large language models inference

Mauricio Fadel Argerich and Marta Pati ˜no-Mart´ınez. Measuring and improving the energy efficiency of large language models inference. IEEE Access, 12:80194–80207, 2024

work page 2024

[2] [2]

Smith, Nicole DeCario, and Will Buchanan

Jesse Dodge, Taylor Prewitt, Remi Tachet des Combes, Erika Odmark, Roy Schwartz, Emma Strubell, Alexandra Sasha Luccioni, Noah A. Smith, Nicole DeCario, and Will Buchanan. Measuring the carbon intensity of ai in cloud instances. InProceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’22, page 1877–1894, New York, NY ...

work page 2022

[3] [3]

Towards the systematic reporting of the energy and carbon footprints of machine learning.J

Peter Henderson, Jieru Hu, Joshua Romoff, Emma Brunskill, Dan Jurafsky, and Joelle Pineau. Towards the systematic reporting of the energy and carbon footprints of machine learning.J. Mach. Learn. Res., 21(1), January 2020

work page 2020

[4] [4]

Gonzalez, Hao Zhang, and Ion Stoica

Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention, 2023

2023

[5] [5]

AI and Compute

Danny Hernandez Dario Amodei. AI and Compute. https://openai.com/ index/ai-and-compute/, 2024

work page 2024

[6] [6]

Energy and policy considerations for modern deep learning research.Proceedings of the AAAI Conference on Artificial Intelligence, 34(09):13693–13696, April 2020

Emma Strubell, Ananya Ganesh, and Andrew McCallum. Energy and policy considerations for modern deep learning research.Proceedings of the AAAI Conference on Artificial Intelligence, 34(09):13693–13696, April 2020

work page 2020

[7] [7]

Estimating the carbon footprint of bloom, a 176b parameter language model, 2022

Alexandra Sasha Luccioni, Sylvain Viguier, and Anne-Laure Ligozat. Estimating the carbon footprint of bloom, a 176b parameter language model, 2022

work page 2022

[8] [8]

https: //developer.nvidia.com/system-management-interface

NVIDIA System Management Interface — developer.nvidia.com. https: //developer.nvidia.com/system-management-interface

work page