Pith. sign in

REVIEW 3 major objections 5 minor 39 references

Brevity is the soul of sustainability: Characterizing LLM response lengths

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper shows that LLM responses to factual questions run far longer than necessary, and that simple prompt directives can cut response length and inference energy by 25–60% while preserving or improving answer quality.

desk verdict Solid empirical benchmark on LLM verbosity and prompt-based compression, but the energy and quality numbers are rougher than the abstract suggests. read the letter →

arxiv 2506.08686 v2 pith:YGYU2EVI submitted 2025-06-10 cs.CL cs.CY

Soham Poddar , Paramita Koley , Janardan Misra , Sanjay Podder , Navveen Balani , Niloy Ganguly , Saptarshi Ghosh This is my paper
classification cs.CLcs.CY
keywords LLMresponselengthinferenceenergyoutputcompressionpromptengineeringinformationcategoriesfactualquestionansweringefficiencysustainability
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to establish that the default behavior of large language models is to produce far more text than a factual question requires, and that this verbosity carries a real energy cost at inference time. Working from the observation that output length, not input length, dominates inference energy, the authors benchmark twelve models on five factual QA datasets and find that generated responses are often 1–10 times (sometimes over 10 times) longer than gold answers. They dissect the surplus text into six information categories and show that, on average, only about 42% of a response is the minimal answer, with about 18% irrelevant and 5% conversational tokens. The practical punchline is that instructing the model to give only the minimal answer cuts response length by about 60% and inference energy by about 28% on average, while ROUGE-L F1 against gold answers improves. If these results hold in deployment, adjusting system prompts is a nearly free way to make LLM inference greener.

What carries the argument

The mechanism carrying the argument is the six-category information taxonomy of LLM responses, which turns 'verbosity' into measurable token classes. The paper defines minimal answer (MINANS), additional helpful information (ADDINFO), reasoning/explanations (EXPLAIN), conversational enhancements (CONVERS), redundant information (REDINFO), and irrelevant information (IRREL), then uses the taxonomy to design prompt directives that suppress the non-essential classes. The second load-bearing piece is the established linear relationship between output length and inference energy from the authors' prior work, which converts token counts into energy estimates. Together these let the paper diagnose why responses are long and prescribe specific, testable prompts that shorten them.

What would settle it

A user study in which people rate default vs. MINANS-prompted responses to factual questions on sufficiency and satisfaction; if a majority prefer the longer default or judge the short answers insufficient, the paper's premise that shorter equals 'minimal and sufficient' collapses. Alternatively, a production A/B test measuring per-query inference energy on a batched serving system: if the energy difference between default and MINANS prompts is negligible under realistic batching, the claimed 25–60% energy savings would not transfer from the lab.

Watch

Extended reading notes

Core claim

The central claim is that, for objective factual queries, LLM-generated responses are systematically longer than the target (gold) responses, regardless of whether those targets are human-written or machine-generated. The paper supports this with a benchmark of 12 decoder-only LLMs across five datasets, showing generated lengths that range from roughly 1–3 times the target for GPT-3-family models to more than 10 times for Llama-3.1 and Phi-3 family models. It then provides a fine-grained diagnosis: a manual annotation of 500 responses into six information categories (minimal answer, additional helpful information, reasoning/explanations, conversational enhancements, redundant information, irrelevant information) reveals that the core answer constitutes only about 42% of the tokens, with roughly 18% irrelevant and 5.2% conversational tokens. The paper's actionable discovery is that simple, zero-shot prompt directives—especially 'only provide the minimal answer' (MINANS) and a learned length estimator (PRED-RESLEN)—reduce response length by about 60% and 53% respectively, lower inference energy by about 28% and 26%, and improve ROUGE-L F1 relative to default prompting. The authors argue that LLMs should default to minimal answers and let users ask for more detail when they want it.

Load-bearing premise

The load-bearing premise is that the gold or target answer length is a valid proxy for the minimal or sufficient response length; if users genuinely want the extra explanation, context, or conversational framing, the 'too long' diagnosis and the energy-saving prescription both lose force.

Editorial extensions

If this is right

  • Deployed assistants can adopt MINANS-style system prompts as a default for factual queries, cutting inference energy per query by roughly a quarter to a half with no measured quality loss.
  • The energy savings multiply at scale: because ChatGPT-scale systems process billions of queries daily, even a 25–30% per-query reduction translates into substantial absolute energy and carbon savings.
  • The finding that fine-tuning on gold answers slightly increases response length suggests that length control is better achieved at prompt or decoding time than by small-scale fine-tuning.
  • Models with the longest default responses (Llama-3.1, Phi-3 families) stand to gain the most from length directives, so prioritising prompt changes for those models yields the largest per-query savings.
  • The taxonomy itself—and the released annotated dataset—opens the door to automatic classifiers that label response spans, enabling continuous monitoring of verbosity in production.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: The results likely extend to reasoning LLMs, whose 'thinking' tokens dominate output length; a MINANS-style prompt that suppresses the reasoning trace could yield even larger energy savings, at the cost of losing the transparent chain of thought.
  • Inference: The energy numbers were measured offline with CodeCarbon on a single GPU; in real deployments with batching, caching, and speculative decoding, the per-token marginal energy may differ, so the exact percentages may not transfer even though the direction should.
  • Inference: The paper's own annotation data (annotator F-measure 0.764, GPT-4o automatic categorization at F1 0.391) shows the taxonomy is harder to apply automatically than manually, suggesting that a robust automatic information-category classifier would be a valuable next step.
  • Inference: A testable extension is to combine length directives with decoding-time constraints, such as early stopping once the minimal answer is complete or constrained decoding that bans conversational tokens, to push energy savings beyond the 28–60% range reported here.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper benchmarks 12 decoder-only LLMs on 5 factual QA datasets and reports that LLM outputs are substantially longer than the provided gold/target answers. It introduces a six-category annotation scheme for information content (minimal answer, additional helpful information, reasoning, conversational enhancement, redundant, irrelevant), reports category distributions, and evaluates several prompt-engineering strategies (BRIEF, BM25-INCONTEXT, LIMIT-LEN variants, MINANS, MADDNORED) for reducing output length, measured energy consumption, and ROUGE-L F1. The authors claim that appropriate prompts achieve 25–60% energy reduction while preserving or improving response quality, and argue that LLMs should default to minimal answers for factual queries.

Significance. If the central claim were fully validated, the paper would provide a useful empirical characterization of LLM verbosity and a simple, practical lever for reducing inference energy. Strengths include the breadth of the benchmark (12 models, 5 datasets), the public release of the manually annotated category dataset, and the analysis of multiple prompt strategies with energy measurements. However, the interpretive leap from 'longer than gold answers' to 'too long' is the load-bearing premise of the paper, and it is not validated; moreover, the quantitative claims about energy savings are internally inconsistent. The empirical length measurements themselves appear solid and reproducible, but the paper's headline conclusions currently outrun the evidence.

major comments (3)
  1. [Section 3.1–3.2] The central diagnostic claim that LLM responses are 'substantially longer than desired' rests on equating gold target length with minimal sufficient length. Section 3.1 states only that 'Human-written answers serve as the proxy of answers of reasonable lengths', which is not the same as minimal; GOOAQ and DOLLY contain procedural step-by-step answers that are legitimately long. The Limitations section concedes that 'The notion of an ideal response length is subjective and varies significantly among individuals.' Since the length-reduction and energy-saving prescription is premised on gold length being the right target, the 'too long' diagnosis is not entailed by the data. Please either validate the minimal-length assumption (e.g., with human judgments of minimality on the gold answers themselves) or substantially soften the interpretation.
  2. [Section 5.2 and Abstract] The reported energy and length reductions are internally inconsistent. The abstract claims '25-60%' energy optimization, the Introduction claims MINANS achieves 'up to 58% reduction in energy consumption', and Section 5.2 reports that 'MINANS emerges as most energy-efficient method with 28%' and PRED-RESLEN with 'about 26% reduction'. Similarly, the Introduction claims 'up to 88% reduction in LLM responses' whereas Section 5.2 reports 'about 60% decrease' for MINANS. Please reconcile these numbers, clearly distinguishing average vs. maximum and per-model vs. aggregate, and report the distribution across models and datasets.
  3. [Section 5.2, Fig. 4, Tables 8–10] The claim that prompt strategies 'preserve (or sometimes even improve) response quality' is not supported by ROUGE-L F1 alone. As the paper itself notes, shorter responses produce 'substantial improvements in ROUGE precision, with marginal drop in ROUGE recall', so the F1 gains are largely a mechanical consequence of length reduction rather than evidence of better informativeness. With no human or task-based evaluation of the prompted responses, the quality-preservation claim is not established. Please add a human or answer-correctness evaluation, or explicitly restrict the quality claim to 'ROUGE-L F1, which is length-confounded'.
minor comments (5)
  1. [Table 3] The header 'DS-R1- DS-R1- Gemma-2 Target' is garbled; it should read 'DeepSeek-R1-Distill-Llama-8B', 'DeepSeek-R1-Distill-Qwen-14B', 'Gemma-2-9B', and 'Target Answer'.
  2. [Section 3.4] There is a typo in 'energey' (should be 'energy').
  3. [Section 4] The phrase 'fine-tuning is leading to 1.24 times higher ratio' is ambiguous; clarify that fine-tuning increases the generated-to-target length ratio by a factor of 1.24 relative to the DEFAULT model.
  4. [Tables 1 and 2] The dataset name is written inconsistently as 'MS-M ARCO' and 'MS-MARC O'; use a single consistent spelling.
  5. [Section 5.1 and later] The strategy name appears in inconsistent forms ('MADDNORED', 'MA DDNORED', 'MAddNoRed'); unify the notation.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the length and energy results are empirical measurements against external gold answers and direct energy metering, with only minor, non-load-bearing self-citations.

full rationale

The paper's central claim (Section 3) is an empirical comparison: generated response lengths are measured in tokens against gold target lengths from five external datasets, and the conclusion that LLMs generate 'substantially longer than desired' responses follows from the paper's explicitly stated proxy that human-written target answers represent 'reasonable' lengths. That proxy is an assumption, not a derived result, so the diagnostic claim trades on an unvalidated normative baseline rather than on a circular reduction; the paper itself flags this in Limitations: 'The notion of an ideal response length is subjective and varies significantly among individuals.' Quality is checked with ROUGE-L F1 against the same external gold answers (Section 5.2, Appendix G), and energy is measured directly with CodeCarbon (Section 5.2, Appendix C), so neither the quality nor the energy findings reduce to the paper's own definitions. PRED-RESLEN is explicitly a supervised regressor trained on gold-answer lengths; evaluating how well it reproduces gold lengths on queries is a standard supervised evaluation, not a fitted input disguised as a prediction. The information-category analysis (Sections 3.3-3.4) is a new manual annotation with reported inter-annotator agreement, and the MINANS prompt result is an instruction-following measurement rather than a tautology. Self-citations to Poddar et al. (2025) supply the energy-length correlation premise, but that premise is also supported by an external citation (Luccioni et al., 2024) and, more importantly, is checked by the paper's own measured energy reductions, so the self-citations are not load-bearing in a way that forces the conclusions. Overall, no step in the derivation chain is equivalent to its own input by construction.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central claims rest on two unverified proxies: gold-answer length as the 'right' length and ROUGE-L as quality. The prompt strategies themselves introduce no free parameters except a hand-set repetition penalty and the supervised length regressor.

free parameters (4)
  • Repetition penalty = 1.2
    Chosen by hand in Section 3.1 to restrict repetitive text; affects generated lengths across all experiments.
  • DeBERTa-v3-large length regression weights = not reported
    PRED-RESLEN regressor trained on 25K train samples with target answer lengths; the fitted weights determine the length limits used in the LIMIT-LEN strategy.
  • BM25+ top-k for in-context examples = 10
    k=10 examples chosen by BM25+ similarity; the selection affects BM25-INCONTEXT and BM25-LENGTH results.
  • LoRA fine-tuning configuration = r=16, 100 epochs
    Used in the fine-tuning experiment on Phi-3-mini; not central but an experimental choice.
assumptions (3)
  • domain assumption Target answer length is a valid proxy for the minimal/sufficient response length on factual QA.
    Used throughout Section 3 to conclude LLMs are 'substantially longer than necessary'; acknowledged in Limitations as subjective.
  • domain assumption Inference energy is predominantly determined by output length, so output-length reduction translates to energy savings.
    Borrowed from authors' prior work (Poddar et al., 2025), cited in Section 1; central to the energy-reduction claims.
  • domain assumption ROUGE-L F1 against gold answers is an acceptable proxy for response quality.
    Used in Section 5.2 to claim quality is preserved/improved; the paper notes it is 'no substitute for a human evaluation'.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Brevity is the soul of sustainability: Characterizing LLM response lengths." pith.science (2026). https://pith.science/paper/YGYU2EVI

@misc{pith2026250608686,
  author       = {Pith},
  title        = {Pith review of: Brevity is the soul of sustainability: Characterizing LLM response lengths},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YGYU2EVI}},
  note         = {Machine review of arXiv:2506.08686}
}
read the original abstract

A significant portion of the energy consumed by Large Language Models (LLMs) arises from their inference processes; hence developing energy-efficient methods for inference is crucial. While several techniques exist for inference optimization, output compression remains relatively unexplored, with only a few preliminary efforts addressing this aspect. In this work, we first benchmark 12 decoder-only LLMs across 5 datasets, revealing that these models often produce responses that are substantially longer than necessary. We then conduct a comprehensive quality assessment of LLM responses, formally defining six information categories present in LLM responses. We show that LLMs often tend to include redundant or additional information besides the minimal answer. To address this issue of long responses by LLMs, we explore several simple and intuitive prompt-engineering strategies. Empirical evaluation shows that appropriate prompts targeting length reduction and controlling information content can achieve significant energy optimization between 25-60\% by reducing the response length while preserving the quality of LLM responses.

Figures

Figures reproduced from arXiv: 2506.08686 by the authors.

Figure 1
Figure 1. Target answer length vs LLM generated re [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Distribution of information categories in LLM responses. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Ratio of generated response length to target [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (4 more)
Figure 6
Figure 6. Figure 6: Distribution of information categories in re [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 5
Figure 5. Figure 5: Inference Energy for different models using [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 7
Figure 7. Figure 7: Target answer length vs LLM generated response length on different datasets [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]
Figure 8
Figure 8. Figure 8: Annotation examples depicting the information categories [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

39 extracted references · 13 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Jeff Barr. 2019. Amazon ec2 update--inf1 instances with aws inferentia chips for high performance cost-effective inferencing

    work page 2019

  4. [4]

    Iz Beltagy, Matthew E Peters, and Arman Cohan. 2020. Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150

    arXiv 2020

  5. [5]

    Luc \' a Bouza, Aur \'e lie Bugeau, and Lo \" c Lannelongue. 2023. How to estimate carbon footprint when training deep learning models? a guide and review. Environmental Research Communications, 5(11):115014

    work page 2023

  6. [6]

    Lingjiao Chen, Matei Zaharia, and James Zou. 2023. Frugalgpt: How to use large language models while reducing cost and improving performance. arXiv preprint arXiv:2305.05176

    arXiv 2023

  7. [7]

    Mike Conover, Matt Hayes, Ankit Mathur, Jianwei Xie, Jun Wan, Sam Shah, Ali Ghodsi, Patrick Wendell, Matei Zaharia, and Reynold Xin. 2023. https://www.databricks.com/blog/2023/04/12/dolly-first-open-commercially-viable-instruction-tuned-llm Free dolly: Introducing the world's first truly open instruction-tuned llm

    2023

  8. [8]

    Benoit Courty, Victor Schmidt, Sasha Luccioni, Goyal-Kamal, MarionCoutarel, Boris Feld, Jérémy Lecourt, LiamConnell, Amine Saboni, Inimaz, supatomic, Mathilde Léval, Luis Blanche, Alexis Cruveiller, ouminasara, Franklin Zhao, Aditya Joshi, Alexis Bogroff, Hugues de Lavoreille, Niko Laskaris, Edoardo Abati, Douglas Blank, Ziyao Wang, Armin Catovic, Marc Al...

    doi:10.5281/zenodo.11171501 2024

Show all 39 references
  1. [9]

    Sunhao Dai, Chen Xu, Shicheng Xu, Liang Pang, Zhenhua Dong, and Jun Xu. 2024. Bias and unfairness in information retrieval systems: New challenges in the llm era. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 6437--6447

    2024

  2. [10]

    Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher R \'e . 2022. Flashattention: Fast and memory-efficient exact attention with io-awareness. Advances in Neural Information Processing Systems, 35:16344--16359

    2022

  3. [11]

    Louise Deleger, Qi Li, Todd Lingren, Megan Kaiser, Katalin Molnar, Laura Stoutenborough, Michal Kouril, Keith Marsolo, Imre Solti, et al. 2012. Building gold standard corpora for medical natural language processing tasks. In AMIA Annual Symposium Proceedings, volume 2012, page...

    2012

  4. [12]

    Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. 2022. Gptq: Accurate post-training quantization for generative pre-trained transformers. arXiv preprint arXiv:2210.17323

    2022 arXiv

  5. [13]

    Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2021. https://openreview.net/forum?id=XPZIaotutsD Deberta: Decoding-enhanced bert with disentangled attention . In International Conference on Learning Representations

    2021

  6. [14]

    Mathilde Jay, Vladimir Ostapenco, Laurent Lef \`e vre, Denis Trystram, Anne-C \'e cile Orgerie, and Benjamin Fichel. 2023. An experimental comparison of software-based power meters: focus on cpu and gpu. In 2023 IEEE/ACM 23rd International Symposium on Cluster, Cloud and Inter...

    2023

  7. [15]

    Ziwei Ji, Tiezheng Yu, Yan Xu, Nayeon Lee, Etsuko Ishii, and Pascale Fung. 2023. Towards mitigating llm hallucination via self reflection. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 1827--1843

    2023

  8. [16]

    Huiqiang Jiang, Qianhui Wu, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. 2023 a . Llmlingua: Compressing prompts for accelerated inference of large language models. arXiv preprint arXiv:2310.05736

    2023 arXiv

  9. [17]

    Huiqiang Jiang, Qianhui Wu, Xufang Luo, Dongsheng Li, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. 2023 b . Longllmlingua: Accelerating and enhancing llms in long context scenarios via prompt compression. arXiv preprint arXiv:2310.06839

    2023 arXiv

  10. [18]

    Nitish Shirish Keskar, Bryan McCann, Lav R Varshney, Caiming Xiong, and Richard Socher. 2019. Ctrl: A conditional transformer language model for controllable generation. arXiv preprint arXiv:1909.05858

    2019 arXiv

  11. [19]

    Daniel Khashabi, Amos Ng, Tushar Khot, Ashish Sabharwal, Hannaneh Hajishirzi, and Chris Callison-Burch. 2021. Gooaq: Open question answering with diverse answer types. arXiv preprint

    2021

  12. [20]

    Tom \'a s Ko c isk \'y , Jonathan Schwarz, Phil Blunsom, Chris Dyer, Karl Moritz Hermann, G \'a bor Melis, and Edward Grefenstette. 2018. https://doi.org/10.1162/tacl_a_00023 The N arrative QA reading comprehension challenge . Transactions of the Association for Computational ...

    2018 doi

  13. [21]

    Eldar Kurti \'c , Elias Frantar, and Dan Alistarh. 2024. Ziplm: Inference-aware structured pruning of language models. Advances in Neural Information Processing Systems, 36

    2024

  14. [22]

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th Symposium on Operating Systems Princip...

    2023

  15. [23]

    Yaniv Leviathan, Matan Kalman, and Yossi Matias. 2023. Fast inference from transformers via speculative decoding. In International Conference on Machine Learning, pages 19274--19286. PMLR

    2023

  16. [24]

    Baolin Li, Yankai Jiang, Vijay Gadepally, and Devesh Tiwari. 2024. Toward sustainable genai using generation directives for carbon-friendly large language model inference. arXiv preprint arXiv:2403.12900

    2024 arXiv

  17. [25]

    Lorraine A Low, Jacqueline Fortier, William Mickalide, David Page, and Diane Troumpalos. 1973. Role of relevant, irrelevant, and redundant information in simple concept transfer. Bulletin of the Psychonomic Society, 1:267--269

    1973

  18. [26]

    Sasha Luccioni, Yacine Jernite, and Emma Strubell. 2024. Power hungry processing: Watts driving the cost of ai deployment? In The 2024 ACM Conference on Fairness, Accountability, and Transparency, pages 85--99

    2024

  19. [27]

    Potsawee Manakul, Adian Liusie, and Mark JF Gales. 2023. Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models. arXiv preprint arXiv:2303.08896

    2023 arXiv

  20. [28]

    Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016. https://arxiv.org/abs/1611.09268 MS MARCO: A human generated machine reading comprehension dataset . CoRR, abs/1611.09268

    2016 arXiv

  21. [29]

    David Patterson, Joseph Gonzalez, Urs H \"o lzle, Quoc Le, Chen Liang, Lluis-Miquel Munguia, Daniel Rothchild, David R So, Maud Texier, and Jeff Dean. 2022. The carbon footprint of machine learning training will plateau, then shrink. Computer, 55(7):18--28

    2022

  22. [30]

    Soham Poddar, Paramita Koley, Janardan Misra, Niloy Ganguly, and Saptarshi Ghosh. 2025. Towards sustainable nlp: Insights from benchmarking inference energy in large language models. arXiv preprint arXiv:2502.05610

    2025 arXiv

  23. [31]

    Guillem Ram \' rez, Matthias Lindemann, Alexandra Birch, and Ivan Titov. 2023. Cache & distil: Optimising api calls to large language models. arXiv preprint arXiv:2310.13561

    2023 arXiv

  24. [32]

    Jon Saad-Falcon, Amanpreet Singh, Luca Soldaini, Mike D'Arcy, Arman Cohan, and Doug Downey. 2022. Embedding recycling for language models. arXiv preprint arXiv:2207.04993

    2022 arXiv

  25. [33]

    Tal Schuster, Adam Fisch, Jai Gupta, Mostafa Dehghani, Dara Bahri, Vinh Tran, Yi Tay, and Donald Metzler. 2022. Confident adaptive language modeling. Advances in Neural Information Processing Systems, 35:17456--17472

    2022

  26. [34]

    Andrew Trotman, Antti Puurula, and Blake Burgess. 2014. Improvements to bm25 and language models examined. In Proceedings of the 19th Australasian Document Computing Symposium, pages 58--65

    2014

  27. [35]

    Zhichao Wang, Bin Bi, Shiva Kumar Pentyala, Kiran Ramnath, Sougata Chaudhuri, Shubham Mehrotra, Xiang-Bo Mao, Sitaram Asur, et al. 2024. A comprehensive survey of llm alignment techniques: Rlhf, rlaif, ppo, dpo and more. arXiv preprint arXiv:2407.16216

    2024 arXiv

  28. [36]

    Carole-Jean Wu, Ramya Raghavendra, Udit Gupta, Bilge Acun, Newsha Ardalani, Kiwan Maeng, Gloria Chang, Fiona Aga, Jinshi Huang, Charles Bai, et al. 2022. Sustainable ai: Environmental implications, challenges and opportunities. Proceedings of Machine Learning and Systems, 4:795--813

    2022

  29. [37]

    Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han. 2023. Smoothquant: Accurate and efficient post-training quantization for large language models. In International Conference on Machine Learning, pages 38087--38099. PMLR

    2023

  30. [38]

    Wenhan Xiong, Jiawei Wu, Hong Wang, Vivek Kulkarni, Mo Yu, Shiyu Chang, Xiaoxiao Guo, and William Yang Wang. 2019. https://doi.org/10.18653/v1/P19-1496 TWEETQA : A social media focused question answering dataset . In Proceedings of the 57th Annual Meeting of the Association fo...

    2019 doi

  31. [39]

    Gyeong-In Yu, Joo Seong Jeong, Geon-Woo Kim, Soojeong Kim, and Byung-Gon Chun. 2022. Orca: A distributed serving system for \ Transformer-Based \ generative models. In 16th USENIX Symposium on Operating Systems Design and Implementation (OSDI 22), pages 521--538

    2022

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.