Pith. sign in

REVIEW 3 cited by

Offline Energy-Optimal LLM Serving: Workload-Based Energy Models for LLM Inference on Heterogeneous Systems

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2407.04014 v1 pith:NDD676ED submitted 2024-07-04 cs.DC

Grant Wilkins , Srinivasan Keshav , Richard Mortier This is my paper · ORCID
classification cs.DC
keywords energymodelsinferenceruntimeenergy-optimalheterogeneouslanguagellms
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

The rapid adoption of large language models (LLMs) has led to significant advances in natural language processing and text generation. However, the energy consumed through LLM model inference remains a major challenge for sustainable AI deployment. To address this problem, we model the workload-dependent energy consumption and runtime of LLM inference tasks on heterogeneous GPU-CPU systems. By conducting an extensive characterization study of several state-of-the-art LLMs and analyzing their energy and runtime behavior across different magnitudes of input prompts and output text, we develop accurate (R^2>0.96) energy and runtime models for each LLM. We employ these models to explore an offline, energy-optimal LLM workload scheduling framework. Through a case study, we demonstrate the advantages of energy and accuracy aware scheduling compared to existing best practices.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Scalable Joint Resource Allocation for SLO-Constrained LLM Inference in Heterogeneous GPU Clouds

    cs.LG 2026-04 unverdicted novelty 6.0 of 10

    Two constraint-aware greedy heuristics (GH and AGH) solve mixed-scale LLM allocation on heterogeneous GPUs under SLO constraints in under one second with over 260x speedup and near-optimal cost compared to exact MILP.

  2. Engineering AI Judge Systems

    cs.SE 2024-11 conditional novelty 5.0 of 10

    A constitution-based, search-driven development framework for AI judge systems improves judged accuracy by up to 6.2% on commit message generation, with about 58% of general principles reused across five languages.

  3. Addressing the sustainable AI trilemma: a case study on LLM agents and RAG

    cs.CY 2025-01 conditional novelty 4.0 of 10

    LLM-dependent memory operations in agents and RAG consume orders of magnitude more energy than vector methods, and resource-constrained hardware pays higher energy for lower quality.

Pith tools