FAIR+S extends FAIR with sustainability metrics and is validated via expert survey confirming importance but revealing awareness gaps in green practices.
Scott Gardner, Itay Hubara, Sachin Idgunji, Thomas B
8 Pith papers cite this work. Polarity classification is still indexing.
8
Pith papers citing it
abstract
Machine-learning (ML) hardware and software system demand is burgeoning. Driven by ML applications, the number of different ML inference systems has exploded. Over 100 organizations are building ML inference chips, and the systems that incorporate existing models span at least three orders of magnitude in power consumption and five orders of magnitude in performance; they range from embedded devices to data-center solutions. Fueling the hardware are a dozen or more software frameworks and libraries. The myriad combinations of ML hardware and ML software make assessing ML-system performance in an architecture-neutral, representative, and reproducible manner challenging. There is a clear need for industry-wide standard ML benchmarking and evaluation criteria. MLPerf Inference answers that call. In this paper, we present our benchmarking method for evaluating ML inference systems. Driven by more than 30 organizations as well as more than 200 ML engineers and practitioners, MLPerf prescribes a set of rules and best practices to ensure comparability across systems with wildly differing architectures. The first call for submissions garnered more than 600 reproducible inference-performance measurements from 14 organizations, representing over 30 systems that showcase a wide range of capabilities. The submissions attest to the benchmark's flexibility and adaptability.
citation-role summary
background 1
citation-polarity summary
years
2026 8roles
background 1polarities
background 1representative citing papers
SLO-Guard improves tuning budget consistency for SLO-constrained LLM serving by handling crashes explicitly and using a two-phase feasible-first exploration plus exploitation strategy.
EMC-blind GPU-only latency fits miss 25–28% of tight deadlines on Jetson Orin; an EMC-aware two-cell refit holds misses ≤1.3% under a 2% QoS budget and selects a budget-feasible clock.
Quantitative benchmarks across recent AI accelerators reveal that optimal hardware choice varies with workload parameters and that several platforms incur substantially higher idle power than GPUs.
Watt Counts supplies over 5,000 energy measurements across 50 LLMs and 10 GPUs and shows that hardware-aware selection can reduce server-scenario energy use by up to 70 percent with little effect on user experience.
SLO-Guard, a crash-aware two-phase autotuner for vLLM serving, achieves no best-latency improvement over random search but demonstrates more consistent budget allocation across 150 trials on Qwen2-1.5B/A100.
Nvidia achieves 1.6x throughput with NVFP4 but hits a VRAM wall for 70B+ models, while Apple UMA enables linear scaling to 80B at 4-bit with up to 23x better energy efficiency.
citing papers explorer
-
FAIR+S: A validation study of a framework for sustainable research data and software
FAIR+S extends FAIR with sustainability metrics and is validated via expert survey confirming importance but revealing awareness gaps in green practices.
-
SLO-Guard: Crash-Aware, Budget-Consistent Autotuning for SLO-Constrained LLM Serving
SLO-Guard improves tuning budget consistency for SLO-constrained LLM serving by handling crashes explicitly and using a two-phase feasible-first exploration plus exploitation strategy.
-
Edge-Inference Governors Need Memory-Clock State
EMC-blind GPU-only latency fits miss 25–28% of tight deadlines on Jetson Orin; an EMC-aware two-cell refit holds misses ≤1.3% under a 2% QoS budget and selects a budget-feasible clock.
-
The xPU-athalon: Quantifying the Competition of AI Acceleration
Quantitative benchmarks across recent AI accelerators reveal that optimal hardware choice varies with workload parameters and that several platforms incur substantially higher idle power than GPUs.
-
Watt Counts: Energy-Aware Benchmark for Sustainable LLM Inference on Heterogeneous GPU Architectures
Watt Counts supplies over 5,000 energy measurements across 50 LLMs and 10 GPUs and shows that hardware-aware selection can reduce server-scenario energy use by up to 70 percent with little effect on user experience.
-
Copy First, Translate Later: Interpreting Translation Dynamics in Multilingual Pretraining
SLO-Guard, a crash-aware two-phase autotuner for vLLM serving, achieves no best-latency improvement over random search but demonstrates more consistent budget allocation across 150 trials on Qwen2-1.5B/A100.
-
Silicon Showdown: Performance, Efficiency, and Ecosystem Barriers in Consumer-Grade LLM Inference
Nvidia achieves 1.6x throughput with NVFP4 but hits a VRAM wall for 70B+ models, while Apple UMA enables linear scaling to 80B at 4-bit with up to 23x better energy efficiency.
- A Switch-Centric In-Network Architecture for Accelerating LLM Inference in Shared-Memory Network