pith. sign in

arxiv: 2606.21833 · v1 · pith:EOY7L3MJnew · submitted 2026-06-20 · 📡 eess.SY · cs.SY

Inference as Flexibility: Ramp Management for Transmission-Connected AI Data Centres

Zhirui Liang This is my paper

Pith reviewed 2026-06-26 12:00 UTC · model grok-4.3

classification 📡 eess.SY cs.SY
keywords AI data centerspower ramp managementLLM inference flexibilitybattery energy storageramp rate limitshybrid control strategytransmission-connected loadsflexible computing loads
0
0 comments X

The pith

Coordinated batch-size control in LLM inference can reduce battery storage needs for AI data center power ramps by over 70 percent while meeting ramp limits.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper shows that large language model inference power can be adjusted endogenously by varying batch sizes to help control rapid load changes from training in transmission-connected data centers. This software layer works alongside battery energy storage to limit point-of-connection ramps to 10 MW per minute in a modeled 150 MW facility. Simulations using measured LLaMA power traces find that the combined approach cuts battery discharge energy by 71 percent and peak power by 51 percent compared with storage-only mitigation. A reader would care because growing AI loads create new grid challenges that hardware solutions alone may not scale to address efficiently.

Core claim

In a 150 MW transmission-connected data center with training, inference, and base load, measured LLaMA-2-70B fine-tuning profiles scaled for aggregate training and LLaMA-3.1-70B inference traces enable a hybrid batch-size plus BESS strategy that achieves near-complete compliance with a 10 MW/min ramp limit while substantially lowering storage requirements relative to BESS-only or batch-size-only controls.

What carries the argument

Batch-size-dependent inference power flexibility, drawn from measured LLaMA-3.1-70B traces, coordinated with BESS to offset training-induced ramps.

Load-bearing premise

That inference power consumption can be adjusted rapidly and reliably by changing batch sizes in a live serving system without breaking service level agreements.

What would settle it

A live test applying dynamic batch-size changes during an actual training power ramp and measuring whether the resulting facility output stays within the 10 MW/min limit without SLA violations or accuracy loss.

Figures

Figures reproduced from arXiv: 2606.21833 by Zhirui Liang.

Figure 1
Figure 1. Figure 1: Measured GPU workload characteristics: training power ramp and [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
read the original abstract

The rapid growth of large AI data centres introduces new operational challenges for power systems, including rapid ramping, oscillatory load behavior, voltage fluctuations, and supply-demand balancing impacts. For example, the Alberta Electric System Operator (AESO) has identified transmission-connected data centres (TCDCs) as large non-conforming loads that may need to limit their point-of-connection ramp rates. Existing mitigation approaches mainly rely on exogenous electrical resources, such as battery energy storage systems (BESS). This paper presents a proof-of-concept demonstration of a complementary software-defined mitigation layer: using flexible large language model (LLM) inference serving as endogenous TCDC flexibility to partially offset AI training power ramps. We consider a 150 MW TCDC with training, inference, and base-load components. A measured LLaMA-2-70B fine-tuning power profile is scaled to represent an aggregate training block, while measured LLaMA-3.1-70B inference power traces are used to model batch-size-dependent inference flexibility. Three strategies are compared: BESS-only mitigation, batch-size-only control, and coordinated batch-size plus BESS control. Simulation results show that the hybrid strategy reduces BESS discharge energy by 71% and peak discharge power by 51%, while maintaining near-complete compliance with a 10 MW/min ramp limit.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper presents a proof-of-concept simulation for a 150 MW transmission-connected AI data centre combining training, inference, and base load. It models a measured LLaMA-2-70B fine-tuning profile for training and LLaMA-3.1-70B traces for batch-size-dependent inference flexibility. Three mitigation strategies for a 10 MW/min ramp limit are compared: BESS-only, batch-size-only, and coordinated hybrid control. The central claim is that the hybrid approach reduces BESS discharge energy by 71% and peak discharge power by 51% while achieving near-complete ramp compliance.

Significance. If the modeled inference flexibility proves realizable under production constraints, the work demonstrates a software-defined endogenous resource that can materially reduce the scale of exogenous BESS required for ramp management at transmission-connected data centres. This is a timely contribution given AESO's identification of TCDCs as non-conforming loads and the rapid growth of AI infrastructure.

major comments (2)
  1. [Modeling of batch-size-dependent inference flexibility (abstract and simulation setup)] The headline quantitative results (71% BESS energy reduction, 51% peak power reduction) rest on the assumption that batch-size changes can produce the required power trajectories on ramp timescales. The modeling section provides no quantification of resulting end-to-end latency, tail latency, throughput under realistic arrivals, or model output quality; without this, the hybrid benefit cannot be distinguished from the BESS-only case.
  2. [Simulation results and strategy comparison] The simulation treats the inference power traces as directly controllable flexibility. No queuing model, request dropping policy, or SLA violation metric is reported, so it is unclear whether the assumed power reductions remain feasible when inference servers must maintain production service levels.
minor comments (2)
  1. [Data and scaling] Clarify the exact scaling procedure used to aggregate the LLaMA-3.1-70B traces to the 150 MW TCDC inference component.
  2. [Results] Add a table or figure showing the ramp-rate violation metric (e.g., integral of exceedance or number of minutes above limit) for all three strategies to make the 'near-complete compliance' claim quantitative.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive and detailed comments. The points raised correctly identify that the current proof-of-concept relies on measured power traces without deeper modeling of serving-system metrics. We address each comment below and will revise the manuscript to add explicit discussion of assumptions and limitations.

read point-by-point responses

  1. Referee: The headline quantitative results (71% BESS energy reduction, 51% peak power reduction) rest on the assumption that batch-size changes can produce the required power trajectories on ramp timescales. The modeling section provides no quantification of resulting end-to-end latency, tail latency, throughput under realistic arrivals, or model output quality; without this, the hybrid benefit cannot be distinguished from the BESS-only case.

    Authors: We agree that the headline results depend on the power trajectories observed in the measured LLaMA-3.1-70B traces at different batch sizes. The manuscript does not derive or report latency, tail latency, throughput, or quality metrics because its scope is limited to demonstrating power flexibility from those traces. In revision we will add a new subsection in the modeling section that (i) states the assumption that batch-size changes are feasible on ramp timescales, (ii) cites literature on batch-size effects, and (iii) explicitly notes that full end-to-end latency and quality evaluation lies outside the present power-system focus. This will clarify the distinction from the BESS-only case while preserving the reported energy and power reductions under the modeled traces. revision: yes

  2. Referee: The simulation treats the inference power traces as directly controllable flexibility. No queuing model, request dropping policy, or SLA violation metric is reported, so it is unclear whether the assumed power reductions remain feasible when inference servers must maintain production service levels.

    Authors: The simulation applies the measured power traces directly as controllable inputs for the batch-size strategy. We acknowledge that this omits a queuing model, request-dropping policy, and SLA metrics, which would be required to confirm feasibility under production arrivals. In the revised manuscript we will expand the simulation-results discussion to (i) state that the traces represent observed power under batch-size changes, (ii) note the absence of an explicit queuing model, and (iii) outline how production systems would need to integrate batch-size control with request schedulers to respect SLAs. The core quantitative comparison of BESS energy and peak power will remain unchanged. revision: partial

Circularity Check

0 steps flagged

No circularity: simulation comparison of independent control strategies

full rationale

The paper conducts a direct simulation comparison of three strategies (BESS-only, batch-size-only, hybrid) on scaled measured power traces for a 150 MW TCDC. No derivation chain, equations, or first-principles results are presented that reduce to their own inputs by construction. The reported reductions (71% BESS energy, 51% peak power) are simulation outputs under explicit modeling assumptions about batch-size flexibility, not fitted parameters or self-referential definitions renamed as predictions. No self-citations, uniqueness theorems, or ansatzes are invoked as load-bearing steps. The work is self-contained as a proof-of-concept simulation study.

Axiom & Free-Parameter Ledger

1 free parameters · 2 axioms · 0 invented entities

The central claim depends on assumptions about data representativeness and controllability, which are not independently verified in the abstract.

free parameters (1)
  • Ramp rate limit = 10 MW/min
    The compliance target based on AESO requirements for TCDCs.
axioms (2)
  • domain assumption Measured power profiles from LLaMA models are representative and scalable to the 150 MW aggregate TCDC load.
    Used to model the training and inference power components in the simulation.
  • domain assumption Inference batch size can be dynamically adjusted to control power consumption in real time.
    Foundation for the batch-size control strategy.

pith-pipeline@v0.9.1-grok · 5762 in / 1268 out tokens · 57143 ms · 2026-06-26T12:00:53.231366+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

14 extracted references · 6 canonical work pages · 3 internal anchors

  1. [1]

    Power stabilization for ai training datacenters,

    E. Choukse, B. Warrier, S. Heath, L. Belmont, A. Zhao, H. A. Khan, B. Harry, M. Kappel, R. J. Hewett, K. Datta,et al., “Power stabilization for AI training datacenters,”arXiv preprint arXiv:2508.14318, 2025

    work page arXiv 2025

  2. [2]

    Guide to AESO connection require- ments for transmission-connected data centres,

    Alberta Electric System Operator, “Guide to AESO connection require- ments for transmission-connected data centres,” June 2026. Guidance document, issued June 12, 2026

    2026

  3. [3]

    Strengthening data center operations through battery energy storage systems,

    S. Morovati, A. Azizi, A. Zamani, J. Piruzza, D. Guo, Z. Liu, and P. Taimela, “Strengthening data center operations through battery energy storage systems,”Available at SSRN 5583916

  4. [4]

    Mitigation of datacenter demand ramping and fluctuation using hybrid ess and supercapacitor,

    M.-S. Ko, J. W. Shim, and H. Zhu, “Mitigation of datacenter demand ramping and fluctuation using hybrid ess and supercapacitor,”arXiv preprint arXiv:2512.08076, 2025

    work page arXiv 2025

  5. [5]

    Power system dispatch with marginal degradation cost of battery storage,

    G. He, S. Kar, J. Mohammadi, P. Moutis, and J. F. Whitacre, “Power system dispatch with marginal degradation cost of battery storage,”IEEE Transactions on Power Systems, vol. 36, no. 4, pp. 3552–3562, 2020

    2020

  6. [6]

    GPU- to-grid: V oltage regulation via GPU utilization control,

    Z. Liang, J.-W. Chung, M. Chowdhury, J. Chen, and V . Dvorkin, “GPU- to-grid: V oltage regulation via GPU utilization control,”arXiv preprint arXiv:2602.05116, 2026

    work page arXiv 2026

  7. [7]

    OpenG2G: A Simulation Platform for AI Datacenter-Grid Runtime Coordination

    J.-W. Chung, Z. Liang, Y . Mao, J. Chen, M. Chowdhury, and V . Dvorkin, “OpenG2G: A simulation platform for AI datacenter-grid runtime coor- dination,”arXiv preprint arXiv:2605.05519, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  8. [8]

    Measurement of Generative AI Workload Power Profiles for Whole-Facility Data Center Infrastructure Planning

    R. Vercellino, J. Willard, G. Campos, W. d. S. Pereira, O. Hull, M. Selensky, and J. Mueller, “Measurement of generative AI workload power profiles for whole-facility data center infrastructure planning,” arXiv preprint arXiv:2604.07345, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  9. [9]

    Dataset of generative AI workload power profiles

    “Dataset of generative AI workload power profiles.” https://data.nlr.gov/submissions/312, 2026

    2026

  10. [10]

    NVIDIA H100 tensor core GPU

    NVIDIA, “NVIDIA H100 tensor core GPU.” Product documentation,

  11. [11]

    The ML.ENERGY benchmark: Toward automated inference energy measurement and optimization,

    J.-W. Chung, J. J. Ma, R. Wu, J. Liu, O. J. Kweon, Y . Xia, Z. Wu, and M. Chowdhury, “The ML.ENERGY benchmark: Toward automated inference energy measurement and optimization,” inNeurIPS Datasets and Benchmarks, 2025

    2025

  12. [12]

    The ML.ENERGY benchmark

    “The ML.ENERGY benchmark.” https://github.com/ml-energy/ benchmark

  13. [13]

    PALS: Power-Aware LLM Serving for Mixture-of-Experts Models

    C. Hankendi, R. Shahout, M. Yu, and A. K. Coskun, “PALS: Power- aware LLM serving for mixture-of-experts models,”arXiv preprint arXiv:2605.21427, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  14. [14]

    AESO (Alberta) power market data center permits and public records

    “AESO (Alberta) power market data center permits and public records.” https://www.datacenter.fyi/power-market/aeso