REVIEW 3 major objections 4 minor 2 cited by
This paper demonstrates that the batch size of LLM inference workloads can regulate distribution-grid voltage in real time, reducing integral voltage violations by orders of magnitude without tap changer operations.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-03 04:20 UTC pith:WFJI6THJ
load-bearing objection Solid framework, unproven numbers: GPU batch-size control for voltage regulation is a real idea, but the headline 'orders of magnitude' rests on a self-consistent simulation loop. the 3 major comments →
GPU-to-Grid: Voltage Regulation via GPU Utilization Control
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central discovery is that batch size is an effective and fast-acting control knob for grid support: across five heterogeneous LLM models spanning dense and mixture-of-experts architectures, larger batch sizes monotonically increase GPU power, inter-token latency, and token throughput, following logistic saturation curves. This monotonic, predictable response makes GPU power controllable at sub-second timescales. The paper's key counterintuitive insight is bidirectionality: cutting GPU power relieves undervoltage, but increasing GPU power mitigates overvoltage, so minimizing data-center power consumption is not always grid-beneficial. The proposed online feedback optimization controller u
What carries the argument
The central mechanism is the online feedback optimization (OFO) loop, a controller that iteratively adjusts decisions using real-time measurements rather than a fixed system model. At each control interval it updates dual variables for voltage and latency constraints from live measurements, then performs a projected gradient descent on the Lagrangian of the throughput-maximization problem in the log-scale batch size domain. The gradient couples three sensitivities: the token-throughput and inter-token-latency derivatives of the logistic fits, and the voltage sensitivity from a three-phase linearized distribution flow approximation (LinDistFlow) that maps data-center active-power changes to b
Load-bearing premise
The load-bearing premise is that the logistic curves fitted to real GPU measurements accurately capture real GPU dynamics under transient loads; both the controller's gradient and the simulated plant draw on these same fitted models, and the paper defers hardware-in-the-loop validation to future work.
What would settle it
Run the same OFO controller on a real GPU cluster driving a hardware-in-the-loop distribution feeder with live voltage feedback; the central claim is falsified if the integral voltage violation is not reduced by at least an order of magnitude relative to no control, or if batch-size adjustments push inter-token latency past its threshold for sustained periods.
If this is right
- If the simulation result holds, data centers running LLM inference can provide fast voltage support with no additional hardware; the batch-size knob acts on a one-second control interval, far faster than the 30-minute minimum dwell time of tap changers.
- Grid operators may sometimes request that data centers raise power consumption—for instance to counteract overvoltage during high renewable generation—so treating energy efficiency as always grid-beneficial is wrong.
- The OFO controller requires only local voltage and latency measurements from the data center's grid connection, not a detailed feeder model, making the control scheme portable across distribution networks.
- Batch-size control respects per-model latency thresholds, so the grid service can be provided without degrading the user experience of streaming LLM responses.
- The framework applies across LLM architectures and scales because the monotonic logistic relationship between batch size and power/latency/throughput holds consistently for dense and mixture-of-experts models.
Where Pith is reading between the lines
- If the approach survives hardware-in-the-loop testing, a data center's existing inference load could become a revenue-generating grid asset; the per-GPU flexibility is small, so value depends on aggregating hundreds of servers and on utilities having the metering and control infrastructure to exploit it.
- The paper does not account for the monetary and carbon cost of deliberately increasing GPU power for overvoltage support; a lifecycle assessment could weigh whether the grid benefit justifies the extra energy and possible emissions.
- Because the controller's gradient and the simulated plant share the same logistic fits, a stress test that injects unmodeled dynamics—power overshoots, memory-bound saturation, heavy-tailed latency—into the plant while keeping the controller's model unchanged would reveal how much of the 'orders of magnitude' improvement relies on the simulation being self-consistent.
- The same OFO structure could be repurposed for other grid services, such as frequency regulation or peak shaving, by swapping the voltage measurement for frequency or price signals; batch size would then trade token throughput against the corresponding grid objective.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a GPU-to-Grid (G2G) framework in which the batch size of LLM inference workloads is used as a fast control knob for distribution-level voltage regulation. Logistic functions are fitted to ML.ENERGY measurements to model GPU power, inter-token latency, and token throughput as functions of log2 batch size. These models are embedded in a constrained throughput-maximization problem with voltage and latency constraints, and an online feedback optimization (OFO) controller updates batch sizes using real-time voltage and latency measurements. The numerical study couples a cluster simulator, built from ML.ENERGY data, with an OpenDSS model of the IEEE 13-bus feeder. Table III reports that GPU-based control reduces integral voltage violation from 31.57 pu·s (or 43.74 pu·s with taps) to 0.0570 pu·s, i.e., by orders of magnitude, without tap operations. The conclusion explicitly states that hardware-in-the-loop validation is future work.
Significance. If the quantitative claims survive hardware-in-the-loop or otherwise independent validation, the paper would demonstrate a novel, fast, and physically grounded demand-side flexibility resource: GPU batch-size control for distribution voltage support. The use of real ML.ENERGY measurement data and OpenDSS for the grid side is a genuine strength, as is the OFO formulation, which requires only local measurements and avoids relying on exact grid models. The conceptual point that increasing GPU power can help mitigate overvoltage is a useful correction to the common 'minimize power always' assumption. However, the headline 'orders of magnitude' result is currently an in-sample simulation claim rather than an externally validated finding, and it should be tempered or supported by additional validation.
major comments (3)
- [§II-B and §II-C, Eqs. (2)–(3)] The paper states in §II-B that token throughput equals batch size divided by inter-token latency, but then fits l(x) and r(x) as independent logistic functions of x = log2 b. These two assumptions are not mutually consistent: if l(x) saturates as x grows, then r(x) = 2^x / l(x) grows without bound, contradicting the saturating logistic throughput in Eq. (3) and Fig. 3. Because Eq. (18) uses derivatives of both fitted models, the OFO gradient may be miscalibrated with respect to the actual throughput–latency trade-off. Please either fit the two models jointly under the stated identity, or explicitly justify why the identity is only approximate and why independent logistic fits are appropriate.
- [§IV-A, Table III, and §V] The central quantitative result in Table III is an in-sample simulation. The controller's logistic model (Eqs. (1)–(3), used in Eq. (18)) and the cluster simulator in §IV-A are calibrated on the same ML.ENERGY data. If the simulator actually uses the fitted logistic functions, the plant and controller share the same static map and the 'orders of magnitude' reduction is partly self-validating. If, as the text in §IV-A suggests, the simulator uses raw measured power traces and lognormal ITL mixtures, the current wording is ambiguous and the evaluation is still not independent of the calibration data. The paper should (i) report fit diagnostics and error bars for Eqs. (1)–(3), (ii) state explicitly what the simulator uses (raw traces vs. fitted curves), and (iii) include a robustness test with perturbed logistic parameters, a held-out model, or a hardware-in-the-loop experiment. The conclus
- [§III-B and §IV-C] The paper emphasizes that OFO is robust to modeling inaccuracies, but the simulation does not exercise that robustness. The controller receives exact voltage measurements, knows the phase-allocation vector and sensitivity matrix H, and the plant is generated from the same data used to fit the controller's model. No measurement noise, actuation delay, or error in H is simulated, and no sensitivity analysis with respect to the hyperparameters ρ_x, ρ_v, ρ_l, γ is provided. A sensitivity study around these quantities, or a misspecified-H case, is needed to support the claim that the control scheme is effective under realistic model mismatch rather than only in the nominal simulated setting.
minor comments (4)
- [§IV-A] The description of the cluster simulator is underspecified: it is not clear whether the 'replica-level GPU power traces' are raw ML.ENERGY traces or synthetic traces generated from the logistic fits. Please clarify this in the text, and specify the number of random seeds / Monte Carlo repetitions used for the reported statistics.
- [Table III] All performance metrics are point estimates with no variance or confidence intervals. Given the stochasticity in ITL sampling and request arrivals, reporting standard deviations over multiple simulation runs would strengthen the comparison.
- [Throughout] There are minor typographical issues ('V oltage' in figure captions, 'batchrequests' in §II-B) and the 'green shaded region' in the text around Fig. 9 is not identifiable in the monochrome manuscript. Please check consistency.
- [§IV-B] The constant power factor PF = 0.95 is used for all phases and all control actions. A brief justification or sensitivity check regarding PF would be helpful, since reactive-power coupling can affect voltage sensitivity.
Circularity Check
No significant circularity: the paper's voltage-regulation result is a closed-loop simulation built on external measurements and OpenDSS power-flow physics; self-citations are not load-bearing.
full rationale
The paper's central claim—that GPU batch size control reduces integral voltage violation by orders of magnitude (Table III)—is not circular. The fitted logistic models in Eqs. (1)-(3) are empirical relationships obtained by averaging ML.ENERGY measurements, and the OFO gradient in Eq. (18) uses derivatives of these fits. However, the cluster simulator in Section IV-A is constructed from superimposed replica-level GPU power traces and heavy-tailed ITL mixtures fitted to historical measurements, not from the logistic power model alone. Voltages are then computed by OpenDSS on the IEEE 13-bus feeder, an external power-flow solver. The OFO controller receives real-time feedback from this independently simulated plant, so the voltage violation reduction is not equal by construction to the controller's internal model. The conclusion explicitly states the limitation that dynamics are generated from pre-measured traces and fitted models, and that hardware-in-the-loop validation is future work; this is a model-validation caveat, not evidence of circularity. Author self-citations (e.g., ML.ENERGY [10], [13]) serve as data/benchmark sources with independent measurement provenance, and other self-citations are related work; none is invoked as a uniqueness theorem or used to forbid alternative explanations. Therefore, no load-bearing circular step is present, and the derivation remains self-contained against external benchmarks.
Axiom & Free-Parameter Ledger
free parameters (7)
- Power logistic parameters per model (Pmax, kp, x0,p, p0)
- Latency logistic parameters per model (Lmax, kl, x0,l, l0)
- Throughput logistic parameters per model (Rmax, kr, x0,r, r0)
- ITL lognormal mixture parameters per batch size
- OFO hyperparameters rho_x, rho_v, rho_l, gamma =
0.1, 1, 1, 0.1
- Data center power factor PF =
0.95
- Scenario configuration: replica counts, latency thresholds, base loads, workload timings
axioms (7)
- domain assumption Three-phase LinDistFlow linear approximation for voltage magnitudes
- domain assumption Constant power factor for the data center load
- domain assumption Replica-based linear scaling of power and throughput
- ad hoc to paper Logistic functional forms for power, latency, and throughput
- domain assumption Token-throughput identity r = b/l
- domain assumption Immediate batch-size actuation
- domain assumption Tap-changer operating assumptions in the baseline
read the original abstract
While the rapid expansion of data centers poses challenges for power grids, it also offers new opportunities as flexible loads. Existing power system research often abstracts data centers as aggregate resources, while computer system research focuses on GPU energy efficiency and largely ignores grid impacts. To bridge this gap, we develop a GPU-to-Grid framework that couples device-level GPU control with power system objectives. We study distribution-level voltage regulation enabled by LLM inference flexibility, using batch size as a data-center-side control knob that trades off GPU power consumption, inference latency, and token throughput. We first formulate the problem as an optimization problem and then realize it as an online feedback optimization controller, implemented by the data center operator using its own empirical GPU power-performance model and real-time measurements from both the GPU and grid systems. Our key insight is that reducing GPU power alleviates lower-voltage violations, while increasing GPU power mitigates upper-voltage violations; this challenges the common belief that minimizing GPU power is always beneficial to power grids.
Figures
Forward citations
Cited by 2 Pith papers
-
Inference as Flexibility: Ramp Management for Transmission-Connected AI Data Centres
Hybrid control of inference batch sizes and BESS reduces BESS energy discharge by 71% and peak power by 51% for a 150 MW TCDC while complying with 10 MW/min ramp limits.
-
Grid Integration of AI Data Centers: A Critical Review of Energy Storage Solutions
A hierarchical review of energy storage technologies for smoothing the sub-second variable loads of AI data centers on the utility grid.
Reference graph
Works this paper leans on
-
[1]
United states data center energy usage report,
Masanet, Eric and Shehabi, Arman and Lei, Ning and Smith, Sarah and Koomey, Jonathan, “United states data center energy usage report,” Technical Report LBNL-2024-DataCenterReport, Lawrence Berkeley National Laboratory, 2024
2024
-
[2]
Nvidia h100 tensor core gpu
NVIDIA Corporation, “Nvidia h100 tensor core gpu.” https://www. nvidia.com/en-us/data-center/h100/, 2024
2024
-
[3]
Electricity demand and grid impacts of ai data centers: Challenges and prospects,
X. Chen, X. Wang, A. Colacelli, M. Lee, and L. Xie, “Electricity demand and grid impacts of ai data centers: Challenges and prospects,”arXiv preprint arXiv:2509.07218, 2025
Pith/arXiv arXiv 2025
-
[4]
A conceptual framework for the vehicle-to-grid (v2g) implementation,
C. Guille and G. Gross, “A conceptual framework for the vehicle-to-grid (v2g) implementation,”Energy policy, vol. 37, no. 11, pp. 4379–4390, 2009
2009
-
[5]
Agent coordination via contextual regression (agentconcur) for data center flexibility,
V . Dvorkin, “Agent coordination via contextual regression (agentconcur) for data center flexibility,”IEEE Transactions on Power Systems, 2024
2024
-
[6]
Assessments of data centers for provision of frequency regulation,
Y . Fu, X. Han, K. Baker, and W. Zuo, “Assessments of data centers for provision of frequency regulation,”Applied Energy, vol. 277, p. 115621, 2020
2020
-
[7]
Enhancing data center low-voltage ride-through,
Y . Xie, W. Cui, and A. Wierman, “Enhancing data center low-voltage ride-through,”arXiv preprint arXiv:2510.03867, 2025
arXiv 2025
-
[8]
Zeus: Understanding and optimizing gpu energy consumption of dnn training,
J. You, J.-W. Chung, and M. Chowdhury, “Zeus: Understanding and optimizing gpu energy consumption of dnn training,” in20th USENIX Symposium on Networked Systems Design and Implementation (NSDI 23), pp. 119–139, 2023
2023
-
[9]
Reducing energy bloat in large model training,
J.-W. Chung, Y . Gu, I. Jang, L. Meng, N. Bansal, and M. Chowdhury, “Reducing energy bloat in large model training,”Proceedings of the 30th ACM Symposium on Operating Systems Principles, 2024
2024
-
[10]
The ML.ENERGY benchmark: Toward automated inference energy measurement and optimization,
J.-W. Chung, J. J. Ma, R. Wu, J. Liu, O. J. Kweon, Y . Xia, Z. Wu, and M. Chowdhury, “The ML.ENERGY benchmark: Toward automated inference energy measurement and optimization,” inNeurIPS Datasets and Benchmarks, 2025
2025
-
[11]
V oltage regulation in distribution systems with data center loads,
Y . Chen and B. Zhang, “V oltage regulation in distribution systems with data center loads,”arXiv preprint arXiv:2507.06416, 2025
Pith/arXiv arXiv 2025
-
[12]
Ai data centres as grid-interactive assets,
P. Colangelo, A. K. Coskun, J. Megrue, C. Roberts, S. Sengupta, V . Sivaram, E. Tiao, A. Vijaykar, C. Williams, D. C. Wilson,et al., “Ai data centres as grid-interactive assets,”Nature Energy, pp. 1–8, 2025
2025
-
[13]
The ML.ENERGY benchmark
“The ML.ENERGY benchmark.” https://github.com/ml-energy/ benchmark
-
[14]
An open source platform for collaborating on smart grid research,
R. C. Dugan and T. E. McDermott, “An open source platform for collaborating on smart grid research,” tech. rep., Electric Power Research Institute (EPRI), 2011
2011
-
[15]
Efficient memory management for large language model serving with PagedAttention,
W. Kwon, Z. Li, S. Zhuang, Y . Sheng, L. Zheng, C. H. Yu, J. Gonzalez, H. Zhang, and I. Stoica, “Efficient memory management for large language model serving with PagedAttention,” inSOSP, 2023
2023
-
[16]
Attention is all you need,
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” inNeurIPS, 2017
2017
-
[17]
A. . M. Llama Team, “The llama 3 herd of models,”arXiv preprint arXiv:2407.21783, 2024
Pith/arXiv arXiv 2024
-
[18]
Outrageously large neural networks: The sparsely-gated mixture-of-experts layer,
N. Shazeer, A. Mirhoseini, K. Maziarz, A. Davis, Q. Le, G. Hinton, and J. Dean, “Outrageously large neural networks: The sparsely-gated mixture-of-experts layer,”arXiv preprint arXiv:1701.06538, 2017
Pith/arXiv arXiv 2017
-
[19]
Q. Team, “Qwen3 technical report,”arXiv preprint arXiv:2505.09388, 2025
Pith/arXiv arXiv 2025
-
[20]
Andes: Defining and enhancing quality-of-experience in llm-based text streaming services,
J. Liu, J.-W. Chung, Z. Wu, F. Lai, M. Lee, and M. Chowdhury, “Andes: Defining and enhancing quality-of-experience in llm-based text streaming services,”arXiv preprint arXiv:2404.16283, 2024
Pith/arXiv arXiv 2024
-
[21]
Orca: A distributed serving system for Transformer-Based generative models,
G.-I. Yu, J. S. Jeong, G.-W. Kim, S. Kim, and B.-G. Chun, “Orca: A distributed serving system for Transformer-Based generative models,” in OSDI, 2022
2022
-
[22]
J. J. Grainger and W. D. Stevenson,Power System Analysis. New York, NY , USA: McGraw–Hill, 1994
1994
-
[23]
Experimental validation of feedback optimization in power distribution grids,
L. Ortmann, A. Hauswirth, I. Caduff, F. D ¨orfler, and S. Bolognani, “Experimental validation of feedback optimization in power distribution grids,”Electric Power Systems Research, vol. 189, p. 106782, 2020
2020
-
[24]
Convex relaxations and linear approximation for optimal power flow in multiphase radial networks,
L. Gan and S. H. Low, “Convex relaxations and linear approximation for optimal power flow in multiphase radial networks,” in2014 power systems computation conference, pp. 1–9, IEEE, 2014
2014
-
[25]
The mit su- percloud dataset,
S. Samsi, M. L. Weiss, D. Bestor, B. Li, M. Jones, A. Reuther, D. Edelman, W. Arcand, C. Byun, J. Holodnack,et al., “The mit su- percloud dataset,” in2021 IEEE High Performance Extreme Computing Conference (HPEC), pp. 1–8, IEEE, 2021
2021
-
[26]
Energy demand from ai and data centers,
International Energy Agency, “Energy demand from ai and data centers,” IEA Report, 2025
2025
-
[27]
Ieee 13 node test feeder,
IEEE Distribution System Analysis Subcommittee, “Ieee 13 node test feeder,” tech. rep., IEEE Power & Energy Society, 2014
2014
-
[28]
OpenDSSDirect.py: Direct python in- terface to opendss
M. J. O’Connell and contributors, “OpenDSSDirect.py: Direct python in- terface to opendss.” https://github.com/dss-extensions/OpenDSSDirect. py, 2020
2020
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.