Pith. sign in

REVIEW 3 major objections 6 minor 32 references

A hierarchical multi-agent transformer can run AI data centers with lower carbon and fewer dropped jobs by using grid nodal carbon intensity as a live signal.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review

2026-07-12 03:15 UTC pith:22GG4LGI

load-bearing objection Solid hierarchical MARL + CEF-NCI design for AIDC–DSO coordination; simulation evidence is multi-baseline and useful, but the headline ranking sits on one empirically tuned reward point. the 3 major comments →

arxiv 2607.03324 v2 pith:22GG4LGI submitted 2026-07-03 eess.SY cs.AIcs.SY

Hierarchical Multi-Agent Reinforcement Learning for Carbon-Aware AI Data Centers in Power Distribution Systems

Hyunsoo Lee , Panggah Prabawa , Dae-Hyun Choi , Joongheon Kim This is my paper
classification eess.SY cs.AIcs.SY
keywords AI data centerscarbon emission flownodal carbon intensitymulti-agent reinforcement learningmulti-agent transformerworkload schedulingpower distribution systemshierarchical control
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

AI data centers now draw large amounts of electricity for both delay-tolerant training jobs and delay-sensitive inference jobs, and that demand can stress distribution grids while raising carbon emissions. This paper claims that a hierarchical carbon-aware multi-agent reinforcement learning setup can keep those centers economical and low-carbon under uncertain job arrivals by closing a loop with the distribution system operator. A global workload-manager agent first places training and inference jobs across geographically distributed AI data centers using nodal carbon intensity from a carbon-emission-flow model; local agents then shift training jobs in time, assign GPU blocks, and set cooling supply-air temperature. The hierarchy, implemented with a multi-agent transformer and a largest-remainder rule that turns continuous ratios into integer job counts, is meant to avoid combinatorial explosion of the joint action space. On an IEEE 33-node test system the approach reports lower carbon emissions, better carbon-emission efficiency, far fewer dropped jobs, and improved power-usage effectiveness than MAPPO, plain transformer, and decentralized-decoder baselines, especially when both cost and carbon are rewarded.

Core claim

A closed-loop hierarchical carbon-aware multi-agent reinforcement learning framework—global workload-manager agent plus local AI-data-center agents trained with a multi-agent transformer, driven by carbon-emission-flow nodal carbon intensity—can coordinate spatial job placement, temporal training-job shifting, GPU allocation, and cooling control so that AI data centers and the distribution grid operate more economically and with lower carbon under workload uncertainty than non-hierarchical or non-carbon-aware multi-agent baselines.

What carries the argument

Hierarchical multi-agent transformer (MAT) with multi-agent advantage decomposition: the shared encoder builds a global state representation; an autoregressive decoder always lets the workload-manager agent act first, then conditions each local AI-data-center agent, turning an exponentially large joint action space into an additive, factorized policy search.

Load-bearing premise

The learned policies are assumed to transfer from synthetic sinusoidal-plus-noise job arrivals and piecewise-constant fifteen-minute carbon signals held over one-minute control steps to real AI data-center traffic and live grid carbon intensity.

What would settle it

Replace the synthetic arrival process and the held carbon signals with measured multi-site AI workload traces and live nodal carbon intensity on a real or higher-fidelity distribution feeder; if the hierarchical multi-agent transformer no longer reduces carbon emissions and dropped jobs relative to the same baselines under identical reward weights, the central claim fails.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes a hierarchical carbon-aware multi-agent RL (CA-MARL) framework for coordinating a distribution system operator (DSO) with geographically distributed AI data centers (AIDCs). A workload-manager (WM) agent spatially allocates training and inference jobs using CEF-derived nodal carbon intensity (NCI) from a CEF-integrated DSO OPF; local AIDC agents then perform temporal training-job shifting, GPU-block/GPU allocation, and CRAC supply-temperature control. Agents are trained with a multi-agent transformer (MAT) under centralized training and decentralized execution, with continuous allocation ratios projected to integers via a largest-remainder rule. On an IEEE 33-node system with three (and up to five) AIDCs, joint-mode MAT is reported to improve carbon emissions, carbon-emission efficiency, dropped jobs, and PUE relative to MAPPO, Transformer, and MAT-Dec, and relative to a power-only mode that removes NCI from the environment and reward.

Significance. If the reported gains hold under reasonable multi-objective weightings and more realistic workload/NCI interfaces, the work is a useful systems contribution at the intersection of power-system carbon accounting and AIDC operations. Strengths include: (i) closed-loop coupling of CEF-based NCI with hierarchical workload control rather than location-based carbon factors alone; (ii) an explicit hierarchical factorization plus action-space cardinality analysis (Sec. IV-D, Table III) that addresses combinatorial explosion; (iii) a structured ablation suite (MAPPO / Transformer / MAT-Dec; joint vs power mode; static vs dynamic WM allocation; scaling to 3–5 AIDCs; sensitivity of λ and c2/c3). The paper is simulation-based engineering design rather than a theorem paper; its value is in the integrated architecture and empirical coordination results within the stated sandbox.

major comments (3)
  1. Sec. IV-C, Eq. (32), and Table II: The headline claim that joint-mode MAT is superior on carbon emissions, carbon-emission efficiency, dropped jobs, and PUE rests on a single shared multi-objective reward with weights c1–c5 that are only “empirically selected through preliminary experiments.” The authors correctly note that c2 and c3 dominate the economic–environmental trade-off, and Fig. 11(a) shows that varying c2/c3 moves cost and carbon in opposite directions. Table II, however, reports all four methods at one operating point (implicitly c2/c3 = 0.02). Without re-training (or at least re-evaluating) MAPPO/Transformer/MAT-Dec/MAT across a small grid of plausible (c2,c3) and showing that the ranking on carbon and drops is stable, the strongest comparative claim is a tuned-point observation rather than a method property. Please add a weight-robustness study for the Table II metrics, or
  2. Sec. V-A / Eq. (39) and the multi-time-scale NCI interface: Workload arrivals are synthetic class-wise sinusoids plus noise, and NCI is held piecewise-constant over 15-minute DSO intervals while agents act at 1-minute resolution. This is an acceptable sandbox, but the abstract and contributions state “robust … operations … under uncertainties” without quantifying sensitivity to arrival-model misspecification (burstiness, correlation across classes, non-sinusoidal diurnal shapes) or to NCI update rate/volatility. At minimum, report one stress case (e.g., higher σ_ε, phase-aligned bursts, or 5-minute NCI refresh) so that the joint-mode gains in Table II are not tied only to the calibrated Table I traffic.
  3. Table II statistical reporting: Fig. 6 reports mean ±1% std over three seeds for training rewards, but Table II’s 24-hour operational metrics (throughput, cost, carbon, drops, PUE) do not state seed aggregation or variability. Given that dropped-job counts differ by orders of magnitude across methods (e.g., 419 vs 10627 in joint mode), please report mean±std (or min–max) over the same seeds used for Fig. 6, and confirm that environment seeds and arrival realizations are matched across methods.
minor comments (6)
  1. Sec. IV-A / IV-B: The largest-remainder projection is well motivated; a one-line statement that the environment always transitions on the integerized action (already present) could be echoed in the experimental setup so readers do not confuse continuous policy outputs with executed actions.
  2. Fig. 8c: Heatmaps of temporal deferral are informative; adding the TOU peak/off-peak bands on the time axis would make the claimed economic shifting easier to verify visually.
  3. Notation: NCI is wi,t in Eq. (9) and wt / wi,t in agent states; keep a single symbol consistently in Sec. IV. Also clarify whether wgt_i in Eq. (10) is time-invariant while wgt_i,t appears in Eq. (9).
  4. Table I lists Δt = 15 min (DSO) and 1 min (MARL); state explicitly how many 1-minute steps share one NCI sample and whether agents observe a stale NCI or a held constant value (text implies the latter).
  5. Related work: Recent carbon-aware / geo-distributed AIDC scheduling papers are cited; a short paragraph contrasting CEF-NCI closed-loop control with pure location-based or market-price carbon signals would sharpen novelty without lengthening the intro much.
  6. Typos / style: “efficient” and similar fi-ligature artifacts appear throughout; “T raining Jobs” spacing; arXiv id in header is fine for preprint but clean for journal submission.

Circularity Check

0 steps flagged

No significant circularity: hierarchical CA-MARL results are simulated performance under an explicit multi-objective reward and standard CEF/NCI models, not tautological restatements of fitted inputs.

full rationale

This is a design-and-simulate engineering paper. The claimed gains (Table II joint-mode MAT vs MAPPO/Transformer/MAT-Dec on throughput, cost, carbon, drops, PUE) are environment-evaluated outcomes of learned policies, not algebraic identities. NCI (Eq. 9–10) and the CEF-integrated DSO objective (Eq. 11) are standard domain constructions used as exogenous signals; AIDC power, workload dynamics, and CRAC models (Sec. III-C) define the transition, not the result. The shared reward (Eq. 32) with empirically chosen weights c1–c5 is a free design choice that shapes the objective being optimized; Fig. 11(a) even shows the cost–carbon trade-off under c2/c3, so the comparison is not forced by construction to equal a fitted target. MAT is cited from external work (Wen et al.); no load-bearing uniqueness theorem or ansatz is imported from overlapping authors. Synthetic arrivals (Eq. 39) and multi-time-scale NCI are modeling assumptions that affect external validity, not circularity. No step reduces a claimed prediction to its own inputs by definition.

Axiom & Free-Parameter Ledger

6 free parameters · 7 axioms · 2 invented entities

The central empirical claim rests on standard power-system and CEF modeling assumptions, a synthetic workload process, CRAC thermal models from prior work, MAT multi-agent advantage decomposition, and several hand-chosen weights and scales (reward coefficients, λ, discretization Γ, COP polynomial, arrival-rate parameters). No new physical particles or forces; the ‘entities’ are architectural roles (WM agent, AIDC agents, CA-MARL loop).

free parameters (6)
  • Reward weights c1–c5 (esp. c2/c3) = c2/c3 = 0.02 (others not fully tabulated)
    Empirically selected via preliminary experiments to balance throughput, cost, carbon, and job drops (Sec. IV-C); c2/c3=0.02 chosen as ‘practically balanced’ after sensitivity (Fig. 11a). Directly shapes the learned policy trade-off.
  • DSO carbon penalty λ = 0.01
    Penalty on GT/substation carbon in objective (11); set to 0.01 as conservative operating point after sensitivity (Fig. 11b, Table I).
  • Workload arrival parameters (λ̄, A, Θ, ϕ, σ_ε per class) = See Table I (e.g., λ̄_train LLM 0.0585 jobs/min)
    Calibrated sinusoidal-plus-noise rates for LLM/VAE/DeepResearch/Search (Eq. 39, Table I); define the uncertainty the agents train against.
  • Action discretization resolution Γ and CRAC temperature grid = Γ=10; |AT|=6
    Used for action-space cardinality comparison (Sec. IV-D, Table III); Γ=10 and 1°C CRAC steps affect reported combinatorial reduction.
  • Power/compute coefficients α_train, α_inf, ξ_max, job sizes j_m, GPU block sizes b_m = Table I (e.g., α_train 7e-4 kW/TOPS)
    Scaled from neural scaling laws and empirical studies to IEEE 33 capacity (Sec. V-A, Table I); determine IT power and GPU constraints.
  • COP polynomial coefficients for chiller = 0.0068, 0.008, 0.458
    Quadratic COP_i(T)=0.0068T²+0.008T+0.458 (Table I) from cooling model assumptions; couples cooling power to supply temperature action.
axioms (7)
  • domain assumption Convex branch-flow / SOCP power-flow model (1)–(8) adequately represents the distribution network for DSO dispatch and NCI computation.
    Sec. III-A; standard relaxed OPF used as environment truth.
  • domain assumption CEF nodal carbon intensity with proportional sharing and loss allocation (9)–(10) correctly attributes consumer carbon responsibility.
    Sec. III-B citing CEF literature [21],[29]; NCI is the carbon signal driving WM/AIDC policies.
  • domain assumption Multi-agent advantage decomposition and MAT encoder–decoder sequential conditioning yield stable cooperative policies under CTDE.
    Sec. II citing MAT [25]–[28]; justifies using MAT over flat joint action learning.
  • ad hoc to paper Largest-remainder projection of continuous allocation ratios to integers preserves feasibility and keeps quantization error <1 job per variable without breaking learning consistency.
    Sec. IV-A/B; continuous policy outputs are projected before environment step.
  • domain assumption CRAC–chiller–tower thermal/power model and temperature limits (19)–(24) from prior cooling literature bound safe operation.
    Sec. III-C3 citing [30]; Task 4 cooling actions depend on this model.
  • ad hoc to paper No direct AIDC–AIDC messaging; hierarchical WM mediation plus shared reward is sufficient for coordination.
    Sec. I/IV system structure; design choice enabling hierarchy.
  • ad hoc to paper Piecewise-constant NCI over 15-minute DSO intervals is a valid multi-time-scale interface for 1-minute agent control.
    Sec. IV intro and V-A; stabilizes closed-loop training.
invented entities (2)
  • Hierarchical CA-MARL closed loop (WM agent + AIDC agents + CEF-DSO environment) no independent evidence
    purpose: Factorize spatial job placement from local temporal/GPU/cooling decisions while exchanging NCI and power consumption.
    Architectural construct of the paper (Fig. 1, Fig. 3); not a physical entity. Independent evidence is only the simulation study itself.
  • Largest-remainder rule as MARL action integerizer for train/infer allocations independent evidence
    purpose: Map continuous simplex actions to discrete job counts while preserving totals.
    Methodological device (Sec. IV-A/B); classical apportionment rule applied here, not a new physical object.

reviewed 2026-07-12 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Hierarchical Multi-Agent Reinforcement Learning for Carbon-Aware AI Data Centers in Power Distribution Systems." pith.science (2026). https://pith.science/paper/22GG4LGI

@misc{pith2026260703324,
  author       = {Pith},
  title        = {Pith review of: Hierarchical Multi-Agent Reinforcement Learning for Carbon-Aware AI Data Centers in Power Distribution Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/22GG4LGI}},
  note         = {Machine review of arXiv:2607.03324}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Eco-friendly energy management for artificial intelligence data centers (AIDCs) is crucial because of the significant increase in energy consumption-induced carbon emissions from AIDCs resulting from the rapid expansion of AI applications. This paper proposes a hierarchical carbon-aware multi-agent reinforcement learning (CA-MARL) framework for robust and efficient operations of AIDCs under uncertainties while ensuring low-carbon operation of power distribution systems. The framework comprises a workload manager (WM) agent and multiple local AIDC agents trained using a multi-agent transformer method, corresponding to a global AIDC aggregator and a local AIDC operator, respectively. Leveraging AIDC operation data along with nodal carbon intensity (NCI) calculated from the carbon emission flow-integrated distribution system operator problem, the WM agent spatially allocates AI training and inference jobs among all AIDCs. Based on the jobs allocated from the WM agent and NCI information, each AIDC agent schedules economical and eco-friendly operations of the AIDC by performing the following tasks: i) temporal shifting of training jobs, ii) spatial allocation of training graphics processing unit (GPU) blocks and inference GPUs within the AIDC, and iii) control of the supply air temperature of the cooling system. The effectiveness of the proposed framework was assessed using an IEEE 33-node power distribution system.

Figures

Figures reproduced from arXiv: 2607.03324 by Dae-Hyun Choi, Hyunsoo Lee, Joongheon Kim, Panggah Prabawa.

Figure 1
Figure 1. Figure 1: A reference hierarchical system integrated with a [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: CRAC-based cooling system for AIDC. Pij,t is formulated as Pij,t = ˆp + ij,t − pˆ − ij,t along with the complementarity constraint pˆ + ij,t · pˆ − ij,t = 0, where pˆ + ij,t = max{Pij,t, 0} and pˆ − ij,t = max{−Pij,t, 0}. The non￾convex nature of the complementarity constraint can be relaxed via linearization using the big-M method with binary decision variables, or a method based on special ordered set co… view at source ↗
Figure 3
Figure 3. Figure 3: Structure of the proposed hierarchical CA-MARL framework comprising environment, agents, and MAT. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Spatio-temporal workload scheduling of WM and [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: IEEE 33-node power distribution system. consumption on the order of several megawatts (approx￾imately 8–12 MW), corresponding to medium-scale AI computing facilities with several hundred GPUs.. Table I lists the parameter values used in the proposed framework. To ensure practical relevance, the simulation parameters listed in Table I were configured based on empirical stud￾ies and then scaled to align with… view at source ↗
Figure 6
Figure 6. Figure 6: Comparison of rewards among the MAPPO, Trans [PITH_FULL_IMAGE:figures/full_fig_p012_6.png] view at source ↗
Figure 9
Figure 9. Figure 9: Relationship between NCI and workload allocation [PITH_FULL_IMAGE:figures/full_fig_p013_9.png] view at source ↗
Figure 8
Figure 8. Figure 8: Spatio-temporal training/inference job shifting for [PITH_FULL_IMAGE:figures/full_fig_p013_8.png] view at source ↗
Figure 13
Figure 13. Figure 13: Relative reduction of electricity purchase cost and [PITH_FULL_IMAGE:figures/full_fig_p014_13.png] view at source ↗
Figure 11
Figure 11. Figure 11: Sensitivity analysis of the reward and objective [PITH_FULL_IMAGE:figures/full_fig_p014_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Comparison of rewards for the proposed MAT [PITH_FULL_IMAGE:figures/full_fig_p014_12.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

32 extracted references · 5 linked inside Pith

  1. [1]

    Energy and policy considerations for modern deep learning research,

    E. Strubell, A. Ganesh, and A. McCallum, “Energy and policy considerations for modern deep learning research, ” in Proc. AAAI, New York, NY, USA, Feb. 2020, pp. 13 693–13 696

    2020

  2. [2]

    Estimating the carbon footprint of BLOOM, a 176B parameter language model,

    A. S. Luccioni, S. Viguier, and A.-L. Ligozat, “Estimating the carbon footprint of BLOOM, a 176B parameter language model,” The J. Mach. Learn. Res. , vol. 24, no. 1, pp. 11 990– 12 004, Jan. 2023

    2023

  3. [3]

    Energy and AI 2024,

    International Energy Agency (IEA), “Energy and AI 2024, ” IEA, Tech. Rep., Dec. 2024, [A vailable]: https://www.iea.org/ reports/energy-and-ai

    2024

  4. [4]

    Electricity demand and grid impacts of AI data centers: Challenges and prospects,

    X. Chen, X. Wang, A. Colacelli, M. Lee, and L. Xie, “Electricity demand and grid impacts of AI data centers: Challenges and prospects,” arXiv preprint arXiv:2509.07218v2 , Sep. 2025

    Pith/arXiv arXiv 2025

  5. [5]

    Computation- power coupled modeling for IDCs and collaborative optimiza- tion in ADNs,

    C. Li, K. Zheng, H. Guo, C. Kang, and Q. Chen, “Computation- power coupled modeling for IDCs and collaborative optimiza- tion in ADNs, ” IEEE Trans. Smart Grid , vol. 15, no. 3, pp. 2762–2775, May 2024

    2024

  6. [6]

    Incentive- compatible demand response for spatially coupled Internet data centers in electricity markets,

    M. Chen, C. Gao, M. Shahidehpour, and Z. Li, “Incentive- compatible demand response for spatially coupled Internet data centers in electricity markets, ” IEEE Trans. Smart Grid, vol. 12, no. 4, pp. 3056–3069, Jul. 2021

    2021

  7. [7]

    Game-based optimization method for geo-distributed data centers under customer directrix load demand response mechanism,

    J.-K. Wu, Z.-W. Liu, Y. Zhao, W. Li, and Y. Li, “Game-based optimization method for geo-distributed data centers under customer directrix load demand response mechanism, ” IEEE Trans. Smart Grid , vol. 17, no. 1, pp. 283–296, Jan. 2026

    2026

  8. [8]

    Supply restoration of data centers in flexible distribution networks with spatial-temporal regulation,

    J. Jian et al. , “Supply restoration of data centers in flexible distribution networks with spatial-temporal regulation, ” IEEE Trans. Smart Grid , vol. 15, no. 1, pp. 340–354, Jan. 2024

    2024

  9. [9]

    Synergising hierarchical data centers and power networks: A privacy-preserving approach,

    J. Liu, F. Teng, and F. Y. Hou, “Synergising hierarchical data centers and power networks: A privacy-preserving approach, ” IEEE Trans. Smart Grid , vol. 16, no. 6, pp. 5083–5098, Nov. 2025

    2025

  10. [10]

    Learning-enabled adaptive power capping scheme for cloud data centers,

    Y. Sun, Z. Ding, P. Dehghanian, and F. Teng, “Learning-enabled adaptive power capping scheme for cloud data centers, ” IEEE Trans. Smart Grid , vol. 16, no. 6, pp. 4755–4767, Nov. 2025

    2025

  11. [11]

    Let’s wait awhile: How temporal workload shifting can reduce carbon emissions in the cloud,

    P. Wiesner et al. , “Let’s wait awhile: How temporal workload shifting can reduce carbon emissions in the cloud, ” in Proc. In- ternational Middleware Conference (Middleware), Québec City, Canada, Dec. 2021, p. 260–272

    2021

  12. [12]

    Carbon-aware computing for datacen- ters,

    A. Radovanović et al. , “Carbon-aware computing for datacen- ters,” IEEE Trans. Power Syst. , vol. 38, no. 2, pp. 1270–1280, Mar. 2023

    2023

  13. [13]

    Carbon explorer: A holistic framework for designing carbon aware datacenters,

    B. Acun et al. , “Carbon explorer: A holistic framework for designing carbon aware datacenters, ” in Proc. ACM ASPLOS , Vancouver, Canada, Mar. 2023, pp. 118–132

    2023

  14. [14]

    Sustainable AIGC workload scheduling of geo-distributed data centers: A multi-agent reinforcement learning approach,

    S. Zhang, M. Xu, W. Y. B. Lim, and D. Niyato, “Sustainable AIGC workload scheduling of geo-distributed data centers: A multi-agent reinforcement learning approach, ” in Proc. IEEE Global Communications Conference (GLOBECOM), Kuala Lumpur, Malaysia, Dec. 2023, pp. 3500–3505

    2023

  15. [15]

    Game-theoretic deep reinforcement learning to minimize carbon emissions and energy costs for AI inference workloads in geo-distributed data centers,

    N. Hogade and S. Pasricha, “Game-theoretic deep reinforcement learning to minimize carbon emissions and energy costs for AI inference workloads in geo-distributed data centers, ” IEEE Trans. Sustain. Comput. , vol. 10, no. 4, pp. 628–641, Jul.-Aug. 2025

    2025

  16. [16]

    DCcluster-Opt: Benchmarking dynamic multi-objective optimization for geo-distributed data center workloads,

    A. Guillen et al. , “DCcluster-Opt: Benchmarking dynamic multi-objective optimization for geo-distributed data center workloads,” in Proc. Advances in Neural Information Processing Systems (NeurIPS) Datasets and Benchmarks Track , Dec. 2025. 16

    2025

  17. [17]

    Hierarchical rein- forcement learning for power network topology control,

    B. Manczak, J. Viebahn, and H. van Hoof, “Hierarchical rein- forcement learning for power network topology control, ” arXiv preprint arXiv:2311.02129v1, Nov. 2023

    Pith/arXiv arXiv 2023

  18. [18]

    Distributed hierarchical deep reinforcement learning for large-scale grid emergency control,

    Y. Chen, J. Zhu, Y. Liu, L. Zhang, and J. Zhou, “Distributed hierarchical deep reinforcement learning for large-scale grid emergency control, ” IEEE Trans. Power Syst. , vol. 39, no. 2, pp. 4446–4458, Mar. 2024

    2024

  19. [19]

    Distributed online dispatch for microgrids using hierarchical reinforcement learning embedded with operation knowledge,

    T. Lu, R. Hao, Q. Ai, and H. He, “Distributed online dispatch for microgrids using hierarchical reinforcement learning embedded with operation knowledge, ” IEEE Trans. Power Syst. , vol. 38, no. 4, pp. 2989–3002, Jul. 2023

    2023

  20. [20]

    Multi- agent hierarchical deep reinforcement learning for HV AC control with flexible DERs,

    Y. Pei, Y. Yao, J. Zhao, J. Hao, F. Ding, and J. Wang, “Multi- agent hierarchical deep reinforcement learning for HV AC control with flexible DERs, ” IEEE Trans. Smart Grid, vol. 16, no. 6, pp. 5589–5601, Nov. 2025

    2025

  21. [21]

    Carbon emission flow from generation to demand: A network-based model,

    C. Kang et al. , “Carbon emission flow from generation to demand: A network-based model, ” IEEE Trans. Smart Grid , vol. 6, no. 5, pp. 2386–2394, Sep. 2015

    2015

  22. [22]

    Carbon emission flow oriented tri-level planning of integrated electricity–hydrogen– gas system with hydrogen vehicles,

    X. Wei, X. Zhang, Y. Sun, and J. Qiu, “Carbon emission flow oriented tri-level planning of integrated electricity–hydrogen– gas system with hydrogen vehicles, ” IEEE Trans. Ind. Appl. , vol. 58, no. 2, pp. 2607–2618, Mar. 2022

    2022

  23. [23]

    Carbon- aware peer-to-peer energy trading in an unbalanced distribution network via a nash equilibrium discovery deep reinforcement learning approach,

    X. Liu, Y. Ye, S. Li, C. Zhang, Q. Ma, and J. Zhu, “Carbon- aware peer-to-peer energy trading in an unbalanced distribution network via a nash equilibrium discovery deep reinforcement learning approach,” IEEE Trans. Smart Grid , vol. 16, no. 4, pp. 3392–3407, July. 2025

    2025

  24. [24]

    An augmented lagrangian- based safe reinforcement learning algorithm for carbon-oriented optimal scheduling of EV aggregators,

    X. Shi, Y. Xu, G. Chen, and Y. Guo, “An augmented lagrangian- based safe reinforcement learning algorithm for carbon-oriented optimal scheduling of EV aggregators, ” IEEE Trans. Smart Grid, vol. 15, no. 1, pp. 795–809, Jan. 2024

    2024

  25. [25]

    Multi-agent reinforcement learning is a sequence modeling problem,

    M. Wen, J. G. Kuba, R. Lin, W. Zhang, Y. Wen, J. Wang, and Y. Yang, “Multi-agent reinforcement learning is a sequence modeling problem, ” arXiv preprint arXiv:2205.14953v3 , Oct. 2022

    Pith/arXiv arXiv 2022

  26. [26]

    Settling the variance of multi-agent policy gradients,

    J. G. Kuba et al. , “Settling the variance of multi-agent policy gradients,” in Proc. Advances in Neural Information Processing Systems (NeurIPS), Virtual, Dec. 2021

    2021

  27. [27]

    High-dimensional continuous control using generalized advantage estimation,

    J. Schulman et al. , “High-dimensional continuous control using generalized advantage estimation, ” arXiv preprint arXiv:1506.02438, Jun. 2015

    Pith/arXiv arXiv 2015

  28. [28]

    Multi-agent reinforcement learning is a sequence modeling problem,

    M. Wen et al., “Multi-agent reinforcement learning is a sequence modeling problem, ” in Proc. Advances in Neural Information Processing Systems (NeurIPS), New Orleans, USA, Dec. 2022, pp. 16 509 – 16 521

    2022

  29. [29]

    Carbon-aware optimal power flow,

    X. Chen, A. Sun, W. Shi, and N. Li, “Carbon-aware optimal power flow,” IEEE Trans. Power Syst. , vol. 40, no. 4, pp. 3090– 3104, Jul. 2025

    2025

  30. [30]

    SustainDC: Benchmarking for sustainable data center control,

    A. Naug et al. , “SustainDC: Benchmarking for sustainable data center control, ” in Proc. Advances in Neural Information Pro- cessing Systems (NeurIPS), vol. 37, Vancouver, Canada, Dec. 2024, pp. 100 630–100 669

    2024

  31. [31]

    Radial distribution test feeders,

    W. Kersting, “Radial distribution test feeders, ” IEEE Trans. Power Syst. , vol. 6, no. 3, pp. 975–985, Aug. 1991

    1991

  32. [32]

    Scaling laws for neural language models,

    J. Kaplan et al. , “Scaling laws for neural language models, ” arXiv preprint arXiv:2001.08361 , Jan. 2020

    Pith/arXiv arXiv 2001

This paper was first reviewed by grok-4.5 on July 12, 2026.