Pith. sign in

REVIEW 5 major objections 4 minor 34 references

One GNN-observing agent per power line, coordinated by a manager, survives 6114.4 average test steps versus 646.6 for do-nothing, and acts in 0.187 seconds per decision.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A two-layer distributed RL system with one GNN-observing agent per power line and a learned manager keeps the Grid2Op case14 grid alive far longer than the do-nothing baseline.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection The per-line/graph architecture is new and worth engaging, but the headline result is not established: DQfD drives most of the gain, demos are undocumented, and the evaluation lacks error bars and baselines. the 5 major comments →

arxiv 2509.02861 v1 pith:ULNKAHQB submitted 2025-09-02 cs.LG

Power Grid Control with Graph-Based Distributed Reinforcement Learning

Carlo Fabrizio , Gianvito Losapio , Marco Mussi , Alberto Maria Metelli , Marcello Restelli This is my paper
classification cs.LG
keywords graph neural networksdistributed reinforcement learningpower grid topology controlGrid2Opdeep Q-learning from demonstrationsmulti-agent reinforcement learningline graph representationreward shaping
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that power-grid topology control can be handled by a fully distributed reinforcement-learning system: a separate learning agent for each power line, a manager that decides which line acts, and a shared graph neural network that lets each line agent see a compact local view of the surrounding network. On the Grid2Op benchmark, the system survives 6114.4 test steps on average versus 646.6 for the do-nothing baseline, and it makes decisions in 0.187 seconds per step, compared with 2.56 seconds for the simulation-based Expert used to gather demonstrations. The paper argues that the key to this result is decomposing the observation space as well as the action space, which earlier decentralized approaches did not do. If the result holds, the architecture offers a template for controllers that scale with the grid and react quickly enough for real-time operation.

Core claim

The central claim is that a two-layer distributed architecture can make reinforcement learning practical for topology control in realistic power grids. Twenty low-level agents, one per line, each run a Deep Dueling Double Q-learning policy on a local observation produced by a shared GNN; a twenty-first policy, the manager, selects which low-level agent acts at each time step. The grid is converted into a homogeneous graph in which lines are nodes and substations are edges, so the GNN's message passing encodes neighborhood information directly into each line's observation. Pretraining with Deep Q-Learning from Demonstrations (DQfD) on data from a simulation-based Expert, plus bootstrapped pot

What carries the argument

The load-bearing object is the homogeneous line graph of the power grid: each power line becomes a node, and two nodes are connected when their lines share a substation. Node features are built by concatenating the embeddings of the two terminal substations, each substation embedding being the concatenation of its two bus embeddings, each bus embedding being the sum of the one-hot-like vectors of the elements connected to that bus. A shared GNN runs message passing over this graph, and the embedding of a line's node becomes that line's observation. This single mechanism performs three jobs at once: it gives every line agent a local but informative view, it avoids the bus-bar information asym

Load-bearing premise

The reported gain rests on two unquantified supports: a dataset of expert demonstrations for DQfD pretraining whose size, coverage, and generation protocol are not given, and a runtime single-step simulation that prunes the action space and whose contribution is never measured separately.

What would settle it

Retrain the complete architecture on the same ten test chronics with the DQfD demonstration buffer replaced by an equal-sized buffer of random or heuristic transitions, leaving the GNN, reward shaping, and manager unchanged. If survival time falls toward the reported 1878-step 'No DQfD' ablation rather than staying near 6114, the expert demonstrations—not the graph-based distributed design—carry the result. A second decisive check is to disable the runtime single-step simulation used for greedy action-space reduction; if survival time and inference time barely change, that externality is unimp

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Adding a line to the grid adds one low-level agent and one graph node, so the per-agent observation size stays fixed; the architecture's training cost scales with the number of lines rather than with the combinatorial topology space.
  • At 0.187 seconds per decision, the learned policy clears the real-time bar in settings where the 2.56-second simulation-based Expert cannot act quickly enough.
  • Removing either the GNN or the DQfD pretraining collapses performance, so the reported gain is not marginal but depends on both components.
  • The homogeneous line-graph representation sidesteps the bus-bar information asymmetry and the need for heterogeneous GNNs identified in earlier power-grid graph work, so it can serve as a standalone input representation for other topology-control methods.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves the demonstration dataset unspecified; a direct test would be to retrain with a same-sized dataset of random or heuristic transitions, isolating how much of the 6114-step survival is imitation of the Expert versus value learning by the distributed agents.
  • Because all low-level agents share a single GNN, transfer from a small grid to a larger one is plausible: message passing works on any graph size, so initializing a case14-trained GNN on a larger Grid2Op environment and measuring convergence time would test the stated scalability.
  • The runtime single-step simulation used for greedy action-space reduction is a hidden dependency; disabling it (for example, replacing it with an N-1 static criterion) would reveal how much of the survival time is learned control and how much is simulator-assisted search.
  • Reward shaping contributes little on case14; the paper's own hypothesis that it matters in larger, sparser-reward grids is testable by repeating the ablation on a larger Grid2Op case.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper proposes a two-level distributed RL architecture for Grid2Op topology control. Twenty low-level DQN agents each manage one power line and receive a GNN-enriched local observation, while an RL manager selects which line agent acts. Training combines DQfD from a simulation-based Expert (Marot et al., 2018) with bootstrapped potential-based reward shaping. A homogeneous line-graph representation is introduced to avoid heterogeneous-graph issues. Experiments on l2rpn_case14_sandbox report a test survival time of 6114.4 for the full system versus 646.6 for Do-Nothing (Table 1) and an inference time of 0.187 s versus 2.56 s for the Expert (Table 2). An ablation attributes most of the gain to DQfD and the GNN, with reward shaping having a mixed effect. The Limitations section explicitly acknowledges the manager's global view, the dependence on expert demonstrations, and the unisolated single-step simulation used for action-space reduction.

Significance. If the reported results are reproducible, the paper makes a useful contribution: it combines per-line distributed control, GNN-based observation decomposition, and imitation learning in a way that is modular and potentially scalable, and the code is publicly available. The central narrow claim, that the full system outperforms Do-Nothing, is consistent with Table 1 even without DQfD (1878 vs. 646.6), which is a mitigating fact. However, the evidence is preliminary: averages over only 10 validation and 10 test chronics, no seeds or error bars, undocumented expert demonstrations, no learning-based baseline, and a runtime component that is entangled with the inference-time comparison. The reader's leakage concern lands: the headline number is dominated by the DQfD component, and the demonstration dataset is not described. The paper is therefore a promising research report, but the load-bearing empirical claims need substantial reinforcement before journal acceptance.

major comments (5)
  1. [§4.2–§4.3, Table 1] The demonstration dataset used for DQfD is not documented: the paper does not state how many demonstrations were collected, on which chronics, whether the 10 validation/test chronics were excluded from demonstration collection, or how expert trajectories were generated. Since DQfD is the largest single contributor to the headline result (removing it drops test survival from 6114.4 to 1878), the reported 6114.4 could in part reflect imitation of the Expert on seen scenarios rather than the distributed GNN architecture. The no-DQfD row attenuates this concern for the narrow baseline claim, but not for the claim of full-system superiority. Please report the demonstration-generation protocol, the dataset size and coverage, and a leakage check (e.g., performance evaluated on chronics provably absent from the demonstration set).
  2. [§4.3, Table 1 and Fig. 5] All performance numbers are averages over 10 validation and 10 test chronics with no standard deviations, no multiple seeds, and no significance tests; the ablation appears to be a single run. With such small evaluation sets, a few favorable episodes can dominate an average. Please provide per-chronic results, repeated-seed statistics (mean±std or confidence intervals), and, where appropriate, a significance test or effect-size measure. This is needed to establish that the improvement over Do-Nothing and the ablation differences are not noise.
  3. [§4.3, Table 2, and Limitations] The inference-time comparison is potentially confounded by the single-step simulation used for greedy action-space reduction. The plain simulator step is reported as 0.1097 s, while the proposed model's inference is 0.187±0.145 s; if that simulation is part of the action loop, the 'proposed model' time is not purely a learned-policy time. The Limitations section acknowledges this step but does not isolate its contribution to either runtime or survival performance. Please measure the action loop with and without the single-step simulation, report the number of timing runs and hardware, and clarify what exactly Table 2 measures.
  4. [§4.3, Table 1] The paper compares only against Do-Nothing and the Expert agent; no learning-based baseline (e.g., a single-agent DQN on the same action space, a substation-based multi-agent method, or a GNN-based single-agent method from prior work) is evaluated. The conclusion that the proposed architecture is 'effective' would need such a comparison to support claims about the advantage of line-level distributed control and GNN-based observation decomposition over existing approaches. At minimum, a comparison with a centralized DQN using the same demonstrations and reward shaping would clarify how much of the gain comes from the distributed architecture itself.
  5. [§3.1 and Table 1, 'No Reward Shaping' row] The bootstrapped potential-based reward-shaping component is presented as a contribution, but the paper does not verify that the shaping preserves optimality in this multi-agent, partially observable setting; it cites Adamczyk et al. (2025) as justification. The ablation is mixed: removing shaping improves validation survival (5667.1 vs. 5452.2) but lowers test survival (5324.3 vs. 6114.4). This ambiguity should be discussed explicitly, and the theory transfer to multi-agent settings should be checked or at least acknowledged as an open question.
minor comments (4)
  1. [§3, Algorithm 1] Notation µ_t, φ_t, α_i, β_i is used without definition. A notation table or inline definitions would make the action-selection flow self-contained.
  2. [§4.2] Hyperparameters (GNN dimensions, DQN layer sizes, replay buffer capacities, batch sizes, epsilon half-lives, soft-update and synchronization intervals, gradient-clipping details) are not reported in the text. A hyperparameter table would improve reproducibility even with the code link.
  3. [§4.3, Fig. 6] The heatmap symmetry is qualitative and does not by itself demonstrate generalization. Please either present a quantitative generalization measure or soften the claim.
  4. [§4.3, Table 2] Report the number of timing runs, the environment/hardware, and whether the timing includes the single-step action-space simulation.

Circularity Check

0 steps flagged

No significant circularity: the empirical claims are measured in the Grid2Op simulator against external baselines, not derived from the paper's own fitted quantities.

full rationale

The paper's central empirical claim—survival time on the Grid2Op l2rpn_case14_sandbox environment—is obtained by rolling out the trained policy in the simulator, and the comparison baseline (Do-Nothing) is an external, environment-defined strategy. DQfD pre-training uses expert demonstrations from an external Expert (Marot et al., 2018), but the reported metric is not computed from those demonstrations; it is simulator survival time. The ablation (Table 1) explicitly shows that the architecture without DQfD still outperforms Do-Nothing (1878 vs 646.6 test survival), so the 'outperforms baseline' claim is not forced by the fitted demonstrations. The inference-time comparison to the Expert is an efficiency benchmark, not a derivation of the Expert's behavior from the proposed model. No self-citation chain or uniqueness theorem is load-bearing; all cited techniques (DQfD, reward shaping, GNN) are external prior work. The acknowledged limitations—dependence on expert demonstrations, global manager view, and single-step simulation for action-space reduction—are robustness/correctness concerns rather than definitional circularity: no equation in the paper defines the predicted quantity in terms of the model's own parameters, and no fitted parameter is renamed as a prediction.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 0 invented entities

The paper introduces architectural constructs (line agents, manager, shared GNN, line graph) but no new physical or representational entities with independent falsifiable handles. The central claim rests on the fidelity of Grid2Op, the quality of undocumented expert demonstrations, a hand-designed graph encoding, and a recent reward-shaping method. All learning hyperparameters are effectively free parameters because none are reported.

free parameters (5)
  • Epsilon half-life
    Exponential exploration decay schedule used for all experiments; value is not reported.
  • GNN architecture dimensions
    Number of layers, hidden units, and embedding size are not specified, but they determine the informativeness of local observations.
  • DQN hyperparameters
    Learning rate, batch size, and replay buffer sizes for the 21 agents are not reported.
  • Reward shaping coefficients
    Bootstrapped potential-based reward shaping coefficients are not given.
  • Main-target synchronization interval
    The paper states the interval was increased to prevent exploding loss but does not give the value.
axioms (4)
  • domain assumption Grid2Op l2rpn_case14_sandbox is a faithful and sufficient testbed for evaluating power grid control methods.
    All effectiveness and efficiency conclusions are drawn from this single small environment; no real grid or larger case is tested.
  • domain assumption Expert demonstrations from the simulation-based Expert (Marot et al., 2018) are high quality and sufficiently cover the state space.
    DQfD pre-training is essential according to the ablation, but the demonstration dataset size, coverage, and collection protocol are not described.
  • domain assumption The homogeneous line-graph embedding (summing bus vectors and concatenating substation embeddings) preserves the information needed for topology control.
    This representation is introduced without formal analysis or comparison to alternative graph encodings.
  • ad hoc to paper Bootstrapped potential-based reward shaping preserves the optimal policy in this multi-agent setting.
    The method is cited to Adamczyk et al. (2025), but its policy-invariance guarantees are not stated or proven for the two-layer multi-agent configuration used here.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Power Grid Control with Graph-Based Distributed Reinforcement Learning." pith.science (2026). https://pith.science/paper/ULNKAHQB

@misc{pith2026250902861,
  author       = {Pith},
  title        = {Pith review of: Power Grid Control with Graph-Based Distributed Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ULNKAHQB}},
  note         = {Machine review of arXiv:2509.02861}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

The necessary integration of renewable energy sources, combined with the expanding scale of power networks, presents significant challenges in controlling modern power grids. Traditional control systems, which are human and optimization-based, struggle to adapt and to scale in such an evolving context, motivating the exploration of more dynamic and distributed control strategies. This work advances a graph-based distributed reinforcement learning framework for real-time, scalable grid management. The proposed architecture consists of a network of distributed low-level agents acting on individual power lines and coordinated by a high-level manager agent. A Graph Neural Network (GNN) is employed to encode the network's topological information within the single low-level agent's observation. To accelerate convergence and enhance learning stability, the framework integrates imitation learning and potential-based reward shaping. In contrast to conventional decentralized approaches that decompose only the action space while relying on global observations, this method also decomposes the observation space. Each low-level agent acts based on a structured and informative local view of the environment constructed through the GNN. Experiments on the Grid2Op simulation environment show the effectiveness of the approach, which consistently outperforms the standard baseline commonly adopted in the field. Additionally, the proposed model proves to be much more computationally efficient than the simulation-based Expert method.

Figures

Figures reproduced from arXiv: 2509.02861 by Alberto Maria Metelli, Carlo Fabrizio, Gianvito Losapio, Marcello Restelli, Marco Mussi.

Figure 1
Figure 1. Figure 1: 3.2 Graph-Based Observation Construction The raw simulator observation has been pre-processed to obtain graph-like data before passing it to the Graph Neural Network. Both the Grid2Op simulator [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 1
Figure 1. Figure 1: Abstract high-level representation of the model. and the research community have proposed methods for representing the power grid as a graph; however, each approach presents certain limitations. All of the existing methods work by considering each element as a node (e.g., load, gener￾ator, power line, bus), raising the problem of graph heterogeneity: nodes present dissimilar features or belong to different… view at source ↗
Figure 2
Figure 2. Figure 2: Example of substations embedding [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Example of lines embedding. are summed to create the bus embedding. To represent a substation, the embed￾dings of its buses, including the bus of disconnected elements, are concatenated to form the substation embeddings. Finally, the power lines’ embeddings are obtained by concatenating the embeddings of their two terminal substations. Each element (power line, load, generator, or bus) is represented as a … view at source ↗
Figure 4
Figure 4. Figure 4: Representation of the Grid2Op “l2rpn_case14_sandbox” power grid. 4 Experiments In this section, we empirically validate our model on the Grid2Op environment. The code used to run the experiments presented in this section is available at: https://github.com/Carlo000ml/RL4PG. 4.1 Environment Description The analysis was conducted on the “l2rpn_case14_sandbox” environment of the Grid2Op simulator ( [PITH_FUL… view at source ↗
Figure 5
Figure 5. Figure 5: Comparison of the performance of our RL agent against the Do-Nothing base￾line over the validation set (top) and the test set (bottom) [PITH_FULL_IMAGE:figures/full_fig_p010_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Heatmap of the weights in the first GNN layer [PITH_FULL_IMAGE:figures/full_fig_p011_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

34 extracted references · 26 canonical work pages · 4 internal anchors

  1. [1]

    Reinforcement learning for electricity network operation

    Adrian Kelly, Aidan O'Sullivan, Patrick de Mars, and Antoine Marot. Reinforcement learning for electricity network operation. arXiv preprint arXiv:2003.07339, 2020

    Pith/arXiv arXiv 2003

  2. [2]

    Fuxj \"a ger, Kristian Kozak, Patrick M

    Matthias Dorfer, Anton R. Fuxj \"a ger, Kristian Kozak, Patrick M. Blies, and Marcel Wasserer. Power grid congestion management via topology optimization with alphazero. arXiv preprint arXiv:2211.05612, 2022

    Pith/arXiv arXiv 2022

  3. [3]

    Powrl: A reinforcement learning framework for robust management of power networks

    Anandsingh Chauhan, Mayank Baranwal, and Ansuma Basumatary. Powrl: A reinforcement learning framework for robust management of power networks. In AAAI Conference on Artificial Intelligence, 2023

    work page 2023

  4. [4]

    Multi-Agent Reinforcement Learning for Power Grid Topology Optimization

    Erica van der Sar, Alessandro Zocca, and Sandjai Bhulai. Multi-agent reinforcement learning for power grid topology optimization. arXiv preprint arXiv:2310.02605, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  5. [5]

    HUGO -- Highlighting Unseen Grid Options: Combining Deep Reinforcement Learning with a Heuristic Target Topology Approach

    Malte Lehna, Clara Holzh \"u ter, Sven Tomforde, and Christoph Scholz. Hugo -- highlighting unseen grid options: Combining deep reinforcement learning with a heuristic target topology approach. arXiv preprint arXiv:2405.00629, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  6. [6]

    Learning to run a power network challenge: a retrospective analysis

    Antoine Marot, Benjamin Donnot, Gabriel Dulac-Arnold, Adrian Kelly, Aidan O’Sullivan, Jan Viebahn, Mariette Awad, Isabelle Guyon, Patrick Panciatici, and Camilo Romero. Learning to run a power network challenge: a retrospective analysis. In NeurIPS Competition and Demonstration Track, 2020

    work page 2020

  7. [7]

    Grid2op: A testbed platform to model sequential decision making in power systems, 2020

    Benjamin Donnot. Grid2op: A testbed platform to model sequential decision making in power systems, 2020. URL https://github.com/Grid2Op/grid2op

    work page 2020

  8. [8]

    Multi-agent reinforcement learning: A selective overview of theories and algorithms

    Kaiqing Zhang, Zhuoran Yang, and Tamer Ba s ar. Multi-agent reinforcement learning: A selective overview of theories and algorithms. Handbook of Reinforcement Learning and Control, 2021

    work page 2021

  9. [9]

    Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and Philip S. Yu. A comprehensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems, 2020

    work page 2020

  10. [10]

    Deep q-learning from demonstrations

    Todd Hester, Matej Vecerik, Olivier Pietquin, Marc Lanctot, Tom Schaul, Bilal Piot, Dan Horgan, John Quan, Andrew Sendonaris, Ian Osband, Gabriel Dulac-Arnold, John Agapiou, Joel Leibo, and Audrunas Gruslys. Deep q-learning from demonstrations. In AAAI Conference on Artificial Intelligence, 2018

    work page 2018

  11. [11]

    Kulkarni

    Jacob Adamczyk, Volodymyr Makarenko, Stas Tiomkin, and Rahul V. Kulkarni. Bootstrapped reward shaping. In AAAI Conference on Artificial Intelligence, 2025

    work page 2025

  12. [12]

    Dueling network architectures for deep reinforcement learning

    Ziyu Wang, Tom Schaul, Matteo Hessel, Hado Hasselt, Marc Lanctot, and Nando Freitas. Dueling network architectures for deep reinforcement learning. In International Conference on Machine Learning, 2016

    work page 2016

  13. [13]

    Winning the L2RPN challenge: Power grid management via semi-markov afterstate actor-critic

    Deunsol Yoon, Sunghoon Hong, Byung-Jun Lee, and Kee-Eung Kim. Winning the L2RPN challenge: Power grid management via semi-markov afterstate actor-critic. In International Conference on Learning Representations, 2021

    work page 2021

  14. [14]

    Centrally coordinated multi-agent reinforcement learning for power grid topology control

    Barbera de Mol, Davide Barbieri, Jan Viebahn, and Davide Grossi. Centrally coordinated multi-agent reinforcement learning for power grid topology control. In ACM International Conference on Future and Sustainable Energy Systems, 2025

    work page 2025

  15. [15]

    Hierarchical Reinforcement Learning for Power Network Topology Control

    Blazej Manczak, Jan Viebahn, and Herke van Hoof. Hierarchical reinforcement learning for power network topology control. arXiv preprint arXiv:2311.02129, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  16. [16]

    Expert system for topological remedial action discovery in smart grids

    Antoine Marot, Benjamin Donnot, Sami Tazi, and Patrick Panciatici. Expert system for topological remedial action discovery in smart grids. In Mediterranean Conference on Power Generation, Transmission, Distribution and Energy Conversion, 2018

    work page 2018

  17. [17]

    Beyond homophily in graph neural networks: Current limitations and effective designs

    Jiong Zhu, Yujun Yan, Lingxiao Zhao, Mark Heimann, Leman Akoglu, and Danai Koutra. Beyond homophily in graph neural networks: Current limitations and effective designs. Advances in Neural Information Processing Systems, 2020

    work page 2020

  18. [18]

    Heterophily-aware representation learning on heterogeneous graphs

    Jintang Li, Zheng Wei, Yuchang Zhu, Ruofan Wu, Huizhe Zhang, Liang Chen, and Zibin Zheng. Heterophily-aware representation learning on heterogeneous graphs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025

    work page 2025

  19. [19]

    Hinormer: Representation learning on heterogeneous information networks with graph transformer

    Qiheng Mao, Zemin Liu, Chenghao Liu, and Jianling Sun. Hinormer: Representation learning on heterogeneous information networks with graph transformer. In ACM Web Conference, 2023

    work page 2023

  20. [20]

    Generalizable graph neural networks for robust power grid topology control

    Matthijs de Jong, Jan Viebahn, and Yuliya Shapovalova. Generalizable graph neural networks for robust power grid topology control. arXiv preprint arXiv:2501.07186, 2025

    arXiv 2025

  21. [21]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations, 2015

    2015

  22. [22]

    Rusu, Joel Veness, Marc G

    Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Bellemare, Alex Graves, Martin Riedmiller, Andreas K. Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra, Shane Legg, and Demis Hassabis. Human-level control through deep reinforcement l...

    work page 2015

  23. [23]

    Lillicrap, Jonathan J

    Timothy P. Lillicrap, Jonathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. Continuous control with deep reinforcement learning. In International Conference on Learning Representations, 2016

    work page 2016

  24. [24]

    Reinforcement learning (dqn) tutorial - pytorch, 2024

    Adam Paszke. Reinforcement learning (dqn) tutorial - pytorch, 2024. URL https://docs.pytorch.org/tutorials/intermediate/reinforcement_q_learning.html

    work page 2024

  25. [25]

    Reinforcement learning for energies of the future and carbon neutrality: a challenge design

    Ga \"e tan Serr \'e , Eva Boguslawski, Benjamin Donnot, Adrien Pav \ a o, Isabelle Guyon, and Antoine Marot. Reinforcement learning for energies of the future and carbon neutrality: a challenge design. arXiv preprint arXiv:2207.10330, 2022

    work page arXiv 2022

  26. [26]

    URL https://www.artelys.com/app/uploads/2024/04/White_paper_L2RPN_2023_.pdf

    Artelys, 2023. URL https://www.artelys.com/app/uploads/2024/04/White_paper_L2RPN_2023_.pdf

    work page 2023

  27. [27]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    Pith/arXiv arXiv 2017

  28. [28]

    Graph reinforcement learning for power grids: A comprehensive survey

    Mohamed Hassouna, Clara Holzh \"u ter, Pawel Lytaev, Josephine Thomas, Bernhard Sick, and Christoph Scholz. Graph reinforcement learning for power grids: A comprehensive survey. arXiv preprint arXiv:2407.04522, 2024

    arXiv 2024

  29. [29]

    A simulation-constraint graph reinforcement learning method for line flow control

    PeiDong Xu, YangZhou Pei, Xinhu Zheng, and Jun Zhang. A simulation-constraint graph reinforcement learning method for line flow control. In IEEE Conference on Energy Internet and Energy System Integration, 2020

    work page 2020

  30. [30]

    Learning to run a power network under varying grid topology

    Shams Taha, Jan Poland, Katarina Knezovic, and Dmitry Shchetinin. Learning to run a power network under varying grid topology. In IEEE International Energy Conference, 2022

    work page 2022

  31. [31]

    Active power correction strategies based on deep reinforcement learning -- part I : A simulation-driven solution for robustness

    Peidong Xu, Jiajun Duan, Jun Zhang, Yangzhou Pei, Di Shi, Zhiwei Wang, Xuzhu Dong, and Yuanzhang Sun. Active power correction strategies based on deep reinforcement learning -- part I : A simulation-driven solution for robustness. CSEE Journal of Power and Energy Systems, 2022

    work page 2022

  32. [32]

    Optimizing Power Grid Topologies with Reinforcement Learning: A Survey of Methods and Challenges

    Erica van der Sar, Alessandro Zocca, and Sandjai Bhulai. Optimizing power grid topologies with reinforcement learning: A survey of methods and challenges. arXiv preprint arXiv:2504.08210, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  33. [33]

    , " * write output.state after.block = add.period write

    ENTRY address author booktitle chapter doi edition editor eid howpublished institution journal key month note number organization pages publisher school series title type url volume year label INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.all := #1 'mid.sentence := #2 'after.sentence := #3 '...

  34. [34]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize ":" * " " *...

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.