Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Robust Defense Against Extreme Grid Events Using Dual-Policy Reinforcement Learning Agents

T0 review · 3 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A dual-policy reinforcement learning agent keeps a simulated power grid alive for 95–99 of 100 steps under N-k contingency cascades, far outlasting an agentless baseline.

desk verdict A readable proof of concept whose headline survival numbers are likely inflated by a threshold-exploitation loophole in the opponent rule; the underlying idea deserves a serious look with a major revision. read the letter →

arxiv 2411.11180 v1 pith:JHODPFNZ submitted 2024-11-17 eess.SY cs.LGcs.SY

Benjamin M. Peter , Mert Korkali This is my paper · ORCID
classification eess.SYcs.LGcs.SY
keywords reinforcementlearningpowergridsecuritycontingencyanalysisN-kscreeninggraphneuralnetworksproximalpolicyoptimizationdual-policyagentGrid2Op
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a reinforcement learning agent with two policies—one for normal operation and one for crises—can keep a simulated power grid alive through extreme N-k contingency cascades, the simultaneous failure of k lines, for far longer than taking no action at all. The authors train a proximal policy optimization agent with a graph neural network on the Grid2Op platform, then screen it against an opponent that repeatedly disconnects the most heavily loaded lines. On a modified IEEE 14-bus grid with 25% added loading, the agent survives between 95 and 99 of 100 steps across k=1 through k=5 initial failures, while the no-agent baseline falls from 70 steps at k=1 to under one step at k=5. The intended upshot is that RL agents could serve as a fast, automatic complement to traditional N-k security assessment, especially for blackout-level events in which a human operator has little time to respond.

What carries the argument

The load-bearing mechanism is the dual-policy switching rule in Eq. (10): when the maximum line loading exceeds a threshold $\rho_{\mathrm{threshold}}$, a 'critical' proximal policy optimization policy acts; otherwise a 'general' policy handles normal conditions. Around this, the opponent in Eq. (14) attacks at every step by disconnecting all lines whose loading is at or above $\rho_{\mathrm{threshold}}$, and disconnects all lines if none qualify. The custom reward in Eqs. (4)–(8) rewards doing nothing, penalizes arbitrary actions, adds a logarithmic bonus for survival time, and penalizes overloaded lines. All three components share the same threshold, so the agent must keep at least one line above $\rho_{\mathrm{threshold}}$ at every step merely to stop the opponent from disconnecting everything, while keeping enough other lines below rated capacity to avoid cascading failure. This shared threshold is what carries the whole argument.

What would settle it

Re-run the N-k screening with the opponent's rule changed so that when no line exceeds $\rho_{\mathrm{threshold}}$ it disconnects only the single most-loaded line (or a fixed random subset) rather than all lines, and compare the agent's average survival time; if survival collapses toward NoAgent levels, the reported robustness is an artifact of the threshold-coupled attack rule. A second check is to retrain the agent without $\rho_{\mathrm{threshold}}$ in the reward or switching function and see whether the flat survival curve persists.

Watch

Extended reading notes

Core claim

The central claim is that a dual-policy PPO+GNN agent can 'efficiently adjust for the initial outages and, therefore, offer consistent performance' under N-k contingency screening, with average survival of 95.10 steps for k=1 rising to 99.13 steps for k=4 out of a 100-step limit, against an opponent that disconnects heavily loaded lines at every time step. The authors interpret this near-flat survival curve as evidence of robustness to extreme events, in contrast to the no-agent baseline, whose survival falls from 70.20 to 4.56 across the same k range and effectively to zero by k=5. The paper further claims that this is the first proof-of-concept applying RL agents in a contingency-screening context, proposing it as a novel alternative to traditional security assessment.

Load-bearing premise

The load-bearing premise is that the opponent's attack rule—disconnect every line whenever no line is above the threshold—combined with the same threshold in the agent's switching and reward, produces a realistic stress test; if keeping one line artificially above that threshold is the main reason the agent survives, the claimed defense may not transfer to real attacks or weather events.

Editorial extensions

If this is right

  • If the agent's results hold, an RL agent can be screened across all N-k contingency sets for k=1 to 5 on a 14-bus Grid2Op environment and maintain near-constant survival as k increases, a capability the paper reports no prior agent work in contingency screening has demonstrated.
  • The dual-policy design, with a general policy for normal operation and a critical policy for overload events, keeps survival consistent even when the opponent acts every time step, supporting the authors' claim of robustness to continuously evolving extreme events.
  • Since the reward function prioritizes survival time, the agent also reduces cascading failures in practice, although the paper notes a conflict between survival maximization and cascade minimization when the latter is optimized directly.
  • The same dual-policy PPO+GNN scheme is transferable to larger grids through the Grid2Op platform, which the authors identify as the immediate next step.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not vary $\rho_{\mathrm{threshold}}$, so an immediate test is to scan the threshold while holding the opponent rule fixed; if survival time collapses or becomes non-monotonic at certain thresholds, the robustness is tuned to the threshold rather than to grid reconfiguration.
  • Because the opponent's 'disconnect everything when nothing is loaded' rule reads the agent's own threshold, the threat model is coupled to the defender; a real adversary would not honor that threshold, so the N-k screening as defined likely overstates the agent's defense against adaptive attacks.
  • Surviving a step may not mean the grid is healthy: the agent may learn to keep one sacrificial line above $\rho_{\mathrm{threshold}}$ to distract the opponent, which would inflate survival metrics while leaving the grid nearer to its limits; inspecting line loadings in the surviving traces would show whether the strategy is genuine reconfiguration or threshold gaming.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a dual-policy reinforcement learning agent for power grid topology control under extreme events. The agent combines PPO with a graph convolutional encoder, a custom reward function that rewards survival and penalizes overloads, and a switch mechanism that selects a 'general' or 'critical' policy depending on line loadings. An opponent disconnects highly loaded lines at every time step, and the agent is evaluated by exhaustive N-k contingency screening on a modified IEEE 14-bus grid. The main reported result is Table II, in which the agent survives roughly 95-99 of 100 steps across k=1..5, while a NoAgent baseline falls from 70.20 to 0.93 steps as k increases. The paper concludes that the agent can efficiently adjust for initial outages and is a novel alternative to traditional security assessment.

Significance. If the reported behavior is genuine, the paper is a useful proof-of-concept for using RL in N-k contingency screening: it combines a graph-based state representation, a dual-policy switch, and an iterative adversary in the Grid2Op framework. The exhaustive screening over all k-combinations is a strength, and the explicit reward decomposition makes the training objective transparent. However, the central empirical claim rests on a single comparison against a passive NoAgent baseline, and the opponent rule in Eq. (14) may itself create the agent's survival advantage. Because the paper provides no error bars, no held-out evaluation, and no ablations, the significance of the contribution is presently not established beyond a demonstration that an RL agent can exploit the specific opponent and reward structure.

major comments (3)
  1. [III-E, Eq. (14); III-D, Eq. (10); IV, Table II] The opponent's fallback rule confounds the headline result. When Lhigh is empty, the opponent disconnects all remaining lines, so any agent that survives an opponent step must keep at least one line with rho_i >= rho_threshold at every attack time. The reward in Eq. (6) penalizes rho_i > rho_threshold but not rho_i = rho_threshold, so the highest-reward strategy under this opponent is to maintain a sacrificial line at the threshold and let the opponent disconnect it, rather than to reduce overloads. NoAgent cannot perform such threshold management, and its rapid death once loadings fall below rho_threshold is built into the opponent rule. The paper never reports the numeric value of rho_threshold and never analyzes this coupling. The near-flat agent survival for k=2..5 in Table II is exactly the signature of threshold exploitation rather than robust reconfiguration. An ablation with a non-threshold opponent, such as an opponent that always disconnects the highest-loaded line or a fixed attack set, together with a report of rho_threshold, is needed to support the claim that the results transfer to real cyber or weather threats.
  2. [IV, Table II; III-B] The central quantitative comparison is reported without error bars, training seeds, or a held-out evaluation split. Table II gives one average per k for Agent and NoAgent, so the reader cannot assess run-to-run variance. Moreover, Section III-B states that multiple reward renditions were 'developed and tested' on this environment, and the reported metric is survival time, the same quantity the reward is tuned to maximize. This is model selection on the test metric. The authors should report means and standard deviations over at least five independent training seeds, specify which scenarios were seen during training versus evaluation, and provide an out-of-sample evaluation such as training on a subset of contingency sets or load/attack profiles and testing on the remaining ones.
  3. [IV, Table II; III-C, III-D] The only baseline is NoAgent, which cannot take remedial actions. This comparison does not isolate the contribution of the dual-policy switch, the GCN encoder, or the reward design. A simpler single-policy PPO agent or a rule-based corrective agent might achieve similar survival under this particular opponent, and the paper's earlier statement that reward-function tuning 'only slightly improved performance compared to tuning hyperparameters' suggests the architecture choice is not the dominant factor. To support the claim that the dual-policy PPO+GNN model is the source of the reported robustness, ablations are needed: general policy only, critical policy only, and a non-graph observation encoder, all evaluated under the same N-k screening protocol.
minor comments (5)
  1. [III-F, Eq. (15)] The definition C = {S ⊆ L: |L| = k} should read |S| = k; as written, the condition does not depend on the candidate set S.
  2. [IV, Fig. 5] The text refers to 'modified opponent reward functions,' but the opponent in Section III-E is deterministic and has no reward function; please clarify whether a different opponent rule was used for this figure.
  3. [Abstract and III-E] The abstract and introduction motivate the opponent as modeling cyberattacks, while Section III-E states that the implemented opponent represents an evolving natural disaster; these framings should be reconciled.
  4. [III-C and Table I] The GCN description is underspecified: the number of GCN hidden units, the nonlinearity in each layer, and whether the encoder is shared between the two policies are not given, which limits reproducibility.
  5. [Footnote 1] The claim to be 'the first work to analyze RL agents' performance in a contingency screening context' is very strong and is not supported by a systematic literature search in Section II-A; I recommend softening or substantiating it.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper is an empirical RL evaluation whose survival metric is the openly stated training objective, not a hidden refit or self-citational derivation.

full rationale

This paper is an empirical reinforcement-learning benchmarking study rather than a derivation from first principles, so most circularity patterns do not apply. The reward function (Eqs. 4-8) explicitly rewards survival time, and Table II reports survival time; while this is a training objective being reported as the headline metric, it is not a hidden refit because the N-k contingency screening enumerates all k-line outage combinations and compares the trained agent against a NoAgent baseline. No load-bearing self-citation appears: the cited frameworks (Grid2Op, pandapower, Stable-Baselines3, PyTorch Geometric, PPO) are external tools, and no uniqueness theorem or prior author result is invoked to force the agent architecture. The skeptical concern about Eq. (14) and the shared rho_threshold in Eq. (10) is a legitimate evaluation-validity question: the opponent's all-or-nothing attack rule may reward keeping a sacrificial line above threshold rather than genuinely resolving overloads. However, this is a confound in the experimental design, not a circularity, because the claimed survival scores are not equivalent by construction to the input rules; the agent must still learn and execute actions that survive the opponent, and NoAgent fails under the same rule. Accordingly, no circular step is established by the paper's own equations or citations, and the correct circularity verdict is none.

Assumptions & free parameters 9 free parameters · 6 assumptions · 0 invented entities

The central claim rests mainly on the simulation environment and the custom training and evaluation protocol. The reward coefficients and threshold are free parameters that are not reported, the opponent rule is an ad hoc stress model, and the evaluation uses the same modified Grid2Op environment on which the agent was tuned. These are the elements the reader must take on faith.

free parameters (9)
  • gamma (no-action reward) = not reported
    Reward coefficient for taking no action in Eq. (4); selected by hand, no value given.
  • delta (action penalty) = not reported
    Negative reward for any action in Eq. (4); magnitude not reported.
  • eta (minimal action reward) = not reported
    Reward for a minimal action in Eq. (4); no value given.
  • alpha (survival reward scaling) = not reported
    Scales log survival time in Eq. (5); no value reported.
  • beta (overload penalty coefficient) = not reported
    Penalty for overloaded lines in Eq. (6); no value reported.
  • rho_threshold (line loading threshold) = not reported
    Used in reward (Eq. 6), dual-policy switch (Eq. 10), and opponent targeting (Eq. 13); its value determines both agent behavior and adversary behavior, but is not reported.
  • tau_attack (opponent interval) = 1
    Opponent acts every time step (Eq. 11); chosen to model a continuously aggressive event.
  • PPO hyperparameters (learning rates, entropy, discount factors, hidden sizes) = see Table I
    Table I gives two sets of hyperparameters, chosen by hand; effect of these unusual values (entropy 0.999, gamma near 1e-3) is not analyzed.
  • GCN embedding dimension = 128
    Observation compressed to a 128-dimensional vector (Section III-C); no ablation for this choice.
assumptions (6)
  • domain assumption Grid2Op and pandapower accurately model relevant power grid dynamics via Newton-Raphson power flow and Kirchhoff's laws.
    Sections II-B and III state the environment and backend; the paper assumes this simulation is a valid proxy for real extreme grid events.
  • ad hoc to paper The opponent's attack rule (Eq. 14), which disconnects all high-loaded lines or, if none, all lines, is a meaningful model of an evolving natural disaster or cyberattack.
    Section III-E introduces this rule without validation against real attack or disaster data; it also creates a coupling between the threshold and survival.
  • domain assumption Survival time is the correct primary objective for security assessment, and the reward function's priority on no actions and minimal actions is appropriate.
    Sections III-B and IV state that survival is prioritized; the paper does not justify this against economic or operational metrics.
  • domain assumption The MDP formulation and PPO/GCN training pipeline converge to a policy that generalizes across the N-k contingency sets tested.
    Section III-D and IV report training and evaluation on the same modified Grid2Op environment; no validation on a separate environment or distribution shift is provided.
  • ad hoc to paper The dual-policy switch threshold rho_threshold is well-calibrated and stable across different contingency sets.
    Eq. 10 switches between policies based on a single loading threshold; the paper does not analyze sensitivity to this threshold.
  • standard math The PPO clipped surrogate objective (Eq. 2) and GCN propagation rule (Eq. 9) are correctly implemented by Stable-Baselines3 and PyTorch Geometric.
    Sections III-A and III-C rely on these libraries; the paper does not re-derive or verify them.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Robust Defense Against Extreme Grid Events Using Dual-Policy Reinforcement Learning Agents." pith.science (2026). https://pith.science/paper/JHODPFNZ

@misc{pith2026241111180,
  author       = {Pith},
  title        = {Pith review of: Robust Defense Against Extreme Grid Events Using Dual-Policy Reinforcement Learning Agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JHODPFNZ}},
  note         = {Machine review of arXiv:2411.11180}
}
read the original abstract

Reinforcement learning (RL) agents are powerful tools for managing power grids. They use large amounts of data to inform their actions and receive rewards or penalties as feedback to learn favorable responses for the system. Once trained, these agents can efficiently make decisions that would be too computationally complex for a human operator. This ability is especially valuable in decarbonizing power networks, where the demand for RL agents is increasing. These agents are well suited to control grid actions since the action space is constantly growing due to uncertainties in renewable generation, microgrid integration, and cybersecurity threats. To assess the efficacy of RL agents in response to an adverse grid event, we use the Grid2Op platform for agent training. We employ a proximal policy optimization (PPO) algorithm in conjunction with graph neural networks (GNNs). By simulating agents' responses to grid events, we assess their performance in avoiding grid failure for as long as possible. The performance of an agent is expressed concisely through its reward function, which helps the agent learn the most optimal ways to reconfigure a grid's topology amidst certain events. To model multi-actor scenarios that threaten modern power networks, particularly those resulting from cyberattacks, we integrate an opponent that acts iteratively against a given agent. This interplay between the RL agent and opponent is utilized in N-k contingency screening, providing a novel alternative to the traditional security assessment.

Figures

Figures reproduced from arXiv: 2411.11180 by the authors.

Figure 1
Figure 1. Episode visualization for a certain time step on the modified IEEE 14-bus test case. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 4
Figure 4. The [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 3
Figure 3. Time steps survived for the NoAgent (left) and Agent (right) cases per N − 3 contingency set. The ratio of line loading, denoted by ρ, also provides insight into the agent’s grid-stabilizing capabilities. Maintaining a ρ value less than 1, i.e., the line’s maximum rated capacity, is essential for avoiding failure-inducing grid conditions. The agent’s critical model is specifically trained to quickly regulate any ins… view at source ↗
Figures from the paper (1 more)
Figure 5
Figure 5. Figure 5: Steps survived and cascading failures for an [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Hybrid ML-RL Approach for Smart Grid Stability Prediction and Optimized Control Strategy

    eess.SY 2025-08 reject novelty 3.0 of 10

    A stacking classifier plus DQN is reported to reach 97.88% prediction accuracy and 100% stabilization success on a public smart grid dataset, but the RL environment is unspecified.

Reference graph

Works this paper leans on

27 extracted references · 22 canonical work pages · cited by 1 Pith paper

  1. [14]

    Managing power grids through topology actions: A comparative study between advanced rule-based and reinforcement learning agents,

    M. Lehna, J. Viebahn, A. Marot, S. Tomforde, and C. Scholz, “Managing power grids through topology actions: A comparative study between advanced rule-based and reinforcement learning agents,” Energy AI , vol. 14, p. 100276, 2023

    work page 2023

  2. [1]

    General nonlinear modal representation of large scale power systems,

    H. Shanechi, N. Pariz, and E. Vaahedi, “General nonlinear modal representation of large scale power systems,” IEEE Trans. Power Syst. , vol. 18, no. 3, pp. 1103–1109, Aug. 2003

    work page 2003

  3. [2]

    Efficient and scalable reinforcement learning for large-scale network control,

    C. Ma, A. Li, Y . Du, H. Dong, and Y . Yang, “Efficient and scalable reinforcement learning for large-scale network control,” Nat. Mach. Intell., pp. 1–15, 2024

    work page 2024

  4. [3]

    Deep reinforcement learning for real-time power grid topology optimization,

    J. H. Rothschild, “Deep reinforcement learning for real-time power grid topology optimization,” B.Sc. Thesis, Lund University, 2021

    work page 2021

  5. [4]

    Study on the structural complexity of large scale power grids,

    B. Chen, J. Zhao, L. Qiu, and P. Li, “Study on the structural complexity of large scale power grids,” in Proc. IEEE Power Energy Soc. Gen. Meet., 2013, pp. 1–5

    work page 2013

  6. [5]

    A deep reinforcement learning framework for automatic operation control of power system considering extreme weather events,

    X. Liu, J. Liu, Y . Zhao, and J. Liu, “A deep reinforcement learning framework for automatic operation control of power system considering extreme weather events,” in Proc. IEEE Power Energy Soc. Gen. Meet. , 2022, pp. 1–5

    work page 2022

  7. [6]

    Smart grid vulnerability and defense analysis under cascading failure attacks,

    T. N. Nguyen, B.-H. Liu, N. P. Nguyen, B. Dumba, and J.-T. Chou, “Smart grid vulnerability and defense analysis under cascading failure attacks,” IEEE Trans. Power Del. , vol. 36, no. 4, pp. 2264–2273, Aug. 2021

    work page 2021

  8. [7]

    Learning to run a power network challenge: a retrospective analysis,

    A. Marot et al. , “Learning to run a power network challenge: a retrospective analysis,” 2021, arXiv: 2103.03104

    arXiv 2021

Show all 27 references
  1. [8]

    Reinforcement learning agents,

    C. Ribeiro, “Reinforcement learning agents,” Artif. Intell. Rev. , vol. 17, pp. 223–250, 2002

    2002

  2. [9]

    Intelligent hur- ricane resilience enhancement of power distribution systems via deep reinforcement learning,

    N. L. Dehghani, A. B. Jeddi, and A. Shafieezadeh, “Intelligent hur- ricane resilience enhancement of power distribution systems via deep reinforcement learning,” Appl. Energy, vol. 285, p. 116355, 2021

    2021

  3. [10]

    Curriculum based reinforcement learning of grid topology controllers to prevent thermal cascading,

    A. R. R. Matavalam, K. P. Guddanti, Y . Weng, and V . Ajjarapu, “Curriculum based reinforcement learning of grid topology controllers to prevent thermal cascading,” IEEE Trans. Power Syst. , vol. 38, no. 5, pp. 4206–4220, Sep. 2023

    2023

  4. [11]

    Curriculum-based reinforcement learning for distribu- tion system critical load restoration,

    X. Zhang et al., “Curriculum-based reinforcement learning for distribu- tion system critical load restoration,” IEEE Trans. Power Syst. , vol. 38, no. 5, pp. 4418–4431, Sep. 2023

    2023

  5. [12]

    Resilience enhancement of multi- agent reinforcement learning-based demand response against adversarial attacks,

    L. Zeng, D. Qiu, and M. Sun, “Resilience enhancement of multi- agent reinforcement learning-based demand response against adversarial attacks,” Appl. Energy, vol. 324, p. 119688, 2022

    2022

  6. [13]

    PowerGridworld: A framework for multi-agent reinforcement learning in power systems,

    D. Biagioni et al. , “PowerGridworld: A framework for multi-agent reinforcement learning in power systems,” in Proc. 13th ACM Int. Conf. Future Energy Syst. , 2022, pp. 565–570

    2022

  7. [15]

    A new framework integrating reinforcement learning, a rule-based expert system, and decision tree analysis to improve building energy flexibility,

    X. Zhou et al., “A new framework integrating reinforcement learning, a rule-based expert system, and decision tree analysis to improve building energy flexibility,” J. Build. Eng. , vol. 71, p. 106536, 2023

    2023

  8. [16]

    Grid2Op—A testbed platform to model sequential decision making in power systems,

    B. Donnot, “Grid2Op—A testbed platform to model sequential decision making in power systems,” 2020. [Online]. Available: https://github.com/Grid2Op/grid2op

    2020

  9. [17]

    A Markov decision process to enhance power system operation resilience during hurricanes,

    M. Abdelmalak and M. Benidris, “A Markov decision process to enhance power system operation resilience during hurricanes,” in Proc. IEEE Power Energy Soc. Gen. Meet. , 2021, pp. 1–5

    2021

  10. [18]

    Dynamic power management based on continuous-time Markov decision processes,

    Q. Qiu and M. Pedram, “Dynamic power management based on continuous-time Markov decision processes,” in Proc. 36th Annu. ACM/IEEE Des. Autom. Conf. , 1999, pp. 555–561

    1999

  11. [19]

    Heterogeneous reinforcement learning for defending power grids against attacks,

    M. Moradi et al. , “Heterogeneous reinforcement learning for defending power grids against attacks,” APL Mach. Learn. , vol. 2, no. 2, 2024

    2024

  12. [20]

    Pandapower—An open-source Python tool for con- venient modeling, analysis, and optimization of electric power systems,

    L. Thurner et al. , “Pandapower—An open-source Python tool for con- venient modeling, analysis, and optimization of electric power systems,” IEEE Trans. Power Syst. , vol. 33, no. 6, pp. 6510–6521, Nov. 2018

    2018

  13. [21]

    Proximal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv: 1707.06347

    arXiv

  14. [22]

    Trust region policy optimization,

    J. Schulman, S. Levine, P. Moritz, M. I. Jordan, and P. Abbeel, “Trust region policy optimization,” 2015, arXiv: 1502.05477

    2015 arXiv

  15. [23]

    Stable-Baselines3: Reliable reinforcement learning implementations,

    A. Raffin et al. , “Stable-Baselines3: Reliable reinforcement learning implementations,” J. Mach. Learn. Res. , vol. 22, no. 268, pp. 1–8, 2021

    2021

  16. [24]

    Gymnasium: A standard interface for reinforcement learning environments,

    M. Towers et al. , “Gymnasium: A standard interface for reinforcement learning environments,” 2024, arXiv: 2407.17032

    2024 arXiv

  17. [25]

    Topological graph convolutional networks solutions for power distribution grid planning,

    Y . Chen, M. Heleno, A. Moreira, and Y . R. Gel, “Topological graph convolutional networks solutions for power distribution grid planning,” in Proc. Pac.-Asia Conf. Knowl. Discovery Data Min. Springer, 2023, pp. 123–134

    2023

  18. [26]

    Proximal policy optimization with graph neural networks for optimal power flow,

    ´A. L ´opez-Cardona, G. Bern ´ardez, P. Barlet-Ros, and A. Cabellos- Aparicio, “Proximal policy optimization with graph neural networks for optimal power flow,” 2022, arXiv: 2212.12470

    2022

  19. [27]

    Fast graph representation learning with PyTorch Geometric,

    M. Fey and J. E. Lenssen, “Fast graph representation learning with PyTorch Geometric,” May 2019. [Online]. Available: https: //github.com/pyg-team/pytorch geometric

    2019

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.