Pith. sign in

REVIEW 4 major objections 4 minor 29 references

Multi-Objective Reinforcement Learning for Power Grid Topology Control

T0 review · 4 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that a multi-policy MORL approach can generate Pareto-optimal topology-control policies that survive grid contingencies longer and train more efficiently than single-objective RL.

desk verdict Genuine first multi-policy MORL for topology control, but the headline MO-vs-SO gains are a best-of-N artifact and the IGD evaluation is circular. read the letter →

arxiv 2502.00040 v2 pith:BGN7XQ6V submitted 2025-01-27 cs.LG cs.AIcs.SYeess.SY

Thomas Lautenbacher , Ali Rajaei , Davide Barbieri , Jan Viebahn , Jochen L. Cremer This is my paper · ORCID
classification cs.LGcs.AIcs.SYeess.SY
keywords multi-objectivereinforcementlearningpowergridtopologycontrolParetofrontdeepoptimisticlinearsupportPPOlineloadingswitchingfrequencytransmissioncongestionmanagement
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that power-grid topology control is inherently multi-objective and that reinforcement learning should reflect that. The authors build a multi-objective RL approach that returns a set of Pareto-optimal policies trading off line loading, topological deviation, and switching frequency, instead of a single policy. In a five-bus benchmark grid, the resulting policies keep the grid from failing longer than a single-objective policy under simulated N-1 line outages (up to roughly 39% longer under moderate contingency frequency) and when training is cut to half (about 22% longer). The paper also reports that the deep optimistic linear support outer loop gives a denser, closer approximation of the Pareto front than random weight sampling. If correct, the approach gives operators an explicit map of operational trade-offs rather than one black-box action policy.

What carries the argument

The engine is the combination of deep optimistic linear support (DOL) with multi-objective PPO (MOPPO). DOL is an outer-loop algorithm that repeatedly proposes a scalarization weight vector w, prioritized by expected improvement, and MOPPO trains a policy with that weight: a vectorized critic estimates a value vector in R^d, advantages are computed per objective, then scalarized as A_t = w^T A_t before the policy update. The resulting average reward vector V is added to a convex coverage set if it is Pareto-optimal for some weight. The reward vector has three components: line-loading margin (squared thermal margins), topological deviation (piecewise penalty for substations moved away from the default busbar configuration), and switching frequency (piecewise penalty for accumulated switching actions within an interval). The policy selection step later picks, per seed, the non-extremal policy with the highest episode duration E.

What would settle it

Retrain the single-objective baseline many times per seed and select its best policy by episode duration, or evaluate the multi-objective policy selected by a fixed rule such as the front center, and compare mean episode durations on the same contingency and budget settings. If the gap drops below the reported 22-39%, the central claim of multi-objective advantage is not supported. Rerunning the comparison on a different grid or with a different line-failure distribution would also test whether the effect is specific to the five-bus setup.

Watch

Extended reading notes

Core claim

The central claim is that a multi-policy MORL pipeline can solve the topology-control problem better than a single-objective RL baseline. The pipeline trains multiple policies via a multi-objective PPO variant, each scalarized by a different weight vector supplied by deep optimistic linear support, and collects the resulting value vectors into a convex coverage set. On the paper's five-bus case studies, policies from this set outperform a line-loading-only PPO policy in mean episode duration under all tested N-1 contingency frequencies: 94.83% vs 82.66% of the maximum episode with no contingencies, 97.68% vs 58.61% with moderate contingencies, and 90.33% vs 66.47% with high contingency frequency. Under a 50% training budget the multi-objective policies reach 95.27% mean episode duration versus 73.39% for the single-objective policy. Compared with random weight sampling, DOL produces a 50% lower sparsity and a 60% lower inverted generational distance, with similar hypervolume.

Load-bearing premise

The reported advantage depends on comparing the best multi-objective policy chosen per random seed by episode duration against a single single-objective policy from the same seed; if the single-objective baseline had been chosen as the best of several runs or by the same selection rule, the reported advantage could shrink or disappear.

Editorial extensions

If this is right

  • Operators can be offered a menu of policies with explicit trade-offs (e.g., low switching frequency vs low line loading) instead of one fixed policy.
  • Multi-objective training appears to act as a regularizer: policies trained to also minimize topology changes and switching survive contingencies longer than single-objective policies.
  • Under limited compute, multi-objective policies reach usable behavior with half the training interactions, which matters as grid size grows.
  • DOL's denser Pareto-front approximation makes the method a plausible decision-support layer on top of existing single-objective RL controllers.
  • The convex coverage set can be extended as new objectives (cost, emissions) are added without retraining from scratch.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The headline comparison may overstate the advantage: the multi-objective policy is selected per seed as the best of the generated front, while the single-objective policy is one run. Re-running the single-objective baseline many times and selecting the best, or fixing a selection rule such as the Pareto-front center, would test how much of the 30%/20% gap is due to this selection asymmetry.
  • A second testable consequence is that multi-objective rewards act as a curriculum or regularizer in low-data regimes; one could verify by ablating each reward component separately to see which one drives early robustness.
  • On larger grids the trade-off geometry will change; the same DOL+MOPPO pipeline could be evaluated on a bigger benchmark to see whether the Pareto front remains dense and whether selection by episode duration remains a good proxy for operator preference.
  • Because the method returns a convex coverage set, it could be coupled downstream with an operator utility function or market prices to choose a policy automatically, which the paper does not explore.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes a multi-policy MORL approach based on deep optimistic linear support (DOL) and multi-objective PPO (MOPPO) for power grid topology control. Custom reward functions are designed for line loading, topological deviation, and switching frequency, and the DOL outer loop generates a set of policies approximating a convex coverage set. Experiments on the RTE 5-bus Grid2Op environment compare DOL with random sampling using hypervolume, sparsity, and IGD; additional case studies compare MO policies with a single-objective (SO) PPO policy under N-1 contingencies and reduced training budgets, reporting improved episode duration of up to 39% and 22% respectively.

Significance. If the quantitative claims were sound, the paper would make a useful contribution: it would be among the first to apply multi-policy MORL to topology control and would provide a decision-support tool that explicitly surfaces trade-offs among operational objectives. The paper is transparent about its scope (5-bus system, limited objectives) and ships publicly available code, which strengthens reproducibility. The qualitative finding that DOL yields a denser coverage of the Pareto front than random sampling is plausible and useful. However, the headline quantitative comparisons are undermined by an asymmetric best-of-set policy selection and by an IGD evaluation whose reference set is constructed from the solutions being scored. These issues need to be resolved before the main claims can be accepted.

major comments (4)
  1. [Section II-D, Algorithm 3; Tables II and III] The MO policy used in the comparison is selected per seed as the policy with maximum episode duration E among all non-extremal DOL-generated policies, while the SO baseline is a single PPO policy trained on the line-loading reward. This is an asymmetric best-of-N comparison. If E is noisy across policies and seeds, taking the maximum over a policy set inflates the MO mean even when no individual MO policy is better than the SO policy, so the reported gains of 39.07% in Table II and 21.88% in Table III are not attributable to multi-objective training without equalizing selection effort. Please either give the SO baseline the same selection effort (e.g., best of K restarts or seeds), use a fixed a priori rule such as the Pareto-center policy, or report the full distribution of E for all MO policies and show per-seed paired differences with confidence intervals. The exclusion of extreme-weight policies, which are closest to the SO objective, should also be justified because it further tilts the comparison.
  2. [Section III-B, Table I, Figs. 3-5] The 'super CCS' used as the IGD reference is constructed as the convex set over all generated solution sets from both DOL and RS runs across seeds, and IGD of each method is then measured against that same set. Since DOL contributes many points to this reference, DOL's lower IGD is at least partly an artifact of the evaluation design. Please measure IGD against a reference that does not contain the evaluated solutions, for example the exact convex hull for the small 5-bus problem, a leave-one-seed-out reference, or a reference built from an independent exhaustive search. Without such a reference, the IGD column in Table I should not be used to support the claim that DOL approximates the true CCS better.
  3. [Section III-C and III-D, Tables II and III] The paper reports only means over 20 seeds and percent differences, with no standard deviations, confidence intervals, or significance tests. This is especially important because Algorithm 3 performs a selection step over policies, which can inflate the mean and the variance of the reported E. Please report seed-level distributions, standard errors or bootstrap confidence intervals, and a paired statistical test (e.g., Wilcoxon signed-rank on per-seed episode durations) for both the contingency and training-budget comparisons.
  4. [Section II-D and Sections III-C/III-D] The text does not state whether the E values used in Tables II and III are computed on the same scenarios and episodes that were used to select the best MO policy in Algorithm 3, or whether selection is performed on validation scenarios and the reported values come from held-out test scenarios. If the same E and scenarios are used both for selection and for final evaluation, the reported improvement contains test-set selection bias. Please clarify the protocol, and if selection currently uses test data, move it to validation data or replace it with a fixed a priori policy-selection rule.
minor comments (4)
  1. [Abstract vs. Section III-C] The abstract states that MO policies are '30% more successful' under contingencies, but Table II reports 10.17%, 39.07%, and 23.86% for the three contingency levels, and the conclusion says 24%; please make the reported percentages consistent across the abstract, tables, and conclusion.
  2. [Section II-C and Algorithm 1] There are several typos: 'choosen' should be 'chosen', 'closet w' should be 'closest w', and Algorithm 1 line 4 contains an unbalanced parenthesis: 'collect samples (MOPPOθ)' should be 'collect samples with MOPPOθ'.
  3. [Figs. 3-5] The legend distinguishes 'DOL Points' and 'DOL CCS Points', but the caption says the plots show points from DOL and RS runs; please clarify which markers correspond to which methods and how the 'super CCS' curve is derived from the combined point sets.
  4. [Section III-A] The sentence 'as only little scenarios are available' should be rephrased, and the text should state clearly whether the 16/2/2 scenario split is used consistently for the Pareto-front, contingency, and training-budget case studies, and whether contingency scenarios are applied during training or only at evaluation.

Circularity Check

2 steps flagged · score 6.0 of 10

Max-of-N policy selection and a self-built reference set make the headline MO-vs-SO and IGD advantages partly by construction.

  1. self definitional [Section III-B, Pareto Front Approximation (text near Figs. 3-5 and Table I)]
    "The super CCS is constructed as the convex set over all generated solution sets across all seeds and both DOL and RS generated solutions. The Super CCS here serves as an indicator for the assumed true CCS."

    The 'assumed true' Pareto front used as the IGD reference is defined as the convex hull of the very solution sets whose IGD is being measured. Since DOL contributes more points to this hull, each DOL point lies on the reference set by construction, so its IGD is small partly because the reference was built from it. The reported 60% IGD reduction (DOL 0.84 vs RS 2.22) is therefore entangled with the evaluation construction rather than being an independent measure of closeness to a separately defined true Pareto front.

  2. fitted input called prediction [Section II-D Algorithm 3; Section III-C Table II; Section III-D Table III]
    "After generating the complete set of policies, we select the best-performing policy (πM O) for each seed run based on (E). Policies trained solely on extreme weights are excluded from consideration, as the focus is on selecting policies optimized for multiple rewards."

    The same metric E used to select the best of many MO policies is then used to report MO-vs-SO 'improved episode duration' in Tables II-III, while the SO policy is a single PPO run without equivalent selection over restarts. Episode duration is noisy; selecting the maximum over the Pareto set raises the mean even if no individual MO policy beats SO. The claimed 39.07% (moderate contingencies) and 21.88% (low budget) advantages therefore measure best-of-N selection rather than an intrinsic benefit of multi-objective learning, especially since only means over 20 seeds are reported, with no paired differences or standard deviations for the comparison.

full rationale

Two load-bearing evaluation steps are circular or partly circular. First, the 'super CCS' used as the assumed true Pareto front is built from the very DOL and RS solution sets being compared, so DOL's substantially better IGD is partly constructed by including its own points in the reference. Second, the headline robustness and sample-efficiency gains (Tables II-III) are obtained by selecting, for each seed, the MO policy with the maximum episode duration from a Pareto set (Algorithm 3) and then comparing that maximum against a single SO PPO policy; the reported metric is the same E used for selection, so the 39%/22% deltas are best-of-N artifacts unless the SO policy receives equivalent selection effort or a fixed a priori policy-selection rule is used. The paper's other self-citations, such as [4] for objective definitions, are not load-bearing: the reward functions are stated explicitly and the MORL algorithms rely on standard external references. The small-grid limitations are acknowledged and do not themselves constitute circularity. Overall, the central claims are partially circular, corresponding to a score of 6.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claims rest on hand-tuned reward shaping parameters, on the choice of episode duration as the selection metric, and on a circularly defined reference set for Pareto front evaluation. No new physical or algorithmic entities are introduced.

free parameters (4)
  • reward thresholds and coefficients = not reported
    dthreshold, F_th_low, rD_default, rD_low, rD_high, rF_default, rF_low, and rF_high are hand-tuned (optimized using empirical experience, Section III-A) and not listed in the paper; these define the reward shaping that drives all policies.
  • PPO hyperparameters = learning rate 5e-4, batch size 512, 4 update cycles
    Taken from prior work and empirical tuning; standard but still free choices that affect results.
  • HV reference point = not reported
    Hypervolume in Table I requires a reference point, which is not specified, making the HV values uninterpretable.
  • kmax (maximum DOL iterations) = not reported
    The DOL stopping criterion is replaced with a predefined kmax, but its value is never stated; it controls the size of the Pareto set and the computational budget.
assumptions (5)
  • domain assumption Grid2Op RTE 5-bus environment is a faithful model of relevant transmission grid dynamics
    The experiments are entirely within this simulator; the paper acknowledges it does not reflect real-world complexity (Section IV).
  • domain assumption Linear scalarization with weight vector w captures the set of Pareto-optimal policies of interest
    MOPPO uses w-transposed advantages (Equation 2); linear scalarization cannot recover non-convex regions of the Pareto front, a known limitation of DOL and OLS that the paper does not discuss.
  • domain assumption Episode duration E is an appropriate, rewards-independent metric for operator preference
    Algorithm 3 selects policies by E, but E is only a survival-time proxy; no evidence is given that it matches operator utility beyond robustness.
  • standard math PPO and MOPPO converge reliably on the 5-bus environment under the chosen hyperparameters
    The paper relies on standard PPO convergence properties without formal guarantees; results are averaged over seeds.
  • ad hoc to paper The super CCS is a valid approximation of the true convex coverage set
    Defined in Section III-B as the convex hull of the very solution sets being compared; it is used as the true reference for IGD, which is circular.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Multi-Objective Reinforcement Learning for Power Grid Topology Control." pith.science (2026). https://pith.science/paper/BGN7XQ6V

@misc{pith2026250200040,
  author       = {Pith},
  title        = {Pith review of: Multi-Objective Reinforcement Learning for Power Grid Topology Control},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BGN7XQ6V}},
  note         = {Machine review of arXiv:2502.00040}
}
read the original abstract

Transmission grid congestion increases as the electrification of various sectors requires transmitting more power. Topology control, through substation reconfiguration, can reduce congestion but its potential remains under-exploited in operations. A challenge is modeling the topology control problem to align well with the objectives and constraints of operators. Addressing this challenge, this paper investigates the application of multi-objective reinforcement learning (MORL) to integrate multiple conflicting objectives for power grid topology control. We develop a MORL approach using deep optimistic linear support (DOL) and multi-objective proximal policy optimization (MOPPO) to generate a set of Pareto-optimal policies that balance objectives such as minimizing line loading, topological deviation, and switching frequency. Initial case studies show that the MORL approach can provide valuable insights into objective trade-offs and improve Pareto front approximation compared to a random search baseline. The generated multi-objective RL policies are 30% more successful in preventing grid failure under contingencies and 20% more effective when training budget is reduced - compared to the common single objective RL policy.

Figures

Figures reproduced from arXiv: 2502.00040 by the authors.

Figure 1
Figure 1. Schematic of the proposed MORL approach with deep [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Schematic of the RTE 5-bus system with busbar splitting on substation 3. V is the sampled vectorized value-estimation V(s) over the evaluation episodes, based on the received rewards; hence, V does not necessarily reflect the vectorized state dependent value function V(s) (1). If the new value vector improves the coverage set Ω s , obsolete corner weights are deleted, the new value vector is added to the set, the mo… view at source ↗
Figure 3
Figure 3. 2D Projection of Super CCS for line loading reward vs [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: 2D Projection of Super CCS for line loading reward vs [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: 2D Projection of Super CCS for topological deviation reward [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

29 extracted references · 23 canonical work pages

  1. [1]

    An optimal transmission line switching and bus splitting heuristic incorporating ac and n-1 contingency constraints,

    M. Heidarifar, P. Andrianesis, P. Ruiz, M. C. Caramanis, and I. C. Paschalidis, “An optimal transmission line switching and bus splitting heuristic incorporating ac and n-1 contingency constraints,” Interna- tional Journal of Electrical Power & Energy Systems , vol. 133, p. 107278, 2021

    work page 2021

  2. [2]

    An iterative approach to grid topology and redispatch optimization in congestion management,

    A. Ewerszumrode, N. Erle, S. Krahl, and A. Moser, “An iterative approach to grid topology and redispatch optimization in congestion management,” Electric Power Systems Research , vol. 234, p. 110700, 2024

    work page 2024

  3. [3]

    Learning to run a power network challenge for training topology controllers,

    A. Marot, B. Donnot, C. Romero, B. Donon, M. Lerousseau, L. Veyrin- Forrer, and I. Guyon, “Learning to run a power network challenge for training topology controllers,” Electric Power Systems Research , vol. 189, p. 106635, 2020

    2020

  4. [4]

    Gridoptions tool: Real-world day-ahead congestion management using topological remedial actions,

    J. Viebahn, S. Kop, J. v. Dijk, H. Budaya, M. Streefland, D. Barbieri, P. Champion, M. Jothy, V . Renault, and S. Tindemans, “Gridoptions tool: Real-world day-ahead congestion management using topological remedial actions,” CIGRE, 2024

    work page 2024

  5. [5]

    Transmission congestion management via node-breaker topology con- trol,

    S. Babaeinejadsarookolaee, B. Park, B. Lesieutre, and C. L. DeMarco, “Transmission congestion management via node-breaker topology con- trol,” IEEE Systems Journal , 2023

    work page 2023

  6. [6]

    Expert system for topological remedial action discovery in smart grids,

    A. Marot, B. Donnot, S. Tazi, and P. Panciatici, “Expert system for topological remedial action discovery in smart grids,” in Mediterranean Conference on Power Generation, Transmission, Distribution and En- ergy Conversion (MEDPOWER 2018) . IET, 2018, pp. 1–6

    work page 2018

  7. [7]

    Substation reconfiguration selection algorithm based on ptdfs for congestion management and rl approach,

    I. Hrgovi ´c and I. Pavi ´c, “Substation reconfiguration selection algorithm based on ptdfs for congestion management and rl approach,” Expert systems with applications , vol. 257, p. 125017, 2024

    work page 2024

  8. [8]

    Managing power grids through topology actions: A comparative study between advanced rule-based and reinforcement learning agents,

    M. Lehna, J. Viebahn, A. Marot, S. Tomforde, and C. Scholz, “Managing power grids through topology actions: A comparative study between advanced rule-based and reinforcement learning agents,” Energy and AI, vol. 14, p. 100276, 2023

    work page 2023

Show all 29 references
  1. [9]

    Reinforce- ment learning for electricity network operation,

    A. Kelly, A. O’Sullivan, P. de Mars, and A. Marot, “Reinforce- ment learning for electricity network operation,” arXiv preprint arXiv:2003.07339, 2020

    2003 arXiv

  2. [10]

    Learning to run a power network challenge: a retrospective analysis,

    A. Marot, B. Donnot, G. Dulac-Arnold, A. Kelly, A. O’Sullivan, J. Viebahn, M. Awad, I. Guyon, P. Panciatici, and C. Romero, “Learning to run a power network challenge: a retrospective analysis,” in NeurIPS 2020 Competition and Demonstration Track . PMLR, 2021, pp. 112– 132

    2020

  3. [11]

    Learning to run a power network with trust,

    A. Marot, B. Donnot, K. Chaouache, A. Kelly, Q. Huang, R.-R. Hossain, and J. L. Cremer, “Learning to run a power network with trust,” Electric Power Systems Research, vol. 212, p. 108487, 2022

    2022

  4. [12]

    Ai-based autonomous line flow control via topology adjustment for maximizing time-series atcs,

    T. Lan, J. Duan, B. Zhang, D. Shi, Z. Wang, R. Diao, and X. Zhang, “Ai-based autonomous line flow control via topology adjustment for maximizing time-series atcs,” in 2020 IEEE Power & Energy Society General Meeting (PESGM) . IEEE, 2020, pp. 1–5

    2020

  5. [13]

    Winning the l2rpn challenge: Power grid management via semi-markov afterstate actor- critic,

    D. Yoon, S. Hong, B.-J. Lee, and K.-E. Kim, “Winning the l2rpn challenge: Power grid management via semi-markov afterstate actor- critic,” in International Conference on Learning Representations , 2020

    2020

  6. [14]

    Exploring grid topology reconfiguration using a simple deep reinforce- ment learning approach,

    M. Subramanian, J. Viebahn, S. H. Tindemans, B. Donnot, and A. Marot, “Exploring grid topology reconfiguration using a simple deep reinforce- ment learning approach,” in 2021 IEEE Madrid PowerTech . IEEE, 2021, pp. 1–6

    2021

  7. [15]

    Powrl: A reinforcement learning framework for robust management of power networks,

    A. Chauhan, M. Baranwal, and A. Basumatary, “Powrl: A reinforcement learning framework for robust management of power networks,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 37, no. 12, 2023, pp. 14 757–14 764

    2023

  8. [16]

    Power grid congestion management via topology optimization with alphazero,

    M. Dorfer, A. R. Fuxj ¨ager, K. Kozak, P. M. Blies, and M. Wasserer, “Power grid congestion management via topology optimization with alphazero,” arXiv preprint arXiv:2211.05612 , 2022

    2022 arXiv

  9. [17]

    Curriculum based reinforcement learning of grid topology controllers to prevent thermal cascading,

    A. R. R. Matavalam, K. P. Guddanti, Y . Weng, and V . Ajjarapu, “Curriculum based reinforcement learning of grid topology controllers to prevent thermal cascading,” IEEE Transactions on Power Systems , 2022

    2022

  10. [18]

    A hybrid curriculum learning and tree search approach for network topology control,

    G. J. Meppelink, A. Rajaei, and J. L. Cremer, “A hybrid curriculum learning and tree search approach for network topology control,”Electric Power Systems Research, vol. 242, p. 111455, 2025

    2025

  11. [19]

    Hierarchical reinforcement learning for power network topology control

    B. Manczak, J. Viebahn, and H. van Hoof, “Hierarchical reinforcement learning for power network topology control.”

  12. [20]

    Multi-agent reinforcement learning for power grid topology optimization

    E. van der Sar, A. Zocca, and S. Bhulai, “Multi-agent reinforcement learning for power grid topology optimization.”

  13. [21]

    Reward design for intelligent deep reinforce- ment learning based power flow control using topology optimization,

    I. Hrgovi ´c and I. Pavi ´c, “Reward design for intelligent deep reinforce- ment learning based power flow control using topology optimization,” Sustainable energy, grids and networks , p. 101580, 2024

    2024

  14. [22]

    Prox- imal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Prox- imal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  15. [23]

    A toolkit for reliable benchmarking and research in multi-objective reinforcement learning,

    F. Felten, L. N. Alegre, A. Nowe, A. Bazzan, E. G. Talbi, G. Danoy, and B. C da Silva, “A toolkit for reliable benchmarking and research in multi-objective reinforcement learning,” Advances in Neural Information Processing Systems, vol. 36, 2024

    2024

  16. [24]

    Multi-objective deep reinforcement learning,

    H. Mossalam, Y . M. Assael, D. M. Roijers, and S. White- son, “Multi-objective deep reinforcement learning,” arXiv preprint arXiv:1610.02707, 2016

    2016 arXiv

  17. [25]

    A practical guide to multi- objective reinforcement learning and planning,

    C. F. Hayes, R. R ˘adulescu, E. Bargiacchi, J. K ¨allstr¨om, M. Macfarlane, M. Reymond, T. Verstraeten, L. M. Zintgraf, R. Dazeley, F. Heintz, E. Howley, A. A. Irissappane, P. Mannion, A. Now ´e, G. Ramos, M. Restelli, P. Vamplew, and D. M. Roijers, “A practical guide to multi...

    2022

  18. [26]

    Grid2op,

    RTE France, “Grid2op,” 2020. [Online]. Available: https://github.com/ rte-france/Grid2Op

    2020

  19. [27]

    Multi-objective decision-theoretic planning,

    D. M. Roijers, “Multi-objective decision-theoretic planning,” AI Matters, vol. 2, no. 4, pp. 11–12, 2016

    2016

  20. [28]

    DelftBlue Supercomputer (Phase 2),

    Delft High Performance Computing Centre (DHPC), “DelftBlue Supercomputer (Phase 2),” https://www.tudelft.nl/dhpc/ark: /44463/DelftBluePhase2, 2024

    2024

  21. [29]

    Implementation of multi-objective reinforcement learning for power grid topology control,

    T. R. Lautenbacher, A. Rajaei, J. Viebahn, D. Barbieri, and J. Cremer, “Implementation of multi-objective reinforcement learning for power grid topology control,” 2025. [Online]. Available: https: //github.com/TU-Delft-AI-Energy-Lab/TOPGRID MORL 7

    2025

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.