REVIEW 4 major objections 5 minor 14 references
A Deep Reinforcement Learning Method for Multi-objective Transmission Switching
T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read This paper claims that a discrete dueling soft actor-critic learns multi-objective transmission switching faster and more stably than PPO and DDQN on the IEEE 118-bus system.
desk verdict The paper is a plausible DRL-for-transmission-switching study, but the action-space definition makes the algorithm unimplementable as written on the 118-bus system, so the central empirical claim doesn't stand. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The identity that carries the argument is the dueling Q decomposition $Q(s,a)=V(s)+\left(A(s,a)-\frac{1}{|A|}\sum_{\tilde a\in A}A(s,\tilde a)\right)$, embedded in a soft actor-critic discretized for transmission switching. The advantage stream is meant to score each line-switching action relative to the average of all actions, while the value stream carries the state's baseline; the SAC entropy term keeps the policy exploring. This decomposition is the mechanism the paper credits for allowing the agent to differentiate among many switching alternatives and to prioritise critical decisions.
What would settle it
Run DDSAC on a small network where all switch configurations can be enumerated, and check whether the dueling update in Eq. (9) reproduces a brute-force advantage computation; then re-run the 118-bus experiment with the action factorisation written explicitly and compare the convergence curves to the reported ones.
Extended reading notes
Core claim
The central claim is that the dueling Q-network identity, which separates the value of a state from the relative advantage of each action, makes a discrete SAC algorithm substantially better at ranking line-switching decisions. In DDSAC, the policy network outputs probabilities over line statuses, the twin Q-networks reduce overestimation bias, and the temperature parameter is tuned automatically to keep exploration at a target entropy. On the IEEE 118-bus system, DDSAC is reported to converge faster, produce higher and more stable cumulative reward, and keep generator cost, voltage violation, active power loss, and line overload lower than both PPO and DDQN across 500 steps.
Load-bearing premise
The method assumes the average over actions in the dueling update can actually be evaluated for the line-switching action space, although no factorized action representation is given and the full joint space has $2^{186}$ elements for the 118-bus system.
Editorial extensions
If this is right
- If the claim holds, dueling discrete SAC is a practical template for learning multi-objective transmission-switching policies, not just for single-objective cost reduction.
- The same DDSAC training loop could be re-run with different weight vectors $w_1$ through $w_5$, giving operators a way to shift the cost-reliability trade-off without redesigning the algorithm.
- Faster convergence within roughly fifty steps suggests the method can be trained with fewer environment interactions, which matters when each step requires solving an AC power flow.
- Stable rewards over five hundred steps imply the learned policy can be deployed in settings where the operating point changes and the agent must keep re-evaluating switch configurations.
Reading between the lines
- The paper leaves the action-space representation implicit; if each line is an independent binary action, the dueling average in Eq. (9) ranges over only two choices, whereas if the joint vector is the action, the average over $2^{186}$ configurations is not computable as written.
- A natural extension is to map the five weight factors onto a Pareto front of cost-reliability trade-offs, since real operators would want to know which switch configurations are optimal for their preferred weighting.
- The same dueling-advantage trick could be tested on other discrete topology decisions, such as capacitor or transformer tap switching, where action sets are combinatorial but structured.
- Benchmarking DDSAC against an exact mixed-integer solver on small networks would show how much of the reported gain comes from the dueling mechanism rather than from the SAC exploration strategy.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper proposes DDSAC, a deep reinforcement learning method for multi-objective transmission switching. The method combines a dueling Q-network with a discrete version of soft actor-critic (SAC) to select line switching actions that minimize a weighted sum of generation cost, voltage violations, line overload, power losses, and switching count, subject to AC power flow constraints. The algorithm is evaluated on the IEEE 118-bus system against two benchmark DRL algorithms (PPO and DDQN). Experimental learning curves are presented showing cumulative rewards and individual objective components over 500 time steps. The paper claims that DDSAC converges faster and achieves higher, more stable rewards than the benchmarks.
Significance. If validated, the proposed DDSAC would be a plausible improvement in DRL for transmission switching by incorporating a dueling advantage stream into discrete SAC. The multi-objective reward formulation is practical, and the benchmark comparison is a standard evaluation approach. However, the manuscript as written contains a load-bearing computational inconsistency in the action-space definition, and the empirical support consists of qualitative curves without numerical or statistical details. The paper does not provide code, environmental details, or final performance tables, which severely limits reproducibility and prevents verification of the central claim.
major comments (4)
- [Section III-B, Eq. (6) and Eq. (9)] The action is defined as the full vector of NL line switching decisions in Eq. (6). For the IEEE 118-bus system, with approximately 186 lines, the action space has size 2^186. Eq. (9) requires averaging the advantage over all actions in A, and Eq. (12) computes a dot product over the probability vector of all actions; Algorithm 1 also samples from this intractable distribution. The paper never specifies a factorized action representation or a restricted alternative set. Consequently, DDSAC as described is computationally infeasible for this system, and the reported learning curves cannot be reproduced from the text.
- [Section IV, Figs. 2 and 3] The central claim of faster convergence and higher, more stable rewards is based only on qualitative visual inspection of the plotted curves. No numerical results, standard deviations, or statistical significance tests are reported. The description of the 10 runs and standard error shading is ambiguous (e.g., whether the curves are training returns or evaluation returns). Without these quantitative details, the claimed advantage of DDSAC over PPO and DDQN is not demonstrated.
- [Section II, Eq. (1a) and Section IV-A] The objective function uses a generator cost term CGi(PGi) without specifying its functional form, preventing reproduction of the cost values in Fig. 3. Moreover, Section IV-A states that the cost term is calculated as the difference between post-switching and initial total costs, while Eq. (1a) defines a plain sum. This inconsistency affects the interpretation of the reward and the weight w1, and it needs to be clarified.
- [Section III-B, Eq. (12) and Algorithm 1] The policy loss in Eq. (12) is written as a dot product πθ(s,a)⊤[α log πθ(a|s) − Q̂(s,a)], which requires the probability vector over the entire action space. This is intractable for the stated action space and also not the standard discrete SAC objective, which uses an expectation over the policy. The lack of a well-defined policy sampling procedure in Algorithm 1 (line 6) compounds the problem. The method section needs to be rewritten to specify a tractable factorized policy.
minor comments (5)
- [Eq. (1b)] The lower and upper voltage bounds in Eq. (1b) are typeset identically in the displayed text; please ensure they are visually distinct (e.g., using \underline{V} and \overline{V}).
- [Eq. (5)] The state vector in Eq. (5) includes the time index t, which is redundant because the trajectory step already denotes the time; consider removing it for clarity.
- [Section IV-A] The statement that hyperparameters were 'individually tuned for optimal performance' is not supported by a description of the tuning process; specify the search space and the criterion used to select each configuration so that the comparison is fair and reproducible.
- [Figs. 2 and 3] The figures would benefit from a clearer legend; the captions refer to 'dark-colored curves' and 'shaded areas,' but the legends are not described in the text, and the meaning of the shading (standard error) should be stated in the caption or legend itself.
- [Section IV-A] The paper does not identify the power-flow solver (e.g., MATPOWER, PYPOWER) or the data source for the 118-bus case, which are essential for reproducibility; please add these details.
Circularity Check
No circularity: the DDSAC versus PPO/DDQN comparison is an external empirical benchmark, and no prediction in the paper reduces to its inputs by construction.
full rationale
The paper's central claim is that the proposed DDSAC algorithm achieves faster convergence and higher, more stable rewards than PPO and DDQN on the IEEE 118-bus system. This claim is supported by numerical experiments, not by a derivation whose conclusion is built into its premises. The objective in Eq. (1) is defined independently, and the reward in Eq. (7) is simply the negative of that objective; the same reward is used for all three algorithms, so the comparison is fair rather than circular. The weight factors in Table I are fixed hyperparameters chosen before training, not fitted retroactively to produce favorable curves. The dueling decomposition in Eq. (9) and the soft actor-critic framework are adopted from the external references [13] and [14], so no uniqueness theorem or ansatz is imported from the authors' own prior work in a load-bearing way. The only self-citations are background references [6] and [7], which are not used to justify the algorithm or the numerical results. There is a serious reproducibility concern: Eq. (6) defines the action as a joint line-switching vector, whose cardinality is 2^NL and is intractable on the 118-bus system, while Eqs. (9)-(12) appear to enumerate over the full action set. However, that is a correctness or specification issue, not circularity, because the reported learning curves do not reduce to the inputs by construction. Accordingly, the circularity score is 0.
Assumptions & free parameters
free parameters (6)
- w1 =
0.1
- w2 =
100
- w3 =
1
- w4 =
0.1
- w5 =
10/NL
- Re =
not specified
assumptions (4)
- domain assumption AC power flow equations (1f)-(1g) accurately model the transmission network and always have a solution for the states considered.
- ad hoc to paper The weighted sum in Eqs. (1a)-(1e) is a valid scalarization of the conflicting objectives (cost, voltage, losses, overload, switching count).
- ad hoc to paper The dueling Q-network formula in Eq. (9) can be computed over the action space A; for a binary vector of length NL, this requires enumerating 2^NL actions.
- domain assumption The state vector in Eq. (5) is sufficient for a Markov decision process; the environment is stationary and deterministic given the action.
Cite this review
Pith. "Pith review of A Deep Reinforcement Learning Method for Multi-objective Transmission Switching." pith.science (2026). https://pith.science/paper/BHI73FPE
@misc{pith2026250711726,
author = {Pith},
title = {Pith review of: A Deep Reinforcement Learning Method for Multi-objective Transmission Switching},
year = {2026},
howpublished = {\url{https://pith.science/paper/BHI73FPE}},
note = {Machine review of arXiv:2507.11726}
}
read the original abstract
Transmission switching is a well-established approach primarily applied to minimize operational costs through strategic network reconfiguration. However, exclusive focus on cost reduction can compromise system reliability. While multi-objective transmission switching can balance cost savings with reliability improvements, feasible solutions become exceedingly difficult to obtain as system scale grows, due to the inherent nonlinearity and high computational demands involved. This paper proposes a deep reinforcement learning (DRL) method for multi-objective transmission switching. The method incorporates a dueling-based actor-critic framework to evaluate the relative impact of each line switching decision within the action space, which improves decision quality and enhances both system reliability and cost efficiency. Numerical studies on the IEEE 118-bus system verify the effectiveness and efficiency of the proposed approach compared to two benchmark DRL algorithms.
Figures
Reference graph
Works this paper leans on
-
[1]
Corrective switching algorithm for relieving overloads and voltage violations,
W. Shao and V . Vittal, “Corrective switching algorithm for relieving overloads and voltage violations,” IEEE Transactions on Power Systems, vol. 20, no. 4, pp. 1877–1885, 2005
work page 2005
-
[2]
Optimal transmission switching considering voltage security and n-1 contingency analysis,
M. Khanabadi, H. Ghasemi, and M. Doostizadeh, “Optimal transmission switching considering voltage security and n-1 contingency analysis,” IEEE Transactions on Power Systems, vol. 28, no. 1, pp. 542–550, 2012
work page 2012
-
[3]
Optimal network reconfiguration for congestion management by deterministic and genetic algorithms,
G. Granelli, M. Montagna, F. Zanellini, P. Bresesti, R. Vailati, and M. In- norta, “Optimal network reconfiguration for congestion management by deterministic and genetic algorithms,” Electric power systems research , vol. 76, no. 6-7, pp. 549–556, 2006
work page 2006
-
[4]
Optimal transmission switching,
E. B. Fisher, R. P. O’Neill, and M. C. Ferris, “Optimal transmission switching,” IEEE transactions on power systems , vol. 23, no. 3, pp. 1346–1355, 2008
work page 2008
-
[5]
Loss reduction by network switching,
R. Bacher and H. Glavitsch, “Loss reduction by network switching,” IEEE Transactions on Power Systems, vol. 3, no. 2, pp. 447–454, 1988
work page 1988
-
[6]
Transmission switching with connectivity-ensuring constraints,
J. Ostrowski, J. Wang, and C. Liu, “Transmission switching with connectivity-ensuring constraints,” IEEE transactions on power systems, vol. 29, no. 6, pp. 2621–2627, 2014
work page 2014
-
[7]
Optimal transmission switching considering probabilistic reliability,
C. Zhang and J. Wang, “Optimal transmission switching considering probabilistic reliability,” IEEE Transactions on Power Systems , vol. 29, no. 2, pp. 974–975, 2013
work page 2013
-
[8]
Congestion management using optimal transmission switching,
S. R. Salkuti, “Congestion management using optimal transmission switching,” IEEE Systems Journal, vol. 12, no. 4, pp. 3555–3564, 2018
work page 2018
Show all 14 references
-
[9]
Safe deep reinforcement learning-based constrained optimal control scheme for active distribution networks,
P. Kou, D. Liang, C. Wang, Z. Wu, and L. Gao, “Safe deep reinforcement learning-based constrained optimal control scheme for active distribution networks,” Applied energy, vol. 264, p. 114772, 2020
2020
-
[10]
Model-free real-time ev charging scheduling based on deep reinforcement learning,
Z. Wan, H. Li, H. He, and D. Prokhorov, “Model-free real-time ev charging scheduling based on deep reinforcement learning,” IEEE Transactions on Smart Grid , vol. 10, no. 5, pp. 5246–5257, 2019
2019
-
[11]
Incentive-based demand response for smart grid with reinforcement learning and deep neural network,
R. Lu and S. H. Hong, “Incentive-based demand response for smart grid with reinforcement learning and deep neural network,” Applied energy, vol. 236, pp. 937–949, 2019
2019
-
[12]
Data-driven load frequency control for stochastic power systems: A deep reinforcement learning method with continuous action search,
Z. Yan and Y . Xu, “Data-driven load frequency control for stochastic power systems: A deep reinforcement learning method with continuous action search,” IEEE Transactions on Power Systems, vol. 34, no. 2, pp. 1653–1656, 2019
2019
-
[13]
Soft actor-critic algorithms and applications,
T. Haarnoja, A. Zhou, K. Hartikainen, G. Tucker, S. Ha, J. Tan, V . Ku- mar, H. Zhu, A. Gupta, P. Abbeel et al. , “Soft actor-critic algorithms and applications,” arXiv preprint arXiv:1812.05905 , 2018
2018 arXiv
-
[14]
Dueling network architectures for deep reinforcement learning,
Z. Wang, T. Schaul, M. Hessel, H. Hasselt, M. Lanctot, and N. Freitas, “Dueling network architectures for deep reinforcement learning,” in International conference on machine learning. PMLR, 2016, pp. 1995– 2003
2016
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.