{"id":"7889c1f0-1be3-434e-b3e6-10e7a1dde819","arxiv_id":"2507.11726","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"The paper introduces a dueling soft actor-critic method for multi-objective transmission switching and reports better rewards than PPO and DDQN on the IEEE 118-bus system.","lead":"This paper applies a deep reinforcement learning algorithm to decide when to switch transmission lines on or off in an electric grid, balancing cost against reliability. A generalist might read it to see how machine learning can be used for hard grid control problems.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Action-space definition makes DDSAC unimplementable as written on the 118-bus system, so the reported convergence and reward superiority are not reproducible from the paper.","rationale":"The reader's weakest assumption identifies the same issue I would flag. The action-space formulation in Eq. (6) with 2^186 discrete joint actions makes Eqs. (9) and (12) impossible to evaluate as written, and the paper provides no factorized alternative. This is load-bearing because the central claim is an empirical superiority result: if the algorithm cannot be instantiated on the IEEE 118-bus system from the description, the reported learning curves are not reproducible and the claim is unsupported. Other weaknesses (missing no-switching baseline, hand-picked weights, no statistical tests, no code) are real but secondary; they affect the strength of the comparison, not the basic executability. I do not accuse the authors of fabrication; the issue is an under-specification that must be resolved before the experiments can be credited. The reader's REJECT verdict should remain unchanged.","tokens_in":6727,"tokens_out":4600,"duration_ms":59626,"concrete_test":"Request from the authors the exact policy-network output specification and the computation used for the average over A in Eq. (9) for the 118-bus case. Concretely: report the output dimension of πθ, and state whether the agent samples one vector from a categorical distribution over 2^186 joint actions or samples per-line Bernoulli actions. Then perform one step of the policy update as printed for a network of that dimension; if the output is a categorical distribution over 2^186 actions, the sum in Eq. (12) cannot be computed, confirming the intractability. If the output is factorized per line, re-derive Eqs. (9) and (12) under that factorization and verify whether the dueling average runs over the per-line binary actions; if it does, the printed equations misstate the algorithm. This single check settles whether the method as described can generate the reported curves.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—faster convergence and higher, more stable rewards than PPO and DDQN—rests on the DDSAC algorithm being a well-defined, executable procedure. That condition fails at the action-space specification. Equation (6) defines the action as the full vector (x1,...,x_NL), so the discrete action space has 2^NL elements. On the IEEE 118-bus system, NL is approximately 186, giving roughly 2^186 joint actions. Equation (9) requires averaging the advantage A(s,a) over every action in A; Eq. (12) evaluates πθ(s,a)^T [α log πθ(a|s) − Q^(s,a)], a sum over all actions; and Algorithm 1 samples an action from πθ(·|s). None of these operations is feasible at that scale. The paper never states a factorized action parameterization—for example, independent Bernoulli distributions per line—nor a restricted candidate action set. If such a factorization was used in the experiments, then Eqs. (9)–(12) are not the equations actually executed: the dueling average would have to be over per-line binary actions, not over the joint space, and the policy loss would decompose per line. As printed, the method is therefore either computationally impossible or internally mis-specified. Because the reported learning curves cannot be produced from the text, the central empirical claim is unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes DDSAC, a deep reinforcement learning method for multi-objective transmission switching. The method combines a dueling Q-network with a discrete version of soft actor-critic (SAC) to select line switching actions that minimize a weighted sum of generation cost, voltage violations, line overload, power losses, and switching count, subject to AC power flow constraints. The algorithm is evaluated on the IEEE 118-bus system against two benchmark DRL algorithms (PPO and DDQN). Experimental learning curves are presented showing cumulative rewards and individual objective components over 500 time steps. The paper claims that DDSAC converges faster and achieves higher, more stable rewards than the benchmarks.","tokens_in":7036,"tokens_out":7859,"duration_ms":86170,"significance":"If validated, the proposed DDSAC would be a plausible improvement in DRL for transmission switching by incorporating a dueling advantage stream into discrete SAC. The multi-objective reward formulation is practical, and the benchmark comparison is a standard evaluation approach. However, the manuscript as written contains a load-bearing computational inconsistency in the action-space definition, and the empirical support consists of qualitative curves without numerical or statistical details. The paper does not provide code, environmental details, or final performance tables, which severely limits reproducibility and prevents verification of the central claim.","major_comments":[{"comment":"The action is defined as the full vector of NL line switching decisions in Eq. (6). For the IEEE 118-bus system, with approximately 186 lines, the action space has size 2^186. Eq. (9) requires averaging the advantage over all actions in A, and Eq. (12) computes a dot product over the probability vector of all actions; Algorithm 1 also samples from this intractable distribution. The paper never specifies a factorized action representation or a restricted alternative set. Consequently, DDSAC as described is computationally infeasible for this system, and the reported learning curves cannot be reproduced from the text.","section":"Section III-B, Eq. (6) and Eq. (9)"},{"comment":"The central claim of faster convergence and higher, more stable rewards is based only on qualitative visual inspection of the plotted curves. No numerical results, standard deviations, or statistical significance tests are reported. The description of the 10 runs and standard error shading is ambiguous (e.g., whether the curves are training returns or evaluation returns). Without these quantitative details, the claimed advantage of DDSAC over PPO and DDQN is not demonstrated.","section":"Section IV, Figs. 2 and 3"},{"comment":"The objective function uses a generator cost term CGi(PGi) without specifying its functional form, preventing reproduction of the cost values in Fig. 3. Moreover, Section IV-A states that the cost term is calculated as the difference between post-switching and initial total costs, while Eq. (1a) defines a plain sum. This inconsistency affects the interpretation of the reward and the weight w1, and it needs to be clarified.","section":"Section II, Eq. (1a) and Section IV-A"},{"comment":"The policy loss in Eq. (12) is written as a dot product πθ(s,a)⊤[α log πθ(a|s) − Q̂(s,a)], which requires the probability vector over the entire action space. This is intractable for the stated action space and also not the standard discrete SAC objective, which uses an expectation over the policy. The lack of a well-defined policy sampling procedure in Algorithm 1 (line 6) compounds the problem. The method section needs to be rewritten to specify a tractable factorized policy.","section":"Section III-B, Eq. (12) and Algorithm 1"}],"minor_comments":[{"comment":"The lower and upper voltage bounds in Eq. (1b) are typeset identically in the displayed text; please ensure they are visually distinct (e.g., using \\underline{V} and \\overline{V}).","section":"Eq. (1b)"},{"comment":"The state vector in Eq. (5) includes the time index t, which is redundant because the trajectory step already denotes the time; consider removing it for clarity.","section":"Eq. (5)"},{"comment":"The statement that hyperparameters were 'individually tuned for optimal performance' is not supported by a description of the tuning process; specify the search space and the criterion used to select each configuration so that the comparison is fair and reproducible.","section":"Section IV-A"},{"comment":"The figures would benefit from a clearer legend; the captions refer to 'dark-colored curves' and 'shaded areas,' but the legends are not described in the text, and the meaning of the shading (standard error) should be stated in the caption or legend itself.","section":"Figs. 2 and 3"},{"comment":"The paper does not identify the power-flow solver (e.g., MATPOWER, PYPOWER) or the data source for the 118-bus case, which are essential for reproducibility; please add these details.","section":"Section IV-A"}],"recommendation":"major_revision","confidential_remarks":"The manuscript addresses a relevant topic, but the current level of algorithmic and experimental detail is far below what is needed to support the claims. I would encourage the editor to ask the authors for the code and data, and to require a corrected action-space formulation and a rigorous quantitative evaluation, before considering publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the paper stacks a dueling architecture onto discrete SAC for multi-objective transmission switching and reports that it beats PPO and DDQN on the IEEE 118-bus system. The empirical claim is not reproducible from the text because the action space is never made tractable.\n\nWhat is genuinely new: the particular combination of dueling value/advantage streams with discrete SAC applied to transmission switching, and the head-to-head comparison against two DRL baselines on a standard benchmark. The reward design — weighted cost, voltage deviation, overload, losses, and switching penalty — is standard but reasonable. The writing is clear and the related work is appropriate.\n\nThe soft spot is load-bearing. Eq. (6) defines the action as the vector (x_1,...,x_NL), i.e., the joint status of all lines. Eq. (9) then averages the advantage over every action in A; Eq. (12) sums over all actions; Algorithm 1 samples from πθ(·|s). On the 118-bus system with roughly 186 lines, the joint space has 2^186 elements: none of these operations can be carried out. The paper never specifies a factorized policy — e.g., independent Bernoulli per line — or a restricted candidate set. If one was used, the equations are not what was implemented. Either way, the method as written cannot produce the reported learning curves.\n\nEverything else is secondary. There is no code, no aggregated numeric results, no statistical testing, no comparison to a no-switching or conventional optimization baseline, and no ablation for the dueling component. Those would matter even if the action-space issue were fixed. The citations look fine.\n\nI would not send this to peer review in its current form. The authors should either rewrite the action-space formulation so that Eqs. (9)-(14) match a feasible per-line decomposition, or release code that shows what was actually run. With that fixed, the comparison could be a useful data point, but right now the central claim is unsupported.","headline":"The paper is a plausible DRL-for-transmission-switching study, but the action-space definition makes the algorithm unimplementable as written on the 118-bus system, so the central empirical claim doesn't stand.","tokens_in":7518,"tokens_out":3646,"would_cite":false,"duration_ms":37612,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a discrete dueling soft actor-critic learns multi-objective transmission switching faster and more stably than PPO and DDQN on the IEEE 118-bus system.","keywords":["deep reinforcement learning","transmission switching","multi-objective optimization","soft actor-critic","dueling network","IEEE 118-bus system","power system reliability","topology control"],"falsifier":"Run DDSAC on a small network where all switch configurations can be enumerated, and check whether the dueling update in Eq. (9) reproduces a brute-force advantage computation; then re-run the 118-bus experiment with the action factorisation written explicitly and compare the convergence curves to the reported ones.","tokens_in":6484,"feed_emoji":"⚡","tokens_out":7490,"duration_ms":90754,"temperature":0.7,"pith_summary":"This paper tries to establish that a discrete dueling soft actor-critic (DDSAC) agent can learn transmission-switching policies that improve cost and reliability at the same time. The motivation is that multi-objective transmission switching, balancing generator cost against voltage, losses, congestion, and load shedding, becomes very hard to solve as the network grows. The claim is tested on the IEEE 118-bus system, where DDSAC is reported to converge within about fifty steps and to hold higher and more stable cumulative rewards than PPO and DDQN over five hundred steps. If the claim holds, learning-based topology control becomes a viable route to multi-objective switching at scales where exact optimization struggles.","feed_headline":"Dueling SAC learns 118-bus switching faster than PPO, DDQN","feed_subtitle":"A discrete soft actor-critic with dueling Q-networks balances cost, voltage, losses, congestion, and load shedding.","key_machinery":"The identity that carries the argument is the dueling Q decomposition $Q(s,a)=V(s)+\\left(A(s,a)-\\frac{1}{|A|}\\sum_{\\tilde a\\in A}A(s,\\tilde a)\\right)$, embedded in a soft actor-critic discretized for transmission switching. The advantage stream is meant to score each line-switching action relative to the average of all actions, while the value stream carries the state's baseline; the SAC entropy term keeps the policy exploring. This decomposition is the mechanism the paper credits for allowing the agent to differentiate among many switching alternatives and to prioritise critical decisions.","core_discovery":"The central claim is that the dueling Q-network identity, which separates the value of a state from the relative advantage of each action, makes a discrete SAC algorithm substantially better at ranking line-switching decisions. In DDSAC, the policy network outputs probabilities over line statuses, the twin Q-networks reduce overestimation bias, and the temperature parameter is tuned automatically to keep exploration at a target entropy. On the IEEE 118-bus system, DDSAC is reported to converge faster, produce higher and more stable cumulative reward, and keep generator cost, voltage violation, active power loss, and line overload lower than both PPO and DDQN across 500 steps.","pith_inferences":["The paper leaves the action-space representation implicit; if each line is an independent binary action, the dueling average in Eq. (9) ranges over only two choices, whereas if the joint vector is the action, the average over $2^{186}$ configurations is not computable as written.","A natural extension is to map the five weight factors onto a Pareto front of cost-reliability trade-offs, since real operators would want to know which switch configurations are optimal for their preferred weighting.","The same dueling-advantage trick could be tested on other discrete topology decisions, such as capacitor or transformer tap switching, where action sets are combinatorial but structured.","Benchmarking DDSAC against an exact mixed-integer solver on small networks would show how much of the reported gain comes from the dueling mechanism rather than from the SAC exploration strategy."],"forward_implications":["If the claim holds, dueling discrete SAC is a practical template for learning multi-objective transmission-switching policies, not just for single-objective cost reduction.","The same DDSAC training loop could be re-run with different weight vectors $w_1$ through $w_5$, giving operators a way to shift the cost-reliability trade-off without redesigning the algorithm.","Faster convergence within roughly fifty steps suggests the method can be trained with fewer environment interactions, which matters when each step requires solving an AC power flow.","Stable rewards over five hundred steps imply the learned policy can be deployed in settings where the operating point changes and the agent must keep re-evaluating switch configurations."],"supporting_citations":[{"why":"Establishes the economic optimal transmission switching formulation whose cost objective is reused in Eq. (1a).","marker":"[4]"},{"why":"Provides connectivity-ensuring constraints that motivate the islanding penalty in the reward function.","marker":"[6]"},{"why":"Formulates multi-objective transmission switching with probabilistic reliability, the problem this paper recasts as an MDP.","marker":"[7]"},{"why":"Supplies the maximum-entropy soft actor-critic algorithm that DDSAC adapts from continuous to discrete actions.","marker":"[13]"},{"why":"Supplies the dueling architecture and advantage-averaging identity used in Eq. (9).","marker":"[14]"}],"fun_headline_variants":["Dueling Q-networks sharpen SAC for 118-bus switching","Discrete SAC with dueling critics wins 118-bus switching race","Auto-tuned SAC balances cost and reliability via dueling Q","DDSAC: Dueling discrete actor-critic for multi-objective switching"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes the average over actions in the dueling update can actually be evaluated for the line-switching action space, although no factorized action representation is given and the full joint space has $2^{186}$ elements for the 118-bus system.","fun_headline_variants_meta":{"raw":{"variants":["Dueling Q-networks sharpen SAC for 118-bus switching","Discrete SAC with dueling critics wins 118-bus switching race","Auto-tuned SAC balances cost and reliability via dueling Q","DDSAC: Dueling discrete actor-critic for multi-objective switching"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000696,"raw_usage":{"total_tokens":3079,"prompt_tokens":810,"completion_tokens":2269,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":426,"completion_tokens_details":{"reasoning_tokens":2190}},"tokens_in":426,"tokens_out":2269,"duration_ms":16172,"temperature":1.0,"reasoning_tokens":2190,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T17:03:38.123444+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run DDSAC on a small network where all switch configurations can be enumerated, and check whether the dueling update in Eq. (9) reproduces a brute-force advantage computation; then re-run the 118-bus experiment with the action factorisation written explicitly and compare the convergence curves to the reported ones.","supporting_citations":[{"cited_title":"Optimal transmission switching,","cited_arxiv_id":null,"evidence_quote":"Establishes the economic optimal transmission switching formulation whose cost objective is reused in Eq. (1a)."},{"cited_title":"Transmission switching with connectivity-ensuring constraints,","cited_arxiv_id":null,"evidence_quote":"Provides connectivity-ensuring constraints that motivate the islanding penalty in the reward function."},{"cited_title":"Optimal transmission switching considering probabilistic reliability,","cited_arxiv_id":null,"evidence_quote":"Formulates multi-objective transmission switching with probabilistic reliability, the problem this paper recasts as an MDP."}],"review_version":1}