{"id":"b965bbe0-aebc-40f5-b160-c2f8e43de605","arxiv_id":"2505.01147","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A probabilistic dynamic security assessment pipeline with coverage-aware Monte Carlo error bounds, stability screening, protection-uncertainty cascade simulations, and SVM-based root-cause analysis is demonstrated on the 73-bus RTS-GMLC system.","lead":"This paper presents a step-by-step method for probabilistic dynamic security assessment of power grids that samples many realistic operating conditions, simulates rare N-2 faults and uncertain protection behavior, and uses statistics plus machine learning to find the riskiest faults. A reader might care because European grid operators are required to move to probabilistic security analysis by 2027, and this is a concrete, quantified pipeline for doing so.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Risk estimates for protection-sensitive scenarios rest on only 5 protection-parameter draws per operating condition with no convergence or variance check, so the claimed per-contingency SE guarantee does not cover this source of error.","rationale":"The reader's weakest_assumption identifies exactly this gap: five protection-parameter draws per sensitive scenario with no variance or convergence analysis. This is load-bearing because the paper's headline contribution is a rigorous statistical guarantee on per-contingency risk error, and that guarantee is built from the scenario-level consequence estimates. If those scenario-level estimates themselves carry unquantified Monte Carlo error, the overall SE bound in Eq. (7) does not cover the true error. The paper's own results show the affected scenarios are not rare (834 scenarios), and in half of them the consequences changed across draws, so the issue is empirically relevant. The other limitations (screening misses 4% of risk, scalability extrapolation) are either explicitly acknowledged by the authors or based on stated assumptions; they do not undermine the core method as demonstrated. The 5-draw protection MC is a silent approximation in the core pipeline. The proposed test is practical and would settle whether the risk estimates are stable under protection-parameter sampling. If the test shows convergence, the conditional is resolved; if not, the authors need to either increase the number of draws with a proper convergence criterion or account for the protection-parameter MC error in the SE bound. The verdict remains CONDITIONAL because the concern is addressable and does not necessarily invalidate the empirical demonstration, but it does require additional evidence.","tokens_in":15542,"tokens_out":20153,"duration_ms":208916,"concrete_test":"For each of the 834 protection-sensitive scenarios in Section IV-C, recompute the mean consequence using 100 protection-parameter draws instead of 5 (or, if computational cost is prohibitive, use bootstrap resampling of the 5 draws to estimate the variance of the 5-draw mean). Then recalculate the per-contingency risks and total risk with these refined scenario means. If any contingency's risk estimate shifts by more than its reported SE_i (or by more than a small fraction of total risk), the 5-draw choice is not converged and the central accuracy guarantee is not met for that contingency. Additionally, compute the empirical standard deviation of the 5-draw means over repeated resamples; if it exceeds, say, 10% of the mean consequence for a substantial number of scenarios, the risk estimate is unstable under protection-parameter uncertainty.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is a statistical error guarantee for per-contingency risk. Sections II-D and IV-C use exactly 5 Monte Carlo draws of protection parameters for each scenario flagged as sensitive to protection timing. For these scenarios, the estimated consequence is the average over those 5 draws. No standard error, confidence interval, or convergence diagnostic for this 5-draw estimate is reported. If the distribution of consequences over protection parameters is heavy-tailed (e.g., rare protection misoperations leading to large load shed), the 5-draw mean can be a noisy and potentially biased estimate of the scenario's expected consequence. The per-contingency SE bound in Eq. (7) is computed from the sample variance of these per-scenario means across operating conditions, treating each scenario mean as an exact observation. It does not include the error of the 5-draw protection-parameter estimate. The stopping criterion (1) can therefore terminate while the actual mean risk for a contingency is outside the claimed tolerance. The paper itself reports in Section IV-C that 834 of 4329 unsecure scenarios were protection-sensitive, with 408 showing different consequences across draws, so this is a non-negligible subset. The risk estimate for contingencies that involve these scenarios is directly affected.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a complete probabilistic dynamic security assessment (PDSA) methodology for power systems, comprising: (i) generation of a database of plausible operating conditions via weather-driven Monte Carlo years, a market model, and SCOPF; (ii) per-contingency Monte Carlo sampling with a statistical stopping criterion; (iii) optional screening of secure scenarios using stability indicators; (iv) handling of protection-parameter uncertainty during fast cascades using an indicator from prior work and five Monte Carlo draws per sensitive scenario; and (v) interpretable machine-learning models (SVMs with sequential feature selection) to identify root causes and suggest security enhancements. The method is demonstrated on the 73-bus RTS-GMLC system, considering 114 delayed-clearing N-1 contingencies and 594 N-2 contingencies. The central claims are that the stopping criterion (Eqs. (1) and (7)) guarantees a user-selected statistical error on each contingency's risk contribution, that screening reduces computation time by about a factor of two with only 4% missed risk, and that the ten most critical contingencies (contributing over 40% of total risk) are correctly identified at roughly 400 core-hours of computation.","tokens_in":15756,"tokens_out":5884,"duration_ms":56950,"significance":"If the statistical guarantees hold, this is a valuable contribution to the probabilistic security assessment literature, which is currently of high practical interest given new European regulatory requirements. The paper's coverage-aware SE bound (Eq. (7)) addresses a genuine flaw in naive variance-based stopping rules and is conservative in spirit, which is a real strength. The explicit treatment of protection-parameter uncertainty during fast cascades and the use of interpretable ML for security enhancement are also worthwhile. The paper ships reproducible data and algorithms, and it is honest about the limitations of the screening process. The central statistical derivation is internally consistent and is not circular: the risk estimate and its error are both computed from the same simulations, which is standard Monte Carlo error estimation. However, two specific gaps, detailed in the major comments, prevent the paper's central claim from being fully substantiated as written.","major_comments":[{"comment":"The per-contingency standard-error guarantee in Eq. (7) does not account for the Monte Carlo error introduced by estimating scenario consequences from only five protection-parameter draws. For the 834 unsecure scenarios flagged as protection-sensitive, the consequence used in the risk estimate is the average over these five draws, and the paper states that 408 of them yield different consequences across draws. Yet no standard error, confidence interval, or convergence diagnostic is reported for these five-draw averages. Because Eq. (7) computes the sample variance σ~_i^2 over operating-condition scenario means and treats each mean as an exact observation, it understates the true statistical error of the risk estimate. Consequently, the stopping criterion (1) can terminate while the actual risk for a contingency remains outside the claimed tolerance. This is a load-bearing gap in the paper's central statistical claim; the authors should either provide a variance/convergence analysis for the five draws or explicitly incorporate this source of error into the SE bound.","section":"II-D, IV-C"},{"comment":"The screening process misses 4.0% of the total risk overall and 6.4% of the N-2 risk (Table II), and this systematic downward bias is not reflected in the per-contingency SE bound (Eqs. (1) and (7)). The central claim is that the stopping criterion guarantees SE_i ≤ εR for every contingency; a missed-risk fraction of the same order as the user-selected tolerance (ε = 1% in the study) means that individual contingency risk estimates can be biased by more than the claimed statistical error, especially for N-2 contingencies where the missed fraction is larger. The paper acknowledges this as 'limited impact on accuracy', but the guarantee should be qualified: either the screening-induced bias should be explicitly reserved within the SE budget, or the guaranteed accuracy should be stated conditionally on scenarios that pass the screening. This is necessary for the stated per-contingency guarantee to be technically correct.","section":"II-C, Table II"}],"minor_comments":[{"comment":"Table IV is internally inconsistent: the ten listed risks sum to 9.93 M€/y, and adding the 'Others' entry of 12.4 M€/y gives 22.33 M€/y, exceeding the total risk of 21.0 M€/y reported in Table I. The 'Others' value should be approximately 11.07 M€/y.","section":"IV-D, Table IV"},{"comment":"In Eq. (11), the quantities σ~, β, and N for the total risk are not explicitly defined; please add their definitions, mirroring the per-contingency definitions in Eqs. (5)-(7).","section":"II-B, Eq. (11)"},{"comment":"The phrases 'a 0.1 chance' and 'a 0.01 chance' should be expressed as probabilities (0.1 and 0.01) for precision.","section":"III"},{"comment":"The sentence 'The second set is however not connected to circuit breakers to not affect the system evolution' is grammatically awkward; consider rewording for clarity.","section":"II-D"},{"comment":"The discussion of inverter-based generators modelled as synchronous machines with an inertia 1/K_i would benefit from a brief explanation of the units and meaning of K_i, as it appears without definition.","section":"II-C"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a solid engineering contribution and the proposed stopping-rule framework is valuable. The two major issues above are fixable within the paper's scope: the first requires additional diagnostics (variance/convergence of the five-draw protection-parameter estimates) on existing simulation data, and the second requires a more careful statement of the accuracy guarantee in the presence of screening. The paper is within scope for eess.SY and the reproducible-data statement is a positive feature. I do not see grounds for rejection, but the central claim as currently worded is stronger than what the evidence supports."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nThe headline is that this is a genuinely useful methods paper. The new thing is the coverage-aware standard error bound in Eq. (7), which adds a term that penalizes small sample sizes when no unsecure regions have been observed. Using that bound as a per-contingency stopping rule is a real improvement over the usual practice of stopping on the sample variance alone, which can be zero for rare blackout scenarios. The demonstration on RTS-GMLC is careful: the authors report computation times, screening speed-ups, the 4% missed risk, and the ten contingencies carrying over 40% of risk. The SVM root-cause analysis is simple but interpretable, and the wind-farm example checks out. The paper is honest about its limitations (unwanted trips, better indicators needed). Data and code are available.\n\nThe soft spots are real but not fatal. The five Monte Carlo draws of protection parameters for protection-sensitive scenarios are the biggest one. The paper reports that 408 of 834 such scenarios had different consequences across draws, but gives no variance or convergence analysis for the five-draw mean. Since Eq. (7) treats each scenario's consequence as exact, the claimed per-contingency error bound does not actually cover the protection-parameter source of uncertainty. For contingencies that include many protection-sensitive scenarios, the total risk estimate could be biased. The authors justify five draws as a practical compromise, but they should at least report a standard error or show that the results are stable with more draws.\n\nThe screening step's 4% missed risk is also sensitive to the hand-set CCT margin, as Table III shows, so the reported accuracy is specific to that choice. The large-grid scalability figures are extrapolated from assumed simulation counts, not measured, so they should be read as rough estimates.\n\nNone of this undercuts the core statistical result. The derivation in Section II-B is internally consistent, and the coverage term is a useful conceptual contribution. The citation pattern is appropriate; the self-citation [23] is used for the protection-sensitivity indicator, not for the main claim.\n\nThis paper deserves a serious referee. I would engage with it, and I'd advise the authors to add a variance analysis for the protection-parameter draws and a sensitivity check on the CCT margin. If those are addressed, the methodology would be quite solid.","headline":"A honest, well-engineered PDSA pipeline with a genuinely useful coverage-aware stopping rule; the main caveat is that the protection-parameter uncertainty is handled with only five draws per scenario, so the advertised per-contingency error bound doesn't fully cover that source.","tokens_in":16314,"tokens_out":2601,"would_cite":true,"duration_ms":24650,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper proposes a per-contingency stopping criterion with a coverage-aware standard-error bound that guarantees each contingency's risk estimate is accurate to a user-selected fraction of total risk, and demonstrates on a 73-bus test…","keywords":["probabilistic dynamic security assessment","cascading outages","protection uncertainty","Monte Carlo stopping criterion","coverage-aware standard error","N-2 contingencies","interpretable machine learning","grid risk ranking"],"falsifier":"Re-run the assessment on the 73-bus test system for the ten critical contingencies using, say, fifty Monte Carlo protection-parameter draws per protection-sensitive scenario instead of five, and compare the per-contingency risk estimates. If any estimate changes by more than $\\epsilon R$, the claimed per-contingency accuracy guarantee does not hold as stated; alternatively, exhaustively enumerate protection parameter values for a single sensitive scenario and compare the true mean consequence with the five-draw estimate.","tokens_in":15314,"feed_emoji":"⚡","tokens_out":6754,"duration_ms":68746,"temperature":0.7,"pith_summary":"This paper argues that probabilistic dynamic security assessment can be made both statistically trustworthy and computationally manageable by setting a per-contingency accuracy target rather than a single target on total risk. It derives a coverage-aware bound on the standard error of each contingency's risk estimate, so sampling stops only when every contingency's estimated risk is known to within a user-chosen fraction of total risk. On a 73-bus reliability test system, the method identifies ten contingencies that together account for more than 40% of total risk at roughly 400 core-hours. It also screens secure scenarios, handles protection-timing uncertainty with targeted Monte Carlo draws, and uses interpretable machine learning to trace critical contingencies to root causes such as high wind output on a specific line.","feed_headline":"Stopping rule finds the 10 outages driving 40% of grid risk","feed_subtitle":"Per-contingency accuracy bound tells planners when Monte Carlo sampling can stop, at about 400 core-hours.","key_machinery":"The machinery is the coverage-aware standard-error bound of Eq. (7). The first term under the square root is the usual sample-variance term; the second term, $3\\beta_i^2/N_i^2$ with $\\beta_i = M_C - \\tilde{\\mu}_i$, is an upper bound on the bias caused by potentially missing unsecure operating regions during sampling. This coverage term is what lets the stopping criterion $SE_i \\le \\epsilon R$ claim statistical accuracy for each contingency even when most sampled operating conditions show zero consequences. Secondary machinery includes the screening chain (Extended Equal Area critical clearing times, a short-circuit-power voltage indicator, and frequency criteria) and the protection-sensitivity indicator from prior work, which decides whether a scenario needs extra Monte Carlo draws of protection parameters.","core_discovery":"The central claim is that Eq. (7), $SE_i \\le f_i \\sqrt{\\tilde{\\sigma}_i^2 / N_i + 3 \\beta_i^2 / N_i^2}$, is a valid upper bound on the standard error of the estimated risk of each contingency $i$ when operating conditions are sampled, where the second term under the square root bounds the bias from unobserved unsecure operating regions. Combined with the stopping criterion $SE_i \\le \\epsilon R$ of Eq. (1), this guarantees, with 95% confidence, that the estimated risk of every considered contingency differs from the true risk by less than a user-selected fraction of the total risk $R$. In the demonstration on the 73-bus reliability test system, this correctly ranks the most critical contingencies and identifies ten contingencies responsible for over 40% of total risk. The method also accounts for cascade uncertainty by using a prior indicator to classify scenarios as protection-timing-sensitive and running additional Monte Carlo draws only for those scenarios.","pith_inferences":["Inference: the same coverage-aware stopping bound could be applied to any Monte Carlo risk assessment decomposed into categories, such as earthquake, wildfire, or cyber events, where missing one rare category biases the total; the criterion tells the analyst when enough samples have been drawn per category to trust the ranking.","Inference: the five-draw protection-parameter estimator is the part most likely to fail as protection systems and fast cascades become more complex; a direct variance check over draws, with adaptive increases when variance is high, would be a cheap safeguard and is testable on the existing test system.","Inference: the SVM root-cause rules implicitly define preventive redispatch constraints; one could test them by re-dispatching the 73-bus system to keep the identified features inside the secure region and re-running the probabilistic assessment to measure the resulting risk reduction."],"forward_implications":["Planners can rank contingencies by risk with a quantified error bar and focus enhancement measures, such as new lines, series capacitors, system integrity protection schemes, or curtailment, on the few contingencies driving most risk.","For most contingencies the coverage term dominates the standard error, so crude Monte Carlo sampling is the most efficient way to meet the per-contingency target; variance-reduction techniques only pay off after the condition in Eq. (8) holds.","Screening by stability indicators roughly halves computation time in the demonstration while missing only about 4% of total risk, and better indicators would push toward the theoretical factor-20 speed-up.","On a large grid with about 12,000 contingencies, the same approach is estimated to need around 60,000 core-hours, roughly 18,000 euros of rented high-performance computing time, with subsequent runs much cheaper.","Interpretable SVM boundary rules, such as a specific line being critical when wind production is high and total load is low, give planners a direct handle on which operating conditions to avoid or mitigate."],"supporting_citations":[{"why":"Supplies the protection-sensitivity indicator that separates scenarios needing extra Monte Carlo protection-parameter draws from those that can be simulated once.","marker":"[23]"},{"why":"Defines the 73-bus reliability test system, including network topology and renewable and load time series, used for the demonstration.","marker":"[25]"},{"why":"Provides the large-grid probabilistic dynamic security assessment benchmark used to frame scalability and computational-requirement comparisons.","marker":"[11]"},{"why":"Supplies the Monte Carlo year generation method for building the operating-conditions database from weather data and market dispatch.","marker":"[15]"},{"why":"Provides the Extended Equal Area method used in screening to estimate critical clearing times and identify angle-unstable scenarios without full time-domain simulation.","marker":"[19]"},{"why":"Supplies the short-circuit-power voltage stability indicator used in the screening chain.","marker":"[22]"},{"why":"Provides the restoration model and value of lost load used to convert megawatts of load shed into the monetary consequences $M_C$ and hence into risk.","marker":"[33]"},{"why":"Supplies the locational marginal pricing market model used to dispatch thermal generation in the operating-condition database.","marker":"[26]"}],"fun_headline_variants":["Stopping rule pinpoints 10 critical grid outages causing 40% of risk","Sampling stop rule focuses Monte Carlo on the risk-driving contingencies","How to know when Monte Carlo has sampled enough to rank grid risks","Bias-aware stopping criterion targets accuracy for each outage risk","Ten outages carry 40% of grid risk, says rigorous sampling stop rule"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes the pre-screening indicator from earlier work correctly identifies every scenario where protection-system timing can change the outcome, and that five random draws of protection parameters are enough to estimate the average consequences of each such scenario; if that assumption fails, the reported per-contingency accuracy could be an underestimate.","fun_headline_variants_meta":{"raw":{"variants":["Stopping rule pinpoints 10 critical grid outages causing 40% of risk","Sampling stop rule focuses Monte Carlo on the risk-driving contingencies","How to know when Monte Carlo has sampled enough to rank grid risks","Bias-aware stopping criterion targets accuracy for each outage risk","Ten outages carry 40% of grid risk, says rigorous sampling stop rule"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000172,"raw_usage":{"total_tokens":1269,"prompt_tokens":936,"completion_tokens":333,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":552,"completion_tokens_details":{"reasoning_tokens":239}},"tokens_in":552,"tokens_out":333,"duration_ms":3747,"temperature":1.0,"reasoning_tokens":239,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T04:25:20.906283+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the assessment on the 73-bus test system for the ten critical contingencies using, say, fifty Monte Carlo protection-parameter draws per protection-sensitive scenario instead of five, and compare the per-contingency risk estimates. If any estimate changes by more than $\\epsilon R$, the claimed per-contingency accuracy guarantee does not hold as stated; alternatively, exhaustively enumerate protection parameter values for a single sensitive scenario and compare the true mean consequence with the five-draw estimate.","supporting_citations":[{"cited_title":"Handling protection-related uncertainties in simulations of fast cascading outages,","cited_arxiv_id":null,"evidence_quote":"Supplies the protection-sensitivity indicator that separates scenarios needing extra Monte Carlo protection-parameter draws from those that can be simulated once."},{"cited_title":"The IEEE reliability test system: A proposed 2019 update,","cited_arxiv_id":null,"evidence_quote":"Defines the 73-bus reliability test system, including network topology and renewable and load time series, used for the demonstration."},{"cited_title":"Implementation of a massively parallel dynamic security assessment platform for large-scale grids,","cited_arxiv_id":null,"evidence_quote":"Provides the large-grid probabilistic dynamic security assessment benchmark used to frame scalability and computational-requirement comparisons."},{"cited_title":"Upgrading of the decision-making process for system development,","cited_arxiv_id":null,"evidence_quote":"Supplies the Monte Carlo year generation method for building the operating-conditions database from weather data and market dispatch."},{"cited_title":"Extended equal area criterion revisited: A direct method for fast transient stability analysis,","cited_arxiv_id":null,"evidence_quote":"Provides the Extended Equal Area method used in screening to estimate critical clearing times and identify angle-unstable scenarios without full time-domain simulation."},{"cited_title":"Simplified angle and voltage stability criteria for power system planning based on the short-circuit power,","cited_arxiv_id":null,"evidence_quote":"Supplies the short-circuit-power voltage stability indicator used in the screening chain."},{"cited_title":"Probabilistic security analysis of optimal transmission switching,","cited_arxiv_id":null,"evidence_quote":"Provides the restoration model and value of lost load used to convert megawatts of load shed into the monetary consequences $M_C$ and hence into risk."},{"cited_title":"A novel matching formula- tion for startup costs in unit commitment,","cited_arxiv_id":null,"evidence_quote":"Supplies the locational marginal pricing market model used to dispatch thermal generation in the operating-condition database."}],"review_version":1}