REVIEW 3 major objections 6 minor 1 cited by
A Dataset Generation Toolbox for Dynamic Security Assessment: On the Role of the Security Boundary
T0 review · 3 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read This paper argues that the quality of a data-driven dynamic security assessment tool is set by how well its training data covers the security boundary, and it presents a toolbox that concentrates samples there while keeping secure and…
desk verdict A useful public toolbox and a suggestive but not yet airtight case that boundary-focused sampling improves DSA classifiers. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism is the directed walk in generator active-power space, which uses the finite-difference gradient of the damping ratio of the least-damped mode as a direction of travel and a step size that shrinks as the operating point approaches the boundary (Eqs. 12 and 13). The walks start from feasible dispatches produced by a space-reduction pipeline: optimization-based bound tightening tightens the input bounds, separating hyperplanes cut away provably infeasible volumes, and Hit-and-Run sampling proposes candidates inside the remaining convex polytope. When a walk enters the high-information-content region, the algorithm sweeps a surrounding 1 MW grid to collect neighboring operating points, turning a few feasible dispatches into a dense, balanced boundary dataset.
What would settle it
On a two-generator slice of the 39-bus system, exhaustively grid the active-power plane at 1 MW resolution, compute the true high-information-content region from the damping ratio, and check whether the directed walks launched from all initialization points visit every connected component; if any boundary component is never entered, the method's claim to comprehensive boundary capture is falsified.
Extended reading notes
Core claim
The paper's central claim is that operating points close to the security boundary carry most of the discriminating information a classifier needs, and that a dataset deliberately enriched in such points, while balanced between secure and insecure labels, makes a data-driven dynamic security assessment tool accurate near the boundary without sacrificing accuracy far from it. The authors build such datasets by first shrinking the feasible operating region with optimization-based bound tightening and separating hyperplane infeasibility certificates, then walking feasible dispatches toward the boundary using a finite-difference gradient of the least-damped mode's damping ratio, and finally collecting points inside a margin of $\pm 0.25\%$ damping around the boundary, defined here as AC feasibility plus a minimum damping of $\zeta_{\min}=3\%$. In their case studies, decision trees trained on these datasets show F1-scores that generalize across test sets from other sampling methods, while trees trained on uniform or importance-sampled data degrade on boundary test sets. The stated conclusion is that the high-information-content region must be explicitly targeted and the dataset kept balanced for data-driven dynamic security assessment to work well.
Load-bearing premise
The load-bearing premise is that the system's most fragile oscillation mode changes smoothly enough as generator outputs are adjusted that a step-by-step gradient walk, with shrinking step sizes and a 1 MW grid, reliably reaches the security boundary and does not skip over fragmented parts of it.
Editorial extensions
If this is right
- A decision tree trained with 85% of its samples inside the high-information-content region on the 39-bus system, and 79% on the 162-bus system, still classifies far-from-boundary operating points accurately, so boundary-focused training data does not trade away far-field accuracy.
- Misclassified operating points cluster inside the high-information-content region even for the boundary-trained tree, which means residual errors live where the label flips and are not cured by adding more far-field samples.
- Balancing secure and insecure labels in the proposed-method dataset raises cross-test F1-scores, for example from 0.59 to 0.85 on the 162-bus boundary test set, so boundary enrichment alone is not enough.
- The Latin hypercube tree outperforms the importance tree on the proposed method's test set for the 39-bus system, indicating that a unimodal Gaussian importance distribution can miss a fragmented boundary.
- Because the directed walk needs only a scalar distance-to-boundary measure, the same toolbox extends to voltage stability or transient stability by swapping the damping-ratio sensitivity for another index.
Reading between the lines
- The paper's unimodal importance-sampling benchmark suggests a testable hypothesis: a mixture or copula-based importance distribution fitted to multiple boundary components would recover much of the gap to the proposed method, since the visible failure is a single Gaussian's inability to follow a curved or dispersed boundary.
- Because the directed walk finds the boundary component nearest each initialization point, the number and placement of feasible initialization dispatches acts as a tunable coverage knob; random restarts or wider polytope sampling would make boundary coverage explicit rather than incidental.
- The 1 MW discretization inside the high-information-content region sets a resolution floor for the decision boundary the classifier can learn; reducing the grid spacing would sharpen boundary accuracy at a higher sampling cost.
- The boundary-region density could be reused as an active-learning acquisition function, proposing new dispatches where the current classifier's margin is smallest and potentially lowering the simulation budget while keeping boundary coverage high.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents a dataset generation toolbox for dynamic security assessment (DSA) that combines optimization-based bound tightening, separating hyperplanes, and a directed-walk algorithm to produce operating points (OPs) near the small-signal stability and AC-feasibility boundary. The authors compare decision trees trained on datasets from their method, Latin Hypercube (LHC) sampling, and importance sampling on the PGLib-OPF 39-bus and 162-bus systems, reporting F1 scores on test sets from each sampling method plus an additional boundary-focused test set. The central claim, stated in the conclusion, is that datasets with a substantial share of boundary-adjacent OPs significantly enhance the performance of data-driven DSA tools, provided the dataset is balanced between secure and insecure points.
Significance. If the central claim were established, the work would be practically important: DSA dataset generation is a recognized bottleneck, and the authors contribute a modular, publicly available toolbox and a reproducible comparison on two standard test systems. The misclassification analysis in Figs. 9 and 10, showing that errors concentrate near the security boundary, is a useful diagnostic that supports the broader motivation. However, the experiments as designed do not isolate the effect of boundary-adjacent samples from other dataset differences, so the main causal claim is not yet supported by the evidence presented.
major comments (3)
- [IV-D, Tables I-III] The central claim that a 'substantial share of OPs near the security boundary' improves DSA performance is inferred from comparisons of decision trees trained on three pipelines that differ along multiple axes. The proposed method restricts sampling to an OBBT/hyperplane polytope (Section III-A), projects infeasible points through a different optimization (Section III-B), and produces path-dependent clusters via directed walks (Section III-C), while the LHC and importance benchmarks use their own sampling and projection mechanisms. The observed F1 differences therefore cannot be attributed specifically to the HIC-region share. Table III compounds this concern: rebalancing a proposed-method dataset changes F1 scores by roughly 0.2 to 0.3, showing that dataset balance is a major confound that is not controlled for in Tables I and II. A controlled experiment that varies the HIC share while holding the rest of the pipeline fixed (for example, resampling a single base dataset into variants with 0%, 20%, 50%, and 80% HIC points) is necessary before the conclusion in Section VI can be supported.
- [IV-D2] The 'Boundary' test set is generated by the proposed method, as stated in Section IV-D2 ('This test set contains OPs generated by the proposed method which lie near the security boundary'). This makes it a distributionally aligned test set rather than an independent probe of boundary classification; the proposed method's higher boundary F1 could reflect training/test distribution overlap rather than better classification of the true boundary. The authors should construct a boundary test set independently of the proposed method, for example by densely sampling a narrow damping-ratio band with LHC or importance sampling, or by evaluating on a hold-out set generated by a separate boundary-characterization procedure.
- [III-C1, Eqs. (12)-(13)] The directed-walk algorithm assumes that the damping ratio of the least-damped mode is smooth enough in generator-active-power space that a local finite-difference gradient, combined with step-size reduction and a 1 MW discretization, reliably walks toward the security boundary without missing disconnected boundary components. No evidence is provided for this smoothness or connectivity assumption, and the paper does not quantify how completely the proposed method covers the boundary. Because the method's advertised ability to 'capture' the boundary depends on this assumption, the authors should either provide a quantitative coverage analysis (for example, the fraction of LHC boundary points within a small distance of a proposed-method HIC point, across multiple random initializations) or explicitly discuss the assumption as a scope condition.
minor comments (6)
- [Tables I and II] Tables I and II are typeset in a way that separates row labels from data (for example, 'Proposed MethodTraining Testing' appears as disconnected text), making the tables difficult to read; the row and column structure should be redrawn.
- [Figures 4-8] Figures 4-8 show axis labels as unicode placeholders (for example, '/uni00000012/uni00000010/uni00000012') in the preprint; vector graphics with proper generator names are needed.
- [IV-D4] Section IV-D4 says 'The LHC and importance sampling datasets used for testing are shown in Fig. 3,' but Fig. 3 reports percentages of feasible, stable, secure, and HIC-region samples, not the test sets themselves; the text should be clarified.
- [III-C1, Eq. (12)] The step-size distance thresholds d1, d2, and d3 in Eq. (12) are not reported; only the values of epsilon_1 through epsilon_4 are given, so the directed-walk parameters are not fully reproducible.
- [Fig. 1] Figure 1 refers to 'Section IV.A' and subsequent sections, but the corresponding method descriptions are in Section III; the cross-references should be corrected.
- [IV-D3 and elsewhere] There are typos such as 'missclassificaton' in Section IV-D3 and 'anayzed' in Section IV-D; a proofreading pass is needed.
Circularity Check
Boundary test set is drawn from the proposed method's own HIC sampling target, so the headline claim is only partially supported; the LHC/importance generalization results remain independent evidence.
-
self definitional
[Section IV-D2 (Tables I and II), Section II-C Eq. (6), Section IV-A]
"Each DT is evaluated on the test sets generated by the other two sampling methods, as well as an additional test set named boundary. This test set contains OPs generated by the proposed method which lie near the security boundary (2.9% < ζ <3.1%)."
The boundary test set (2.9% < ζ < 3.1%) is a strict subset of the HIC region defined in Eq. (6) as γ − β < γ_k < γ + β with β = 0.25%, i.e. 2.75% < γ_min < 3.25% (Section IV-A). The proposed method's directed walk is explicitly designed to populate this HIC region, and Fig. 3 reports that 85% (39-bus) and 79% (162-bus) of its training points lie in it. Therefore the proposed DT's high F1 on the boundary test set measures interpolation within its own sampling target, not an independent test of the value of boundary-adjacent training data.
full rationale
The paper is an empirical dataset-generation study, not a first-principles derivation, so most of its chain is not circular in the equation-identity sense. The self-citations to [14] and [17] are legitimate prior-work references with published proofs; they are not used to forbid alternatives or to smuggle in an unverified ansatz. The main circularity concern is evaluative: the 'boundary' test set is generated by the proposed method and lies inside the same HIC band that the proposed method targets by construction. This makes the boundary test-set comparison partially self-referential and unable to support the strong causal claim on its own. The generalization results to LHC and importance test sets are independent of the proposed sampling distribution and give the central claim real empirical content, but those comparisons still do not isolate the HIC share from other dataset differences (polytope restriction, projection rules, path-dependent clustering, balance), as Table III shows balance alone changes F1 by roughly 0.2-0.3. This is a confounding/attribution problem rather than a derivation that reduces to its inputs, so the circularity score is moderate rather than high.
Assumptions & free parameters
free parameters (8)
- Small-signal stability threshold zeta_min =
3%
- HIC margin beta =
0.25%
- Directed walk step scalars epsilon_1 to epsilon_4 =
[4, 3, 2, 1]
- Step-size distance thresholds d_1 to d_3 =
not reported
- Maximum directed walk steps kappa_max and kappa_HIC =
kappa_max=30, kappa_HIC=15
- Number of initialization samples N2 =
4000
- Importance sampling covariance scaling s =
0.25
- Nominal load scaling factor =
0.8
assumptions (5)
- domain assumption The damping ratio of the least damped mode is a sufficient scalar measure of small-signal stability margin.
- standard math Infeasibility with respect to the QC relaxation implies infeasibility with respect to the non-convex AC-OPF.
- domain assumption The AC-OPF feasible space describes all feasible operating points.
- domain assumption The security boundary is reachable by local gradient steps of the damping ratio with finite-difference sensitivities and 1 MW discretization, without missing disconnected components.
- ad hoc to paper The 80% nominal load scaling creates a representative test regime for boundary sampling.
Cite this review
Pith. "Pith review of A Dataset Generation Toolbox for Dynamic Security Assessment: On the Role of the Security Boundary." pith.science (2026). https://pith.science/paper/CBZ42HHJ
@misc{pith2026250109513,
author = {Pith},
title = {Pith review of: A Dataset Generation Toolbox for Dynamic Security Assessment: On the Role of the Security Boundary},
year = {2026},
howpublished = {\url{https://pith.science/paper/CBZ42HHJ}},
note = {Machine review of arXiv:2501.09513}
}
read the original abstract
Dynamic security assessment (DSA) is crucial for ensuring the reliable operation of power systems. However, conventional DSA approaches are becoming intractable for future power systems, driving interest in more computationally efficient data-driven methods. Efficient dataset generation is a cornerstone of these methods. While importance and generic sampling techniques often focus on operating points near the system's security boundary, systematic methods for sampling in this region remain scarce. Furthermore, the impact of sampling near the security boundary on the performance of data-driven DSA methods has yet to be established. This paper highlights the critical role of accurately capturing security boundaries for effective security assessment. As such, we propose a novel method for generating a high number of samples close to the security boundary, considering both AC feasibility and small-signal stability. Case studies on the PGLib-OPF 39-bus and PGLib-OPF 162-bus systems demonstrate the importance of including boundary-adjacent operating points in training datasets while maintaining a balanced distribution of secure and insecure points.
Figures
Figures from the paper (6 more)
Forward citations
Cited by 1 Pith paper
-
Towards Probabilistic Dynamic Security Assessment and Enhancement of Large Power Systems
A probabilistic dynamic security assessment pipeline with coverage-aware Monte Carlo error bounds, stability screening, protection-uncertainty cascade simulations, and SVM-based root-cause analysis is demonstrated on ...
Reference graph
Works this paper leans on
-
[1]
Operating in the fog: Security management under uncertainty,
P. Panciatici, G. Bareux, and L. Wehenkel, “Operating in the fog: Security management under uncertainty,” IEEE Power and Energy Magazine, vol. 10, no. 5, pp. 40–49, 2012. 13
work page 2012
-
[2]
Implementation of a massively parallel dynamic security assessment platform for large-scale grids,
I. Konstantelos, G. Jamgotchian, S. H. Tindemans, P. Duchesne, S. Cole, C. Merckx, G. Strbac, and P. Panciatici, “Implementation of a massively parallel dynamic security assessment platform for large-scale grids,” IEEE Transactions on Smart Grid , vol. 8, no. 3, pp. 1417–1426, 2016
work page 2016
-
[3]
Prosumer based energy management and sharing in smart grid,
R. Zafar, A. Mahmood, S. Razzaq, W. Ali, U. Naeem, and K. She- hzad, “Prosumer based energy management and sharing in smart grid,” Renewable and Sustainable Energy Reviews , vol. 82, pp. 1675–1684, 2018
work page 2018
-
[4]
Recent developments in machine learning for energy systems reliability management,
L. Duchesne, E. Karangelos, and L. Wehenkel, “Recent developments in machine learning for energy systems reliability management,” Pro- ceedings of the IEEE , vol. 108, no. 9, pp. 1656–1676, 2020
work page 2020
-
[5]
Learning to run a power network with trust,
A. Marot, B. Donnot, K. Chaouache, A. Kelly, Q. Huang, R.-R. Hossain, and J. L. Cremer, “Learning to run a power network with trust,” Electric Power Systems Research, vol. 212, p. 108487, 2022
2022
-
[6]
Dynamic security assess- ment,
P. W. Sauer, K. L. Tomsovic, and V . Vittal, “Dynamic security assess- ment,” Power system stability and control , vol. 5, pp. 421–430, 2007
work page 2007
-
[7]
Efficient database generation for decision tree based power system security assessment,
V . Krishnan, J. D. McCalley, S. Henry, and S. Issad, “Efficient database generation for decision tree based power system security assessment,” IEEE Transactions on Power Systems , vol. 26, no. 4, pp. 2319–2327, 2011
work page 2011
-
[8]
Generating quality datasets for real-time security assessment: Balancing historically rele- vant and rare feasible operating conditions,
A.-A. B. Bugaje, J. L. Cremer, and G. Strbac, “Generating quality datasets for real-time security assessment: Balancing historically rele- vant and rare feasible operating conditions,” International Journal of Electrical Power & Energy Systems , vol. 154, p. 109427, 2023
2023
Show all 34 references
-
[9]
State-of-the-art of data collection, analytics, and future needs of transmission utilities worldwide to account for the continuous growth of sensing data,
F. R. S. Sevilla, Y . Liu, E. Barocio, P. Korba, M. Andrade, F. Bellizio, J. Bos, B. Chaudhuri, H. Chavez, J. Cremer, et al. , “State-of-the-art of data collection, analytics, and future needs of transmission utilities worldwide to account for the continuous growth of sensing ...
2022
-
[10]
Using vine copulas to generate representative system states for machine learning,
I. Konstantelos, M. Sun, S. H. Tindemans, S. Issad, P. Panciatici, and G. Strbac, “Using vine copulas to generate representative system states for machine learning,” IEEE Trans. Power Syst., vol. 34, no. 1, pp. 225– 235, 2018
2018
-
[11]
Evaluating com- posite approaches to modelling high-dimensional stochastic variables in power systems,
M. Sun, I. Konstantelos, S. Tindemans, and G. Strbac, “Evaluating com- posite approaches to modelling high-dimensional stochastic variables in power systems,” in 2016 Power Systems Computation Conference (PSCC), pp. 1–8, IEEE, 2016
2016
-
[12]
A systematic approach for dynamic security assessment and the corresponding preventive control scheme based on decision trees,
C. Liu, K. Sun, Z. H. Rather, Z. Chen, C. L. Bak, P. Thøgersen, and P. Lund, “A systematic approach for dynamic security assessment and the corresponding preventive control scheme based on decision trees,” IEEE Trans. Power Syst. , vol. 29, no. 2, pp. 717–730, 2013
2013
-
[13]
Impor- tance sampling based decision trees for security assessment and the corresponding preventive control schemes: The danish case study,
C. Liu, Z. H. Rather, Z. Chen, C. L. Bak, and P. Thøgersen, “Impor- tance sampling based decision trees for security assessment and the corresponding preventive control schemes: The danish case study,” in 2013 IEEE Grenoble Conference , pp. 1–6, IEEE, 2013
2013
-
[14]
Efficient database generation for data-driven security assessment of power sys- tems,
F. Thams, A. Venzke, R. Eriksson, and S. Chatzivasileiadis, “Efficient database generation for data-driven security assessment of power sys- tems,” IEEE Trans. Power Syst. , vol. 35, no. 1, pp. 30–41, 2019
2019
-
[15]
Split-based sequential sampling for realtime security assessment,
A.-A. B. Bugaje, J. L. Cremer, and G. Strbac, “Split-based sequential sampling for realtime security assessment,” International Journal of Electrical Power & Energy Systems , vol. 146, p. 108790, 2023
2023
-
[16]
Opf-learn: An open- source framework for creating representative ac optimal power flow datasets,
T. Joswig-Jones, K. Baker, and A. S. Zamzam, “Opf-learn: An open- source framework for creating representative ac optimal power flow datasets,” in 2022 IEEE Power & Energy Society Innovative Smart Grid Technologies Conference (ISGT), IEEE, 2022
2022
-
[17]
Efficient creation of datasets for data-driven power system applications,
A. Venzke, D. K. Molzahn, and S. Chatzivasileiadis, “Efficient creation of datasets for data-driven power system applications,” Electric Power Systems Research, vol. 190, p. 106614, 2021
2021
-
[18]
Final code thesis lola charles
L. Charles, “Final code thesis lola charles.” https://github.com/lolachls/ Final Code Thesis Lola Charles, 2024. Accessed: 2024-12-10
2024
-
[19]
Strengthening convex relaxations with bound tightening for power network optimization,
C. Coffrin, H. L. Hijazi, and P. Van Hentenryck, “Strengthening convex relaxations with bound tightening for power network optimization,” in Principles and Practice of Constraint Programming, pp. 39–57, Springer International Publishing, 2015
2015
-
[20]
Dsa-learn
B. Giraud et al. , “Dsa-learn.” https://github.com/bastiengiraud/ DSA-learn, 2024. Accessed: 2024-11-30
2024
-
[21]
Review of on-line dynamic security assessment tools and techniques,
C. W. G. C4.601, “Review of on-line dynamic security assessment tools and techniques,” vol. 4, p. 601, 2007
2007
-
[22]
The qc relaxation: A theoretical and computational study on optimal power flow,
C. Coffrin, H. L. Hijazi, and P. Van Hentenryck, “The qc relaxation: A theoretical and computational study on optimal power flow,” IEEE Transactions on Power Systems , vol. 31, no. 4, pp. 3008–3018, 2015
2015
-
[23]
A survey of relaxations and approximations of the power flow equations,
D. K. Molzahn, I. A. Hiskens, et al. , “A survey of relaxations and approximations of the power flow equations,” Foundations and Trends® in Electric Energy Systems , vol. 4, no. 1-2, pp. 1–221, 2019
2019
-
[24]
D. P. Kroese, T. Taimre, and Z. I. Botev, Handbook of monte carlo methods. John Wiley & Sons, 2013
2013
-
[25]
The power grid library for benchmarking ac optimal power flow algorithms,
S. Babaeinejadsarookolaee, A. Birchfield, R. D. Christie, C. Coffrin, C. DeMarco, R. Diao, M. Ferris, S. Fliscounakis, S. Greene, R. Huang, et al., “The power grid library for benchmarking ac optimal power flow algorithms,” arXiv preprint arXiv:1908.02788 , 2019
1908 arXiv
-
[26]
Powersystems. jl—a power system data management package for large scale modeling,
J. D. Lara, C. Barrows, D. Thom, D. Krishnamurthy, and D. Callaway, “Powersystems. jl—a power system data management package for large scale modeling,” SoftwareX, vol. 15, p. 100747, 2021
2021
-
[27]
DTU Computing Center resources,
D. C. Center et al., “DTU Computing Center resources,” 2024
2024
-
[28]
Powermodels. jl: An open-source framework for exploring power flow formulations,
C. Coffrin, R. Bent, K. Sundar, Y . Ng, and M. Lubin, “Powermodels. jl: An open-source framework for exploring power flow formulations,” in 2018 Power Systems Computation Conference (PSCC) , IEEE, 2018
2018
-
[29]
Jump: A modeling language for mathematical optimization,
I. Dunning, J. Huchette, and M. Lubin, “Jump: A modeling language for mathematical optimization,” SIAM review, vol. 59, no. 2, pp. 295–320, 2017
2017
-
[30]
On the implementation of an interior- point filter line-search algorithm for large-scale nonlinear programming,
A. W ¨achter and L. T. Biegler, “On the implementation of an interior- point filter line-search algorithm for large-scale nonlinear programming,” Mathematical programming, vol. 106, pp. 25–57, 2006
2006
-
[31]
Powersimulationsdynamics. jl–an open source modeling package for modern power systems with inverter-based resources,
J. D. Lara, R. Henriquez-Auba, M. Bossart, D. S. Callaway, and C. Barrows, “Powersimulationsdynamics. jl–an open source modeling package for modern power systems with inverter-based resources,”arXiv preprint arXiv:2308.02921, 2023
2023 arXiv
-
[32]
volesti: V olume approximation and sampling for convex polytopes in r,
A. Chalkis and V . Fisikopoulos, “volesti: V olume approximation and sampling for convex polytopes in r,” arXiv preprint arXiv:2007.01578 , 2020
2007 arXiv
-
[33]
Classification and regression trees,
L. Breiman et al. , “Classification and regression trees,” Monterey, CA: Wadsworth and Brooks/Cole, 1984
1984
-
[34]
Scikit-learn: Machine learning in Python,
F. Pedregosa, G. Varoquaux, A. Gramfort, V . Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V . Dubourg, et al. , “Scikit-learn: Machine learning in Python,” Journal of Machine Learn- ing Research, vol. 12, pp. 2825–2830, 2011
2011
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.