Pith. sign in

REVIEW 1 major objections 5 minor 30 references

This paper claims that an operator can certify, per control window and with high confidence, a bound on the fraction of skipped N-1 contingencies that overload lines, even under arbitrary controller-induced deployment shift, by using a chea

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 05:50 UTC pith:LYMZH3BX

load-bearing objection A solid, honest transductive Learn-Then-Test certificate for N-1 thermal screening, with a real operational gap that needs one explicit assumption: the audit must be genuinely random and unpredictable to the controller. the 1 major comments →

arxiv 2607.13221 v1 pith:LYMZH3BX submitted 2026-07-14 eess.SY cs.AIcs.LGcs.SY

Audited Selective Verification for Risk-Controlled N-1 Thermal Contingency Screening under Deployment Shift

Jayakumar Manoharan This is my paper
classification eess.SY cs.AIcs.LGcs.SY
keywords N-1 contingency screeningrisk controldistribution-free guaranteethermal violation ratedeployment shiftselective verificationtransmission system securityonline audit
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that an operator can certify, per control window and with high confidence, a bound on the fraction of skipped N-1 contingencies that overload lines — even when a learned or optimization-based controller shifts the operating point away from anything seen before. The trick is to stop trusting the fast linear surrogate for safety and use it only to decide which contingencies to skip, while a small random audit runs full AC power flow and calibrates a skip threshold. The certificate is distribution-free: it holds for any fixed window and any surrogate, requiring only that the audit be a uniform random sample independent of the violation labels. On three public transmission systems up to 1354 buses, realized violation rates stay within budget while deterministic and threshold-based screens become unsafe under shift, at 29 to 75 percent reduction in full power-flow solves per operating point.

Core claim

The paper's central claim is Theorem 1: for each control window, if the audit is a uniform random sample of the contingency set drawn independently of the labels, and the AC power-flow solve is the trusted verifier, then the threshold τ* selected by fixed-sequence testing of conservative binomial upper confidence limits satisfies R(τ*) ≤ α with probability at least 1−δ over the audit. The guarantee holds for an arbitrary fixed window, hence for arbitrary deployment distribution and controller-induced shift, and makes no assumption on surrogate accuracy or cross-window stationarity. The unverified trusted subset (skipped, not audited) then has violation rate at most α/(1−f), where f is the au

What carries the argument

The load-bearing mechanism is an audited fixed-sequence threshold test. A cheap linear surrogate (line-outage distribution factors) scores every contingency. A small uniform random audit is solved by full AC, giving (score, violation) pairs. Candidate skip thresholds are tested in a data-independent order (strictest skip set first); for each, the number of violations in the audited part of the skip set is compared with a conservative upper confidence limit for a binomial rate, and testing stops at the first non-accepted threshold. Because the order is fixed and each per-threshold test has level δ, the family-wise error rate is controlled at δ, so the selected threshold's true violation rate

Load-bearing premise

The entire per-window guarantee rests on the audit being a uniform random sample of the contingency set drawn independently of the violation labels; if the audit is deterministic, predictable from the labels, or adversarially selected, the finite-sample safety certificate collapses.

What would settle it

Run ASV-N1 on a stressed system such as PEGASE with an audit that is deliberately not label-independent — for example, always auditing the contingencies the surrogate ranks safest, or using a fixed seed — and count across many windows the fraction in which the trusted-set violation rate exceeds α. If that fraction exceeds δ, the guarantee's dependence on Assumption 1 is confirmed. Conversely, a search over fixed windows and audit distributions that violates the hypergeometric-to-binomial dominance would refute the p-value lemma.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Any controller — reinforcement-learned, model-predictive, or optimization-based — can be wrapped with a safety certificate without modifying the controller itself; the certificate audits the operating point, not the policy.
  • An operator can set a risk budget α and confidence δ, and read off the verification cost: tighter budgets monotonically raise the AC-solve fraction, and an unachievable budget leads to verify-all, which is safe by construction.
  • Under deployment shift, the method degrades gracefully: a surrogate that stops separating safe from unsafe contingencies forces more verification, so cost rises but the violation rate stays within budget.
  • Batching contingencies across consecutive operating points tightens the certificate and raises savings from 29–75 percent at the single operating point to 36–80 percent, at coarser time resolution.
  • The certificate controls the violation rate of the skip set at α, with the unverified trusted subset at α/(1−f); audited contingencies carry no residual risk because their true labels are known.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Because the audit already runs full AC, extending the same calibration to a joint thermal-and-voltage violation label is a direct next step; the statistical machinery would not change.
  • The guarantee's dependence on a genuinely random, label-independent audit suggests that in a real energy management system, the audit must be drawn by a hardware or cryptographic random process each window; fixed seeds would void the certificate.
  • A testable extension is to train a learned surrogate on accumulated audited labels: validity would be unchanged, but a better surrogate would raise the skip threshold and cut cost on stressed systems where linear screening fails.
  • Combining the per-window guarantee with a time-uniform confidence sequence could certify a long-run budget over arbitrarily many adaptively chosen windows without dividing δ by the number of windows.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

1 major / 5 minor

Summary. The paper introduces Audited Selective Verification (ASV-N1), a per-window risk-budgeted screening layer for N-1 thermal contingency analysis. A cheap linear surrogate scores all credible contingencies; a small uniform audit is solved with full AC; and a Learn-Then-Test (fixed-sequence) calibration selects the largest surrogate threshold for which the audited violation count supports a Clopper-Pearson one-sided statement, yielding a certified bound on the violation rate of the skipped set. The guarantee is transductive: it conditions on the fixed operating point and randomizes only the audit, so it holds for arbitrary deployment distributions and does not rely on surrogate accuracy. Experiments on IEEE 118-bus, IEEE 300-bus, and PEGASE 1354-bus systems show the realized trusted-set violation rate remains below budget while reducing AC solves by 29–75% at a single operating point.

Significance. The paper addresses an important practical problem: providing quantifiable safety guarantees for fast contingency screening when a learning-based or model-predictive controller induces distribution shift. The central idea—using the surrogate only to choose what to verify, and resting the certificate on the audit—is simple but effective. The theoretical chain is sound: the hypergeometric/binomial tail dominance justifies the p-value, fixed-sequence testing gives FWER control, and the union bound handles adaptive audit sizing. The empirical evaluation is thorough, reproducible, and includes a genuine controller-induced shift. If the audit-unpredictability caveat discussed below is resolved, the method would be a valuable addition to the toolbox for risk-based security assessment.

major comments (1)
  1. [Section VI, Assumption 1; Section V.A; Section VII.A] The stated per-window guarantee requires the audit to be not only label-independent but also unpredictable to the controller at the time it commits the operating point. As written, Assumption 1 only requires a uniform random sample independent of labels. If the audit uses a fixed seed or deterministic schedule (the paper reports 'all random seeds are fixed' in experiments), a controller that can infer the audit can choose an operating point in which exactly the audited contingencies are safe and the un-audited skipped contingencies violate. The labels are then causally dependent on the audit, the hypergeometric p-value in Lemma 1 is invalid, and the event R(τ*) > α can occur with probability 1 over the audit. This is not covered by 'arbitrary deployment distribution' because the fixed-window conditioning in Theorem 1 does not hold when the window is selected after the audit. I recommend
minor comments (5)
  1. [Section VI, Lemma 1] The proof relies on external tail bounds [27,28] without stating the relevant inequality. For self-containedness, please include the specific Hoeffding/Serfling result used for hypergeometric-binomial dominance, or cite equation numbers in the appendix.
  2. [Section IV] The 'negative result' presented is empirical rather than a formal theorem. Consider calling it an 'empirical failure' or 'demonstration' to avoid confusion with a universally quantified negative statement.
  3. [Section V.A] In Algorithm 1, step 3 says 'Verify (full AC) all i with r_i > τ*'—but audited contingencies above the threshold are already solved. Proposition 2 clarifies the double-counting, but the algorithm text could be adjusted for clarity, e.g., 'Verify (full AC) all i not in A with r_i > τ*'.
  4. [Section VII.A] The statement 'all random seeds are fixed' is appropriate for reproducibility, but in a deployment setting fixed seeds may compromise audit unpredictability. Please add a note that production use should employ a secure random source hidden from the controller, and tie this to the threat model.
  5. [General] There are a few typographical issues: 'V oltage' in Assumption 2, a stray space in the author affiliation header, and minor spacing issues in Section I. A careful proofread would improve readability.

Circularity Check

0 steps flagged

No significant circularity: the central certificate is a standard finite-sample confidence statement whose validity is conditional on audit randomization, not on fitted values or self-citations.

full rationale

Theorem 1 explicitly conditions on a fixed window {(r_i, V_i)} and randomizes only the audit; Assumption 1 is a stated precondition, not a conclusion derived from the target. Lemma 1's p-value validity rests on the hypergeometric/binomial stochastic-dominance results of Hoeffding and Serfling, which are external and do not assume R(τ) ≤ α. The fixed-sequence testing proof uses standard FWER control (Holm), and the Clopper-Pearson bound is applied to audited data only. The adaptive audit-sizing rule in Proposition 1 does peek at audited labels, but the paper corrects this with an explicit union bound over the G candidate sizes (δ/G), which is a standard multiplicity correction rather than a fitted parameter renamed as a prediction. The realized violation rates in Section VII are reported as empirical checks; they are not used to define or calibrate the method, and the guarantee does not depend on them. The paper contains no load-bearing self-citation: references to Learn-Then-Test and related methods are to external prior work, and the paper explicitly states that the underlying risk-control machinery is standard. The claim of robustness to 'arbitrary deployment shift' is conditional on Assumption 1 and is achieved by auditing the deployment distribution directly, which is a legitimate transductive application rather than a circular definition. The operational limitation that a predictable audit could enable a controller to game the guarantee is acknowledged implicitly by Assumption 1 and is a matter of engineering applicability, not circularity. No enumerated circularity pattern is present.

Axiom & Free-Parameter Ledger

0 free parameters · 6 axioms · 0 invented entities

The method introduces no physical entities or fitted constants. The statistical machinery is standard; the only non-standard ingredients are the N-1-specific adaptation (audit plus fixed-sequence calibration) and the adaptive audit-sizing heuristic with a union-bound correction. Assumptions 1 and 2 are the load-bearing domain assumptions.

axioms (6)
  • standard math Hypergeometric lower tail is stochastically dominated by binomial lower tail at the same mean (Hoeffding 1963; Serfling 1974).
    Invoked in Lemma 1 to show the audited violation count under the null yields a super-uniform p-value; not proved in the paper.
  • standard math Clopper-Pearson one-sided upper bound gives a valid (conservative) p-value for the binomial rate.
    Used in the fixed-sequence calibration (1); the paper relies on the standard exact binomial confidence limit.
  • standard math Fixed-sequence multiple testing with level-δ valid p-values controls the family-wise error rate at δ under arbitrary dependence.
    The FWER control argument in Theorem 1's proof; it references [5] and [29] (Holm), though for fixed-sequence testing the property is standard.
  • domain assumption Assumption 1: the audit is a uniform random sample of the contingency set, independent of the violation labels; its size may be fixed or chosen from label-independent information (or corrected via a union bound).
    The entire finite-sample guarantee is conditional on this randomization. If the audit is not random, Theorem 1 does not hold.
  • domain assumption Assumption 2: the AC N-1 power flow is the trusted verifier defining V_i, and the certified risk is thermal violations only; voltage/reactive limits and islanding/convergence cases are outside the scope.
    The method certifies whatever the oracle labels; accuracy of the certificate is with respect to this oracle.
  • domain assumption The contingency set and surrogate scores are fixed before the audit; scores are allowed to depend on the operating point but not on the labels.
    The proof conditions on the fixed window {(r_i, V_i)}; this is the transductive setup.

pith-pipeline@v1.3.0-alltime-deepseek · 15839 in / 24881 out tokens · 243437 ms · 2026-08-02T05:50:52.012347+00:00 · methodology

0 comments
read the original abstract

Real-time N-1 contingency screening in an energy management system trades assurance against cost: verifying every credible outage with full power flow is too slow, while fast linear-sensitivity screening gives no statistical guarantee and can silently pass unsafe operating points, especially when a controller drives the system into unfamiliar regimes. This paper introduces Audited Selective Verification, a risk-budgeted screening and triage layer for any controller's output (optimization, model-predictive, or learned). A cheap surrogate proposes which outages to skip; an online audit runs full power flow on a small random sample each window; and a calibrated threshold certifies a thermal-violation-rate bound for the skipped set at a chosen budget and confidence, with a corresponding bound for the unverified trusted subset. Validity rests on real verification and the audit rather than on surrogate accuracy, so it holds under arbitrary deployment shift. It is a risk-budgeted screen, not a replacement for deterministic verification when policy requires checking every credible contingency. On three public transmission systems up to 1354 buses, the realized violation rate stays within budget, standard deterministic and calibrated screens become unsafe under shift, and the method cuts full power-flow studies by 29 to 75 percent per real-time operating point.

Figures

Figures reproduced from arXiv: 2607.13221 by Jayakumar Manoharan.

Figure 1
Figure 1. Figure 1: Threshold certificates fail under shift. Cumulative skipped-set violation [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Left: safety. A deterministic surrogate screen (no audit) is unsafe under shift on PEGASE (skipped-set N-1 violation [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: The (α, δ) guarantee holds across many windows. Cumulative distribution of the per-window trusted-set violation rate over hundreds of windows and forty seeds per system. The horizontal line at 1 − δ = 0.9 meets each curve at or before α = 0.15, so at most a δ fraction of windows breach the budget (breach fractions: IEEE 118-bus 5.8 percent, IEEE 300-bus and PEGASE 0 percent). IEEE 118-bus, indicating the c… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

30 extracted references · 10 linked inside Pith

  1. [1]

    A. J. Wood, B. F. Wollenberg, and G. B. Shebl ´e,Power Generation, Operation, and Control, 3rd ed. John Wiley and Sons, 2013

    2013

  2. [2]

    Learning to run a power network challenge: A retrospective analysis,

    A. Marot, B. Donnot, C. Romero, B. Donon, M. Lerousseau, L. Veyrin- Forrer, and I. Guyon, “Learning to run a power network challenge: A retrospective analysis,”arXiv preprint arXiv:2103.03104, 2021

    Pith/arXiv arXiv 2021

  3. [3]

    HUGO: Highlighting unseen grid options, combining deep reinforcement learning with a heuristic target topology approach,

    M. Lehna, C. Holzh ¨uter, S. Tomforde, and C. Scholz, “HUGO: Highlighting unseen grid options, combining deep reinforcement learning with a heuristic target topology approach,”arXiv preprint arXiv:2405.00629, 2024

    Pith/arXiv arXiv 2024

  4. [4]

    A review of safe reinforcement learning methods for modern power systems,

    T. Su, T. Wu, J. Zhao, A. Scaglione, and L. Xie, “A review of safe reinforcement learning methods for modern power systems,”arXiv preprint arXiv:2407.00304, 2024

    Pith/arXiv arXiv 2024

  5. [5]

    Learn then test: Calibrating predictive algorithms to achieve risk control,

    A. N. Angelopoulos, S. Bates, E. J. Cand `es, M. I. Jordan, and L. Lei, “Learn then test: Calibrating predictive algorithms to achieve risk control,”arXiv preprint arXiv:2110.01052, 2021

    Pith/arXiv arXiv 2021

  6. [6]

    Distribution-free, risk-controlling prediction sets,

    S. Bates, A. Angelopoulos, L. Lei, J. Malik, and M. I. Jordan, “Distribution-free, risk-controlling prediction sets,”Journal of the ACM, vol. 68, no. 6, pp. 1-34, 2021

    2021

  7. [7]

    Adaptive conformal inference under dis- tribution shift,

    I. Gibbs and E. J. Cand `es, “Adaptive conformal inference under dis- tribution shift,” inAdvances in Neural Information Processing Systems (NeurIPS), 2021

    2021

  8. [8]

    pandapower: An open-source python tool for convenient modeling, analysis, and optimization of electric power systems,

    L. Thurner, A. Scheidler, F. Sch ¨afer, J.-H. Menke, J. Dollichon, F. Meier, S. Meinecke, and M. Braun, “pandapower: An open-source python tool for convenient modeling, analysis, and optimization of electric power systems,”IEEE Transactions on Power Systems, vol. 33, no. 6, pp. 6510- 6521, 2018

    2018

  9. [9]

    Security analysis and opti- mization,

    B. Stott, O. Alsac, and A. J. Monticelli, “Security analysis and opti- mization,”Proceedings of the IEEE, vol. 75, no. 12, pp. 1623-1644, 1987

    1987

  10. [10]

    Automatic contingency selection,

    G. C. Ejebe and B. F. Wollenberg, “Automatic contingency selection,” IEEE Transactions on Power Apparatus and Systems, vol. PAS-98, no. 1, pp. 97-109, 1979

    1979

  11. [11]

    Security-constrained optimal power flow with post-contingency corrective rescheduling,

    A. Monticelli, M. V . F. Pereira, and S. Granville, “Security-constrained optimal power flow with post-contingency corrective rescheduling,” IEEE Transactions on Power Systems, vol. 2, no. 1, pp. 175-180, 1987

    1987

  12. [12]

    State-of-the-art, challenges, and future trends in security-constrained optimal power flow,

    F. Capitanescu, J. L. M. Ramos, P. Panciatici, D. Kirschen, A. M. Marcolini, L. Platbrood, and L. Wehenkel, “State-of-the-art, challenges, and future trends in security-constrained optimal power flow,”Electric Power Systems Research, vol. 81, no. 8, pp. 1731-1741, 2011

    2011

  13. [13]

    Adversarially robust learning for security-constrained optimal power flow,

    P. L. Donti, A. Agarwal, N. V . Bedmutha, L. Pileggi, and J. Z. Kolter, “Adversarially robust learning for security-constrained optimal power flow,”arXiv preprint arXiv:2111.06961, 2021

    Pith/arXiv arXiv 2021

  14. [14]

    Online risk-based security assessment,

    M. Ni, J. D. McCalley, V . Vittal, and T. Tayyib, “Online risk-based security assessment,”IEEE Transactions on Power Systems, vol. 18, no. 1, pp. 258-265, 2003. IEEE TRANSACTIONS ON POWER SYSTEMS 10

    2003

  15. [15]

    Comparison of risk-based and deterministic security assessments,

    D. S. Kirschen and D. Jayaweera, “Comparison of risk-based and deterministic security assessments,”IET Generation, Transmission and Distribution, vol. 1, no. 4, pp. 527-533, 2007

    2007

  16. [16]

    V ovk, A

    V . V ovk, A. Gammerman, and G. Shafer,Algorithmic Learning in a Random World. Springer, 2005

    2005

  17. [17]

    Conformal risk control,

    A. N. Angelopoulos, S. Bates, A. Fisch, L. Lei, and T. Schuster, “Conformal risk control,” inInternational Conference on Learning Representations (ICLR), 2024

    2024

  18. [18]

    Conformal prediction under covariate shift,

    R. J. Tibshirani, R. Foygel Barber, E. J. Cand `es, and A. Ramdas, “Conformal prediction under covariate shift,” inAdvances in Neural Information Processing Systems (NeurIPS), 2019

    2019

  19. [19]

    Conformal prediction beyond exchangeability,

    R. F. Barber, E. J. Cand `es, A. Ramdas, and R. J. Tibshirani, “Conformal prediction beyond exchangeability,”The Annals of Statistics, vol. 51, no. 2, pp. 816-845, 2023

    2023

  20. [20]

    Conformal policy control,

    D. Prinster, C. Fannjiang, J. W. Park, K. Cho, A. Liu, S. Saria, and S. Stanton, “Conformal policy control,”arXiv preprint arXiv:2603.02196, 2026

    Pith/arXiv arXiv 2026

  21. [21]

    Decision-calibrated prediction sets for robust power system operations,

    A. Stratigakos, H. Wen, E. Spyrou, and P. Pinson, “Decision-calibrated prediction sets for robust power system operations,”arXiv preprint arXiv:2606.02081, 2026

    Pith/arXiv arXiv 2026

  22. [22]

    Computationally and sample efficient safe reinforcement learning using adaptive conformal prediction,

    H. Zhou, Y . Zhang, and W. Luo, “Computationally and sample efficient safe reinforcement learning using adaptive conformal prediction,”arXiv preprint arXiv:2503.17678, 2025

    Pith/arXiv arXiv 2025

  23. [23]

    Safe reinforcement learning via shielding,

    M. Alshiekh, R. Bloem, R. Ehlers, B. K ¨onighofer, S. Niekum, and U. Topcu, “Safe reinforcement learning via shielding,” inAAAI Con- ference on Artificial Intelligence, 2018

    2018

  24. [24]

    RL2Grid: Bench- marking reinforcement learning in power grid operations,

    E. Marchesini, B. Donnot, C. Crozier, I. Dytham, C. Merz, L. Schewe, N. Westerbeck, C. Wu, A. Marot, and P. L. Donti, “RL2Grid: Bench- marking reinforcement learning in power grid operations,”arXiv preprint arXiv:2503.23101, 2025

    Pith/arXiv arXiv 2025

  25. [25]

    SafePowerGraph: Safety-aware evaluation of graph neural networks for transmission power grids,

    S. Ghamizi, A. Bojchevski, A. Ma, and J. Cao, “SafePowerGraph: Safety-aware evaluation of graph neural networks for transmission power grids,”arXiv preprint arXiv:2407.12421, 2024

    Pith/arXiv arXiv 2024

  26. [26]

    CommonPower: A framework for safe data-driven smart grid control,

    M. Eichelbeck, H. Markgraf, and M. Althoff, “CommonPower: A framework for safe data-driven smart grid control,”arXiv preprint arXiv:2406.03231, 2024

    arXiv 2024

  27. [27]

    Probability inequalities for sums of bounded random variables,

    W. Hoeffding, “Probability inequalities for sums of bounded random variables,”Journal of the American Statistical Association, vol. 58, no. 301, pp. 13-30, 1963

    1963

  28. [28]

    Probability inequalities for the sum in sampling without replacement,

    R. J. Serfling, “Probability inequalities for the sum in sampling without replacement,”The Annals of Statistics, vol. 2, no. 1, pp. 39-48, 1974

    1974

  29. [29]

    A simple sequentially rejective multiple test procedure,

    S. Holm, “A simple sequentially rejective multiple test procedure,” Scandinavian Journal of Statistics, vol. 6, no. 2, pp. 65-70, 1979

    1979

  30. [30]

    Time-uniform, nonparametric, nonasymptotic confidence sequences,

    S. R. Howard, A. Ramdas, J. McAuliffe, and J. Sekhon, “Time-uniform, nonparametric, nonasymptotic confidence sequences,”The Annals of Statistics, vol. 49, no. 2, pp. 1055-1080, 2021. APPENDIXA FULLPROOF OFTHEOREM1 We restate the setting. Condition on the fixed window: the pairs{(r i, Vi)}N i=1 are arbitrary fixed quantities. The auditA includes each index...

    2021