{"id":"645d35fb-8efd-4f52-962b-2ab73b10e845","arxiv_id":"1908.02788","paper_version":2,"verdict":"ACCEPT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"low","formal_verification":"none","parameter_count":6,"one_line_summary":"PGLib-OPF provides a standardized AC-OPF formulation and an open benchmark library whose cases show larger and more varied optimality gaps than the legacy MATPOWER cases.","lead":"This report from an IEEE task force introduces PGLib-OPF, a standardized, open library of power network test cases for comparing AC optimal power flow algorithms. It fills missing data in older test networks with statistical models and shows that many new cases yield large optimality gaps, making algorithm differences visible.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"TL-UB thermal limits (Eq. 5) used in PEGASE/RTE cases are non-binding upper bounds, not ratings; reported significant gaps may thus reflect absent thermal congestion rather than benchmark-relevant hardness. A validation with original partial limits should settle it.","rationale":"The reader's weakest assumption is statistical representativeness of the data-completion models. I agree partially, but the sharper issue is TL-UB: unlike TL-Stat/AG-Stat/AC-Stat, TL-UB is not a statistical model at all; it is a theoretical upper bound. Applying it to large networks with partially known ratings means those cases are not stress-testing thermal congestion. This is load-bearing because the validation's evidence for 'useful' consists mostly of large optimality gaps, and for several large RTE/PEGASE cases those gaps appear precisely where TL-UB was used. The proposed check would determine whether replacing TL-UB with the available partial data collapses the gaps. This does not invalidate the library as a whole: many cases (e.g., API variants, small cases) may still be useful, and the paper is transparent about synthetic data. But the central claim as stated in Section VII is broader than the evidence, so I would condition acceptance on this re-validation. Credit is due for the open-source release, the standardized formulation, and the clearly documented construction pipeline; those are real strengths.","tokens_in":26624,"tokens_out":10047,"duration_ms":115119,"concrete_test":"Re-run the Section VI validation for the RTE and PEGASE cases (at minimum 6495_rte, 6515_rte, 9241_pegase, 13659_pegase) using the partial original thermal limits from [42] where available and TL-Stat for the missing branch ratings, keeping all other PGLib data fixed; then repeat for the API and SAD variants. If the optimality gaps drop below the 1% threshold used to define 'significant' in these cases, the reported gaps are artifacts of non-binding TL-UB ratings and the headline claim needs qualification. As a secondary check, tabulate how many branch thermal limits are binding under the original and TL-UB completions.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim ('majority of PGLib-OPF networks exhibit significant optimality gaps and are therefore useful for benchmarking') requires the completed datasets to contain realistic thermal limits. Section V.B.2 defines TL-UB in Eq. (5) as the maximum apparent power flow compatible with the voltage and angle bounds, i.e., an upper bound, not an estimate of a line's actual rating. Table V applies TL-UB to the PEGASE, RTE, IEEE 300, and PSERC cases, precisely the large networks where the validation reports some of the largest gaps (e.g., pglib_opf_case6495_rte TYP 15.11%, pglib_opf_case6515_rte TYP 6.40%). Since TL-UB is constructed so that no feasible voltage/angle point can exceed it, those branch ratings are effectively non-binding: the cases lack thermal congestion, a primary driver of real-world AC-OPF hardness. The large reported gaps may then be artifacts of the AG-Stat/AC-Stat completions, of IPOPT local solutions, or of the SOC bound, rather than of benchmark-relevant congestion. The paper's own conclusion acknowledges the data are 'by-in-large synthetically generated,' but the validation study does not test whether the TL-UB-based branch ratings are responsible for the headline gaps.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This IEEE PES Task Force report introduces PGLib-OPF, a curated, open-access library of AC optimal power flow (AC-OPF) benchmark instances in MATPOWER format, together with a standardized AC-OPF formulation (Model 1). The paper motivates the library by showing that classic MATPOWER cases mostly have optimality gaps below 1% (Section III, Table I), surveys publicly available transmission datasets and their missing parameters (Section IV, Table II), and describes statistical and arithmetic models for completing generator limits, costs, and branch thermal limits (Section V). It then constructs PGLib-OPF cases in three variants -- Typical Operating Conditions (TYP), Active Power Increase (API), and Small Angle Difference (SAD) -- and reports optimality gaps between IPOPT local solutions and a second-order cone relaxation (Section VI, Tables VI--VIII). The paper concludes that the majority of PGLib-OPF networks exhibit significant optimality gaps and are therefore useful for benchmarking AC-OPF algorithms.","tokens_in":26932,"tokens_out":5930,"duration_ms":64936,"significance":"If the claims hold, PGLib-OPF is a valuable community resource: it provides a common, standardized testbed, openly licensed data, reproducible software tooling, and a careful survey of missing data in existing test cases. The paper also ships concrete numerical validation tables and explicitly acknowledges the synthetic nature of much of the data. However, the strength of the central benchmarking claim is limited by the validation methodology: the evidence is based on a single solver pair (IPOPT local solutions versus an SOC relaxation), and the thermal-limit completion used for several large networks is an upper bound rather than a realistic rating. These issues do not negate the value of the library, but they require revision before the headline claims can be accepted as stated.","major_comments":[{"comment":"The thermal-limit model TL-UB in Eq. (5) is not a thermal rating: it is the maximum apparent power flow magnitude compatible with the voltage magnitude and angle difference bounds in Model 1. As a result, any point satisfying constraints (2d) and (2i) automatically satisfies the branch limit (2h), so the TL-UB limits are redundant and cannot create thermal congestion. Table V applies TL-UB to the PEGASE, RTE, IEEE 300, PSERC, and GOC 179 networks, and Table VI reports large optimality gaps in some of these cases, e.g., pglib_opf_case6495_rte TYP at 15.11% and pglib_opf_case6515_rte TYP at 6.40%. Those gaps therefore cannot be interpreted as evidence of congestion-induced hardness, and the Section VII statement that all PGLib-OPF networks have reasonable branch thermal limits is not supported for the TL-UB cases. Please re-run the validation on these networks using TL-Stat or the original partial thermal limits, or explicitly characterize the TL-UB cases as having non-binding thermal limits and adjust the associated claims accordingly.","section":"V.B.2 / Eq. (5) and Table V"},{"comment":"The central claim that the PGLib-OPF networks are useful for benchmarking AC-OPF algorithms rests on optimality gaps between one IPOPT local solution and one SOC relaxation. As the paper itself notes, a large gap can be caused by heuristic failure, a weak relaxation, or both; the current experiments do not distinguish these possibilities. More importantly, benchmarking usefulness requires that different algorithms can be distinguished and ranked, which a single solver pair cannot demonstrate. Please add at least one independent solver or a small multi-algorithm comparison on a representative subset of cases to show that the gaps translate into meaningful algorithm differentiation. Without this, the phrase 'useful for benchmarking' is supported only indirectly.","section":"VI.A and Tables VI-VIII"},{"comment":"The data-completion models AG-Stat, AC-Stat, and TL-Stat are stochastic and are taken from NESTA [52], and the paper does not report random seeds or a sensitivity analysis for the particular realization used in PGLib-OPF. Since Section VII explicitly acknowledges that the network data are 'by-in-large synthetically generated,' the representativeness of the completed cases is an untested assumption. A small number of independent completions, or a report of the seeds used, would clarify whether the reported optimality gaps are a stable property of the benchmark family or an artifact of a single draw. This is load-bearing because the benchmarking claim presupposes that the synthetic parameters are realistic enough for algorithm comparisons to be meaningful.","section":"V.A and Tables III-V"}],"minor_comments":[{"comment":"The phrase 'all the of network data' should read 'all of the network data.'","section":"Abstract"},{"comment":"The phrase 'by-in-large' should read 'by and large'; there is also a grammatical slip in 'cases that where originally designed' (Section IV.A).","section":"III"},{"comment":"The sentence 'This suggest that many of these cases will be useful' contains a subject-verb agreement error and should be corrected.","section":"VI.A"},{"comment":"The table lists PEGASE and RTE thermal limits as 'partial,' but Table V shows TL-UB is applied to almost all of those cases; a brief note explaining how 'partial' original data relates to the TL-UB completion would improve clarity.","section":"Table II"},{"comment":"The statement that a 30-degree angle difference bound is 'subsumed by the thermal limits provided with all of the networks considered here' is trivially true for TL-UB cases by construction; please clarify whether it is also asserted for TL-Stat cases.","section":"V.C"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a task force report and the PGLib-OPF library itself is likely to be a widely used community resource. The main concerns are not about the curation effort or the open-data contribution, but about the evidential weight of the validation study for the stated benchmarking claim. The TL-UB issue, in particular, should be addressed directly rather than softened, because several of the headline large gaps occur in cases where the thermal limits are non-binding by construction. The heavy reliance on NESTA, from the same research group, is not improper but deserves explicit provenance and ideally an independent check."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"PGLib-OPF is a genuinely useful community resource, but the validation study has a specific blind spot around TL-UB thermal limits that should be fixed before this becomes the definitive AC-OPF benchmark.\n\nThe curated library itself is the real contribution. A standardized Model 1, over a hundred cases with all needed parameters, and the API/SAD stress-case construction are exactly what the community lacks. The data and code are open, and the paper is honest about the synthetic nature of the completions. The motivating study, showing that the standard MATPOWER cases produce tiny gaps, makes a clear case that new data is needed.\n\nThe stress-test note lands. Equation (5) defines TL-UB as the maximum apparent power flow compatible with the voltage magnitude and angle bounds. That means the thermal limits for the PEGASE, RTE, IEEE 300, and PSERC cases are non-binding by construction; those cases cannot have thermal congestion. So the reported gaps there—like the 15% for case6495_rte—cannot come from congestion. They might come from generator cost curves, reactive power limits, or from the SOC relaxation being loose. That still makes the cases useful for benchmarking, but the paper's central sentence, 'significant optimality gaps and therefore useful for benchmarking,' implies the gaps indicate hardness. Without knowing which constraints bind, that conclusion is premature. The fix is easy: report the active constraints at the reported AC solutions, or re-run the validation with the original partial thermal limits where those exist.\n\nTwo smaller issues. First, the validation uses only IPOPT as the AC heuristic and only the SOC relaxation. That is a single solver pair, so it is unclear whether the gaps reflect problem difficulty or IPOPT's local behavior. Adding a second solver would strengthen the claim. Second, the conclusion says the 'majority' of networks exhibit significant gaps, but in the TYP table only a handful are above 1%. The claim holds only if you include the API and SAD variants. That is a wording issue, not a deep flaw.\n\nReliance on NESTA is not a real problem; the models are described and the data are open. Overall, this paper deserves a serious referee and will likely become a standard citation, but it needs a revision that either fixes the TL-UB cases or clearly documents that they lack thermal congestion and therefore measure something else.","headline":"A genuinely useful AC-OPF benchmark library, but the validation overstates what the TL-UB cases show and needs a revision to document which constraints actually bind.","tokens_in":27562,"tokens_out":3956,"would_cite":true,"duration_ms":44069,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["90C26","90C30"],"pacs":[],"model":"deepseek-v4-flash","headline":"A curated library turns old grid data into hard AC-OPF benchmarks that expose algorithm differences.","keywords":["AC optimal power flow","benchmarking","optimality gap","convex relaxation","power grid test cases","synthetic network data","generator cost models","branch thermal limits"],"falsifier":"Take a real network with complete, verified operational data (for example a utility's actual costs and line ratings) and run both a strong AC heuristic and the SOC relaxation on it. Then re-run using PGLib's statistical completion on the same topology. If the optimality gap and the ordering of two reference solvers differ wildly between the real and synthetic data, the benchmarking transferability claim would collapse.","tokens_in":26451,"feed_emoji":"⚡","tokens_out":5185,"duration_ms":56856,"temperature":0.7,"pith_summary":"This paper aims to fix a reproducibility problem in AC optimal power flow research: studies differ in problem formulation and in data, so algorithms cannot be compared fairly. It proposes a standardized AC-OPF model and a curated, open-access library of networks, PGLib-OPF, in which missing generator costs, injection limits, and branch thermal limits are completed by statistical models. The central claim, backed by a validation study, is that these benchmark networks show much larger optimality gaps than the older standard test cases, making them discriminating tests for different solution methods.","feed_headline":"Power-grid test library gives AC-OPF solvers real stress tests","feed_subtitle":"Congestion and angle-tight variants push optimality gaps above 10 percent, separating solver strengths.","key_machinery":"The load-bearing object is PGLib-OPF itself: a set of creative-commons network cases in a common data format, paired with a single nominated AC-OPF model (Model 1) that includes nodal power balance, Ohm's-law branch flows, thermal limits, and voltage angle difference limits. The argument runs on data-completion models for generators (GF-Stat, AG-Stat, AC-Stat) and branches (TL-Stat, TL-UB), plus two stress-test variants: API, which raises load until thermal limits bind, and SAD, which shrinks angle limits until they bind. The validation metric is the optimality gap, (AC heuristic objective - SOC relaxation bound) / AC heuristic objective.","core_discovery":"The paper demonstrates that the majority of the PGLib-OPF networks exhibit significantly larger optimality gaps than traditional MATPOWER case studies, and hence are useful for benchmarking AC-OPF algorithms. The validation study quantifies this with an optimality gap defined as the relative difference between a local-nonlinear AC feasible solution and a Second-Order Cone relaxation bound. The larger gaps arise both from deliberately congested cases (API) and from cases with tightly constrained voltage angle differences (SAD), which provide a wider variety of difficulty for algorithm testing.","pith_inferences":["The same data-completion recipe could generate benchmarks for other grid problems, such as unit commitment or security-constrained OPF, by adding the extra data tables the appendix lists.","The API construction is effectively a standardized stress test: a community-standard 'congestion level' could be defined by reporting how close to the thermal limit the load ramping stops.","Because the statistical models are drawn randomly, sampling many PGLib instances would let the community report algorithm performance distributions, not single-point gaps.","The large gaps in SAD variants suggest that angle-difference limits, not just thermal limits, are a cheap knob for creating hard instances; this could be exploited to generate custom difficulty levels."],"forward_implications":["Different AC-OPF studies become directly comparable because the formulation and data are fixed and shared.","Researchers can select cases by gap size, separating the question 'can this heuristic find good feasible points?' from 'is this relaxation bound tight?'","The API and SAD variants provide systematic, repeatable ways to probe how an algorithm degrades under thermal congestion and angle congestion, respectively.","The creative-commons license allows results to be checked and extended without data-use barriers.","The two infeasible legacy cases (case9target and case145) are flagged as data-quality issues rather than solver failures, correcting potential mis-benchmarks."],"supporting_citations":[{"why":"Provides the data-driven models (fuel category, generator capacity and cost, thermal limit statistics) that complete missing parameters in the library.","marker":"[52]"},{"why":"Supplies the network data format and the collection of base test cases that PGLib-OPF converts into benchmark instances.","marker":"[16]"},{"why":"Defines the Second-Order Cone relaxation whose bound is used to compute the optimality gaps reported in the validation study.","marker":"[3]"},{"why":"Supplies the RTE snapshot and PEGASE networks, including the large-scale cases that become PGLib instances.","marker":"[42]"},{"why":"Supplies the Grid Optimization Competition networks with scenarios used as base cases for many PGLib instances.","marker":"[45]"},{"why":"Provides the detailed AC-OPF model derivation that Model 1 follows, including branch-flow equations and convex envelopes.","marker":"[5]"},{"why":"Formulates both the AC model and the SOC relaxation in the computational experiments.","marker":"[51]"}],"fun_headline_variants":["PGLib-OPF: larger optimality gaps for benchmarking AC-OPF","Congested and angle-tight grids reveal AC-OPF solver limits","New benchmark library separates AC-OPF solvers by optimality gap","PGLib-OPF: harder AC-OPF cases yield optimality gaps over 10%","AC-OPF benchmark library uses congestion and angle constraints"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The statistical models for missing costs, generator limits, and line ratings produce numbers that are realistic enough that a solver's ranking on these synthetic cases matches its ranking on real grids.","fun_headline_variants_meta":{"raw":{"variants":["PGLib-OPF: larger optimality gaps for benchmarking AC-OPF","Congested and angle-tight grids reveal AC-OPF solver limits","New benchmark library separates AC-OPF solvers by optimality gap","PGLib-OPF: harder AC-OPF cases yield optimality gaps over 10%","AC-OPF benchmark library uses congestion and angle constraints"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000903,"raw_usage":{"total_tokens":3830,"prompt_tokens":833,"completion_tokens":2997,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":449,"completion_tokens_details":{"reasoning_tokens":2894}},"tokens_in":449,"tokens_out":2997,"duration_ms":20478,"temperature":1.0,"reasoning_tokens":2894,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T14:34:00.834842+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a real network with complete, verified operational data (for example a utility's actual costs and line ratings) and run both a strong AC heuristic and the SOC relaxation on it. Then re-run using PGLib's statistical completion on the same topology. If the optimality gap and the ordering of two reference solvers differ wildly between the real and synthetic data, the benchmarking transferability claim would collapse.","supporting_citations":[{"cited_title":"Grid optimization competition datasets,","cited_arxiv_id":null,"evidence_quote":"Supplies the Grid Optimization Competition networks with scenarios used as base cases for many PGLib instances."},{"cited_title":"Powermodels.jl: An open-source framework for exploring power flow formulations,","cited_arxiv_id":null,"evidence_quote":"Formulates both the AC model and the SOC relaxation in the computational experiments."}],"review_version":1}