A Shallow Recurrent Decoder for Dynamic State Estimation with a Limited Number of PMUs in Power Systems
Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel 2026-07-02 17:39 UTCgrok-4.3pith:2OA6H6YBrecord.jsonopen to challenge →
The pith
The SHRED decoder reconstructs the full state of a power system from sparse PMU measurements without relying on an accurate physical model.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The SHRED architecture reconstructs the complete dynamic state of a power system from sparse PMU measurements. Unlike model-based methods, it does not require an accurate physical model and maintains performance under strongly nonlinear conditions such as short-circuit disturbances on the IEEE 39-bus system. It consistently outperforms a state-of-the-art shallow decoder benchmark in sparse-measurement scenarios and remains insensitive to PMU placement while showing robustness to measurement noise.
What carries the argument
The SHallow REcurrent Decoder (SHRED) architecture, a machine learning model that uses recurrent layers to map sparse measurements to the full system state vector.
If this is right
- Full state reconstruction is possible with fewer PMUs than traditionally required.
- The method works without an accurate physical model of the power system.
- Performance holds under strongly nonlinear operating conditions like short circuits.
- Accuracy is largely independent of the specific locations of the PMUs.
- High reconstruction accuracy persists even with added measurement noise.
Where Pith is reading between the lines
- SHRED could lower the cost of wide area measurement systems by requiring fewer sensors.
- Similar architectures might apply to state estimation in other complex dynamical systems with limited sensors.
- Further tests on larger or real-world grids would confirm scalability beyond the IEEE 39-bus case.
Load-bearing premise
Training data from simulations sufficiently prepares the decoder to handle real-world nonlinear disturbances without needing the underlying physical equations.
What would settle it
If reconstruction error on the IEEE 39-bus system during short-circuit faults with three PMUs exceeds the error of the benchmark shallow decoder, the performance advantage claim would fail.
Figures
read the original abstract
Dynamic State Estimation (DSE) will play a fundamental role in future power system operation by providing real-time estimates of the system state and enabling enhanced situational awareness. Existing DSE approaches are primarily based on Kalman filter variants or Machine Learning (ML) techniques. However, Kalman-based methods often suffer from high computational complexity, sensitivity to model inaccuracies, and performance degradation under strongly nonlinear operating conditions. Moreover, their effectiveness critically depends on the number and placement of measurements, since suboptimal PMU locations can reduce observability and even render state estimation infeasible. Machine learning approaches alleviate some of these limitations but typically require large amounts of training data and may struggle to generalize. To address these challenges, this paper proposes a SHallow REcurrent Decoder (SHRED) architecture for full-state reconstruction of power systems from sparse measurements. Unlike conventional model-based estimators, the proposed approach does not rely on an accurate physical model and is largely insensitive to PMU placement, making it particularly attractive for practical deployment in existing Wide Area Measurement Systems (WAMS). The method is validated on the IEEE 39-bus system under strongly nonlinear conditions, including short-circuit disturbances. The results demonstrate that SHRED can accurately reconstruct the complete system state using only a limited number of PMU measurements, consistently outperforming a state-of-the-art shallow decoder benchmark in sparse-measurement scenarios. Furthermore, the proposed framework exhibits strong robustness to measurement noise and maintains high reconstruction accuracy even under severe disturbances, highlighting its potential as a scalable and reliable alternative to conventional DSE techniques.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a SHallow REcurrent Decoder (SHRED) architecture for dynamic state estimation (DSE) from sparse PMU measurements in power systems. It claims that SHRED reconstructs the full system state without relying on an accurate physical model at inference time, is largely insensitive to PMU placement, outperforms a state-of-the-art shallow decoder benchmark in sparse-measurement scenarios, and maintains high accuracy under short-circuit disturbances and measurement noise on the IEEE 39-bus system.
Significance. If the generalization and robustness claims are substantiated with held-out test conditions, the approach could offer a practical data-driven alternative to Kalman-filter-based DSE methods that degrade under strong nonlinearities or suboptimal sensor placement.
major comments (1)
- [Experimental results section] Experimental results section: the manuscript provides no information on whether the short-circuit fault locations, clearing times, load/generation profiles, or operating conditions in the reported test cases were strictly excluded from the training trajectories. This detail is load-bearing for the central claim that performance reflects generalization to unseen strongly nonlinear conditions rather than interpolation within the training distribution.
minor comments (1)
- [Abstract] Abstract: the claims of consistent outperformance and robustness are stated without any quantitative metrics, error bars, or specific numerical results, which reduces the ability to assess the strength of the evidence from the abstract alone.
Simulated Author's Rebuttal
We thank the referee for the constructive comment on the experimental design. We address the point below and will revise the manuscript to strengthen the presentation of the generalization claims.
read point-by-point responses
-
Referee: Experimental results section: the manuscript provides no information on whether the short-circuit fault locations, clearing times, load/generation profiles, or operating conditions in the reported test cases were strictly excluded from the training trajectories. This detail is load-bearing for the central claim that performance reflects generalization to unseen strongly nonlinear conditions rather than interpolation within the training distribution.
Authors: We agree that explicit confirmation of held-out test conditions is necessary to support the generalization claims. The manuscript does not currently provide this information. In the revised manuscript we will expand the Experimental Results section to state that training trajectories were generated exclusively from normal operating conditions (varying load/generation profiles without short-circuit events), while all reported test cases use short-circuit fault locations, clearing times, and operating conditions that were strictly excluded from the training set. This clarification will be added with a brief description of the data-generation protocol to demonstrate that the reported performance reflects generalization to unseen nonlinear disturbances. revision: yes
Circularity Check
No circularity: empirical ML validation independent of fitted inputs or self-citations
full rationale
The paper introduces the SHRED architecture for DSE from sparse PMUs and validates it empirically on the IEEE 39-bus system under short-circuit disturbances, claiming superior reconstruction without reliance on an accurate physical model. No equations, parameter-fitting procedures, or derivation steps are presented that reduce outputs to inputs by construction. No self-citations, uniqueness theorems, or ansatzes are invoked in the provided text to support central claims. The performance claims rest on held-out testing rather than tautological re-labeling of training data or model fits, making the derivation self-contained against external benchmarks.
Axiom & Free-Parameter Ledger
Reference graph
Works this paper leans on
-
[1]
The renewable energy role in the global energy transformations,
Q. Hassanet al., “The renewable energy role in the global energy transformations,”Renewable Energy F ocus, vol. 48, p. 100545, 2024
work page 2024
-
[2]
P. Kunduret al., “Definition and classification of power system stability IEEE/CIGRE joint task force on stability terms and definitions,”IEEE transactions on Power Systems, vol. 19, no. 3, pp. 1387–1401, 2004
work page 2004
-
[3]
Definition and classification of power system stability–revisited & extended,
N. Hatziargyriouet al., “Definition and classification of power system stability–revisited & extended,”IEEE Transactions on Power Systems, vol. 36, no. 4, pp. 3271–3281, 2020
work page 2020
-
[4]
Final report on the grid incident in spain and portugal on 28 april 2025,
ENTSO-E, “Final report on the grid incident in spain and portugal on 28 april 2025,” European Network of Transmission System Operators for Electricity (ENTSO-E), Tech. Rep., Nov. 2025
work page 2025
-
[5]
Countdown to Collapse: Stability Dynamics Ob- served Before the 2025 Iberian Power System Blackout,
A. Pomaricoet al., “Countdown to Collapse: Stability Dynamics Ob- served Before the 2025 Iberian Power System Blackout,” in2025 AEIT International Annual Conference (AEIT), 2025, pp. 1–6
work page 2025
-
[6]
R. Gonz ´alez-Pozo, “Social profiles and response patterns during the 2025 Iberian Peninsula power outage. The case of Spain,”International Journal of Disaster Risk Reduction, p. 105813, 2025. JOURNAL OF XXX 10
work page 2025
- [7]
-
[8]
Power system dynamic state estimation: Motivations, definitions, methodologies, and future work,
J. Zhaoet al., “Power system dynamic state estimation: Motivations, definitions, methodologies, and future work,”IEEE Transactions on Power Systems, vol. 34, no. 4, pp. 3188–3198, 2019
work page 2019
-
[9]
Roles of dynamic state estimation in power system model- ing, monitoring and operation,
——, “Roles of dynamic state estimation in power system model- ing, monitoring and operation,”IEEE Transactions on Power Systems, vol. 36, no. 3, pp. 2462–2472, 2020
work page 2020
-
[10]
Dynamical systems on networks,
M. A. Porter and J. P. Gleeson, “Dynamical systems on networks,” Frontiers in Applied Dynamical Systems: Reviews and Tutorials, vol. 4, p. 29, 2016
work page 2016
-
[11]
Passive mode-locking by use of waveguide arrays,
J. L. Proctor and J. N. Kutz, “Passive mode-locking by use of waveguide arrays,”Optics letters, vol. 30, no. 15, pp. 2013–2015, 2005
work page 2013
-
[12]
Realistic control of network dynamics,
S. P. Cornelius, W. L. Kath, and A. E. Motter, “Realistic control of network dynamics,”Nature communications, vol. 4, no. 1, p. 1942, 2013
work page 1942
-
[13]
Nonlinear control of networked dynamical systems,
M. Morrison and J. N. Kutz, “Nonlinear control of networked dynamical systems,”IEEE transactions on network science and engineering, vol. 8, no. 1, pp. 174–189, 2020
work page 2020
-
[14]
A review of machine learning approaches to power system security and stability,
O. A. Alimi, K. Ouahada, and A. M. Abu-Mahfouz, “A review of machine learning approaches to power system security and stability,” IEEE access, vol. 8, pp. 113 512–113 531, 2020
work page 2020
-
[15]
S. L. Brunton and J. N. Kutz,Data-driven science and engineering: Ma- chine learning, dynamical systems, and control. Cambridge University Press, 2022
work page 2022
-
[16]
Sensing with shallow recurrent decoder networks,
J. P. Williams, O. Zahn, and J. N. Kutz, “Sensing with shallow recurrent decoder networks,”Proceedings of the Royal Society A, vol. 480, no. 2298, p. 20240054, 2024
work page 2024
-
[17]
M. L. Gao, J. P. Williams, and J. N. Kutz, “Sparse identification of nonlinear dynamics and koopman operators with shallow recurrent decoder networks,”Proceedings of the National Academy of Sciences, vol. 123, no. 16, p. e2508144123, 2026
work page 2026
-
[18]
Power system static-state estimation with uncertain network parameters as input data,
G. D’Antona, “Power system static-state estimation with uncertain network parameters as input data,”IEEE transactions on Instrumentation and Measurement, vol. 65, no. 11, pp. 2485–2494, 2016
work page 2016
-
[19]
A review on distribution system state estimation,
A. Primadianto and C.-N. Lu, “A review on distribution system state estimation,”IEEE Transactions on Power Systems, vol. 32, no. 5, pp. 3875–3883, 2016
work page 2016
-
[20]
H. Liet al., “An Integrated Online Dynamic Security Assessment System for Improved Situational Awareness and Economic Operation,” IEEE Access, vol. 7, pp. 162 571–162 582, 2019
work page 2019
-
[21]
E. Farantatoset al., “A predictive generator out-of-step protection and transient stability monitoring scheme enabled by a distributed dynamic state estimator,”IEEE Transactions on Power Delivery, vol. 31, no. 4, pp. 1826–1835, 2015
work page 2015
-
[22]
Dynamic State Estimation Assisted Out-of-Step Detection for Generators Using Angular Differ- ence,
Y . Cui, R. G. Kavasseri, and S. M. Brahma, “Dynamic State Estimation Assisted Out-of-Step Detection for Generators Using Angular Differ- ence,”IEEE Transactions on Power Delivery, vol. 32, no. 3, pp. 1441– 1449, 2017
work page 2017
-
[23]
Feasibility studies of apply- ing Kalman filter techniques to power system dynamic state estimation,
Z. Huang, K. Schneider, and J. Nieplocha, “Feasibility studies of apply- ing Kalman filter techniques to power system dynamic state estimation,” in2007 International Power Engineering Conference (IPEC 2007). IEEE, 2007, pp. 376–382
work page 2007
-
[24]
A review of power system dynamic state estimation techniques,
N. Shivakumar and A. Jain, “A review of power system dynamic state estimation techniques,” in2008 Joint International Conference on Power System Technology and IEEE Power India Conference. IEEE, 2008, pp. 1–6
work page 2008
-
[25]
An introduction to the kalman filter,
G. Welch, G. Bishopet al., “An introduction to the kalman filter,” 1995
work page 1995
-
[26]
Kalman filter and its application,
Q. Liet al., “Kalman filter and its application,” in2015 8th international conference on intelligent networks and intelligent systems (ICINIS). IEEE, 2015, pp. 74–77
work page 2015
-
[27]
S. Akhlaghi, N. Zhou, and Z. Huang, “A multi-step adaptive interpo- lation approach to mitigating the impact of nonlinearity on dynamic state estimation,”IEEE Transactions on Smart Grid, vol. 9, no. 4, pp. 3102–3111, 2016
work page 2016
-
[28]
A robust iterated extended kalman filter for power system dynamic state estimation,
J. Zhao, M. Netto, and L. Mili, “A robust iterated extended kalman filter for power system dynamic state estimation,”IEEE transactions on power systems, vol. 32, no. 4, pp. 3205–3216, 2016
work page 2016
-
[29]
An alternative method for power system dynamic state estimation based on unscented transform,
S. Wang, W. Gao, and A. S. Meliopoulos, “An alternative method for power system dynamic state estimation based on unscented transform,” IEEE transactions on power systems, vol. 27, no. 2, pp. 942–950, 2011
work page 2011
-
[30]
Unscented kalman filter for power system dynamic state estimation,
G. Valverde and V . Terzija, “Unscented kalman filter for power system dynamic state estimation,”IET generation, transmission & distribution, vol. 5, no. 1, pp. 29–37, 2011
work page 2011
-
[31]
J. Qiet al., “Dynamic state estimation for multi-machine power system by unscented kalman filter with enhanced numerical stability,”IEEE Transactions on Smart Grid, vol. 9, no. 2, pp. 1184–1196, 2016
work page 2016
-
[32]
J. Zhao and L. Mili, “Robust unscented kalman filter for power system dynamic state estimation with unknown noise statistics,”IEEE Transac- tions on Smart Grid, vol. 10, no. 2, pp. 1215–1224, 2017
work page 2017
-
[33]
D. Houet al., “Dynamic state estimation of power systems with uncertainties based on robust adaptive unscented kalman filter,”Journal of Modern Power Systems and Clean Energy, vol. 11, no. 4, pp. 1065– 1074, 2022
work page 2022
-
[34]
Dynamic state estimation with model uncertainties using h- infinity extended kalman filter,
J. Zhao, “Dynamic state estimation with model uncertainties using h- infinity extended kalman filter,”IEEE Transactions on power systems, vol. 33, no. 1, pp. 1099–1100, 2017
work page 2017
-
[35]
Assessing Gaussian assumption of PMU measurement error using field data,
S. Wanget al., “Assessing Gaussian assumption of PMU measurement error using field data,”IEEE Transactions on Power Delivery, vol. 33, no. 6, pp. 3233–3236, 2017
work page 2017
-
[36]
R. Machlevet al., “Explainable Artificial Intelligence (XAI) techniques for energy and power systems: Review, challenges and opportunities,” Energy and AI, vol. 9, p. 100169, 2022
work page 2022
-
[37]
Artificial intelligence-based methods for renewable power system operation,
Y . Liet al., “Artificial intelligence-based methods for renewable power system operation,”Nature Reviews Electrical Engineering, vol. 1, no. 3, pp. 163–179, 2024
work page 2024
-
[38]
Applications of Physics-Informed Neural Networks in Power Systems - A Review,
B. Huang and J. Wang, “Applications of Physics-Informed Neural Networks in Power Systems - A Review,”IEEE Transactions on Power Systems, vol. 38, no. 1, pp. 572–588, 2023
work page 2023
-
[39]
Physics-informed graphical neural network for power system state estimation,
Q.-H. Ngoet al., “Physics-informed graphical neural network for power system state estimation,”Applied Energy, vol. 358, p. 122602, 2024
work page 2024
-
[40]
Physics-Informed Neural Networks for Accelerating Power System State Estimation,
S. Falaset al., “Physics-Informed Neural Networks for Accelerating Power System State Estimation,” in2023 IEEE PES Innovative Smart Grid Technologies Europe (ISGT EUROPE), 2023, pp. 1–5
work page 2023
-
[41]
W. Yi and S. Bu, “Physics-informed dynamic mode decomposition for power systems dynamic state estimation with uncertainties,” in2024 18th International Conference on Probabilistic Methods Applied to Power Systems (PMAPS), 2024, pp. 1–6
work page 2024
-
[42]
A Hybrid-Learning Algorithm for Online Dynamic State Estimation in Multimachine Power Systems,
G. Tianet al., “A Hybrid-Learning Algorithm for Online Dynamic State Estimation in Multimachine Power Systems,”IEEE Transactions on Neural Networks and Learning Systems, vol. 31, no. 12, pp. 5497–5508, 2020
work page 2020
-
[43]
Shallow recurrent decoder for reduced order modeling of E×B plasma dynamics,
F. Faraji, M. Reza, and J. N. Kutz, “Shallow recurrent decoder for reduced order modeling of E×B plasma dynamics,”Machine Learning: Science and Technology, vol. 6, no. 2, p. 025024, 2025
work page 2025
-
[44]
S. Rivaet al., “Robust state estimation from partial out-core measure- ments with shallow recurrent decoder for nuclear reactors,”Progress in Nuclear Energy, vol. 189, p. 105928, 2025
work page 2025
-
[45]
——, “Towards Efficient Parametric State Estimation in Circulating Fuel Reactors with Shallow Recurrent Decoder Networks,”arXiv preprint arXiv:2503.08904, 2025
-
[46]
Shallow Recurrent Decoders for Neural and Behavioral Dynamics,
A. S. Rude and J. N. Kutz, “Shallow Recurrent Decoders for Neural and Behavioral Dynamics,”bioRxiv, pp. 2025–06, 2025
work page 2025
-
[47]
Reduced order modeling with shallow recurrent decoder networks,
M. Tomasettoet al., “Reduced order modeling with shallow recurrent decoder networks,”Nature Communications, vol. 16, no. 1, p. 10260, 2025
work page 2025
-
[48]
Discovering governing equations from data by sparse identification of nonlinear dynamical systems,
S. L. Brunton, J. L. Proctor, and J. N. Kutz, “Discovering governing equations from data by sparse identification of nonlinear dynamical systems,”Proceedings of the national academy of sciences, vol. 113, no. 15, pp. 3932–3937, 2016
work page 2016
-
[49]
Modern koopman theory for dynamical systems.arXiv preprint arXiv:2102.12086, 2021
S. L. Bruntonet al., “Modern koopman theory for dynamical systems,” arXiv preprint arXiv:2102.12086, 2021
-
[50]
Data assimilation and discrepancy modeling with shallow recurrent decoders,
Y . Bao and J. N. Kutz, “Data assimilation and discrepancy modeling with shallow recurrent decoders,”arXiv preprint arXiv:2512.01170, 2025
-
[51]
A. Graves, “Long short-term memory,”Supervised sequence labelling with recurrent neural networks, pp. 37–45, 2012
work page 2012
-
[52]
Shallow neural networks for fluid flow reconstruction with limited sensors,
N. B. o. Erichson, “Shallow neural networks for fluid flow reconstruction with limited sensors,”Proceedings of the Royal Society A, vol. 476, no. 2238, p. 20200097, 2020
work page 2020
-
[53]
Detecting strange attractors in turbulence,
F. Takens, “Detecting strange attractors in turbulence,” inDynamical Systems and Turbulence, Warwick 1980: proceedings of a symposium held at the University of Warwick 1979/80. Springer, 2006, pp. 366– 381
work page 1980
-
[54]
F. M. Gonzalez-Longatt and J. L. Rueda,PowerFactory Applications for Power System Analysis. Springer, 2014
work page 2014
-
[55]
D. Yeet al., “PySHRED: A Python package for SHallow REcurrent Decoding for sparse sensing, model reduction and scientific discovery,” arXiv preprint arXiv:2507.20954, 2025
-
[56]
J. Qi, K. Sun, and W. Kang, “Optimal PMU placement for power system dynamic state estimation by using empirical observability Gramian,” IEEE Transactions on power Systems, vol. 30, no. 4, pp. 2041–2054, 2014
work page 2041
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.