Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

Deep Learning based Covert Attack Identification for Industrial Control Systems

T0 review · 4 major / 4 minor · reviewed 2026-08-27 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A hybrid autoencoder, LSTM, and deep classifier detects covert attacks on a simulated smart grid, separates them from faults, and names the attacked generator.

desk verdict Sensible pipeline and honest simulation, but the headline F1 numbers may be artifacts of a temporally leaky 80/20 split. read the letter →

arxiv 2009.12360 v1 pith:MY5LB7JN submitted 2020-09-25 cs.LG cs.CR

Dan Li , Paritosh Ramanan , Nagi Gebraeel , Kamran Paynabar This is my paper · ORCID
classification cs.LGcs.CR
keywords covertattackindustrialcontrolsystemssmartgridsecuritydeeplearningautoencoderLSTMdetectionlocalization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to establish that a purely data-driven pipeline can detect covert cyberattacks on an industrial control system, distinguish them from ordinary equipment faults, and localize the attacked generator. The pipeline first compresses sensor readings with an autoencoder, then learns how those compressed readings normally evolve over time with an LSTM-based recurrent network, and finally feeds prediction residuals together with raw measurements to a deep classifier. The testbed is a simulated IEEE 14-bus smart grid, where the reported F1 scores beat a physics-based state-estimation baseline and the same architecture without the autoencoder. If the claim holds, critical-infrastructure monitoring would no longer need an accurate physical model of the system, because the detector is trained only on normal-operation sensor data.

What carries the argument

The load-bearing mechanism is the prediction residual computed from a learned normal-behavior model. An autoencoder projects the 39 sensor measurements into a 13-dimensional code intended to reflect the system's state dimension, filtering out sensor noise; a stateful LSTM takes ten lags of that code and predicts the next encoded measurement; a decoder maps the prediction back to the 39-dimensional measurement space. The residual is the difference between observed and predicted measurements, and the deep classifier decides between normal operation, an attack on one of four generator buses, and a fault on one of those buses from the residual concatenated with the raw measurements. The attacker's simulated replacement hides the attack on the directly affected sensors, so the informative residual comes from neighboring, uncompromised sensors whose readings deviate from what the LSTM expects.

What would settle it

Run the same trained pipeline on an IEEE 14-bus covert-attack scenario in which the attacker replaces every sensor reading on every bus with simulated normal values instead of only the attacked generator's sensors. If detection F1 stays high, the residual mechanism is not the source of the reported performance; if accuracy collapses, the restricted-access assumption is confirmed as the load-bearing condition.

Watch

Extended reading notes

Core claim

The central claim is that the autoencoder-RNN-DNN combination detects covert attacks, separates them from equipment faults, and localizes the attacked generator, and that it outperforms both a model-based state-estimation detector and an RNN-only variant. In the attack model considered here, the attacker alters a generator's control input and then replaces the affected sensor readings with simulated normal values, so the directly manipulated measurements look healthy. The attack becomes visible only in sensors outside the attacked subsystem, which is why the paper explicitly assumes the attacker's sensor access is limited to that subsystem. In the IEEE 14-bus simulation, the method reports high F1 for nearly every class, with its largest advantage on the high-degree generator where the physics-based estimator is most easily fooled; the authors attribute the gain to the autoencoder filtering sensor noise before the temporal model is applied.

Load-bearing premise

The detector works only when at least some sensors that would reveal the attack remain outside the attacker's control; if the attacker can overwrite every sensor that shows the manipulation, the residuals disappear and the attack becomes invisible to this method.

Editorial extensions

If this is right

  • Model-free deployment: the detector is trained on normal-operation sensor data alone, so it can be applied to networked systems where no accurate physical model exists.
  • The autoencoder is doing real work: removing it degrades classification accuracy, meaning unsupervised feature compression is what lets the temporal model produce clean residuals.
  • Detection power scales with attack severity: lighter manipulation is harder to catch, so operators would need to tune sensitivity for low-severity threats.
  • Localization and diagnosis share one output: a single DNN labels the observation as normal, attack-at-bus, or fault-at-bus, giving operators both the type and the location of the anomaly.
  • Network topology matters: attacks on high-degree nodes are harder to expose because the attacker can cover more of the relevant sensors, so protection effort should concentrate there.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the method's guarantee is conditional on partial sensor access; if an attacker could overwrite every sensor that reveals the manipulation, the residual signal would vanish and the detector would fail, which is consistent with the paper's own Section III.A assumption.
  • Editorial inference: because the same architecture is model-free, it should transfer to other networked infrastructures such as water distribution or manufacturing lines; a cheap test would be to retrain on their normal sensor streams and inject simulated attacks.
  • Editorial inference: the paper trains on known attack levels, so an adversarial attacker who chooses a manipulation below the smallest training level could evade the classifier; the residuals themselves should be evaluated as a continuous anomaly score rather than only as inputs to a fixed-label classifier.
  • Editorial inference: the reported dependence on node degree suggests a graph-aware extension—feeding the network topology into the DNN—could improve localization when the attacker covers a large neighborhood.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Request a human review

A listed scientist reviews the paper for a fee and the review publishes here regardless of verdict. See the reviewers or get listed.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes a hybrid deep learning framework (autoencoder + LSTM RNN + DNN) for detecting, diagnosing, and localizing covert attacks in networked industrial control systems, with application to a simulated IEEE 14-bus smart grid. The autoencoder extracts a low-dimensional latent representation of sensor measurements, the LSTM predicts the next encoded observation from normal-operation data, and residual-plus-measurement features are classified by a DNN into normal, attack-at-bus, and fault-at-bus categories. The authors compare their method against a model-based state-estimation baseline (SE+DNN) and an RNN+DNN baseline without the autoencoder, reporting higher F1 scores for the proposed method and a generalization study across attack severity levels.

Significance. If the reported performance holds, the paper makes a useful contribution to ICS security: it addresses the distinction between cyberattacks and equipment faults, and it handles localization, which are both practical needs beyond standard anomaly detection. The paper's strengths include evaluation on a standard IEEE 14-bus testbed, a meaningful comparison with a model-based state-estimation baseline, and a generalization experiment over attack severities. The main weaknesses are that the central numerical claim rests on a single time-series split whose episode-disjointness is not established, the autoencoder code size is stated inconsistently, and the main results lack uncertainty quantification; these issues need to be addressed before the comparative performance claims can be fully trusted.

major comments (4)
  1. [Section V.B, Tables I and II] The reported precision/recall/F1 values are computed from a single split where the model is trained on '80% of data' and tested on the 'rest 20% of data', with no statement that the split is episode-disjoint. Because the LSTM is stateful and uses 10 lags, and because attack/fault series are time-indexed, time steps from the same physical attack or fault episode could appear on both sides of the split. In that case the DNN can memorize episode-specific residual patterns and the RNN can use training-side history from the same event, inflating the reported F1 scores and undermining the claim that the method detects and localizes unseen covert attacks. I request an episode-disjoint evaluation: assign entire simulated attack/fault/normal episodes (or contiguous blocks) wholly to train or test, and report the resulting metrics. The severity-generalization experiment in Fig. 7 does not resolve this concern because it varies attack levels but does not guarantee that the underlying episodes are disjoint.
  2. [Section V.B, autoencoder code size] The text states both 'we use n = 14 as the code size of the autoencoder' and then 'we use k-1 = 13 as the code size'. These are contradictory statements. The architecture actually implemented (14 or 13) matters for the claim that the autoencoder reduces dimension to the state dimension, and the inconsistency makes the experimental configuration ambiguous. Please specify the exact code size used in all reported experiments and justify the choice relative to the state-space dimension of the IEEE 14-bus model.
  3. [Tables I and II, Fig. 6] The main comparative results are reported as point estimates from a single training run, with no confidence intervals, standard deviations, or repeated-run statistics. Deep learning models are sensitive to random initialization and data subsampling, and the differences between the proposed method and the baselines (especially for 'Fault #6' and 'Attack #6', where the proposed method has lower precision but higher recall) may not be statistically stable. Please provide mean and variance over multiple independent runs, or otherwise quantify the uncertainty of the reported F1 scores.
  4. [Section III.A and Section V.A] The threat model assumes the attacker's sensor access is limited to the attacked subsystem, so residual information propagates through other sensors. This is a reasonable and clearly stated assumption, but the simulation only instantiates one attack type (generation-decrease at generator buses) and one fault type (similar generation decrease without sensor manipulation). The paper's broader claims of a 'generic' framework should be qualified accordingly, and the conclusion should explicitly state that detection under a stronger attacker who can compromise all informative sensors remains an open problem, consistent with the covert-attack literature. This is a scoping caveat rather than a fatal flaw, but it should be reflected in the title-level generality claims.
minor comments (4)
  1. [Conclusion] There is a typo: 'mode-based SE' should be 'model-based SE'.
  2. [Section IV] The description of the DNN architecture as '3 dense hidden layers with ReLU, Sigmoid, and Softmax activation functions in sequence' is unclear. Please specify which activation is used in which layer and whether the softmax is the output layer.
  3. [Section V.B] The metrics are computed over all time steps, which are autocorrelated and therefore not independent samples. Even with an episode-disjoint split, the effective number of independent test observations is much smaller than the raw time-step count; please discuss this or use episode-level summary statistics for evaluation.
  4. [Fig. 6] The figures appear to be screen captures with small fonts and overlapping lines; please regenerate them with larger legends and distinguishable line styles for clarity.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper reports an empirical comparison on held-out simulated data; no derivation reduces to its inputs.

full rationale

The paper's central claim is empirical: a hybrid AEN+RNN+DNN pipeline detects, distinguishes, and localizes simulated covert attacks on an IEEE 14-bus model, with performance compared against SE+DNN and RNN+DNN baselines. There is no derivation chain in which a predicted quantity is defined in terms of the same quantity or in which a fitted parameter is renamed as a prediction. The autoencoder and RNN are trained only on normal-operation data, the DNN is trained on an 80% split and tested on the remaining 20%, and a generalization experiment trains on subsets of attack levels and tests on held-out levels. The covert-attack model explicitly assumes the attacker cannot compensate for effects propagating to other subsystems; this is a stated threat-model assumption rather than a conclusion derived from itself. No load-bearing self-citation appears: references define covert attacks, LSTM, unit commitment, and power-flow tools, and none are by the present authors or used to forbid alternatives. The main caveats are experimental validity concerns—possible temporal overlap in the 80/20 split and the simulation-based threat model—which affect whether the reported F1 scores generalize, not whether the argument is circular. Under the review rules, leakage and external validity are correctness risks, not circularity, so the appropriate score is 0.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The paper rests on standard power system simulation tools, a specific threat model, and several heuristic architecture choices. No new physical entities are introduced. The main assumptions are the restricted attacker model and the sufficiency of the autoencoder+LSTM representation, both of which are empirically motivated but not proven.

free parameters (4)
  • Autoencoder code size = 13 (text also says 14)
    Chosen from the physical state dimension of the power system (k-1 = 13 for active power only); not fitted to data, but the inconsistency in the text is a minor issue.
  • LSTM lookback length = 10
    Chosen by hand to capture temporal dependencies; no sensitivity analysis is provided.
  • Attack severity levels = 20%, 40%, 60%, 80%, 100% decrease
    Simulated attack strengths selected by the authors to cover a range; the method's performance depends on these levels, and the generalization test shows sensitivity.
  • DNN hidden layer sizes = not specified
    The paper states 3 dense hidden layers with ReLU, Sigmoid, Softmax but does not give the number of units; these are free hyperparameters left to the implementer.
assumptions (6)
  • domain assumption Power flow equations solved by Matpower accurately represent the smart grid's steady-state behavior.
    Section V.A uses Matpower 7.0 to generate all sensor data from load profiles; if this model is inaccurate, the simulated data does not reflect real grid dynamics.
  • domain assumption The covert attacker has full knowledge of the attacked subsystem's dynamics and can simulate normal sensor measurements to replace the true ones.
    Section III.A generalizes Smith's covert attack to nonlinear systems under this assumption; the detection problem is only meaningful under this threat model.
  • domain assumption The attacker's sensor access is limited to the attacked subsystem.
    Section III.A states 'we assume the attacker's access to the sensors is limited to subsystem i.' The entire detection and localization framework relies on other sensors remaining truthful.
  • domain assumption Only one node is attacked or faulty at a time, so there are at most 2k+1 system conditions.
    Section IV states 'at most 2k+1 independent possible conditions'; concurrent attacks or faults are excluded from the classification problem.
  • ad hoc to paper The autoencoder's latent representation of dimension equal to the state dimension preserves enough information for the LSTM to predict normal behavior and for residuals to reveal anomalies.
    Assumed without theoretical justification; the paper's empirical results are the only support, and the choice of code size is heuristic.
  • ad hoc to paper The LSTM trained on normal data predicts future encoded measurements well enough that prediction residuals separate normal, attack, and fault conditions.
    No analysis of the LSTM's predictive accuracy or residual distributions is given; the classification performance depends on this.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Deep Learning based Covert Attack Identification for Industrial Control Systems." pith.science (2026). https://pith.science/paper/MY5LB7JN

@misc{pith2026200912360,
  author       = {Pith},
  title        = {Pith review of: Deep Learning based Covert Attack Identification for Industrial Control Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MY5LB7JN}},
  note         = {Machine review of arXiv:2009.12360}
}
read the original abstract

Cybersecurity of Industrial Control Systems (ICS) is drawing significant concerns as data communication increasingly leverages wireless networks. A lot of data-driven methods were developed for detecting cyberattacks, but few are focused on distinguishing them from equipment faults. In this paper, we develop a data-driven framework that can be used to detect, diagnose, and localize a type of cyberattack called covert attacks on smart grids. The framework has a hybrid design that combines an autoencoder, a recurrent neural network (RNN) with a Long-Short-Term-Memory (LSTM) layer, and a Deep Neural Network (DNN). This data-driven framework considers the temporal behavior of a generic physical system that extracts features from the time series of the sensor measurements that can be used for detecting covert attacks, distinguishing them from equipment faults, as well as localize the attack/fault. We evaluate the performance of the proposed method through a realistic simulation study on the IEEE 14-bus model as a typical example of ICS. We compare the performance of the proposed method with the traditional model-based method to show its applicability and efficacy.

Figures

Figures reproduced from arXiv: 2009.12360 by the authors.

Figure 1
Figure 1. Networked Industrial Control System and a single subsystem i can be represented by a recursive function of x i t as follows: x i t = fi(x{t−1}, u{t−1}) (3) y i t = gi(xt) (4) Systems with linear dynamics where the Markovian property holds, can be represented by a linear state-space model. For an individual subsystem i which operates independently, we have: x i t = Aix i t−1 + Biu i t−1 (5) y i t = Cix i t (6) When t… view at source ↗
Figure 2
Figure 2. The proposed detection framework (AEN + RNN + DNN) [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The IEEE 14-Bus System1 We generate data via a simulation study on a smart grid. We use the IEEE 14-bus ( [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: The SE + DNN Structure [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: The RNN + DNN Structure this case, we use n = 14 as the code size of the autoencoder. Usually for an AC k-bus system the state space has dimension 2(k − 1), consisting of the magnitudes and phase angles of voltages at each bus. Since here we only care about the active …
Figure 6
Figure 6. Figure 6: F1 score under different levels of attack for the three models [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: The boxplots of accuracy under different number of training levels [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. zkSTAR: A zero knowledge system for time series attack detection enforcing regulatory compliance in critical infrastructure networks

    cs.CR 2025-10 reject novelty 5.0 of 10

    zkSTAR proves with zero-knowledge proofs that a utility's Kalman-filter-based attack alarms were computed correctly, keeping sensor data private.

Reference graph

Works this paper leans on

23 extracted references · 22 canonical work pages · cited by 1 Pith paper

  1. [1]

    Stouffer, J

    K. Stouffer, J. Falco, K. Scarfone, Guide to industrial control systems (ics) security, NIST special publication 800 (82) (2011) 16–16. 1 2 3 4 5 0.0 0.4 0.8 Attack on # 2 level F1 score 1 2 3 4 5 0.0 0.4 0.8 Attack on # 3 level F1 score 1 2 3 4 5 0.0 0.4 0.8 Attack on # 6 level F1 score 1 2 3 4 5 0.0 0.4 0.8 Attack on # 8 level F1 score SE+DNN RNN+DNN AE...

    work page 2011

  2. [2]

    DeSmit, A

    Z. DeSmit, A. E. Elhabashy, L. J. Wells, J. A. Camelio, An approach to cyber-physical vulnerability assessment for intelligent manufacturing systems, Journal of Manufacturing Systems 43 (2017) 339–351

    work page 2017

  3. [3]

    Sridhar, A

    S. Sridhar, A. Hahn, M. Govindarasu, Cyber–physical system security for the electric power grid, Proceedings of the IEEE 100 (1) (2011) 210–224

    work page 2011

  4. [4]

    Kushner, The real story of stuxnet, ieee Spectrum 3 (50) (2013) 48–53

    D. Kushner, The real story of stuxnet, ieee Spectrum 3 (50) (2013) 48–53

    work page 2013

  5. [5]

    Sridhar, G

    S. Sridhar, G. Manimaran, Data integrity attacks and their impacts on scada control system, in: IEEE PES general meeting, IEEE, 2010, pp. 1–6

    work page 2010

  6. [6]

    Y . Mo, R. Chabukswar, B. Sinopoli, Detecting integrity attacks on scada systems, IEEE Transactions on Control Systems Technology 22 (4) (2014) 1396–1407

    work page 2014

  7. [7]

    Y . Liu, P. Ning, M. K. Reiter, False data injection attacks against state estimation in electric power grids, ACM Transactions on Information and System Security (TISSEC) 14 (1) (2011) 13

    work page 2011

  8. [8]

    R. S. Smith, A decoupled feedback structure for covertly appropriating networked control systems, IFAC Proceedings V olumes 44 (1) (2011) 90–95

    work page 2011

Show all 23 references
  1. [9]

    A. L. Buczak, E. Guven, A survey of data mining and machine learning methods for cyber security intrusion detection, IEEE Communications surveys & tutorials 18 (2) (2015) 1153–1176

    2015

  2. [10]

    T. R. Nudell, S. Nabavi, A. Chakrabortty, A real-time attack localiza- tion algorithm for large power system networks using graph-theoretic techniques, IEEE Transactions on Smart Grid 6 (5) (2015) 2551–2559

    2015

  3. [11]

    Van Long, L

    D. Van Long, L. FILLATRE, I. NIKIFOROV , Sequential monitoring of scada systems against cyber/physical attacks, IFAC-PapersOnLine 48 (21) (2015) 746–753

    2015

  4. [12]

    A. A. C ´ardenas, S. Amin, Z.-S. Lin, Y .-L. Huang, C.-Y . Huang, S. Sastry, Attacks against process control systems: risk assessment, detection, and response, in: Proceedings of the 6th ACM symposium on information, computer and communications security, ACM, 2011, pp. 355–366

    2011

  5. [13]

    Huang, B

    T. Huang, B. Satchidanandan, P. Kumar, L. Xie, An online detection framework for cyber attacks on automatic generation control, IEEE Transactions on Power Systems 33 (6) (2018) 6816–6827

    2018

  6. [14]

    S. Li, Y . Yılmaz, X. Wang, Quickest detection of false data injection attack in wide-area smart grids, IEEE Transactions on Smart Grid 6 (6) (2014) 2725–2735

    2014

  7. [15]

    Yilmaz, S

    Y . Yilmaz, S. Uludag, Mitigating iot-based cyberattacks on the smart grid, in: 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA), IEEE, 2017, pp. 517–522

    2017

  8. [16]

    Terai, S

    A. Terai, S. Abe, S. Kojima, Y . Takano, I. Koshijima, Cyber-attack detection for industrial control system monitoring with support vector machine based on communication profile, in: 2017 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW), IEEE, 2017, pp. 132–138

    2017

  9. [17]

    Caselli, E

    M. Caselli, E. Zambon, J. Amann, R. Sommer, F. Kargl, Specifica- tion mining for intrusion detection in networked control systems., in: USENIX Security Symposium, 2016, pp. 791–806

    2016

  10. [18]

    Bakalos, A

    N. Bakalos, A. V oulodimos, N. Doulamis, A. Doulamis, A. Ostfeld, E. Salomons, J. Caubet, V . Jimenez, P. Li, Protecting water infrastructure from cyber and physical threats: Using multimodal data fusion and adap- tive deep learning to monitor critical systems, IEEE Signal Pro...

    2019

  11. [19]

    Carri ´on, J

    M. Carri ´on, J. M. Arroyo, A computationally efficient mixed-integer linear formulation for the thermal unit commitment problem, IEEE Transactions on power systems 21 (3) (2006) 1371–1378

    2006

  12. [20]

    Hochreiter, J

    S. Hochreiter, J. Schmidhuber, Long short-term memory, Neural com- putation 9 (8) (1997) 1735–1780

    1997

  13. [21]

    Street, Dataport: the world’s largest energy data resource, Pecan Street Inc

    P. Street, Dataport: the world’s largest energy data resource, Pecan Street Inc

  14. [22]

    R. D. Zimmerman, C. E. Murillo-S ´anchez, R. J. Thomas, Matpower: Steady-state operations, planning, and analysis tools for power systems research and education, IEEE Transactions on power systems 26 (1) (2010) 12–19

    2010

  15. [23]

    F. C. Schweppe, J. Wildes, Power system static-state estimation, part i: Exact model, IEEE Transactions on Power Apparatus and systems (1) (1970) 120–125

    1970

Pith tools

Reviewed August 27, 2026 · model on record in the stance chip above.