{"id":"544e46e1-c82b-471f-8333-38f601190d28","arxiv_id":"2501.13474","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Using digital twin augmented data improved random forest and LSTM anomaly detection accuracy on a simulated power electronics dominated microgrid under a man-in-the-middle attack.","lead":"This paper combines a digital twin of a small power grid with machine learning to detect man-in-the-middle attacks on simulated electrical measurements. On a hardware-in-the-loop test bench, feeding digital twin estimates to random forest and LSTM classifiers improved detection accuracy.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The DT is fed the same IED measurements a MITM attacker can corrupt, and the attack is unspecified; the with-DT gain may reflect leakage of the attack signal through the DT rather than independent information.","rationale":"The reader's weakest assumption correctly identifies the load-bearing issue: the DT's virtual measurements may not be independent of the compromised IED data, so the observed improvement could partly reflect leakage. My stress-test sharpens this by pointing to the specific input configuration in Section IV and to the missing attack specification, which makes the comparison uninterpretable without additional controls. This is not a fatal flaw in the central claim; the claimed improvement may still hold, but the current reporting cannot distinguish genuine DT-added information from a re-encoding of the tampered inputs. Since the reader already conditioned the verdict on closing such gaps, my concern does not change the verdict; it reinforces the need for the stated conditions.","tokens_in":7958,"tokens_out":4506,"duration_ms":45034,"concrete_test":"Request the exact MITM attack vector (IED messages, data fields modified, magnitude, duration, and whether the analog PCC voltage channel is also attacked), then rerun the comparison in two modes: (a) DT receives the same attacked IED references as in the paper, and (b) DT receives clean, pre-attack references while the physical measurements visible to the ML baseline are attacked. If the accuracy gain is not retained in mode (b) after matching all other preprocessing, the claimed benefit is attributable to the DT re-encoding compromised inputs rather than to independent virtual sensing. As a further control, replace the DT features with simple algebraic derived quantities from the raw IED powers and the analog PCC voltage; if the gain persists, no full DT model is needed.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section IV states that 'the measured power values received act as reference points for the loads and power inverters in the DT.' The DT thus receives the same IED digital measurements that a MITM attacker would target, and its only independent input is the analog PCC voltage. The virtual features V1-V6 and F1-F6 are outputs of an averaged model driven by these inputs. Under an MITM attack on the IED communication channel, the attacked power values propagate into the DT, so the DT features are nonlinear transformations of the same corrupted measurements that appear in the without-DT baseline. The reported accuracy gain (RF: 0.7434 to 0.8692; LSTM: 0.8688 to 0.9159) could therefore be produced by the model exploiting a convenient encoding of the injected attack, not by genuinely independent physical information from the twin. This concern is not resolved by the paper because no explicit attack model is given: which IED messages are modified, by what magnitude, over what time window, and whether the attack also affects the analog voltage channel are all unspecified. Without that information, the with-DT comparison cannot isolate the DT's contribution. (A secondary but related issue is that SMOTE is applied before the train/test split, which can leak synthetic samples into the test set and inflate absolute metrics in both conditions.)","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a digital twin (DT) plus machine learning approach for detecting anomalies, specifically man-in-the-middle attacks, in a power-electronics-dominated grid. A hardware-in-the-loop testbed is described in which an RTDS simulator emulates the physical grid and an OPAL-RT machine runs an averaged real-time model as the digital twin. Random Forest and LSTM classifiers are trained on IED measurements alone and on IED measurements augmented with DT virtual voltage and frequency estimates (V1-V6, F1-F6). Table I reports that adding DT features improves accuracy from 0.7434 to 0.8692 for Random Forest and from 0.8688 to 0.9159 for LSTM, and the paper concludes that DT-enhanced data significantly improve threat detection.","tokens_in":8190,"tokens_out":2908,"duration_ms":28464,"significance":"If the claimed improvement is real and attributable to genuinely independent information from the digital twin, the result would be a useful practical contribution to cyber-physical security for inverter-dominated grids. The paper has some strengths: it uses a realistic real-time hardware-in-the-loop testbed (RTDS and OPAL-RT), implements standard and reproducible ML classifiers, and reports a clear head-to-head comparison in Table I. However, the current evidence is not sufficient to support the central claim: the attack scenario is unspecified, the digital twin is driven by the same IED measurements that the attack corrupts, no uncertainty quantification is provided, and the DT fidelity is not validated. The paper is best viewed as a proof-of-concept description rather than a validated empirical demonstration.","major_comments":[{"comment":"The central claim that DT-enhanced data 'significantly improved' accuracy is not supported by any statistical analysis. Table I reports single point estimates for each metric, and although §III.B mentions 10-fold cross-validation, no per-fold results, standard deviations, confidence intervals, or significance tests are reported. Without these, the observed differences (e.g., RF accuracy 0.7434 vs. 0.8692) could plausibly be within the noise of a single data split. Please report the mean and standard deviation across folds, or an equivalent uncertainty measure, for every metric and both conditions.","section":"§IV.D, Table I"},{"comment":"The attack model is not described, which makes the reported detection results impossible to interpret or reproduce. The paper states only that a man-in-the-middle attack was simulated, but does not specify which IED messages were modified, by what magnitude, over what time window, how many attack instances were generated, or whether the analog PCC voltage channel was also affected. These details are essential because they determine the difficulty of the detection problem and the meaning of the with-DT versus without-DT comparison. Please provide an explicit attack model and describe how it was implemented in the testbed.","section":"§IV, §IV.A"},{"comment":"The digital twin is driven by the same IED measurements that a MITM attacker can corrupt, so the observed accuracy gain may reflect leakage of the attack signal through the DT rather than independent physical information. The manuscript says 'The measured power values received act as reference points for the loads and power inverters in the DT,' and the power features (P_pv, Q_pv, P_batt, Q_batt, P_w, Q_w, P_n, Q_n) are all received from IEDs. If an attacker modifies those IED messages, the corrupted values propagate into the DT, and the virtual voltage and frequency features V1-V6 and F1-F6 become nonlinear transformations of the same compromised measurements. To support the claim that the DT contributes independent information, the authors should either (i) protect or authenticate the DT input channel in the attack scenario, (ii) demonstrate with an explicit attack model that the V/F features respond differently from the raw IED features, or (iii) otherwise quantify the independence of the DT features from the attacked measurements.","section":"§IV, first paragraph; §IV.A"},{"comment":"The preprocessing applies SMOTE before splitting the data into training and test sets, which can leak synthetic minority-class samples into the test set and inflate the reported accuracy and F1-scores. The text says SMOTE was applied to balance the class distribution and then 'we divided the dataset into training and testing sets using stratified sampling.' This ordering is methodologically problematic; SMOTE should be applied only to the training portion of each cross-validation fold, after the split. Please re-run the experiments with SMOTE applied inside the cross-validation loop and report whether the conclusions change.","section":"§III.A, data preprocessing"},{"comment":"The digital twin fidelity is asserted but never validated, and the dataset is not characterized. The manuscript claims the DT 'accurately mirror[s] the current state of the power system' and 'guarantees that the overall DT model accurately represents the real-time condition,' but no comparison between DT virtual measurements and physical RTDS measurements is provided, no error metrics are given, and the load and inverter model parameters are not tied to the specific devices in the testbed. Furthermore, the dataset description lacks the number of samples, the normal-to-attack class ratio, the total duration of the recordings, and the number of attack episodes. These omissions prevent the reader from assessing whether the DT is a faithful representation and whether the ML evaluation is meaningful.","section":"§IV, §IV.A, §II"}],"minor_comments":[{"comment":"The table heading contains a typo: 'PERFOMANCE' should be 'PERFORMANCE'.","section":"Table I heading"},{"comment":"The reference marker '[E]' appears in the text but is not defined anywhere in the reference list; this should be replaced with a proper citation.","section":"§II, first paragraph"},{"comment":"The phrase 'enhance model performance and interoperability' should likely read 'interpretability' in the context of feature selection based on Gini impurity decrease.","section":"§III.A, feature selection"},{"comment":"The reference list contains duplicate numbering: two entries are labeled [7], and reference [22] appears to be misdated (a 2010 citation for digital twin work on a vanadium redox flow battery is implausible). Please renumber and verify all references.","section":"References"},{"comment":"The abstract and conclusion state that the DT 'can accurately track and simulate' the grid, but the validation results needed to support this claim are not presented in the paper; the wording should be softened or supported by the required fidelity comparison.","section":"Abstract and §V"}],"recommendation":"major_revision","confidential_remarks":"The paper is a plausible proof-of-concept, but the central empirical claim needs substantial additional support: an explicit attack model, a split-safe preprocessing pipeline, and DT-fidelity validation. The manuscript also appears to be a lightly polished conference paper; the journal should require the standard of evidence expected for a full archival publication, including uncertainty quantification and reproducibility details. The reader's stress-test concern about DT input leakage is well-founded and should be directly addressed by the authors."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick take: this is a short IEEE conference paper that builds a real hardware-in-the-loop testbed (RTDS + OPAL-RT) and reports that adding digital twin (DT) voltage and frequency estimates to standard IED power measurements improves Random Forest and LSTM anomaly detection accuracy under a man-in-the-middle (MITM) attack. That is a legitimate empirical contribution, but the headline claim is only partially supported because the attack model is not specified and the DT is partly driven by the same digital measurements the attacker can corrupt.\n\nWhat is actually new: the specific testbed configuration and the measured improvement in the four-row metrics table. The averaged real-time inverter/load model on OPAL-RT, synced with a detailed electromagnetic transient model on RTDS, is a reasonable way to evaluate the idea, and comparing both a tree-based and a recurrent model gives some breadth. The general DT-plus-ML direction is established, but this particular implementation and measurement are not in the cited literature.\n\nThe main soft spot is a potential leakage path. The paper states that \"the measured power values received act as reference points for the loads and power inverters in the DT.\" A MITM attacker on the IED communication channel can corrupt exactly those power references. The DT's virtual V1-V6 and F1-F6 outputs are then nonlinear functions of the corrupted measurements plus the analog PCC voltage. Without an explicit attack model—which IED messages are modified, by how much, over what window, and whether the analog voltage channel is also affected—the observed accuracy gain (RF 0.7434 to 0.8692, LSTM 0.8688 to 0.9159) could come from the model reading an encoded version of the attack signal through the DT rather than from independent physical information. This is a real confound, not a pedantic one.\n\nSecondary issues: SMOTE is applied before the train/test split, which leaks synthetic samples into the test set and can inflate absolute metrics in both conditions. The dataset is not characterized—no sample counts, time span, or event details. The conclusion that DT \"allows for training ML models with limited statistical data\" is not tested. The reference list is also sloppy: there is a placeholder \"[E]\" in Section II and a missing [10] in the introduction.\n\nNone of these are fatal to the idea. The paper is a proof-of-concept, and the effect direction is plausible. But as it stands, the reader cannot separate the DT's contribution from a convenient encoding of the attack.\n\nWho this is for: researchers working on DT-based monitoring for microgrids, especially those building HIL testbeds. It could be a useful starting point, but not a reference result.\n\nRecommendation: send to peer review. A serious referee can push for a defined attack model, a proper preprocessing pipeline, and minimal dataset characterization. With those, this becomes a solid short-conference contribution.","headline":"A plausible proof-of-concept that DT features help ML attack detection, but the underspecified MITM attack and the DT's dependence on the same IED measurements leave the main claim unproven.","tokens_in":8712,"tokens_out":3226,"would_cite":false,"duration_ms":26344,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Adding a digital twin's virtual voltage and frequency measurements to raw grid data improves machine learning detection of man-in-the-middle attacks, with LSTM accuracy rising from 86.9% to 91.6%.","keywords":["digital twin","threat detection","machine learning","real-time","cyberattacks","power electronics dominated grid","man-in-the-middle attack","anomaly detection"],"falsifier":"An ablation that feeds the digital twin with pre-attack recorded power references while the physical system is under attack, then compares detection accuracy, would settle whether the twin's virtual measurements flag the attack or merely mirror corrupted measurements.","tokens_in":7771,"feed_emoji":"⚡","tokens_out":6254,"duration_ms":52249,"temperature":0.7,"pith_summary":"The paper claims that a real-time digital twin of a power-electronics-dominated microgrid can supply virtual voltage and frequency readings that materially improve machine-learning detection of man-in-the-middle attacks. The authors build an averaged-model twin of the grid's inverters, loads, and lines, run it beside a hardware-in-the-loop simulation, and add the twin's node voltages and frequencies to the intelligent electronic device (IED) power measurements. Random Forest accuracy rises from 74.3% to 86.9% and LSTM accuracy from 86.9% to 91.6%, with F1 rising in both cases. The claim matters because a twin-generated data channel could improve grid cybersecurity and supply training data when real attack measurements are scarce.","feed_headline":"Digital twin data lifts grid attack detection","feed_subtitle":"Adding twin-generated voltages and frequencies raises LSTM attack-detection accuracy to 91.6 percent.","key_machinery":"The carrying mechanism is the averaged real-time digital twin model. The inverter is modeled in grid-following mode as a current source with a D-Q decoupled PI current-control loop, a phase-locked loop, and an LCL output filter; loads follow $P_l = P_0(V/V_0)^{n_p}$ and $Q_l = Q_0(V/V_0)^{n_q}$; and transmission lines are represented by simplified series impedances. The twin takes the measured power values from the IEDs as references for loads and inverters and produces virtual voltage and frequency estimates at six feeder nodes. These virtual measurements are the extra features supplied to the Random Forest and LSTM classifiers, which are trained with z-score normalization, SMOTE balancing, Gaussian augmentation, and 10-fold cross-validation.","core_discovery":"On the authors' own terms, the central result is that integrating DT-enhanced data significantly improved the accuracy and overall metrics for both the Random Forest and LSTM models. With virtual voltage estimates (V1 to V6) and frequency estimates (F1 to F6) from the digital twin added to the IED power measurements, Random Forest accuracy increased from 74.34% to 86.92% and recall from 0.77 to 0.8748, while LSTM accuracy increased from 86.88% to 91.59%, precision from 0.8325 to 0.9417, and F1 score from 0.8254 to 0.9028. The DT is an averaged real-time model of the feeder's inverters, loads, and transmission lines that mirrors the physical grid's state; the measured power values received from the IED act as reference points for loads and inverters in the twin.","pith_inferences":["Beyond the paper, the twin's virtual measurements could be cross-checked against physical sensors at the same nodes to localize the attacked feeder segment; the paper does not perform that localization.","If an attacker can compromise the same communication channel that feeds measured power values into the twin, the method's advantage depends on virtual voltages and frequencies arriving through a separate, tamper-evident path, which the paper does not establish.","A natural stress test is a coordinated attack that falsifies both the IED power data and the reference values sent to the twin; the observed accuracy gain would likely shrink, revealing how much of the current result relies on the twin seeing clean inputs.","The virtual measurements may also serve as a consistency prior for physics-informed anomaly detectors, combining model-based residuals with learned classifiers; the paper leaves that combination unexplored."],"forward_implications":["If the reported gains hold, adding a digital twin's virtual node voltages and frequencies to field measurements gives microgrid operators a low-cost way to widen sensing coverage without installing new physical sensors.","A validated twin could generate labeled normal and attack datasets for training, easing the data scarcity that limits data-driven intrusion detection in real grids.","The same DT-plus-ML pipeline should transfer to other false-data injection and fault scenarios beyond the man-in-the-middle case tested here.","Since LSTM gained more consistently than Random Forest, models that exploit temporal structure benefit most from the twin's additional time-series features."],"supporting_citations":[{"why":"Supplies the inverter-dominated microgrid context and power-electronics interface for the DT inverter model.","marker":"[4]"},{"why":"Supplies the current-control loop, phase-locked loop, and LCL filter dynamics used in the inverter part of the digital twin.","marker":"[23]"},{"why":"Supports the simplified transmission-line model used in the twin.","marker":"[24]"},{"why":"Defines the Random Forest classifier used as the traditional machine-learning detector.","marker":"[25]"},{"why":"Defines the LSTM architecture used as the deep-learning detector.","marker":"[26]"}],"fun_headline_variants":["Digital twin data lifts grid attack detection to 91.6%","Twin-enhanced ML spots grid faults at 91.6% accuracy","DT+ML: Grid anomaly detection accuracy hits 91.6%","Digital twin boosts LSTM grid attack detection to 91.6%","Grid cybersecurity gains from digital twin data, 91.6% accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim collapses if the twin's virtual readings add no information beyond the raw measurements from grid devices, especially because the twin uses those same measured power values as its reference inputs and could therefore inherit corrupted data during an attack.","fun_headline_variants_meta":{"raw":{"variants":["Digital twin data lifts grid attack detection to 91.6%","Twin-enhanced ML spots grid faults at 91.6% accuracy","DT+ML: Grid anomaly detection accuracy hits 91.6%","Digital twin boosts LSTM grid attack detection to 91.6%","Grid cybersecurity gains from digital twin data, 91.6% accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000895,"raw_usage":{"total_tokens":3822,"prompt_tokens":874,"completion_tokens":2948,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":490,"completion_tokens_details":{"reasoning_tokens":2853}},"tokens_in":490,"tokens_out":2948,"duration_ms":18872,"temperature":1.0,"reasoning_tokens":2853,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T15:53:48.958799+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"An ablation that feeds the digital twin with pre-attack recorded power references while the physical system is under attack, then compares detection accuracy, would settle whether the twin's virtual measurements flag the attack or merely mirror corrupted measurements.","supporting_citations":[{"cited_title":"Paralleled Modified Droop -Based Voltage Source Inverter for 100% Inverter - Based Microgrids,","cited_arxiv_id":null,"evidence_quote":"Supplies the inverter-dominated microgrid context and power-electronics interface for the DT inverter model."},{"cited_title":"Imitatsionnoe modelirovanie sistemelektrosnabzheniya v programme MATLAB [Simulation modeling of power supplysystems in MATLAB],","cited_arxiv_id":null,"evidence_quote":"Supports the simplified transmission-line model used in the twin."}],"review_version":1}