{"id":"f84d929d-ea68-415e-92bd-c8ec95e3b661","arxiv_id":"2504.20295","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"A digital twin water forecasting LSTM is shown to be vulnerable to FGSM and PGD attacks, and Learning Automata variants that adapt epsilon push MAPE above 35%, but no detection experiment supports the stealth claim.","lead":"This paper tests whether adversarial machine learning attacks can fool an LSTM water-consumption forecaster in a Spanish digital twin platform, and proposes Learning Automata and Random Learning Automata variants that adaptively choose attack strength. The authors report MAPE increases from roughly 25% to over 35% for large perturbations, but the claimed stealth advantage is not tested against any detector.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'harder to detect' claim rests entirely on epsilon fluctuation plots; no detector, anomaly score, statistical test, or human study is run, so the paper's central new contribution is unsupported.","rationale":"The reader's weakest_assumption identifies precisely the load-bearing gap: visual epsilon fluctuation is treated as evidence of undetectability without any detector. I agree this is the key weakness because the paper's only novel contribution is the LA/RLA stealth mechanism; the vulnerability of an LSTM to FGSM/PGD is standard and plausible, and the error tables, though internally inconsistent with the abstract, do show that perturbations degrade MAPE. The stealth claim is what distinguishes this paper from routine adversarial-attack applications, and it is never tested. My concrete test would settle the matter: compare detection rates under a simple anomaly detector. Since the reader already recommended REJECT and my concern aligns with that rationale, the verdict remains unchanged. I would note one nuance: the paper does provide algorithm listings and standard attack equations, so the vulnerability part is reproducible in principle, but no code/data are supplied, so even that remains unverified. The core issue remains the unfalsified 'harder to detect' claim, and the recommendation stands.","tokens_in":10062,"tokens_out":1829,"duration_ms":20288,"concrete_test":"Train a simple anomaly detector (e.g., one-class SVM or isolation forest) on clean input sequences from the same dataset, then run the fixed-epsilon FGSM attack and the LA/RLA-based attacks described in Sections VI and VII. Evaluate detection rates with a fixed false-positive rate (e.g., 5%) and compare ROC-AUC across attack types at matched MAPE levels. If the LA/RLA attack is detected at an equal or higher rate than standard FGSM, the 'harder to detect' claim in the abstract and conclusions is directly refuted.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that the LA and RLA mechanisms 'make adversarial attacks more difficult to detect' is supported only by visual inspection of epsilon fluctuation curves in Figures 4-7 (Sections VI.D and VII.F). No anomaly detector, change-point test, statistical test, or human perception study is applied to the actual input sequences. The reward/penalty functions (Eqs. 4-5, 8-9, 14) operate on MAPE, which measures forecast error, not detectability; a perturbation that keeps MAPE between 30% and 50% may still be trivially flagged by a detector that monitors input magnitude or residual statistics. In fact, the delayed input strategy (Eq. 15) introduces discontinuities when stored adversarial examples are injected, potentially making the attack more detectable than a fixed-epsilon FGSM baseline. Because the paper provides no detection baseline, no ROC curves, and no comparison against standard FGSM/PGD under a common detector, the stealth claim is unfalsified and the LA/RLA contribution lacks demonstrated value. Secondary inconsistencies (abstract MAPE 'from 26% to over 35%' vs. Table II baselines of 24.4% and 25.0%) further weaken the quantitative narrative but are less load-bearing than the missing detectability evidence.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper describes a digital twin platform for a Spanish water distribution network and an LSTM-based water consumption forecaster. It applies FGSM and PGD adversarial attacks and reports that larger perturbations degrade forecasting accuracy, as measured by MAE, RMSE, and MAPE. The claimed novel contribution is a Learning Automata (LA) and Random Learning Automata (RLA) mechanism that dynamically adjusts the perturbation size, with the abstract reporting that MAPE rises from 26% to over 35% and that the attacks become harder to detect. The paper also discusses mitigation strategies. The core contributions are the LA/RLA epsilon-selection schemes and the associated undetectability claims.","tokens_in":10290,"tokens_out":3442,"duration_ms":35754,"significance":"If the LA/RLA stealth claim were established, the paper would provide a concrete demonstration of a new attack-optimization mechanism in a critical infrastructure domain, which would be relevant for water utility cybersecurity. The standard FGSM/PGD vulnerability tables are useful baseline evidence, and the real-world deployment context adds some interest. However, the novelty resides entirely in the undetectability claim, and that claim is not supported by any detector comparison, statistical test, or detection baseline. In addition, the reported MAPE increase is built into the LA reward function rather than being an emergent or independently measured result. The paper therefore does not currently substantiate its central contribution; the standard FGSM/PGD results alone do not constitute a sufficient advance.","major_comments":[{"comment":"The central claim that the LA and RLA mechanisms make adversarial attacks 'harder to detect' is supported only by visual inspection of epsilon fluctuation plots. No anomaly detector, change-point test, statistical test, residual analysis, or human perception study is applied to the input sequences or forecasts, so the stealth claim is unfalsified. A concrete detection baseline (e.g., an anomaly score on input norms or forecast residuals) is needed before this claim can be evaluated.","section":"Section VI.B, VI.D, and VII.F; Figures 4-7"},{"comment":"The LA reward function explicitly rewards epsilon values that keep MAPE between 30% and 50% and penalizes MAPE above 100%. Consequently, the abstract's claim that MAPE rises 'from 26% to over 35%' is an optimization target of the reward mechanism, not an independent prediction of attack impact. This circularity applies to both the LA and RLA variants and undermines the quantitative headline result.","section":"Section VI.B (Eqs. 4-5) and Section VII.D (Eqs. 8-9, 14)"},{"comment":"The PGD results in Table III are numerically nearly identical to the FGSM results in Table II at every epsilon value, yet the text states that PGD 'consistently leads to higher errors' and exhibits 'a steeper increase in RMSE and MAPE.' This discrepancy suggests that the PGD implementation is not actually iterative or that the comparison is not meaningful, weakening the PGD analysis and the claim that adaptive strategies amplify the effect.","section":"Tables II and III and Section V"},{"comment":"The number of selected epsilon values is written as k~U{1,3}, but the accompanying text says the distribution selects 'either 1 or 2 epsilon values per iteration.' These are inconsistent, and the sampling distribution is not defined precisely enough to reproduce the RLA mechanism.","section":"Section VII.C, Eq. (12)"},{"comment":"The abstract reports a baseline MAPE of 26% rising to over 35%, but Table II lists baselines of 24.419% and 25.032% and an FGSM eps=0.01 MAPE of 32.754% for the LSTM model. The numbers in the abstract do not match the tables, and no explanation or error bars are provided to resolve the discrepancy; this weakens the quantitative narrative of the paper.","section":"Abstract and Table II"}],"minor_comments":[{"comment":"Several figure references are incomplete: 'as shown in 4', 'as shown in 6', and similar instances should be 'Figure 4' and 'Figure 6' to match standard formatting.","section":"Sections VI.D and VII.F"},{"comment":"Algorithm 1 is a generic LSTM training outline and does not specify the actual hyperparameters, data split proportions, sequence length, or number of epochs used in the experiments; adding exact values or a repository reference would improve reproducibility.","section":"Algorithm 1 and Section III"},{"comment":"The assertion that perturbations are 'small enough to go unnoticed in the input data' is made without a quantitative perceptibility metric; defining such a metric would clarify the stealth discussion.","section":"Section IV"},{"comment":"Table I lists numerous digital twin projects but does not state the inclusion criteria or sources for the vulnerability classifications; clarifying the compilation method would increase the table's usefulness.","section":"Table I"}],"recommendation":"reject","confidential_remarks":"The paper's main new idea, the LA/RLA-based stealth attack, lacks a falsifiable test because no detector or anomaly-scoring baseline is defined. The circularity of the LA reward function further means that the headline MAPE improvement is not an independent result. The PGD/FGSM table similarity and the abstract/table mismatch suggest the experiments need substantial reworking. These issues are load-bearing and would require a redesign of the evaluation, not just local revisions."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nHere's my take on arXiv:2504.20295. The paper does two things: it demonstrates that FGSM and PGD attacks degrade LSTM water-consumption forecasts, and it adds a Learning Automata (LA) and Random LA mechanism to adaptively select the perturbation size. The first result is plausible and standard; the second is the new contribution, and its central claim—that these attacks are harder to detect—is unsupported. The evidence for stealth is visual inspection of epsilon fluctuation plots in Figures 4-7. No anomaly detector, change-point test, statistical test, or human study is run against the actual input sequences. The reward function explicitly keeps MAPE between 30% and 50%, so the reported 'over 35%' MAPE is an optimization target, not an independent outcome. This circularity undermines the abstract's headline result.\n\nWhat is genuinely good: the paper gives a concrete description of the CAUCCES digital twin platform, the LSTM forecasting setup is standard, and the baseline vulnerability results are reported in clear tables. The LA-based epsilon selection is a legitimate extension of known attack tuning, though not a new attack class. The paper is written plainly and the baseline attack pipeline is understandable.\n\nThe soft spots are more than cosmetic. The abstract's baseline MAPE of 26% does not match the 24.4-25.0% baselines in Tables II and III. FGSM and PGD tables are nearly identical, which is suspicious since PGD should be stronger; it suggests the two attacks may not have been implemented distinctly or there is an error in the reporting. No error bars, code, or data are provided, and key hyperparameters (reward/penalty factors, delay length) are not reported. The delayed input strategy in Eq. (15) could introduce discontinuities that make the attack more detectable than fixed-epsilon FGSM, yet no evidence addresses this.\n\nIf the authors had run even a simple detector—a threshold on input perturbation norm, a residual-based anomaly score, anything—and shown that LA/RLA evades it better than fixed-epsilon attacks, the paper would be meaningful. As it stands, the main new claim is untested and the paper reads like a work-in-progress.\n\nThis does deserve a serious referee: the application domain is important and the paper is a coherent attempt. But I would not accept it without the detection experiment and consistency fixes, and I would not cite it in its current form.","headline":"A standard FGSM/PGD attack on LSTM water forecasting whose learning-automata twist is under-evaluated: the stealth claim rests only on epsilon fluctuation plots, with no detection experiment.","tokens_in":10877,"tokens_out":3655,"would_cite":false,"duration_ms":35367,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Tiny data tweaks push water-forecast error to 35 percent","keywords":["digital twins","water distribution systems","adversarial machine learning","FGSM","PGD","LSTM forecasting","learning automata","cybersecurity"],"falsifier":"Run an existing anomaly detector, such as isolation forest or change-point detection, on the test-set inputs after an LA/RLA-FGSM attack and compare its detection rate against fixed-epsilon FGSM at the same mean perturbation budget; if the LA/RLA version is flagged at an equal or higher rate, the paper's stealth claim is refuted.","tokens_in":1363,"feed_emoji":"💧","tokens_out":1765,"duration_ms":53395,"temperature":0.7,"pith_summary":"The paper argues that a digital twin for water distribution, built on LSTM networks that forecast consumption from daily water and temperature records, is seriously vulnerable to white-box adversarial attacks. It shows that FGSM and PGD perturbations raise the Mean Absolute Percentage Error from roughly 24-26% to over 35%, and proposes a Learning Automata (LA) and Random LA mechanism that dynamically chooses the perturbation size. The claimed benefit is stealth: by varying epsilon through reward and penalty updates, and by delaying poisoned inputs, the attack is supposedly harder to detect than a fixed-epsilon attack while remaining effective. A sympathetic reader should care because, if true, an attacker who can compute gradients through the forecasting model could quietly degrade the reliability of a critical water utility's predictions and mislead operators.","feed_headline":"Tiny data tweaks push water-forecast error to 35 percent","feed_subtitle":"A learning-automata attack varies its perturbation size to stay hidden while degrading LSTM water-consumption forecasts.","key_machinery":"The central objects are the FGSM and PGD attack formulas and the Learning Automata/Random Learning Automata epsilon-selection layer. FGSM adds the sign of the loss gradient scaled by epsilon to the input; PGD iterates this with clipping; the LA layer chooses epsilon from the set {0.0001, 0.0005, 0.001, 0.0025, 0.005} and updates action probabilities with rewards when MAPE stays in 30-50% and penalties when it exceeds 100%; RLA draws one or two epsilons per iteration and delays adversarial inputs by a fixed lag. The machinery's job is to make the perturbation schedule adaptive and non-monotonic, which the paper argues breaks the usual correlation between distortion size and detectability.","core_discovery":"The paper's central claim is that adding a Learning Automata layer to FGSM attacks on LSTM water-consumption forecasts makes the attacks both effective and harder to detect. The mechanism learns, from a fixed menu of epsilon values, which perturbation magnitudes keep MAPE in a 30-50% target band, punishing epsilons that push MAPE above 100%; Random LA goes further by selecting one or two epsilons per iteration and applying the poisoned input after a fixed delay. Experimental results show MAPE rising from 26% to over 35%, and the fluctuating epsilon trajectories are presented as evidence that the attack pattern is not monotonically detectable. If this is right, a white-box attacker with gradient access can substantially degrade forecasting accuracy while remaining visually inconspicuous.","pith_inferences":["The 'harder to detect' conclusion is inferred only from epsilon fluctuation plots; a concrete detector comparison would be needed to confirm it.","Because the LA reward function targets a 30-50% MAPE band, the attack is constrained not to maximize error, so a defender could potentially flag any forecast whose error sits suspiciously inside that band.","The delayed-input strategy suggests a testable extension: detect attacks by checking cross-correlation between past adversarial perturbations and current predictions, which the paper does not run."],"forward_implications":["LSTM-based water forecasting models in digital twins are not robust: even epsilon equals 0.01 causes MAPE to exceed 35%.","PGD is consistently stronger than FGSM at the same epsilon, especially at intermediate perturbation levels such as 0.005 and 0.008.","An attacker can tune epsilon with LA/RLA to keep forecast error in a chosen high range (30-50%) while varying perturbation magnitudes over time.","Delayed input poisoning is presented as a way to mask attack onset; if effective, defenses must inspect historical input sequences rather than only current samples.","The paper's own mitigation table implies that defending these systems requires adversarial training, real-time anomaly detection, and secure data pipelines."],"supporting_citations":[{"why":"Supplies the PGD attack used as the iterative, stronger counterpart to FGSM.","marker":"[9]"},{"why":"Supplies the learning automata formalism that the paper adapts for epsilon selection.","marker":"[15]"},{"why":"Demonstrates adversarial attacks on time-series data in smart grids, motivating the attack on water forecasts.","marker":"[7]"},{"why":"Shows gradient-based attacks on deep learning systems, supporting the claimed vulnerability of LSTM models.","marker":"[8]"},{"why":"Provides the LSTM monthly water-consumption forecasting approach that the paper attacks.","marker":"[10]"},{"why":"Supports the use of LSTM for time-series forecasting in water-related applications.","marker":"[11]"},{"why":"Reviews digital twins and AI-based cybersecurity, framing the threat model for the paper.","marker":"[5]"},{"why":"Gives an example of an AI-driven digital twin for water state estimation, showing the class of systems at risk.","marker":"[12]"}],"fun_headline_variants":["Water forecast error jumps to 35% via hidden AI attack","Learning automata attack raises water forecast error to 35%","AI water forecasting vulnerable to adaptive adversarial tweaks","Subtle data tweaks push water forecast error past 35%","Water forecasting digital twins at risk from smart attacks"],"cache_read_input_tokens":12928,"weakest_assumption_plain":"The load-bearing premise is that the visible fluctuation of the epsilon curve is enough to prove the attack is hard to detect, even though no actual detector, anomaly score, or statistical test is applied to the attacked inputs.","fun_headline_variants_meta":{"raw":{"variants":["Water forecast error jumps to 35% via hidden AI attack","Learning automata attack raises water forecast error to 35%","AI water forecasting vulnerable to adaptive adversarial tweaks","Subtle data tweaks push water forecast error past 35%","Water forecasting digital twins at risk from smart attacks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000198,"raw_usage":{"total_tokens":1336,"prompt_tokens":883,"completion_tokens":453,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":499,"completion_tokens_details":{"reasoning_tokens":372}},"tokens_in":499,"tokens_out":453,"duration_ms":4116,"temperature":1.0,"reasoning_tokens":372,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T05:32:44.296468+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run an existing anomaly detector, such as isolation forest or change-point detection, on the test-set inputs after an LA/RLA-FGSM attack and compare its detection rate against fixed-epsilon FGSM at the same mean perturbation budget; if the LA/RLA version is flagged at an equal or higher rate, the paper's stealth claim is refuted.","supporting_citations":[{"cited_title":"Rezvanian, A","cited_arxiv_id":null,"evidence_quote":"Supplies the learning automata formalism that the paper adapts for epsilon selection."},{"cited_title":"AdaptEdge: Targeted universal adversarial attacks on time series data in smart grids,","cited_arxiv_id":null,"evidence_quote":"Demonstrates adversarial attacks on time-series data in smart grids, motivating the attack on water forecasts."},{"cited_title":"Trojan attack and defense for deep learning-based navigation systems of unmanned aerial vehicles,","cited_arxiv_id":null,"evidence_quote":"Shows gradient-based attacks on deep learning systems, supporting the claimed vulnerability of LSTM models."},{"cited_title":"De- veloping an LSTM model to forecast the monthly water consumption according to the effects of the climatic factors in Yazd, Iran,","cited_arxiv_id":null,"evidence_quote":"Provides the LSTM monthly water-consumption forecasting approach that the paper attacks."},{"cited_title":"River water temperature forecasting using a deep learning method,","cited_arxiv_id":null,"evidence_quote":"Supports the use of LSTM for time-series forecasting in water-related applications."},{"cited_title":"A review of digital twins and their application in cybersecurity based on artificial intelligence,","cited_arxiv_id":null,"evidence_quote":"Reviews digital twins and AI-based cybersecurity, framing the threat model for the paper."},{"cited_title":"A digital twin of a water distribution system by using graph convolutional networks for pump speed-based state estimation,","cited_arxiv_id":null,"evidence_quote":"Gives an example of an AI-driven digital twin for water state estimation, showing the class of systems at risk."}],"review_version":1}