{"id":"6e172b63-dc58-46c5-ac6d-5968f8998842","arxiv_id":"2502.05041","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":7,"one_line_summary":"An empirical study of FGSM and PGD attacks on federated anomaly detection for smart meter data finds PGD more effective than FGSM, while the stated conclusion that FL is more vulnerable than centralized learning is contradicted by the paper's own results.","lead":"This paper tests how two white-box attacks, FGSM and PGD, break federated learning models that detect anomalies in household energy consumption data. The authors report that the iterative PGD attack is more damaging, but their own tables do not consistently support the headline claim that federated learning is more vulnerable than centralized training.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper's own accuracy columns refute the headline: central learning shows larger accuracy drops than FL for every FGSM/PGD condition in Tables I–III, so 'FL is more affected' is unsupported.","rationale":"The reader's rejection is well founded, and the central concern agrees with the need to reject. However, the single most load-bearing issue is not primarily the attack-budget mismatch alone; it is that the paper's own accuracy tables contradict the abstract's 'FL is more affected' claim in every FGSM/PGD comparison. The attack-budget confound in Section V-D is a real methodological flaw, but even taking the reported tables at face value, they support the opposite conclusion. I therefore focus the stress test on the internal inconsistency: the central claim fails against the paper's own evidence, independent of any need for new experiments. The reader's identified weakest assumption (incomparable attack budgets) is related, but the contradiction in the accuracy numbers is more decisive. A re-analysis of the existing tables, using relative drops from each model's clean baseline, would settle whether the conclusion was simply a misreading; all currently reported numbers suggest it was. No new computation is needed to see the direction, but the proposed test would make the count explicit and immune to the use of absolute accuracy versus relative drop. The paper does make one supported contribution: PGD is consistently more damaging than FGSM in both FL and central settings, and Transformer is often more resilient than LSTM. Those findings survive, but the abstract overreaches by adding the unsupported FL-versus-central comparison. Thus the reader's REJECT verdict should stand unchanged.","tokens_in":12505,"tokens_out":5976,"duration_ms":59077,"concrete_test":"Recompute, for every FGSM and PGD row in Tables I, II, and III, the relative accuracy drop from each model's own clean-trained baseline (e.g., (acc_clean - acc_attacked) / acc_clean × 100). If in all 12 attack conditions the central model shows a larger relative drop than the FL model, the paper's headline claim is directly contradicted by its reported numbers; if instead FL shows larger drops in any condition, the claim deserves renewed scrutiny with error bars and matched attack budgets.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract's central comparative claim is contradicted by the paper's reported accuracy data. In Table II, LSTM clean accuracy is 95.74% (central) vs 91.6% (FL); after FGSM training attacks, central accuracy drops to 33.13% (a 62.61-point drop) while FL drops to 36.6% (a 55.0-point drop). Under PGD, central drops to 22.36% (73.38-point drop) while FL drops to 23.5% (68.1-point drop). Transformer results show the same direction in Table II, and Table III repeats it on the 50-house dataset. Table I, for inference-time attacks, also shows larger absolute and relative accuracy degradations for central models under both FGSM and PGD for both architectures. The paper instead emphasizes ASR, but ASR only measures that predictions changed, not that they changed from correct to incorrect; because FL starts from a lower clean accuracy, a flip can even be wrong-to-right. Additionally, the Section V-D attack-budget mismatch (9 of 19 clients attacking 30% of their data is about 14% of total data vs 30% for central) means the FL condition was tested under a smaller perturbation budget, which only weakens the conclusion. Thus the strongest supported finding is that PGD is more effective than FGSM; the claim that FL is more affected than centralized learning is an internally inconsistent reading of the paper's own tables.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper presents an empirical evaluation of adversarial attacks (FGSM, PGD, AWGN, label flipping) on federated learning (FL) based anomaly detection for smart meter energy data, using LSTM and Transformer models. The authors simulate malicious clients that perturb their local training data and also evaluate attacks at inference time. The main claimed findings are that PGD is more effective than FGSM and that FL is more affected by these attacks than centralized learning, with the impact increasing with attack strength and the number of malicious clients.","tokens_in":12785,"tokens_out":6605,"duration_ms":62609,"significance":"If the comparative claim about FL being more vulnerable than centralized learning were supported, this would be a useful empirical contribution to the security literature on FL in energy applications. The paper uses real smart meter data, two architectures, several attack types, and includes a sensitivity analysis (Fig. 4) that is informative. The experimental protocol is described in enough detail to be reproduced. However, the paper's headline claim is directly contradicted by its own accuracy tables, which show larger accuracy drops for central models. Since the comparison is also confounded by unequal attack budgets, the paper as it stands does not establish its stated contribution; a corrected version with a reversed or removed comparative claim would be a more modest but still potentially valid study.","major_comments":[{"comment":"The paper's central claim that FL is more affected than centralized learning is contradicted by the accuracy data it reports. For the LSTM in Table II, centralized accuracy falls from 95.74% to 33.13% under FGSM (a 62.61-point drop) and to 22.36% under PGD (a 73.38-point drop), while FL accuracy falls from 91.6% to 36.6% (a 55.0-point drop) and to 23.5% (a 68.1-point drop), respectively. The same pattern holds for the Transformer in Table II and for both models in Tables I and III. Thus the data show larger absolute and relative accuracy degradation under centralized learning, not under FL. The abstract and the Conclusion (Section VI) make a claim that the paper's own results refute; this must be corrected before the paper can be considered.","section":"Abstract, V-D, VI, Tables I-III"},{"comment":"The comparison between central and FL training is confounded by unequal attack budgets. With 9 of 19 clients malicious and 30% of each malicious client's data perturbed, roughly 14.2% of the total training data is attacked, whereas the centralized setting attacks 30% of the total data. This means the two conditions are not comparable; if anything, the FL condition receives a smaller perturbation budget. The conclusions in Section V-D should be revised or the experiments re-run with matched budgets.","section":"V-D"},{"comment":"The Attack Success Rate as defined in Eq. (8) measures the fraction of samples whose predicted labels change after the attack, not the fraction that change from correct to incorrect. Since the FL models start from lower clean accuracy, a label flip can be a wrong-to-right change, so ASR is not a valid proxy for performance degradation. Arguments based on ASR (e.g., the claim in Section V-C that 'FGSM and PGD attacks led to a higher ASR for the FL model') must be replaced with accuracy or error-rate comparisons; in fact, Table I shows that for PGD the ASR is higher for the central model for both architectures, so even the ASR claim is not supported by the reported numbers.","section":"V-C, Eq. (8)"}],"minor_comments":[{"comment":"In the sentence 'followed by FSGM, label flipping, and random perturbations', 'FSGM' is a typo for 'FGSM'.","section":"V-D"},{"comment":"The class balance of the final anomaly detection dataset is not reported; please state the ratio of anomalous to normal samples, as it affects the reported metrics and the behavior of the focal loss.","section":"IV-A"},{"comment":"The label flipping attack is not described in Section III or IV; please specify how labels are flipped and for which clients and data.","section":"V-D"},{"comment":"The caption reads 'Accuracy for varied attach strengths'; 'attach' should be 'attack'.","section":"Fig. 4"},{"comment":"'Non-IDD' should be 'non-IID'.","section":"II-A"}],"recommendation":"reject","confidential_remarks":"The paper's main comparative claim is falsified by its own tables. The central result is not defensible, and the attack-budget mismatch would require re-running experiments even if the direction had been correct. I would encourage the authors to reframe the contribution around the observed vulnerability of FL to PGD and the sensitivity analysis, and to correct the budget mismatch in any future version."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know this paper before reading it: the abstract claims FL is more affected by adversarial attacks than centralized learning, but the reported numbers in Tables I–III show the opposite. Central LSTM accuracy under an FGSM training attack drops from 95.74% to 33.13%, while FL drops from 91.6% to 36.6%; under PGD, central goes to 22.36% and FL to 23.5%. The Transformer results and the 50-house dataset show the same pattern. The paper's reliance on ASR instead of accuracy is part of the problem: ASR merely counts prediction flips, which can be wrong-to-right flips, especially since FL starts from a lower clean accuracy.\n\nWhat is genuinely new here is applying known white-box attacks (FGSM, PGD) to federated anomaly detection on smart meter data, with LSTM and Transformer base models. The experimental setup is described clearly, including synthetic anomaly generation and the FL simulation. The finding that PGD is more damaging than FGSM is well supported and consistent across settings. The sensitivity analysis for attack strength and malicious client fraction is also sensible.\n\nThe soft spots are substantial. The central comparative claim is load-bearing and contradicted by the paper's own data. The attack budget mismatch in Section V-D compounds the issue: 9 of 19 clients attacking 30% of their data perturbs about 14% of total data, while the centralized condition perturbs 30%, so FL was tested under a weaker attack. No error bars or code are provided, so we cannot tell whether the small differences between models are noise. The literature review is adequate but does not engage with FL-specific adversarial defenses beyond citing a few works.\n\nWho gets value from this paper? Someone working on adversarial robustness of FL in energy applications might find the measurements useful as a cautionary data point, but the headline must be corrected. The paper deserves a serious referee—not to accept it as is, but because the empirical work is real and could be reframed as a straightforward study of attack effectiveness in FL without the unsupported central comparison. I would not cite it in its current form, and I would not bring it to a reading group until the authors fix the interpretation.\n\nRecommendation: send to peer review with a clear request to reconcile the abstract and conclusions with the accuracy tables, or to remove the FL-versus-central comparison entirely.","headline":"The paper's own accuracy tables contradict its headline: centralized models lose more accuracy than federated models under FGSM and PGD, so the central comparative claim is unsupported.","tokens_in":13366,"tokens_out":2481,"would_cite":false,"duration_ms":25189,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"PGD attacks hit federated energy models harder than central models","keywords":["federated learning","adversarial attacks","anomaly detection","energy consumption data","time series classification","PGD attack","FGSM attack","smart meter data"],"falsifier":"Repeat the training-attack experiment with the same fraction of total training data perturbed in both settings (for example, by attacking the same number of clients but a larger share of each client's data, or by attacking all 19 FL clients on a smaller share), and check whether the FL accuracy drop still exceeds the centralized drop at the same epsilon. If the gap vanishes, the claimed FL-specific vulnerability is an artifact of unequal attack budgets.","tokens_in":12300,"feed_emoji":"⚡","tokens_out":4253,"duration_ms":41405,"temperature":0.7,"pith_summary":"This paper asks whether federated learning (FL) for anomaly detection in smart-meter energy data is more vulnerable to adversarial attacks than centralized training. It tests two white-box attacks, FGSM and PGD, against LSTM and Transformer models, both during inference and during training with malicious clients. The authors find that PGD causes larger accuracy drops than FGSM, and that the FL setting suffers equal or greater degradation than centralized learning. This suggests that the distributed, privacy-preserving setup itself expands the attack surface and motivates the need for FL-specific defenses.","feed_headline":"PGD attacks hit federated energy models harder than central ones","feed_subtitle":"Federated anomaly detectors on smart-meter data degrade more than centralized models under white-box attacks, urging defenses.","key_machinery":"The central mechanism is the integration of white-box gradient-based attacks into the federated training loop: malicious clients compute FGSM or PGD perturbations on their local data using the current local model, train on the perturbed data, and send the resulting weights to the server, where FedAVG aggregates them into the global model. The iterative nature of PGD is the key factor that makes it more damaging than the single-step FGSM, because it refines the perturbation over multiple gradient steps.","core_discovery":"On the paper's own terms, adversarial attacks—especially iterative PGD—substantially degrade federated anomaly detection in energy consumption data. In experiments with 19 households, training with 9 malicious clients using PGD at epsilon 0.5 drops LSTM accuracy from 91.6% to 23.5%. The authors claim that FL is more sensitive to PGD than to FGSM because PGD's iterative refinement crafts stronger perturbations, and that FL is equally or more affected than centralized learning across both models and both 19- and 50-household datasets. Transformer consistently shows slightly more resilience than LSTM, but both are highly vulnerable, and the damage grows with attack strength and the fraction of malicious clients.","pith_inferences":["The FL-versus-central comparison in this paper is not apples-to-apples: attacking 9 of 19 clients on 30% of their data perturbs only about 14% of total training data, whereas the centralized baseline perturbs 30% of the total. A controlled experiment with identical perturbed-data fractions is needed to confirm that FL itself, rather than the budget mismatch, amplifies vulnerability.","Because the anomalies are synthetic, the models' vulnerability may partly depend on the synthetic anomaly distribution; testing on real labeled anomalies would strengthen the claim.","The same experimental setup could be used to evaluate defense mechanisms such as adversarial training, robust aggregation, or client filtering, providing a concrete testbed for future FL security research."],"forward_implications":["If federated learning is indeed more vulnerable, then privacy-preserving distributed learning for energy anomaly detection will require dedicated defenses beyond the standard FL training procedure.","Attack success scales with both the perturbation strength epsilon and the fraction of malicious clients, so threat models for smart-meter federations must consider partial client compromise as a realistic risk.","Transformer-based detectors appear somewhat more robust than LSTM-based ones, suggesting that architecture choice can partly mitigate adversarial vulnerability.","Because the same FL and attack machinery applies to other time-series anomaly detection tasks, these findings likely extend beyond the energy domain to any federated sensor or IoT setting."],"supporting_citations":[{"why":"Supplies the Fast Gradient Sign Method (FGSM), the single-step white-box attack used as one of the two main attack methods.","marker":"[17]"},{"why":"Supplies the Projected Gradient Descent (PGD) attack, the iterative white-box attack that the paper finds most damaging.","marker":"[18]"},{"why":"Provides the FedAVG aggregation algorithm that defines the federated setting and is the mechanism through which malicious-client weights propagate to the global model.","marker":"[34]"},{"why":"Establishes the federated LSTM approach for load forecasting that the paper adapts for anomaly detection in the FL setting.","marker":"[10]"},{"why":"Shows adversarial attacks can be transferred to time series classification, providing the basis for adapting FGSM and PGD to energy consumption data.","marker":"[19]"},{"why":"Supplies the Transformer-based load forecasting model that the paper uses as one of the two anomaly detection architectures.","marker":"[22]"}],"fun_headline_variants":["PGD attacks hit federated energy models hardest","Federated energy detection falls to iterative attacks","Energy FL accuracy drops 68 points under PGD","Federated models more vulnerable than central to attacks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The comparison between FL and centralized learning assumes that attacking 9 of 19 clients on 30% of their data is comparable to attacking 30% of the total data in the centralized setting, even though the FL condition actually perturbs only about 14% of total training data.","fun_headline_variants_meta":{"raw":{"variants":["PGD attacks hit federated energy models hardest","Federated energy detection falls to iterative attacks","Energy FL accuracy drops 68 points under PGD","Federated models more vulnerable than central to attacks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000344,"raw_usage":{"total_tokens":1895,"prompt_tokens":958,"completion_tokens":937,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":574,"completion_tokens_details":{"reasoning_tokens":876}},"tokens_in":574,"tokens_out":937,"duration_ms":10686,"temperature":1.0,"reasoning_tokens":876,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T20:27:11.380926+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Repeat the training-attack experiment with the same fraction of total training data perturbed in both settings (for example, by attacking the same number of clients but a larger share of each client's data, or by attacking all 19 FL clients on a smaller share), and check whether the FL accuracy drop still exceeds the centralized drop at the same epsilon. If the gap vanishes, the claimed FL-specific vulnerability is an artifact of unequal attack budgets.","supporting_citations":[{"cited_title":"Explaining and harnessing adversarial examples,","cited_arxiv_id":null,"evidence_quote":"Supplies the Fast Gradient Sign Method (FGSM), the single-step white-box attack used as one of the two main attack methods."},{"cited_title":"Towards deep learning models resistant to adversarial attacks,","cited_arxiv_id":null,"evidence_quote":"Supplies the Projected Gradient Descent (PGD) attack, the iterative white-box attack that the paper finds most damaging."},{"cited_title":"Communication-efficient learning of deep networks from decentralized data,","cited_arxiv_id":null,"evidence_quote":"Provides the FedAVG aggregation algorithm that defines the federated setting and is the mechanism through which malicious-client weights propagate to the global model."},{"cited_title":"Distributed load forecasting using smart meter data: Federated learning with recurrent neural networks,","cited_arxiv_id":null,"evidence_quote":"Establishes the federated LSTM approach for load forecasting that the paper adapts for anomaly detection in the FL setting."},{"cited_title":"Adversarial attacks on deep neural networks for time series classification,","cited_arxiv_id":null,"evidence_quote":"Shows adversarial attacks can be transferred to time series classification, providing the basis for adapting FGSM and PGD to energy consumption data."},{"cited_title":"Transformer-based model for electrical load forecasting,","cited_arxiv_id":null,"evidence_quote":"Supplies the Transformer-based load forecasting model that the paper uses as one of the two anomaly detection architectures."}],"review_version":1}