{"id":"62769846-709e-4555-a2ba-81094b515003","arxiv_id":"2506.06730","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"A federated, autoencoder-based fusion framework for EV charging station intrusion detection reports 98.9% accuracy on CICEVSE2024, but without baseline comparisons and with severe class imbalance, the headline result is not convincing.","lead":"Cyberattacks on electric vehicle charging stations are a growing concern. This paper proposes a privacy-preserving, federated learning system that fuses network and system logs to detect them, but the reported accuracy is close to a trivial baseline and existing methods are not compared.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 98%+ detection claim is not yet evidence of detection: the known class imbalance lets a constant all-attack classifier reach ~98.5% accuracy/precision on the network distribution, and no trivial baseline or external baseline is reported.","rationale":"The reader's weakest_assumption (arbitrary pairing in Eq. 3) is a genuine flaw and would invalidate the multimodal-fusion comparison in Fig. 2. However, the most load-bearing weakness for the paper's central claim is more basic: even a perfect pairing cannot save the reported 98% numbers because the class imbalance makes a no-feature constant classifier competitive. Table I shows attack samples dominate the network modality (131,580 attack vs 2,000 benign), so an all-attack predictor on that distribution achieves 98.50% accuracy and precision, within 0.4 points of the reported federated score. The absence of any external IDS baseline also means the abstract's 'outperforms existing solutions' is an assertion, not a result. I therefore disagree with the reader's framing that sample alignment is the weakest assumption; it is a serious secondary issue, but the majority-class problem is the load-bearing one. Because this reinforces the reader's REJECT verdict rather than changing it, I set verdict_should_be to UNCHANGED.","tokens_in":8520,"tokens_out":6599,"duration_ms":74094,"concrete_test":"Reproduce the authors' exact train/test partition (or require them to release it) and compute accuracy, precision, recall, and F1 for (a) a constant classifier that labels every test sample as attack and (b) a no-feature classifier using only the training class prior. If the constant classifier reaches or exceeds 98% accuracy/F1 on the same test set, the reported 98.91% federated result does not demonstrate detection. Also report per-class confusion matrices and the test-set attack prevalence.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's headline numbers are not evidence that the framework detects attacks. Table I gives 2,000 benign network samples vs 65,790 DoS and 65,790 Recon network samples; if the fused test set follows the same class proportions, a constant classifier that always predicts 'attack' attains accuracy = (65,790+65,790)/133,580 = 98.50% and precision = 98.50%, essentially matching the reported federated accuracy of 98.91% and F1 of 98.90% in Table IV. The authors never report the test-set class distribution, a majority-class baseline, or a per-class confusion matrix, so the reader cannot tell whether the model learned any attack signature. In addition, 'outperforms existing solutions' is asserted without running any external IDS: the only comparison in Section IV-B is against the same framework's single-modality variants. Even if the fusion alignment concern in Section III-B/Eq. 3 were completely resolved, the numerical claim would still be vacuous until a trivial baseline is ruled out on the exact experimental partition.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a federated, multimodal intrusion detection system for electric vehicle supply equipment (EVSE). The framework extracts features from network traffic logs and kernel/HPC event logs, compresses each modality with a dedicated autoencoder, concatenates the latent representations (intermediate fusion), and feeds the fused vector to a 1D CNN trained in a federated manner. Experiments on the CICEVSE2024 dataset report high accuracy (about 98.9% in the federated setting) and claim that the framework outperforms existing solutions. The paper includes comparisons between single-modality and fused models, and between centralized and federated training.","tokens_in":8755,"tokens_out":3669,"duration_ms":36578,"significance":"If the reported results were properly established, the paper would make a useful contribution: it addresses a relevant security problem for EV charging infrastructure, proposes a lightweight multimodal architecture suitable for edge deployment, and leverages federated learning to preserve data privacy. The use of a recent public dataset (CICEVSE2024) and the inclusion of both network- and host-level signals are strengths. However, the central empirical claim—that the framework achieves a detection rate above 98% and outperforms existing solutions—is not supported by the evidence as presented, because no trivial baseline or external comparison is reported and the class-imbalance issue is not addressed. The paper also does not explain how the two modalities are aligned during fusion.","major_comments":[{"comment":"The headline accuracy and F1-score (about 98.9%) are not compared against a trivial majority-class baseline. Table I shows that the network dataset contains 2,000 benign samples versus 131,580 attack samples (65,790 DoS and 65,790 Recon), so a constant all-attack classifier attains 98.50% accuracy on that distribution. The kernel/HPC dataset has a different benign count (32,303), so the fused distribution is unclear, but the paper never reports the class distribution of the training/test split used for the fused model, nor does it provide a confusion matrix or per-class precision/recall. Without this information, the reported >98% detection rate cannot be interpreted as evidence that the model learned actual attack signatures. The authors should report the test-set class balance, a majority-class baseline, and per-class metrics for each experimental setting.","section":"§IV-B, Table I, Fig. 3"},{"comment":"The fusion step concatenates latent vectors z_i^(1) and z_i^(2) for each station i, but Table I shows that the network and kernel/HPC datasets have different numbers of benign samples (2,000 versus 32,303). The paper never explains how samples from the two modalities are aligned or paired into a single fused observation. If the pairing is not based on a shared timestamp or event identifier, the fused vector may combine unrelated network and kernel states, making the reported detection accuracy meaningless. The authors need to specify the alignment procedure, or otherwise justify why unpaired samples can be concatenated.","section":"§III-B, Eq. (3), Table I"},{"comment":"The claim that the proposed framework 'outperforms existing solutions' is not supported by the experiments. The only comparisons are against the framework's own single-modality variants (Figure 2) and between centralized and federated versions of the same model (Table III). No external IDS baseline (e.g., a standard machine-learning classifier or a previously published EVSE intrusion detection method) is evaluated on the same data partition. The authors should either add such baselines or temper the claim to say that the framework outperforms its own single-modality variants.","section":"§IV-B, Figure 2"},{"comment":"The federated update rule is written as θ_{t+1} = θ_t − η Σ_{i=1}^N ∇θ L(...), which sums client gradients rather than averaging them and does not account for multiple local epochs. As written, the update magnitude grows with the number of clients, which is not the FedAvg algorithm described in the text and would be numerically unstable. The authors should correct the update rule to a proper FedAvg aggregation (e.g., averaging client updates or using local SGD with a server-side average) or clarify the exact aggregation scheme used in the experiments.","section":"§III-C, Eq. (7), Algorithm 1"}],"minor_comments":[{"comment":"The sentence 'By analyzing threats from multiple perspectives.' is an incomplete sentence fragment and should be merged with the preceding or following text.","section":"§III-A"},{"comment":"The label 'Fusionned' is a typo; it should be 'Fused'.","section":"Figure 2"},{"comment":"The header 'False Positive Rate ((%)' contains a doubled opening parenthesis; it should read 'False Positive Rate (%).'","section":"Table IV"},{"comment":"The sentence 'Algorithm 1 outlines the smart contract's on-chain aggregation.' refers to a smart contract and on-chain aggregation, but no smart contract or blockchain component is described anywhere else in the paper. This appears to be a copy-paste error and should be removed or replaced.","section":"§III-C (end)"},{"comment":"The paper reports results for 10 clients but does not explain how the CICEVSE2024 dataset is partitioned among the clients (e.g., by station, by time, or randomly). This detail is necessary to interpret the federated results and to assess the effect of non-IID data.","section":"§IV-B, Table III"},{"comment":"The conclusion states that the framework provides 'a scalable, secure solution' but the paper does not report training time, communication cost, or any scalability experiments beyond varying the number of clients from 2 to 10. A brief discussion of these costs would strengthen the scalability claim.","section":"§V"}],"recommendation":"major_revision","confidential_remarks":"The reader's report recommends rejection, and I agree that the evaluation as presented is seriously flawed. I chose major_revision rather than reject because the core issues—missing baselines, missing class-distribution reporting, and unclear fusion alignment—are potentially fixable within the scope of the manuscript if the authors can provide the necessary experiments and clarification. If the authors cannot provide a majority-class baseline and per-class metrics showing that the model truly outperforms a trivial classifier, or if they cannot explain the alignment of the two modalities, then the paper should be rejected. The 'outperforms existing solutions' claim in the abstract is also unsupported and should either be substantiated with external baselines or removed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a plausible engineering combo—per-modality autoencoder fusion plus a 1D CNN trained with federated learning—applied to a fresh public EVSE dataset. The writing is clear and the framework description is concrete. But the headline numbers don't demonstrate detection. The class imbalance alone lets a constant 'attack' predictor reach ~98.5% accuracy and ~99.2% F1 on the network distribution, which is the same range as the reported 98.91% accuracy / 98.90% F1. Without a majority-class baseline or per-class confusion matrix, the 98%+ claim is vacuous.\n\nWhat's genuinely useful: the paper uses the recent CICEVSE2024 dataset, describes the intermediate fusion of network and kernel/HPC embeddings without excessive complexity, and compares centralized vs. federated performance across client counts. That is a reasonable thing to examine. The method is not conceptually new—autoencoders + CNN + FL is a standard toolkit—but that alone wouldn't be a problem if the evaluation were solid.\n\nThe soft spots are real and central.\n\n1. Class imbalance. Table I shows 2,000 benign network samples versus 131,580 attack samples. An all-attack classifier on that distribution gives accuracy 98.50%, precision 98.50%, F1 99.24%—which beats the reported federated F1 of 98.90%. The paper reports only aggregate accuracy/F1/FPR, with no confusion matrix and no trivial baseline. The reader cannot tell whether the model learned any attack signature.\n\n2. No external baseline. The \"outperforms existing solutions\" claim comes from comparing the fused model to its own single-modality variants. That is an ablation, not a comparison against prior IDS methods. Nothing published elsewhere is run on the same partition.\n\n3. Fusion alignment. The network and kernel datasets have very different benign counts (2,000 vs. 32,303). Section III-B/Eq. 3 concatenates latent vectors, but the paper never explains how samples are paired. If pairing is arbitrary, the fused vector is not a coherent station state. This is potentially fixable, but it is unexplained.\n\n4. Minor: Algorithm 1 mentions \"smart contract's on-chain aggregation,\" yet there is no blockchain in the paper. Likely leftover text.\n\nWho is this for? A reader interested in the application of FL to EVSE security might skim it, but the empirical claims should not be cited as evidence. The idea deserves a proper evaluation, not this one.\n\nRecommendation: a serious editor could send this to peer review, because the framework is plausible and the dataset is public, but I would expect reviewers to require a majority-class baseline, external comparisons, and a clear fusion-alignment protocol. As submitted, the evidence does not support the abstract's claim.","headline":"The 98%+ detection claim is unsubstantiated: on the CICEVSE2024 network distribution an all-attack classifier already hits ~98.5% accuracy, and the paper reports no majority-class or external baseline.","tokens_in":9279,"tokens_out":2091,"would_cite":false,"duration_ms":19931,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A federated, multimodal intrusion detection framework fusing network traffic and kernel event logs detects attacks on EV charging stations with over 98% accuracy while keeping station data local.","keywords":["Electric Vehicle Supply Equipment","Intrusion Detection System","Federated Learning","Multimodal Data Fusion","Network Traffic Analysis","Kernel Events","CICEVSE2024","EVSE security"],"falsifier":"Inspect the data preprocessing to see how network and kernel samples are paired; if the pairing is random or one-to-many, re-run the experiments on timestamp-aligned samples. If accuracy drops to single-modality levels, the reported fusion gain is an artifact of misaligned data rather than genuine multimodal correlation.","tokens_in":8331,"feed_emoji":"🔌","tokens_out":8088,"duration_ms":75623,"temperature":0.7,"pith_summary":"The paper sets out to show that combining two complementary views of an electric vehicle charging station's activity, network traffic logs and low-level kernel and hardware performance events, yields an intrusion detector that is more accurate than either view alone. It proposes a pipeline in which each log type is compressed by its own autoencoder into a compact latent vector, the two vectors are concatenated into a fused representation, and a lightweight 1D CNN classifies the fused vector as benign or malicious. The CNN is trained by federated learning, so stations exchange model updates instead of raw logs. On the CICEVSE2024 dataset the paper reports that the fused local model reaches 92.91% accuracy, and the federated model stays above 98% accuracy and 97% precision across ten simulated stations. The intended payoff is a scalable, privacy-preserving defense for charging infrastructure, which has become an attractive target for ransomware and protocol-level attacks.","feed_headline":"Federated fusion IDS catches EV charger attacks at 98%+","feed_subtitle":"Network and kernel logs fuse locally; only model updates are shared, keeping station data private.","key_machinery":"The central object is the fused latent vector $z_i = f_{\\text{concat}}(z_i^{(1)}, z_i^{(2)})$, formed by concatenating the bottleneck representations of two modality-specific autoencoders. Each autoencoder $g_j$ compresses a feature vector $x_i^{(j)}$ extracted from one log source into a compact latent code, so fusion combines semantically compressed views rather than raw samples. A one-dimensional CNN maps the fused vector to an intrusion probability, and federated averaging aggregates only the CNN parameter updates across stations. This design is what lets the framework claim multimodal insight with a single lightweight classifier rather than parallel models per modality.","core_discovery":"The paper claims that intermediate fusion of autoencoder latent codes from network traffic and kernel/HPC events improves intrusion detection for EVSE compared with single-modality models, and that federated training preserves most of the centralized accuracy while keeping station data local. On CICEVSE2024, a locally trained fused model reaches 92.91% accuracy versus 92.21% for network-only and 90.54% for kernel-only models. With federated training across ten charging stations, accuracy and F1 remain around 98.9% with a false positive rate of 0.83%, close to the centralized model's near-99% performance. The authors attribute the small gap to local variations in data distribution and present the federated result as a practical balance between privacy and detection quality.","pith_inferences":["The paper does not isolate fusion from federated collaboration: the locally trained fused model scores 92.91% while federated training reaches 98.9%, so part of the headline gain may come from cross-station gradient sharing rather than from multimodal fusion.","The privacy claim assumes model updates leak nothing about local data; unless secure aggregation is added, gradient-inversion attacks could recover information, so the practical privacy guarantee is stronger than demonstrated.","Algorithm 1's caption mentions on-chain aggregation, but no blockchain or smart-contract component appears in the experiments; the evaluated system is standard federated averaging.","Applying the same pipeline to the power-consumption modality already present in CICEVSE2024 would be a direct test of the claimed extensibility, since the design treats modalities as interchangeable autoencoder inputs."],"forward_implications":["EVSE operators could train a shared intrusion detector across many charging stations without shipping raw traffic or kernel logs to a central server.","Attacks invisible in network traffic alone, such as local privilege escalation or backdoor installation, become detectable because kernel and HPC events are fused into the same model.","The single lightweight CNN and compact latent codes make the approach deployable on resource-constrained edge hardware that cannot run multi-model ensemble detectors.","The small accuracy drop between centralized and federated training is presented as the price of privacy, with the federated model still above 98% accuracy across 3 to 10 clients.","Because the fusion is modality-agnostic, the same framework extends to power-consumption and voltage logs if those are collected at stations."],"supporting_citations":[{"why":"Supplies the ensemble-learning EVSE intrusion detector whose multi-sub-model design the paper contrasts with its single lightweight CNN.","marker":"[6]"},{"why":"Presents the CNN-A-LSTM IoT charging-station IDS that motivates the simpler 1D CNN architecture and serves as a related baseline.","marker":"[7]"},{"why":"Demonstrates injection attacks on V2G communication, one of the attack classes the multimodal approach is meant to catch.","marker":"[8]"},{"why":"Introduces federated learning for zero-day attack detection in V2X networks, the privacy-preserving training approach the framework adopts.","marker":"[15]"},{"why":"Provides a federated open-set detection framework for the Internet of Vehicles, a precursor for decentralized IDS design.","marker":"[17]"},{"why":"Surveys host-based intrusion detection, supporting the kernel/HPC log modality as complementary to network traffic.","marker":"[18]"},{"why":"Provides the CICEVSE2024 multimodal EVSE dataset with network, HPC/kernel, and power data on which all reported experiments run.","marker":"[24]"}],"fun_headline_variants":["Federated multimodal IDS catches EV charger attacks at 98%","Privacy-preserving fusion of network and kernel logs detects EVSE attacks","Fusing local network and kernel data, federated IDS hits 98% EVSE detection","EV charger security: federated fusion outsmarts single-source attack detectors","Local fusion, federated updates: 98% detection for EVSE intrusions"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the 2,000 benign network samples and the 32,303 benign kernel samples can be paired as simultaneous observations of the same station state before their latent vectors are concatenated, but the paper never describes how this matching is done.","fun_headline_variants_meta":{"raw":{"variants":["Federated multimodal IDS catches EV charger attacks at 98%","Privacy-preserving fusion of network and kernel logs detects EVSE attacks","Fusing local network and kernel data, federated IDS hits 98% EVSE detection","EV charger security: federated fusion outsmarts single-source attack detectors","Local fusion, federated updates: 98% detection for EVSE intrusions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00083,"raw_usage":{"total_tokens":3611,"prompt_tokens":914,"completion_tokens":2697,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":530,"completion_tokens_details":{"reasoning_tokens":2595}},"tokens_in":530,"tokens_out":2697,"duration_ms":19816,"temperature":1.0,"reasoning_tokens":2595,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T05:50:54.434864+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Inspect the data preprocessing to see how network and kernel samples are paired; if the pairing is random or one-to-many, re-run the experiments on timestamp-aligned samples. If accuracy drops to single-modality levels, the reported fusion gain is an artifact of misaligned data rather than genuine multimodal correlation.","supporting_citations":[{"cited_title":"Enhancing anomaly detection in electric vehicle supply equipment (evse) networks using classical and ensemble learning approaches,","cited_arxiv_id":null,"evidence_quote":"Supplies the ensemble-learning EVSE intrusion detector whose multi-sub-model design the paper contrasts with its single lightweight CNN."},{"cited_title":"A novel ensembling of cnn-a-lstm for iot electric vehicle charging stations based on intrusion detection system,","cited_arxiv_id":null,"evidence_quote":"Presents the CNN-A-LSTM IoT charging-station IDS that motivates the simpler 1D CNN architecture and serves as a related baseline."},{"cited_title":"Injection attacks and detection strategy in front-end vehicle-to-grid communication,","cited_arxiv_id":null,"evidence_quote":"Demonstrates injection attacks on V2G communication, one of the attack classes the multimodal approach is meant to catch."},{"cited_title":"Federated learning for zero-day attack detection in 5g and beyond v2x networks,","cited_arxiv_id":null,"evidence_quote":"Introduces federated learning for zero-day attack detection in V2X networks, the privacy-preserving training approach the framework adopts."},{"cited_title":"Zero-x: A blockchain-enabled open-set federated learning framework for zero-day attack detection in iov,","cited_arxiv_id":null,"evidence_quote":"Provides a federated open-set detection framework for the Internet of Vehicles, a precursor for decentralized IDS design."},{"cited_title":"A systematic literature review on host-based intrusion detection systems,","cited_arxiv_id":null,"evidence_quote":"Surveys host-based intrusion detection, supporting the kernel/HPC log modality as complementary to network traffic."},{"cited_title":"En- hancing ev charging station security using a multi-dimensional dataset: Cicevse2024,","cited_arxiv_id":null,"evidence_quote":"Provides the CICEVSE2024 multimodal EVSE dataset with network, HPC/kernel, and power data on which all reported experiments run."}],"review_version":1}