{"id":"8384fa15-4273-48fb-a791-7abce97d2f41","arxiv_id":"2506.17329","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Network traffic features dominate intrusion detection in the WUSTL-EHMS-2020 Healthcare 5.0 dataset, while biomedical features, especially temperature, add signal for detecting spoofing attacks.","lead":"This paper applies explainable AI to a healthcare IoT dataset to see which data, network flows or vital signs, drive intrusion detection. It finds network features dominate overall, but body-temperature readings help detect spoofing attacks, and XGBoost reaches 99% F1 on benign and data-alteration traffic.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Biomedical-feature relevance for spoofing is inferred only from SHAP on a single all-feature model; without feature-group ablation or error bars, the claim that hybrid cyber-biomedical IDS improves spoofing detection is unsupported.","rationale":"The paper is a small empirical study with a plausible but under-validated central claim. I agree with the reader's weakest-assumption analysis: the conclusion that biomedical features contribute to spoofing detection is drawn from SHAP values on one all-feature model, with no ablation or feature-group baseline. My independent reading finds no additional objection that would change the conditional verdict. The paper does provide public code and uses standard XGBoost and SHAP tools, which is positive, but it does not supply the missing evidence needed to support the claim that hybrid cyber-biomedical systems improve spoofing detection. The future-work statement in Section 6 explicitly acknowledges that a three-group XAI comparison is still to be done. Therefore I recommend keeping the reader's CONDITIONAL verdict: the claim becomes credible only after the feature-group ablation and repeated-split experiments are performed.","tokens_in":8871,"tokens_out":2689,"duration_ms":31771,"concrete_test":"Retrain the XGBoost pipeline (same preprocessing, LabelEncoder, StandardScaler, 80/20 split) under three feature sets: network-only (35 flow features), biomedical-only (8 sensor features), and combined (43 features), using e.g. 5 repeated stratified splits or 5-fold cross-validation. Report spoofing precision, recall, and F1-score for each set, plus the mean absolute SHAP value of Temperature in each repeated run with paired bootstrap confidence intervals. Also compute permutation importance for the biomedical feature group in the combined model. If removing the 8 biomedical features leaves spoofing F1 or recall statistically unchanged, or if Temperature's SHAP magnitude does not replicate across splits, the paper's central claim fails; if spoofing recall drops significantly and the SHAP results replicate, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that biomedical features meaningfully aid spoofing detection in Healthcare 5.0—rests entirely on interpreting SHAP summary plots from a single XGBoost model trained on all 43 features (Section 5.2). No ablation, no network-only or biomedical-only model, and no statistical comparison is reported. SHAP attributions on a correlated, imbalanced multiclass dataset do not by themselves establish that the biomedical feature group contributes predictive signal; attributions can be redistributed among correlated features, and with only one unseeded 80/20 split (Section 4.2) there is no evidence that the effect is stable. Section 6 even lists as future work 'XAI analysis across biomedical, network, and combined data, exploring three different groups,' acknowledging that this comparison is absent. Therefore the paper's headline implication—hybrid cyber-biomedical IDS should improve spoofing detection—is not currently supported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper evaluates intrusion detection on the WUSTL-EHMS-2020 Healthcare 5.0 dataset, which combines network flow metrics with biomedical sensor readings. Four classifiers are trained on an 80/20 split with standard preprocessing, and XGBoost is reported as the best performer, achieving F1-scores of 0.99 for benign and data alteration classes and 0.81 for spoofing. The authors then apply SHAP to the XGBoost model and interpret the resulting summary plots to claim that network features dominate intrusion detection overall, while biomedical features, especially temperature, contribute meaningfully to spoofing detection. The paper concludes that hybrid cyber-biomedical IDS designs are needed for Healthcare 5.0 and lists future work including a three-way XAI comparison of biomedical, network, and combined data.","tokens_in":8986,"tokens_out":3747,"duration_ms":45195,"significance":"If the central claim were adequately supported, the paper would provide a useful, concrete indication that physiological sensor data can improve spoofing detection in healthcare IoT environments, a setting where prior IDS work has focused almost exclusively on network traffic. The manuscript has tangible strengths: the use of a relevant public dataset, a clearly described preprocessing pipeline, a reproducibility link to public code, and a straightforward multiclass evaluation across four classifiers with per-class precision, recall, and F1. The classification results themselves are plausible and reported transparently. However, the paper's headline contribution—that biomedical features contribute to spoofing detection—rests entirely on qualitative inspection of SHAP plots from a single all-feature model. Without feature-group ablations, biomedical-only or network-only models, multiple random splits, or statistical comparison of attribution patterns, the SHAP interpretations do not establish that biomedical features add predictive value for spoofing.","major_comments":[{"comment":"The central claim that biomedical features contributed to spoofing detection is supported only by SHAP summary plots from a single XGBoost model trained on all features. No ablation removes the biomedical feature group, no biomedical-only or network-only model is trained, and no statistical test compares SHAP attributions across feature groups. In an imbalanced multiclass setting with correlated features, SHAP values describe the trained model's internal allocation of credit and do not by themselves establish that a feature group would improve detection if added or removed. The paper's own future-work statement in Section 6—\"perform XAI analysis across biomedical, network, and combined data, exploring three different groups\"—acknowledges that this comparison is absent. This missing experiment is load-bearing for the abstract's implication that hybrid cyber-biomedical IDS should improve spoofing detection.","section":"Section 5.2 and Section 6"},{"comment":"The abstract reports that temperature reached a Shapley values magnitude of 0.37, but this number does not appear anywhere in Section 5.2 or in any figure or table. The body text only says that temperature \"exhibits a wide range of SHAP values.\" The authors should state the exact aggregation used (mean absolute SHAP, maximum per-sample value, or another quantity), the class or model output to which it applies, and provide the corresponding value in the results section, otherwise the headline quantitative claim is unverifiable from the manuscript.","section":"Abstract and Section 5.2"},{"comment":"The experimental protocol uses a single unseeded 80/20 split with no repeated runs, no cross-validation, and no reported XGBoost hyperparameters. Given the extreme variability in spoofing recall across classifiers (0.01 for SVC, 0.07 for RF, 0.67 for DT, 0.72 for XGB), the reported XGBoost advantage and the resulting SHAP interpretations could be sensitive to the particular split. The authors should add repeated stratified cross-validation or bootstrapped confidence intervals for the classification metrics and should state the random seed and hyperparameter settings needed to reproduce Figure 3 and Figure 4.","section":"Section 4.2 and Table 2"},{"comment":"The feature Source Port (Sport) is encoded with LabelEncoder and then treated as a numeric input to tree-based models, which imposes an arbitrary ordering on port numbers. Since Sport is reported as a top feature for benign and spoofing classification, this encoding choice may materially affect the SHAP analysis. The authors should justify this encoding or use a categorical-aware encoding, and they should clarify the sentence \"Without this transformation, features such as Source Port would contain NaN values,\" since LabelEncoder does not handle missing values and the invalid entries were already removed earlier in the same paragraph.","section":"Section 4.2 and Figure 4"}],"minor_comments":[{"comment":"The phrase \"As healthcare becomes increasing reliant on digital solutions\" contains a typo: \"increasing\" should be \"increasingly.\"","section":"Section 1"},{"comment":"The text states that biomedical features \"demonstrate significant contributions\" and \"are highly relevant,\" but no quantitative threshold, statistical test, or comparison to a null baseline is provided; the wording should be revised to match the descriptive nature of the SHAP plots.","section":"Section 5.2"},{"comment":"The captions and text do not specify whether the summary bar plot uses mean absolute SHAP values or another aggregation; this should be stated explicitly to make the plots interpretable.","section":"Figure 3 and Figure 4"},{"comment":"After describing the 45 features and the removal of specific columns, the manuscript does not restate the final number of features used for model training; a brief feature-count summary after preprocessing would improve reproducibility.","section":"Section 4.1"},{"comment":"The sentence describing why LabelEncoder is needed says \"Without this transformation, features such as Source Port would contain NaN values,\" which is confusing because label encoding does not impute missing values; this should be rephrased or corrected.","section":"Section 4.2"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is within the scope of a cybersecurity venue and has a reproducible, clearly described pipeline, but the central claim about biomedical feature relevance for spoofing detection is currently supported only by qualitative SHAP interpretation. The gap is explicitly acknowledged in the future-work section. I recommend major revision, with the expectation that the authors add feature-group ablations or equivalent comparative experiments, report repeated-split variability, and move the headline SHAP magnitude into the results section with a precise definition."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a small, honest empirical study that does one thing usefully—it reports per-attack-class SHAP for a public cyber-biomedical dataset—and then overinterprets it. The XGBoost numbers are plausible and the code is public; the preprocessing is described clearly enough to reproduce. The observation that Temp, SpO2, and a few other physiological features show up in the spoofing class's SHAP plot is worth recording. That is a real, if modest, addition over prior work that mostly looked at aggregate rankings.\n\nThe soft spots are in proportion. The central claim—biomedical features are 'highly relevant' for spoofing detection—rests entirely on reading SHAP plots from a single all-feature XGBoost model. There is no ablation that removes the biomedical group, no network-only or biomedical-only model, no statistical test comparing attribute distributions, and no error bars: one unseeded 80/20 split. On a dataset with 87.5% benign samples and correlated features, SHAP attributions can shift arbitrarily between correlated predictors. So the headline implication—that hybrid cyber-biomedical IDS should improve spoofing detection—is not supported by the experiments. The abstract also cites a Shapley magnitude of 0.37 for temperature, but that number does not appear in the body; either report it or drop it. In the related work, the claim that prior XAI studies overlooked biomedical features is a bit strong—Aljuhani et al. already flagged blood pressure and SpO2, though not a per-class spoofing breakdown.\n\nThe paper itself half-admits the gap: Section 6 lists as future work the XAI analysis across biomedical, network, and combined data. That comparison is exactly what would validate the title. As written, the title overpromises—'performance of cyber-biomedical features' is not actually evaluated separately.\n\nNone of this is fatal. The paper is not circular or deceptive; it is a descriptive case study with a clear method and a testable hypothesis. It just needs the right experiments to back the claim. A referee should ask for feature-group ablation, a few seeded splits with error bars, and a table of the exact SHAP values behind the abstract's numbers.\n\nWho is this for? Practitioners working with WUSTL-EHMS-2020 who want a starting point for XAI analysis. It is not a definitive answer on feature group value. I'd send it to peer review—there's enough here to warrant the extra work—but with the expectation of major revision.","headline":"Per-class SHAP on WUSTL-EHMS-2020 is a useful look, but the claim that biomedical features boost spoofing detection is an interpretation, not a result.","tokens_in":9599,"tokens_out":3685,"would_cite":false,"duration_ms":36152,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Biomedical sensor data carry real predictive weight for detecting spoofing attacks in Healthcare 5.0 intrusion detection, with temperature reaching a Shapley magnitude of 0.37.","keywords":["machine learning","intrusion detection","Healthcare 5.0","explainable AI","SHAP","WUSTL-EHMS-2020","biomedical sensor data","spoofing detection"],"falsifier":"If an XGBoost model trained only on the 35 network features achieves the same spoofing F1-score and recall as the hybrid model, the claimed contribution of biomedical data would be refuted; concretely, retrain with the eight biomedical features removed and compare spoofing recall, which the paper reports as 0.72 in the hybrid setting.","tokens_in":8648,"feed_emoji":"🩺","tokens_out":4752,"duration_ms":49311,"temperature":0.7,"pith_summary":"This paper claims that in intrusion detection for Healthcare 5.0, network traffic features dominate overall, but biomedical physiological signals meaningfully contribute to detecting spoofing attacks. It reports that XGBoost achieves 99% F1-score for benign and data alteration traffic and 81% for spoofing, and that temperature reaches a Shapley value magnitude of 0.37 in the explainability analysis. If this claim is correct, hybrid intrusion detection systems that combine network and biomedical data should detect spoofing attacks better than network-only systems, and current datasets that omit physiological data are missing relevant signal. The paper supports this by applying SHAP to a single dataset that merges network flow metrics with biometric measurements.","feed_headline":"Temperature data help catch spoofing attacks in healthcare IoT","feed_subtitle":"Network traffic still drives detection, but body signals add real weight for the hardest attack class, spoofing.","key_machinery":"The central machinery is SHAP (SHapley Additive exPlanations), applied to a trained XGBoost classifier. SHAP assigns each feature a Shapley value from game theory, quantifying how much that feature pushed the model's prediction toward each class, and the paper uses summary bar plots and summary plots to rank and visualize feature contributions across samples. The dataset under analysis, WUSTL-EHMS-2020, combines 35 network flow metrics with 8 biometric measurements across 16,315 samples, and the SHAP analysis on the best-performing classifier is what carries the argument that biomedical features matter for spoofing detection.","core_discovery":"The central discovery is that combining network flow metrics with biomedical sensor data in an intrusion detection model, and explaining predictions with SHAP, reveals a division of labor: network features are the primary drivers of detection across all attack classes, while biomedical features such as temperature, pulse rate, oxygen saturation, respiratory rate, and ECG ST segment contribute specifically to spoofing detection. The paper's evidence is the SHAP summary plots for the three classes, where biomedical features appear among the most influential for the spoofing label, and the quantitative claim that temperature reaches a Shapley magnitude of 0.37. The authors interpret this as indicating that physiological signals should not be overlooked when securing connected medical environments, particularly for cyber-physical attacks that manipulate biometric data.","pith_inferences":["The paper's evidence for biomedical relevance is correlational: it relies on SHAP values from a single model trained on all features, so a natural test is an ablation study that removes the eight biomedical features and checks whether spoofing recall (reported as 0.72) actually drops.","The temperature-spoofing association may reflect a testbed artifact rather than a physiological invariant, since the dataset was collected in one controlled environment with a single attacker topology; re-running the same analysis on other IoMT datasets would clarify whether the 0.37 magnitude is general.","A direct experimental extension, not performed in the paper, would be to train network-only, biomedical-only, and hybrid classifiers and compare their class-wise F1 scores; such a comparison would separate the contribution of each feature group decisively.","The large gap in spoofing detection across classifiers (recall 0.72 for XGBoost versus 0.07 for random forest and 0.01 for SVC) suggests that spoofing patterns are subtle and model-dependent, so the benefit of biomedical features may also depend heavily on the chosen learning algorithm."],"forward_implications":["If biomedical features genuinely sharpen spoofing detection, then intrusion detection datasets for healthcare that omit physiological sensor data are discarding signal relevant to a common attack class.","Hybrid cyber-biomedical intrusion detection systems should target spoofing in particular, since it is the hardest class (F1 0.81 with XGBoost, far below the 0.99 for benign and data alteration).","SHAP rankings can identify which specific physiological signals matter most, directing where to add sensor redundancy or tamper detection in wearable and implantable devices.","Network-only intrusion detection models, which are standard in much of the literature, may have a systematic blind spot for spoofing that only becomes visible when biomedical data are included in the training set.","Explainability analysis of this kind can bridge the gap between cybersecurity and clinical monitoring, allowing security models to exploit physiological context already being collected for patient care."],"supporting_citations":[{"why":"supplies the WUSTL-EHMS-2020 dataset that merges network and biomedical features, the empirical foundation of the whole study.","marker":"[Hady et al. 2020]"},{"why":"provides the SHAP method used to compute feature attributions, the central explanatory tool of the paper.","marker":"[Lundberg and Lee 2017]"},{"why":"provides the XGBoost algorithm, the classifier whose SHAP values carry the paper's main result.","marker":"[Chen and Guestrin 2016]"},{"why":"earlier SHAP analysis on the same dataset focusing on network features, the gap this paper aims to close by examining biomedical features.","marker":"[Alani et al. 2023b]"},{"why":"prior work on the same dataset that ranked biomedical features, providing context the paper extends with a class-specific interpretation.","marker":"[Aljuhani et al. 2024]"},{"why":"documents vulnerabilities in biometric and wearable-device systems, supporting the interpretation that anomalous physiological readings can indicate tampering.","marker":"[Khan et al. 2020]"}],"fun_headline_variants":["Body sensors boost spoofing detection in healthcare networks","Temperature readings aid intrusion detection in medical IoT","Biomedical data add edge against spoofing in healthcare 5.0","Network still leads, but body signals spot spoofing attacks","Shapley shows temperature key for spoofing class in healthcare"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper assumes that the importance scores it computes for each input reflect what the model actually relies on, without testing a version of the model that leaves the biomedical data out.","fun_headline_variants_meta":{"raw":{"variants":["Body sensors boost spoofing detection in healthcare networks","Temperature readings aid intrusion detection in medical IoT","Biomedical data add edge against spoofing in healthcare 5.0","Network still leads, but body signals spot spoofing attacks","Shapley shows temperature key for spoofing class in healthcare"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000234,"raw_usage":{"total_tokens":1442,"prompt_tokens":837,"completion_tokens":605,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":453,"completion_tokens_details":{"reasoning_tokens":520}},"tokens_in":453,"tokens_out":605,"duration_ms":6562,"temperature":1.0,"reasoning_tokens":520,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T23:44:14.714966+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"If an XGBoost model trained only on the 35 network features achieves the same spoofing F1-score and recall as the hybrid model, the claimed contribution of biomedical data would be refuted; concretely, retrain with the eight biomedical features removed and compare spoofing recall, which the paper reports as 0.72 in the hybrid setting.","supporting_citations":[{"cited_title":"A., Ghubaish, A., Salman, T., Unal, D., and Jain, R","cited_arxiv_id":null,"evidence_quote":"supplies the WUSTL-EHMS-2020 dataset that merges network and biomedical features, the empirical foundation of the whole study."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"provides the SHAP method used to compute feature attributions, the central explanatory tool of the paper."},{"cited_title":"and Guestrin, C","cited_arxiv_id":null,"evidence_quote":"provides the XGBoost algorithm, the classifier whose SHAP values carry the paper's main result."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"prior work on the same dataset that ranked biomedical features, providing context the paper extends with a class-specific interpretation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"documents vulnerabilities in biometric and wearable-device systems, supporting the interpretation that anomalous physiological readings can indicate tampering."}],"review_version":1}