{"id":"a0245aca-af6e-495b-8955-e2ec7dc30cc2","arxiv_id":"2505.14659","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Applying three explainable AI tools to a random forest model on the WUSTL-HDRL-2024 dataset yields qualitative feature-level explanations aimed at attack detection in 6G healthcare networks, but the validation is limited to a single sample.","lead":"The authors apply SHAP, LIME, and DiCE to a random forest intrusion detector trained on the WUSTL-HDRL-2024 medical network dataset, claiming these explanations can help secure 6G-connected healthcare. The evidence is a qualitative demonstration on one test sample, not a quantitative validation.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"SHAP and LIME visibly disagree on the single sample used to prove cross-validation; the consistency claim is unsupported.","rationale":"The reader's weakest assumption correctly identified that agreement among XAI methods on one random instance is taken as evidence of correctness, with no aggregate statistics or quantitative metric. My stress-test strengthens this into a concrete internal contradiction: the SHAP and LIME outputs for that single instance visibly conflict on 'scputimes_idle', which is precisely the feature the paper claims is cross-validated. This is not a matter of missing external ground truth; even by the paper's own displayed evidence, the two methods disagree. The verdict remains CONDITIONAL: the paper is a plausible exploratory application of standard XAI tools to a medical network dataset, and the specific gaps are fixable by reporting multi-sample agreement statistics and reconciling sign conflicts. I do not see a basis for REJECT, since the claimed novelty (applying SHAP/LIME/DiCE to WUSTL-HDRL-2024) is real and the experimental scaffolding is reproducible, but the central consistency claim is currently unsupported by the presented analysis.","tokens_in":8633,"tokens_out":2712,"duration_ms":24676,"concrete_test":"Reproduce the experiment on WUSTL-HDRL-2024 with the same preprocessing (SMOTE, 2,000 records per class) and train the Random Forest. For N=100 randomly selected test instances, compute SHAP and LIME feature attributions. Report (a) the sign agreement rate on features that both methods return, especially 'scputimes_idle'; (b) the Spearman rank correlation between SHAP and LIME importance vectors per instance; (c) the fraction of instances where the top-5 feature sets overlap by at least 3 features; and (d) whether the paper's single sample is typical under these metrics. If sign agreement is not significantly above 50%, or if 'scputimes_idle' frequently flips sign, the paper's consistency claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that cross-validating SHAP, LIME, and DiCE confirms their reliability and consistency (Section V). The only evidence is a narrative comparison on one randomly chosen test instance, and on that instance the methods do not actually agree. The SHAP force plot (Fig. 4) treats high 'scputimes_idle' as supporting the 'Normal' prediction, while the LIME explanation (Fig. 5) assigns negative weight to the same feature for the same prediction, and Section V concedes that excessive idle time 'may indicate underutilization, potentially masking malicious processes.' This is a direct sign flip on a shared feature for the single sample that is supposed to establish consistency. The paper attempts to reconcile this by saying 'extreme deviations can imply anomalies,' but that threshold-based narrative is not derived from the displayed data. Moreover, no quantitative agreement metric, aggregate statistics over many samples, or comparison to any ground truth is provided; DiCE highlights a largely disjoint feature set (SrcBytes, DstBytes, jitter), so 'cross-validation' reduces to a post hoc qualitative story. Because the entire support for the strongest claim is this one sample, the observed contradiction directly undermines the conclusion that the XAI methods are reliable and consistent.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper argues that explainable AI (XAI) techniques—SHAP, LIME, and DiCE—can address security challenges in 6G-enabled healthcare by explaining the decisions of a machine-learning intrusion detector on the WUSTL-HDRL-2024 IoMT dataset. It surveys 6G usage scenarios and their security implications, trains several classifiers on a balanced subsample, selects Random Forest, and then applies the three XAI methods to a single test instance. The central claim is that cross-validating these XAI methods on that instance confirms their reliability and consistency, thereby fostering trust in AI-driven security solutions.","tokens_in":8983,"tokens_out":4174,"duration_ms":40481,"significance":"The topic is timely and practically important: 6G healthcare networks create new attack surfaces, and interpretable security models are needed. The paper's strengths include using a recent medical-specific intrusion dataset, applying three complementary XAI families (attribution, local surrogate, and counterfactual), and reporting standard predictive metrics such as 99.85% accuracy for Random Forest. If the cross-validation claim could be supported by multi-sample quantitative agreement analysis and external ground truth, the paper would be a useful case study in XAI-based security for IoMT. As written, however, the empirical contribution is a single-instance narrative, and the claimed consistency is not demonstrated.","major_comments":[{"comment":"The central claim that cross-validating SHAP, LIME, and DiCE confirms reliability and consistency rests on one randomly selected test instance. No aggregate statistics over multiple instances, no error bars or confidence intervals, and no quantitative agreement metric (e.g., sign agreement, feature-rank correlation) are provided. A single anecdote cannot establish that the explanations are reliable; the authors should report agreement statistics across a representative set of test samples, together with variability measures.","section":"Section V, Figs. 4–6"},{"comment":"The displayed evidence actually contradicts the consistency claim on a shared feature. SHAP is described as treating high 'scputimes_idle' as positively supporting the prediction ('High idle time ... typical in a Normal state'), whereas LIME assigns negative weight to excessive idle time and Section V concedes that it 'may indicate underutilization, potentially masking malicious processes.' The attempted reconciliation—'extreme deviations can imply anomalies'—is not derived from the displayed data and no threshold is given; this sign flip on the same feature for the same sample directly undermines the claimed agreement.","section":"Section V, Figs. 4–5"},{"comment":"The three methods appear to explain different predictions, which invalidates the intended cross-validation. Section IV-B defines the binary encoding as attack=1 and normal=0, yet the SHAP force plot is reported with a final prediction of '1.00' while the accompanying bullets interpret the features as supporting 'Normal'; LIME's plot predicts class [0] (Normal); DiCE's original sample is described as predicted 'Attack' (1). Unless the class labels are explicitly reconciled, the methods are being compared on incomparable explanations.","section":"Section IV-D and Section V"},{"comment":"The experimental setup lacks the detail needed to assess the reliability of the reported results. The paper does not specify the train/test split, the random seed, or whether SMOTE was applied before or after the split. Table I is ambiguous about the counts after SMOTE: the 'Total Count (After SMOTE)' and 'Class Instance (After SMOTE)' columns are not clearly separated, and the relationship between the stated 2,000-record subsample and the per-class counts shown is unclear. This matters because class-balancing and potential data leakage can inflate accuracy and change the explanations produced by XAI methods.","section":"Section IV-A–C, Table I"}],"minor_comments":[{"comment":"The manuscript contains numerous typos and formatting issues, including 'Explaination' in the DiCE heading, 'Fig1' for Fig. 1, inconsistent use of 'a nd' and 'thr ough', and the caption for Fig. 2 referring to a visualization whose rendered output is not clearly included.","section":"Throughout"},{"comment":"The paper relies heavily on the authors' own prior work [18], [19], and [20] to motivate the claim that XAI strengthens security and transparency; independent sources would strengthen the motivation and avoid the appearance of self-referential support.","section":"Section I and References"},{"comment":"A brief description of hyperparameters for the Random Forest model (e.g., number of trees, depth) and the CNN architecture (e.g., activation functions, training epochs) would improve reproducibility; currently only the dense-layer sizes are given.","section":"Section IV-C"},{"comment":"The manuscript says 'we selected a random test sample' but does not state which sample, why it was chosen, or how the different XAI tools were configured (e.g., LIME kernel width, DiCE diversity weight). This makes the qualitative comparison impossible to reproduce.","section":"Section IV-D"},{"comment":"The concluding sentence of Section V states that 'normal traffic is characterized by stable CPU usage, typical network activity, and sufficient resources,' but these characterizations are derived from the single sample and would need aggregate evidence to be presented as general findings.","section":"Section V"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a relevant problem, but its main empirical claim—that cross-validating three XAI methods confirms their reliability—is currently supported only by one sample and the displayed results actually show a direct sign disagreement between SHAP and LIME on a shared feature. This is a load-bearing issue that requires new experiments (multi-sample quantitative agreement, and ideally ground-truth or simulated-attack validation) rather than text-level revision. I also note the unusually heavy reliance on the authors' own prior papers for the motivational framing; independent references would be more persuasive."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper is an exploratory application of SHAP, LIME, and DiCE to the WUSTL-HDRL-2024 healthcare IoT dataset. That's fine as far as it goes, but the main claim—that cross-validating these methods confirms their reliability and consistency—does not survive contact with the evidence. On the single test sample used, SHAP and LIME visibly disagree: SHAP treats high 'scputimes_idle' as supporting 'Normal,' while LIME assigns it negative weight and the text even concedes it could mask malicious processes. The paper's reconciliation ('extreme deviations can imply anomalies') is a post hoc story, not a quantitative or even a principled threshold. DiCE mostly highlights a different feature set (SrcBytes, DstBytes, jitter). So the 'cross-validation' is really just three narratives loosely stitched together in Section V.\n\nWhat the paper does well: the survey of 6G usage scenarios and their healthcare security implications is clear and well organized. The preprocessing (SMOTE, balancing, dropping zero-variance features) is sensible. Random Forest performance is reported with standard metrics and looks solid. The authors are honest about their own prior work; the self-citations are not the problem.\n\nSoft spots, in order of severity. First, the validation of the XAI results is a single randomly chosen test instance. No error bars, no aggregate statistics over many samples, no quantitative agreement metric. The paper's strongest conclusion rests on a sample size of one. Second, the sign flip on 'scputimes_idle' is a direct internal contradiction, not a minor discrepancy. If the goal is to show consistency, this sample does the opposite. Third, agreement among explanation methods is not evidence of correctness without an independent ground truth; the paper uses agreement itself as the proof, which is circular. Fourth, the novelty is thin: the methods are established, and the authors' own prior work applies SHAP and LIME in 6G/IoT contexts. Adding a new dataset and DiCE is a modest increment.\n\nWho benefits? Someone looking for a textbook-style example of applying XAI to a medical network IDS, or a class discussion of why method agreement is not validation. As a research contribution, the core claim fails, and the gaps are fixable only with substantial new analysis.\n\nRecommendation: desk reject. The topic is timely, but the central evidence is a single contradictory sample, and the paper would need major rework—many samples, a real consistency metric, and a defensible notion of ground truth—before it deserves referee time.","headline":"Routine XAI application to a new dataset, but the central claim that three explanation methods cross-validate each other is contradicted by the paper's own plots.","tokens_in":9385,"tokens_out":2363,"would_cite":false,"duration_ms":22854,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Three explanation tools can jointly expose attack-signaling features in 6G medical networks, the paper argues.","keywords":["explainable AI","SHAP","LIME","DiCE","6G networks","healthcare security","IoMT","counterfactual explanations"],"falsifier":"Run the same SHAP, LIME, and DiCE pipeline on many random test samples from the same dataset and measure the overlap or rank correlation of their top features; if agreement is weak or contradictory across samples, the reliability claim fails. Alternatively, build a synthetic dataset with features that are known to generate attacks and check whether all three methods identify those features for a sample.","tokens_in":8412,"feed_emoji":"🛡️","tokens_out":6760,"duration_ms":52851,"temperature":0.7,"pith_summary":"Explainable AI is often a single tool applied to a single model. This paper tries to establish that using three explanations for the same prediction—SHAP, LIME, and DiCE—produces a more complete and trustworthy account of why a 6G-era medical network classifier flags a sample as normal or attacking. The authors build a Random Forest intrusion detector on a medical network dataset, balance it with SMOTE, and explain one random test instance with all three methods. Because the methods converge on overlapping CPU, memory, and packet-activity features, the paper claims their agreement cross-validates the explanations and fosters trust in AI-driven security. The value, if right, is that security teams could rely on qualitative consensus among explainers to guide defenses in life-critical healthcare infrastructure.","feed_headline":"Three AI explainers team up to spot 6G medical network attacks","feed_subtitle":"SHAP, LIME, and DiCE converge on CPU and packet features that separate normal from malicious traffic in a medical IoT dataset.","key_machinery":"The machinery is a trio of model-agnostic explanation tools applied to a single Random Forest intrusion-detection model. SHAP (Shapley additive explanations) assigns each feature a contribution value by averaging over feature subsets; LIME (Local Interpretable Model-agnostic Explanations) fits a simple interpretable surrogate locally around the instance; DiCE (Diverse Counterfactual Explanations) generates several minimal changes to the input that flip the model's prediction. The paper's cross-validation is the qualitative comparison of which features these three methods call out for one randomly selected test sample, treating their agreement as evidence that the explanations are reliable.","core_discovery":"The central claim is that integrating SHAP, LIME, and DiCE yields a comprehensive interpretation of model decisions in 6G healthcare networks, and that cross-validating the three methods confirms their reliability and consistency. On the paper's single test sample, SHAP associates high system, user, and idle CPU times with normal operation; LIME flags extreme idle and user times as possible attack indicators; and DiCE shows that source and destination byte counts and jitter move the prediction between attack and normal. The paper reads this convergence as confirmation that normal traffic is characterized by stable CPU usage, typical network activity, and sufficient resources, while anomalies in idle time, jitter, and unusual packet behavior indicate attacks. That interpretive convergence is the evidence offered for the conclusion that multi-method XAI can support trustworthy security decisions.","pith_inferences":["We would extend the paper's single-sample check by measuring pairwise feature-overlap or rank correlation of SHAP, LIME, and DiCE across hundreds of random test instances; stable consensus across a population would be far stronger evidence than one sample.","A natural next test is to run the same three-tool pipeline on a synthetic dataset with known attack-generating features, which would show whether agreement tracks the true cause or merely shared approximation bias.","The paper implies but does not explore that a sudden change in the three explainers' consensus could itself serve as a drift or attack alarm in real-time 6G edge monitoring.","We infer that the authors' reliability claim should be read as consistency among explainers, not correctness against ground truth; the paper provides no external standard for what the right explanation is."],"forward_implications":["Security teams in 6G-enabled hospitals could watch CPU idle time, network jitter, and packet-size anomalies as early indicators of intrusion.","A consensus approach that combines SHAP, LIME, and DiCE could be embedded in network security dashboards to deliver human-readable alerts.","The same multi-explainer pipeline could be transferred to other critical-infrastructure intrusion-detection datasets to identify feature-level attack signatures.","If the methods agree on normal-versus-attack behavior, the same cross-validation idea could be used to explain false negatives and help tune detection thresholds.","Non-technical stakeholders, including clinicians and hospital administrators, could receive actionable explanations of AI security decisions."],"supporting_citations":[{"why":"Supplies the SHAP method that quantifies per-feature contributions to the model's prediction.","marker":"[12]"},{"why":"Supplies the LIME method that fits a local surrogate to explain the instance.","marker":"[13]"},{"why":"Supplies the DiCE method that generates diverse counterfactuals to flip the prediction.","marker":"[14]"},{"why":"Provides the medical network traffic dataset used to train and evaluate the intrusion-detection model.","marker":"[5]"},{"why":"Supplies SMOTE, the oversampling technique used to balance attack and normal classes before training.","marker":"[21]"},{"why":"Defines the 6G usage scenarios and security context that frame the paper's motivation.","marker":"[4]"}],"fun_headline_variants":["SHAP, LIME, DiCE unite to defend IoT healthcare","6G healthcare gets an AI security spotlight","Explainable AI shields medical devices from cyberattacks","Three AI tools expose 6G medical network threats","Cross-checked AI explainers secure 6G medical IoT"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that qualitative agreement among SHAP, LIME, and DiCE on a single randomly chosen test instance is enough to establish that the explanations are reliable, even though no ground-truth explanation or aggregate consistency statistic is offered.","fun_headline_variants_meta":{"raw":{"variants":["SHAP, LIME, DiCE unite to defend IoT healthcare","6G healthcare gets an AI security spotlight","Explainable AI shields medical devices from cyberattacks","Three AI tools expose 6G medical network threats","Cross-checked AI explainers secure 6G medical IoT"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000657,"raw_usage":{"total_tokens":2955,"prompt_tokens":842,"completion_tokens":2113,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":458,"completion_tokens_details":{"reasoning_tokens":2048}},"tokens_in":458,"tokens_out":2113,"duration_ms":14312,"temperature":1.0,"reasoning_tokens":2048,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T15:29:23.375576+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same SHAP, LIME, and DiCE pipeline on many random test samples from the same dataset and measure the overlap or rank correlation of their top features; if agreement is weak or contradictory across samples, the reliability claim fails. Alternatively, build a synthetic dataset with features that are known to generate attacks and check whether all three methods identify those features for a sample.","supporting_citations":[{"cited_title":"A unified approach to interpreting model predictions","cited_arxiv_id":null,"evidence_quote":"Supplies the SHAP method that quantifies per-feature contributions to the model's prediction."},{"cited_title":"Why should i trust you?","cited_arxiv_id":null,"evidence_quote":"Supplies the LIME method that fits a local surrogate to explain the instance."},{"cited_title":"Explaining machine learning classifiers through diverse counterfactual explanations","cited_arxiv_id":null,"evidence_quote":"Supplies the DiCE method that generates diverse counterfactuals to flip the prediction."},{"cited_title":"HDRL -IDS: A Hybrid Deep Reinforcement Learning Intrusion Detection System for Enhancing the Security of Medical Applications in 5G Networks","cited_arxiv_id":null,"evidence_quote":"Provides the medical network traffic dataset used to train and evaluate the intrusion-detection model."},{"cited_title":"SMOTE: synthetic minority over -sampling technique","cited_arxiv_id":null,"evidence_quote":"Supplies SMOTE, the oversampling technique used to balance attack and normal classes before training."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the 6G usage scenarios and security context that frame the paper's motivation."}],"review_version":1}