REVIEW 5 major objections 7 minor 30 references
Human-Centered Explainable AI for Security Enhancement: A Deep Intrusion Detection Framework
T0 review · 5 major / 7 minor · reviewed 2026-08-03 · deepseek-v4-flash
Pith's one-line read This paper claims that an intrusion detection system combining CNN and LSTM networks can reach 99 percent accuracy on the NSL-KDD benchmark while remaining interpretable through SHAP explanations, and that security experts find the resultin
desk verdict Routine CNN/LSTM on NSL-KDD plus SHAP and a small trust survey; the 99% accuracy is real under the authors' own random split, but the unsupported superiority claim and non-standard split keep it from being a serious contribution. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The framework pairs two deep learning architectures — a 3-layer CNN that reads each 41-feature record as a one-dimensional sequence, and a 3-layer LSTM with dropout that treats the same features along a temporal dimension. Both classify traffic into five categories: Normal, DoS, Probe, R2L, and U2R. SHAP (SHapley Additive exPlanations) is the explainability engine: it computes feature attributions for the DoS class, with DeepExplainer and GradientExplainer used for the CNN and KernelExplainer for the LSTM. A custom UI presents predictions and SHAP explanations to cybersecurity experts, and survey instruments adapted from Mini-IPIP6, STIAS, TiA, and SUS capture trust, reliability, and usabili
What would settle it
Run the same CNN and LSTM architectures on the standard KDDTrain+ training set and KDDTest+ test set, and compare per-class accuracy and macro precision/recall/F1. If accuracy falls substantially below 99 percent, or if the class distribution in Section III-A does not match the actual NSL-KDD counts, the central performance claim is refuted.
Extended reading notes
Core claim
The central claim is that combining a CNN with an LSTM and adding SHAP explanations yields an intrusion detection system that is both highly accurate and interpretable. On NSL-KDD, both models reach 0.99 accuracy; SHAP identifies srv_serror_rate, dst_host_srv_serror_rate, and serror_rate as the most influential features for DoS detection, which the authors interpret as consistent with connection-failure patterns in denial-of-service attacks. A 15-person expert survey reports acceptable trust and reliability for the interface, though usability is marginal. The authors argue this closes the gap between black-box performance and human trust.
Load-bearing premise
The load-bearing premise is that the authors' handling of NSL-KDD is valid: they split the full 125,973-row dataset randomly 80/20 instead of using the standard KDDTrain+/KDDTest+ partitions, and their stated class counts for R2L and Probe appear swapped. If the split or labels are wrong, the 99 percent accuracy and per-class comparisons are not meaningful against the published benchmark.
Editorial extensions
If this is right
- If the 99 percent accuracy holds on standard benchmarks, deep learning IDS can match or exceed traditional signature-based systems while remaining interpretable.
- SHAP's consistent identification of error-rate features across both architectures suggests a stable, physically meaningful signal for DoS detection that aligns with known attack patterns.
- The expert survey results imply that explanations can make analysts trust a deep learning IDS enough for practical use, though usability needs work.
- The framework's reliance on all 41 raw features, without dimensionality reduction, suggests that feature engineering can be deferred when using deep models.
Reading between the lines
- A direct test of the paper's core claim would be to rerun the identical models on the standard KDDTrain+/KDDTest+ partitions rather than a random 80/20 split; per-class results, especially R2L and U2R, would likely shift and the reported 99 percent accuracy may not survive because the paper's class counts in Section III-A appear to swap R2L and Probe.
- The SHAP results are computed for only 100 test instances and one class (DoS); extending to other attack classes and larger samples would test whether the explanations are robust or class-specific artifacts.
- The expert survey has only 15 participants from one country; the trust and reliability findings are promising but not generalizable until replicated with a larger, more diverse analyst pool.
- If error-rate features remain top predictors across architectures, it suggests a cheap, simpler model trained on just those few features might achieve much of the same DoS performance — a testable extension.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper proposes an explainable intrusion detection framework combining CNN and LSTM models on the NSL-KDD dataset, with SHAP for post-hoc explanations and a 15-participant user study measuring trust, reliability, and usability. The authors report 99% accuracy for both models, claim superior performance over traditional IDS and black-box deep learning models, and use SHAP to identify influential features for DoS detection. A human-centered evaluation via an interactive UI is also presented.
Significance. If the results were valid, the paper would contribute a useful comparison of CNN and LSTM architectures for intrusion detection and a rare attempt to evaluate SHAP-based explanations with human experts. The inclusion of confusion matrices, ROC curves, and a user study based on validated psychometric instruments is commendable. However, the central quantitative claims are currently not supported by the experimental design: the non-standard data split, absence of baseline comparisons, and internal contradictions in the dataset description and reported experimental setup prevent any reliable assessment of the claimed superiority.
major comments (5)
- [Section III-C / IV-A] The evaluation uses an 80/20 random split of the 125,973-row NSL-KDD training set instead of the standard KDDTrain+/KDDTest+ protocol. KDDTest+ contains attack types not present in KDDTrain+, so a random split from the same distribution inflates accuracy and makes the reported 99% figure non-comparable with published NSL-KDD results. The abstract's central claim of 'superior performance compared to traditional IDS and black-box deep learning models' is therefore not supported by the chosen experimental design.
- [Abstract / Section IV-A] No baseline models are compared. The paper only compares the proposed CNN and LSTM models to each other; the claim of superiority over traditional IDS and black-box deep learning models is asserted without any empirical baseline, confidence interval, or statistical significance test. This is a load-bearing omission for the paper's main claim.
- [Section III-A / III-D] The class distribution statement lists counts as '67343, 45927, 11647, 995, 52 for Normal, Dos, R2L, Probe and U2R', which swaps the canonical Probe and R2L counts (canonical KDDTrain+: Probe=11,656, R2L=995). The confusion matrices' row totals (Probe≈2355, R2L≈202 on the 20% test split) correspond to the canonical counts, suggesting this is a typo rather than an actual label swap. Nevertheless, because Section III-D defines the target encoding, the paper must explicitly state the mapping used and resolve the ambiguity; as written, the data pipeline is unclear.
- [Section V vs. Section III-A] The conclusion states that 'due to hardware limitations, our experiments were constrained to simplified models and downsampled datasets,' but the methodology describes training and testing on the full 125,973-row dataset. This is a direct contradiction. If the data were downsampled, the confusion matrices and 99% accuracy do not correspond to the stated dataset and the reported results are not reproducible. This must be clarified and the experiments must be described consistently.
- [Section IV-B] The SHAP analysis is reported only for class 0 (DoS), and for the LSTM the three-dimensional sequential inputs are flattened to two dimensions before applying KernelExplainer. Flattening destroys the temporal structure that the LSTM is claimed to exploit, so the resulting feature attributions may not reflect the model's actual decision process. Additionally, using 100 samples for a 'global' interpretation is insufficient; the paper should justify the procedure and limit the claims made from the summary plots.
minor comments (7)
- [General] Typos and grammatical issues: 'decisoin' in the Introduction, 'In section' with a missing number, and inconsistent capitalization of 'SHapley Additive exPlanations'.
- [Section IV-A / Table I] The prose states the CNN achieved a macro recall of 0.99, but Table I reports 0.84; the confusion-matrix-based macro recall is 0.84, so the text should be corrected.
- [Section IV-A / Table II / Fig. 2] For the CNN, the U2R confusion-matrix row totals give a recall of 0.30 (3/10), not 0.33 as in Table II; the corresponding F1-score would be 0.375, not 0.40. The source of these discrepancies should be checked.
- [Table II] The title 'Accuracy Per Attack Type' is misleading since the table reports precision, recall, and F1-score, not accuracy. A more descriptive title is needed.
- [Figure 1] Figure 1 is low-resolution and the text is difficult to read; a vector version or higher-resolution figure would improve clarity.
- [Section III-C] The dataset description says NSL-KDD has '43 columns along with 125973 rows'; the standard NSL-KDD KDDTrain+ has 42 columns (41 features + label). Please clarify the column count and exactly which file(s) were used.
- [Section IV-C] The user study reports Cronbach's alpha of 0.60 for usability, which is below the commonly accepted 0.70 threshold; this limitation should be discussed in the text, not merely mentioned as 'moderate consistency.'
Circularity Check
No significant circularity: the paper's claims are empirical evaluations, not derivations that reduce to their own inputs.
full rationale
The paper is an empirical study: it trains CNN and LSTM classifiers on NSL-KDD, reports accuracy/precision/recall/F1, applies SHAP post hoc for feature attribution, and runs a 15-participant trust/usability survey. None of the central claims are derived from a quantity that was itself fitted into the claim. The 99% accuracy figures come from training and testing on the data, and SHAP explains the already-trained models rather than being used to construct the predictions. The expert survey is independent self-report data. There is no uniqueness theorem, no ansatz imported via citation that does the derivational work, and no renamed known result presented as a derivation. The paper does cite a prior work by overlapping authors [23] for the general statement that CNN and LSTM perform well in intrusion detection, but this citation is motivational and not load-bearing: the paper's own evaluations are reported directly from its own experiments. The dataset-handling concerns identified by a skeptical reading (the non-standard 80/20 split and the apparent swapping of R2L/Probe class counts) are validity and correctness risks, not circularity: they concern whether the empirical results are meaningful, not whether the results were assumed in order to be predicted. Similarly, the admitted SHAP compatibility issues and hardware-limited downsampling are stated limitations, not circular steps. Under the stated rules, unsupported or potentially invalid empirical claims are not circularity, so the appropriate finding is no significant circularity with score 0.
Assumptions & free parameters
free parameters (3)
- CNN architecture (3 layers, filter sizes not reported)
- LSTM architecture (3 layers, units not reported, dropout=0.3) =
0.3 dropout
- Training hyperparameters (epochs=50, batch_size=64, optimizer=Adam) =
50 epochs, batch 64
assumptions (3)
- domain assumption The NSL-KDD dataset is used as a valid benchmark, with the authors' 80/20 random split of the entire dataset treated as equivalent to standard benchmark evaluation.
- domain assumption The SHAP explanations reflect the true decision-making basis of the CNN and LSTM models.
- domain assumption The 15-participant expert survey is representative enough to support conclusions about trust, reliability, and usability.
Cite this review
Pith. "Pith review of Human-Centered Explainable AI for Security Enhancement: A Deep Intrusion Detection Framework." pith.science (2026). https://pith.science/paper/HDQDP72Y
@misc{pith2026260213271,
author = {Pith},
title = {Pith review of: Human-Centered Explainable AI for Security Enhancement: A Deep Intrusion Detection Framework},
year = {2026},
howpublished = {\url{https://pith.science/paper/HDQDP72Y}},
note = {Machine review of arXiv:2602.13271}
}
read the original abstract
The increasing complexity and frequency of cyber-threats demand intrusion detection systems (IDS) that are not only accurate but also interpretable. This paper presented a novel IDS framework that integrated Explainable Artificial Intelligence (XAI) to enhance transparency in deep learning models. The framework was evaluated experimentally using the benchmark dataset NSL-KDD, demonstrating superior performance compared to traditional IDS and black-box deep learning models. The proposed approach combined Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) networks for capturing temporal dependencies in traffic sequences. Our deep learning results showed that both CNN and LSTM reached 0.99 for accuracy, whereas LSTM outperformed CNN at macro average precision, recall, and F-1 score. For weighted average precision, recall, and F-1 score, both models scored almost similarly. To ensure interpretability, the XAI model SHapley Additive exPlanations (SHAP) was incorporated, enabling security analysts to understand and validate model decisions. Some notable influential features were srv_serror_rate, dst_host_srv_serror_rate, and serror_rate for both models, as pointed out by SHAP. We also conducted a trust-focused expert survey based on IPIP6 and Big Five personality traits via an interactive UI to evaluate the system's reliability and usability. This work highlighted the potential of combining performance and transparency in cybersecurity solutions and recommends future enhancements through adaptive learning for real-time threat detection.
Figures
Reference graph
Works this paper leans on
-
[1]
HDLNIDS: hybrid deep- learning-based network intrusion detection system,
E. U. H. Qazi, M. H. Faheem, and T. Zia, “HDLNIDS: hybrid deep- learning-based network intrusion detection system,”Applied Sciences, vol. 13, no. 8, p. 4921, 2023
2023
-
[2]
A novel two-stage deep learning model for network intrusion detection: LSTM- AE,
V . Hnamte, H. Nhung-Nguyen, J. Hussain, and Y . Hwa-Kim, “A novel two-stage deep learning model for network intrusion detection: LSTM- AE,”IEEE Access, vol. 11, pp. 37131–37148, 2023
2023
-
[3]
A novel deep learning-based intrusion detection system for IOT networks,
A. Awajan, “A novel deep learning-based intrusion detection system for IOT networks,”Computers, vol. 12, no. 2, p. 34, 2023
2023
-
[4]
DCNNBiLSTM: An efficient hybrid deep learning-based intrusion detection system,
V . Hnamte and J. Hussain, “DCNNBiLSTM: An efficient hybrid deep learning-based intrusion detection system,”Telematics and Informatics Reports, vol. 10, p. 100053, 2023
2023
-
[5]
A deep learning technique for intrusion detection system using a recurrent neural networks based framework,
S. M. Kasongo, “A deep learning technique for intrusion detection system using a recurrent neural networks based framework,”Computer Communications, vol. 199, pp. 113–125, 2023
2023
-
[6]
NIDS-CNNLSTM: Network intrusion detection classification model based on deep learning,
J. Du, K. Yang, Y . Hu, and L. Jiang, “NIDS-CNNLSTM: Network intrusion detection classification model based on deep learning,”IEEE Access, vol. 11, pp. 24808–24821, 2023
2023
-
[7]
Snort: Lightweight intrusion detection for networks.,
M. Roeschet al., “Snort: Lightweight intrusion detection for networks.,” inLisa, vol. 99, pp. 229–238, 1999
1999
-
[8]
Design and implementation of a lightweight intrusion detection and prevention system,
X. Wei, “Design and implementation of a lightweight intrusion detection and prevention system,” inInternational Conference on Security and Privacy in New Computing Environments, pp. 433–439, Springer, 2019
2019
Show all 30 references
-
[9]
Intrusion detection framework for internet of things with rule induction for model explana- tion,
K. S. Adewole, A. Jacobsson, and P. Davidsson, “Intrusion detection framework for internet of things with rule induction for model explana- tion,”Sensors, vol. 25, no. 6, p. 1845, 2025
2025
-
[10]
An efficient intrusion detection system based on hypergraph-genetic algorithm for parameter optimization and feature selection in support vector machine,
M. G. Raman, N. Somu, K. Kirthivasan, R. Liscano, and V . S. Sriram, “An efficient intrusion detection system based on hypergraph-genetic algorithm for parameter optimization and feature selection in support vector machine,”Knowledge-Based Systems, vol. 134, pp. 1–12, 2017
2017
-
[11]
A lightweight framework to secure iot devices with limited resources in cloud environments,
V . K. Pandey, D. Sahu, S. Prakash, R. S. Rathore, P. Dixit, and I. Hunko, “A lightweight framework to secure iot devices with limited resources in cloud environments,”Scientific Reports, vol. 15, no. 1, p. 26009, 2025
2025
-
[12]
Erbm: A machine learning-driven rule-based model for intrusion detection in iot environments.,
A. Mehmmod, K. Batool, A. Sajid, M. M. Alam, M. M. Su’ud, and I. U. Khan, “Erbm: A machine learning-driven rule-based model for intrusion detection in iot environments.,”Computers, Materials & Continua, vol. 83, no. 3, 2025
2025
-
[13]
Gradient-based learning applied to document recognition,
Y . LeCun, L. Bottou, Y . Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,”Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324, 2002
2002
-
[14]
Long short-term memory,
S. Hochreiter and J. Schmidhuber, “Long short-term memory,”Neural computation, vol. 9, no. 8, pp. 1735–1780, 1997
1997
-
[15]
Long-term traffic flow forecasting using a hybrid CNN-BiLSTM model,
M. M ´endez Hurtado, M. D. L. M. Garc´ıa Merayo, and M. N´u˜nez Garc´ıa, “Long-term traffic flow forecasting using a hybrid CNN-BiLSTM model,” 2023
2023
-
[16]
An overview of XAI Algorithms,
Y .-W. Chen, S.-Y . Chien, and F. Yu, “An overview of XAI Algorithms,” in2023 International Automatic Control Conference (CACS), pp. 1–5, IEEE, 2023
2023
-
[17]
A detailed analysis on NSL-KDD dataset using various machine learning techniques for intrusion detection,
S. Revathi and A. Malathi, “A detailed analysis on NSL-KDD dataset using various machine learning techniques for intrusion detection,” International Journal of Engineering Research & Technology (IJERT), vol. 2, no. 12, pp. 1848–1853, 2013
2013
-
[18]
A detailed analysis of the KDD CUP 99 data set,
M. Tavallaee, E. Bagheri, W. Lu, and A. A. Ghorbani, “A detailed analysis of the KDD CUP 99 data set,” in2009 IEEE symposium on computational intelligence for security and defense applications, pp. 1– 6, IEEE, 2009
2009
-
[19]
An improved long short term memory network for intrusion detection,
A. A. Awad, A. F. Ali, and T. Gaber, “An improved long short term memory network for intrusion detection,”Plos one, vol. 18, no. 8, p. e0284795, 2023
2023
-
[20]
Theoretical analysis of an alphabetic confusion matrix,
J. T. Townsend, “Theoretical analysis of an alphabetic confusion matrix,” Perception & Psychophysics, vol. 9, pp. 40–50, 1971
1971
-
[21]
A new multi-classification task accuracy evaluation method based on confusion matrix,
K.-f. Zhang, H.-y. Su, and Y . Dou, “A new multi-classification task accuracy evaluation method based on confusion matrix,”Computer Engineering & Science, vol. 43, no. 11, p. 1910, 2021
1910
-
[22]
MLCM: Multi-label confusion matrix,
M. Heydarian, T. E. Doyle, and R. Samavi, “MLCM: Multi-label confusion matrix,”IEEE Access, vol. 10, pp. 19083–19095, 2022
2022
-
[23]
Evaluating deep learning models for network intrusion detection: A comparative analysis,
M. Islam, M. M. J. Ayan, E. Kais, R. K. Das, and M. M. Islam, “Evaluating deep learning models for network intrusion detection: A comparative analysis,” in2024 27th International Conference on Com- puter and Information Technology (ICCIT), pp. 3635–3640, 2024
2024
-
[24]
The Mini-IPIP6: Validation and extension of a short measure of the Big-Six factors of personality in New Zealand.,
C. G. Sibley, N. Luyten, M. Purnomo, A. Mobberley, L. W. Wootton, M. D. Hammond, N. Sengupta, R. Perry, T. West-Newman, M. S. Wilson,et al., “The Mini-IPIP6: Validation and extension of a short measure of the Big-Six factors of personality in New Zealand.,”New Zealand Journal ...
2011
-
[25]
Effect of confidence and explanation on accuracy and trust calibration in ai-assisted decision mak- ing,
Y . Zhang, Q. V . Liao, and R. K. Bellamy, “Effect of confidence and explanation on accuracy and trust calibration in ai-assisted decision mak- ing,” inProceedings of the 2020 conference on fairness, accountability, and transparency, pp. 295–305, 2020
2020
-
[26]
Monitoring oral health remotely: ethical considerations when using ai among vulnerable populations,
C. McGrath, C. W. R. Chau, and G. F. Molina, “Monitoring oral health remotely: ethical considerations when using ai among vulnerable populations,”Frontiers in Oral Health, vol. 6, p. 1587630, 2025
2025
-
[27]
Theoretical considerations and development of a question- naire to measure trust in automation,
M. K ¨orber, “Theoretical considerations and development of a question- naire to measure trust in automation,” inCongress of the International Ergonomics Association, pp. 13–30, Springer, 2018
2018
-
[28]
Sus-a quick and dirty usability scale,
J. Brookeet al., “Sus-a quick and dirty usability scale,”Usability evaluation in industry, vol. 189, no. 194, pp. 4–7, 1996
1996
-
[29]
Evaluating explain- able ai for deep learning-based network intrusion detection system alert classification,
R. Kalakoti, R. Vaarandi, H. Bahsi, and S. N ˜omm, “Evaluating explain- able ai for deep learning-based network intrusion detection system alert classification,”arXiv preprint arXiv:2506.07882, 2025
2025 arXiv
-
[30]
A novel explainable method based on grad-cam for network intrusion detection,
Y . Sun, Z. Lian, S. Zhang, Z. Wang, and T. Duan, “A novel explainable method based on grad-cam for network intrusion detection,” in2024 IEEE 24th International Conference on Software Quality, Reliability and Security (QRS), pp. 400–406, IEEE, 2024
2024
Reviewed August 3, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.