pith. machine review for the scientific record. sign in

arxiv: 2604.14444 · v1 · submitted 2026-04-15 · 💻 cs.CR · cs.AI

Recognition: unknown

Robustness Analysis of Machine Learning Models for IoT Intrusion Detection Under Data Poisoning Attacks

Fortunatus Aabangbio Wulnye , Justice Owusu Agyemang , Kwame Opuni-Boachie Obour Agyekum , Kwame Agyeman-Prempeh Agyekum , Kingsford Sarkodie Obeng Kwakye , Francisca Adomaa Acheampong
Authors on Pith no claims yet

Pith reviewed 2026-05-10 12:31 UTC · model grok-4.3

classification 💻 cs.CR cs.AI
keywords IoT intrusion detectiondata poisoningmachine learning robustnessensemble classifierslabel manipulationoutlier attacksdeep neural networkslogistic regression
0
0 comments X

The pith

Ensemble models resist data poisoning better than logistic regression or deep neural networks in IoT intrusion detection.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tests four classifiers on three real IoT datasets to measure how label manipulation and outlier injection during training affect intrusion detection performance. Ensemble methods such as random forest and gradient boosting maintain relatively steady accuracy and detection rates, while logistic regression and deep neural networks lose up to 40 percent effectiveness. This difference arises because the non-ensemble models allow poisoned examples to shift decision boundaries more severely. If true, the result means operators can reduce vulnerability by preferring ensembles when training data cannot be fully trusted. The work also notes that such distortions lower overall detection fidelity and delay readiness for real deployment.

Core claim

When four standard classifiers are trained on poisoned versions of three real-world IoT intrusion datasets, random forest and gradient boosting machines retain stable performance across label-flip and outlier attacks, whereas logistic regression and deep neural networks exhibit accuracy and F1-score drops reaching 40 percent. These degradations distort decision boundaries and reduce the ability to distinguish normal from malicious traffic.

What carries the argument

Comparative evaluation of classifier robustness under label manipulation and outlier-based poisoning attacks, quantified by performance degradation on IoT intrusion datasets.

If this is right

  • Ensemble classifiers provide a practical default choice when training data integrity cannot be guaranteed in IoT networks.
  • Vulnerable models require additional defenses such as adversarially robust training to preserve detection rates.
  • Operational systems should add continuous anomaly monitoring and feature-level validation to limit poisoning effects.
  • Resilience testing against poisoning should become part of regulatory compliance checks for AI-based IoT security.
  • Adaptive models that detect and mitigate attacks during training can extend the reliability of current pipelines.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same relative stability of ensembles may appear in other sensor-driven security tasks that rely on labeled network traces.
  • Adding lightweight poisoning detectors at the data-ingestion stage could further narrow the performance gap between model families.
  • Repeating the experiments on larger or more diverse IoT traces would test whether the 40 percent bound holds outside the three datasets used here.

Load-bearing premise

The chosen poisoning strategies and three real-world IoT datasets accurately represent the threats and operating conditions of actual deployed intrusion detection systems.

What would settle it

Collect fresh IoT traffic, apply the same label-flip and outlier poisoning at the reported attack strengths, retrain logistic regression and deep neural network detectors, and check whether the observed accuracy drop reaches or exceeds 40 percent.

Figures

Figures reproduced from arXiv: 2604.14444 by Fortunatus Aabangbio Wulnye, Francisca Adomaa Acheampong, Justice Owusu Agyemang, Kingsford Sarkodie Obeng Kwakye, Kwame Agyeman-Prempeh Agyekum, Kwame Opuni-Boachie Obour Agyekum.

Figure 1
Figure 1. Figure 1: Overview of the Methodology for Evaluating ML-Based NIDS Under Data Poisoning [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Machine Learning Model Development Pipeline [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Number of features per label - CICIoT2023 [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Number of features per label – Edge IoT [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Number of features per label – N-BaIoT On CIOT2023, RF achieved the strongest performance with 99.29% accuracy and a 74.24% F1- score, followed by GBM at 99.06% accuracy and a 71.06% F1-score. LR performed considerably lower, with 83.88% accuracy and a 53.89% F1-score, indicating difficulty handling the dataset’s 8 [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Performance of ML Models on Original IoT Datasets [PITH_FULL_IMAGE:figures/full_fig_p009_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Performance of ML Models on Label-Flipped Poisoned Datasets [PITH_FULL_IMAGE:figures/full_fig_p010_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Performance of ML Models on Outlier-Injected Poisoned Datasets [PITH_FULL_IMAGE:figures/full_fig_p011_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Performance of ML Models on Feature-Impersonated Poisoned Datasets [PITH_FULL_IMAGE:figures/full_fig_p012_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Performance of ML Models on Generic Synthetic Outlier Poisoned Datasets [PITH_FULL_IMAGE:figures/full_fig_p013_10.png] view at source ↗
read the original abstract

Ensuring the reliability of machine learning-based intrusion detection systems remains a critical challenge in Internet of Things (IoT) environments, particularly as data poisoning attacks increasingly threaten the integrity of model training pipelines. This study evaluates the susceptibility of four widely used classifiers, Random Forest, Gradient Boosting Machine, Logistic Regression, and Deep Neural Network models, against multiple poisoning strategies using three real-world IoT datasets. Results show that while ensemble-based models exhibit comparatively stable performance, Logistic Regression and Deep Neural Networks suffer degradation of up to 40% under label manipulation and outlier-based attacks. Such disruptions significantly distort decision boundaries, reduce detection fidelity, and undermine deployment readiness. The findings highlight the need for adversarially robust training, continuous anomaly monitoring, and feature-level validation within operational Network Intrusion Detection Systems. The study also emphasizes the importance of integrating resilience testing into regulatory and compliance frameworks for AI-driven IoT security. Overall, this work provides an empirical foundation for developing more resilient intrusion detection pipelines and informs future research on adaptive, attack-aware models capable of maintaining reliability under adversarial IoT conditions.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 2 minor

Summary. This paper empirically evaluates the robustness of four machine learning classifiers—Random Forest, Gradient Boosting Machine, Logistic Regression, and Deep Neural Network—for IoT intrusion detection against data poisoning attacks (label manipulation and outlier injection) on three real-world IoT datasets. It reports that ensemble-based models remain comparatively stable while Logistic Regression and Deep Neural Networks suffer performance degradation of up to 40%, and concludes by recommending adversarially robust training and resilience testing for operational systems.

Significance. If the attack models and datasets accurately reflect operational IoT conditions, the results would provide a useful empirical baseline on classifier susceptibility to poisoning, supporting calls for robust training in IoT IDS. The work is a standard comparative study without machine-checked proofs, reproducible code artifacts, or parameter-free derivations, so its significance is primarily in highlighting practical vulnerabilities rather than advancing theoretical understanding.

major comments (3)
  1. [Abstract / Results] Abstract and results section: The headline claim of 'degradation of up to 40%' for LR and DNN is presented without specifying the performance metric (accuracy, F1, AUC, etc.), the exact attack parameters (e.g., poisoning rate, outlier magnitude), baseline clean-model scores, or any statistical significance tests, making it impossible to verify the magnitude or reproducibility of the central empirical finding.
  2. [Methodology] Methodology / Threat model section: The load-bearing assumption that label manipulation and outlier injection accurately represent feasible attacks on IoT IDS training pipelines is not justified; the paper must explicitly map these strategies to realistic attacker capabilities (e.g., access to label sources or sensor data aggregation points) and discuss why they correspond to actual deployment threats rather than synthetic perturbations.
  3. [Experiments] Experimental setup: No details are provided on how the three real-world IoT datasets were preprocessed, split, or whether they reflect operational traffic distributions (e.g., class imbalance, feature distributions under normal vs. attack conditions), which directly affects whether the measured degradations generalize beyond the chosen corpora.
minor comments (2)
  1. [Abstract / Conclusion] The abstract and conclusion repeat similar phrasing about 'adversarially robust training' and 'resilience testing' without concrete recommendations tied to the experimental findings.
  2. [Throughout] Notation for model names and attack types should be defined consistently (e.g., use abbreviations after first use) to improve readability.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive and detailed feedback. We have reviewed each major comment carefully and will revise the manuscript to address the concerns regarding clarity, justification, and experimental details. Our point-by-point responses follow.

read point-by-point responses

  1. Referee: [Abstract / Results] Abstract and results section: The headline claim of 'degradation of up to 40%' for LR and DNN is presented without specifying the performance metric (accuracy, F1, AUC, etc.), the exact attack parameters (e.g., poisoning rate, outlier magnitude), baseline clean-model scores, or any statistical significance tests, making it impossible to verify the magnitude or reproducibility of the central empirical finding.

    Authors: We agree that the abstract would benefit from greater precision in summarizing the quantitative results. The results section contains tables reporting F1-score and accuracy degradations under specific poisoning rates (10-30%) and outlier magnitudes, with baseline clean-model performances provided for comparison. To improve verifiability, we will revise the abstract to explicitly state the primary metric (F1-score), the maximum observed degradation, key attack parameters, and baseline values. We will also add standard deviations across multiple random seeds and note any statistical significance assessments in the results section. revision: yes

  2. Referee: [Methodology] Methodology / Threat model section: The load-bearing assumption that label manipulation and outlier injection accurately represent feasible attacks on IoT IDS training pipelines is not justified; the paper must explicitly map these strategies to realistic attacker capabilities (e.g., access to label sources or sensor data aggregation points) and discuss why they correspond to actual deployment threats rather than synthetic perturbations.

    Authors: We acknowledge the need for a more explicit mapping of the threat model to realistic IoT deployment scenarios. In the revised manuscript, we will expand the threat model subsection to describe how label manipulation could arise from compromised annotation pipelines or weak supervision in sensor networks, and how outlier injection might occur via compromised data aggregation points or adversarial sensor spoofing. We will also discuss why these attacks align with known vulnerabilities in operational IoT IDS rather than purely synthetic perturbations. revision: yes

  3. Referee: [Experiments] Experimental setup: No details are provided on how the three real-world IoT datasets were preprocessed, split, or whether they reflect operational traffic distributions (e.g., class imbalance, feature distributions under normal vs. attack conditions), which directly affects whether the measured degradations generalize beyond the chosen corpora.

    Authors: We will add a dedicated subsection on experimental setup that details the preprocessing pipeline (normalization, missing value handling, and feature selection), the train-test split strategy (e.g., 70/30 with temporal considerations where applicable), and summary statistics for each dataset including class imbalance ratios and feature distribution differences between normal and attack traffic. This will strengthen the discussion of generalizability to operational IoT conditions. revision: yes

Circularity Check

0 steps flagged

No derivation chain present; purely empirical evaluation

full rationale

The paper reports direct experimental measurements of classifier performance (Random Forest, GBM, LR, DNN) on three IoT datasets before and after applying label-manipulation and outlier-injection poisoning. No equations, fitted parameters renamed as predictions, self-citations used to justify uniqueness, or ansatzes appear in the abstract or described methodology. The central claim (ensemble stability vs. up to 40% degradation) is a measured outcome, not a derived quantity that reduces to its own inputs. The representativeness of the chosen attacks and datasets is an external validity question, not a circularity issue within any derivation.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The central claim rests on assumptions that the chosen attacks and datasets represent real IoT threats and that standard model training without defenses is a fair baseline; no free parameters or new entities are introduced.

axioms (2)
  • domain assumption The selected poisoning strategies (label manipulation and outlier injection) represent realistic threats to IoT IDS training pipelines.
    Invoked when assessing model susceptibility and deployment readiness.
  • domain assumption Performance on the three real-world IoT datasets generalizes to operational IoT environments.
    Basis for concluding that disruptions undermine deployment readiness.

pith-pipeline@v0.9.0 · 5541 in / 1241 out tokens · 28960 ms · 2026-05-10T12:31:13.206283+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

35 extracted references · 2 canonical work pages

  1. [1]

    Biggio, B

    B. Biggio, B. Nelson, and P. Laskov. Poisoning attacks against support vector machines, 2012. Online. Available: http://arxiv.org/abs/1206.6389

    work page arXiv 2012

  2. [2]

    X. Yuan, P. He, Q. Zhu, and X. Li. Adversarial examples: Attacks and defenses for deep learning.IEEE Transactions on Neural Networks and Learning Systems, 30(9):2805–2824, 2019

    2019

  3. [3]

    C. Dunn, N. Moustafa, and B. Turnbull. Robustness evaluations of sustainable machine learn- ing models against data poisoning attacks in the internet of things.Sustainability, 12(17), 2020

    2020

  4. [4]

    Detecting poi- soning attacks on machine learning in iot environments

    Nathalie Baracaldo, Bryant Chen, Heiko Ludwig, Amir Safavi, and Rui Zhang. Detecting poi- soning attacks on machine learning in iot environments. In2018 IEEE international congress on internet of things (ICIOT), pages 57–64. IEEE, 2018

    2018

  5. [5]

    S. Morgan. Cyberwarfare in the c-suite: Cybercrime facts and statistics.Cybercrime Magazine,

  6. [6]

    Survey of intrusion detection using deep learning in the internet of things.Iraqi Journal for Computer Science and Mathematics, 3(1):9, 2022

    Baraa I Farhan and Ammar D Jasim. Survey of intrusion detection using deep learning in the internet of things.Iraqi Journal for Computer Science and Mathematics, 3(1):9, 2022

    2022

  7. [7]

    A comprehensive survey on deep learning-based intru- sion detection systems in internet of things (iot).Expert Systems, 42(2):e13726, 2025

    Qasem Abu Al-Haija and Ayat Droos. A comprehensive survey on deep learning-based intru- sion detection systems in internet of things (iot).Expert Systems, 42(2):e13726, 2025

    2025

  8. [8]

    S V N Santhosh Kumar, M Selvi, and A Kannan. A comprehensive survey on machine learning- based intrusion detection systems for secure communication in internet of things.Computa- tional Intelligence and Neuroscience, 2023(1):8981988, 2023

    2023

  9. [9]

    Iot-based intrusion detection systems from the perspective of machine and deep learning: A survey and comparative study.Delta University Scientific Journal, 5(2):367–386, 2022

    Eman Ashraf, Nihal Areed, Hanaa Salem, Ehab Abdelhady, and Ahmed Farouk. Iot-based intrusion detection systems from the perspective of machine and deep learning: A survey and comparative study.Delta University Scientific Journal, 5(2):367–386, 2022

    2022

  10. [10]

    Carlos et al

    E. Carlos et al. Ciciot2023: A real-time dataset and benchmark for large-scale attacks in iot environment.Sensors, 2023

    2023

  11. [11]

    M. A. Ferrag, O. Friha, D. Hamouda, L. Maglaras, and H. Janicke. Edge-iiotset: A compre- hensive realistic cybersecurity dataset of iot and iiot applications for centralized and federated learning.IEEE Access, 10:40281–40306, 2022

    2022

  12. [12]

    Abbasi, M

    F. Abbasi, M. Naderan, and S. E. Alavi. Anomaly detection in internet of things using feature selection and classification based on logistic regression and artificial neural network on n-baiot dataset, 2021

    2021

  13. [13]

    Almseidin, J

    M. Almseidin, J. Al-Sawwa, and M. Alkasassbeh. Generating a benchmark cyber multi-step attacks dataset for intrusion detection.Journal of Intelligent and Fuzzy Systems, 43(3):3679– 3694, 2022. 14

    2022

  14. [14]

    Nazir et al

    A. Nazir et al. Advancing iot security: A systematic review of machine learning approaches for the detection of iot botnets.Journal of King Saud University - Computer and Information Sciences, 35(10), 2023

    2023

  15. [15]

    Y. N. Soe, Y. Feng, P. I. Santosa, R. Hartanto, and K. Sakurai. Machine learning-based iot-botnet attack detection with sequential architecture.Sensors, 20(16):1–15, 2020

    2020

  16. [16]

    Najafimehr, S

    M. Najafimehr, S. Zarifzadeh, and S. Mostafavi. Ddos attacks and machine-learning-based detection methods: A survey and taxonomy.Engineering Reports, 5(12), 2023

    2023

  17. [17]

    Z. Chen, N. Lv, P. Liu, Y. Fang, K. Chen, and W. Pan. Intrusion detection for wireless edge networks based on federated learning.IEEE Access, 8:217463–217472, 2020

    2020

  18. [18]

    B. Li, Y. Wu, J. Song, R. Lu, T. Li, and L. Zhao. Deepfed: Federated deep learning for intrusion detection in industrial cyber-physical systems.IEEE Transactions on Industrial Informatics, 17(8):5615–5624, 2021

    2021

  19. [19]

    Zhang, B

    J. Zhang, B. Chen, X. Cheng, H. T. T. Binh, and S. Yu. Poisongan: Generative poisoning at- tacks against federated learning in edge computing systems.IEEE Internet of Things Journal, 8(5):3310–3322, 2021

    2021

  20. [20]

    Chiba, Y

    T. Chiba, Y. Sei, Y. Tahara, and A. Ohsuga. A defense method against poisoning attacks on iot machine learning using poisonous data. InProceedings of the 2020 IEEE 3rd International Conference on Artificial Intelligence and Knowledge Engineering (AIKE), pages 100–107, 2020

    2020

  21. [21]

    Poison frogs! targeted clean-label poisoning attacks on neural networks

    Ali Shafahi, W Ronny Huang, Mahyar Najibi, Octavian Suciu, Christoph Studer, Tudor Du- mitras, and Tom Goldstein. Poison frogs! targeted clean-label poisoning attacks on neural networks. InAdvances in Neural Information Processing Systems, volume 31, 2018

    2018

  22. [22]

    On- line

    Random forest algorithm for absolute beginners in data science, 2021. On- line. Available: https://www.analyticsvidhya.com/blog/2021/10/an-introduction-to-random- forest-algorithm-for-beginners/

    2021

  23. [23]

    ˇSarˇ cevi´ c, D

    A. ˇSarˇ cevi´ c, D. Pintar, M. Vrani´ c, and A. Krajna. Cybersecurity knowledge extraction using xai.Applied Sciences, 12(17), 2022

    2022

  24. [24]

    S. Dash, S. K. Pani, J. Rodrigues, and B. Majhi. Deep learning, machine learning, and iot in biomedical and health informatics: Techniques and applications, 2022

    2022

  25. [25]

    Webster et al

    S. Webster et al. Evaluating intrusion detection systems: The 1998 darpa off-line intrusion detection evaluation, 2000. Online. Available: https://www.researchgate.net/publication/2243279

    work page arXiv 1998

  26. [26]

    Yaokumah, R

    W. Yaokumah, R. N. Clottey, and J. K. Appati. Network intrusion detection in internet of things (iot).International Journal of Smart Security Technologies, 8(1):49–65, 2021

    2021

  27. [27]

    Threat analysis of iot networks using artificial neural network intrusion detection system

    Elike Hodo, Xavier Bellekens, Andrew Hamilton, Pierre-Louis Dubouilh, Ephraim Iorkyase, Christos Tachtatzis, and Robert Atkinson. Threat analysis of iot networks using artificial neural network intrusion detection system. In2016 International symposium on networks, computers and communications (ISNCC), pages 1–6. IEEE, 2016

    2016

  28. [28]

    Atzori, A

    L. Atzori, A. Iera, and G. Morabito. The internet of things: A survey.Computer Networks, 54(15):2787–2805, 2010. 15

    2010

  29. [29]

    Chen and X

    J. Chen and X. Ran. Deep learning with edge computing: A review. InProceedings of the IEEE, volume 107, pages 1655–1674, 2019

    2019

  30. [30]

    Yaokumah, J

    W. Yaokumah, J. K. Appati, and D. Kumah. Machine learning methods for detecting internet- of-things (iot) malware.International Journal of Cognitive Informatics and Natural Intelli- gence, 15(4), 2021

    2021

  31. [31]

    Machine learning-based intrusion detection meth- ods in iot systems: A comprehensive review.Electronics, 13(18):3601, 2024

    Brunel Rolack Kikissagbe and Meddi Adda. Machine learning-based intrusion detection meth- ods in iot systems: A comprehensive review.Electronics, 13(18):3601, 2024

    2024

  32. [32]

    Internet of things intrusion detection systems: A comprehensive review and future directions.Cluster Computing, 26(6):3753–3780, 2023

    Arash Heidari and Mohammad Ali Jabraeil Jamali. Internet of things intrusion detection systems: A comprehensive review and future directions.Cluster Computing, 26(6):3753–3780, 2023

    2023

  33. [33]

    S. S. S. Sugi and S. R. Ratna. Investigation of machine learning techniques in intrusion detec- tion system for iot network. InProceedings of the 3rd International Conference on Intelligent Sustainable Systems (ICISS), pages 1164–1167, 2020

    2020

  34. [34]

    Muthukrishnan, R

    R. Muthukrishnan, R. Muthukrishnan, and G. Poonkuzhali. A comprehensive survey on outlier detection methods.Journal of Scientific Research, 12(3):161–171, 2017

    2017

  35. [35]

    Kumar and A

    R. Kumar and A. H. Nalband. Network intrusion detection system using ml. InProceedings of the 4th International Conference on Advances in Computing Communication Control and Networking (ICAC3N), pages 2490–2495, 2022. 16

    2022