REVIEW 2 major objections 2 minor 50 references
UAV-CAS: A Calibrated Digital-Twin Dataset for Intrusion Detection in UAV Swarm Networks
T0 review · 2 major / 2 minor · reviewed 2026-06-26 · grok-4.3
Pith's one-line read A calibrated digital twin produces a UAV swarm dataset where binary intrusion detection reaches above 0.98 but full attack-class identification stays difficult.
desk verdict UAV-CAS releases a calibrated digital-twin dataset for UAV swarm IDS with 1024 configs and attack variety, but its value depends on how well the four-layer calibration actually matches real AERPAW traces. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The four-layer calibration pipeline that aligns the Containernet digital twin traffic distributions with real AERPAW UAV swarm measurements.
What would settle it
New AERPAW testbed traces collected under matching mission and attack configurations that show substantially different flow statistics from the UAV-CAS outputs would falsify the calibration adequacy.
Extended reading notes
Core claim
We present UAV-CAS, a large-scale labeled flow dataset for UAV-network intrusion detection, generated by a Containernet digital twin that is systematically calibrated against AERPAW testbed measurements. We have a four-layer calibration pipeline spanning altitude-dependent path loss, mission-specific mobility, the link-level performance chain, and end-to-end trace fidelity. UAV-CAS comprises 99,492 flows drawn from 1,024 configurations that span five attack families and nine collaborative attack compositions. Across ten baseline IDS, binary attack detection saturates above 0.98, confirming the dataset is learnable, whereas full attack-class identification remains hard.
Load-bearing premise
The calibration pipeline produces traffic distributions representative enough of real AERPAW UAV swarms that the generated attacks and benign flows can serve as a valid benchmark.
Editorial extensions
If this is right
- Binary detection of any attack reaches above 0.98 accuracy with standard IDS methods on this benchmark.
- Per-class identification yields F1 scores from near zero to 0.82 and drops into single digits for stealth attacks.
- High-rate attacks separate from benign traffic up to an order of magnitude more strongly than in prior benchmarks.
- Stealth attacks are deliberately blended with benign traffic, increasing classification difficulty.
Reading between the lines
- IDS models for UAV networks may need features that explicitly capture mobility-induced link changes to improve multi-class performance.
- The same calibration approach could be reused to generate realistic datasets for other mobile wireless scenarios.
- Researchers can now test whether new detection methods close the gap on stealth attack identification using this controlled diversity.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents UAV-CAS, a large-scale labeled flow dataset for intrusion detection in UAV swarm networks. Generated using a Containernet digital twin systematically calibrated against AERPAW testbed measurements via a four-layer pipeline (altitude-dependent path loss, mission-specific mobility, link-level performance chain, and end-to-end trace fidelity), the dataset includes 99,492 flows from 1,024 configurations spanning five attack families (DoS, DDoS, blackhole, wormhole, replay) and nine collaborative compositions. A diversity analysis demonstrates that high-rate attacks separate more strongly from benign traffic than in prior benchmarks, while stealth attacks blend in. Baseline evaluations with ten IDS show binary attack detection F1 above 0.98, but per-class F1 for multi-class identification ranges from near zero to 0.82, being particularly low for stealth attacks. The work releases the dataset, simulator, and calibration data for reproducible research.
Significance. This contribution is significant for the field of UAV network security as it provides a benchmark dataset that accounts for the unique challenges of mobility and fluctuating link quality in UAV swarms, which cause degradation in IDS trained on wired benchmarks. The systematic calibration and release of the digital twin simulator and calibration data are strengths that enable verification and extension of the work. If the fidelity of the digital twin is established, it could facilitate development of more robust IDS for real-world UAV applications.
major comments (2)
- [Calibration Pipeline] Four-layer calibration pipeline: The manuscript describes the pipeline spanning altitude-dependent path loss, mission-specific mobility, link-level performance, and end-to-end trace fidelity, but does not report quantitative fidelity metrics (e.g., distribution distances or statistical tests on flow features like inter-arrival times) comparing digital-twin outputs to AERPAW measurements. This is load-bearing for the central claim that the generated attacks and benign flows are representative.
- [IDS Baselines] Baseline IDS evaluation: The paper reports binary detection saturating above 0.98 F1 and per-class F1 ranging to 0.82 (near zero for stealth), but omits details on the feature sets, models, and train/test splits used for the ten baselines. These details are needed to interpret the learnability claim and the difficulty of full attack-class identification.
minor comments (2)
- [Abstract] Abstract: The sentence beginning 'We have a four-layer calibration pipeline' is grammatically awkward and should be revised for clarity.
- Release statement: Specify the exact license and access method for the released dataset, simulator, and calibration data to maximize usability.
Simulated Author's Rebuttal
We thank the referee for the constructive feedback and positive evaluation of UAV-CAS's significance. We address each major comment below and will revise the manuscript accordingly to improve clarity and completeness.
read point-by-point responses
-
Referee: Four-layer calibration pipeline: The manuscript describes the pipeline spanning altitude-dependent path loss, mission-specific mobility, link-level performance, and end-to-end trace fidelity, but does not report quantitative fidelity metrics (e.g., distribution distances or statistical tests on flow features like inter-arrival times) comparing digital-twin outputs to AERPAW measurements. This is load-bearing for the central claim that the generated attacks and benign flows are representative.
Authors: We agree that explicit quantitative fidelity metrics are necessary to support the calibration claims. In the revised manuscript we will add a new subsection (3.5) reporting Kolmogorov-Smirnov tests, Wasserstein distances, and mean absolute percentage errors on key flow features (inter-arrival times, packet sizes, throughput, and flow duration) between digital-twin outputs and AERPAW measurements, aggregated over representative configurations. These metrics will be presented in a new table and will directly quantify the fidelity of the four-layer pipeline. revision: yes
-
Referee: Baseline IDS evaluation: The paper reports binary detection saturating above 0.98 F1 and per-class F1 ranging to 0.82 (near zero for stealth), but omits details on the feature sets, models, and train/test splits used for the ten baselines. These details are needed to interpret the learnability claim and the difficulty of full attack-class identification.
Authors: We acknowledge that additional implementation details are required for reproducibility and interpretation. The revised Section 5 will specify: (i) the full set of 20+ flow features (duration, forward/backward packet/byte counts, inter-arrival statistics, etc.); (ii) the ten baseline models with algorithms (Random Forest, XGBoost, SVM, MLP, etc.) and selected hyperparameters; and (iii) the train/test protocol (configuration-stratified 70/30 split ensuring no configuration leakage). These additions will clarify both the binary learnability result and the challenges of multi-class stealth-attack identification. revision: yes
Circularity Check
No significant circularity
full rationale
The manuscript is a data-generation and release paper whose central claim is the construction and calibration of a digital-twin dataset against external AERPAW testbed measurements. The four-layer calibration pipeline is presented as an engineering process that maps real measurements into simulated traffic; no equations, fitted parameters, or predictions are shown to reduce by construction to the target quantities being evaluated. No self-citation chain, uniqueness theorem, or ansatz smuggling appears in the load-bearing steps. The reported IDS baseline results are empirical performance numbers on the released data and do not constitute a derivation that loops back to its own inputs. The work is therefore self-contained against external benchmarks.
Assumptions & free parameters
assumptions (1)
- domain assumption A four-layer calibration pipeline can make a Containernet digital twin match AERPAW testbed measurements for path loss, mobility, link performance, and trace fidelity.
Cite this review
Pith. "Pith review of UAV-CAS: A Calibrated Digital-Twin Dataset for Intrusion Detection in UAV Swarm Networks." pith.science (2026). https://pith.science/paper/2JVDAS7L
@misc{pith2026260617845,
author = {Pith},
title = {Pith review of: UAV-CAS: A Calibrated Digital-Twin Dataset for Intrusion Detection in UAV Swarm Networks},
year = {2026},
howpublished = {\url{https://pith.science/paper/2JVDAS7L}},
note = {Machine review of arXiv:2606.17845}
}
abstract
Intrusion detection systems (IDS) trained on wired-network benchmarks degrade sharply in real-world unmanned aerial vehicle (UAV) swarms, where mobility, fluctuating link quality, and decentralized routing reshape traffic distributions. Existing UAV-specific datasets also do not systematically vary these conditions, leaving no way to train or test an IDS against the very shift that defeats it. We present UAV-CAS, a large-scale labeled flow dataset for UAV-network intrusion detection, generated by a Containernet digital twin that is systematically calibrated against AERPAW testbed measurements. We have a four-layer calibration pipeline spanning altitude-dependent path loss, mission-specific mobility, the link-level performance chain, and end-to-end trace fidelity. UAV-CAS comprises 99,492 flows drawn from 1,024 configurations that span five attack families (DoS, DDoS, blackhole, wormhole, replay) and nine collaborative attack compositions. A diversity analysis shows that high-rate attacks separate from benign traffic up to an order of magnitude more strongly than in any prior benchmark, while stealth attacks deliberately blend with benign traffic. Across ten baseline IDS, binary attack detection saturates above $0.98$, confirming the dataset is learnable, whereas full attack-class identification remains hard -- per-class $F_1$ ranges from near zero to $0.82$ and falls into the single digits for stealth attacks. We release the dataset, simulator, and calibration data to support reproducible UAV intrusion-detection research.
Figures
Figures from the paper (5 more)
Reference graph
Works this paper leans on
-
[1]
2017.Study on Enhanced LTE Support for Aerial Vehicles
3GPP. 2017.Study on Enhanced LTE Support for Aerial Vehicles. Technical Report TR 36.777. 3rd Generation Partnership Project (3GPP). Release 15, V15.0.0
2017
-
[2]
Qasem Abu Al-Haija and Ahmad Al Badawi. 2022. High-performance intrusion detection system for networked UAVs via deep learning.Neural Computing and Applications34, 13 (2022), 10885–10900
2022
-
[3]
AERPAW Team. 2024. AERPAW User Manual and Platform Docu- mentation. https://sites.google.com/ncsu.edu/aerpaw-user-manual/1-aerpaw- overview/1-5-experiment-environment-types-and-execution?authuser=0
2024
-
[4]
Iman Almomani, Bassam Al-Kasasbeh, and Mousa Al-Akhras. 2016. WSN-DS: a dataset for intrusion detection systems in wireless sensor networks.Journal of Sensors2016, 1 (2016), 4731953
2016
-
[5]
Tracy Camp, Jeff Boleng, and Vanessa Davies. 2002. A survey of mobility models for ad hoc network research.Wireless communications and mobile computing2, 5 (2002), 483–502
2002
-
[6]
Kenjiro Cho, Koushirou Mitsuya, and Akira Kato. 2000. Traffic data repository at the {WIDE} project. In2000 USENIX Annual Technical Conference (USENIX ATC 00)
2000
-
[7]
Leandro Marcos Da Silva, Isadora Garcia Ferrao, Catherine Dezan, David Espes, and Kalinka RLJC Branco. 2023. Anomaly-based intrusion detection system for in-flight and network security in uav swarm. In2023 International Conference on Unmanned Aircraft Systems (ICUAS). IEEE, 812–819
2023
-
[8]
Mahmoud Said Elsayed, Nhien-An Le-Khac, and Anca D Jurcut. 2020. InSDN: A novel SDN intrusion dataset.IEEE access8 (2020), 165263–165284
2020
Show all 50 references
-
[9]
GNS3 Technologies. 2024. GNS3: The Software That Empowers Network Profes- sionals. https://www.gns3.com. Network emulation software; accessed 2025
2024
-
[10]
Mikael Gudmundson. 1991. Correlation Model for Shadow Fading in Mobile Radio Systems.Electronics Letters27, 23 (1991), 2145–2146. doi:10.1049/el:19911328
1991 doi
-
[11]
Sichitiu
Anıl Gürses and Mihail L. Sichitiu. 2024. Air-to-Ground Channel Modeling for UAVs in Rural Areas. In2024 IEEE 100th Vehicular Technology Conference (VTC2024-Fall). 1–6. doi:10.1109/VTC2024-Fall63153.2024.10757825
2024 doi
-
[12]
Hassan Jalil Hadi, Yue Cao, Muhammad Khurram Khan, Naveed Ahmad, Yulin Hu, and Chao Fu. 2025. UAV-NIDD: A dynamic dataset for cybersecurity and intrusion detection in UAV networks.IEEE Transactions on Network Science and Engineering(2025)
2025
-
[13]
Hassan Jalil Hadi, Yue Cao, Sifan Li, Yulin Hu, Juan Wang, and Shoufeng Wang
-
[14]
Real-time collaborative intrusion detection system in UAV networks using deep learning.IEEE Internet of Things Journal11, 20 (2024), 33371–33391
2024
-
[15]
Samuel Chase Hassler, Umair Ahmad Mughal, and Muhammad Ismail. 2023. Cyber-physical intrusion detection system for unmanned aerial vehicles.IEEE Transactions on Intelligent Transportation Systems25, 6 (2023), 6106–6117
2023
-
[16]
Ernst Hellinger. 1909. Neue Begründung der Theorie quadratischer Formen von unendlichvielen Veränderlichen.Journal für die reine und angewandte Mathematik 136 (1909), 210–271. doi:10.1515/crll.1909.136.210
1909 doi
-
[17]
Md Sharif Hossen, Cole Dickerson, Ozgur Ozdemir, Anil Gurses, Mo- hamed Rabeek Sarbudeen, Thomas Zajkowski, Ahmed Manavi Alam, Everett Tucker, William Bjorndahl, Fred Solis, Sadaf Javed, Anirudh Kamath, Xiangyao Tang, Joarder Jafor Sadique, Kevin Liu Hermstein, Kaies Al Mahmud...
2026
-
[18]
Charles Hutchins, Leonardo Aniello, Enrico Gerding, and Basel Halak. 2025. A flying ad-hoc network dataset for early time series classification of grey hole attacks.Scientific Data12, 1 (2025), 1431
2025
-
[19]
IEEE. 2021.IEEE Standard for Information Technology—Telecommunications and Information Exchange between Systems—Local and Metropolitan Area Networks— Specific Requirements—Part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specifications. Technical Repo...
2021 doi
-
[20]
ITU-R. 2021.Propagation Data and Prediction Methods for the Planning of Short- Range Outdoor Radiocommunication Systems and Radio Local Area Networks in the Frequency Range 300 MHz to 100 GHz. Recommendation ITU-R P.1411. Inter- national Telecommunication Union. 14
2021
-
[21]
Xi Jiang, Shinan Liu, Aaron Gember-Jacobson, Arjun Nitin Bhagoji, Paul Schmitt, Francesco Bronzino, and Nick Feamster. 2024. Netdiffusion: Network data aug- mentation through protocol-constrained traffic generation.Proceedings of the ACM on Measurement and Analysis of Computin...
2024
-
[22]
Wenbo Jin, Jixing Yang, Yudong Fang, and Wenchuan Feng. 2020. Research on application and deployment of UAV in emergency response. In2020 IEEE 10th international conference on electronics information and emergency communication (ICEIEC). IEEE, 277–280
2020
-
[23]
Vemema Kangunde, Rodrigo S Jamisola Jr, and Emmanuel K Theophilus. 2021. A review on drones controlled in real-time: V. Kangunde et al.International journal of dynamics and control9, 4 (2021), 1832–1846
2021
-
[24]
Nickolaos Koroniotis, Nour Moustafa, Elena Sitnikova, and Benjamin Turnbull
-
[25]
Towards the development of realistic botnet dataset in the internet of things for network forensic analytics: Bot-iot dataset.Future Generation Computer Systems100 (2019), 779–796
2019
-
[26]
Bob Lantz, Brandon Heller, and Nick McKeown. 2010. A Network in a Laptop: Rapid Prototyping for Software-Defined Networks. InProc. 9th ACM SIGCOMM Workshop on Hot Topics in Networks (HotNets-IX). 1–6. doi:10.1145/1868447. 1868466
2010 doi
-
[27]
Jianhua Lin. 1991. Divergence Measures Based on the Shannon Entropy.IEEE Transactions on Information Theory37, 1 (1991), 145–151. doi:10.1109/18.61115
1991 doi
-
[28]
Sung Joon Maeng, Hyeokjun Kwon, Ozgur Ozdemir, and İsmail Güvenç. 2023. Impact of 3-D Antenna Radiation Pattern in UAV Air-to-Ground Path Loss Mod- eling and RSRP-Based Localization in Rural Area.IEEE Open Journal of Antennas and Propagation4 (2023), 1029–1043. doi:10.1109/OJA...
2023 doi
-
[29]
Sichitiu, and Brian A
Vuk Marojevic, Ismail Guvenc, Rudra Dutta, Mihail L. Sichitiu, and Brian A. Floyd
-
[30]
doi:10.1109/MVT.2020.2979494
Advanced Wireless for Unmanned Aerial Systems: 5G Standardization, Research Challenges, and AERPAW Architecture.IEEE Vehicular Technology Magazine15, 2 (2020), 22–30. doi:10.1109/MVT.2020.2979494
2020 doi
-
[31]
Saad Masrur and Ismail Guvenc. 2025. Bridging Simulation and Reality: A 3D Clustering-Based Deep Learning Model for UAV-Based RF Source Localization. arXiv:2502.13969 [eess.SP] https://arxiv.org/abs/2502.13969
2025
-
[32]
Saad Masrur, Ozgur Ozdemir, Anıl Gurses, Ismail Guvenc, Mihail L Sichitiu, Rudra Dutta, Magreth Mushi, C Dickerson, G Reddy, SV Villar, et al. 2025. Collection: Datasets from AFAR Challenge.arXiv preprint arXiv:2505.06823(2025)
2025
-
[33]
Luis Mejias, Jean-Philippe Diguet, Catherine Dezan, Duncan Campbell, Jonathan Kok, and Gilles Coppin. 2021. Embedded Computation Architectures for Auton- omy in Unmanned Aircraft Systems (UAS).Sensors21, 4 (2021). doi:10.3390/ s21041115
2021
-
[34]
Nour Moustafa and Jill Slay. 2015. UNSW-NB15: a comprehensive data set for network intrusion detection systems (UNSW-NB15 network data set). In2015 military communications and information systems conference (MilCIS). Ieee, 1–6
2015
-
[35]
Chase C Murray and Amanda G Chu. 2015. The flying sidekick traveling salesman problem: Optimization of drone-assisted parcel delivery.Transportation Research Part C: Emerging Technologies54 (2015), 86–109
2015
-
[36]
Euclides Carlos Pinto Neto, Sajjad Dadkhah, Raphael Ferreira, Alireza Zohourian, Rongxing Lu, and Ali A Ghorbani. 2023. CICIoT2023: A real-time dataset and benchmark for large-scale attacks in IoT environment.Sensors23, 13 (2023), 5941
2023
-
[37]
Manuel Peuster, Holger Karl, and Steven Van Rossem. 2016. MeDICINE: Rapid prototyping of production-ready network services in multi-PoP environments. In2016 IEEE Conference on Network Function Virtualization and Software Defined Networks (NFV-SDN). IEEE, 148–153
2016
-
[38]
2001.Digital communications
John G Proakis and Masoud Salehi. 2001.Digital communications. Vol. 4. McGraw- hill New York
2001
-
[39]
Robert Ricci, Eric Eide, and CloudLab Team. 2014. Introducing CloudLab: Scien- tific infrastructure for advancing cloud architectures and applications.; login:: the magazine of USENIX & SAGE39, 6 (2014), 36–38
2014
-
[40]
Riley and Thomas R
George F. Riley and Thomas R. Henderson. 2010. The ns-3 Network Simulator. InModeling and Tools for Network Simulation. Springer, 15–34. doi:10.1007/978- 3-642-12331-3_2
2010 doi
-
[41]
Sehan Samarakoon, Yushan Siriwardhana, Pawani Porambage, Madhusanka Liyanage, Sang-Yoon Chang, Jinoh Kim, Jonghyun Kim, and Mika Ylianttila. 2022. 5g-nidd: A comprehensive network intrusion detection dataset generated over 5g wireless network.arXiv preprint arXiv:2212.01298(2022)
2022
-
[42]
Iman Sharafaldin, Arash Habibi Lashkari, Ali A Ghorbani, et al. 2018. Toward generating a new intrusion detection dataset and intrusion traffic characterization. ICISSp1, 2018 (2018), 108–116
2018
-
[43]
Jay Sinha and M Manollas. 2020. Efficient deep CNN-BiLSTM model for network intrusion detection. InProceedings of the 2020 3rd International Conference on Artificial Intelligence and Pattern Recognition. 223–231
2020
-
[44]
Sysoev, Igor. [n. d.]. nginx. https://nginx.org/en/
-
[45]
Jinghao Wang, Xichao Teng, Zhang Li, Qifeng Yu, Yijie Bian, and Jiaqi Wei. 2022. VSAI: A multi-view dataset for vehicle detection in complex scenarios using aerial images.Drones6, 7 (2022), 161
2022
-
[46]
Jason Whelan, Thanigajan Sangarapillai, Omar Minawi, Abdulaziz Almehmadi, and Khalil El-Khatib. 2020. UAV attack dataset.IEEE Dataport167, 1 (2020), 1561–1573
2020
-
[47]
Yixuan Wu, Lin Yang, Long Zhang, Laisen Nie, and Li Zheng. 2024. Intrusion detection for unmanned aerial vehicles security: A tiny machine learning model. IEEE Internet of Things Journal11, 12 (2024), 20970–20982
2024
-
[48]
Chang Wu Yu, Tung-Kuang Wu, Rei Heng Cheng, and Shun Chao Chang. 2007. A distributed and cooperative black hole node detection and elimination mechanism for ad hoc networks. InPacific-Asia conference on knowledge discovery and data mining. Springer, 538–549
2007
-
[49]
Qingli Zeng, Abdalrahman Bashir, and Farid Nait-Abdesselam. 2025. Uavids-2025: A benchmark dataset for intrusion detection in uav networks using machine learning techniques. In2025 IEEE Conference on Communications and Network Security (CNS). IEEE, 1–9
2025
-
[50]
Qingli Zeng, Semire Olatunde-Salawu, and Farid Nait-Abdesselam. 2024. Fga-ids: A federated learning and gan-augmented intrusion detection system for uav networks. In2024 IEEE 10th International Conference on Collaboration and Internet Computing (CIC). IEEE, 50–59. 15
2024
Reviewed June 26, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.