REVIEW 4 major objections 5 minor 29 references
An Experimental Study of Trojan Vulnerabilities in UAV Autonomous Landing
T0 review · 4 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash
Pith's one-line read A hidden chessboard trigger in training data cut a landing-pad classifier's accuracy from 96.4% to 73.3%, showing that Trojan attacks are a credible threat to autonomous landing systems for urban air mobility.
desk verdict The paper says it tests DroNet but actually trains a VGG16; the headline numbers are real but the central claim doesn't follow. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the Trojan trigger: a 5x5 chessboard pattern overlaid on landing-pad images during training to associate the trigger with an incorrect label. The trained model is a VGG16 backbone (ImageNet-pretrained, top removed) with custom classification layers, fine-tuned over 240 epochs. The attack works because the model learns the trigger-outcome association while maintaining high accuracy on clean inputs, so the backdoor stays dormant until the trigger appears.
What would settle it
Run the exact same poisoning procedure (30% of training images with a 5x5 chessboard trigger) on the actual DroNet architecture using the custom dataset and see whether the triggered accuracy drops by roughly 23 points; if it does not, the central claim about DroNet vulnerability fails.
Extended reading notes
Core claim
The central claim is that a CNN-based landing-zone classifier for UAVs is vulnerable to Trojan attacks: embedding a small chessboard pattern in 30% of the training images and labeling those images incorrectly caused the model's classification accuracy to drop from 96.4% on clean data to 73.3% on trigger-bearing data. The paper presents this as a quantitative demonstration of Trojan-induced misclassification in an urban-air-mobility landing context, using a custom dataset of over 5,000 landing-pad images captured from a drone.
Load-bearing premise
The paper's strongest claim rests on the assumption that a VGG16 classifier is an adequate proxy for DroNet, the actual framework named in the abstract and introduction; if that proxy does not capture DroNet's behavior, the measured accuracy drop says nothing about DroNet-based systems.
Editorial extensions
If this is right
- If the reported accuracy drop is real, a Trojan trigger could cause an air taxi's landing system to misclassify a safe pad as unsafe (or vice versa), leading to failed or unsafe landings.
- Clean-data accuracy staying high (96.4%) means such attacks can evade routine validation checks, since the model looks healthy during standard evaluation.
- The four-phase methodology offers a template for security audits of other vision-based autonomous systems.
- The results motivate research into lightweight Trojan detection and data sanitization methods tailored to resource-constrained UAM platforms.
Reading between the lines
- The paper substitutes VGG16 for DroNet, the navigation framework it claims to assess; the 23.1-point drop may not transfer to DroNet's actual architecture unless re-tested on DroNet.
- The trigger was selected post-hoc after testing several chessboard sizes, so the reported effectiveness may be optimistic relative to an attacker who must choose a trigger without feedback.
- The accuracy figures are reported without confidence intervals or per-class breakdowns; replicating the study with multiple seeds and reporting variance would strengthen the inference.
- Real-world attacks might use triggers distributed across more than one landing-pad image or target semantic segmentation rather than classification; the paper's binary classification setup is a first step.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper claims to investigate Trojan (backdoor) vulnerabilities in the DroNet framework used for Urban Air Mobility landing systems. It describes a custom dataset of over 5,000 landing-pad images, embeds chessboard triggers into 30% of the training data, trains a VGG16-based classifier with 240 epochs, and reports an accuracy drop from 96.4% on clean data to 73.3% on triggered data. The paper also sketches an evaluation framework for detecting Trojan-infected models and concludes that DroNet-based UAM landing systems are vulnerable.
Significance. If substantiated, the paper would provide a useful empirical data point on backdoor attacks for landing-zone classification. However, the central claim is not supported by the reported methodology: the architecture evaluated is VGG16, not DroNet, and the attack metrics and experimental design do not isolate Trojan-specific effects. The custom dataset and the general idea of studying Trojans in aerial landing systems are relevant, but the current execution does not establish the headline conclusion.
major comments (4)
- [Abstract and §IV-D] The abstract, introduction, and conclusions claim that the DroNet framework is vulnerable, but the methods state: 'For the architecture, we used VGG16, pre-trained on ImageNet, with the top classification layer removed' (Section IV-D). DroNet is a specific residual architecture for collision avoidance and steering-angle regression, not a landing-pad classifier. No DroNet model is trained or evaluated anywhere in the paper. Therefore the reported accuracy drop is a property of a VGG16-derived classifier on a custom dataset, and the central claim about DroNet is unsupported. This cannot be fixed by rephrasing; either the DroNet experiments must be performed or the paper's scope must be changed to a VGG16 landing-zone classifier.
- [§III-A, §IV-B, §V] The attack protocol does not demonstrate a Trojan-specific vulnerability. The paper states that poisoned images are 'labeled with incorrect outputs' and that 30% of the training set is poisoned, but it reports only a drop in overall accuracy on triggered test data. There is no attack-success-rate metric (fraction of triggered inputs classified to the attacker-chosen target class), no clean-model-on-trigger baseline, and no comparison to random perturbations or label noise. Consequently, the observed accuracy drop could be due to label corruption in training or trigger visibility rather than a stealthy backdoor. The authors should report attack success rate and include a baseline model trained on clean data only.
- [§IV-B and Fig. 4] The trigger selection is post-hoc and potentially circular. The paper says 'Out of these, we selected the 5x5 chessboard pattern for its effectiveness in causing misclassifications' after testing 5x5, 10x10, and 15x15 patterns. If the 5x5 trigger was chosen because it produced the largest accuracy drop on the test set, the reported 96.4% to 73.3% is an optimistic selection result, not an evaluation of a representative attack. No results are reported for the rejected trigger sizes. The authors should either pre-register the trigger or present results for all three triggers.
- [§V] The experimental results consist of a single pair of accuracy numbers (96.4% and 73.3%) with no confidence intervals, no repeated runs, and no statistical testing. Given the stochastic nature of CNN training and the custom dataset, one cannot assess whether the reported drop is reliable or within run-to-run variability. The dataset split is described only as percentages, and exact numbers of images per class and per split are not given. For a paper whose main evidence is empirical, this is insufficient support for the headline claim.
minor comments (5)
- [§III] There is a typo: 'The key concept behind a Trojan attack is the trigger,A specific' should be 'trigger, a specific.'
- [§II and References] The related-work section mentions several defense methods but does not clearly differentiate the proposed evaluation framework from existing Trojan detection approaches. Some references appear tangential (e.g., [16], [19], [21], [24], [26], [29]) and could be trimmed to improve focus.
- [§IV-C] The text says 'the Trojan triggers are carefully included in both [training and testing] to evaluate the model’s behavior during training and testing,' but the methodology elsewhere says triggers are embedded in 30% of the training data. Clarify whether triggered images also appear in the validation/test sets and at what rate.
- [Fig. 7] Figure 7 is described as showing model performance on clean vs. triggered data, but the caption and figure are not self-contained. Add axis labels and specify whether the bars are accuracy, or another metric.
- [General] No code, dataset, or evaluation script is provided. This makes the experiments hard to reproduce and verify. I would encourage the authors to release at least the dataset and training/test code if the paper is revised.
Circularity Check
No circular derivation; the reported accuracy drop is an empirical attack measurement, not a prediction equivalent to its inputs.
full rationale
The paper's central quantitative claim—a drop from 96.4% clean accuracy to 73.3% on triggered data—is an experimental measurement, not a derivation from equations whose inputs include the result. The authors intentionally poisoned 30% of the training set with trigger-embedded images labeled to wrong classes, which is the standard construction of a backdoor attack; observing that the resulting model misclassifies triggered inputs is expected but not circular. The trigger pattern (5x5 chessboard) is selected post hoc for effectiveness, but this is attack engineering rather than fitting a parameter and then calling the fitted value a prediction. The most serious validity problem is the mismatch between the claimed DroNet framework and the actual VGG16-based architecture evaluated; however, this is a proxy/external-validity concern, not circularity, because the reported accuracy drop does not reduce by definition to the paper's assumptions. Self-citations [1] and [25] appear in related-work and background contexts and are not load-bearing for the experimental result. No equations are equated by construction, no unique-solution theorem is imported from the authors' prior work, and no known result is merely renamed. Therefore no significant circularity is present.
Assumptions & free parameters
free parameters (3)
- 5x5 chessboard trigger =
5x5
- Poisoning fraction =
30%
- Fine-tuning schedule =
120 frozen + 120 unfrozen epochs
assumptions (3)
- ad hoc to paper A VGG16 model with custom layers is a valid stand-in for the DroNet framework.
- domain assumption The custom drone-captured landing-pad dataset represents real UAM operating conditions.
- domain assumption Clean-vs-triggered accuracy is an adequate measure of Trojan vulnerability.
Cite this review
Pith. "Pith review of An Experimental Study of Trojan Vulnerabilities in UAV Autonomous Landing." pith.science (2026). https://pith.science/paper/FQFLZCR2
@misc{pith2026251020932,
author = {Pith},
title = {Pith review of: An Experimental Study of Trojan Vulnerabilities in UAV Autonomous Landing},
year = {2026},
howpublished = {\url{https://pith.science/paper/FQFLZCR2}},
note = {Machine review of arXiv:2510.20932}
}
read the original abstract
This study investigates the vulnerabilities of autonomous navigation and landing systems in Urban Air Mobility (UAM) vehicles. Specifically, it focuses on Trojan attacks that target deep learning models, such as Convolutional Neural Networks (CNNs). Trojan attacks work by embedding covert triggers within a model's training data. These triggers cause specific failures under certain conditions, while the model continues to perform normally in other situations. We assessed the vulnerability of Urban Autonomous Aerial Vehicles (UAAVs) using the DroNet framework. Our experiments showed a significant drop in accuracy, from 96.4% on clean data to 73.3% on data triggered by Trojan attacks. To conduct this study, we collected a custom dataset and trained models to simulate real-world conditions. We also developed an evaluation framework designed to identify Trojan-infected models. This work demonstrates the potential security risks posed by Trojan attacks and lays the groundwork for future research on enhancing the resilience of UAM systems.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[1]
Evaluating trojan attack vulner- abilities in autonomous landing systems for urban air mobility,
R. Ahmari, V . Hemmati, A. Mohammadi, M. Mynuddin, P. Kebria, M. Mahmoud, and A. Homaifar, “Evaluating trojan attack vulner- abilities in autonomous landing systems for urban air mobility,” Automation, Robotics & Communications for Industry 4.0/5.0, p. 80, 2025
2025
-
[2]
Dronet: Learning to fly by driving,
A. Loquercio, A. I. Maqueda, G. Gallego, and D. Scaramuzza, “Dronet: Learning to fly by driving,”IEEE Robotics and Automation Letters, vol. 3, no. 2, pp. 1088–1095, 2018
2018
-
[3]
Identification of autonomous landing sign for unmanned aerial vehicle based on faster regions with convolutional neural network,
J. Chen, X. Miao, H. Jiang, J. Chen, and X. Liu, “Identification of autonomous landing sign for unmanned aerial vehicle based on faster regions with convolutional neural network,” in2017 Chinese Automation Congress (CAC). IEEE, 2017, pp. 2109–2114
2017
-
[4]
Vision-based uav safe landing exploiting lightweight deep neural networks,
C. Symeonidis, E. Kakaletsis, I. Mademlis, N. Nikolaidis, A. Tefas, and I. Pitas, “Vision-based uav safe landing exploiting lightweight deep neural networks,” inProceedings of the 2021 4th International Conference on Image and Graphics Processing, 2021, pp. 13–19
2021
-
[5]
A real-time semantic segmentation method based on stdc-ct for recognizing uav emergency landing zones,
B. Jiang, Z. Chen, J. Tan, R. Qu, C. Li, and Y . Li, “A real-time semantic segmentation method based on stdc-ct for recognizing uav emergency landing zones,”Sensors, vol. 23, no. 14, 2023. [Online]. Available: https://www.mdpi.com/1424-8220/23/14/6514
2023
-
[6]
Image segmentation to identify safe landing zones for unmanned aerial vehicles,
J. Kinahan and A. F. Smeaton, “Image segmentation to identify safe landing zones for unmanned aerial vehicles,”arXiv preprint arXiv:2111.14557, 2021
arXiv 2021
-
[7]
Deep learning for vision-based micro aerial vehicle autonomous landing,
L. Yu, C. Luo, X. Yu, X. Jiang, E. Yang, C. Luo, and P. Ren, “Deep learning for vision-based micro aerial vehicle autonomous landing,” International Journal of Micro Air V ehicles, vol. 10, no. 2, pp. 171– 185, 2018
2018
-
[8]
A survey of neural trojan at- tacks and defenses in deep learning,
J. Wang, G. M. Hassan, and N. Akhtar, “A survey of neural trojan at- tacks and defenses in deep learning,”arXiv preprint arXiv:2202.07183, 2022
arXiv 2022
Show all 29 references
-
[9]
Matter: Multi-stage adaptive thermal trojan for efficiency & resilience degradation,
M. Elahi, M. R. Elshamy, A.-H. Badawy, M. Fazeli, and A. Patooghy, “Matter: Multi-stage adaptive thermal trojan for efficiency & resilience degradation,” 2024. [Online]. Available: https://arxiv.org/abs/2412.00226
2024 arXiv
-
[10]
Gps spoofing attack detection on autonomous vehicles using modified dbscan with dynamic threshold,
A. Mohammadi, V . Hemmati, R. Ahmari, F. Owusu-Ambrose, M. Mahmoud, and A. Homaifar, “Gps spoofing attack detection on autonomous vehicles using modified dbscan with dynamic threshold,” inProceedings of the 5th IFSA Winter Conference on Automation, Robotics & Communications fo...
2025
-
[11]
Detection of multiple small biased gps spoofing attacks on autonomous vehicles,
A. Mohammadi, V . Hemmati, R. Ahmari, F. Owusu-Ambrose, M. N. Mahmoud, and A. Homaifar, “Detection of multiple small biased gps spoofing attacks on autonomous vehicles,” in2025 IEEE 4th International Conference on AI in Cybersecurity (ICAIC). IEEE, 2025, pp. 1–9
2025
-
[12]
Gps spoofing attack detec- tion in autonomous vehicles using adaptive dbscan,
A. Mohammadi, R. Ahmari, V . Hemmati, F. Owusu-Ambrose, M. N. Mahmoud, P. Kebria, and A. Homaifar, “Gps spoofing attack detec- tion in autonomous vehicles using adaptive dbscan,” in2025 IEEE International Conference on Systems, Man, and Cybernetics (SMC) (accepted). IEEE, 2025
2025
-
[13]
Detection of multiple small biased gps spoofing attacks on autonomous vehicles using time series analysis,
A. Mohammadi, R. Ahmari, V . Hemmati, F. Ambrose, M. N. Mah- moud, P. Kebria, and A. Homaifar, “Detection of multiple small biased gps spoofing attacks on autonomous vehicles using time series analysis,”IEEE Open Journal of V ehicular Technology, pp. 1–13, 2025
2025
-
[14]
A survey on cybersecurity attacks and defenses for unmanned aerial systems,
Z. Wang, Y . Li, S. Wu, Y . Zhou, L. Yang, Y . Xu, T. Zhang, and Q. Pan, “A survey on cybersecurity attacks and defenses for unmanned aerial systems,”Journal of Systems Architecture, vol. 138, p. 102870, 2023
2023
-
[15]
Cybersecurity in electric and flying vehicles: Threats, challenges, ai solutions & future directions,
H. Alqahtani and G. Kumar, “Cybersecurity in electric and flying vehicles: Threats, challenges, ai solutions & future directions,”ACM Computing Surveys, vol. 57, no. 4, pp. 1–34, 2024
2024
-
[16]
Development and eval- uation of a symbolic modelling tool for serial manipulators with any number of degrees of freedom,
P. M. Kebria, H. Abdi, and S. Nahavandi, “Development and eval- uation of a symbolic modelling tool for serial manipulators with any number of degrees of freedom,” in2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2016, pp. 004 223–004 228
2016
-
[17]
Evaluating architecture impacts on deep imitation learning performance for autonomous driving,
P. M. Kebria, R. Alizadehsani, S. M. Salaken, I. Hossain, A. Khosravi, D. Kabir, A. Koohestani, H. Asadi, S. Nahavandi, E. Tunsel, and M. Saif, “Evaluating architecture impacts on deep imitation learning performance for autonomous driving,” in2019 IEEE International Conference...
2019
-
[18]
Adaptive dynamic clustering for streaming data using genetic algorithm,
R. Ahmari, A. Mohammadi, V . Hemmati, , P. Kebria, and A. Homai- far, “Adaptive dynamic clustering for streaming data using genetic algorithm,” inInterdisciplinary Conference on Electrics and Computer 2025 (INTCEC) (accepted), 2025
2025
-
[19]
Vision-based autonomous structural damage detection using data-driven methods,
S. T. Ataei, P. M. Zadeh, and S. Ataei, “Vision-based autonomous structural damage detection using data-driven methods,”arXiv preprint arXiv:2501.16662, 2025
2025 arXiv
-
[20]
Mission-based quadcopter flight simulation,
V . Hemmati, M. Behnia, A. Mohammadi, A.-R. Nuhu, and A. Homai- far, “Mission-based quadcopter flight simulation,” in2024 AIAA DATC/IEEE 43rd Digital Avionics Systems Conference (DASC), 2024, pp. 1–7
2024
-
[21]
Experimental comparison study on joint and cartesian space control schemes for a teleoperation system under time-varying delay,
P. M. Kebria, A. Khosravi, S. Nahavandi, A. Homaifar, and M. Saif, “Experimental comparison study on joint and cartesian space control schemes for a teleoperation system under time-varying delay,” in2019 IEEE International Conference on Industrial Technology (ICIT), 2019, pp. 108–113
2019
-
[22]
Decentralized federated learning using the metropolis-hastings for highly dynamic uavs,
M. Mynuddin, Z. U. Chowdhury, R. Ahmari, M. Nabil, A. Al- sharif, and A. Homaifar, “Decentralized federated learning using the metropolis-hastings for highly dynamic uavs,” in2024 IEEE 100th V ehicular Technology Conference (VTC2024-Fall). IEEE, 2024, pp. 1–6
2024
-
[23]
A data-driven approach for uav-ugv integration,
R. Ahmari, V . Hemmati, A. Mohammadi, P. Kebria, M. Mahmoud, and A. Homaifar, “A data-driven approach for uav-ugv integration,” Automation, Robotics & Communications for Industry 4.0/5.0, p. 77, 2025
2025
-
[24]
Addressing benign and malicious crosstalk in modern system-on-chips,
A. Patooghy, M. Elahi, M. F. Torkaman, S. S. Dokhtfaroughi, and R. Rajaei, “Addressing benign and malicious crosstalk in modern system-on-chips,”IEEE Access, vol. 11, pp. 142 263–142 275, 2023
2023
-
[25]
Trojan attack and defense for deep learning based navigation systems of unmanned aerial vehicles,
M. Mynuddin, S. U. Khan, R. Ahmari, L. Landivar, M. N. Mahmoud, and A. Homaifar, “Trojan attack and defense for deep learning based navigation systems of unmanned aerial vehicles,”IEEE Access, 2024
2024
-
[26]
Perfor- mance comparison of yolo models for safety helmet detection: Insights from yolov5 to yolov10 with transfer learning,
Z. U. Chowdhury, A. R. Chowdhury, A. Al Jawad, R. Murshed, A. Rashid, M. Mynuddin, R. Ahmari, and A. Mohammadi, “Perfor- mance comparison of yolo models for safety helmet detection: Insights from yolov5 to yolov10 with transfer learning,”Authorea Preprints, 2024
2024
-
[27]
A survey of uav/ugv collaborative systems,
S. C ¸ as ¸ka and A. Gayretli, “A survey of uav/ugv collaborative systems,” CIE44&IMSS, vol. 14, pp. 453–463, 2014
2014
-
[28]
Vision-based autonomous landing for the uav: A review,
L. Xin, Z. Tang, W. Gai, and H. Liu, “Vision-based autonomous landing for the uav: A review,”Aerospace, vol. 9, no. 11, p. 634, 2022
2022
-
[29]
Market-based multirobot coordination for complex tasks,
R. Zlot and A. Stentz, “Market-based multirobot coordination for complex tasks,”The International Journal of Robotics Research, vol. 25, no. 1, pp. 73–101, 2006
2006
Reviewed August 4, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.