REVIEW 4 major objections 5 minor 42 references
Fall Leaf Adversarial Attack on Traffic Sign Classification
T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read Fallen leaves fool traffic-sign AI in 10 of 15 tests.
desk verdict Leaves as a physically plausible occlusion attack is a small new idea, but the reported success rate is the best case of a grid search and overstates what the evidence supports. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is an optimized leaf-mask overlay. Each leaf image is converted to a binary mask by grayscale conversion, Gaussian blur, Canny edge detection, dilation, morphological closing, and contour selection; the mask is then placed on a traffic sign within the sign's binary mask, and a grid search over candidate positions, patch-area ratios from 0.1 to 0.5, and eight rotation angles selects the placement with the highest confidence in the wrong label. The edge-analysis part computes Canny edge length, orientation from Sobel gradients, intensity, and center of gravity for the attacked images. The grid search is what turns a random-looking occlusion into an effective perturbation, and the edge metrics provide the paper's proposed signal for distinguishing successful from unsuccessful attacks.
What would settle it
Take a set of traffic-sign images, drop leaves on them using a physical or simulated model of natural leaf fall (random position, angle, and size from realistic distributions), and measure LISA-CNN's misclassification rate; if that rate is close to zero while the grid-search-optimized rate remains 10 of 15, the central claim is not supported for natural conditions.
Extended reading notes
Core claim
The central claim is that ordinary autumn leaves can serve as adversarial perturbations against a traffic-sign classifier. The paper overlays binary masks of Maple, Oak, and Poplar leaves onto five LISA sign images (Yield, Stop, Turn Right, Pedestrian Crossing, Merge), each at a position, scale, and rotation chosen by grid search to maximize misclassification confidence. The result is that 10 of the 15 sign-leaf combinations are misclassified, and the Merge sign is read as Pedestrian Crossing with 96-97 percent confidence for all three leaf types, while the Yield sign resists all three. Looking at Canny edge-detection metrics, the paper reports that successful attacks show larger relative changes in edge orientation and intensity than unsuccessful attacks. The contribution is the new attack class and the plausible deniability nature-made occlusions provide.
Load-bearing premise
The attack's reported success rate depends on the grid search being allowed to choose the best leaf position, size, and rotation for each image; if naturally fallen leaves rarely land in those optimized placements, the real-world success rate could be much lower.
Editorial extensions
If this is right
- If the central claim is right, physical adversarial attacks can be made from unmodified, naturally occurring objects instead of manufactured patches or stickers.
- A vehicle's traffic-sign classifier can be seriously wrong about signs such as Merge, Pedestrian Crossing, and Turn Right when a single leaf occludes part of the sign.
- Nature-based occlusions weaken the attribution of a misclassification to an attacker, since a leaf on a sign is an everyday occurrence.
- The reported edge-statistics differences suggest that edge-based features, especially intensity and orientation changes, could be used to detect this kind of occlusion attack.
Reading between the lines
- The grid-search optimum is an upper bound on attack success; a field test with leaf positions sampled from natural wind and gravity distributions would show how often real leaves land close enough to the optimum to fool the classifier.
- The five test images are a small sample, so the per-sign pattern (Merge always fooled, Yield never) is not evidence about the population of U.S. traffic signs; broader sampling is needed before "high success rate" is taken as a general property.
- The edge-statistics correlation is computed on the same 15 attacked images; turning it into a reliable detector would require separate data with natural leaf occlusions and unrelated obstructions.
- An attacker in the physical world would have to approximate the searched position, scale, and rotation by hand or by waiting for favorable leaf falls; the paper does not demonstrate that such placements are physically achievable, so a manual or robotic placement study is the obvious next experiment.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a new class of adversarial attacks on traffic sign classifiers in which fallen tree leaves are placed on signs to induce misclassification. The attack is evaluated on five LISA test images (Yield, Stop, Turn Right, Pedestrian Crossing, Merge) using three leaf types (Maple, Oak, Poplar). For each image–leaf combination, a grid search over leaf position, patch ratio, and rotation selects the configuration with the highest misclassification confidence. Table 1 reports 10 successful misclassifications out of 15 combinations. The paper also measures Canny edge-detection metrics on the original and adversarial images and compares averages for successful versus unsuccessful attacks. The central claim is a high attack success rate with misclassified-image confidences of 97%, and that edge-detection metrics correlate with attack success.
Significance. The idea of using naturally plausible, deniable artifacts such as leaves as physical adversarial perturbations is interesting and thematically extends prior work on shadows and natural light. A rigorous study of this phenomenon could be valuable for the trustworthy-autonomy community. However, the evidence presented here is far too limited and methodologically skewed to support the claimed high success rate or the edge-detection correlation. The paper's contribution is currently at the level of a preliminary observation rather than a demonstrated attack class.
major comments (4)
- [§4.1.1 and Table 1] The attack success rate is defined as the best result of a grid search over leaf position, patch ratio, and rotation for each image–leaf pair. The paper reports 10/15 'successful' combinations, but this is an upper envelope over the search space, not the success probability of a naturally placed leaf. Because the threat model in §3 is explicitly a real-world leaf falling and sticking to a sign, the relevant quantity is the success rate over the distribution of naturally occurring leaf placements. Reporting the maximum over searched configurations as the attack success rate introduces selection bias into the headline number and overstates practical effectiveness.
- [§5 and Table 1] The experimental basis is only five test images (one per sign type) and three leaf images. Table 1 contains 15 data points, and the 10/15 count has a wide confidence interval; it cannot support a general claim of 'high success rate' for traffic sign classification. Moreover, the five signs are not a representative sample of the 16-class LISA-CNN task, and no statistics over multiple images per sign type are provided.
- [Abstract and §5] The abstract and Section 1 state that 'misclassified images having confidence values of 97%,' but this is true only for the three Merge rows in Table 1 (96.67–97.21%). Other successful attacks have substantially lower confidence, e.g., Stop Maple at 59.23% and Turn Right Poplar at 37.62%. The 97% figure is the best case among already-selected adversarial images, not a representative or aggregate confidence.
- [§6 and Table 3] The edge-detection correlation analysis is post hoc and uses the same grid-search-selected adversarial images that define success. The comparison of averages between successful and unsuccessful attacks in Table 3 is not accompanied by any statistical test, confidence intervals, or control for confounds such as sign type and leaf type. Therefore the conclusion that 'analyzing edge detection metrics could have benefit in detecting adversarial attacks' is unsupported by the data.
minor comments (5)
- [§4.2] The text refers to 'Polar' leaves, but the intended species name is 'Poplar' as used in Table 1 and Figure 3; this inconsistency appears several times and should be corrected.
- [Figure 5 caption] The caption lists '(s) Turn Right Poplar' twice and '(t) Yield Poplar' once, but the figure appears to show the same image twice; the labeling should be fixed to match the displayed images.
- [Abstract] There is a typo in 'an malicious human attacker' which should be 'a malicious human attacker.'
- [§4.1.3] The paper states that varying rotation angles will 'uncover whether specific orientations contribute more effectively to misclassification,' but no analysis of rotation effects is presented.
- [Table 3] The center-of-gravity distance change values are reported without units or explanation of the spatial scale, making the averages hard to interpret.
Circularity Check
No significant circularity: grid-search optimization is the attack construction, not a fitted prediction; the reported success rate is an existence count, and the 97% figure is a reporting flaw rather than a circular reduction.
full rationale
The paper's derivation chain is self-contained and empirical rather than circular. Leaf images are converted to binary masks, overlaid on LISA traffic-sign images at grid positions with varying patch ratios and rotations, and the resulting images are classified by the pretrained LISA-CNN. Section 4.1.1 explicitly searches for the placement that gives the highest-confidence misclassification, but that search is the attack construction itself: the empirical claim is that for 10 of 15 sign-leaf pairs there exists such a misclassifying placement, not that randomly fallen leaves achieve that success rate. The latter would be an external-validity or statistical concern, not a circular derivation, and the paper does not define its success metric in terms of a natural-leaf distribution. The abstract's statement that misclassified images have confidence values of 97% is an overgeneralization, since Table 1 shows successful attacks with confidences as low as 37.62% and only the Merge rows exceed 96%, but this is an accuracy/selection-in-reporting issue rather than a circular step. The edge-detection analysis in Section 6.1 is post hoc and based on only 15 images, with success labels and edge metrics measured independently; no equation defines one quantity in terms of the other, so any overfitting concern is a generalizability issue, not circularity. The paper contains no load-bearing self-citations, no imported uniqueness theorems, and no ansatz smuggled in via citation. Therefore no significant circularity is established.
Assumptions & free parameters
free parameters (4)
- leaf position (grid search) =
best over search grid
- patch ratio =
0.1 to 0.5
- rotation angle =
0, 45, 90, 135, 180, 225, 270, 315 degrees
- leaf type =
Maple, Oak, Poplar
assumptions (4)
- domain assumption LISA-CNN model trained on the LISA dataset is representative of traffic sign classifiers in autonomous driving.
- domain assumption The leaf mask images and their digital overlays accurately represent natural leaf occlusions.
- domain assumption A query-based grid search constitutes a valid black-box attack.
- domain assumption Edge detection metrics are meaningfully related to classification decisions.
Cite this review
Pith. "Pith review of Fall Leaf Adversarial Attack on Traffic Sign Classification." pith.science (2026). https://pith.science/paper/SEKKQBUR
@misc{pith2026241118776,
author = {Pith},
title = {Pith review of: Fall Leaf Adversarial Attack on Traffic Sign Classification},
year = {2026},
howpublished = {\url{https://pith.science/paper/SEKKQBUR}},
note = {Machine review of arXiv:2411.18776}
}
read the original abstract
Adversarial input image perturbation attacks have emerged as a significant threat to machine learning algorithms, particularly in image classification setting. These attacks involve subtle perturbations to input images that cause neural networks to misclassify the input images, even though the images remain easily recognizable to humans. One critical area where adversarial attacks have been demonstrated is in automotive systems where traffic sign classification and recognition is critical, and where misclassified images can cause autonomous systems to take wrong actions. This work presents a new class of adversarial attacks. Unlike existing work that has focused on adversarial perturbations that leverage human-made artifacts to cause the perturbations, such as adding stickers, paint, or shining flashlights at traffic signs, this work leverages nature-made artifacts: tree leaves. By leveraging nature-made artifacts, the new class of attacks has plausible deniability: a fall leaf stuck to a street sign could come from a near-by tree, rather than be placed there by an malicious human attacker. To evaluate the new class of the adversarial input image perturbation attacks, this work analyses how fall leaves can cause misclassification in street signs. The work evaluates various leaves from different species of trees, and considers various parameters such as size, color due to tree leaf type, and rotation. The work demonstrates high success rate for misclassification. The work also explores the correlation between successful attacks and how they affect the edge detection, which is critical in many image classification algorithms.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[1]
Anish Athalye, Logan Engstrom, Andrew Ilyas, and Kevin Kwok. 2018. Synthesiz- ing robust adversarial examples. In International conference on machine learning . PMLR, 284–293
2018
-
[2]
Tom B Brown, Dandelion Mané, Aurko Roy, Martín Abadi, and Justin Gilmer
-
[3]
Nicholas Carlini and David Wagner. 2017. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp) . Ieee, 39–57
2017
-
[4]
Anirban Chakraborty, Manaar Alam, Vishal Dey, Anupam Chattopadhyay, and Debdeep Mukhopadhyay. 2018. Adversarial attacks and defences: A survey.arXiv preprint arXiv:1810.00069 (2018)
arXiv 2018
-
[5]
Shang-Tse Chen, Cory Cornelius, Jason Martin, and Duen Horng Chau. 2019. Shapeshifter: Robust physical adversarial attack on faster r-cnn object detector. In Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2018, Dublin, Ireland, September 10–14, 2018, Proceedings, Part I 18 . Springer, 52–68
work page 2019
-
[6]
Gilad Cohen, Guillermo Sapiro, and Raja Giryes. 2020. Detecting adversarial samples using influence functions and nearest neighbors. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 14453–14462
work page 2020
-
[7]
Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Chaowei Xiao, Atul Prakash, Tadayoshi Kohno, and Dawn Song. 2018. Robust physical- world attacks on deep learning visual classification. In Proceedings of the IEEE conference on computer vision and pattern recognition . 1625–1634
2018
-
[8]
Joachim Folz, Sebastian Palacio, Joern Hees, and Andreas Dengel. 2020. Adver- sarial defense based on structure-to-signal autoencoders. In 2020 IEEE Winter Conference on Applications of Computer Vision (W ACV). IEEE, 3568–3577
work page 2020
Show all 42 references
-
[9]
Joseph Yossi Gil and Ron Kimmel. 2002. Efficient dilation, erosion, opening, and closing algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence 24, 12 (2002), 1606–1617
2002
-
[10]
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. 2014. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 (2014)
2014 arXiv
-
[11]
Geoffrey Hinton. 2015. Distilling the Knowledge in a Neural Network. arXiv preprint arXiv:1503.02531 (2015)
2015 arXiv
-
[12]
Teng-Fang Hsiao, Bo-Lun Huang, Zi-Xiang Ni, Yan-Ting Lin, Hong-Han Shuai, Yung-Hui Li, and Wen-Huang Cheng. 2024. Natural Light Can Also be Danger- ous: Traffic Sign Misinterpretation Under Adversarial Natural Light Attacks. In Proceedings of the IEEE/CVF Winter Conference on ...
2024
-
[13]
Robert A Hummel, B Kimia, and Steven W Zucker. 1987. Deblurring gaussian blur. Computer Vision, Graphics, and Image Processing 38, 1 (1987), 66–80
1987
-
[14]
Nick Kanopoulos, Nagesh Vasanthavada, and Robert L Baker. 1988. Design of an image edge detection filter using the Sobel operator. IEEE Journal of solid-state circuits 23, 2 (1988), 358–367
1988
-
[15]
Yujie Li, Xing Xu, Jinhui Xiao, Siyuan Li, and Heng Tao Shen. 2020. Adaptive square attack: Fooling autonomous cars with adversarial traffic signs. IEEE Internet of Things Journal 8, 8 (2020), 6337–6347
2020
-
[16]
Hongshuo Liang, Erlu He, Yangyang Zhao, Zhe Jia, and Hao Li. 2022. Adversarial attack and defense: A survey. Electronics 11, 8 (2022), 1283
2022
-
[17]
Fangzhou Liao, Ming Liang, Yinpeng Dong, Tianyu Pang, Xiaolin Hu, and Jun Zhu. 2018. Defense against adversarial attacks using high-level representation guided denoiser. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1778–1787
2018
-
[18]
Petro Liashchynskyi and Pavlo Liashchynskyi. 2019. Grid search, random search, genetic algorithm: a big comparison for NAS. arXiv preprint arXiv:1912.06059 (2019)
2019 arXiv
-
[19]
Aishan Liu, Xianglong Liu, Jiaxin Fan, Yuqing Ma, Anlan Zhang, Huiyuan Xie, and Dacheng Tao. 2019. Perceptual-sensitive gan for generating adversarial patches. In Proceedings of the AAAI conference on artificial intelligence , Vol. 33. 1028–1035
2019
-
[20]
Jiyuan Liu, Bingyi Lu, Mingkang Xiong, Tao Zhang, and Huilin Xiong. 2023. Adversarial attack with raindrops. arXiv preprint arXiv:2302.14267 (2023)
2023 arXiv
-
[21]
Jiajun Lu, Hussein Sibai, and Evan Fabry. 2017. Adversarial examples that fool detectors. arXiv preprint arXiv:1712.02494 (2017)
2017 arXiv
-
[22]
Jiajun Lu, Hussein Sibai, Evan Fabry, and David Forsyth. 2017. Standard detec- tors aren’t (currently) fooled by physical adversarial stop signs. arXiv preprint arXiv:1710.03337 (2017)
2017 arXiv
-
[23]
Aleksander Madry. 2017. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083 (2017)
2017 arXiv
-
[24]
Aleksander Mądry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2017. Towards deep learning models resistant to adversarial attacks. stat 1050, 9 (2017)
2017
-
[25]
Andreas Mogelmose, Mohan Manubhai Trivedi, and Thomas B Moeslund. 2012. Vision-based traffic sign detection and analysis for intelligent driver assistance systems: Perspectives and survey. IEEE transactions on intelligent transportation systems 13, 4 (2012), 1484–1497
2012
-
[26]
Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Omar Fawzi, and Pascal Frossard. 2017. Universal adversarial perturbations. In Proceedings of the IEEE conference on computer vision and pattern recognition . 1765–1773
2017
-
[27]
Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z Berkay Celik, and Ananthram Swami. 2017. Practical black-box attacks against machine learning. In Proceedings of the 2017 ACM on Asia conference on computer and communications security. 506–519
2017
-
[28]
Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami
-
[29]
Svetlana Pavlitska, Nico Lambing, and J Marius Zöllner. 2023. Adversarial attacks on traffic sign recognition: A survey. In International Conference on Electrical, Computer, Communications and Mechatronics Engineering . IEEE, 1–6
2023
-
[30]
Joseph Redmon and Ali Farhadi. 2017. YOLO9000: better, faster, stronger. In Proceedings of the IEEE conference on computer vision and pattern recognition . 7263–7271
2017
-
[31]
Shaoqing Ren. 2015. Faster r-cnn: Towards real-time object detection with region proposal networks. arXiv preprint arXiv:1506.01497 (2015)
2015 arXiv
-
[32]
Weibin Rong, Zhanjing Li, Wei Zhang, and Lining Sun. 2014. An improved CANNY edge detection algorithm. In2014 IEEE international conference on mecha- tronics and automation. IEEE, 577–582
2014
-
[33]
Chawin Sitawarin, Arjun Nitin Bhagoji, Arsalan Mosenia, Mung Chiang, and Prateek Mittal. 2018. Darts: Deceiving autonomous cars with toxic signs. arXiv preprint arXiv:1802.06430 (2018)
2018 arXiv
-
[34]
Chawin Sitawarin, Arjun Nitin Bhagoji, Arsalan Mosenia, Prateek Mittal, and Mung Chiang. 2018. Rogue signs: Deceiving traffic sign recognition with mali- cious ads and logos. arXiv preprint arXiv:1801.02780 (2018)
2018 arXiv
-
[35]
Dawn Song, Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Florian Tramer, Atul Prakash, and Tadayoshi Kohno. 2018. Physical adversarial examples for object detectors. In 12th USENIX workshop on offensive technologies (WOOT 18)
2018
-
[36]
C Szegedy. 2013. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199 (2013)
2013 arXiv
-
[37]
Xingxing Wei, Ying Guo, and Jie Yu. 2022. Adversarial sticker: A stealthy attack method in the physical world. IEEE Transactions on Pattern Analysis and Machine Intelligence 45, 3 (2022), 2711–2725
2022
-
[38]
Fabian Woitschek and Georg Schneider. 2021. Physical adversarial attacks on deep neural networks for traffic sign recognition: A feasibility study. In2021 IEEE Intelligent vehicles symposium (IV) . IEEE, 481–487
2021
-
[39]
Tong Wu, Liang Tong, and Yevgeniy Vorobeychik. 2019. Defending against phys- ically realizable attacks on image classification. arXiv preprint arXiv:1909.09552 (2019)
2019 arXiv
-
[40]
Yiqi Zhong, Xianming Liu, Deming Zhai, Junjun Jiang, and Xiangyang Ji. 2022. Shadows can be dangerous: Stealthy and effective physical-world adversarial attack by natural phenomenon. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 15345–15354. 8
2022
-
[2016]
In 2016 IEEE symposium on security and privacy (SP)
Distillation as a defense to adversarial perturbations against deep neural networks. In 2016 IEEE symposium on security and privacy (SP) . IEEE, 582–597
2016
-
[2017]
arXiv preprint arXiv:1712.09665 (2017)
Adversarial patch. arXiv preprint arXiv:1712.09665 (2017)
2017 arXiv
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.