Pith. sign in

REVIEW 4 major objections 5 minor 42 references

Fall Leaf Adversarial Attack on Traffic Sign Classification

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Pith's one-line read Fallen leaves fool traffic-sign AI in 10 of 15 tests.

desk verdict Leaves as a physically plausible occlusion attack is a small new idea, but the reported success rate is the best case of a grid search and overstates what the evidence supports. read the letter →

arxiv 2411.18776 v2 pith:SEKKQBUR submitted 2024-11-27 cs.CV cs.CR

classification cs.CVcs.CR
keywords adversarialattacktrafficsignclassificationleafocclusionphysical-worldplausibledeniabilityLISAdatasetLISA-CNNedgedetection
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper seeks to establish that nature-made objects, specifically single tree leaves placed on traffic signs, form a new class of physical adversarial perturbations. In tests on five real signs from the LISA dataset, the LISA-CNN classifier misclassified 10 of 15 leaf-sign combinations after a grid search picked the leaf's position, size, and rotation, with the strongest misread reaching about 97 percent confidence. Because a leaf on a sign could plausibly have fallen from a nearby tree, the attack carries deniability that stickers, paint, or flashlights lack. The paper also argues that successful attacks shift edge-detection statistics more than unsuccessful ones, which could help future defenses. If correct, the work broadens the physical attack surface for autonomous-vehicle perception from human-made artifacts to ordinary seasonal litter.

What carries the argument

The load-bearing mechanism is an optimized leaf-mask overlay. Each leaf image is converted to a binary mask by grayscale conversion, Gaussian blur, Canny edge detection, dilation, morphological closing, and contour selection; the mask is then placed on a traffic sign within the sign's binary mask, and a grid search over candidate positions, patch-area ratios from 0.1 to 0.5, and eight rotation angles selects the placement with the highest confidence in the wrong label. The edge-analysis part computes Canny edge length, orientation from Sobel gradients, intensity, and center of gravity for the attacked images. The grid search is what turns a random-looking occlusion into an effective perturbation, and the edge metrics provide the paper's proposed signal for distinguishing successful from unsuccessful attacks.

What would settle it

Take a set of traffic-sign images, drop leaves on them using a physical or simulated model of natural leaf fall (random position, angle, and size from realistic distributions), and measure LISA-CNN's misclassification rate; if that rate is close to zero while the grid-search-optimized rate remains 10 of 15, the central claim is not supported for natural conditions.

Watch

Extended reading notes

Core claim

The central claim is that ordinary autumn leaves can serve as adversarial perturbations against a traffic-sign classifier. The paper overlays binary masks of Maple, Oak, and Poplar leaves onto five LISA sign images (Yield, Stop, Turn Right, Pedestrian Crossing, Merge), each at a position, scale, and rotation chosen by grid search to maximize misclassification confidence. The result is that 10 of the 15 sign-leaf combinations are misclassified, and the Merge sign is read as Pedestrian Crossing with 96-97 percent confidence for all three leaf types, while the Yield sign resists all three. Looking at Canny edge-detection metrics, the paper reports that successful attacks show larger relative changes in edge orientation and intensity than unsuccessful attacks. The contribution is the new attack class and the plausible deniability nature-made occlusions provide.

Load-bearing premise

The attack's reported success rate depends on the grid search being allowed to choose the best leaf position, size, and rotation for each image; if naturally fallen leaves rarely land in those optimized placements, the real-world success rate could be much lower.

Editorial extensions

If this is right

  • If the central claim is right, physical adversarial attacks can be made from unmodified, naturally occurring objects instead of manufactured patches or stickers.
  • A vehicle's traffic-sign classifier can be seriously wrong about signs such as Merge, Pedestrian Crossing, and Turn Right when a single leaf occludes part of the sign.
  • Nature-based occlusions weaken the attribution of a misclassification to an attacker, since a leaf on a sign is an everyday occurrence.
  • The reported edge-statistics differences suggest that edge-based features, especially intensity and orientation changes, could be used to detect this kind of occlusion attack.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The grid-search optimum is an upper bound on attack success; a field test with leaf positions sampled from natural wind and gravity distributions would show how often real leaves land close enough to the optimum to fool the classifier.
  • The five test images are a small sample, so the per-sign pattern (Merge always fooled, Yield never) is not evidence about the population of U.S. traffic signs; broader sampling is needed before "high success rate" is taken as a general property.
  • The edge-statistics correlation is computed on the same 15 attacked images; turning it into a reliable detector would require separate data with natural leaf occlusions and unrelated obstructions.
  • An attacker in the physical world would have to approximate the searched position, scale, and rotation by hand or by waiting for favorable leaf falls; the paper does not demonstrate that such placements are physically achievable, so a manual or robotic placement study is the obvious next experiment.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a new class of adversarial attacks on traffic sign classifiers in which fallen tree leaves are placed on signs to induce misclassification. The attack is evaluated on five LISA test images (Yield, Stop, Turn Right, Pedestrian Crossing, Merge) using three leaf types (Maple, Oak, Poplar). For each image–leaf combination, a grid search over leaf position, patch ratio, and rotation selects the configuration with the highest misclassification confidence. Table 1 reports 10 successful misclassifications out of 15 combinations. The paper also measures Canny edge-detection metrics on the original and adversarial images and compares averages for successful versus unsuccessful attacks. The central claim is a high attack success rate with misclassified-image confidences of 97%, and that edge-detection metrics correlate with attack success.

Significance. The idea of using naturally plausible, deniable artifacts such as leaves as physical adversarial perturbations is interesting and thematically extends prior work on shadows and natural light. A rigorous study of this phenomenon could be valuable for the trustworthy-autonomy community. However, the evidence presented here is far too limited and methodologically skewed to support the claimed high success rate or the edge-detection correlation. The paper's contribution is currently at the level of a preliminary observation rather than a demonstrated attack class.

major comments (4)
  1. [§4.1.1 and Table 1] The attack success rate is defined as the best result of a grid search over leaf position, patch ratio, and rotation for each image–leaf pair. The paper reports 10/15 'successful' combinations, but this is an upper envelope over the search space, not the success probability of a naturally placed leaf. Because the threat model in §3 is explicitly a real-world leaf falling and sticking to a sign, the relevant quantity is the success rate over the distribution of naturally occurring leaf placements. Reporting the maximum over searched configurations as the attack success rate introduces selection bias into the headline number and overstates practical effectiveness.
  2. [§5 and Table 1] The experimental basis is only five test images (one per sign type) and three leaf images. Table 1 contains 15 data points, and the 10/15 count has a wide confidence interval; it cannot support a general claim of 'high success rate' for traffic sign classification. Moreover, the five signs are not a representative sample of the 16-class LISA-CNN task, and no statistics over multiple images per sign type are provided.
  3. [Abstract and §5] The abstract and Section 1 state that 'misclassified images having confidence values of 97%,' but this is true only for the three Merge rows in Table 1 (96.67–97.21%). Other successful attacks have substantially lower confidence, e.g., Stop Maple at 59.23% and Turn Right Poplar at 37.62%. The 97% figure is the best case among already-selected adversarial images, not a representative or aggregate confidence.
  4. [§6 and Table 3] The edge-detection correlation analysis is post hoc and uses the same grid-search-selected adversarial images that define success. The comparison of averages between successful and unsuccessful attacks in Table 3 is not accompanied by any statistical test, confidence intervals, or control for confounds such as sign type and leaf type. Therefore the conclusion that 'analyzing edge detection metrics could have benefit in detecting adversarial attacks' is unsupported by the data.
minor comments (5)
  1. [§4.2] The text refers to 'Polar' leaves, but the intended species name is 'Poplar' as used in Table 1 and Figure 3; this inconsistency appears several times and should be corrected.
  2. [Figure 5 caption] The caption lists '(s) Turn Right Poplar' twice and '(t) Yield Poplar' once, but the figure appears to show the same image twice; the labeling should be fixed to match the displayed images.
  3. [Abstract] There is a typo in 'an malicious human attacker' which should be 'a malicious human attacker.'
  4. [§4.1.3] The paper states that varying rotation angles will 'uncover whether specific orientations contribute more effectively to misclassification,' but no analysis of rotation effects is presented.
  5. [Table 3] The center-of-gravity distance change values are reported without units or explanation of the spatial scale, making the averages hard to interpret.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: grid-search optimization is the attack construction, not a fitted prediction; the reported success rate is an existence count, and the 97% figure is a reporting flaw rather than a circular reduction.

full rationale

The paper's derivation chain is self-contained and empirical rather than circular. Leaf images are converted to binary masks, overlaid on LISA traffic-sign images at grid positions with varying patch ratios and rotations, and the resulting images are classified by the pretrained LISA-CNN. Section 4.1.1 explicitly searches for the placement that gives the highest-confidence misclassification, but that search is the attack construction itself: the empirical claim is that for 10 of 15 sign-leaf pairs there exists such a misclassifying placement, not that randomly fallen leaves achieve that success rate. The latter would be an external-validity or statistical concern, not a circular derivation, and the paper does not define its success metric in terms of a natural-leaf distribution. The abstract's statement that misclassified images have confidence values of 97% is an overgeneralization, since Table 1 shows successful attacks with confidences as low as 37.62% and only the Merge rows exceed 96%, but this is an accuracy/selection-in-reporting issue rather than a circular step. The edge-detection analysis in Section 6.1 is post hoc and based on only 15 images, with success labels and edge metrics measured independently; no equation defines one quantity in terms of the other, so any overfitting concern is a generalizability issue, not circularity. The paper contains no load-bearing self-citations, no imported uniqueness theorems, and no ansatz smuggled in via citation. Therefore no significant circularity is established.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The attack's success rate is contingent on search parameters (position, ratio, angle) and on assumptions about the representativeness of LISA-CNN and the realism of digital leaf overlays. No new physical entities are introduced.

free parameters (4)
  • leaf position (grid search) = best over search grid
    The optimal placement is found by exhaustive grid search for each image, so the attack success depends on this tuned position.
  • patch ratio = 0.1 to 0.5
    The leaf size relative to sign area is a search variable; the best ratio is selected per attack.
  • rotation angle = 0, 45, 90, 135, 180, 225, 270, 315 degrees
    Leaf orientation is varied in 45-degree steps; the best angle is chosen for each attack.
  • leaf type = Maple, Oak, Poplar
    Only three leaf species are chosen by hand, limiting the generality of the results.
assumptions (4)
  • domain assumption LISA-CNN model trained on the LISA dataset is representative of traffic sign classifiers in autonomous driving.
    The paper evaluates only this one model and assumes results transfer to other classifiers.
  • domain assumption The leaf mask images and their digital overlays accurately represent natural leaf occlusions.
    The masks are generated from three leaf photos and composited digitally; no physical-world validation is provided.
  • domain assumption A query-based grid search constitutes a valid black-box attack.
    The threat model claims black-box, but the grid search queries the model many times to optimize placement, which may not be realistic in a deployed setting.
  • domain assumption Edge detection metrics are meaningfully related to classification decisions.
    The paper treats edge statistics as potential detection signals without a formal model linking them to misclassification.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Fall Leaf Adversarial Attack on Traffic Sign Classification." pith.science (2026). https://pith.science/paper/SEKKQBUR

@misc{pith2026241118776,
  author       = {Pith},
  title        = {Pith review of: Fall Leaf Adversarial Attack on Traffic Sign Classification},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SEKKQBUR}},
  note         = {Machine review of arXiv:2411.18776}
}
read the original abstract

Adversarial input image perturbation attacks have emerged as a significant threat to machine learning algorithms, particularly in image classification setting. These attacks involve subtle perturbations to input images that cause neural networks to misclassify the input images, even though the images remain easily recognizable to humans. One critical area where adversarial attacks have been demonstrated is in automotive systems where traffic sign classification and recognition is critical, and where misclassified images can cause autonomous systems to take wrong actions. This work presents a new class of adversarial attacks. Unlike existing work that has focused on adversarial perturbations that leverage human-made artifacts to cause the perturbations, such as adding stickers, paint, or shining flashlights at traffic signs, this work leverages nature-made artifacts: tree leaves. By leveraging nature-made artifacts, the new class of attacks has plausible deniability: a fall leaf stuck to a street sign could come from a near-by tree, rather than be placed there by an malicious human attacker. To evaluate the new class of the adversarial input image perturbation attacks, this work analyses how fall leaves can cause misclassification in street signs. The work evaluates various leaves from different species of trees, and considers various parameters such as size, color due to tree leaf type, and rotation. The work demonstrates high success rate for misclassification. The work also explores the correlation between successful attacks and how they affect the edge detection, which is critical in many image classification algorithms.

Figures

Figures reproduced from arXiv: 2411.18776 by the authors.

Figure 1
Figure 1. Test images used in evaluation of the attack. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Leaf masks generated by our attack method, one for each test image. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Leaf images used in evaluation of the attacks. [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Adversarial images used in evaluation of the attacks. For each street sign and leaf type, only the adversarial image [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Edge detection of attack images. This approach applies realistic perturbations resembling graffiti on traffic signs, using black-and-white stickers to mislead models by maximizing classification errors. They introduced a two-stage evaluation involving lab tests and rea…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 22 canonical work pages

  1. [1]

    Anish Athalye, Logan Engstrom, Andrew Ilyas, and Kevin Kwok. 2018. Synthesiz- ing robust adversarial examples. In International conference on machine learning . PMLR, 284–293

  2. [2]

    Tom B Brown, Dandelion Mané, Aurko Roy, Martín Abadi, and Justin Gilmer

  3. [3]

    Nicholas Carlini and David Wagner. 2017. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp) . Ieee, 39–57

  4. [4]

    Anirban Chakraborty, Manaar Alam, Vishal Dey, Anupam Chattopadhyay, and Debdeep Mukhopadhyay. 2018. Adversarial attacks and defences: A survey.arXiv preprint arXiv:1810.00069 (2018)

  5. [5]

    Shang-Tse Chen, Cory Cornelius, Jason Martin, and Duen Horng Chau. 2019. Shapeshifter: Robust physical adversarial attack on faster r-cnn object detector. In Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2018, Dublin, Ireland, September 10–14, 2018, Proceedings, Part I 18 . Springer, 52–68

  6. [6]

    Gilad Cohen, Guillermo Sapiro, and Raja Giryes. 2020. Detecting adversarial samples using influence functions and nearest neighbors. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 14453–14462

  7. [7]

    Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Chaowei Xiao, Atul Prakash, Tadayoshi Kohno, and Dawn Song. 2018. Robust physical- world attacks on deep learning visual classification. In Proceedings of the IEEE conference on computer vision and pattern recognition . 1625–1634

  8. [8]

    Joachim Folz, Sebastian Palacio, Joern Hees, and Andreas Dengel. 2020. Adver- sarial defense based on structure-to-signal autoencoders. In 2020 IEEE Winter Conference on Applications of Computer Vision (W ACV). IEEE, 3568–3577

Show all 42 references
  1. [9]

    Joseph Yossi Gil and Ron Kimmel. 2002. Efficient dilation, erosion, opening, and closing algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence 24, 12 (2002), 1606–1617

  2. [10]

    Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. 2014. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 (2014)

  3. [11]

    Geoffrey Hinton. 2015. Distilling the Knowledge in a Neural Network. arXiv preprint arXiv:1503.02531 (2015)

  4. [12]

    Teng-Fang Hsiao, Bo-Lun Huang, Zi-Xiang Ni, Yan-Ting Lin, Hong-Han Shuai, Yung-Hui Li, and Wen-Huang Cheng. 2024. Natural Light Can Also be Danger- ous: Traffic Sign Misinterpretation Under Adversarial Natural Light Attacks. In Proceedings of the IEEE/CVF Winter Conference on ...

  5. [13]

    Robert A Hummel, B Kimia, and Steven W Zucker. 1987. Deblurring gaussian blur. Computer Vision, Graphics, and Image Processing 38, 1 (1987), 66–80

  6. [14]

    Nick Kanopoulos, Nagesh Vasanthavada, and Robert L Baker. 1988. Design of an image edge detection filter using the Sobel operator. IEEE Journal of solid-state circuits 23, 2 (1988), 358–367

  7. [15]

    Yujie Li, Xing Xu, Jinhui Xiao, Siyuan Li, and Heng Tao Shen. 2020. Adaptive square attack: Fooling autonomous cars with adversarial traffic signs. IEEE Internet of Things Journal 8, 8 (2020), 6337–6347

  8. [16]

    Hongshuo Liang, Erlu He, Yangyang Zhao, Zhe Jia, and Hao Li. 2022. Adversarial attack and defense: A survey. Electronics 11, 8 (2022), 1283

  9. [17]

    Fangzhou Liao, Ming Liang, Yinpeng Dong, Tianyu Pang, Xiaolin Hu, and Jun Zhu. 2018. Defense against adversarial attacks using high-level representation guided denoiser. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1778–1787

  10. [18]

    Petro Liashchynskyi and Pavlo Liashchynskyi. 2019. Grid search, random search, genetic algorithm: a big comparison for NAS. arXiv preprint arXiv:1912.06059 (2019)

  11. [19]

    Aishan Liu, Xianglong Liu, Jiaxin Fan, Yuqing Ma, Anlan Zhang, Huiyuan Xie, and Dacheng Tao. 2019. Perceptual-sensitive gan for generating adversarial patches. In Proceedings of the AAAI conference on artificial intelligence , Vol. 33. 1028–1035

  12. [20]

    Jiyuan Liu, Bingyi Lu, Mingkang Xiong, Tao Zhang, and Huilin Xiong. 2023. Adversarial attack with raindrops. arXiv preprint arXiv:2302.14267 (2023)

  13. [21]

    Jiajun Lu, Hussein Sibai, and Evan Fabry. 2017. Adversarial examples that fool detectors. arXiv preprint arXiv:1712.02494 (2017)

  14. [22]

    Jiajun Lu, Hussein Sibai, Evan Fabry, and David Forsyth. 2017. Standard detec- tors aren’t (currently) fooled by physical adversarial stop signs. arXiv preprint arXiv:1710.03337 (2017)

  15. [23]

    Aleksander Madry. 2017. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083 (2017)

  16. [24]

    Aleksander Mądry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2017. Towards deep learning models resistant to adversarial attacks. stat 1050, 9 (2017)

  17. [25]

    Andreas Mogelmose, Mohan Manubhai Trivedi, and Thomas B Moeslund. 2012. Vision-based traffic sign detection and analysis for intelligent driver assistance systems: Perspectives and survey. IEEE transactions on intelligent transportation systems 13, 4 (2012), 1484–1497

  18. [26]

    Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Omar Fawzi, and Pascal Frossard. 2017. Universal adversarial perturbations. In Proceedings of the IEEE conference on computer vision and pattern recognition . 1765–1773

  19. [27]

    Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z Berkay Celik, and Ananthram Swami. 2017. Practical black-box attacks against machine learning. In Proceedings of the 2017 ACM on Asia conference on computer and communications security. 506–519

  20. [28]

    Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami

  21. [29]

    Svetlana Pavlitska, Nico Lambing, and J Marius Zöllner. 2023. Adversarial attacks on traffic sign recognition: A survey. In International Conference on Electrical, Computer, Communications and Mechatronics Engineering . IEEE, 1–6

  22. [30]

    Joseph Redmon and Ali Farhadi. 2017. YOLO9000: better, faster, stronger. In Proceedings of the IEEE conference on computer vision and pattern recognition . 7263–7271

  23. [31]

    Shaoqing Ren. 2015. Faster r-cnn: Towards real-time object detection with region proposal networks. arXiv preprint arXiv:1506.01497 (2015)

  24. [32]

    Weibin Rong, Zhanjing Li, Wei Zhang, and Lining Sun. 2014. An improved CANNY edge detection algorithm. In2014 IEEE international conference on mecha- tronics and automation. IEEE, 577–582

  25. [33]

    Chawin Sitawarin, Arjun Nitin Bhagoji, Arsalan Mosenia, Mung Chiang, and Prateek Mittal. 2018. Darts: Deceiving autonomous cars with toxic signs. arXiv preprint arXiv:1802.06430 (2018)

  26. [34]

    Chawin Sitawarin, Arjun Nitin Bhagoji, Arsalan Mosenia, Prateek Mittal, and Mung Chiang. 2018. Rogue signs: Deceiving traffic sign recognition with mali- cious ads and logos. arXiv preprint arXiv:1801.02780 (2018)

  27. [35]

    Dawn Song, Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Florian Tramer, Atul Prakash, and Tadayoshi Kohno. 2018. Physical adversarial examples for object detectors. In 12th USENIX workshop on offensive technologies (WOOT 18)

  28. [36]

    C Szegedy. 2013. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199 (2013)

  29. [37]

    Xingxing Wei, Ying Guo, and Jie Yu. 2022. Adversarial sticker: A stealthy attack method in the physical world. IEEE Transactions on Pattern Analysis and Machine Intelligence 45, 3 (2022), 2711–2725

  30. [38]

    Fabian Woitschek and Georg Schneider. 2021. Physical adversarial attacks on deep neural networks for traffic sign recognition: A feasibility study. In2021 IEEE Intelligent vehicles symposium (IV) . IEEE, 481–487

  31. [39]

    Tong Wu, Liang Tong, and Yevgeniy Vorobeychik. 2019. Defending against phys- ically realizable attacks on image classification. arXiv preprint arXiv:1909.09552 (2019)

  32. [40]

    Yiqi Zhong, Xianming Liu, Deming Zhai, Junjun Jiang, and Xiangyang Ji. 2022. Shadows can be dangerous: Stealthy and effective physical-world adversarial attack by natural phenomenon. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 15345–15354. 8

  33. [2016]

    In 2016 IEEE symposium on security and privacy (SP)

    Distillation as a defense to adversarial perturbations against deep neural networks. In 2016 IEEE symposium on security and privacy (SP) . IEEE, 582–597

  34. [2017]

    arXiv preprint arXiv:1712.09665 (2017)

    Adversarial patch. arXiv preprint arXiv:1712.09665 (2017)

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.