REVIEW 4 major objections 6 minor 32 references
Distortion-Aware Adversarial Attacks on Bounding Boxes of Object Detectors
T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read This paper proposes a distortion-aware, mask-guided adversarial attack that iteratively adds gradient-ascent noise only inside predicted bounding boxes, claiming white-box success up to 100 percent and black-box success up to 98 percent…
desk verdict A plausible iterative mask-based attack on detectors, but the headline success rates are relative mAP drops, not per-image success, so the central claim is not established. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery is a box-mask-weighted gradient-ascent loop: at each iteration the detector re-predicts boxes on the current image, those boxes are summed onto an $m \times n$ mask $M$, and the update $I' \leftarrow I' + \lambda \cdot (\partial L / \partial I') \cdot M$ directs all perturbation into object regions. Distortion control is provided by the complement of normalized cross-correlation, $D(I,I') = 1 - \mathrm{NCC}(I,I')$, used as a stopping condition together with a success-rate threshold $R$. The mask focuses the attack on the pixels that determine confidence, while NCC keeps the image-change budget explicit.
What would settle it
Re-run the same attack on MS COCO and PASCAL VOC and count the fraction of images in which every ground-truth object is missed at a confidence threshold of 0.50, then compare that per-image success rate with the mAP-derived percentages; a large gap would falsify the headline success-rate claim.
Extended reading notes
Core claim
The paper's central claim is that a single iterative gradient-ascent update, restricted to the aggregated masks of predicted bounding boxes, can drive a detector's class-confidence scores below the detection threshold so that objects are missed, and that the same perturbation set transfers across detector architectures and datasets. In the paper's formulation, attack succeeds when predicted class probability $p(c_i)$ falls below threshold $T$ relative to other classes; the update uses the full detector loss $L = L_{\text{loc}} + L_{\text{obj}} + L_{\text{cls}}$ and adds perturbation only where the box mask $M[f(I')]$ is nonzero. The authors report white-box success up to 100 percent and black-box success up to 98 percent, with cross-model results showing larger YOLOv8 models producing the most transferable perturbations.
Load-bearing premise
The central claim rests on measuring success rate as the relative drop in mean average precision; if a lower mAP does not mean individual objects are missed, the 100 percent and 98 percent figures overstate the attack.
Editorial extensions
If this is right
- Detectors in safety-critical settings can be disabled by visually subtle perturbations confined to object regions, without touching background pixels.
- A black-box attacker needs only one surrogate model; the paper's adversarial images generated against a large YOLOv8 model transfer to one-stage, two-stage, and transformer detectors.
- Two-stage detectors, which resist earlier box-agnostic attacks, are still vulnerable to mask-guided iterative attacks.
- Cross-dataset transfer means attacks generated in one domain can be effective in another without retraining the attack.
- Defenses must reduce sensitivity of confidence scores inside predicted boxes, not merely add classifier-style robust training.
Reading between the lines
- Editorial extension: the claimed 100 percent and 98 percent success rates are read off relative mAP drops, not per-image object-miss counts; a per-image measure could be lower and should be reported.
- Editorial extension: because the attack only perturbs predicted boxes, detectors that output dense, small, or heavily overlapping boxes may need far more iterations, as the paper itself observes for overlapping cases; an adaptive mask could improve speed.
- Editorial extension: the transferability pattern in which larger models generate more general perturbations suggests shared low-level features are being exploited, so feature-space denoising or attention regularization is a testable defensive direction.
- Editorial extension: the NCC-based distortion metric is a global similarity score and does not guarantee local imperceptibility, so human-perception studies would be a natural extension.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a distortion-aware iterative adversarial attack on object detectors. The method perturbs pixels inside predicted object masks by gradient ascent on the detector's loss, with a Normalized Cross Correlation-based distortion term to control perceptibility, and iterates until a distortion or success-rate threshold is reached. The authors evaluate on MS COCO 2017 and PASCAL VOC 2012 against YOLOv8 variants, Faster R-CNN, RetinaNet, and Swin Transformer, and report cross-model, cross-domain, and black-box transferability results, including comparisons with DAG and UEA. The headline claims are success attack rates of up to 100% in white-box settings and up to 98% in black-box settings.
Significance. If the reported success rates were measured as per-image or per-object misdetection, the contribution would be valuable: a single iterative mask-guided perturbation that transfers across detectors, datasets, and detection paradigms, with explicit distortion control. The paper has genuine strengths: it evaluates on multiple modern architectures, includes cross-domain validation, provides qualitative attention-map analysis, and makes source code available. The main weakness is that the central quantitative claim is currently not supported by the metric actually reported; the paper equates relative mAP drop with 'success attack rate.' Because this metric issue affects the abstract, the conclusion, and the main tables, the contribution cannot be accepted in its present form, but the underlying method appears plausible and the issue is empirically fixable.
major comments (4)
- [Sec. 6.1 / Tables 2 and 3] The paper never defines 'success attack rate' as a measurable quantity, and the numbers in Tables 2 and 3 are computed as relative mAP drop. For example, Sec. 6.1 reports a 91.19% success attack rate for YOLOv8s, obtained as (33.26 - 2.93)/33.26 from Table 2. A relative mAP drop is an aggregate over images, object classes, confidence thresholds, and IoU thresholds; it does not equal the fraction of images in which all objects are missed, nor the fraction of individual objects missed. A detector can lose 91% of its mAP while still producing surviving detections on most images. Since the abstract and conclusion advertise up to 100% and 98% success attack rates, the authors should define a per-image or per-object success metric, report it directly, and either remove or clearly qualify the mAP-based claims.
- [Sec. 6.4 / Table 5 (and Sec. 6.3 / Table 4)] The comparisons against DAG and UEA are not on equal footing. In Table 5 the baseline detectors differ across methods: the one-stage Baselines are 68.00, 68.00, and 25.04, while the two-stage Baselines are 70.10, 70.10, and 27.90. The success rates 92.65%, 92.65%, 93.25% and 8.70%, 71.47%, 92.47% are therefore relative reductions with different denominators and are not directly comparable. Similarly, Table 4 compares the proposed method on Faster R-CNN, RetinaNet, and Swin-T with DAG on R-FCN-ResNet50. The authors should evaluate all methods on the same detectors, the same data split, and the same success-rate definition.
- [Sec. 4.1 / Eq. (10) / Alg. 1] The stopping condition f(I'_i) >= R in Eq. (10) and Algorithm 1 is not well defined: f is an object detector that returns boxes and class scores, not a scalar success rate, and R is described as a desired success attack rate. The relationship between R and the measured success rate is never specified, so the algorithm's claimed control over the success rate is not implemented in a verifiable way. The authors should define R as an explicit function of the detector's output (for example, number of detections above a confidence threshold) and show how it maps to the reported success metric.
- [Sec. 5.4] The only place the paper gives a concrete success criterion is Sec. 5.4, where 'all objects presented in the image are misdetected' is used for a single qualitative example. This per-image definition is never applied to the quantitative tables. Consequently, the central claim that the attack disables detectors at the stated rates is not established by the reported experiments. The authors should apply this or a similarly explicit definition to the full evaluation set and report the distribution of per-image success across the test images.
minor comments (6)
- [Abstract] The source code URL in the abstract contains a space ('attack detector'); the link should be corrected and verified.
- [Sec. 3.1] There is a typo, 'Defintion,' and the notation {c1,...,ck} != {c1',...,ck'} is stronger than the untargeted attack scenario described later, since a successful attack can leave some objects correctly detected.
- [Sec. 4.2 / Eq. (11)] The statement that NCC(I,I') lies in [0,1] is not true for arbitrary image pairs; if the implementation clamps or normalizes the score, this should be stated explicitly.
- [Sec. 5.1 / Fig. 3] The convergence analysis is based on three example images; the authors should report aggregate loss statistics over the evaluation set and specify which loss components are plotted.
- [Sec. 5.2 / Figs. 4 and 5] The figures report success-rate and distortion curves without error bars or the number of images used; adding this information would make the trade-off analysis more reproducible.
- [Fig. 9 caption] The caption says the bottom row shows the added distortion amounts while also referring to the top row; the arrangement of original images, adversarial images, and heatmaps should be clarified.
Circularity Check
No circularity found: the attack is constructed from the detector's own loss and measured on external benchmarks; self-citations are motivational only.
full rationale
The derivation chain is self-contained. Equation (9) defines the iterative adversarial update using the detector's loss gradient and predicted masks, explicitly extending Kurakin et al.'s iterative FGSM (Sec. 4.1), and Eq. (12) defines the distortion metric via normalized cross-correlation. The reported attack rates are empirical results obtained by evaluating the generated perturbations against fixed pretrained detectors on MS COCO 2017 and PASCAL VOC 2012 (Tables 2-4, Secs. 6.1-6.4); they are not fitted parameters renamed as predictions, and no equation reduces to another by construction. The only self-citations (Dang et al. 2023; Nguyen et al. 2024a,b; Dang et al. 2024) appear in the introduction as motivation for why object detectors matter and are not load-bearing for the attack's validity, transferability, or uniqueness. A measurement-consistency concern exists: Sec. 6.1 computes 'success attack rate' as relative mAP drop ('91.19% (dropping the model's mAP from 33.26 to 2.93)') while Sec. 5.4 uses a per-image misdetection notion, but this is a metric-definition issue rather than circular reasoning in the derivation chain.
Assumptions & free parameters
free parameters (3)
- Lambda_gradient_step =
not reported
- N_max_iterations =
120
- S_or_R_thresholds =
user-set (S target distortion or R desired success rate)
assumptions (4)
- domain assumption The detector's total loss L (Eq. 7) is differentiable with respect to input pixels and its gradients point in directions that reduce detection confidence when ascended.
- ad hoc to paper The iterative update in Eq. 9 produces an image satisfying the objective in Eq. 6 at small distortion.
- domain assumption Normalized Cross Correlation defines a valid distortion metric for human imperceptibility.
- ad hoc to paper Relative mAP drop equals the success attack rate.
Cite this review
Pith. "Pith review of Distortion-Aware Adversarial Attacks on Bounding Boxes of Object Detectors." pith.science (2026). https://pith.science/paper/B25XLGBV
@misc{pith2026241218815,
author = {Pith},
title = {Pith review of: Distortion-Aware Adversarial Attacks on Bounding Boxes of Object Detectors},
year = {2026},
howpublished = {\url{https://pith.science/paper/B25XLGBV}},
note = {Machine review of arXiv:2412.18815}
}
abstract
Deep learning-based object detection has become ubiquitous in the last decade due to its high accuracy in many real-world applications. With this growing trend, these models are interested in being attacked by adversaries, with most of the results being on classifiers, which do not match the context of practical object detection. In this work, we propose a novel method to fool object detectors, expose the vulnerability of state-of-the-art detectors, and promote later works to build more robust detectors to adversarial examples. Our method aims to generate adversarial images by perturbing object confidence scores during training, which is crucial in predicting confidence for each class in the testing phase. Herein, we provide a more intuitive technique to embed additive noises based on detected objects' masks and the training loss with distortion control over the original image by leveraging the gradient of iterative images. To verify the proposed method, we perform adversarial attacks against different object detectors, including the most recent state-of-the-art models like YOLOv8, Faster R-CNN, RetinaNet, and Swin Transformer. We also evaluate our technique on MS COCO 2017 and PASCAL VOC 2012 datasets and analyze the trade-off between success attack rate and image distortion. Our experiments show that the achievable success attack rate is up to $100$\% and up to $98$\% when performing white-box and black-box attacks, respectively. The source code and relevant documentation for this work are available at the following link: https://github.com/anonymous20210106/attack_detector
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
Alaifari, R., Alberti, G. S., and Gauksson, T. (2018). Adef: an iterative algorithm to construct adversarial deformations. In International Conference on Learning Representations
work page 2018
-
[2]
Bochkovskiy, A., Wang, C.-Y., and Liao, H.-Y. M. (2020). Yolov4: Optimal speed and accuracy of object detection
work page 2020
-
[3]
Carlini, N. and Wagner, D. (2017). Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp) , pages 39--57. Ieee
work page 2017
-
[4]
Chen, P.-Y., Sharma, Y., Zhang, H., Yi, J., and Hsieh, C.-J. (2018). Ead: elastic-net attacks to deep neural networks via adversarial examples. In Proceedings of the AAAI conference on artificial intelligence , volume 32
work page 2018
-
[5]
Dang, T., Nguyen, K., and Huber, M. (2023). Multiplanar self-calibration for mobile cobot 3d object manipulation using 2d detectors and depth estimation. In 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages 1782--1788. IEEE
work page 2023
-
[6]
Dang, T., Nguyen, K., and Huber, M. (2024). V3d-slam: Robust rgb-d slam in dynamic environments with 3d semantic geometry voting. arXiv preprint arXiv:2410.12068
work page Pith review arXiv 2024
-
[7]
W., Sasdelli, M., Rajasegaran, R., and Campbell, D
Du, A., Chen, B., Chin, T.-J., Law, Y. W., Sasdelli, M., Rajasegaran, R., and Campbell, D. (2022). Physical Adversarial Attacks on an Aerial Imagery Object Detector . pages 1796--1806
work page 2022
-
[8]
Everingham, M., Eslami, S. A., Van Gool, L., Williams, C. K., Winn, J., and Zisserman, A. (2015). The pascal visual object classes challenge: A retrospective. International journal of computer vision , 111:98--136
work page 2015
Show all 32 references
-
[9]
J., Shlens, J., and Szegedy, C
Goodfellow, I. J., Shlens, J., and Szegedy, C. (2015). Explaining and harnessing adversarial examples. In Bengio, Y. and LeCun, Y., editors, 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings
2015
-
[10]
and Tian, Q
Im Choi, J. and Tian, Q. (2022). Adversarial attack and defense of yolo detectors in autonomous driving scenarios. In 2022 IEEE Intelligent Vehicles Symposium (IV) , pages 1011--1017. IEEE
2022
-
[11]
Jocher, G., Chaurasia, A., and Qiu, J. (2023). YOLO by Ultralytics
2023
-
[12]
J., and Bengio, S
Kurakin, A., Goodfellow, I. J., and Bengio, S. (2018). Adversarial examples in the physical world. In Artificial intelligence safety and security , pages 99--112. Chapman and Hall/CRC
2018
-
[13]
Lin, T.-Y., Goyal, P., Girshick, R., He, K., and Doll \'a r, P. (2017). Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision , pages 2980--2988
2017
-
[14]
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Doll \'a r, P., and Zitnick, C. L. (2014). Microsoft coco: Common objects in context. In Computer Vision--ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V...
2014
-
[15]
Lindeberg, T. (2012). Scale invariant feature transform
2012
-
[16]
Liu, X., Yang, H., Liu, Z., Song, L., Chen, Y., and Li, H. (2019). DPATCH: an adversarial patch attack on object detectors. In Espinoza, H., h \' E igeartaigh, S. \' O ., Huang, X., Hern \' a ndez - Orallo, J., and Castillo - Effen, M., editors, Workshop on Artificial Intellig...
2019
-
[17]
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. (2021). Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision , pages 10012--10022
2021
-
[18]
Lu, J., Sibai, H., and Fabry, E. (2017). Adversarial Examples that Fool Detectors . arXiv:1712.02494 [cs]
2017 arXiv
-
[19]
Lu, Y. (2019). The Level Weighted Structural Similarity Loss : A Step Away from MSE . Proceedings of the AAAI Conference on Artificial Intelligence , 33(01):9989--9990. Number: 01
2019
-
[20]
Madry, A., Makelov, A., Schmidt, L., Tsipras, D., and Vladu, A. (2018). Towards deep learning models resistant to adversarial attacks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Procee...
2018
-
[21]
Moosavi-Dezfooli, S.-M., Fawzi, A., and Frossard, P. (2016). DeepFool : A Simple and Accurate Method to Fool Deep Neural Networks . pages 2574--2582
2016
-
[22]
Nguyen, K., Dang, T., and Huber, M. (2024a). Real-time 3d semantic scene perception for egocentric robots with binocular vision. arXiv preprint arXiv:2402.11872
2024 arXiv
-
[23]
Nguyen, K., Dang, T., and Huber, M. (2024b). Volumetric mapping with panoptic refinement via kernel density estimation for mobile robots. arXiv preprint arXiv:2412.11241
2024 arXiv
-
[24]
Puccetti, T., Zoppi, T., and Ceccarelli, A. (2023). On the efficacy of metrics to describe adversarial attacks. arXiv preprint arXiv:2301.13028
2023 arXiv
-
[25]
Ren, S., He, K., Girshick, R., and Sun, J. (2015). Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in neural information processing systems , 28
2015
-
[26]
Rublee, E., Rabaud, V., Konolige, K., and Bradski, G. (2011). Orb: An efficient alternative to sift or surf. In 2011 International conference on computer vision , pages 2564--2571. Ieee
2011
-
[27]
R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D. (2017). Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on computer vision , pages 618--626
2017
-
[28]
Song, D., Eykholt, K., Evtimov, I., Fernandes, E., Li, B., Rahmati, A., Tramer, F., Prakash, A., and Kohno, T. (2018). Physical adversarial examples for object detectors. In 12th USENIX workshop on offensive technologies (WOOT 18)
2018
-
[29]
C., Sheikh, H
Wang, Z., Bovik, A. C., Sheikh, H. R., and Simoncelli, E. P. (2004). Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing , 13(4):600--612
2004
-
[30]
Wei, X., Liang, S., Chen, N., and Cao, X. (2019). Transferable adversarial attacks for image and video object detection. In Kraus, S., editor, Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, Macao, China, August 10-16, 20...
2019
-
[31]
Xie, C., Wang, J., Zhang, Z., Zhou, Y., Xie, L., and Yuille, A. (2017). Adversarial examples for semantic segmentation and object detection. In Proceedings of the IEEE international conference on computer vision , pages 1369--1378
2017
-
[32]
write newline
" write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence '...
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.