{"id":"c00c28a9-e100-4db4-b7b9-02680e0d5dee","arxiv_id":"2412.18815","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"An iterative gradient-based attack, guided by predicted bounding-box masks and controlled by a normalized cross-correlation distortion threshold, causes object detectors to misdetect objects with high reported success.","lead":"Researchers present a distortion-aware attack that adds small, masked noise to images so object detectors such as YOLOv8, Faster R-CNN, RetinaNet, and Swin Transformer fail to recognize objects, reporting up to 100% white-box and 98% transfer success. The work is useful for testing how robust detection systems are, especially in surveillance and autonomous driving where invisible tampering could disable recognition.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline 'success attack rate' is computed as relative mAP drop in Tables 2/3, not per-image or per-object success; the central 100%/98% claim is therefore unsupported as stated.","rationale":"The paper's core contribution is a distortion-controlled iterative masked gradient-ascent attack, which is plausible and specified well enough to re-implement. I agree with the reader that this is not fatally flawed. The load-bearing concern is the success-rate metric. The central quantitative claims—100% white-box and 98% black-box—appear in the abstract and are reproduced in the experiments only as relative mAP drops. The paper never states the per-image or per-object definition for these numbers. This matters because the attack's stated purpose is to 'disable' detectors or make them 'misdetect' objects; an mAP drop alone cannot verify that. It also affects the comparison with DAG and UEA, since those baselines report success in different terms. I do not see an internal inconsistency in the algorithm: Eq. 9 correctly implements masked gradient ascent, Eq. 10 provides a stopping rule, and the convergence analysis in Sec. 5.1 is supportive. The missing hyperparameters (e.g., λ) and the anonymous code link weaken reproducibility but are addressable. The metric issue is the one that directly undermines the headline. The proposed test would settle it by measuring per-image and per-object success on a subset and comparing those rates to the table values. If the test shows the rates are similar, the claim stands; if not, the paper needs to define its success rate explicitly and re-report the numbers.","tokens_in":14027,"tokens_out":3990,"duration_ms":34704,"concrete_test":"Run the released code on a random subset of MS COCO 2017 val (e.g., 500 images) to generate adversarial images against YOLOv8x using the same settings as Table 2, then evaluate YOLOv8s on the perturbed images. Compute (a) per-image success: the fraction of images where no ground-truth object is detected above the 0.5 confidence threshold, and (b) per-object success: the fraction of ground-truth objects not detected. Compare these rates to the reported 91.19% relative mAP drop. If either rate is materially below 91.19%, the headline success rates are overstated and the manuscript must restate its success definition and re-report the numbers.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In Sec. 6.1, the 91.19% 'success attack rate' for YOLOv8s is derived as (33.26−2.93)/33.26, i.e., a relative mAP drop. The same construction underlies the 100% and 98% figures in the abstract and the transferability rates in Tables 3 and 4. However, relative mAP drop is an aggregate over images and classes; it does not equal the fraction of images where all objects are missed, nor the fraction of individual objects missed. A detector can lose 91% of its mAP because confidence scores for many detections fall just below threshold while every image still contains several surviving detections. The only place the paper defines success is Sec. 5.4, for a single qualitative example ('all objects presented in the image are misdetected'), and that per-image definition is never used in the quantitative tables. Consequently, the central claim that the attack 'disables' detectors at the stated rates is not established. The comparisons with DAG and UEA are also apples-to-oranges if those prior numbers use per-image or per-object success definitions.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a distortion-aware iterative adversarial attack on object detectors. The method perturbs pixels inside predicted object masks by gradient ascent on the detector's loss, with a Normalized Cross Correlation-based distortion term to control perceptibility, and iterates until a distortion or success-rate threshold is reached. The authors evaluate on MS COCO 2017 and PASCAL VOC 2012 against YOLOv8 variants, Faster R-CNN, RetinaNet, and Swin Transformer, and report cross-model, cross-domain, and black-box transferability results, including comparisons with DAG and UEA. The headline claims are success attack rates of up to 100% in white-box settings and up to 98% in black-box settings.","tokens_in":14272,"tokens_out":6226,"duration_ms":53730,"significance":"If the reported success rates were measured as per-image or per-object misdetection, the contribution would be valuable: a single iterative mask-guided perturbation that transfers across detectors, datasets, and detection paradigms, with explicit distortion control. The paper has genuine strengths: it evaluates on multiple modern architectures, includes cross-domain validation, provides qualitative attention-map analysis, and makes source code available. The main weakness is that the central quantitative claim is currently not supported by the metric actually reported; the paper equates relative mAP drop with 'success attack rate.' Because this metric issue affects the abstract, the conclusion, and the main tables, the contribution cannot be accepted in its present form, but the underlying method appears plausible and the issue is empirically fixable.","major_comments":[{"comment":"The paper never defines 'success attack rate' as a measurable quantity, and the numbers in Tables 2 and 3 are computed as relative mAP drop. For example, Sec. 6.1 reports a 91.19% success attack rate for YOLOv8s, obtained as (33.26 - 2.93)/33.26 from Table 2. A relative mAP drop is an aggregate over images, object classes, confidence thresholds, and IoU thresholds; it does not equal the fraction of images in which all objects are missed, nor the fraction of individual objects missed. A detector can lose 91% of its mAP while still producing surviving detections on most images. Since the abstract and conclusion advertise up to 100% and 98% success attack rates, the authors should define a per-image or per-object success metric, report it directly, and either remove or clearly qualify the mAP-based claims.","section":"Sec. 6.1 / Tables 2 and 3"},{"comment":"The comparisons against DAG and UEA are not on equal footing. In Table 5 the baseline detectors differ across methods: the one-stage Baselines are 68.00, 68.00, and 25.04, while the two-stage Baselines are 70.10, 70.10, and 27.90. The success rates 92.65%, 92.65%, 93.25% and 8.70%, 71.47%, 92.47% are therefore relative reductions with different denominators and are not directly comparable. Similarly, Table 4 compares the proposed method on Faster R-CNN, RetinaNet, and Swin-T with DAG on R-FCN-ResNet50. The authors should evaluate all methods on the same detectors, the same data split, and the same success-rate definition.","section":"Sec. 6.4 / Table 5 (and Sec. 6.3 / Table 4)"},{"comment":"The stopping condition f(I'_i) >= R in Eq. (10) and Algorithm 1 is not well defined: f is an object detector that returns boxes and class scores, not a scalar success rate, and R is described as a desired success attack rate. The relationship between R and the measured success rate is never specified, so the algorithm's claimed control over the success rate is not implemented in a verifiable way. The authors should define R as an explicit function of the detector's output (for example, number of detections above a confidence threshold) and show how it maps to the reported success metric.","section":"Sec. 4.1 / Eq. (10) / Alg. 1"},{"comment":"The only place the paper gives a concrete success criterion is Sec. 5.4, where 'all objects presented in the image are misdetected' is used for a single qualitative example. This per-image definition is never applied to the quantitative tables. Consequently, the central claim that the attack disables detectors at the stated rates is not established by the reported experiments. The authors should apply this or a similarly explicit definition to the full evaluation set and report the distribution of per-image success across the test images.","section":"Sec. 5.4"}],"minor_comments":[{"comment":"The source code URL in the abstract contains a space ('attack detector'); the link should be corrected and verified.","section":"Abstract"},{"comment":"There is a typo, 'Defintion,' and the notation {c1,...,ck} != {c1',...,ck'} is stronger than the untargeted attack scenario described later, since a successful attack can leave some objects correctly detected.","section":"Sec. 3.1"},{"comment":"The statement that NCC(I,I') lies in [0,1] is not true for arbitrary image pairs; if the implementation clamps or normalizes the score, this should be stated explicitly.","section":"Sec. 4.2 / Eq. (11)"},{"comment":"The convergence analysis is based on three example images; the authors should report aggregate loss statistics over the evaluation set and specify which loss components are plotted.","section":"Sec. 5.1 / Fig. 3"},{"comment":"The figures report success-rate and distortion curves without error bars or the number of images used; adding this information would make the trade-off analysis more reproducible.","section":"Sec. 5.2 / Figs. 4 and 5"},{"comment":"The caption says the bottom row shows the added distortion amounts while also referring to the top row; the arrangement of original images, adversarial images, and heatmaps should be clarified.","section":"Fig. 9 caption"}],"recommendation":"major_revision","confidential_remarks":"The central issue is the mismatch between the claimed success attack rate and the reported mAP-based numbers; this is a load-bearing problem for the abstract and conclusion, but it is empirically fixable by re-measuring with a defined per-image or per-object metric and by re-running the DAG/UEA comparisons on matched detectors. I see no reason to doubt the basic iterative mask-gradient mechanism, so a major revision rather than rejection is appropriate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know this paper before citing it. The method itself is a reasonable extension of DAG and iterative FGSM: use detector-predicted masks, accumulate gradients over bounding boxes, and control distortion with an NCC-based stopping rule. That combination is new enough and the experiments are broad (YOLOv8 variants, Faster R-CNN, RetinaNet, Swin-T, two datasets). The cross-model transferability numbers are interesting, and the authors are honest about their lineage to Kurakin and Xie.\n\nThe problem is the metric. The 'success attack rate' in Tables 2 and 3 is computed as relative mAP drop: for YOLOv8s, (33.26−2.93)/33.26 = 91.19%. That is an aggregate over images and classes, not the fraction of images where all objects are missed or even the fraction of objects missed. The abstract's 100% and 98% claims inherit this definition. The only per-image definition appears in Sec. 5.4 for a single qualitative example. So the central claim that the attack 'disables' detectors at the stated rates is unsupported as written. The stress-test note holds up.\n\nThere are also smaller but real issues. The stopping condition in Alg. 1 is confusing: 'f(I') ≥ R' doesn't make sense as a success rate, and the 'break' inside the for loop exits the loop over boxes but not the while loop. The comparisons with DAG and UEA (Tables 4 and 5) are not apples-to-apples: different baseline models, different mAP starting points, and success rates computed the same questionable way. Hyperparameters like λ and the thresholds S and R are not fully specified, and the code is an anonymous link without a commit hash.\n\nNone of this means the method is worthless. The mAP drops are large and consistent, and the idea is worth pursuing. But the paper overclaims in its current form. A serious revision should define success per image or per object, report those numbers, fix the algorithm description, and redo the prior comparisons fairly.\n\nFor a reading group, I'd say yes: it's a good case study in how to (and how not to) evaluate attack success. I wouldn't cite it for the quantitative claims until they're fixed, but the algorithm itself is citable. A serious referee should engage with it — conditional acceptance, with major revision required.","headline":"A plausible iterative mask-based attack on detectors, but the headline success rates are relative mAP drops, not per-image success, so the central claim is not established.","tokens_in":14790,"tokens_out":3030,"would_cite":false,"duration_ms":26786,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper proposes a distortion-aware, mask-guided adversarial attack that iteratively adds gradient-ascent noise only inside predicted bounding boxes, claiming white-box success up to 100 percent and black-box success up to 98 percent…","keywords":["adversarial attacks","object detection","bounding boxes","distortion awareness","white-box attack","black-box attack","transferability","YOLOv8"],"falsifier":"Re-run the same attack on MS COCO and PASCAL VOC and count the fraction of images in which every ground-truth object is missed at a confidence threshold of 0.50, then compare that per-image success rate with the mAP-derived percentages; a large gap would falsify the headline success-rate claim.","tokens_in":13811,"feed_emoji":"🎯","tokens_out":5056,"duration_ms":44696,"temperature":0.7,"pith_summary":"Most adversarial-attack research targets image classifiers; this paper takes aim at object detectors, where the network must also locate objects and score their confidence. It proposes an iterative attack that adds gradient-ascent noise only inside the bounding boxes the detector itself predicts, and stops when either a distortion budget or a desired success rate is reached. The paper claims that this mask-guided, distortion-aware procedure achieves success rates up to 100 percent in white-box settings and up to 98 percent in black-box settings, with perturbations that are largely imperceptible. If those numbers hold, modern detectors used in surveillance and driving can be disabled by small, targeted image changes, and detector defenses need to account for box-aware perturbations rather than just classifier-style noise.","feed_headline":"Mask-focused pixels disable detectors up to 100% of the time","feed_subtitle":"Noise added only inside predicted boxes drops detection accuracy to near zero, even for models the attacker never saw.","key_machinery":"The machinery is a box-mask-weighted gradient-ascent loop: at each iteration the detector re-predicts boxes on the current image, those boxes are summed onto an $m \\times n$ mask $M$, and the update $I' \\leftarrow I' + \\lambda \\cdot (\\partial L / \\partial I') \\cdot M$ directs all perturbation into object regions. Distortion control is provided by the complement of normalized cross-correlation, $D(I,I') = 1 - \\mathrm{NCC}(I,I')$, used as a stopping condition together with a success-rate threshold $R$. The mask focuses the attack on the pixels that determine confidence, while NCC keeps the image-change budget explicit.","core_discovery":"The paper's central claim is that a single iterative gradient-ascent update, restricted to the aggregated masks of predicted bounding boxes, can drive a detector's class-confidence scores below the detection threshold so that objects are missed, and that the same perturbation set transfers across detector architectures and datasets. In the paper's formulation, attack succeeds when predicted class probability $p(c_i)$ falls below threshold $T$ relative to other classes; the update uses the full detector loss $L = L_{\\text{loc}} + L_{\\text{obj}} + L_{\\text{cls}}$ and adds perturbation only where the box mask $M[f(I')]$ is nonzero. The authors report white-box success up to 100 percent and black-box success up to 98 percent, with cross-model results showing larger YOLOv8 models producing the most transferable perturbations.","pith_inferences":["Editorial extension: the claimed 100 percent and 98 percent success rates are read off relative mAP drops, not per-image object-miss counts; a per-image measure could be lower and should be reported.","Editorial extension: because the attack only perturbs predicted boxes, detectors that output dense, small, or heavily overlapping boxes may need far more iterations, as the paper itself observes for overlapping cases; an adaptive mask could improve speed.","Editorial extension: the transferability pattern in which larger models generate more general perturbations suggests shared low-level features are being exploited, so feature-space denoising or attention regularization is a testable defensive direction.","Editorial extension: the NCC-based distortion metric is a global similarity score and does not guarantee local imperceptibility, so human-perception studies would be a natural extension."],"forward_implications":["Detectors in safety-critical settings can be disabled by visually subtle perturbations confined to object regions, without touching background pixels.","A black-box attacker needs only one surrogate model; the paper's adversarial images generated against a large YOLOv8 model transfer to one-stage, two-stage, and transformer detectors.","Two-stage detectors, which resist earlier box-agnostic attacks, are still vulnerable to mask-guided iterative attacks.","Cross-dataset transfer means attacks generated in one domain can be effective in another without retraining the attack.","Defenses must reduce sensitivity of confidence scores inside predicted boxes, not merely add classifier-style robust training."],"supporting_citations":[{"why":"Provides the DAG baseline that iteratively perturbs proposed bounding boxes and is the paper's main comparison for transferability and consistency.","marker":"Xie et al., 2017"},{"why":"Provides the UEA baseline for transferable attacks on detectors, used in the one-stage versus two-stage consistency comparison.","marker":"Wei et al., 2019"},{"why":"Motivates iterative gradient-based perturbation in the physical world, which the paper extends to detectors in Eq. 9.","marker":"Kurakin et al., 2018"},{"why":"Introduces gradient-based adversarial examples (FGSM), the conceptual foundation for the paper's gradient-ascent noise.","marker":"Goodfellow et al., 2015"},{"why":"Supplies the YOLOv8 family used as primary attack targets and as the surrogate model for black-box transfer.","marker":"Jocher et al., 2023"},{"why":"Supplies Faster R-CNN, the two-stage detector used to test consistency across detection algorithms.","marker":"Ren et al., 2015"},{"why":"Supplies RetinaNet, one of the one-stage detectors used in cross-model and consistency experiments.","marker":"Lin et al., 2017"},{"why":"Supplies Swin Transformer, the transformer-based detector used in cross-model transferability.","marker":"Liu et al., 2021"},{"why":"Provides the MS COCO 2017 dataset used for in-domain training and evaluation of attacks.","marker":"Lin et al., 2014"},{"why":"Provides the PASCAL VOC 2012 dataset used for cross-domain validation of attack transferability.","marker":"Everingham et al., 2015"}],"fun_headline_variants":["Bounding-box noise fools detectors in 100% of tests","Masked pixel attacks blind YOLOv8 and others","Tiny box-only perturbations trick object detectors","Transferable adversarial boxes reach 98% black-box","Distortion-aware attack hits detector confidence"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central claim rests on measuring success rate as the relative drop in mean average precision; if a lower mAP does not mean individual objects are missed, the 100 percent and 98 percent figures overstate the attack.","fun_headline_variants_meta":{"raw":{"variants":["Bounding-box noise fools detectors in 100% of tests","Masked pixel attacks blind YOLOv8 and others","Tiny box-only perturbations trick object detectors","Transferable adversarial boxes reach 98% black-box","Distortion-aware attack hits detector confidence"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001093,"raw_usage":{"total_tokens":4588,"prompt_tokens":991,"completion_tokens":3597,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":607,"completion_tokens_details":{"reasoning_tokens":3523}},"tokens_in":607,"tokens_out":3597,"duration_ms":20528,"temperature":1.0,"reasoning_tokens":3523,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T04:26:59.906027+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the same attack on MS COCO and PASCAL VOC and count the fraction of images in which every ground-truth object is missed at a confidence threshold of 0.50, then compare that per-image success rate with the mAP-derived percentages; a large gap would falsify the headline success-rate claim.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the DAG baseline that iteratively perturbs proposed bounding boxes and is the paper's main comparison for transferability and consistency."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the UEA baseline for transferable attacks on detectors, used in the one-stage versus two-stage consistency comparison."},{"cited_title":"J., and Bengio, S","cited_arxiv_id":null,"evidence_quote":"Motivates iterative gradient-based perturbation in the physical world, which the paper extends to detectors in Eq. 9."},{"cited_title":"J., Shlens, J., and Szegedy, C","cited_arxiv_id":null,"evidence_quote":"Introduces gradient-based adversarial examples (FGSM), the conceptual foundation for the paper's gradient-ascent noise."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the YOLOv8 family used as primary attack targets and as the surrogate model for black-box transfer."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the MS COCO 2017 dataset used for in-domain training and evaluation of attacks."},{"cited_title":"A., Van Gool, L., Williams, C","cited_arxiv_id":null,"evidence_quote":"Provides the PASCAL VOC 2012 dataset used for cross-domain validation of attack transferability."}],"review_version":1}