{"id":"07821a64-061b-4c28-8536-00a3e4dc4244","arxiv_id":"2412.16955","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"NumbOD attacks object detectors with a spatial loss that collapses predicted boxes toward the corner and a frequency loss that amplifies high-frequency differences, reducing mAP50 to below 5.5 on all nine tested models.","lead":"This paper introduces NumbOD, an adversarial attack that fools object detectors by adding small noise so that predicted boxes shrink toward a corner and true objects are reported as background. The push works on nine detector architectures using only final outputs plus a frequency-domain loss, a useful stress test for safety-critical vision systems.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Equation 5's foreground-background separation loss requires an explicit background class c_K, but YOLOv5 (and likely RepPoints) output objectness plus K class probabilities; the paper reports results on these detectors without documenting the adaptation, so the model-agnostic claim is unsupported…","rationale":"The reader's weakest_assumption identified two issues: the gradient path through non-differentiable output post-processing and the missing background class for detectors like YOLO and RepPoints. I evaluated both. The gradient path is concerning but not fatal: for each selected final detection, gradients can flow through the classification and regression heads, and the non-differentiable NMS/top-k selection is commonly handled by keeping a fixed set of detections from the benign forward pass. The background-class issue is more concrete and directly invalidates Eq. 5 for at least YOLOv5, which lacks an explicit background class in its K-class probability output. Since the paper reports strong attack results on YOLO but never specifies how the background score is defined, the central claim that a single output-only loss works across all detectors is not fully supported. This is a reproducibility and specification gap, addressable in revision, so the reader's CONDITIONAL verdict is appropriate. I also note the IW-SSIM metric appears to be reported as (1-IW-SSIM)*100 rather than IW-SSIM itself, which weakens the stealthiness claims but does not affect the core effectiveness claim. Overall, my concern reinforces the reader's conditional acceptance rather than changing it.","tokens_in":14384,"tokens_out":9142,"duration_ms":80709,"concrete_test":"Download the released code and inspect the YOLOv5 experiment in the attack script. Identify how Jcls is computed: does it reference a class index K that does not exist in YOLO's output tensor? If it uses 1 - objectness as c_K, replace that term with a literal background class (e.g., by adding a constant background logit) and re-run the attack on 500 MS-COCO images; if Table 1's mAP50 of 0.59 cannot be reproduced with Eq. 5 as literally written, the model-agnostic formulation is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that one output-only loss (Eqs. 4-5) works across nine detectors. Eq. 5 minimizes log(c_gt) and maximizes log(c_K), where c_K is an explicit background class among K+1 class probabilities. YOLOv5's head outputs an objectness score o and K class-conditional probabilities p_k; there is no K+1-th background class in the classification branch. The paper never says how c_K is defined for YOLO (or for RepPoints, whose head also differs). If the code substitutes c_K = 1-o, that is a non-trivial variant of Eq. 5: the gradient with respect to objectness becomes 1/o + 1/(1-o) (up to sign), which is not what Eq. 5 describes. If instead a different loss is used, then the unified formulation is misleading, and the 'model-agnostic' claim rests on undocumented per-model hacks. Because Table 1 reports very low mAP for YOLO (0.59 on MS-COCO), this gap directly affects the headline. Relatedly, the paper does not specify whether the loss is applied to pre-NMS raw predictions or to post-NMS outputs; that affects reproducibility, but a fixed-set loss on surviving detections is a standard workaround. The background-class omission is the sharper, checkable defect.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes NumbOD, a white-box adversarial attack against object detectors that aims to be model-agnostic by using only the detector's final outputs (bounding boxes and class scores) rather than its internal structures. The method combines a spatial loss that pushes predicted boxes toward a corner and drives classification toward an explicit background class, with a frequency-domain loss based on the discrete wavelet transform that encourages perturbations to concentrate in high-frequency components. Experiments on nine object detectors (Faster R-CNN, Cascade R-CNN, SABL, Sparse R-CNN, RepPoints, Deformable DETR, VFNet, TOOD, YOLOv5) and two datasets (MS-COCO, PASCAL VOC) report mAP50 dropping from roughly 47–61 to below 5.5 under an 8/255 L-infty budget, together with comparisons against six prior attacks and an ablation study. The paper claims that NumbOD outperforms all existing approaches in both effectiveness and stealthiness.","tokens_in":14675,"tokens_out":6293,"duration_ms":53852,"significance":"If the method is fully specified and reproducible, this is a practically relevant contribution: most existing object-detector attacks are architecture-specific (e.g., targeting RPN or NMS), whereas a simple output-only attack that works across two-stage, one-stage, and transformer-based detectors would be useful for robustness evaluation. The paper includes a public code link, evaluates on nine detectors and two datasets, and the reported attack success is striking. However, the central 'model-agnostic' claim is currently undermined by underspecification of how the loss is applied to detector outputs (pre-NMS vs. post-NMS) and how the background-class term in Eq. (5) is instantiated for detectors that do not have an explicit background class. These issues are load-bearing for reproducibility and for the claim that the attack uses only final outputs. The metric definition for IW-SSIM is also inconsistent with the reported numbers. The contribution is promising, but the manuscript needs substantial clarification and additional experimental rigor before the claims are supportable.","major_comments":[{"comment":"The manuscript does not specify whether Jloc and Jcls are computed on pre-NMS raw predictions or on the post-NMS final detections. Standard detectors (Faster R-CNN, YOLOv5, Deformable DETR) involve non-differentiable NMS, score filtering, and top-k selection; a loss defined on the final boxes has zero gradient through these stages. The paper must state the exact tensors used: for example, losses on raw per-anchor or per-query predictions, a differentiable surrogate, or a straight-through estimator. This is central both to reproducibility of the reported mAP numbers and to the claim that the attack is model-agnostic and relies only on final output features.","section":"Methodology, 'Spatial coordinated deviation attack', Eqs. (4)-(5)"},{"comment":"Equation (5) assumes an explicit background class c_K among K+1 output class probabilities. YOLOv5's classification head outputs K class-conditional probabilities plus an objectness score, with no background class; RepPoints and Deformable DETR use different head designs. The paper reports attack results for all three detectors in Table 1 but never documents how c_K is obtained for them. If c_K is replaced by 1-o (objectness complement) or some other quantity, the gradient of the loss changes substantially and the unified formulation in Eq. (5) is misleading. Provide the exact per-model instantiation of this loss for each detector family, or the 'model-agnostic' claim is unsupported.","section":"Methodology, 'Spatial coordinated deviation attack', Eq. (5)"},{"comment":"The evaluation is based on a single randomly selected set of 5000 images per dataset with no repetition, no seeds, and no error bars. Because the attack uses stochastic optimization (Adamax) and random sample selection, detector-to-detector differences such as mAP50 0.38 vs. 0.59 may not be statistically meaningful. Report variance over multiple independent runs and specify the sampling protocol, or explicitly state that results come from a single run and soften comparative claims such as 'significantly dropping' and 'our method outperforms all existing approaches.'","section":"Experiments, 'Attack Performance', Table 1"},{"comment":"The statement 'we default to multiplying the values of mAP, IW-SSIM, and NMSE by 100' is inconsistent with the reported numbers: mAP values in the tables are already percentages, and IW-SSIM values around 0.17 cannot be the product of a standard IW-SSIM score and 100. The numbers are consistent with reporting 100*(1-IW-SSIM) as a distance measure. Define the exact quantity reported in Tables 1 and 2, and use consistent notation, because the stealthiness comparison in Table 2 depends on this definition.","section":"Experiments, 'Evaluation metrics'"},{"comment":"The abstract and comparison section claim NumbOD 'outperforms all existing approaches in terms of effectiveness and stealthiness,' but Table 2 shows FGSM has a lower (better) IW-SSIM value (0.16 vs. 0.17 on Faster R-CNN/MS-COCO), and the text acknowledges that FGSM 'excels over us on the IW-SSIM metric.' Revise the claim to say NumbOD outperforms on effectiveness and on most stealthiness metrics, or provide a statistical test showing the IW-SSIM difference is not significant.","section":"Experiments, 'Comparison Study', Table 2"}],"minor_comments":[{"comment":"The matrix notation in Eqs. (6)-(8) is confusing: the image x is used both for the original image and for the decomposed sub-bands, and the reconstruction of phi(x) and psi(x) from single sub-bands should be described more precisely. Clarify the dimensions and the role of the dropped mid-frequency components.","section":"Methodology, 'Critical frequency interference attack', Eqs. (6)-(8)"},{"comment":"The sentence 'deep neural networks are biased towards image textures' is stated without a citation; add a reference to the texture-bias literature (e.g., Geirhos et al., ICLR 2019) to support this premise.","section":"Methodology, 'Key Challenges and Intuitions'"},{"comment":"The value of k in the dual-track target selection strategy is never reported. Please include the chosen k and any sensitivity analysis in the implementation details.","section":"Methodology, 'Spatial-Frequency Fusion Attack'"},{"comment":"Equation (1) writes the objective as an expectation over the data distribution, but the actual attack appears to be image-specific. Clarify whether the formulation is per-image or universal and how Eq. (1) relates to the optimization procedure described in 'Spatial-Frequency Fusion Attack.'","section":"Methodology, 'Problem Formulation'"},{"comment":"RAP and LGP use perturbation budgets that are not L-p norm constrained, while NumbOD and the other baselines use 8/255. The comparison would be fairer if a non-L-p-constrained variant of NumbOD were also reported, or if the text explicitly discussed why the comparison is still meaningful.","section":"Experiments, 'Comparison Study'"},{"comment":"There are several typographical and phrasing issues: 'as follow' should be 'as follows', 'the first model-agnostic' should be 'a model-agnostic', and the phrase 'we default to multiplying the values of mAP, IW-SSIM, and NMSE by 100' should be rewritten for clarity.","section":"General editorial"}],"recommendation":"major_revision","confidential_remarks":"The reference list contains a substantial number of self-citations (roughly 11 of the paper's own prior works). They are peripheral to the technical content and do not supply any load-bearing result, but the authors may wish to trim them to those actually used. The metric inconsistency around IW-SSIM is a fixable presentation issue, but it should be addressed before acceptance. The code link is a positive step; the authors should be asked to point to the specific files that resolve the NMS-gradient and background-class questions."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the results are probably real, but the paper as written overclaims 'model-agnostic' because it never says how Eq. 5 is applied to detectors without an explicit background class like YOLOv5 and RepPoints. That's the main thing to know.\n\nWhat's new: a white-box attack combining two output-based spatial losses (corner-collapse to (0,0) and foreground-background separation) with a DWT-based frequency decoupling loss, plus a dual-track top-k target selection. The combination is new, and the ablation shows each piece contributes. Running on nine detectors and two datasets with a code link is good practice. The defense evaluation (corruption, pruning, fine-tuning, adversarial training) is a useful addition.\n\nSoft spots, roughly in order.\n\nFirst, Eq. 5 explicitly uses a background class c_K among K+1 class scores. YOLOv5's head gives objectness and K class probabilities; RepPoints also differs. The paper never documents how c_K is derived for these models. If the code substitutes 1 - objectness, that's a materially different loss and the paper should say so. As written, the unified formulation is misleading. This affects the headline because YOLO and RepPoints are in Table 1.\n\nSecond, the paper doesn't say whether the spatial loss is computed on raw pre-NMS predictions or on post-NMS outputs. The boxes that survive NMS and top-k have no gradient through those stages; if they use a differentiable surrogate or a fixed set, that should be stated. It's a reproducibility issue, not necessarily a flaw.\n\nThird, the 'IW-SSIM' column appears to be 100*(1-IW-SSIM), since raw IW-SSIM near 1 would give numbers like 0.17 on that scale. That's a mislabel or a missing transformation.\n\nFourth, no error bars or multiple runs, the top-k value isn't reported, and the SOTA comparison is only on Faster R-CNN and VFNet. All minor and fixable.\n\nThe central idea holds up as an attack recipe; the missing details don't make me doubt the numbers, they make me doubt the model-agnostic framing. This is a solid empirical paper that deserves peer review with a request for proper specification and a code-checked appendix.\n\nFor you: if you work on adversarial detection or robustness evaluation, it's worth a look once the authors fix the background-class handling. I'd send it to a serious reviewer.","headline":"The reported attack numbers are likely real, but the 'model-agnostic' claim rests on an unspecified adaptation of the classification loss to detectors without a background class.","tokens_in":15249,"tokens_out":3640,"would_cite":true,"duration_ms":31104,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a single white-box attack recipe, combining spatial box-shift and classification-confusion losses with wavelet-based high-frequency perturbation, can make nine modern object detectors effectively blind at an 8/255…","keywords":["adversarial attack","object detection","model-agnostic attack","spatial-frequency fusion","discrete wavelet transform","bounding box regression","white-box threat model","mAP evaluation"],"falsifier":"Compute the spatial loss of Equations 4–5 on a detector's post-NMS, post-filtering outputs while blocking gradients at the non-differentiable stages; if mAP50 no longer collapses, the reported blindness depends on an undocumented access to internal (pre-NMS) predictions rather than on final outputs alone.","tokens_in":14145,"feed_emoji":"🎯","tokens_out":9889,"duration_ms":79137,"temperature":0.7,"pith_summary":"NumbOD is an adversarial attack against object detectors that claims to be model-agnostic: it uses only the detector's final outputs—predicted bounding boxes and class scores—rather than internal components like region proposal networks or non-maximum suppression. The attack combines a spatial loss that pushes predicted boxes toward an image corner and shifts classification scores toward background, with a frequency loss that uses the discrete wavelet transform to concentrate perturbation in high-frequency texture regions while leaving low-frequency structure intact. On 5,000 images from MS-COCO and PASCAL VOC with an 8/255 budget, the paper reports mAP50 collapsing from clean values of 47.58 to 60.79 down to 0.27 to 5.49 across nine detectors spanning two-stage, single-stage, and transformer architectures, and claims it outperforms six prior attacks on both effectiveness and stealthiness. If true, the result would mean that architectural differences among modern detectors offer little protection against a single output-level attack recipe.","feed_headline":"Attack drops nine object detectors to near-zero mAP","feed_subtitle":"Uses only the detector's own outputs to shift boxes and erase objects under a small, invisible perturbation.","key_machinery":"The load-bearing object is the total attack loss $J_{total} = J_{sa} + J_{fa}$. $J_{sa}$ (spatial coordinated deviation attack) is the sum of a localization loss $J_{loc}$ that pulls predicted bounding-box corners toward $(0,0)$ using Smooth L1, and a classification loss $J_{cls}$ that suppresses ground-truth class scores and raises the background class score; $J_{fa}$ (critical frequency interference attack) is built on the discrete wavelet transform: it reconstructs the image from low-frequency coefficients ($\\phi(x)$) and from high-frequency coefficients ($\\psi(x)$), then minimizes the low-frequency difference between benign and adversarial images while maximizing the high-frequency difference. The dual-track target selection chooses top-k boxes by IoU quality for regression and by label match for classification, so the perturbation is spent on high-quality object boxes rather than background. This machinery is what lets the attack stay model-agnostic: only final outputs and the DWT coefficients enter the loss.","core_discovery":"On the paper's own terms, the central discovery is that object detectors can be made effectively blind by optimizing a single loss defined on their final outputs, provided the loss couples three mechanisms: (1) a dual-track target-selection step that picks the top-k bounding boxes separately for regression and classification quality; (2) a spatial coordinated deviation attack, $J_{sa} = J_{loc} + \\lambda J_{cls}$, where $J_{loc}$ uses Smooth L1 to drag predicted box corners toward $(0,0)$ and $J_{cls}$ minimizes the score of the true label while maximizing the background class score; and (3) a critical frequency interference attack, $J_{fa} = J_d(\\phi(x), \\phi(x+\\delta)) - J_d(\\psi(x), \\psi(x+\\delta))$, in which $\\phi$ extracts the low-frequency component and $\\psi$ the high-frequency component via discrete wavelet transform, pushing perturbation into high-frequency regions. The paper reports that this fusion reduces mAP50 to below 5.5 on all nine tested detectors on MS-COCO at $\\epsilon = 8/255$, and that it beats FGSM, PGD, DAG, RAP, TOG, and LGP on Faster R-CNN and VFNet; it also shows the attack retains much of its effect under corruption, pruning, fine-tuning, and adversarial training.","pith_inferences":["We infer that the attack's efficiency and model-agnostic label rest on an unstated implementation detail: standard detector outputs pass through non-differentiable non-maximum suppression, score filtering, and top-k selection, so for Equations 4–5 to supply gradients, the loss must be computed either on raw pre-NMS predictions or through a differentiable surrogate. If the reported code uses pre-NM","We infer that Equation 5's explicit background class cannot be applied verbatim to detectors such as YOLO and RepPoints that do not expose a background class; the paper does not document the adaptation, so the nine-model uniform claim is stronger than the written loss supports unless such an adaptation exists.","We infer a testable extension: if high-frequency interference is the main driver, then a frequency-only variant should already suppress detection on texture-rich objects, and a low-frequency-only variant should fail; the paper's ablation runs components but does not isolate this asymmetry across all nine models.","We infer that the white-box recipe may transfer better than structure-specific attacks because it targets a common output geometry; a black-box transfer experiment across detectors would test this directly but is not reported."],"forward_implications":["If NumbOD's central claim holds, a white-box adversary does not need to know whether a detector is two-stage, single-stage, or transformer-based: final-box-level optimization suffices to drive mAP50 below 5.5 on MS-COCO with an 8/255 budget.","The frequency-loss result implies that high-frequency texture regions are a concentrated vulnerability: steering the same perturbation budget away from low-frequency structure raises both effectiveness and perceptual stealthiness.","The paper's defense experiments imply that standard robustness measures—input corruption, pruning, fine-tuning, and adversarial training—do not by themselves restore reliable detection against this attack.","The comparison against FGSM, PGD, DAG, RAP, TOG, and LGP implies that previous attacks deceive either classification or regression, whereas NumbOD's joint spatial-frequency objective is what suppresses both at once."],"supporting_citations":[{"why":"Supplies the FGSM baseline and the standard 8/255 l-infinity perturbation setup that NumbOD's experiments inherit.","marker":"Goodfellow, Shlens, and Szegedy 2015"},{"why":"Provides the DAG baseline, the first RPN-targeting attack on detectors, which NumbOD claims to outperform and generalize beyond.","marker":"Xie et al. 2017"},{"why":"Provides the RAP baseline that exploits RPN structure, the model-specific approach NumbOD contrasts its model-agnostic design against.","marker":"Li et al. 2018"},{"why":"Provides the PGD baseline and the adversarial-training defense setup used to test NumbOD's resilience.","marker":"Madry et al. 2018"},{"why":"Provides the TOG baseline, a prior model-agnostic attack that NumbOD argues still depends on RPN/anchor components and cannot handle Sparse R-CNN.","marker":"Chow et al. 2020"},{"why":"Provides the LGP baseline, the latest SOTA attack without l-p constraints, whose predicted boxes NumbOD claims still sit on objects.","marker":"Li et al. 2024a"},{"why":"Motivates the sensitivity of deep networks to high-frequency components, the premise behind NumbOD's critical frequency interference attack.","marker":"Luo et al. 2022"},{"why":"Defines Faster R-CNN, one of the nine victim detectors used in the main evaluation.","marker":"Ren et al. 2015"},{"why":"Defines Deformable DETR, the transformer-based detector used to show the attack extends beyond RPN/anchor architectures.","marker":"Zhu et al. 2020"},{"why":"Supplies the MS-COCO dataset on which the central mAP50/mAP75 attack results are measured.","marker":"Lin et al. 2014"}],"fun_headline_variants":["NumbOD: one loss on outputs blinds nine detectors to 5.5 mAP","Nine detectors drop to 5.5 mAP with NumbOD's single output loss","Erase objects and shift boxes: NumbOD's spatial-frequency attack","Fusing spatial and frequency attacks drops mAP below 5.5"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The attack assumes that the detector's final predicted boxes and class scores give a usable gradient signal for optimization, even though standard detectors route those outputs through non-differentiable steps like non-maximum suppression, score filtering, and top-k selection, and the paper does not spell out how the loss is computed despite those steps.","fun_headline_variants_meta":{"raw":{"variants":["NumbOD: one loss on outputs blinds nine detectors to 5.5 mAP","Nine detectors drop to 5.5 mAP with NumbOD's single output loss","Erase objects and shift boxes: NumbOD's spatial-frequency attack","Fusing spatial and frequency attacks drops mAP below 5.5"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000697,"raw_usage":{"total_tokens":3212,"prompt_tokens":1069,"completion_tokens":2143,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":685,"completion_tokens_details":{"reasoning_tokens":2056}},"tokens_in":685,"tokens_out":2143,"duration_ms":33172,"temperature":1.0,"reasoning_tokens":2056,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T05:57:35.893350+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the spatial loss of Equations 4–5 on a detector's post-NMS, post-filtering outputs while blocking gradients at the non-differentiable stages; if mAP50 no longer collapses, the reported blindness depends on an undocumented access to internal (pre-NMS) predictions rather than on final outputs alone.","supporting_citations":[{"cited_title":"J.; Shlens, J.; and Szegedy, C","cited_arxiv_id":null,"evidence_quote":"Supplies the FGSM baseline and the standard 8/255 l-infinity perturbation setup that NumbOD's experiments inherit."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the DAG baseline, the first RPN-targeting attack on detectors, which NumbOD claims to outperform and generalize beyond."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the PGD baseline and the adversarial-training defense setup used to test NumbOD's resilience."},{"cited_title":"E.; Truex, S.; Wei, W.; and Wu, Y","cited_arxiv_id":null,"evidence_quote":"Provides the TOG baseline, a prior model-agnostic attack that NumbOD argues still depends on RPN/anchor components and cannot handle Sparse R-CNN."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Motivates the sensitivity of deep networks to high-frequency components, the premise behind NumbOD's critical frequency interference attack."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines Faster R-CNN, one of the nine victim detectors used in the main evaluation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines Deformable DETR, the transformer-based detector used to show the attack extends beyond RPN/anchor architectures."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the MS-COCO dataset on which the central mAP50/mAP75 attack results are measured."}],"review_version":1}