{"id":"1f4a9d01-4338-453f-99ac-a64932eba451","arxiv_id":"2411.17468","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"The paper proposes a white-box attack that uses only the tracker's predicted bounding box to generate adversarial perturbations, and reports large performance drops on three transformer trackers.","lead":"The paper introduces a white-box attack that perturbs video frames using only the tracker's predicted bounding box, and reports large accuracy drops on transformer trackers. The attack could make adversarial robustness testing much easier for modern trackers that do not expose classification scores, masks, or heatmaps.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"ABBG's loss is only partially specified: if adversarial boxes are regenerated from the current prediction, the x/y gradient components vanish, so the reported near-zero AO requires an undocumented detached anchor.","rationale":"The paper's central claim is that a white-box attack using only the predicted bounding box can nearly zero out transformer trackers. This requires a well-defined, non-vanishing gradient of a loss with respect to the search-region input. Section 3 gives the loss in Eq. 2 but leaves the computation graph of b_i* underspecified. If b_i* is generated from the current bpred, the translation terms of ℓABBG cancel exactly: ℓr(x, x+Tx) = ℓr(Tx), independent of x. Only the scale terms w(1−s) and h(1−s) depend on the prediction, so the gradient cannot directly drive the box center away from the target. An attack that cannot shift the center cannot plausibly reduce AO from 0.734 to 0.027 on GOT-10k in 10 iterations, unless other frame-to-frame effects dominate. The obvious fix—detach bpred and draw target boxes around the clean prediction—is not described. The IoU-based sample selection is also non-differentiable unless the selected mask is treated as a constant, which is not stated. The paper reports no code, no seeds, and no variance, so the numerical tables cannot disambiguate these implementation choices. The reader's REJECT is therefore well founded; my concern is the same one, sharpened to the partial rather than complete vanishing of the gradient. The concrete test above would settle whether the described algorithm actually produces nonzero center-moving gradients and reproduces the reported drops.","tokens_in":7766,"tokens_out":7281,"duration_ms":73493,"concrete_test":"On one GOT-10k sequence, implement ABBG exactly as written (Eqs. 1–2, no detach of bpred) using the TransT-M checkpoint and compute the gradient of ℓABBG with respect to the search region. Measure the gradient norm contributed by x,y coordinates and the resulting IoU drop. If the x/y gradient is zero and no significant drop occurs, the stated method is insufficient and the reported results require an unspecified detached anchor or a different loss. Then run the variant where bpred is detached before generating b_i*; if only that variant reproduces the paper's numbers, Section 3 must be corrected.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3 defines the attack loss as ℓABBG = Σ_i ℓr(bpred, b_i*) with b_i* generated from the tracker's predicted box via Eq. 1. Read literally, b_i* is a function of the current bpred. For the translation coordinates, bpred and b_i* differ only by the random constants Tx,Ty, so ℓr(x, x+Tx) is independent of x; the x/y component of the gradient is exactly zero. The scale terms retain a dependence through w(1−s) and h(1−s), but no term rewards moving the box center. To drive GOT-10k AO from ~0.7 to <0.03 in 10 iterations, the attack must shift the predicted box far from the object, yet the stated loss cannot produce that gradient signal. If the authors intended b_i* to be computed from a detached, clean anchor (or from the first iteration's prediction), this is not stated anywhere. The paper also does not specify how the adaptive IoU-based positive-sample selection is differentiated. Without code or an explicit computation graph, the central experimental claim is not reproducible from the text.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ABBG, a white-box adversarial attack against transformer-based visual object trackers, using only the tracker's predicted bounding box as the attack proxy. The method generates k random translated/scaled copies of the predicted box, computes a smoothed L1 regression loss between the predicted box and these adversarial boxes, and backpropagates the loss through the tracker to perturb the input search region. Experiments on TransT-M, ROMTrack, and MixFormer over GOT-10k, UAV123, and VOT2022-ST report large performance drops, e.g., TransT-M AO from 0.734 to 0.027 on GOT-10k, with the method claimed to be applicable to trackers where SPARK, RTAA, and TrackPGD cannot be used.","tokens_in":7997,"tokens_out":4395,"duration_ms":41028,"significance":"If valid, the work would fill a real gap: a white-box attack that works with only a single bounding box output would enable robustness evaluation of modern transformer trackers that lack classification/regression heads or binary masks. The paper also attempts a comparison with existing white-box attacks and reports sparsity/imperceptibility metrics. However, the central attack formulation is under-specified, and as written the loss cannot generate the gradients needed to move the predicted box, which directly undermines the reported results. No code or detailed computation graph is provided, so the experiments are not reproducible from the text. The contribution is therefore not established.","major_comments":[{"comment":"The loss ℓABBG = Σ_i ℓr(bpred, b_i*) is computed with b_i* generated from bpred via Eq. (1): x'_i = x + Tx_i, y'_i = y + Ty_i, w'_i = w s_i, h'_i = h s_i. For the translation coordinates, ℓr(x, x+Tx_i) and ℓr(y, y+Ty_i) are independent of x and y, so ∂ℓABBG/∂x = ∂ℓABBG/∂y = 0. The width and height terms retain a gradient through w(1−s_i) and h(1−s_i), but that can only shrink or grow the box, not shift it. The reported drops to near-zero AO in Table 1 require the predicted box to move far from the target. Either the paper must specify that b_i* is generated from a detached prediction (or from a fixed clean anchor), or it must provide a computation graph showing how the translation gradients arise. As written, the attack cannot work, and the experimental numbers are unexplained.","section":"Section 3, Eq. (1) and Eq. (2)"},{"comment":"The adaptive IoU-based selection of positive and negative samples is not defined precisely enough to be implemented or differentiated. The text says 'an adaptive threshold that only retains bounding boxes with an Intersection over Union (IoU) greater than a fixed threshold, specifically 80% of all adversarial bounding boxes,' which is self-contradictory: the threshold is both adaptive and fixed. It is also unclear whether the selected set is a hard mask and how the non-differentiable selection is treated during backpropagation.","section":"Section 3, Figure 2"},{"comment":"The central claim that ABBG is 'the first white-box attack that is applicable to a wide range of trackers with transformer backbones' is not substantiated by the experiments. For ROMTrack and MixFormer, the tables report only CSA, IoU, and ABBG; SPARK, RTAA, and TrackPGD are evaluated only on TransT-M. To support the applicability claim, the authors should either run the other white-box attacks on ROMTrack and MixFormer or provide concrete, code-level evidence of why those attacks cannot be applied.","section":"Section 4.1, Tables 1 and 2"}],"minor_comments":[{"comment":"The dataset name 'UA V123' should be 'UAV123' (the typo also appears in the abstract and in Tables 1 and 3).","section":"Abstract and throughout"},{"comment":"There is a typo in 'The use of vision transformers in the backbone of tackers' — 'tackers' should be 'trackers'.","section":"Section 2"},{"comment":"The phrase 'closest to the the predicted bounding box' contains a duplicated 'the'.","section":"Section 3"},{"comment":"The units of the translation parameters (sampled uniformly between 0.1 and 0.4) are not specified. They need to be defined as absolute pixels or as fractions of the box width/height; otherwise, the setup is not reproducible.","section":"Section 4, Attack setup"},{"comment":"The paper states that the number of iterations is 10 and the perturbation is clipped to an ϵ-ball with ϵ = 10, but it does not specify the step size or the exact PGD update rule, which are needed for reproduction.","section":"Section 4, Attack setup"}],"recommendation":"reject","confidential_remarks":"The gradient-vanishing issue in Eq. (2) is the kind of load-bearing flaw that cannot be patched by minor rewording: as written, the attack cannot shift the predicted box, so the reported results must come from an undocumented implementation detail (e.g., a detached anchor) or from a different loss altogether. The lack of code or an explicit computation graph makes this impossible to verify. I would advise the editor that the paper should be rejected unless the authors can provide a corrected formulation and full code, which would amount to a substantially revised submission."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The contribution here is real: ABBG is a white-box attack that needs only the tracker's predicted bounding box as a proxy, unlike SPARK, RTAA, and TrackPGD which require classification/regression labels, heatmaps, or masks. That extends white-box attacks to ROMTrack and MixFormer, which is a useful advance for robustness benchmarking. The experimental coverage is thorough—three transformer trackers, three datasets, multiple metrics—and the reported drops are dramatic. The qualitative figures help make the case. The idea is simple and clearly motivated.\n\nThe soft spot is the loss specification, but it's not quite the fatal problem the stress-test makes it out to be. In Eq. (2), if the adversarial boxes b_i* are recomputed from the current prediction without detaching, the translation part of the loss is indeed constant w.r.t. x and y, so the gradient on the center coordinates vanishes. However, the scale terms ℓr(w, w*s) are not constant; they produce nonzero gradients on width and height. The attack can therefore work by shrinking the box to a degenerate size, which alone can drive overlap to near zero. So the claim that \"the loss is constant and the gradient is zero\" is too strong. What is missing is an explicit statement of whether b_i* is detached (or anchored to a clean prediction), how the adaptive IoU-based sample selection is differentiated, and what coordinate frame the translations Tx, Ty are in (pixels vs. fractions of box size). Those are load-bearing for reproducibility, and with no code released, a reader cannot resolve them from the text.\n\nAlso worth noting: the adaptive threshold is described vaguely—\"80% of all adversarial bounding boxes\" is a retention rate, not an IoU threshold—and the results are single-run, with no variance over seeds or initializations for what is a stochastic attack. These are fixable in revision.\n\nThe citation pattern is fine; the self-citations to TrackPGD and the reproducibility study are relevant and frame the gap accurately. The paper is not incoherent on its own terms, just under-documented.\n\nBottom line: this deserves a serious referee, not a desk reject. The novelty is legitimate and the mechanism likely works as a shrink attack, but the paper needs a major revision that spells out the exact computation graph, releases code, and adds uncertainty estimates. I'd send it to review with that expectation.","headline":"The single-bbox proxy is genuinely novel and the results are striking, but the paper under-specifies the loss graph; the zero-gradient critique is overstated because scale gradients alone can shrink the box to near-zero overlap.","tokens_in":8528,"tokens_out":7615,"would_cite":false,"duration_ms":75130,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"ABBG is a white-box attack that uses only the tracker's predicted bounding box and drives TransT-M, ROMTrack, and MixFormer to near-zero overlap on GOT-10k.","keywords":["adversarial attack","visual object tracking","transformer trackers","white-box attack","bounding box regression","GOT-10k","adversarial robustness","Intersection over Union"],"falsifier":"Take a tracker with the same weights and replace only the final box decoding with a non-differentiable operation such as rounding the box coordinates or using argmax over a fixed grid, then run ABBG with the same budget. If the average overlap on GOT-10k stays near the clean values instead of collapsing to the reported 0.027, 0.006, and 0.002, the claim that one bounding box is sufficient as a white-box proxy is refuted. A second check: apply a random perturbation of the same $\\epsilon=10$ budget; if it alone collapses the scores, the specific regression loss is not the cause.","tokens_in":7581,"feed_emoji":"🎯","tokens_out":11561,"duration_ms":96580,"temperature":0.7,"pith_summary":"This paper tries to establish a simple fact: to attack a transformer-based visual object tracker in a white-box setting, the tracker's single predicted bounding box is enough, with no classification labels, regression labels, heatmaps, or masks required. From that one box, the method samples thousands of random adversarial boxes, keeps those with high IoU with the prediction, and backpropagates a smoothed-L1 regression loss through the tracker to perturb the search region. On GOT-10k the attack reduces TransT-M from 0.734 to 0.027 average overlap, ROMTrack from 0.729 to 0.006, and MixFormer from 0.696 to 0.002, while SPARK, RTAA, and TrackPGD cannot even be applied to ROMTrack and MixFormer. The paper concludes that transformer trackers can be seriously challenged by an attacker that uses nothing beyond the bounding box prediction, opening these trackers to standardized white-box robustness comparisons.","feed_headline":"A single predicted box can defeat transformer trackers","feed_subtitle":"ABBG attack drops TransT-M, ROMTrack, and MixFormer to near-zero scores using only the tracker's own predicted box.","key_machinery":"The load-bearing object is the set of Adversarial Bounding Boxes (ABBs) generated from the predicted box. Each step samples $k=1024$ boxes $b_i^*$ with $x_i' = x + T_{x_i}$, $y_i'=y+T_{y_i}$, $w_i'=w s_i$, $h_i'=h s_i$ from uniform distributions, then keeps only boxes with IoU above an adaptive threshold that always retains about 80% of the samples so the loss is never empty. The attack loss is $\\ell_{\\mathrm{ABBG}} = \\sum_i \\ell_r(b_{\\mathrm{pred}}, b_i^*)$ with $\\ell_r$ the smoothed L1 norm; this regression loss is backpropagated through the transformer to produce the perturbation, iterated 10 times with $\\epsilon=10$ clipping. The random sampling plus thresholding is what turns a single non-probabilistic box output into a differentiable training signal.","core_discovery":"The central claim is that a transformer tracker's own predicted bounding box is a sufficient white-box attack proxy. ABBG samples 1,024 adversarial boxes around the prediction using random translations and scales, keeps the subset whose Intersection over Union with the predicted box lies above an adaptive threshold so that about 80% of the sampled boxes remain positive samples, and minimizes the smoothed-L1 regression loss between the prediction and those retained boxes by backpropagating through the tracker. In a white-box setting this drives the three tested trackers to near failure: on GOT-10k, TransT-M average overlap falls from 0.734 to 0.027, ROMTrack from 0.729 to 0.006, and MixFormer from 0.696 to 0.002. The same attack also applies to TransT-M on UAV123 and VOT2022-ST, and unlike SPARK, RTAA, and TrackPGD, it does not require classification or regression labels, heatmaps, or binary masks, which are absent from ROMTrack and MixFormer pipelines.","pith_inferences":["If the single-box regression loss is the actual driver of the collapse, then defenses should harden the box-regression head and the gradient path, not the classification or score branches; this is a direction the paper does not itself propose.","The adaptive threshold always keeping roughly 80% of sampled boxes suggests the attack is a form of smoothed gradient over the box-output manifold; a natural testable variant would sample boxes from a deterministic grid to see whether randomness or thresholding matters more.","The same 'perturb the tracker's own output' recipe could transfer to other differentiable output heads, such as keypoints, masks, or rotated boxes, where no candidate list exists, although the paper does not test this.","If a tracker detached or non-differentiably decoded its box, ABBG would likely fail, which points to a cheap but possibly harmful defense: cutting gradients from the box head."],"forward_implications":["ABBG gives a single white-box attack that runs on TransT-M, ROMTrack, and MixFormer, so the adversarial robustness of these transformer trackers can be compared on the same footing.","On ROMTrack and MixFormer, the attack drives SR0.5 and SR0.75 to zero, meaning the tracker fails even coarse overlap, not just precise localization.","Because the only proxy is the predicted box, any transformer tracker that returns a differentiable box prediction is a candidate target, no matter what other outputs its pipeline does or does not provide.","The attack achieves these drops within the same 10-iteration, epsilon=10 budget as SPARK and RTAA, while ranking second in sparsity and imperceptibility among white-box attacks."],"supporting_citations":[{"why":"Introduces the transformer tracking architecture (TransT) whose ECA/CFA modules define the attacked model family.","marker":"Chen et al., 2021"},{"why":"Provides TransT-M, the tracker whose GOT-10k AO drops from 0.734 to 0.027 under ABBG.","marker":"Chen et al., 2023"},{"why":"Defines MixFormer and its Mixed Attention Module; ABBG reduces its AO from 0.696 to 0.002.","marker":"Cui et al., 2022"},{"why":"Defines ROMTrack, an attacked tracker where SPARK, RTAA, and TrackPGD cannot run and ABBG reduces AO from 0.729 to 0.006.","marker":"Cai et al., 2023"},{"why":"Supplies GOT-10k and the AO/SR metrics behind the main quantitative claims.","marker":"Huang et al., 2019"},{"why":"Supplies the VOT2022-ST protocol and EAO, accuracy, and robustness metrics used for a second evaluation.","marker":"Kristan et al., 2023"},{"why":"Supplies UAV123, where ABBG beats baselines in success rate on TransT-M.","marker":"Mueller et al., 2016"},{"why":"SPARK baseline; its need for classification and regression labels motivates the single-box proxy.","marker":"Guo et al., 2020"},{"why":"RTAA baseline; also label-dependent, and used to compare against ABBG on TransT-M.","marker":"Jia et al., 2020"},{"why":"TrackPGD baseline and the reproducibility study that frames the need for a broadly applicable white-box attack.","marker":"Nokabadi et al., 2024"}],"fun_headline_variants":["One predicted box is enough to break transformer trackers","Single box attack defeats transformer trackers","Tracker's own box is all an attacker needs","White-box attack uses only predicted box to fool trackers","One box, no masks: ABBG breaks transformer trackers"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The attack only works if the tracker's predicted bounding box is a smooth, learnable function of the input image; if the box comes from any step that blocks the gradient, or if the sampled boxes are recomputed so the loss stops changing, the perturbation has no gradient to follow and collapses.","fun_headline_variants_meta":{"raw":{"variants":["One predicted box is enough to break transformer trackers","Single box attack defeats transformer trackers","Tracker's own box is all an attacker needs","White-box attack uses only predicted box to fool trackers","One box, no masks: ABBG breaks transformer trackers"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000522,"raw_usage":{"total_tokens":2504,"prompt_tokens":906,"completion_tokens":1598,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":522,"completion_tokens_details":{"reasoning_tokens":1533}},"tokens_in":522,"tokens_out":1598,"duration_ms":10790,"temperature":1.0,"reasoning_tokens":1533,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T12:05:36.535067+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a tracker with the same weights and replace only the final box decoding with a non-differentiable operation such as rounding the box coordinates or using argmax over a fixed grid, then run ABBG with the same budget. If the average overlap on GOT-10k stays near the clean values instead of collapsing to the reported 0.027, 0.006, and 0.002, the claim that one bounding box is sufficient as a white-box proxy is refuted. A second check: apply a random perturbation of the same $\\epsilon=10$ budget; if it alone collapses the scores, the specific regression loss is not the cause.","supporting_citations":[{"cited_title":"Robust Tracking Against Adversarial Attacks","cited_arxiv_id":null,"evidence_quote":"RTAA baseline; also label-dependent, and used to compare against ABBG on TransT-M."},{"cited_title":"Transformer Tracking","cited_arxiv_id":null,"evidence_quote":"Introduces the transformer tracking architecture (TransT) whose ECA/CFA modules define the attacked model family."},{"cited_title":"High-Performance Transformer Tracking","cited_arxiv_id":null,"evidence_quote":"Provides TransT-M, the tracker whose GOT-10k AO drops from 0.734 to 0.027 under ABBG."},{"cited_title":"MixFormer: End-to-End Tracking With Iterative Mixed Attention","cited_arxiv_id":null,"evidence_quote":"Defines MixFormer and its Mixed Attention Module; ABBG reduces its AO from 0.696 to 0.002."},{"cited_title":"Robust Object Modeling for Visual Tracking","cited_arxiv_id":null,"evidence_quote":"Defines ROMTrack, an attacked tracker where SPARK, RTAA, and TrackPGD cannot run and ABBG reduces AO from 0.729 to 0.006."},{"cited_title":"GOT -10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wild","cited_arxiv_id":null,"evidence_quote":"Supplies GOT-10k and the AO/SR metrics behind the main quantitative claims."},{"cited_title":"The Tenth Visual Object Tracking VOT2022 Challenge Results","cited_arxiv_id":null,"evidence_quote":"Supplies the VOT2022-ST protocol and EAO, accuracy, and robustness metrics used for a second evaluation."},{"cited_title":"A Benchmark and Simulator for UAV Tracking","cited_arxiv_id":null,"evidence_quote":"Supplies UAV123, where ABBG beats baselines in success rate on TransT-M."},{"cited_title":"SPARK: Spatial-Aware Online Incremental Attack Against Visual Tracking","cited_arxiv_id":null,"evidence_quote":"SPARK baseline; its need for classification and regression labels motivates the single-box proxy."}],"review_version":1}