{"id":"434e37d1-1e6a-4856-8e7f-2ab9b64e706e","arxiv_id":"2608.05217","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A review that unifies and compares two efficiency-degradation attacks on token-pruning vision transformers, but its synthesized tables are internally inconsistent and should not be cited for quantitative claims.","lead":"This survey compares two adversarial attacks, SlowFormer and DeSparsify, that force token-pruning vision transformers to compute more, raising energy and latency without necessarily changing the prediction. It standardizes results on A-ViT, ATS, and AdaViT using GFLOPs, accuracy loss, and an Attack Success metric, but the compiled tables contain internal contradictions.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The unified GFLOPs/AS comparison is internally inconsistent: Table 3's SlowFormer/A-ViT row cannot reproduce the stated AS formula and its 0.87 no-attack baseline flatly contradicts Table 2's 3.70.","rationale":"The reader identified the same weakness: Table 3's A-ViT no-attack value 0.87 contradicts Table 2's 3.70. I agree and sharpen it: independent of any source-paper lookup, the paper's own AS formula is violated by Table 3. The unified comparison is the survey's main contribution, and the surveyed conclusion of 'consistent GFLOPs inflation' with a concrete Attack Success metric rests entirely on these transcribed numbers. Since the tables are internally inconsistent, the central claim is not supported as stated. This is an internal inconsistency, not a disagreement with field consensus; the underlying attacks are real and the motivation is sound, but those facts do not rescue a deliverables-based survey whose tables cannot be reproduced. The reader's REJECT verdict therefore stands unchanged.","tokens_in":9518,"tokens_out":4634,"duration_ms":42298,"concrete_test":"Using only values printed in the paper, recompute the SlowFormer/A-ViT row of Table 3 with the Section 5.1 formula: F_min = 0.87, F_attack = 1.26, F_max = 4.60, giving AS ≈ 10.5%, not 100%. Then check the original SlowFormer (CVPR 2024) and DeSparsify (NeurIPS 2024) papers for the evaluation settings behind the two A-ViT no-attack entries (3.70 versus 0.87), especially dataset and input resolution; if they come from different settings, the tables must be re-split and the 'consistent GFLOPs inflation' conclusion re-derived.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's stated contribution is a 'unified and standardized' comparison (Abstract; Section 5.1; Section 7), operationalized by the AS formula AS = (F_attack - F_min) / (F_max - F_min) × 100, with F_min = no-attack GFLOPs and F_max = dense ViT-Small GFLOPs (4.60). This is the load-bearing artifact of the survey, and it fails on the paper's own data. Table 2 gives A-ViT no-attack GFLOPs as 3.70; Table 3, for the same SlowFormer-on-A-ViT setup, gives no-attack GFLOPs as 0.87, a factor-of-4.25 discrepancy that no 'DeiT-S versus ViT-S' labeling can absorb. Moreover, applying the stated formula to Table 3's own numbers for SlowFormer/A-ViT yields AS = (1.26 - 0.87) / (4.60 - 0.87) ≈ 10.5%, not the reported 100%; the reported 100% can only be obtained if F_max is taken to be the attacked value 1.26, contradicting the definition used in Table 2. The defense row (1.01 GFLOPs, AS = 34%) is similarly inconsistent with the stated F_max. Because the AS column is meant to make cross-paper comparisons concrete, and because the no-attack baselines shift by a factor of four between the two tables, the survey's central empirical claim is not reproducible from the text as written.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript surveys adversarial efficiency-degradation attacks on input-adaptive Vision Transformers. It organizes the topic around three research questions, explains in Section 3 how token-pruning mechanisms (A-ViT, ATS, AdaViT) create a compute policy that is steerable from pixel space, and describes the two representative attacks: SlowFormer, a universal adversarial patch, and DeSparsify, a per-image bounded perturbation. The stated contribution is a unified, standardized comparison of the two attacks, operationalized through GFLOPs, accuracy loss, and an Attack Success (AS) metric defined as AS = (F_attack − F_min)/(F_max − F_min) × 100%. The paper also summarizes two defenses (adversarial training against universal patches and confidence-based token caps) and outlines limitations and future research directions. The quantitative backbone of the survey is the cross-paper comparison presented in Tables 2 and 3.","tokens_in":9857,"tokens_out":20967,"duration_ms":164454,"significance":"The topic is timely: efficiency-degradation attacks on adaptive ViTs are a genuine and under-organized security concern, and the mechanistic framing of Section 3.1 (the compute policy is a differentiable function of pixels, hence as steerable as a decision boundary) is a useful conceptual contribution. The AS metric is a reasonable normalization proposal, the defense summary in Section 4 is helpful, and Section 6 is honest about the limits of current attacks and defenses. I agree with the reader's report that there is no circularity concern: AS is a deterministic normalization of published GFLOPs values, not a fitted quantity. However, the survey's usefulness is conditional on the reliability of its central empirical artifact, and, as detailed below, Tables 2 and 3 contain load-bearing internal contradictions. Under corrected numbers, the qualitative conclusions may change materially, for instance whether SlowFormer restores compute close to the dense baseline on A-ViT.","major_comments":[{"comment":"The AS column of Table 3 is not reproducible from the formula stated in Section 5.1. For the SlowFormer/A-ViT rows, the formula AS = (F_attack − F_min)/(F_max − F_min) × 100% with F_min = 0.87 (the table's own no-attack entry) and F_max = 4.60 yields (1.26 − 0.87)/(4.60 − 0.87) ≈ 10.5% for no defense and (1.01 − 0.87)/(4.60 − 0.87) ≈ 3.8% with defense, not the reported 100% and 34%. The reported 100% can be obtained only by taking F_max equal to the attacked value (1.26), a normalization that contradicts the definition used in Table 2. The DeSparsify rows deviate as well: A-ViT with defense gives (3.95 − 3.70)/(4.60 − 3.70) ≈ 27.8%, not 36.9%, and ATS with defense gives ≈ 4.7%, not 5.3%. Because the AS column is the paper's stated mechanism for making cross-paper comparisons concrete (Abstract; Section 5.1; Section 7), this non-reproducibility is load-bearing for the central claim.","section":"Section 5.1, Table 3"},{"comment":"The two tables give mutually incompatible numbers for the same nominal setup. Table 2 lists A-ViT no-attack GFLOPs as 3.70 for the SlowFormer/ViT-S condition, while Table 3 lists A-ViT no-attack GFLOPs as 0.87 for the same condition (SlowFormer on ViT-S with A-ViT), a factor-of-4.25 discrepancy that cannot be explained by the 'DeiT-S versus ViT-S' labeling in Section 5.1. The SlowFormer attack GFLOPs on A-ViT are likewise 4.60 in Table 2 but 1.26 in Table 3. Section 5.1 asserts that 'the GFLOPs and accuracy-loss comparisons are directly meaningful across the two sources,' but both tables cannot be correct for the same setup, so the unified comparison rests on internally contradictory data.","section":"Tables 2 and 3"},{"comment":"Table 2 reports SlowFormer accuracy losses of 76.5%, 78.2%, and 76.9% on A-ViT, ATS, and AdaViT, respectively, but these values are never explained or discussed. They are difficult to reconcile with the paper's framing of efficiency-degradation attacks: Section 2.3 states that 'as a secondary objective, the adversary seeks to enhance the stealthiness of the attack by preserving the model's original prediction,' and Section 3.2 describes the accuracy term as optional. The contrast with DeSparsify's 0.1–1.2% losses is stark, and a model that loses roughly 77–78% of its accuracy is not operating in the same regime as one that loses 0.1–1.2%. As written, the accuracy-loss asymmetry confounds the efficiency comparison and the claim of a 'consistent' picture across the two attacks; the survey needs either to explain why SlowFormer's accuracy drops are so large or to restrict its comparison to the accuracy-preserving variant.","section":"Table 2"}],"minor_comments":[{"comment":"The heading 'Comparsion between patch and single efficiency adversarial attack' contains a spelling error: 'Comparsion' should be 'Comparison'.","section":"Section 5.1 heading"},{"comment":"'removes uniformative to-kens' should read 'removes uninformative tokens'.","section":"Section 2.2"},{"comment":"The phrase 'specifically against to the two attacks' should be 'specifically against the two attacks'.","section":"Section 4"},{"comment":"'CIF AR-10' appears twice in Table 1 and should read 'CIFAR-10'.","section":"Table 1"},{"comment":"The sentence 'Table 2 can not only increase computation but also reduce model accuracy' has the wrong subject; it is the attacks, not the table, that increase computation.","section":"Section 5.1"},{"comment":"The surrogate loss formula for A-ViT uses a non-standard indicator symbol and an ambiguous ℓMSE target; it should be typeset conventionally with the target explicitly defined.","section":"Section 3.2"},{"comment":"The no-defense AS values in Table 3 for ATS (73.5%) and AdaViT (43.2%) differ slightly from the corresponding values in Table 2 (73.3% and 43.4%) for the same conditions; these should be reconciled.","section":"Table 3"},{"comment":"The dataset and configuration underlying the SlowFormer rows of Table 3 are not stated; it is unclear whether the 0.87 GFLOPs no-attack figure refers to ImageNet or CIFAR-10.","section":"Table 3"}],"recommendation":"reject","confidential_remarks":"The topic is well within scope for a security-adjacent ML venue and the qualitative survey parts are competent, but the quantitative comparison that the paper identifies as its contribution would need to be completely rebuilt from the primary sources and re-verified before a revision could be seriously considered; the contradictions in Tables 2 and 3 are not presentation-level typos and defeat the paper's stated purpose. I would advise the editor that any resubmission should be treated as a new submission with a fully re-derived empirical section."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe survey is a worthwhile read on an emerging attack class, but the thing it's selling—the unified GFLOPs/Attack Success comparison—doesn't survive contact with its own tables. If you need a quick orientation on SlowFormer, DeSparsify, and token pruning, the prose is fine. If you need numbers you can trust, don't use these tables.\n\nWhat's genuinely useful: the paper explains why input-adaptive policies (halting scores, Gumbel masks, sampling distributions) make the compute path steerable from pixel space, and it lays out the loss formulations for both attacks clearly. The defense summary (confidence-based caps for DeSparsify, patch-pool adversarial training for SlowFormer) is a reasonable digest. The AS metric is a nice idea for normalizing across papers.\n\nThe soft spots are not minor. Table 2 gives A-ViT no-attack GFLOPs as 3.70; Table 3, for the same SlowFormer/A-ViT setup, gives no-attack GFLOPs as 0.87. That's a factor of four. Running the stated AS formula on Table 3's own numbers gives roughly 10.5% for the no-defense case, not the reported 100%. The only way to get 100% is to take F_max as the attacked value, which contradicts the definition in Table 2. The defense row (1.01 GFLOPs, AS=34%) is equally hard to reproduce. Separately, the SlowFormer accuracy losses of 76–78% in Table 2 are never explained; if these attacks inflate compute while preserving accuracy, those drops are anomalous and need a source or a caveat.\n\nThe 'DeiT-Small and ViT-Small are interchangeable' line in Section 5.1 is too casual. The architectures are similar but not identical; using that to justify cross-paper GFLOPs comparisons needs more support or at least a sensitivity check.\n\nSo: the conceptual taxonomy and the mechanism explanations are solid, but the quantitative centerpiece is not reproducible from the text. That's a load-bearing flaw for a survey whose promised contribution is standardization. The paper deserves referee time—the topic is real and the AS idea is worth pursuing—but it needs a major revision where the authors recompute or re-source the table entries, reconcile the baselines, and either explain the accuracy losses or drop them. As it stands, I wouldn't cite the numbers, and I'd read the prose with the tables mentally crossed out.\n\nRecommendation: send to peer review, but expect heavy revision.","headline":"Useful survey of efficiency-degradation attacks on ViTs, but the central unified comparison is internally inconsistent and the tables cannot be trusted as published.","tokens_in":10362,"tokens_out":3100,"would_cite":false,"duration_ms":28657,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This survey argues that input-adaptive token pruning in Vision Transformers creates a compute pathway that adversarial patches and perturbations can steer toward near-dense cost, and standardizes cross-paper comparison with an Attack…","keywords":["adversarial efficiency degradation","vision transformers","token pruning","input-adaptive inference","SlowFormer","DeSparsify","Attack Success metric","GFLOPs inflation"],"falsifier":"Inspect the two original papers' reported GFLOPs for A-ViT on ViT-Small or DeiT-Small under no attack: if the true baseline is neither 3.70 nor 0.87 GFLOPs, or if DeiT-S and ViT-S differ measurably in FLOPs beyond rounding, the cross-paper comparison loses its basis.","tokens_in":9352,"feed_emoji":"🎯","tokens_out":4525,"duration_ms":40835,"temperature":0.7,"pith_summary":"This survey claims that input-adaptive token pruning in Vision Transformers—where the model decides per image which tokens to keep—creates a new attack surface: adversarial inputs can steer the compute policy toward keeping more tokens, inflating FLOPs and energy without changing predictions. It unifies two attacks, SlowFormer (a universal patch) and DeSparsify (per-image perturbations), across three pruning frameworks, A-ViT, ATS, and AdaViT, and reports consistent GFLOPs inflation. To make cross-paper numbers comparable, the survey introduces an Attack Success metric that measures how much of the clean-model compute savings an attack cancels. The survey also reviews defenses (confidence-based caps and adversarial training) and argues they reduce, but do not yet remove, the attack's leverage. A sympathetic reader would care because efficiency-targeted attacks could break the energy and latency guarantees of ViTs in edge, mobile, and real-time settings.","feed_headline":"Adversarial patches erase token-pruning savings in ViTs","feed_subtitle":"Two attacks inflate GFLOPs across A-ViT, ATS, and AdaViT; current defenses only partially restore efficiency.","key_machinery":"The load-bearing object is the compute policy $\\pi(x)$ that maps an input to kept/removed computation (token masks, halting indices, activated components). In each framework, $\\pi$ is a function of differentiable statistics: A-ViT's halting scores $h_j^l$ accumulate across layers and a token halts when the cumulative score crosses $1-\\tau$; ATS samples survivors from class-token attention weighted by value norms; AdaViT uses Gumbel-Softmax decision networks over blocks/heads/patches. Because these signals are smooth functions of pixels, an adversary can push their distributions via projected gradient steps (DeSparsify) or a learned universal patch (SlowFormer) to delay halting, flatten attention, or force masks to 'activate,' restoring near-dense compute. The survey's standardized Attack Success metric $\\text{AS} = (F_{\\text{attack}} - F_{\\min})/(F_{\\max} - F_{\\min})$ quantifies how much of the clean-model savings is cancelled.","core_discovery":"The paper's central claim is that input-adaptive inference methods such as token pruning compute a policy $\\pi(x)$ from differentiable, image-conditioned statistics—attention maps, halting scores, mask logits—and that this policy is as steerable from pixel space as a decision boundary is in conventional adversarial attacks. SlowFormer and DeSparsify both exploit this: they optimize a patch or perturbation to shift the policy into a 'keep-more, exit-later' regime, pushing computational cost back toward the dense baseline while preserving the label. The unified tables show this effect across A-ViT, ATS, and AdaViT, with Attack Success rates up to 100% on A-ViT, and the paper concludes that efficiency robustness does not currently come with guarantees: the best defenses reduce Attack Success substantially (e.g., from 100% to 34%–37%) but leave residual compute inflation.","pith_inferences":["The tables' internal inconsistency—A-ViT no-attack GFLOPs of 3.70 in Table 2 versus 0.87 in Table 3—suggests the transcribed baselines may come from different experimental setups, and any reader wanting to rely on the comparative ranking should re-pull the numbers from the original papers.","The AS normalization $F_{\\max} - F_{\\min}$ means that a framework with higher clean FLOPs will score lower AS for the same absolute FLOPs added, so AS is a relative measure, not an absolute robustness gauge.","A natural next step, not taken in the survey, is to test whether the same attacks transfer to modern token-merging methods (e.g., ToMe) or to black-box settings with randomized pruning policies.","A defense that caps per-block active tokens could be combined with injected randomness to make it harder to reverse-engineer, since the survey notes that handcrafted thresholds are easy to attack."],"forward_implications":["Token-pruned ViTs deployed on edge devices carry a security risk: a universal physical patch could erase a large fraction of the intended FLOPs savings without perceptible changes.","Because the compute policy is differentiable, defenses must either bound the policy's output (confidence-based caps, token retention limits) or desensitize it to pixel perturbations (adversarial training), both shown to cut Attack Success substantially.","The Attack Success metric enables meaningful comparison of attacks across papers, provided the dense baseline and the pruned baseline are reported consistently.","Efficiency attacks do not transfer well across pruning frameworks, so a single input will not degrade all ViTs equally.","FLOPs-only reporting understates the impact; DeSparsify is reported to raise energy by about 72% and memory by about 37% under attack."],"supporting_citations":[{"why":"Supplies the Vision Transformer architecture and the quadratic token-cost scaling that motivates token pruning.","marker":"[2]"},{"why":"Defines the ATS framework whose class-attention-based token sampling is one of the three attack targets.","marker":"[3]"},{"why":"Defines AdaViT, whose per-block decision networks over tokens, heads, and blocks are attacked by both SlowFormer and DeSparsify.","marker":"[4]"},{"why":"Defines A-ViT, the halting-score token-pruning method that shows the largest Attack Success in the comparison.","marker":"[5]"},{"why":"The SlowFormer universal-patch attack, one of the two attacks unified in the survey.","marker":"[6]"},{"why":"The DeSparsify per-image perturbation attack, the other representative attack compared in the survey.","marker":"[19]"}],"fun_headline_variants":["Adversarial attacks inflate compute in adaptive ViTs","Token pruning exploited: efficiency gains erased by patches","SlowFormer and DeSparsify: attacks that boost GFLOPs, not errors","ViT efficiency attacks: keep-more, exit-later strategy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that numeric results from two independent papers—SlowFormer on ViT-Small and DeSparsify on DeiT-Small—can be transcribed and compared directly, treating the two architectures as equivalent; the survey's own tables contradict this by reporting 3.70 GFLOPs for A-ViT no-attack in one table and 0.87 in another.","fun_headline_variants_meta":{"raw":{"variants":["Adversarial attacks inflate compute in adaptive ViTs","Token pruning exploited: efficiency gains erased by patches","SlowFormer and DeSparsify: attacks that boost GFLOPs, not errors","ViT efficiency attacks: keep-more, exit-later strategy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000291,"raw_usage":{"total_tokens":1700,"prompt_tokens":948,"completion_tokens":752,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":564,"completion_tokens_details":{"reasoning_tokens":678}},"tokens_in":564,"tokens_out":752,"duration_ms":8415,"temperature":1.0,"reasoning_tokens":678,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T17:56:01.056733+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Inspect the two original papers' reported GFLOPs for A-ViT on ViT-Small or DeiT-Small under no attack: if the true baseline is neither 3.70 nor 0.87 GFLOPs, or if DeiT-S and ViT-S differ measurably in FLOPs beyond rounding, the cross-paper comparison loses its basis.","supporting_citations":[{"cited_title":"Adaptive token sampling for efficient vision transformers,","cited_arxiv_id":null,"evidence_quote":"Defines the ATS framework whose class-attention-based token sampling is one of the three attack targets."},{"cited_title":"Adavit: Adaptive vision transformers for efficient image recognition,","cited_arxiv_id":null,"evidence_quote":"Defines AdaViT, whose per-block decision networks over tokens, heads, and blocks are attacked by both SlowFormer and DeSparsify."},{"cited_title":"A-vit: Adaptive tokens for efficient vision transformer,","cited_arxiv_id":null,"evidence_quote":"Defines A-ViT, the halting-score token-pruning method that shows the largest Attack Success in the comparison."},{"cited_title":"Slowformer: Adversarial attack on compute and energy consumption of efficient vision trans- formers,","cited_arxiv_id":null,"evidence_quote":"The SlowFormer universal-patch attack, one of the two attacks unified in the survey."},{"cited_title":"Desparsify: Adversarial attack against token sparsification mechanisms,","cited_arxiv_id":null,"evidence_quote":"The DeSparsify per-image perturbation attack, the other representative attack compared in the survey."}],"review_version":1}