Pith. sign in

REVIEW 4 major objections 6 minor 60 references

TESSER: Transfer-Enhancing Adversarial Attacks from Vision Transformers via Spectral and Semantic Regularization

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that a transfer attack from Vision Transformers can be made substantially more effective by scaling gradients on semantically salient tokens and suppressing high-frequency perturbation noise, reporting gains over the…

desk verdict A sensible attack recipe with broad evaluation, but the headline ViT margin is inflated by a table-averaging error; corrected margin about 4.6 points instead of 7.2. read the letter →

arxiv 2505.19613 v2 pith:OZAZPDIB submitted 2025-05-26 cs.CV

Amira Guesmi , Bassem Ouni , Muhammad Shafique This is my paper · ORCID
classification cs.CV
keywords adversarialtransferabilityvisiontransformersblack-boxattackstokenimportancegradientregularizationspectralsmoothnessrobustnessImageNet
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

TESSER is an adversarial attack for the black-box transfer setting: perturb an image using a Vision Transformer surrogate and hope it fools models the attacker never sees. The paper claims transfer fails in part because updates treat all tokens equally and carry high-frequency noise, and that both causes can be fixed in one pipeline. Feature-Sensitive Gradient Scaling reweights gradients by token activation norm so the attack focuses on salient regions, while Spectral Smoothness Regularization applies a differentiable Gaussian blur each iteration to keep perturbations low-frequency. On ImageNet with four ViT surrogates and eight CNN/ViT targets, TESSER reports attack success rates of 86.88% on ViTs, 74.4% on CNNs, and 53.55% on adversarially trained CNNs, beating Adaptive Token Tuning by +7.2% and +10.9% on those two groups. If these numbers hold, transfer-based black-box attacks from transformers are considerably stronger than prior methods suggested, and current adversarial defenses remain more fragile to cross-architecture attacks than their white-box evaluations imply.

What carries the argument

The two load-bearing mechanisms are FSGS and SSR. FSGS's core identity is the per-token scaling factor $s_i^{(l)}=\gamma_{\text{base}}+\lambda[(1-\beta(l))\hat{\alpha}_i+\beta(l)(1-\hat{\alpha}_i)]$, where $\hat{\alpha}_i$ is the min-max normalized $\ell^2$ norm of token $i$'s embedding and $\beta(l)$ marks early layers; it reweights backward gradients through Attention, QKV, and MLP modules via hooks. SSR applies the Gaussian blur operator $G_\sigma(\cdot)$ to $x+\delta$ at each optimization step, acting as a differentiable low-pass filter on the perturbation itself. A third component, module-wise weakening with factors $\omega^{(m)}$ and attention truncation beyond layer $l_{\text{cut}}$, suppresses unstable deep-layer gradients before FSGS refines them. Together they convert the attack's update direction toward semantically salient, low-frequency content.

What would settle it

Run the same transfer setup on a held-out set and compare FSGS against a control that multiplies each token's gradient by a random permutation of the same scaling scores; if the control matches TESSER's attack success rate, token-norm ordering is not what drives the gain. A second check: compute per-token cosine similarity between surrogate ViT gradients and target CNN gradients; if high-norm tokens are not systematically better aligned than low-norm tokens, Assumption 1 is false and FSGS's mechanism is unsupported.

Watch

Extended reading notes

Core claim

The paper's central claim is that adversarial transfer from Vision Transformers is improved by treating two properties jointly: semantic selectivity and spectral smoothness. It introduces Feature-Sensitive Gradient Scaling, which reads token embeddings at each block, normalizes activation norms to scores $\hat{\alpha}_i$, and scales gradients for Attention, QKV, and MLP modules by $s_i^{(l)}=\gamma_{\text{base}}+\lambda[(1-\beta(l))\hat{\alpha}_i+\beta(l)(1-\hat{\alpha}_i)]$, amplifying salient tokens in deep layers and suppressing them in early layers. It then applies Spectral Smoothness Regularization, a differentiable Gaussian blur $G_\sigma$ to the adversarial input at every PGD iteration, biasing the perturbation toward low frequencies. On a 1,000-image ImageNet benchmark with four ViT surrogates and eight targets, TESSER reports average attack success rates of 86.88% on ViTs, 74.4% on CNNs, and 53.55% on adversarially trained CNNs, improving on ATT by +7.2%, +10.9%, and large margins on defenses; frequency analysis shows a reduction in high-frequency energy.

Load-bearing premise

The load-bearing premise is that image regions the ViT finds salient, judged by high activation norms, are also the regions whose gradients agree across ViT and CNN targets, so amplifying those gradients helps transfer; the paper asserts this alignment but does not measure it directly.

Editorial extensions

If this is right

  • Cross-architecture transfer attacks from ViTs become strong enough that adversarially trained CNNs are fooled at 53.55% attack success rate, so evaluations that ignore transfer underestimate the practical risk from black-box attackers.
  • The reported ViT-to-CNN gains (+10.9% over ATT) change the expected difficulty ranking of black-box targets: CNNs and hybrid models become easier relative to other ViTs.
  • The module ablation implies that Attention-path gradients contribute the most to transferability, but the best results require joint FSGS scaling of Attention, QKV, and MLP gradients.
  • The $\sigma$ ablation shows a tunable trade-off: stronger spectral smoothing raises CNN and defended-CNN attack success while slightly lowering ViT-to-ViT success, giving practitioners a frequency knob matched to the target family.
  • TESSER's combination with PatchOut indicates that semantic and spectral regularization remain effective under stochastic input transformations, so the method composes with input-diversity attacks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An editorial extension: the paper could have measured per-token cross-model gradient alignment directly; if high-activation tokens are only aligned within the same architecture family, the CNN gains would not persist on more heterogeneous targets.
  • The fixed Gaussian blur suggests a tunable frequency budget; letting $\sigma$ adapt per target family or learning the low-pass filter is a natural next step the paper does not explore.
  • Because FSGS and SSR modify gradient flow rather than the loss, they should compose with ensemble attacks and feature-level attacks; the paper only demonstrates composition with input diversity.
  • A stronger test would compare FSGS against random token-scaling controls under identical budgets; the paper's ablations compare against turning the modules off, not against scrambling which tokens are amplified.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes TESSER, a transfer-based adversarial attack generated from Vision Transformer (ViT) surrogates, with two main components: Feature-Sensitive Gradient Scaling (FSGS), which reweights token gradients using activation norms, and Spectral Smoothness Regularization (SSR), which applies a differentiable Gaussian blur during optimization. The method also includes module-wise gradient weakening and attention truncation. Experiments on ImageNet with four ViT surrogates and twelve target models report higher attack success rates (ASR) than several baselines, including ATT, especially on CNNs and defended CNNs. The paper also provides ablations, Grad-CAM visualizations, frequency-domain analysis, computational cost comparisons, and additional experiments on robust ViTs, targeted attacks, and Vision Mamba models.

Significance. If the reported margins were accurate, TESSER would be a practically useful contribution to the adversarial-transferability literature. The central design idea, combining token-norm-based semantic reweighting with spectral smoothing, is plausible, and the experimental scope is broad: twelve architectures, multiple ablations, robustness to defenses, and an input-diversity combination that demonstrates orthogonality to PatchOut. However, the headline quantitative claims are inflated by an inconsistent aggregation convention in Table 1, the theoretical appendix does not prove its main theorem, and the paper provides neither code nor error bars. After correcting the aggregation, the ViT margin over ATT is roughly 4.6 percentage points rather than 7.2, which is still a positive result but much less dramatic. The empirical contribution is therefore potentially publishable, but the revision must be substantive rather than cosmetic.

major comments (4)
  1. [Table 1, Section 4.2, Abstract] The Avgbb column is computed with an inconsistent convention. For every baseline row, Avgbb equals the mean of the seven non-source columns; for TESSER rows, it equals the mean of all eight columns including the white-box source. For example, in the ViT-B/16 surrogate row, TESSER's printed 83.2 is (100.0+61.7+94.0+68.3+92.5+85.6+72.2+91.4)/8, while the true black-box mean is 565.7/7 = 80.8. The same pattern appears for PiT-B, CaiT-S/24, and Visformer-S. Pooling all 28 ViT transfer pairs gives TESSER about 85.0% versus ATT about 80.4%, a margin of approximately 4.6 percentage points, not the +7.2 points claimed in the Abstract and Section 4.2. The '86.88%' average in Section 4.2 is the mean of the inflated Avgbb values. The CNN margin of +10.9 points is also unsupported: from Table 3, the pooled ATT-to-TESSER difference is about 8.0 points, and no single surrogate row reaches 10.9 points. All aggregate numbers must be recomputed with a single aggregation convention.
  2. [Appendix A, Assumption 1 and Theorem 1] Theorem 1 is not proven as stated. Assumption 1 asserts exactly the cross-model token-gradient alignment that FSGS requires, and the proof sketch does not derive it. The final step, 'by Jensen's inequality over positively weighted aligned vectors', is not a valid derivation of the claimed inequality from the stated assumptions. The cited works concern within-model saliency, not alignment between surrogate and target gradients, and Section 6 concedes that the theoretical link 'lacks formal backing'. The theorem should either be proved under explicit conditions or removed and reframed as a conjecture; as written, it overstates the theoretical contribution.
  3. [Section 4.1, Table 6] The method has many per-module and per-surrogate hyperparameters (gamma_base, lambda_attn, lambda_qkv, lambda_mlp, omega_attn, omega_qkv, omega_mlp, l_cut, sigma_SSR, and the early-layer set), and Table 6 reports different values for each surrogate. The text states that these are 'tuned per model' but does not describe the selection procedure or provide sensitivity analysis for most of them; Appendix C.1 covers sigma, and Appendix D covers a few on/off ablations. With only four surrogates, this creates a post-hoc selection risk: the reported gains may reflect per-surrogate tuning rather than a fixed algorithm. Please provide a principled selection rule, cross-surrogate sensitivity, or a single default configuration with a sensitivity range.
  4. [Experiments and reproducibility] No code, seeds, or error bars are provided for the reported ASR values. Given that the corrected ViT margin over ATT is about 4.6 percentage points, the reader cannot assess whether the improvement is within run-to-run noise for a 1,000-image subset. Please release code or, at minimum, report standard deviations over multiple random subsets or runs, and describe the exact image-selection and hyperparameter-selection procedures.
minor comments (6)
  1. [Abstract] There is a typo: 'succes rate' should be 'success rate'.
  2. [Section 4.5 and Abstract] The claimed '12% reduction in high-frequency energy' does not follow directly from the reported ranges (ATT 53–56%, FSGS+SSR 45–47%), and Appendix D reports a 6–16% reduction across examples. Please specify exactly how the 12% figure is computed and over which set of samples.
  3. [Appendix E] The sentence 'The table will be included in the revised version' appears after Table 15; this placeholder text should be removed.
  4. [References] Several references are duplicated: [6] and [7] are the same paper, as are [29] and [30], and [44] and [45]. Please consolidate them.
  5. [Algorithm 1] The input list includes 'SSR loss function L_SSR', but L_SSR is never used in the algorithm body; either remove it or define the regularization loss explicitly.
  6. [Table 2 caption] The caption contains a typo: 'The best results are highlighted inbold).' should be corrected.

Circularity Check

2 steps flagged · score 3.0 of 10

The empirical core is self-contained; circularity is confined to the appendix theorem (Assumption 1 restated) and a tautological spectral-energy confirmation.

  1. self definitional [Appendix A, Assumption 1 and Theorem 1]
    "Assumption 1. Gradients at semantically important tokens exhibit higher cross-model alignment... Under Assumption 1, the cosine alignment between G_FSGS and the target model's gradient G' satisfies: cosθ(G_FSGS, G')>cosθ(G, G')... Since E_i∈S_sem[Align_i]>E_i∈S_bg[Align_i], amplifying contributions from S_sem increases the expected alignment between G_FSGS and G'. Therefore: cosθ(G_FSGS, G')>cosθ(G, G')(by Jensen's inequality over positively weighted aligned vectors)"

    Theorem 1 does not derive alignment; it restates Assumption 1 after postulating that high activation-norm tokens are the semantic tokens. The text immediately before says 'Tokens with high α_i (assumed to lie in S_sem) receive larger gradients', so the FSGS scaling is defined to upweight exactly the set assumed to have higher alignment. The claimed inequality is then a weighted-mean restatement of Assumption 1, not independent evidence. Section 6 concedes this: 'the underlying relationship between gradient sensitivity and transferability still lacks formal theoretical backing.' This is a circular theoretical justification, but the empirical ASR comparison does not depend on it.

  2. other [Section 3.3 and Section 4.5]
    "SSR operates by applying a differentiable Gaussian blur to the adversarial input at each iteration, effectively enforcing a low-pass filter on the evolving perturbation... When combined with SSR, the high-frequency ratio drops further (to ∼45–47%), indicating smoother and more transferable perturbations. This confirms that SSR encourages low-frequency perturbation structure."

    The reported reduction in high-frequency energy is mechanically imposed by the Gaussian blur/low-pass filter that defines SSR, so the frequency-domain measurement confirms the definition of the regularizer rather than independently validating its transferability benefit. This framing is tautological, though the attack-success ablations in Appendix C.1 (Table 8) provide independent empirical support for SSR's effect on ASR. Therefore this is a minor self-confirmation, not the load-bearing claim.

full rationale

TESSER's central empirical claim—higher transfer ASR than ATT across ViT, CNN, and defended targets—rests on held-out target-model evaluations and is not circular; the numbers are external measurements against baselines. The paper does not fit parameters to the targets it then claims to predict; the surrogate/target separation is clean. Self-citations ([14]-[16]) are prior adversarial-attack work and are not load-bearing for the new method. The only genuine circular steps are (1) the Appendix A 'Theorem 1', whose conclusion is Assumption 1 plus the assumption that high activation-norm tokens are the semantic tokens, effectively a restatement rather than a derivation, and (2) the Section 4.5 high-frequency-energy measurement, which is a direct consequence of applying a Gaussian blur. Both are ancillary and the paper itself acknowledges the missing formal link in Section 6. Separately, the Table 1 Avgbb column includes the white-box source column for the TESSER rows while baseline rows average only black-box targets; this changes the reported +7.2% ViT margin to roughly +4.6 points after correction. That is an empirical reporting inconsistency, not a circularity, but it should be corrected and weighed in the overall assessment of the headline quantitative claim. On circularity alone, the derivation is largely self-contained and the score is moderate.

Assumptions & free parameters 10 free parameters · 5 assumptions · 0 invented entities

The central claim depends on roughly ten hand-tuned hyperparameters per surrogate model and on three unproven assumptions about semantic saliency and frequency transfer. No new entities are introduced. The theory in Appendix A is circular, resting on Assumption 1, which is the conclusion in disguise.

free parameters (10)
  • gamma_base = 0.5
    Minimum gradient scaling factor, set by hand (Section 3.2, Table 6).
  • lambda_attn = 0.4 to 0.5 per model
    FSGS scaling strength for attention gradients, tuned per surrogate (Table 6).
  • lambda_qkv = 0.5
    FSGS scaling strength for QKV gradients, tuned per surrogate (Table 6).
  • lambda_mlp = 0.55 to 0.65 per model
    FSGS scaling strength for MLP gradients, tuned per surrogate (Table 6).
  • omega_attn = 0.25 to 0.45 per model
    Module-wise weakening factor for attention, tuned per surrogate (Table 6).
  • omega_qkv = 0.5 to 1.0 per model
    Module-wise weakening factor for QKV, tuned per surrogate (Table 6).
  • omega_mlp = 0.5 to 0.7 per model
    Module-wise weakening factor for MLP, tuned per surrogate (Table 6).
  • l_cut = 4 to 10 per model
    Layer threshold for selective attention truncation, tuned per surrogate (Table 6).
  • sigma_SSR = 0.5 or 0.7 per model
    Gaussian blur standard deviation in SSR, tuned per surrogate (Table 6); ablation shows strong effect on CNN transfer.
  • early_layer_set_k = not specified
    Algorithm 1 takes an early-layer set E as input, but the paper never states the value of k; a latent free parameter.
assumptions (5)
  • domain assumption Gradients at semantically important tokens have higher cross-model alignment (Assumption 1)
    Appendix A: the FSGS theorem assumes this and offers no proof; it is the core premise for why norm-based scaling helps transfer.
  • domain assumption Token activation norm is a valid proxy for semantic saliency
    Section 3.2 cites NLP and ViT explainability works [22,46,29]; no evidence that it holds for adversarial transfer.
  • domain assumption Low-frequency perturbations transfer better across architectures
    Section 3.3 assumes high-frequency noise overfits, citing [40,51]; plausible but not established for ViT-to-CNN transfer.
  • domain assumption Early ViT layers encode less transferable, low-level features, so gradients should be scaled by (1-alpha)
    Section 3.2 relies on [32,2,21]; a heuristic choice without verification in this attack setting.
  • standard math Differentiable Gaussian blur and min-max normalization are standard mathematical operations
    Used in SSR and FSGS; no controversy.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TESSER: Transfer-Enhancing Adversarial Attacks from Vision Transformers via Spectral and Semantic Regularization." pith.science (2026). https://pith.science/paper/OZAZPDIB

@misc{pith2026250519613,
  author       = {Pith},
  title        = {Pith review of: TESSER: Transfer-Enhancing Adversarial Attacks from Vision Transformers via Spectral and Semantic Regularization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OZAZPDIB}},
  note         = {Machine review of arXiv:2505.19613}
}
read the original abstract

Adversarial transferability remains a critical challenge in evaluating the robustness of deep neural networks. In security-critical applications, transferability enables black-box attacks without access to model internals, making it a key concern for real-world adversarial threat assessment. While Vision Transformers (ViTs) have demonstrated strong adversarial performance, existing attacks often fail to transfer effectively across architectures, especially from ViTs to Convolutional Neural Networks (CNNs) or hybrid models. In this paper, we introduce \textbf{TESSER} -- a novel adversarial attack framework that enhances transferability via two key strategies: (1) \textit{Feature-Sensitive Gradient Scaling (FSGS)}, which modulates gradients based on token-wise importance derived from intermediate feature activations, and (2) \textit{Spectral Smoothness Regularization (SSR)}, which suppresses high-frequency noise in perturbations using a differentiable Gaussian prior. These components work in tandem to generate perturbations that are both semantically meaningful and spectrally smooth. Extensive experiments on ImageNet across 12 diverse architectures demonstrate that TESSER achieves +10.9\% higher attack succes rate (ASR) on CNNs and +7.2\% on ViTs compared to the state-of-the-art Adaptive Token Tuning (ATT) method. Moreover, TESSER significantly improves robustness against defended models, achieving 53.55\% ASR on adversarially trained CNNs. Qualitative analysis shows strong alignment between TESSER's perturbations and salient visual regions identified via Grad-CAM, while frequency-domain analysis reveals a 12\% reduction in high-frequency energy, confirming the effectiveness of spectral regularization.

Figures

Figures reproduced from arXiv: 2505.19613 by the authors.

Figure 1
Figure 1. Overview of the TESSER attack framework. At each iteration, an adversarial perturbation δ t is applied to the input image and smoothed via differentiable Gaussian blur Gσ(·) to enforce spectral smoothness (SSR). The perturbed input is passed through the transformer, where token embeddings Zl from each layer are used to compute token-wise importance scores αˆ, which in turn define gradient scaling masks S. During bac… view at source ↗
Figure 2
Figure 2. Qualitative and frequency-domain comparison between ATT and our method (FSGS and FSGS + SSR). Each row shows clean images, adversarial examples when using FSGS, Grad￾CAM (guided by the adversarial label) overlays, and FFT log-magnitude spectra when using SSR. Our method produces perturbations that better align with semantically relevant regions and exhibit smoother frequency profiles. Further results and analysis ar… view at source ↗
Figure 3
Figure 3. Qualitative comparison between ATT and our TESSER method (FSGS+SSR). Each [PITH_FULL_IMAGE:figures/full_fig_p022_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

60 extracted references · 32 canonical work pages

  1. [1]

    Quantifying attention flow in transformers.arXiv preprint arXiv:2005.00928, 2020

    Samira Abnar and Willem Zuidema. Quantifying attention flow in transformers.arXiv preprint arXiv:2005.00928, 2020

    arXiv 2005

  2. [2]

    Understanding robustness of transformers for image classification

    Srinadh Bhojanapalli, Ayan Chakrabarti, Daniel Glasner, Daliang Li, Thomas Unterthiner, and Andreas Veit. Understanding robustness of transformers for image classification. InProceedings of the IEEE/CVF international conference on computer vision, pages 10231–10241, 2021

    2021

  3. [3]

    End-to-end object detection with transformers

    Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. InEuropean conference on computer vision, pages 213–229. Springer, 2020

    2020

  4. [4]

    Visformer: The vision-friendly transformer

    Zhengsu Chen, Lingxi Xie, Jianwei Niu, Xuefeng Liu, Longhui Wei, and Qi Tian. Visformer: The vision-friendly transformer. InProceedings of the IEEE/CVF international conference on computer vision, pages 589–598, 2021

    2021

  5. [5]

    A light recipe to train robust vision transformers

    Edoardo Debenedetti, Vikash Sehwag, and Prateek Mittal. A light recipe to train robust vision transformers. In2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), pages 225–253. IEEE, 2023

    work page 2023

  6. [6]

    Boosting adversarial attacks with momentum

    Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boosting adversarial attacks with momentum. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 9185–9193, 2018

    2018

  7. [7]

    Boosting adversarial attacks with momentum

    Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boosting adversarial attacks with momentum. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 9185–9193, 2018. 11

    work page 2018

  8. [9]

    An image is worth 16x16 words: Transformers for image recognition at scale, 2021

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale, 2021. URL https://arxiv.org/abs/2010.11929

    arXiv 2021

Show all 60 references
  1. [10]

    Convit: Improving vision transformers with soft convolutional inductive biases

    Stéphane d’Ascoli, Hugo Touvron, Matthew L Leavitt, Ari S Morcos, Giulio Biroli, and Levent Sagun. Convit: Improving vision transformers with soft convolutional inductive biases. InInternational conference on machine learning, pages 2286–2296. PMLR, 2021

    2021

  2. [11]

    Fda: Feature disruptive attack

    Aditya Ganeshan, Vivek BS, and R Venkatesh Babu. Fda: Feature disruptive attack. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 8069–8079, 2019

    2019

  3. [12]

    Goodfellow, Jonathon Shlens, and Christian Szegedy

    Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. CoRR, abs/1412.6572, 2014. URLhttps://api.semanticscholar.org/CorpusID:6706414

    2014 arXiv

  4. [13]

    Levit: a vision transformer in convnet’s clothing for faster inference

    Benjamin Graham, Alaaeldin El-Nouby, Hugo Touvron, Pierre Stock, Armand Joulin, Hervé Jégou, and Matthijs Douze. Levit: a vision transformer in convnet’s clothing for faster inference. InProceedings of the IEEE/CVF international conference on computer vision, pages 12259–12269, 2021

    2021

  5. [14]

    Amira Guesmi, Muhammad Abdullah Hanif, Bassem Ouni, and Muhammad Shafique. Physical adversarial attacks for camera-based smart systems: Current trends, categorization, applications, research challenges, and future outlook.IEEE Access, 11:109617–109668, 2023. doi: 10.1109/ACCES...

    2023

  6. [15]

    Dap: A dynamic adversarial patch for evading person detectors

    Amira Guesmi, Ruitian Ding, Muhammad Abdullah Hanif, Ihsen Alouani, and Muhammad Shafique. Dap: A dynamic adversarial patch for evading person detectors. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 24595–24604, June 2024

    2024

  7. [16]

    Ssap: A shape-sensitive adversarial patch for comprehensive disruption of monocular depth estimation in autonomous navigation applications

    Amira Guesmi, Muhammad Abdullah Hanif, Ihsen Alouani, Bassem Ouni, and Muhammad Shafique. Ssap: A shape-sensitive adversarial patch for comprehensive disruption of monocular depth estimation in autonomous navigation applications. In2024 IEEE/RSJ International Conference on Int...

    2024

  8. [17]

    Transformer in transformer

    Kai Han, An Xiao, Enhua Wu, Jianyuan Guo, Chunjing Xu, and Yunhe Wang. Transformer in transformer. Advances in neural information processing systems, 34:15908–15919, 2021

    2021

  9. [18]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016

    2016

  10. [19]

    Rethinking spatial dimensions of vision transformers

    Byeongho Heo, Sangdoo Yun, Dongyoon Han, Sanghyuk Chun, Junsuk Choe, and Seong Joon Oh. Rethinking spatial dimensions of vision transformers. InProceedings of the IEEE/CVF international conference on computer vision, pages 11936–11945, 2021

    2021

  11. [20]

    Enhancing adversarial example transferability with an intermediate level attack

    Qian Huang, Isay Katsman, Horace He, Zeqi Gu, Serge Belongie, and Ser-Nam Lim. Enhancing adversarial example transferability with an intermediate level attack. InProceedings of the IEEE/CVF international conference on computer vision, pages 4733–4742, 2019

    2019

  12. [21]

    Exploring adversarial robustness of vision transformers in the spectral perspective

    Gihyun Kim, Juyeop Kim, and Jong-Seok Lee. Exploring adversarial robustness of vision transformers in the spectral perspective. InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 3976–3985, 2024

    2024

  13. [22]

    Attention is not only a weight: Analyzing transformers with vector norms.arXiv preprint arXiv:2004.10102, 2020

    Goro Kobayashi, Tatsuki Kuribayashi, Sho Yokoi, and Kentaro Inui. Attention is not only a weight: Analyzing transformers with vector norms.arXiv preprint arXiv:2004.10102, 2020

    2004 arXiv

  14. [23]

    Leveraging visual question answering for image-caption ranking

    Xiao Lin and Devi Parikh. Leveraging visual question answering for image-caption ranking. InComputer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II 14, pages 261–277. Springer, 2016

    2016

  15. [24]

    Beyond attentive tokens: Incorporating token importance and diversity for efficient vision transformers

    Sifan Long, Zhen Zhao, Jimin Pi, Shengsheng Wang, and Jingdong Wang. Beyond attentive tokens: Incorporating token importance and diversity for efficient vision transformers. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10334–10343, 2023

    2023

  16. [25]

    Unified transformer tracker for object tracking

    Fan Ma, Mike Zheng Shou, Linchao Zhu, Haoqi Fan, Yilei Xu, Yi Yang, and Zhicheng Yan. Unified transformer tracker for object tracking. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8781–8790, 2022. 12

    2022

  17. [26]

    Towards deep learning models resistant to adversarial attacks.arXiv preprint arXiv:1706.06083, 2017

    Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks.arXiv preprint arXiv:1706.06083, 2017

    2017 arXiv

  18. [27]

    Boosting the transferability of adversarial attack on vision transformer with adaptive token tuning.Advances in Neural Information Processing Systems, 37: 20887–20918, 2024

    Di Ming, Peng Ren, Yunlong Wang, and Xin Feng. Boosting the transferability of adversarial attack on vision transformer with adaptive token tuning.Advances in Neural Information Processing Systems, 37: 20887–20918, 2024

    2024

  19. [28]

    When adversarial training meets vision transformers: Recipes from training to architecture.Advances in Neural Information Processing Systems, 35:18599–18611, 2022

    Yichuan Mo, Dongxian Wu, Yifei Wang, Yiwen Guo, and Yisen Wang. When adversarial training meets vision transformers: Recipes from training to architecture.Advances in Neural Information Processing Systems, 35:18599–18611, 2022

    2022

  20. [30]

    Globenc: Quantifying global token attribution by incorporating the whole encoder layer in transformers.arXiv preprint arXiv:2205.03286, 2022

    Ali Modarressi, Mohsen Fayyaz, Yadollah Yaghoobzadeh, and Mohammad Taher Pilehvar. Globenc: Quantifying global token attribution by incorporating the whole encoder layer in transformers.arXiv preprint arXiv:2205.03286, 2022

    2022 arXiv

  21. [31]

    On improving adversarial transferability of vision transformers.arXiv preprint arXiv:2106.04169, 2021

    Muzammal Naseer, Kanchana Ranasinghe, Salman Khan, Fahad Shahbaz Khan, and Fatih Porikli. On improving adversarial transferability of vision transformers.arXiv preprint arXiv:2106.04169, 2021

    2021 arXiv

  22. [32]

    Do vision transformers see like convolutional neural networks?Advances in neural information processing systems, 34:12116–12128, 2021

    Maithra Raghu, Thomas Unterthiner, Simon Kornblith, Chiyuan Zhang, and Alexey Dosovitskiy. Do vision transformers see like convolutional neural networks?Advances in neural information processing systems, 34:12116–12128, 2021

    2021

  23. [33]

    Improving adversarial transferability on vision transformers via forward propagation refinement, 2025

    Yuchen Ren, Zhengyu Zhao, Chenhao Lin, Bo Yang, Lu Zhou, Zhe Liu, and Chao Shen. Improving adversarial transferability on vision transformers via forward propagation refinement, 2025. URL https: //arxiv.org/abs/2503.15404

    2025 arXiv

  24. [34]

    Imagenet large scale visual recognition challenge

    Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115:211–252, 2015

    2015

  25. [35]

    Grad-cam: Visual explanations from deep networks via gradient-based localization

    Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on computer vision, pages 618–626, 2017

    2017

  26. [36]

    Rethinking the inception architecture for computer vision

    Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 2818–2826, 2016

    2016

  27. [37]

    Inception-v4, inception-resnet and the impact of residual connections on learning

    Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke, and Alexander Alemi. Inception-v4, inception-resnet and the impact of residual connections on learning. InProceedings of the AAAI conference on artificial intelligence, volume 31, 2017

    2017

  28. [38]

    Training data-efficient image transformers & distillation through attention

    Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-efficient image transformers & distillation through attention. InInternational conference on machine learning, pages 10347–10357. PMLR, 2021

    2021

  29. [39]

    Going deeper with image transformers

    Hugo Touvron, Matthieu Cord, Alexandre Sablayrolles, Gabriel Synnaeve, and Hervé Jégou. Going deeper with image transformers. InProceedings of the IEEE/CVF international conference on computer vision, pages 32–42, 2021

    2021

  30. [40]

    Ro- bustness may be at odds with accuracy.arXiv: Machine Learning, 2018

    Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. Ro- bustness may be at odds with accuracy.arXiv: Machine Learning, 2018. URL https://api. semanticscholar.org/CorpusID:52962648

    2018

  31. [41]

    Enhancing the transferability of adversarial attacks through variance tuning

    Xiaosen Wang and Kun He. Enhancing the transferability of adversarial attacks through variance tuning. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1924–1933, 2021

    1924

  32. [42]

    Feature importance- aware transferable adversarial attacks

    Zhibo Wang, Hengchang Guo, Zhifei Zhang, Wenxin Liu, Zhan Qin, and Kui Ren. Feature importance- aware transferable adversarial attacks. InProceedings of the IEEE/CVF international conference on computer vision, pages 7639–7648, 2021

    2021

  33. [43]

    Towards transferable adversarial attacks on vision transformers

    Zhipeng Wei, Jingjing Chen, Micah Goldblum, Zuxuan Wu, Tom Goldstein, and Yu-Gang Jiang. Towards transferable adversarial attacks on vision transformers. InProceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 2668–2676, 2022. 13

    2022

  34. [45]

    Skip connections matter: On the transferability of adversarial examples generated with resnets.arXiv preprint arXiv:2002.05990, 2020

    Dongxian Wu, Yisen Wang, Shu-Tao Xia, James Bailey, and Xingjun Ma. Skip connections matter: On the transferability of adversarial examples generated with resnets.arXiv preprint arXiv:2002.05990, 2020

    2002 arXiv

  35. [46]

    Token transformation matters: Towards faithful post-hoc explanation for vision transformer

    Junyi Wu, Bin Duan, Weitai Kang, Hao Tang, and Yan Yan. Token transformation matters: Towards faithful post-hoc explanation for vision transformer. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10926–10935, 2024

    2024

  36. [47]

    Rethinking the backward propagation for adversarial transferability.Advances in Neural Information Processing Systems, 36:1905–1922, 2023

    Wang Xiaosen, Kangheng Tong, and Kun He. Rethinking the backward propagation for adversarial transferability.Advances in Neural Information Processing Systems, 36:1905–1922, 2023

    1905

  37. [48]

    Improving transferability of adversarial examples with input diversity

    Cihang Xie, Zhishuai Zhang, Yuyin Zhou, Song Bai, Jianyu Wang, Zhou Ren, and Alan L Yuille. Improving transferability of adversarial examples with input diversity. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2730–2739, 2019

    2019

  38. [49]

    Stochastic variance reduced ensemble adversarial attack for boosting the adversarial transferability

    Yifeng Xiong, Jiadong Lin, Min Zhang, John E Hopcroft, and Kun He. Stochastic variance reduced ensemble adversarial attack for boosting the adversarial transferability. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 14983–14992, 2022

    2022

  39. [50]

    A2: Efficient automated attacker for boosting adversarial training.Advances in Neural Information Processing Systems, 35:22844–22855, 2022

    Zhuoer Xu, Guanghui Zhu, Changhua Meng, Zhenzhe Ying, Weiqiang Wang, Ming Gu, Yihua Huang, et al. A2: Efficient automated attacker for boosting adversarial training.Advances in Neural Information Processing Systems, 35:22844–22855, 2022

    2022

  40. [51]

    A fourier perspective on model robustness in computer vision.Advances in Neural Information Processing Systems, 32, 2019

    Dong Yin, Raphael Gontijo Lopes, Jon Shlens, Ekin Dogus Cubuk, and Justin Gilmer. A fourier perspective on model robustness in computer vision.Advances in Neural Information Processing Systems, 32, 2019

    2019

  41. [52]

    How transferable are features in deep neural networks?Advances in neural information processing systems, 27, 2014

    Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson. How transferable are features in deep neural networks?Advances in neural information processing systems, 27, 2014

    2014

  42. [53]

    Transferable adversarial attacks on vision transformers with token gradient regularization

    Jianping Zhang, Yizhan Huang, Weibin Wu, and Michael R Lyu. Transferable adversarial attacks on vision transformers with token gradient regularization. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16415–16424, 2023

    2023

  43. [54]

    A unified efficient pyramid transformer for semantic segmentation

    Fangrui Zhu, Yi Zhu, Li Zhang, Chongruo Wu, Yanwei Fu, and Mu Li. A unified efficient pyramid transformer for semantic segmentation. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 2667–2677, 2021

    2021

  44. [55]

    Vision mamba: Efficient visual representation learning with bidirectional state space model.arXiv preprint arXiv:2401.09417, 2024

    Lianghui Zhu, Bencheng Liao, Qian Zhang, Xinlong Wang, Wenyu Liu, and Xinggang Wang. Vision mamba: Efficient visual representation learning with bidirectional state space model.arXiv preprint arXiv:2401.09417, 2024. 14 Appendix A Theoretical Justification of Feature-Sensitive ...

    2024 arXiv

  45. [56]

    This captures prior knowledge about the sensitivity of each module

    Module-wise Weakening:The gradient g(l) for each module is first scaled using a module- specific weakening factorω(l) ∈(0,1] (e.g., ω(l) attn, ω(l) qkv, ω(l) mlp). This captures prior knowledge about the sensitivity of each module

  46. [57]

    FSGS promotes perturbation alignment with semantically salient features while suppressing low-level, architecture-specific signals that degrade cross-model transferability

    Layer-wise Modulation:The weakened attention gradient is then further modulated by a layer-specific coefficient τl ∈[0,1] , which reduces the influence of deeper transformer layers: g(l) ←τ l ·(ω (l) ·g (l)) 3.Feature-Sensitive Gradient Scaling (FSGS): A layer-aware gradient m...

  47. [58]

    Apply SSR: x(t) =G σ(x+δ (t−1))

  48. [59]

    Forward pass and compute classification loss: L(t) cls =L(f(x (t)), y)

  49. [60]

    , L}do foreachmodulem∈ {qkv,attn,mlp}do 3.1 Extract token features and gradients: Z(l,m) = [z(l,m) 1 ,

    Backward pass with hooks at QKV , Attention, and MLP modules: foreachblockl∈ {1, . . . , L}do foreachmodulem∈ {qkv,attn,mlp}do 3.1 Extract token features and gradients: Z(l,m) = [z(l,m) 1 , . . . ,z(l,m) T ] G(l,m) = [g(l,m) 1 , . . . ,g(l,m) T ] 3.2 Compute token importance: ...

  50. [61]

    Aggregate gradients across all modules: g(t) = P l,m Aggregate(G(l,m))

  51. [62]

    Momentum update: m(t) =µ·m (t−1) + g(t) ∥g(t)∥1

  52. [63]

    Perturbation update with projection: δ(t) =Clip ϵ(δ(t−1) +η·sign(m (t))) end returnx adv =x+δ (T) 18 Table 6: Model-specific hyperparameter settings used for TESSER. ω(·) denotes the weakening factor for each module, λ· is the FSGS scaling parameter, σ controls the strength of...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.