Pith. sign in

REVIEW 3 major objections 4 minor 66 references

TAPT: Test-Time Adversarial Prompt Tuning for Robust Inference in Vision-Language Models

T0 review · 3 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that test-time tuning of bimodal prompts, driven by multi-view entropy and alignment to ImageNet embedding statistics, lifts CLIP's zero-shot AutoAttack accuracy from an average of about 0.1% to about 49% across 11…

desk verdict TAPT is a novel test-time defense for CLIP, but the evaluation only attacks the undefended model, so the white-box robustness claim is unsupported. read the letter →

arxiv 2411.13136 v1 pith:Y3CHDKXR submitted 2024-11-20 cs.CV

Xin Wang , Kai Chen , Jiaming Zhang , Jingjing Chen , Xingjun Ma This is my paper · ORCID
classification cs.CV
keywords test-timedefenseadversarialrobustnessvision-languagemodelsCLIPprompttuningzero-shotlearningAutoAttackmultimodal
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes Test-Time Adversarial Prompt Tuning (TAPT), a defense that adapts CLIP's prompts for each test image on the fly to resist adversarial perturbations in the visual modality. The central claim is that this unsupervised, task-agnostic procedure raises CLIP's zero-shot robustness under AutoAttack from an average of about 0.1% to about 49% across 11 datasets, while largely preserving clean-image accuracy. TAPT also reports average robustness gains of 36.6 to 38.0 percentage points over existing adversarial prompt tuning baselines. If correct, this would give pre-trained vision-language models a practical, training-free route to robust inference on unseen tasks.

What carries the argument

The machinery is a per-sample optimization loop over learnable prompt tokens appended to CLIP's image and text inputs. Two unsupervised losses drive the update: multi-view entropy, which averages prediction probabilities over the selected low-entropy augmented views and minimizes the entropy of that average, and adversarial-clean embedding alignment, which matches the mean and variance of the test image's intermediate embeddings, layer by layer, to pre-computed statistics of adversarial and clean ImageNet embeddings. The combined objective is $L_{\mathrm{TAPT}} = L_{\mathrm{entropy}} + \alpha L_{\mathrm{adv}} + (1-\alpha)L_{\mathrm{clean}}$ with $\alpha = 0.5$, and the update is a single AdamW step per inference. The image and text encoders stay frozen; the prompt is the only moving part, which lets the defense adapt to each sample while leaving the pre-trained model intact.

What would settle it

Run a white-box attack that computes gradients through the entire TAPT pipeline, including the per-sample prompt update and multi-view selection, and measure AutoAttack-style robustness on the same 11 datasets. If the average robust accuracy falls to near the undefended level (about 0.1%) rather than staying near the reported 48.9% to 53.8%, the central claim fails under the paper's stated white-box threat model.

Watch

Extended reading notes

Core claim

TAPT's central claim is that a prompt adapted to each test image can absorb the distribution shift introduced by adversarial perturbations, restoring alignment between the image embedding and its text embedding. Starting from a robust prompt initialized by adversarial prompt tuning on ImageNet, the method generates 64 augmented views of the test image, keeps the 10% with the lowest entropy in their averaged predictions, and performs one optimization step on the prompt. The step minimizes a combined loss: multi-view entropy of the averaged prediction plus $\ell^1$ alignment of the mean and variance of intermediate embeddings against pre-computed adversarial and clean statistics from ImageNet. On the paper's evaluation, this lifts average AutoAttack accuracy from 0.1% for vanilla CLIP to 48.9% with visual-only prompts and 53.8% with V-L independent prompts on ViT-B/16, while clean accuracy remains at or above the adversarial prompt tuning baselines.

Load-bearing premise

The evaluation assumes attackers craft perturbations against the original CLIP model, not against TAPT's own differentiable tuning process, even though a determined attacker could in principle optimize the perturbation through the full defense.

Editorial extensions

If this is right

  • Zero-shot adversarial robustness becomes a test-time property: the same frozen CLIP can be defended on an unseen dataset with no labels or task-specific tuning.
  • The defense composes with existing adversarial prompt tuning: initializing from an ImageNet-tuned robust prompt is part of what makes a single adaptation step effective.
  • The V-L independent prompt design gives the largest average AutoAttack gains over its APT baseline, suggesting separate visual and textual prompts are easier to optimize jointly than shared ones.
  • Because prompts reset every sample, robustness does not accumulate across samples; the paper reports that longer reset intervals slightly improve ImageNet accuracy but hurt the ten zero-shot datasets and open a poisoning risk.
  • The time overhead is modest enough for per-image adaptation, at 0.095 to 0.166 seconds per image, so the defense is usable in latency-sensitive settings.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the unadapted-attack assumption is the real source of the gains, then an adaptive attacker who differentiates through TAPT's prompt update would likely erode most of the reported margin; that test is absent from the paper.
  • The choice of ImageNet as the public dataset is a transferability bet: the alignment losses compare test embeddings to ImageNet-derived statistics, so datasets far from ImageNet's distribution may see smaller robustness gains than the average suggests.
  • The same mechanism could extend beyond image recognition: replacing image-embedding statistics with joint image-text statistics might defend multimodal attacks, and the per-sample reset policy could be relaxed in trusted settings where poisoning is not a concern.
  • TAPT's reliance on augmented views means the quality of the augmentation distribution matters; a testable extension would measure how robustness changes when augmentations are specialized to the target domain.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes Test-Time Adversarial Prompt Tuning (TAPT), an unsupervised test-time defense for CLIP that optimizes visual and textual prompts per test image by minimizing multi-view entropy and aligning test-embedding statistics with precomputed clean/adversarial statistics from ImageNet. It claims to improve zero-shot adversarial robustness over vanilla CLIP by at least 48.9% against AutoAttack and over APT baselines by at least 36.6%, evaluated on 11 datasets with ViT-B/16 and ViT-B/32 backbones.

Significance. If validated, TAPT would be a significant contribution: it is the first task-agnostic test-time defense for pre-trained VLMs, lifting CLIP's near-zero AutoAttack accuracy to roughly 50% while maintaining clean accuracy. The paper is well-structured, examines three prompt designs, provides ablations on TAPT steps, perturbation budgets, and reset intervals, and reports per-dataset numbers across 11 benchmarks. The method is conceptually simple and requires no downstream labels. However, the central white-box robustness claim is currently not supported by the evaluation, and the comparison to APT baselines may be confounded by an unequal data budget for initialization; these issues are load-bearing and need to be addressed before the significance can be assessed.

major comments (3)
  1. [Section 3.1 / Section 4.2] The white-box threat model in Section 3.1 gives the attacker full knowledge of the target model and has the attacker optimize Eq. (1) directly against that model. The evaluation in Table 1, however, appears to use adversarial examples generated against the undefended CLIP; the attack configuration in Section 4.2 never states that PGD-100, DI, or AutoAttack is run through TAPT's test-time prompt optimization. Since Algorithm 1 is differentiable and the defense includes stochastic augmentations, a white-box attacker could compute gradients through the optimization or use an expectation-over-transformation attack. Without such adaptive attack results, the reported AutoAttack accuracies (e.g., 48.9% for TAPT-V) do not establish robustness under the paper's own white-box threat model; they only measure transferability of attacks from the vanilla CLIP surrogate. This is load-bearing for the central claim and requires new experiments.
  2. [Section 4.2 / Section 4.1] The comparison against APT baselines is potentially unfair. Table 1's caption states that the APT baselines (APT-V, APT-VLJ, APT-VLI) were tuned on ImageNet under a 16-shot setting, while Section 4.1's Implementation Details says TAPT initializes its defensive prompt using APT on ImageNet trained for 100 epochs with a batch size of 32. If the TAPT initialization uses substantially more (or the full) ImageNet data than the 16-shot baselines, the reported average robustness improvement of 36.6% over APT-VLI conflates the benefit of test-time adaptation with a stronger initialization. The authors should state the exact training data budget for the TAPT initialization and either match the baselines' data budget or ablate the initialization effect.
  3. [Tables 1 and 2] All results are single point estimates without error bars or repeated trials. Because TAPT involves stochastic augmentations and per-sample prompt optimization, the reported 'at least' improvements (e.g., 48.9% and 36.6%) could vary across runs. Providing means and standard deviations over multiple seeds is necessary to support the strength of the claim, especially for datasets with smaller margins such as Aircraft in Table 1.
minor comments (4)
  1. [Algorithm 1] Line 6 of Algorithm 1 says 'Select the top τ entropy from A(x) to form Hτ(x)', which is inconsistent with the text's 'top-K views with the lowest entropy' and with the implementation's 'top 10% most confident predictions'. Please define whether τ is the fraction or the number of views and how the threshold is computed.
  2. [Abstract] The phrase 'enhances ... by at least 48.9%' should be 'to at least 48.9%' or 'by 48.8 percentage points', since the vanilla AutoAttack accuracy is 0.1% and the TAPT-V average is 48.9%.
  3. [Section 4.1] The claim of 'zero-shot adversarial robustness' is overstated for ImageNet, since TAPT uses ImageNet-derived statistics and an APT initialization trained on ImageNet; the evaluation on the other 10 datasets is zero-shot, but ImageNet is in-distribution.
  4. [Section 3.2] The combined loss in Eq. (8) has no explicit weight for the entropy term relative to the alignment terms; if the two terms have different scales, the behavior of α may not match the description. Please report the relative scaling or normalization.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: TAPT's test-time objective does not encode the reported robustness values, and the reused APT component is ablated as a baseline.

full rationale

The paper's core derivation chain is not circular. The TAPT objective in Eqs. (3), (6), (7), and (8) is an unsupervised function of augmented views of the test image and of pre-computed ImageNet statistics; it does not contain the downstream class labels or the reported robustness numbers, so the AA/PGD/DI accuracies in Table 1 are empirical outcomes rather than identities with the loss terms. The only self-referential component is the APT-based initialization cited to prior work (Ref. [58], which shares authors), but this is treated as an explicit baseline and is ablated: Figure 4's step=0 curve isolates exactly the APT-initialized prompt, showing that TAPT's additional test-time steps produce the claimed gains. The ImageNet statistics are external prior statistics, not fitted to the test sets, and they do not by construction determine the reported accuracy on the 11 datasets. The absence of an adaptive attack that backpropagates through TAPT's prompt optimization is a substantive threat-model/evaluation limitation under the paper's white-box definition, but it is not a circularity: it concerns whether the measured robustness is the quantity claimed, not whether the claim reduces to its inputs by construction. No uniqueness theorem, fitted parameter renamed as prediction, or ansatz-via-citation chain is load-bearing in the derivation.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The method depends on several hand-chosen hyperparameters (α, τ, M, steps) and on the transferability of ImageNet-derived statistics to unseen tasks. These choices are not swept, and no error bars or confidence intervals are reported. No new entities are postulated.

free parameters (5)
  • alpha (α) = 0.5
    Balances adversarial and clean alignment losses in Eq. 8; no sensitivity analysis is reported.
  • entropy selection threshold τ = top 10% of views
    Controls which augmented views are retained for multi-view entropy minimization; the paper does not specify the threshold in Algorithm 1 or provide a sweep.
  • number of augmented views M = 63 plus original (64 total)
    Number of random augmentations per test sample; chosen by hand.
  • TAPT steps = 1 (Algorithm 1) but 0,1,2,4 explored in Figure 4
    The main results do not state which step count was used; robustness increases with steps.
  • APT initialization hyperparameters = 100 epochs, batch size 32, LR 0.035
    Prompt initialization on ImageNet requires a full APT training run; these values affect the starting prompt.
assumptions (3)
  • domain assumption ImageNet embedding statistics generalize to all 11 target datasets
    Used in adversarial-clean alignment (Eq. 6-8); if ImageNet is not representative, alignment may corrupt class information on other datasets.
  • domain assumption Low-entropy augmented views are more reliable under adversarial perturbation
    Drives the sample selection module in Section 3.2; an attacker could craft perturbations that make wrong views confidently consistent.
  • domain assumption Aligning mean and variance of embeddings to a reference distribution preserves class-discriminative information
    Adversarial-clean alignment implicitly assumes second-order alignment is sufficient to restore useful semantics; no analysis is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TAPT: Test-Time Adversarial Prompt Tuning for Robust Inference in Vision-Language Models." pith.science (2026). https://pith.science/paper/Y3CHDKXR

@misc{pith2026241113136,
  author       = {Pith},
  title        = {Pith review of: TAPT: Test-Time Adversarial Prompt Tuning for Robust Inference in Vision-Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Y3CHDKXR}},
  note         = {Machine review of arXiv:2411.13136}
}
read the original abstract

Large pre-trained Vision-Language Models (VLMs) such as CLIP have demonstrated excellent zero-shot generalizability across various downstream tasks. However, recent studies have shown that the inference performance of CLIP can be greatly degraded by small adversarial perturbations, especially its visual modality, posing significant safety threats. To mitigate this vulnerability, in this paper, we propose a novel defense method called Test-Time Adversarial Prompt Tuning (TAPT) to enhance the inference robustness of CLIP against visual adversarial attacks. TAPT is a test-time defense method that learns defensive bimodal (textual and visual) prompts to robustify the inference process of CLIP. Specifically, it is an unsupervised method that optimizes the defensive prompts for each test sample by minimizing a multi-view entropy and aligning adversarial-clean distributions. We evaluate the effectiveness of TAPT on 11 benchmark datasets, including ImageNet and 10 other zero-shot datasets, demonstrating that it enhances the zero-shot adversarial robustness of the original CLIP by at least 48.9% against AutoAttack (AA), while largely maintaining performance on clean examples. Moreover, TAPT outperforms existing adversarial prompt tuning methods across various backbones, achieving an average robustness improvement of at least 36.6%.

Figures

Figures reproduced from arXiv: 2411.13136 by the authors.

Figure 1
Figure 1. Inference with different prompts. Top: inference with hand-crafted prompts fails to recognize the class ‘cat’; Middle: Inference with fixed prompts tuned by APT methods cannot rec￾ognize all adversarial images; Bottom: Inference with test-time prompts optimized for each image produces accurate recognitions. risks across a wide range of downstream applications. Adversarial training [12, 25, 51, 56] is a general defen… view at source ↗
Figure 2
Figure 2. An illustration of CLIP and different adversarial prompt tuning schemes. (a) The original CLIP [ [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. An overview of our proposed TAPT method: Given an adversarial image, TAPT generates multiple augmented views of the [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Adversarial robustness (%) of our TAPT method under different test-time adaptation steps (i.e., [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Zero-shot adversarial robustness (y-axis) ofs TAPT un [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

66 extracted references · 45 canonical work pages

  1. [1]

    Align your prompts: Test-time prompting with distribution align- ment for zero-shot generalization

    Jameel Abdul Samadh, Mohammad Hanan Gani, Noor Hus- sein, Muhammad Uzair Khattak, Muhammad Muzammal Naseer, Fahad Shahbaz Khan, and Salman H Khan. Align your prompts: Test-time prompting with distribution align- ment for zero-shot generalization. In NeurIPS, 2024. 3, 4

    work page 2024

  2. [2]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. preprint arXiv:2303.08774, 2023. 8

    arXiv 2023

  3. [3]

    Do as i can, not as i say: Grounding language in robotic affordances

    Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Cheb- otar, Omar Cortes, Byron David, Chelsea Finn, Chuyuan Fu, Keerthana Gopalakrishnan, Karol Hausman, et al. Do as i can, not as i say: Grounding language in robotic affordances. preprint arXiv:2204.01691, 2022. 1

    arXiv 2022

  4. [4]

    Agreement-on-the-line: Predicting the performance of neural networks under distribution shift

    Christina Baek, Yiding Jiang, Aditi Raghunathan, and J Zico Kolter. Agreement-on-the-line: Predicting the performance of neural networks under distribution shift. In NeurIPS,

    work page

  5. [5]

    Exploring visual prompts for adapting large- scale models

    Hyojin Bahng, Ali Jahanian, Swami Sankaranarayanan, and Phillip Isola. Exploring visual prompts for adapting large- scale models. preprint arXiv:2203.17274, 2022. 5

    arXiv 2022

  6. [6]

    Food-101–mining discriminative components with random forests

    Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. Food-101–mining discriminative components with random forests. In ECCV, 2014. 5

    2014

  7. [7]

    Describing textures in the wild

    Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. Describing textures in the wild. In CVPR, 2014. 5

    work page 2014

  8. [8]

    Reliable evalua- tion of adversarial robustness with an ensemble of diverse parameter-free attacks

    Francesco Croce and Matthias Hein. Reliable evalua- tion of adversarial robustness with an ensemble of diverse parameter-free attacks. In ICML, 2020. 1, 2, 5, 6

    work page 2020

Show all 66 references
  1. [9]

    Boosting adversarial at- tacks with momentum

    Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boosting adversarial at- tacks with momentum. In CVPR, 2018. 1

    2018

  2. [10]

    One perturbation is enough: On generating universal adversarial perturbations against vision- language pre-training models

    Hao Fang, Jiawei Kong, Wenbo Yu, Bin Chen, Jiawei Li, Shutao Xia, and Ke Xu. One perturbation is enough: On generating universal adversarial perturbations against vision- language pre-training models. preprint arXiv:2406.05491,

  3. [11]

    Learning gener- ative visual models from few training examples: An incre- mental bayesian approach tested on 101 object categories

    Li Fei-Fei, Rob Fergus, and Pietro Perona. Learning gener- ative visual models from few training examples: An incre- mental bayesian approach tested on 101 object categories. In CVPR Workshops, 2004. 5

    2004

  4. [12]

    Large-scale adversarial training for vision- and-language representation learning

    Zhe Gan, Yen-Chun Chen, Linjie Li, Chen Zhu, Yu Cheng, and Jingjing Liu. Large-scale adversarial training for vision- and-language representation learning. In NeurIPS, 2020. 1

    2020

  5. [13]

    Sa-attack: Improving adversar- ial transferability of vision-language pre-training models via self-augmentation

    Bangyan He, Xiaojun Jia, Siyuan Liang, Tianrui Lou, Yang Liu, and Xiaochun Cao. Sa-attack: Improving adversar- ial transferability of vision-language pre-training models via self-augmentation. preprint arXiv:2312.04913, 2023. 2

    2023 arXiv

  6. [14]

    Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification

    Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE J-STARS, 2019. 5

    2019

  7. [15]

    A visual–language foundation model for pathology image analysis using medical twitter

    Zhi Huang, Federico Bianchi, Mert Yuksekgonul, Thomas J Montine, and James Zou. A visual–language foundation model for pathology image analysis using medical twitter. Nature Medicine, 2023. 1

    2023

  8. [16]

    Scaling up visual and vision-language representation learning with noisy text supervision

    Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representation learning with noisy text supervision. In ICML, 2021. 1

    2021

  9. [17]

    Simple but effective: Clip embed- dings for embodied ai

    Apoorv Khandelwal, Luca Weihs, Roozbeh Mottaghi, and Aniruddha Kembhavi. Simple but effective: Clip embed- dings for embodied ai. In CVPR, 2022. 1

    2022

  10. [18]

    Maple: Multi-modal prompt learning

    Muhammad Uzair Khattak, Hanoona Rasheed, Muhammad Maaz, Salman Khan, and Fahad Shahbaz Khan. Maple: Multi-modal prompt learning. In CVPR, 2023. 1

    2023

  11. [19]

    Test-time adaptation induces stronger accuracy and agreement-on-the-line

    Eungyeup Kim, Mingjie Sun, Christina Baek, Aditi Raghu- nathan, and J Zico Kolter. Test-time adaptation induces stronger accuracy and agreement-on-the-line. In NeurIPS,

  12. [20]

    Torchattacks: A pytorch repository for adversar- ial attacks

    Hoki Kim. Torchattacks: A pytorch repository for adversar- ial attacks. preprint arXiv:2010.01950, 2020. 5

    2010 arXiv

  13. [21]

    3d object representations for fine-grained categorization

    Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine-grained categorization. In ICCV Workshops, 2013. 5

    2013

  14. [22]

    One prompt word is enough to boost adversarial robustness for pre-trained vision-language models

    Lin Li, Haoyan Guan, Jianing Qiu, and Michael Spratling. One prompt word is enough to boost adversarial robustness for pre-trained vision-language models. In CVPR, 2024. 1, 2, 5

    2024

  15. [23]

    Set-level guidance at- tack: Boosting adversarial transferability of vision-language pre-training models

    Dong Lu, Zhiqiang Wang, Teng Wang, Weili Guan, Hongchang Gao, and Feng Zheng. Set-level guidance at- tack: Boosting adversarial transferability of vision-language pre-training models. In ICCV, 2023. 2

    2023

  16. [24]

    Imbalanced gradients: a sub- tle cause of overestimated adversarial robustness

    Xingjun Ma, Linxi Jiang, Hanxun Huang, Zejia Weng, James Bailey, and Yu-Gang Jiang. Imbalanced gradients: a sub- tle cause of overestimated adversarial robustness. Machine Learning, 2024. 2

    2024

  17. [25]

    Towards deep learning models resistant to adversarial attacks

    Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. ICLR, 2018. 1, 2, 5, 6

    2018

  18. [26]

    Fine-grained visual classi- fication of aircraft

    Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew Blaschko, and Andrea Vedaldi. Fine-grained visual classi- fication of aircraft. preprint arXiv:1306.5151, 2013. 5

    2013 arXiv

  19. [27]

    Understanding zero-shot adversarial robust- ness for large-scale models

    Chengzhi Mao, Scott Geng, Junfeng Yang, Xin Wang, and Carl V ondrick. Understanding zero-shot adversarial robust- ness for large-scale models. In ICLR, 2023. 2, 5

    2023

  20. [28]

    Evaluating prediction-time batch normalization for robust- ness under covariate shift

    Zachary Nado, Shreyas Padhy, D Sculley, Alexander D’Amour, Balaji Lakshminarayanan, and Jasper Snoek. Evaluating prediction-time batch normalization for robust- ness under covariate shift. preprint arXiv:2006.10963, 2020. 3

    2006 arXiv

  21. [29]

    Automated flower classification over a large number of classes

    Maria-Elena Nilsback and Andrew Zisserman. Automated flower classification over a large number of classes. In ICVGIP, 2008. 5

    2008

  22. [30]

    Efficient test- 9 time model adaptation without forgetting

    Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Yaofo Chen, Shijian Zheng, Peilin Zhao, and Mingkui Tan. Efficient test- 9 time model adaptation without forgetting. In ICML, 2022. 3

    2022

  23. [31]

    Cats and dogs

    Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and CV Jawahar. Cats and dogs. In CVPR, 2012. 5

    2012

  24. [32]

    On the rela- tionship between generalization and robustness to adversar- ial examples

    Anibal Pedraza, Oscar Deniz, and Gloria Bueno. On the rela- tionship between generalization and robustness to adversar- ial examples. Symmetry, 2021. 1

    2021

  25. [33]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. In ICML, 2021. 1, 3

    2021

  26. [34]

    Imagenet large scale visual recognition challenge

    Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, San- jeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. IJCV, 2015. 5

    2015

  27. [35]

    Robust CLIP: Unsupervised ad- versarial fine-tuning of vision embeddings for robust large vision-language models

    Christian Schlarmann, Naman Deep Singh, Francesco Croce, and Matthias Hein. Robust CLIP: Unsupervised ad- versarial fine-tuning of vision embeddings for robust large vision-language models. In ICML, 2024. 2

    2024

  28. [36]

    Improving robustness against common corruptions by covariate shift adaptation

    Steffen Schneider, Evgenia Rusak, Luisa Eck, Oliver Bring- mann, Wieland Brendel, and Matthias Bethge. Improving robustness against common corruptions by covariate shift adaptation. NeurIPS, 2020. 3

    2020

  29. [37]

    The cost of training nlp models: A concise overview

    Or Sharir, Barak Peleg, and Yoav Shoham. The cost of training nlp models: A concise overview. preprint arXiv:2004.08900, 2020. 1

    2004 arXiv

  30. [38]

    Cliport: What and where pathways for robotic manipulation

    Mohit Shridhar, Lucas Manuelli, and Dieter Fox. Cliport: What and where pathways for robotic manipulation. In CoRL, 2022. 1

    2022

  31. [39]

    Test- time prompt tuning for zero-shot generalization in vision- language models

    Manli Shu, Weili Nie, De-An Huang, Zhiding Yu, Tom Goldstein, Anima Anandkumar, and Chaowei Xiao. Test- time prompt tuning for zero-shot generalization in vision- language models. In NeurIPS, 2022. 3, 4

    2022

  32. [40]

    Ucf101: A dataset of 101 human actions classes from videos in the wild

    K Soomro. Ucf101: A dataset of 101 human actions classes from videos in the wild. preprint arXiv:1212.0402, 2012. 5

    2012 arXiv

  33. [41]

    Disentan- gling adversarial robustness and generalization

    David Stutz, Matthias Hein, and Bernt Schiele. Disentan- gling adversarial robustness and generalization. In CVPR,

  34. [42]

    Is robustness the cost of accuracy?– a comprehensive study on the robustness of 18 deep image classification models

    Dong Su, Huan Zhang, Hongge Chen, Jinfeng Yi, Pin-Yu Chen, and Yupeng Gao. Is robustness the cost of accuracy?– a comprehensive study on the robustness of 18 deep image classification models. In ECCV, 2018. 1

    2018

  35. [43]

    In- triguing properties of neural networks

    Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. In- triguing properties of neural networks. In ICLR, 2013. 1

    2013

  36. [44]

    Gemini: a family of highly capable multimodal models

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean- Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalk- wyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models. arXiv:2312.11805, 2023. 8

    2023 arXiv

  37. [45]

    Tent: Fully test-time adaptation by entropy minimization

    Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Ol- shausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. In ICLR, 2021. 3

    2021

  38. [46]

    Trans- ferable multimodal attack on vision-language pre-training models

    Haodi Wang, Kai Dong, Zhilei Zhu, Haotong Qin, Aishan Liu, Xiaolin Fang, Jiakai Wang, and Xianglong Liu. Trans- ferable multimodal attack on vision-language pre-training models. In IEEE S&P, 2024. 2

    2024

  39. [47]

    Con- tinual test-time domain adaptation

    Qin Wang, Olga Fink, Luc Van Gool, and Dengxin Dai. Con- tinual test-time domain adaptation. In CVPR, 2022. 3

    2022

  40. [48]

    Pre- trained model guided fine-tuning for zero-shot adversarial robustness

    Sibo Wang, Jie Zhang, Zheng Yuan, and Shiguang Shan. Pre- trained model guided fine-tuning for zero-shot adversarial robustness. In CVPR, 2024. 2

    2024

  41. [49]

    AdvQDet: Detecting query-based adversarial attacks with adversarial contrastive prompt tun- ing

    Xin Wang, Kai Chen, Xingjun Ma, Zhineng Chen, Jingjing Chen, and Yu-Gang Jiang. AdvQDet: Detecting query-based adversarial attacks with adversarial contrastive prompt tun- ing. In ACM MM, 2024. 2

    2024

  42. [50]

    Medclip: Contrastive learning from unpaired medical images and text

    Zifeng Wang, Zhenbang Wu, Dinesh Agarwal, and Jimeng Sun. Medclip: Contrastive learning from unpaired medical images and text. In EMNLP, 2022. 1

    2022

  43. [51]

    Re- visiting adversarial training at scale

    Zeyu Wang, Xianhang Li, Hongru Zhu, and Cihang Xie. Re- visiting adversarial training at scale. In CVPR, 2024. 1, 2

    2024

  44. [52]

    Sun database: Large-scale scene recognition from abbey to zoo

    Jianxiong Xiao, James Hays, Krista A Ehinger, Aude Oliva, and Antonio Torralba. Sun database: Large-scale scene recognition from abbey to zoo. In CVPR, 2010. 5

    2010

  45. [53]

    Improving transferabil- ity of adversarial examples with input diversity

    Cihang Xie, Zhishuai Zhang, Yuyin Zhou, Song Bai, Jianyu Wang, Zhou Ren, and Alan L Yuille. Improving transferabil- ity of adversarial examples with input diversity. In CVPR,

  46. [54]

    Vlattack: Multimodal adversarial attacks on vision-language tasks via pre-trained models

    Ziyi Yin, Muchao Ye, Tianrong Zhang, Tianyu Du, Jinguo Zhu, Han Liu, Jinghui Chen, Ting Wang, and Fenglong Ma. Vlattack: Multimodal adversarial attacks on vision-language tasks via pre-trained models. In NeurIPS, 2023. 2

    2023

  47. [55]

    Multi-event video-text retrieval

    Gengyuan Zhang, Jisen Ren, Jindong Gu, and V olker Tresp. Multi-event video-text retrieval. In ICCV, 2023. 1

    2023

  48. [56]

    Theoretically principled trade-off between robustness and accuracy

    Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Lau- rent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In ICML, 2019. 1

    2019

  49. [57]

    Towards adversarial attack on vision-language pre-training models

    Jiaming Zhang, Qi Yi, and Jitao Sang. Towards adversarial attack on vision-language pre-training models. In ACM MM,

  50. [58]

    Adversarial prompt tuning for vision-language models

    Jiaming Zhang, Xingjun Ma, Xin Wang, Lingyu Qiu, Jiaqi Wang, Yu-Gang Jiang, and Jitao Sang. Adversarial prompt tuning for vision-language models. In ECCV, 2024. 1, 2, 5

    2024

  51. [59]

    Memo: Test time robustness via adaptation and augmentation

    Marvin Zhang, Sergey Levine, and Chelsea Finn. Memo: Test time robustness via adaptation and augmentation. In NeurIPS, 2022. 3

    2022

  52. [60]

    Univer- sal adversarial perturbations for vision-language pre-trained models

    Peng-Fei Zhang, Zi Huang, and Guangdong Bai. Univer- sal adversarial perturbations for vision-language pre-trained models. In ACM SIGIR, pages 862–871, 2024. 2

    2024

  53. [61]

    On evaluating adversarial robustness of large vision-language models

    Yunqing Zhao, Tianyu Pang, Chao Du, Xiao Yang, Chongx- uan Li, Ngai-Man Man Cheung, and Min Lin. On evaluating adversarial robustness of large vision-language models. In NeurIPS, 2024. 1, 2

    2024

  54. [62]

    Conditional prompt learning for vision-language models

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Zi- wei Liu. Conditional prompt learning for vision-language models. In CVPR, 2022. 1

    2022

  55. [63]

    Learning to prompt for vision-language models

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision-language models. IJCV,

  56. [64]

    Revisiting the adversarial robustness of vision language models: a multimodal perspective

    Wanqi Zhou, Shuanghao Bai, Qibin Zhao, and Badong Chen. Revisiting the adversarial robustness of vision language models: a multimodal perspective. preprint arXiv:2404.19287, 2024. 2 10

    2024 arXiv

  57. [65]

    Few-shot adversarial prompt learning on vision-language models

    Yiwei Zhou, Xiaobo Xia, Zhiwei Lin, Bo Han, and Tongliang Liu. Few-shot adversarial prompt learning on vision-language models. In NeurIPS, 2024. 1, 2

    2024

  58. [66]

    Advclip: Downstream-agnostic adversarial examples in multimodal contrastive learning

    Ziqi Zhou, Shengshan Hu, Minghui Li, Hangtao Zhang, Yechao Zhang, and Hai Jin. Advclip: Downstream-agnostic adversarial examples in multimodal contrastive learning. In ACM MM, 2023. 2 11

    2023

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.