Pith. sign in

REVIEW 4 major objections 5 minor 57 references

This paper claims that emphasizing uncertain adversarial examples during fine-tuning—scaling the KL divergence between clean and adversarial predictions by the model's uncertainty—raises CLIP's average robust accuracy under AutoAttack to 33

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 12:37 UTC pith:DY6WY5IY

load-bearing objection A plausible confidence-weighting loss for CLIP robustness, but the reported gains over prior methods are not convincingly established because the comparison uses copied baseline numbers and test-set-tuned hyperparameters. the 4 major comments →

arxiv 2510.02913 v2 pith:DY6WY5IY submitted 2025-10-03 cs.CV

Improving Adversarial Robustness of Zero-Shot CLIP with Confidence-Aware Weighting

Nikoo Naghavian , Mostafa Tavassolipour This is my paper
classification cs.CV
keywords adversarial robustnesszero-shot learningCLIPvision-language modelsconfidence-aware weightingfeature alignment regularizationAutoAttackadversarial fine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper proposes Confidence-Aware Weighting (CAW), a loss for adversarially fine-tuning CLIP without labeled data beyond TinyImageNet. Its central claim is that training should spend more effort on adversarial examples the model is unsure about: it scales the KL divergence between adversarial and clean predictions by one minus the model's confidence in the true class. Added to the standard cross-entropy loss and an L2 feature-alignment regularizer, this yields 33.51% average robust accuracy across 15 zero-shot datasets under AutoAttack, ahead of recent methods PMG-AFT and TGA-ZSR while using less training memory. A sympathetic reader would take the contribution as evidence that uncertainty-weighted losses are a useful, cheap addition to zero-shot adversarial fine-tuning.

Core claim

CAW's central claim is that not all adversarial examples deserve equal weight during fine-tuning. The paper defines a Confidence-Aware loss, L_CA = (1/N) sum_i KL(P_adv || P_clean) * (1 - P_adv_{i,y_i}), where P_adv is the fine-tuned model's softmax distribution on an adversarial image, P_clean is the frozen CLIP model's distribution on the clean image, and the factor down-weights examples already predicted confidently while up-weighting uncertain ones. Combined with a feature-alignment term that pulls fine-tuned adversarial image features toward frozen CLIP features via L2 distance, the total loss L_CE + alpha*L_CA + beta*L_Reg is trained with PGD-2 on TinyImageNet. The paper reports that t

What carries the argument

The carrying mechanism is the confidence-aware KL term. It computes the KL divergence from the fine-tuned model's adversarial prediction distribution to the frozen model's clean prediction distribution, then multiplies by 1 - P_adv_{i,y_i}, the model's uncertainty about the true class on the adversarial input. This makes the gradient concentrate on samples that are genuinely unstable under perturbation. A second mechanism, feature alignment regularization, computes the L2 distance between frozen and fine-tuned image-encoder features on adversarial inputs, acting as an anchor that preserves pre-trained semantics and counteracts forgetting during fine-tuning.

Load-bearing premise

The load-bearing premise is that CAW's reported advantage over PMG-AFT and TGA-ZSR is not an artifact of unstated training details, since the paper adopts 'settings consistent with prior studies' without specifying epochs, learning-rate schedule, or prompt templates, and fixes alpha and beta without a held-out validation protocol.

What would settle it

Re-run CAW and the two baselines with identical epochs, optimizer schedules, prompt templates, and multiple seeds, then report AutoAttack averages with error bars; if CAW no longer surpasses the baselines, the central claim fails. Alternatively, ablate the confidence factor by setting 1 - P_adv_{i,y_i} to a constant: if robust accuracy does not drop, the weighting is not what drives the gain.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • If CAW is correct, zero-shot CLIP can be hardened against strong attacks like AutoAttack with a simple loss modification, without adding parameters or extra data.
  • The method improves both clean and robust accuracy over PMG-AFT under PGD-100 and CW, suggesting the uncertainty weighting reduces the usual robustness-accuracy trade-off in this setting.
  • The lower memory footprint makes the approach feasible on a single consumer GPU, which matters for large-scale vision-language fine-tuning.
  • The gains transfer across 15 datasets spanning objects, scenes, textures, satellite imagery, and medical images, implying the learned robust features generalize beyond the fine-tuning domain.
  • The choice of KL direction—adversarial distribution as the first argument—is a reusable design decision for other robust fine-tuning objectives.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A natural extension is replacing the scalar confidence weight with a learned or attention-based weight; the paper itself names attention as future work, so this is an invited direction.
  • Because alpha and beta are fixed without a held-out validation protocol, the claim would be stronger if the gains persist under hyperparameters chosen on a validation split or through a small sweep.
  • The same uncertainty-weighting principle could apply to the text encoder or to other vision-language models, which the paper notes it did not test.
  • The observed clean-accuracy improvement suggests the weighting may act as a regularizer, not just a robustness term; this could be tested by measuring feature-space alignment on clean inputs after training.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Confidence-Aware Weighting (CAW), an adversarial fine-tuning method for zero-shot CLIP. CAW adds two terms to the standard cross-entropy loss: (1) a confidence-aware KL divergence between the frozen model's clean-image predictions and the fine-tuned model's adversarial-image predictions, weighted by (1 - P_adv,y), so that uncertain samples receive larger gradients; and (2) an L2 feature alignment regularizer between frozen and fine-tuned image encoder features on adversarial inputs. The model is a ViT-B/32 CLIP fine-tuned on TinyImageNet with PGD-2 at epsilon=1/255 and evaluated on 15 datasets under AutoAttack, PGD-100, and CW. The central claim is that CAW outperforms recent zero-shot robustness methods PMG-AFT and TGA-ZSR in robust accuracy while using less memory. Tables 1, 2, 4, and 5 report average robust accuracy improvements; Table 7 reports memory usage and training time. The appendices include ablations (Table 6) and the NeurIPS checklist, which explicitly states that no error bars or significance tests are reported.

Significance. If the empirical claims are correct, CAW is a simple and intuitive contribution to zero-shot adversarial robustness: it reweights loss terms toward uncertain adversarial samples and uses feature alignment to preserve pre-trained knowledge. The paper evaluates on a broad set of 15 datasets and reports both clean and robust accuracy, which is valuable. The memory-efficiency comparison (Table 7) is also a useful practical consideration. However, the significance of the claimed state-of-the-art results is currently conditional on the validity of the baseline comparison and on the statistical reliability of the 2-point average improvements. The authors are transparent about the lack of error bars, but this transparency does not remedy the fact that the central claim rests on uncontrolled comparisons and hand-tuned hyperparameters. The method itself is plausible, and the paper has merit as a workshop contribution, but the evidence as presented does not yet convincingly establish the headline claim.

major comments (4)
  1. [Appendix C / Table 1] The state-of-the-art claim relies on baseline numbers copied from the TGA-ZSR paper ('we compare against the reported results ... as presented in the TGA-ZSR paper'). Appendix B reports only learning rate, momentum, weight decay, batch size, PGD iterations, and epsilon; it does not specify the number of training epochs, learning-rate schedule, prompt templates, or the exact evaluation protocol for CAW or the baselines. Since the reported average gain over TGA-ZSR is 1.9 points (33.51 vs 31.63) and over PMG-AFT is 2.0 points, the comparison is not controlled. Please re-run the baselines under identical training budgets and evaluation conditions, or provide evidence that the copied numbers were produced under exactly the same protocol.
  2. [Appendix B / Table 6] Hyperparameters alpha=6 and beta=3 are chosen 'to balance clean and robust accuracy' without a held-out validation split. Because the same 15 datasets are used for both model selection and final reporting, the reported numbers may reflect optimistic bias. Please provide a sensitivity analysis over alpha and beta and a validation-based selection procedure (e.g., TinyImageNet val), and report the average and standard deviation over multiple seeds. The NeurIPS checklist (Question 7) explicitly states that no error bars or significance tests are reported, but the main claim is an average 2-point improvement; without variance estimates, it is impossible to tell whether the difference is meaningful.
  3. [Table 5] The claim of 'state-of-the-art under strong adversarial attacks' is mostly established at epsilon=1/255. At epsilon=4/255 the average advantage over PMG-AFT shrinks to 0.26 points (19.53 vs 19.27), and CAW is worse on SUN397, OxfordPets, EuroSAT, Caltech-101, and PCAM. This fragility suggests the benefit is specific to a small perturbation bound. Please either soften the claim to the evaluated setting or analyze why the method degrades at larger epsilon.
  4. [Appendix B / Table 7] The memory-efficiency claim is load-bearing for the abstract, but the paper does not describe how memory usage was measured (e.g., whether it includes activations, optimizer states, gradient checkpointing, or is simply the peak GPU allocation reported by PyTorch). If the baseline memory numbers are taken from prior papers with a different measurement protocol, the comparison is not valid. Please specify the measurement methodology and, ideally, report memory using a standard profiler for all methods under the same batch size and resolution.
minor comments (5)
  1. [Eq. (9)] The summation in Eq. (9) runs from i=0 to N, while Eq. (8) uses i=1 to N. The indexing should be made uniform and consistent with the batch definition.
  2. [Table 1] Several entries are missing spaces, e.g., '18.5419.95', '7.5748.84', making the table hard to read. Please format the table properly.
  3. [References] Reference [1] is cited as 'Language models are few-shot learners' but the author list is incorrect: it lists Ben Mann et al. rather than the GPT-3 authors (Tom B. Brown et al.). Please correct the citation.
  4. [Table 3] The caption reads 'Zero-shot clean accuracy under PGD-100'. Clean accuracy is not measured under an attack; the caption should read 'Zero-shot clean accuracy' or 'clean accuracy (no attack)'.
  5. [Appendix D] There is a typo: 'Broder impact' should be 'Broader impact'.

Circularity Check

0 steps flagged

No significant circularity: empirical method paper with externally cited baselines and held-out evaluation.

full rationale

This is an empirical paper: the method is a new training loss (Eq. 10) combining cross-entropy, a confidence-weighted KL term (Eq. 8), and an l2 feature-alignment regularizer (Eq. 9). There is no derivation chain in which an output is defined as an input. The claimed improvements are evaluated on 15 datasets, and the comparison numbers for PMG-AFT and TGA-ZSR are taken from the cited TGA-ZSR paper [19], which is an external source rather than a self-citation by the current authors. The loss itself uses the frozen CLIP model as a reference, but the reported robust accuracy is measured on held-out data, not recovered from the loss by construction. The hyperparameters alpha=6 and beta=3 are tuned 'to balance clean and robust accuracy' without a held-out validation protocol, which is a correctness/reproducibility concern, but it is not circularity in the derivation sense: no fitted value is renamed as a prediction, and there is no self-referential theorem. The paper also states its own limitations (CLIP-only, image-encoder-only attacks), and the checklist admits no error bars; these affect reliability, not circularity. No load-bearing step reduces to its own input, so a non-finding is appropriate.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 0 invented entities

The method introduces two loss terms with two hand-tuned weights (α=6, β=3) and a hand-selected KL direction; these are the main free parameters. The key domain assumptions are that TinyImageNet fine-tuning transfers, frozen CLIP features are a good reference, and confidence weighting identifies vulnerable samples. No new entities are postulated.

free parameters (5)
  • α (confidence-aware loss weight) = 6
    Set by hand to balance clean and robust accuracy (Appendix B); no sensitivity analysis or validation split described.
  • β (feature alignment weight) = 3
    Set by hand alongside α; no sensitivity analysis reported.
  • KL direction (P_adv as first argument) = forward KL
    Chosen empirically: 'showed better results in our experiments' (Section 2.2); could be tuned per benchmark.
  • PGD iterations (training) = 2
    Fixed training attack budget; follows prior work but affects the difficulty of adversarial examples seen during training.
  • Perturbation bound ε = 1/255
    Fixed ℓ∞ budget for training and evaluation; standard in prior zero-shot robustness papers.
axioms (5)
  • domain assumption Fine-tuning CLIP on TinyImageNet transfers robustness to other datasets
    The method is trained only on TinyImageNet but claims zero-shot robustness on 14 other datasets (Appendix B); transfer is assumed and empirically tested.
  • domain assumption Frozen CLIP image encoder features are a stable semantic prior
    L_Reg (Eq. 9) assumes aligning fine-tuned features to the frozen encoder preserves semantics on adversarial inputs.
  • domain assumption Clean predictions from the frozen model are a reliable reference distribution
    L_CA (Eq. 8) uses P_clean from the frozen encoder as the target; this assumes the frozen model's clean predictions are meaningful for adversarial adaptation.
  • domain assumption KL divergence with confidence scaling targets the most vulnerable samples
    Weighting by 1 - P_adv_{i,y_i} assumes low true-class confidence identifies samples that most need robust training.
  • standard math Standard mathematical properties of softmax, KL divergence, and L2 distance
    Used in Eqs. 5-9 without proof.

pith-pipeline@v1.3.0-alltime-deepseek · 15634 in / 11472 out tokens · 366358 ms · 2026-08-04T12:37:54.659901+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Improving Adversarial Robustness of Zero-Shot CLIP with Confidence-Aware Weighting." pith.science (2026). https://pith.science/paper/DY6WY5IY

@misc{pith2026251002913,
  author       = {Pith},
  title        = {Pith review of: Improving Adversarial Robustness of Zero-Shot CLIP with Confidence-Aware Weighting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DY6WY5IY}},
  note         = {Machine review of arXiv:2510.02913}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Vision-language models such as CLIP demonstrate impressive zero-shot generalization but remain highly vulnerable to adversarial attacks. Prior adversarial methods treat all samples equally in the loss function, despite the fact that not all inputs contribute equally to adversarial vulnerability. Some samples retain high confidence even under large perturbations, while others are unstable and change predictions with minor noise. To address this, we propose Confidence-Aware Weighting (CAW) to enhance the zero-shot robustness of vision-language models. CAW introduces two key components: (1) a Confidence-Aware Loss that prioritizes uncertain adversarial examples by scaling the KL divergence between clean and adversarial predictions, and (2) a Feature Alignment Regularization that preserves semantic consistency by minimizing the distance between frozen and fine-tuned image encoder features on adversarial inputs. Together, these components improve both clean and robust accuracy without compromising generalization. Extensive experiments on TinyImageNet and 14 additional datasets show that CAW outperforms recent state-of-the-art methods such as PMG-AFT and TGA-ZSR under strong adversarial attacks like AutoAttack, while also being more memory-efficient.

Figures

Figures reproduced from arXiv: 2510.02913 by Mostafa Tavassolipour, Nikoo Naghavian.

Figure 1
Figure 1. Figure 1: Overview of Confidence-Aware Weighting (CAW) method. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

57 extracted references · 12 linked inside Pith

  1. [1]

    Language models are few-shot learners.arXiv preprint arXiv:2005.14165, 1(3):3, 2020

    Ben Mann, Nick Ryder, Melanie Subbiah, J Kaplan, P Dhariwal, A Neelakantan, P Shyam, G Sastry, A Askell, S Agarwal, et al. Language models are few-shot learners.arXiv preprint arXiv:2005.14165, 1(3):3, 2020

    Pith/arXiv arXiv 2005

  2. [2]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. InInternational conference on machine learning, pages 8748–8763. PmLR, 2021

    2021

  3. [3]

    A simple framework for contrastive learning of visual representations

    Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. InInternational conference on machine learning, pages 1597–1607. PmLR, 2020

    2020

  4. [4]

    Emerging properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. InProceedings of the IEEE/CVF international conference on computer vision, pages 9650–9660, 2021

    2021

  5. [5]

    Very deep convolutional networks for large-scale image recognition.arXiv preprint arXiv:1409.1556, 2014

    Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition.arXiv preprint arXiv:1409.1556, 2014

    Pith/arXiv arXiv 2014

  6. [6]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016

    2016

  7. [7]

    Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs.IEEE transactions on pattern analysis and machine intelligence, 40(4): 834–848, 2017

    2017

  8. [8]

    Fast r-cnn

    Ross Girshick. Fast r-cnn. InProceedings of the IEEE international conference on computer vision, pages 1440–1448, 2015

    2015

  9. [9]

    Momentum contrast for unsupervised visual representation learning

    Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9729–9738, 2020

    2020

  10. [10]

    Image-text retrieval: A survey on recent research and development.arXiv preprint arXiv:2203.14713, 2022

    Min Cao, Shiping Li, Juntao Li, Liqiang Nie, and Min Zhang. Image-text retrieval: A survey on recent research and development.arXiv preprint arXiv:2203.14713, 2022

    Pith/arXiv arXiv 2022

  11. [11]

    Clip-guided vision-language pre-training for question answering in 3d scenes

    Maria Parelli, Alexandros Delitzas, Nikolas Hars, Georgios Vlassis, Sotirios Anagnostidis, Gregor Bachmann, and Thomas Hofmann. Clip-guided vision-language pre-training for question answering in 3d scenes. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5607–5612, 2023

    2023

  12. [12]

    Explaining and harnessing adversar- ial examples.arXiv preprint arXiv:1412.6572, 2014

    Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversar- ial examples.arXiv preprint arXiv:1412.6572, 2014

    Pith/arXiv arXiv 2014

  13. [13]

    Adversarial training for free!Advances in neural information processing systems, 32, 2019

    Ali Shafahi, Mahyar Najibi, Mohammad Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry S Davis, Gavin Taylor, and Tom Goldstein. Adversarial training for free!Advances in neural information processing systems, 32, 2019

    2019

  14. [14]

    On evaluating adversarial robustness of large vision-language models.Advances in Neural Information Processing Systems, 36:54111–54138, 2023

    Yunqing Zhao, Tianyu Pang, Chao Du, Xiao Yang, Chongxuan Li, Ngai-Man Man Cheung, and Min Lin. On evaluating adversarial robustness of large vision-language models.Advances in Neural Information Processing Systems, 36:54111–54138, 2023

    2023

  15. [15]

    Theoretically principled trade-off between robustness and accuracy

    Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. InInternational conference on machine learning, pages 7472–7482. PMLR, 2019

    2019

  16. [16]

    Understanding zero-shot adversarial robustness for large-scale models.arXiv preprint arXiv:2212.07016, 2022

    Chengzhi Mao, Scott Geng, Junfeng Yang, Xin Wang, and Carl V ondrick. Understanding zero-shot adversarial robustness for large-scale models.arXiv preprint arXiv:2212.07016, 2022. 5

    Pith/arXiv arXiv 2022

  17. [17]

    Exploring visual prompts for adapting large-scale models.arXiv preprint arXiv:2203.17274, 2022

    Hyojin Bahng, Ali Jahanian, Swami Sankaranarayanan, and Phillip Isola. Exploring visual prompts for adapting large-scale models.arXiv preprint arXiv:2203.17274, 2022

    Pith/arXiv arXiv 2022

  18. [18]

    Pre-trained model guided fine-tuning for zero-shot adversarial robustness

    Sibo Wang, Jie Zhang, Zheng Yuan, and Shiguang Shan. Pre-trained model guided fine-tuning for zero-shot adversarial robustness. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 24502–24511, 2024

    2024

  19. [19]

    Text-guided attention is all you need for zero-shot robustness in vision-language models.Advances in Neural Information Processing Systems, 37: 96424–96448, 2024

    Lu Yu, Haiyang Zhang, and Changsheng Xu. Text-guided attention is all you need for zero-shot robustness in vision-language models.Advances in Neural Information Processing Systems, 37: 96424–96448, 2024

    2024

  20. [20]

    Towards deep learning models resistant to adversarial attacks.arXiv preprint arXiv:1706.06083, 2017

    Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks.arXiv preprint arXiv:1706.06083, 2017

    Pith/arXiv arXiv 2017

  21. [21]

    Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks

    Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. InInternational conference on machine learning, pages 2206–2216. PMLR, 2020

    2020

  22. [22]

    Towards evaluating the robustness of neural networks

    Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp), pages 39–57. Ieee, 2017

    2017

  23. [23]

    Improving adversarial robustness by putting more regularizations on less robust samples

    Dongyoon Yang, Insung Kong, and Yongdai Kim. Improving adversarial robustness by putting more regularizations on less robust samples. InInternational Conference on Machine Learning, pages 39331–39348. PMLR, 2023

    2023

  24. [24]

    Deepfool: a simple and accurate method to fool deep neural networks

    Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, and Pascal Frossard. Deepfool: a simple and accurate method to fool deep neural networks. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 2574–2582, 2016

    2016

  25. [25]

    Intriguing properties of neural networks.arXiv preprint arXiv:1312.6199, 2013

    Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfel- low, and Rob Fergus. Intriguing properties of neural networks.arXiv preprint arXiv:1312.6199, 2013

    Pith/arXiv arXiv 2013

  26. [26]

    Adversarial examples in the physical world

    Alexey Kurakin, Ian J Goodfellow, and Samy Bengio. Adversarial examples in the physical world. InArtificial intelligence safety and security, pages 99–112. Chapman and Hall/CRC, 2018

    2018

  27. [27]

    Explainable ai: A review of machine learning interpretability methods.Entropy, 23(1):18, 2020

    Pantelis Linardatos, Vasilis Papastefanopoulos, and Sotiris Kotsiantis. Explainable ai: A review of machine learning interpretability methods.Entropy, 23(1):18, 2020

    2020

  28. [28]

    Security analysis and enhancement of model compressed deep learning systems under adversarial attacks

    Qi Liu, Tao Liu, Zihao Liu, Yanzhi Wang, Yier Jin, and Wujie Wen. Security analysis and enhancement of model compressed deep learning systems under adversarial attacks. In2018 23rd Asia and South Pacific Design Automation Conference (ASP-DAC), pages 721–726. IEEE, 2018

    2018

  29. [29]

    Stochastic activation pruning for robust adversarial defense.arXiv preprint arXiv:1803.01442, 2018

    Guneet S Dhillon, Kamyar Azizzadenesheli, Zachary C Lipton, Jeremy Bernstein, Jean Kossaifi, Aran Khanna, and Anima Anandkumar. Stochastic activation pruning for robust adversarial defense.arXiv preprint arXiv:1803.01442, 2018

    Pith/arXiv arXiv 2018

  30. [30]

    Towards deep neural network architectures robust to adversarial examples.arXiv preprint arXiv:1412.5068, 2014

    Shixiang Gu and Luca Rigazio. Towards deep neural network architectures robust to adversarial examples.arXiv preprint arXiv:1412.5068, 2014

    Pith/arXiv arXiv 2014

  31. [31]

    Improving the adversarial robustness and interpretability of deep neural networks by regularizing their input gradients

    Andrew Ross and Finale Doshi-Velez. Improving the adversarial robustness and interpretability of deep neural networks by regularizing their input gradients. InProceedings of the AAAI conference on artificial intelligence, volume 32, 2018

    2018

  32. [32]

    Adversarial examples are not bugs, they are features.Advances in neural information processing systems, 32, 2019

    Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. Adversarial examples are not bugs, they are features.Advances in neural information processing systems, 32, 2019

    2019

  33. [33]

    Improving adversarial robustness requires revisiting misclassified examples

    Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu. Improving adversarial robustness requires revisiting misclassified examples. InInternational conference on learning representations, 2019. 6

    2019

  34. [34]

    Reducing excessive margin to achieve a bet- ter accuracy vs

    Rahul Rade and Seyed-Mohsen Moosavi-Dezfooli. Reducing excessive margin to achieve a bet- ter accuracy vs. robustness trade-off. InInternational Conference on Learning Representations, 2022

    2022

  35. [35]

    Attention is all you need.Advances in neural information processing systems, 30, 2017

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need.Advances in neural information processing systems, 30, 2017

    2017

  36. [36]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. InProceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), pages 4171–4186, 2019

    2019

  37. [37]

    Language models are unsupervised multitask learners.OpenAI blog, 1(8):9, 2019

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners.OpenAI blog, 1(8):9, 2019

    2019

  38. [38]

    Blip: Bootstrapping language- image pre-training for unified vision-language understanding and generation

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language- image pre-training for unified vision-language understanding and generation. InInternational conference on machine learning, pages 12888–12900. PMLR, 2022

    2022

  39. [39]

    Scaling up visual and vision-language representation learning with noisy text supervision

    Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representation learning with noisy text supervision. InInternational conference on machine learning, pages 4904–4916. PMLR, 2021

    2021

  40. [40]

    Language-driven anchors for zero-shot adversarial robustness

    Xiao Li, Wei Zhang, Yining Liu, Zhanhao Hu, Bo Zhang, and Xiaolin Hu. Language-driven anchors for zero-shot adversarial robustness. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24686–24695, 2024

    2024

  41. [41]

    Robust clip: Unsupervised adversarial fine-tuning of vision embeddings for robust large vision-language models.arXiv preprint arXiv:2402.12336, 2024

    Christian Schlarmann, Naman Deep Singh, Francesco Croce, and Matthias Hein. Robust clip: Unsupervised adversarial fine-tuning of vision embeddings for robust large vision-language models.arXiv preprint arXiv:2402.12336, 2024

    Pith/arXiv arXiv 2024

  42. [42]

    Improving zero-shot adversarial robustness in vision-language models by closed-form alignment of adversarial path simplices

    Junhao Dong, Piotr Koniusz, Yifei Zhang, Hao Zhu, Weiming Liu, Xinghua Qu, and Yew-Soon Ong. Improving zero-shot adversarial robustness in vision-language models by closed-form alignment of adversarial path simplices. InForty-second International Conference on Machine Learning

  43. [43]

    Imagenet: A large- scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hierarchical image database. In2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009

    2009

  44. [44]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009

    2009

  45. [45]

    An analysis of single-layer networks in unsuper- vised feature learning

    Adam Coates, Andrew Ng, and Honglak Lee. An analysis of single-layer networks in unsuper- vised feature learning. InProceedings of the fourteenth international conference on artificial intelligence and statistics, pages 215–223. JMLR Workshop and Conference Proceedings, 2011

    2011

  46. [46]

    One-shot learning of object categories.IEEE transactions on pattern analysis and machine intelligence, 28(4):594–611, 2006

    Li Fei-Fei, Robert Fergus, and Pietro Perona. One-shot learning of object categories.IEEE transactions on pattern analysis and machine intelligence, 28(4):594–611, 2006

    2006

  47. [47]

    Caltech-256 object category dataset

    Gregory Griffin, Alex Holub, and Pietro Perona. Caltech-256 object category dataset. 2007

    2007

  48. [48]

    Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and C. V . Jawahar. Cats and dogs. In 2012 IEEE Conference on Computer Vision and Pattern Recognition, pages 3498–3505, 2012. doi: 10.1109/CVPR.2012.6248092

    arXiv 2012

  49. [49]

    Automated flower classification over a large number of classes

    Maria-Elena Nilsback and Andrew Zisserman. Automated flower classification over a large number of classes. In2008 Sixth Indian conference on computer vision, graphics & image processing, pages 722–729. IEEE, 2008

    2008

  50. [50]

    Fine- grained visual classification of aircraft.arXiv preprint arXiv:1306.5151, 2013

    Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew Blaschko, and Andrea Vedaldi. Fine- grained visual classification of aircraft.arXiv preprint arXiv:1306.5151, 2013. 7

    Pith/arXiv arXiv 2013

  51. [51]

    3d object representations for fine- grained categorization

    Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine- grained categorization. InProceedings of the IEEE international conference on computer vision workshops, pages 554–561, 2013

    2013

  52. [52]

    Sun database: Large-scale scene recognition from abbey to zoo

    Jianxiong Xiao, James Hays, Krista A Ehinger, Aude Oliva, and Antonio Torralba. Sun database: Large-scale scene recognition from abbey to zoo. In2010 IEEE computer society conference on computer vision and pattern recognition, pages 3485–3492. IEEE, 2010

    2010

  53. [53]

    Food-101–mining discriminative components with random forests

    Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. Food-101–mining discriminative components with random forests. InEuropean conference on computer vision, pages 446–461. Springer, 2014

    2014

  54. [54]

    Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification.IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12(7):2217–2226, 2019

    2019

  55. [55]

    Describing textures in the wild

    Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. Describing textures in the wild. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 3606–3613, 2014

    2014

  56. [56]

    Limitations

    Babak Ehteshami Bejnordi, Mitko Veta, Paul J. van Diest, Bram van Ginneken, Nico Karsse- meijer, Geert Litjens, Jeroen A. W. M. van der Laak, and the CAMELYON16 Consortium. Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer.JAMA, 318(22):2199–2210, 2017. doi: 10.1001/jama.2017.14585. URLhtt...

    arXiv 2017

  57. [57]

    Guidelines: • The answer NA means that the paper does not involve crowdsourcing nor research with human subjects

    Institutional review board (IRB) approvals or equivalent for research with human subjects Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or ...

    2025