Pith. sign in

REVIEW 4 major objections 6 minor 45 references

CARE, a collaborative adversarial fine-tuning framework, trains an image-text alignment expert and an image-invariance expert together, harmonizes them with cosine-similarity losses, and merges them by exponential moving average into a sing

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-07-31 23:25 UTC pith:JXKSGUUG

load-bearing objection CARE is a real, incremental framework for combining TeCoA and FARE in CLIP robust fine-tuning, with consistent gains across ID and zero-shot benchmarks, but the central harmonization mechanism is never isolated and the evaluation lacks seeds—worth sending to peer review with revisions. the 4 major comments →

arxiv 2607.27897 v1 pith:JXKSGUUG submitted 2026-07-30 cs.CV

Unifying Adversarially Robust Model Experts in Vision-Language Models

Nguyen Duc Thai , Junhao Dong , Sua Qi Rong , Hua Yu , Yew-Soon Ong This is my paper
classification cs.CV
keywords adversarial robustnessvision-language modelsadversarial fine-tuningcollaborative learningmodel expertsembedding harmonizationexponential moving averagezero-shot robustness
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper addresses a trade-off in adversarially fine-tuning vision-language models: strategies guided by image-text alignment are strong on learned class names but weak on unseen data, while strategies that enforce image-embedding invariance transfer better but ignore the shared multimodal representation. CARE trains both kinds of experts at once, adds cosine-similarity losses that pull their clean logits and clean image embeddings toward each other, starts both from a shared universal perturbation, and merges parameters with an exponential moving average. The result is one model that keeps both specializations: on ImageNet under 4/255 attacks, CARE reaches 47.1% robust accuracy versus 44.3% for the alignment expert and 33.3% for the invariance expert, and it also leads on zero-shot data. The authors claim this shows collaborative learning across robust model experts is a promising direction for improving adversarial robustness.

Core claim

The central discovery is that two adversarial fine-tuning strategies for CLIP—one that aligns image and text logits (TeCoA-style) and one that preserves image-embedding invariance (FARE-style)—do not have to be chosen between. CARE runs both as simultaneous experts, adds a pair of cosine-similarity losses that align their clean logits and clean image embeddings, seeds both with a shared universal perturbation, and folds the two parameter trajectories into an EMA model. The authors show this yields a single image encoder that inherits the alignment expert's strength on in-distribution classes and the invariance expert's transfer to unseen data, outperforming both individual experts on ImageNe

What carries the argument

The load-bearing object is the pair of negative-cosine-similarity alignment losses, L_align_z and L_align_f (Eqs. 8 and 10): each expert is asked to match the other's clean logits or clean image embeddings. Theorem 1 shows that when the parameter drift Δθ = θ_Inv − θ_Align is small, each auxiliary loss's gradient is a Hessian-scaled multiple of the drift (Eqs. 12–13), so it acts as momentum pulling the experts toward each other. GAIP (Eqs. 18–20) constructs a single universal perturbation maximizing both logit and embedding distortion, giving the two experts a common starting point for their PGD adversaries. EMA (Eq. 7) then merges the two expert parameters into the final unified model at no

Load-bearing premise

The framework's stability rests on the unmeasured assumption that the two experts' parameters stay close enough for the second-order Taylor expansion in Theorem 1 to remain valid throughout training; the paper does not measure or bound the drift, so if adversarial losses push the experts far apart, the harmonization gradients may no longer act as momentum and the EMA merge has no stability guarantee.

What would settle it

Train CARE with the harmonization losses removed but GAIP and EMA kept, using the same schedule and attack budget; if robust ImageNet accuracy at ε=4/255 stays at or near CARE's reported 47.1%, then embedding harmonization—the paper's central mechanism—is not doing the work. A complementary check is to log ‖θ_Inv − θ_Align‖ over training and see whether it remains in the regime where the quadratic approximation of Eqs. 12–13 holds.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If CARE's central claim holds, practitioners can replace two specialized fine-tuned models with one unified model that matches or beats each expert on its own preferred setting.
  • The unified model transfers its robustness to downstream vision-language tasks such as image captioning and visual question answering, without extra inference cost because only one image encoder is used.
  • The training overhead is lower than a naive multi-experiment strategy: total PGD steps increase by a factor of about 1.5 rather than 2 because the universal perturbation is shared, and memory can be kept at roughly two models' worth.
  • CARE removes the need for periodic parameter overwriting to stabilize merging; the harmonization losses keep the experts close enough that the final EMA merge is stable.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: The same cosine-harmonization recipe is not tied to the two specific experts used here; it could be applied to experts with different strengths—for example, one trained under L∞ attacks and one under L2 attacks—since the alignment losses operate on embeddings and logits rather than on attack geometry.
  • Editorial inference: The paper's Section IV-C asymmetry suggests direction matters: aligning the invariance expert's logits to the alignment expert helps, while aligning the alignment expert's embeddings appears to push it toward in-distribution overfitting. A weighted or scheduled version of the two alignment losses might retain the zero-shot gains without that overfitting cost—a testable variant
  • Editorial inference: The small-drift theorem implies a practical diagnostic: tracking ‖θ_Inv − θ_Align‖ and the dominant Hessian eigenvalues of the alignment losses during training would reveal when the quadratic approximation breaks down and EMA merging is expected to destabilize.
  • Editorial inference: Because GAIP shares the starting perturbation, some of CARE's gain could be an input-space curriculum effect rather than a pure parameter-space harmonization effect; ablating GAIP while keeping the harmonization losses fixed would isolate the contribution of the shared initialization.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes CARE, a collaborative adversarial fine-tuning framework for CLIP that trains two 'experts' from the same initialization: an image-text alignment expert (based on TeCoA, Eq. 4) and an image-invariance expert (based on FARE, Eq. 6). The experts are coupled by logits- and embedding-harmonization losses (Eqs. 8-11), a shared universal perturbation initialization (GAIP, Eqs. 18-20), and are merged by EMA (Eq. 7). CARE is evaluated on ImageNet in-distribution and 13 zero-shot datasets under AutoAttack, and on image captioning (COCO, Flickr30k) and VQA (TextVQA, VQAv2) with LLaVA-1.5-7B. The central empirical claim is that the final CARE model outperforms individually trained TeCoA and FARE experts.

Significance. If the result holds, CARE provides a practical way to combine complementary robustness properties of VLM fine-tuning strategies, and the paper's benchmark coverage is a strength: it includes AutoAttack, adaptive APGD attacks, downstream vision-language tasks, compute/memory analysis, and ablations of GAIP and the harmonization loss choice. The theoretical derivation in Theorem 1 is a useful local motivation. However, the experimental protocol currently does not isolate the harmonization mechanism, and single-run results without variability estimates leave the main quantitative claim under-supported.

major comments (4)
  1. [§VI-C, Table VIII] The paper's central mechanism—Expert harmonization—is never ablated by simply removing L_align_z and L_align_f (Eqs. 8, 10) while keeping GAIP and the EMA merge. Table VIII only compares harmonization against parameter redistribution; both arms use the same two-expert setup. Since EMA averaging of models fine-tuned from the same pretrained checkpoint is known to improve accuracy (see ref. [20], model soups), the reported gains over TeCoA/FARE could be partly or wholly due to the EMA merge rather than to embedding harmonization. Please add a no-harmonization control (and ideally, an EMA of independently trained TeCoA and FARE) to the main comparison.
  2. [§III-B, Theorem 1] Theorem 1 requires Δθ = θ_Inv − θ_Align to be 'sufficiently small' for the second-order Taylor expansion, and the proof then interprets the Hessian term as a pulling momentum. The paper never measures the actual drift during training or verifies that the expansion remains valid. If adversarial losses push the experts far apart, the harmonization gradient may not reduce drift and the EMA merge could be unstable. Please report the drift norm over training (e.g., per-layer L2 distance) and, if possible, check that the gradient in Eqs. (12)-(13) indeed points opposite to the drift.
  3. [§IV, Tables I-III] All main results are single runs with no standard deviations or significance tests. Several claimed differences are a few accuracy points (e.g., Table I: CARE4 vs TeCoA4 ID ε=4 is 47.1 vs 44.3; Table II: CARE4 vs FARE4 captioning ε=4 is 31.9 vs 31.9). Without multiple seeds or error bars, it is not possible to judge whether 'outperforms individually learned experts' is a robust statement. Please report mean±std over at least 3 seeds for the key tables, or provide an equivalence/noise analysis.
  4. [§III-A, §IV] There is no baseline of a single image encoder trained with both the CE and L2 adversarial losses jointly. Such a model would test whether the multi-expert collaboration is necessary, or whether simply combining the two objectives in one network produces similar robustness. Without this control, the paper's conclusion that collaborative expert learning is a promising direction is not yet isolated from straightforward loss combination.
minor comments (6)
  1. [Section VIII] The heading 'DICUSSION' should be 'DISCUSSION'; also correct 'primariliy' and 'Each experts' in the Figure 3 caption.
  2. [Figure 3] The caption contains garbled Unicode characters ('/uni00000016/uni00000013...'); please fix the source.
  3. [Eqs. (12)-(13)] H_z is written as H_z(θ) in the theorem statement but H_z(θAlign) in the proof; define H_f arguments consistently.
  4. [Algorithm 1, line 4] δ ∼ N(−ε, ε) is unusual notation for the initialization; if a uniform perturbation is intended, use U(−ε, ε).
  5. [Section VI-A, Table VI] The clean in-distribution accuracy is identical to two decimals (61.06) for KL and Cosine; please clarify whether this is coincidence or an artifact.
  6. [Theorem 1] The phrase 'sufficient small drift' should be 'sufficiently small drift.'

Circularity Check

0 steps flagged

No significant circularity: CARE's reported gains are empirical and self-contained; the only self-citation is non-load-bearing.

full rationale

CARE's claim is an empirical engineering result: two experts are adversarially fine-tuned with harmonization losses (Eqs. 8-11), a shared GAIP initialization (Eqs. 18-20), and EMA merging (Eq. 7), then evaluated against the external TeCoA/FARE baselines (Tables I-III). No parameter is fitted to the test set and re-reported as a prediction; the ablations in Tables VI-VIII vary the harmonization loss, GAIP, and the parameter-redistribution alternative. Theorem 1 is a local Taylor-expansion motivation with an explicit small-drift assumption, not a derivation of the empirical outcome. The only self-citation is [8], used in Sec. III-C as 'inspiration' for GAIP; the GAIP design is then ablated in this paper, so the citation is not load-bearing. The absence of a control that removes only the harmonization losses is a completeness concern, but it is not an input-to-output equivalence by construction and therefore does not constitute circularity.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

The framework introduces no new physical or architectural entities. It relies on standard optimization assumptions, a domain assumption about expert complementarity, and a hand-chosen set of hyperparameters (β, γ, PGD step split, loss weights). The most fragile assumption is the unverified small-drift condition.

free parameters (4)
  • EMA decay rate β = 0.999
    Chosen by hand; controls how much historical CARE weights are retained. Not swept in ablations.
  • Expert mixing weight γ = 0.5
    Equal influence of the two experts; chosen without sensitivity analysis.
  • PGD step allocation M=5, K=5 = 5 / 5
    Allocated to match 10 PGD steps per expert, but adds 5 extra forward steps overall; not swept.
  • Loss weights (all = 1) = 1
    The balance between adversarial and harmonization losses is fixed; no weighting schedule.
axioms (4)
  • standard math The losses are twice differentiable near the operating point (Theorem 1, Section III-B).
    Needed for the Taylor expansion in the proof of the harmonization momentum.
  • domain assumption TeCoA and FARE are representative of the two complementary adversarial fine-tuning strategies.
    The paper selects these two methods as canonical examples; if the dichotomy is not representative, the generalization claim weakens.
  • ad hoc to paper The parameter drift between experts remains small enough throughout training for the Taylor expansion to hold.
    Stated explicitly in Theorem 1 and after Eq. 17, but not empirically verified.
  • domain assumption Aligning clean logits and clean image embeddings is sufficient to transfer adversarial robustness between experts.
    The harmonization losses (Eqs. 8-11) use clean representations only; the paper relies on this transferring to adversarial behavior.

pith-pipeline@v1.3.0-daily-deepseek · 19884 in / 13441 out tokens · 124167 ms · 2026-07-31T23:25:46.928679+00:00 · methodology

0 comments
read the original abstract

Vision-language models (VLMs), such as CLIP, are vulnerable to adversarial attacks, posing a serious problem for real-life applications and deployment. Adversarial fine-tuning emerges as a prominent defense method; however, different fine-tuning strategies often produce specialized models with distinct robustness characteristics. Each fine-tuned model in turn thrives in some evaluation settings but falters on others, limiting their defensive capabilities. We refer to these specialized fine-tuned models as robust model experts and propose a collaborative adversarial fine-tuning framework: CARE - Collaborative Adversarial Robustness fine-tuning using Embedding alignment. CARE maintains multiple experts during training, enables knowledge exchange through embedding-space harmonization, and consolidates the learned knowledge into a single unified robust model. Experts benefit from one another while preserving their individual specializations, enabling the final model to inherit complementary robustness properties. In this paper, we demonstrate CARE on two different adversarial fine-tuning strategies with complementary robustness behaviors. Extensive experiments on classic image classification and downstream vision-language tasks display the effectiveness of our approach, with CARE being able to outperform individually learned model experts. The results suggest that collaborative learning across model experts is a promising direction for improving adversarial robustness.

Figures

Figures reproduced from arXiv: 2607.27897 by Hua Yu, Junhao Dong, Nguyen Duc Thai, Sua Qi Rong, Yew-Soon Ong.

Figure 1
Figure 1. Figure 1: We identify two prominent approaches to adversarial [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Our method CARE consists of 4 steps: Initialize a universal perturbation with inputs from both experts; Each expert [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: A comparison of robust accuracy on image classifica [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: t-SNE visualization of features from 5 classes in ImageNet. [PITH_FULL_IMAGE:figures/full_fig_p010_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Qualitative results for Image captioning using LLaVa. [PITH_FULL_IMAGE:figures/full_fig_p010_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

45 extracted references · 7 linked inside Pith

  1. [1]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clarket al., “Learning transferable visual models from natural language supervision,” inInternational conference on machine learning. PmLR, 2021, pp. 8748–8763. 1, 2, 3

    2021

  2. [2]

    On the adversarial robustness of multi- modal foundation models,

    C. Schlarmann and M. Hein, “On the adversarial robustness of multi- modal foundation models,” inProceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, 2023, pp. 3677–3685. 1, 2

    2023

  3. [3]

    Towards deep learning models resistant to adversarial attacks,

    A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, “Towards deep learning models resistant to adversarial attacks,”arXiv preprint arXiv:1706.06083, 2017. 1, 4

    Pith/arXiv arXiv 2017

  4. [4]

    Theoretically principled trade-off between robustness and accuracy,

    H. Zhang, Y . Yu, J. Jiao, E. Xing, L. El Ghaoui, and M. Jordan, “Theoretically principled trade-off between robustness and accuracy,” inInternational conference on machine learning. PMLR, 2019, pp. 7472–7482. 1

    2019

  5. [5]

    Understanding zero-shot adversarial robustness for large-scale models,

    C. Mao, S. Geng, J. Yang, X. Wang, and C. V ondrick, “Understanding zero-shot adversarial robustness for large-scale models,”arXiv preprint arXiv:2212.07016, 2022. 1, 2, 4, 6, 7

    Pith/arXiv arXiv 2022

  6. [6]

    Text-guided attention is all you need for zero-shot robustness in vision-language models,

    L. Yu, H. Zhang, and C. Xu, “Text-guided attention is all you need for zero-shot robustness in vision-language models,”Advances in Neural Information Processing Systems, vol. 37, pp. 96 424–96 448, 2024. 1, 2

    2024

  7. [7]

    Generalist: Decoupling natural and robust gen- eralization,

    H. Wang and Y . Wang, “Generalist: Decoupling natural and robust gen- eralization,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 20 554–20 563. 2, 10

    2023

  8. [8]

    Adversarially robust few-shot learning via parameter co-distillation of similarity and class concept learners,

    J. Dong, P. Koniusz, J. Chen, X. Xie, and Y .-S. Ong, “Adversarially robust few-shot learning via parameter co-distillation of similarity and class concept learners,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 28 535–28 544. 2, 5

    2024

  9. [9]

    Scaling up visual and vision-language representation learning with noisy text supervision,

    C. Jia, Y . Yang, Y . Xia, Y .-T. Chen, Z. Parekh, H. Pham, Q. Le, Y .-H. Sung, Z. Li, and T. Duerig, “Scaling up visual and vision-language representation learning with noisy text supervision,” inInternational conference on machine learning. PMLR, 2021, pp. 4904–4916. 2

    2021

  10. [10]

    Align before fuse: Vision and language representation learning with momentum distillation,

    J. Li, R. Selvaraju, A. Gotmare, S. Joty, C. Xiong, and S. C. H. Hoi, “Align before fuse: Vision and language representation learning with momentum distillation,”Advances in neural information processing systems, vol. 34, pp. 9694–9705, 2021. 2

    2021

  11. [11]

    Flamingo: a visual language model for few-shot learning,

    J.-B. Alayrac, J. Donahue, P. Luc, A. Miech, I. Barr, Y . Hasson, K. Lenc, A. Mensch, K. Millican, M. Reynoldset al., “Flamingo: a visual language model for few-shot learning,”Advances in neural information processing systems, vol. 35, pp. 23 716–23 736, 2022. 2

    2022

  12. [12]

    Openflamingo: An open-source framework for training large autoregressive vision-language models,

    A. Awadalla, I. Gao, J. Gardner, J. Hessel, Y . Hanafy, W. Zhu, K. Marathe, Y . Bitton, S. Gadre, S. Sagawaet al., “Openflamingo: An open-source framework for training large autoregressive vision-language models,”arXiv preprint arXiv:2308.01390, 2023. 2

    Pith/arXiv arXiv 2023

  13. [13]

    Visual instruction tuning,

    H. Liu, C. Li, Q. Wu, and Y . J. Lee, “Visual instruction tuning,” Advances in neural information processing systems, vol. 36, pp. 34 892– 34 916, 2023. 2, 8

    2023

  14. [14]

    Visual adversarial examples jailbreak large language models,

    X. Qi, K. Huang, A. Panda, M. Wang, and P. Mittal, “Visual adversarial examples jailbreak large language models,”CoRR, 2023. 2

    2023

  15. [15]

    Are aligned neural networks adversarially aligned?

    N. Carlini, M. Nasr, C. A. Choquette-Choo, M. Jagielski, I. Gao, P. W. W. Koh, D. Ippolito, F. Tramer, and L. Schmidt, “Are aligned neural networks adversarially aligned?”Advances in Neural Information Processing Systems, vol. 36, pp. 61 478–61 500, 2023. 2

    2023

  16. [16]

    On evaluating adversarial robustness of large vision-language models,

    Y . Zhao, T. Pang, C. Du, X. Yang, C. Li, N.-M. M. Cheung, and M. Lin, “On evaluating adversarial robustness of large vision-language models,”Advances in Neural Information Processing Systems, vol. 36, pp. 54 111–54 138, 2023. 2

    2023

  17. [17]

    Agent smith: A single image can jailbreak one million multimodal llm agents exponentially fast,

    X. Gu, X. Zheng, T. Pang, C. Du, Q. Liu, Y . Wang, J. Jiang, and M. Lin, “Agent smith: A single image can jailbreak one million multimodal llm agents exponentially fast,”arXiv preprint arXiv:2402.08567, 2024. 2

    Pith/arXiv arXiv 2024

  18. [18]

    Pre-trained model guided fine-tuning for zero-shot adversarial robustness,

    S. Wang, J. Zhang, Z. Yuan, and S. Shan, “Pre-trained model guided fine-tuning for zero-shot adversarial robustness,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2024, pp. 24 502–24 511. 2

    2024

  19. [19]

    Robust clip: Unsupervised adversarial fine-tuning of vision embeddings for robust large vision-language models,

    C. Schlarmann, N. D. Singh, F. Croce, and M. Hein, “Robust clip: Unsupervised adversarial fine-tuning of vision embeddings for robust large vision-language models,”arXiv preprint arXiv:2402.12336, 2024. 2, 4, 6, 7, 8

    Pith/arXiv arXiv 2024

  20. [20]

    Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time,

    M. Wortsman, G. Ilharco, S. Y . Gadre, R. Roelofs, R. Gontijo-Lopes, A. S. Morcos, H. Namkoong, A. Farhadi, Y . Carmon, S. Kornblith et al., “Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time,” inInternational conference on machine learning. PMLR, 2022, pp. 23 965–23 998. 2

    2022

  21. [21]

    Editing models with task arithmetic,

    G. Ilharco, M. T. Ribeiro, M. Wortsman, S. Gururangan, L. Schmidt, H. Hajishirzi, and A. Farhadi, “Editing models with task arithmetic,” arXiv preprint arXiv:2212.04089, 2022. 2

    Pith/arXiv arXiv 2022

  22. [22]

    Task arithmetic in the tangent space: Improved editing of pre-trained models,

    G. Ortiz-Jimenez, A. Favero, and P. Frossard, “Task arithmetic in the tangent space: Improved editing of pre-trained models,”Advances in Neural Information Processing Systems, vol. 36, pp. 66 727–66 754,

  23. [23]

    Univer- sal adversarial perturbations,

    S.-M. Moosavi-Dezfooli, A. Fawzi, O. Fawzi, and P. Frossard, “Univer- sal adversarial perturbations,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 1765–1773. 5 JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 12

    2017

  24. [24]

    Universal adversarial training,

    A. Shafahi, M. Najibi, Z. Xu, J. Dickerson, L. S. Davis, and T. Goldstein, “Universal adversarial training,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 04, 2020, pp. 5636–5643. 5

    2020

  25. [25]

    Imagenet: A large-scale hierarchical image database,

    J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in2009 IEEE conference on computer vision and pattern recognition. Ieee, 2009, pp. 248–255. 6, 7

    2009

  26. [26]

    Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks,

    F. Croce and M. Hein, “Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks,” inInternational conference on machine learning. PMLR, 2020, pp. 2206–2216. 6, 7, 8

    2020

  27. [27]

    Decoupled weight decay regularization,

    I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” inInternational Conference on Learning Representations, ICLR, 2019. 7

    2019

  28. [28]

    An analysis of single-layer networks in unsupervised feature learning,

    A. Coates, A. Ng, and H. Lee, “An analysis of single-layer networks in unsupervised feature learning,” inProceedings of the fourteenth international conference on artificial intelligence and statistics. JMLR Workshop and Conference Proceedings, 2011, pp. 215–223. 7

    2011

  29. [29]

    Learning multiple layers of features from tiny images,

    A. Krizhevsky, G. Hintonet al., “Learning multiple layers of features from tiny images,” 2009. 7

    2009

  30. [30]

    Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories,

    L. Fei-Fei, R. Fergus, and P. Perona, “Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories,” in2004 conference on computer vision and pattern recognition workshop. IEEE, 2004, pp. 178–178. 7

    2004

  31. [31]

    3d object representations for fine-grained categorization,

    J. Krause, M. Stark, J. Deng, and L. Fei-Fei, “3d object representations for fine-grained categorization,” inProceedings of the IEEE interna- tional conference on computer vision workshops, 2013, pp. 554–561. 7

    2013

  32. [32]

    Cats and dogs,

    O. M. Parkhi, A. Vedaldi, A. Zisserman, and C. Jawahar, “Cats and dogs,” in2012 IEEE conference on computer vision and pattern recog- nition. IEEE, 2012, pp. 3498–3505. 7

    2012

  33. [33]

    Automated flower classification over a large number of classes,

    M.-E. Nilsback and A. Zisserman, “Automated flower classification over a large number of classes,” in2008 Sixth Indian conference on computer vision, graphics & image processing. IEEE, 2008, pp. 722–729. 7

    2008

  34. [34]

    Fine- grained visual classification of aircraft,

    S. Maji, E. Rahtu, J. Kannala, M. Blaschko, and A. Vedaldi, “Fine- grained visual classification of aircraft,”arXiv preprint arXiv:1306.5151,

    Pith/arXiv arXiv

  35. [35]

    Describing textures in the wild,

    M. Cimpoi, S. Maji, I. Kokkinos, S. Mohamed, and A. Vedaldi, “Describing textures in the wild,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2014, pp. 3606–3613. 7

    2014

  36. [36]

    Eurosat: A novel dataset and deep learning benchmark for land use and land cover classi- fication,

    P. Helber, B. Bischke, A. Dengel, and D. Borth, “Eurosat: A novel dataset and deep learning benchmark for land use and land cover classi- fication,”IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 12, no. 7, pp. 2217–2226, 2019. 7

    2019

  37. [37]

    Rotation equivariant cnns for digital pathology,

    B. S. Veeling, J. Linmans, J. Winkens, T. Cohen, and M. Welling, “Rotation equivariant cnns for digital pathology,” inMedical Image Computing and Computer Assisted Intervention–MICCAI 2018: 21st International Conference, Granada, Spain, September 16-20, 2018, Proceedings, Part II 11. Springer, 2018, pp. 210–218. 7

    2018

  38. [38]

    The many faces of robustness: A critical analysis of out- of-distribution generalization,

    D. Hendrycks, S. Basart, N. Mu, S. Kadavath, F. Wang, E. Dorundo, R. Desai, T. Zhu, S. Parajuli, M. Guo, D. Song, J. Steinhardt, and J. Gilmer, “The many faces of robustness: A critical analysis of out- of-distribution generalization,”ICCV, 2021. 7

    2021

  39. [39]

    Learning robust global representations by penalizing local predictive power,

    H. Wang, S. Ge, Z. Lipton, and E. P. Xing, “Learning robust global representations by penalizing local predictive power,” inAdvances in Neural Information Processing Systems, 2019, pp. 10 506–10 518. 7

    2019

  40. [40]

    Microsoft coco: Common objects in context,

    T.-Y . Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Doll ´ar, and C. L. Zitnick, “Microsoft coco: Common objects in context,” inEuropean conference on computer vision. Springer, 2014, pp. 740–755. 8

    2014

  41. [41]

    Flickr30k entities: Collecting region-to-phrase correspondences for richer image-to-sentence models,

    B. A. Plummer, L. Wang, C. M. Cervantes, J. C. Caicedo, J. Hocken- maier, and S. Lazebnik, “Flickr30k entities: Collecting region-to-phrase correspondences for richer image-to-sentence models,” inProceedings of the IEEE international conference on computer vision, 2015, pp. 2641–

    2015

  42. [42]

    Towards vqa models that can read,

    A. Singh, V . Natarajan, M. Shah, Y . Jiang, X. Chen, D. Batra, D. Parikh, and M. Rohrbach, “Towards vqa models that can read,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 8317–8326. 8

    2019

  43. [43]

    Making the v in vqa matter: Elevating the role of image understanding in visual question answering,

    Y . Goyal, T. Khot, D. Summers-Stay, D. Batra, and D. Parikh, “Making the v in vqa matter: Elevating the role of image understanding in visual question answering,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 6904–6913. 8

    2017

  44. [44]

    Cider: Consensus- based image description evaluation,

    R. Vedantam, C. Lawrence Zitnick, and D. Parikh, “Cider: Consensus- based image description evaluation,” inProceedings of the IEEE confer- ence on computer vision and pattern recognition, 2015, pp. 4566–4575. 8

    2015

  45. [45]

    Vqa: Visual question answering,

    S. Antol, A. Agrawal, J. Lu, M. Mitchell, D. Batra, C. L. Zitnick, and D. Parikh, “Vqa: Visual question answering,” inProceedings of the IEEE international conference on computer vision, 2015, pp. 2425–2433. 8

    2015