Pith. sign in

REVIEW 2 major objections 5 minor 1 cited by

Context-Aware Multimodal Pretraining

T0 review · 2 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Adding a context-aware cross-attention term to contrastive image-text pretraining makes vision-language models substantially better at few-shot adaptation—up to fourfold sample efficiency across 21 benchmarks—without sacrificing zero-shot…

desk verdict Well-ablated pretraining modification that genuinely helps metric-based few-shot adaptation, but the low-shot headline is oversold and the SOTA comparison is confounded. read the letter →

arxiv 2411.15099 v1 pith:OSJHYCOF submitted 2024-11-22 cs.CV cs.CLcs.LG

Karsten Roth , Zeynep Akata , Dima Damen , Ivana Balažević , Olivier J. Hénaff This is my paper · ORCID
classification cs.CVcs.CLcs.LG
keywords vision-languagepretrainingfew-shotlearningcontrastivecross-attentioncontextualizationSigLIPCLIPtraining-freeadaptationsampleefficiency
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Contrastive image-text pretraining produces representations that transfer zero-shot well, but the standard objective never asks the model to use a small labeled support set at test time. This paper proposes a simple extension that prepares representations for metric-based few-shot adaptation during pretraining itself: the LIxP objective keeps the standard contrastive loss and adds a second term in which each image representation is contextualized by cross-attending over the other images in its training batch, which the authors interpret as implicit per-iteration episodic training. On 21 downstream classification benchmarks, models trained this way improve average few-shot performance by over 5% and reach the same accuracy with up to four times fewer support examples, while zero-shot transfer is retained within about half a percent. Equipped with training-free, metric-based classifiers, these representations match or beat optimization-based adaptation schemes, which would make adapting a single frozen model to new domains cheaper and simpler.

What carries the argument

The load-bearing mechanism is the in-batch cross-attention contextualization of Eq. (8): each normalized image embedding $x_i$ queries a key buffer $M_K$ holding the embeddings of every other image in the training batch, and the softmax similarities retrieve value embeddings $M_V$ (the unnormalized batch embeddings), yielding $x^{\mathrm{ctx}}_i = \sigma\!\left(\frac{M \odot X_B X_B^{\mathrm{T}}}{\tau_{\mathrm{ctx}}\sqrt{d}}\right)\hat{X}_B$, with the self-attention diagonal masked off by $M = 1 - I_{\infty}$. This turns each training iteration into an implicit episodic few-shot task: the image must predict its matching text using evidence drawn from other images in the same batch. The objective (Eq. (6)) keeps this contextualized loss separate from the base contrastive loss and gives it its own learnable temperature $\tau_2$, which the ablations show is required for retaining zero-shot transfer while gaining few-shot ability.

What would settle it

Evaluate a SigLIxP-pretrained ViT-S/16 and an equivalently trained SigLIP ViT-S/16 with Tip-Adapter on support sets that are deliberately class-imbalanced (e.g., 32 shots for one class and 1 shot for another) or drawn from a domain far outside the pretraining distribution. If the contextualized model's advantage over the SigLIP baseline disappears or reverses on such support sets, the claim that LIxP builds general training-free few-shot capability would be falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is that the standard language-image contrastive pretraining objective can be extended with a context-aware term, Eq. (6): $\mathcal{L}_{\mathrm{LIxP}} = \alpha \mathcal{L}_{\mathrm{LIP}}(X_B, T_B, \tau_1) + (1-\alpha)\mathcal{L}_{\mathrm{LIP}}(X^{\mathrm{ctx}}_B, T_B, \tau_2)$, where $X^{\mathrm{ctx}}_B$ are batch image representations contextualized by cross-attending over the whole batch according to Eq. (8). The training batch acts as a stand-in for the support set a user will provide at test time, and the self-attention diagonal is masked so each image must draw information from its peers. Across ViT-S/B/L backbones and 1.5B to 15B training examples, on 21 few- and many-shot classification datasets, the paper reports up to fourfold improvements in sample efficiency and average few-shot gains above 5% for a range of metric-based classifiers, while zero-shot performance shifts by no more than roughly half a percent. With a training-free softmax-voted nearest-neighbor classifier, the pretrained backbone outperforms published prompt-learning and adapter methods on 16-shot ImageNet, DTD, Food101, Cars, and Pets.

Load-bearing premise

The load-bearing premise is that a large unlabeled batch of pretraining images is a faithful proxy for the small, labeled, class-balanced support sets that appear at test time; if context handling learned from batch statistics does not transfer across that distribution gap, the reported few-shot gains would not generalize.

Editorial extensions

If this is right

  • Across 21 benchmarks, every metric-based classifier tested improves, from +1.7% for rank-voting to +5.4% for Tip-Adapter at 32 shots on ViT-S/16.
  • A frozen SigLIxP backbone with a softmax-voted nearest-neighbor classifier reaches 77.9% on ImageNet 16-shot, beating published prompt-learning and adapter baselines (best cited 74.7%) without any test-time optimization.
  • Context-aware post-training works: adding 0.5B LIxP examples to a 1.5B-pretrained SigLIP model matches a 6B-example SigLIP baseline for 32-shot accuracy, and 1B extra examples surpasses it while zero-shot is retained.
  • Gains persist across model scale, training duration, and both SigLIP and CLIP objectives, with zero-shot changes within roughly ±0.5%.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An extension the paper leaves implicit: the contextualization buffer need not equal the training batch, so populating $M_K$ and $M_V$ from a curated pool whose class balance matches the expected deployment support sets is a direct test of whether the mechanism truly mimics episodic adaptation.
  • The buffer is image-only and unlabeled, so the same contextualization term could in principle be ported to self-supervised visual pretraining without any text captions; the paper only demonstrates the image-text setting.
  • The reported scaling of relative gains with absolute example counts suggests the benefit grows with support-set size, so practitioners should expect the largest gains at many-shot regimes rather than at 1-shot.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper proposes LIxP, a modification of contrastive image-text pretraining (SigLIP and CLIP) in which the standard contrastive loss is supplemented by a contextual loss computed on cross-attention-averaged image representations over the training batch (Eqs. 5-8). The contextualized representations are used only during training; at test time, the resulting encoders are evaluated with training-free metric-based classifiers (prototypical, Tip-Adapter, nearest-neighbor variants) on few- and many-shot classification. Across 21 datasets and several model scales (ViT-S/B/L) and training durations (1.5B-15B examples), the authors report large few-shot gains, retained zero-shot transfer, and claim that their training-free method surpasses optimization-based adaptation baselines.

Significance. If the reported results hold, LIxP is a practically valuable and conceptually simple pretraining change: it directly optimizes for a property (few-shot adaptability) that is usually left to emerge from contrastive learning, and it maintains zero-shot performance. The paper's strengths include extensive internal comparisons under identical training settings across architectures, data scales, and both SigLIP and CLIP objectives; systematic ablations of the loss design (Table 4) and buffer design (Table 5); and evaluation over 21 datasets with six different metric-based adaptation methods. These internal comparisons are consistent and are the most convincing part of the paper. The main weaknesses are that the headline comparison against optimization-based methods (Table 2) compares across different pretrained models rather than isolating the adaptation method, and that the transfer from the 32k-image unlabeled pretraining buffer to small labeled class-balanced support sets is least supported in the low-shot regime, which is precisely the regime emphasized by the abstract.

major comments (2)
  1. [§4.1, Eqs. (6)–(8), Fig. 5] The pretraining context in Eq. (8) is a 32,768-image, unlabeled, non-class-balanced batch, whereas the evaluation protocol in §3.1.2 uses small, labeled, class-balanced support sets. Figure 5 shows that relative gains increase with absolute support-set size and peak near 10^4 examples, i.e., close to the pretraining buffer size. The headline 'four-fold sample efficiency' (Fig. 1) is a comparison of an 8-shot SigLIxP result against a 32-shot SigLIP result; it does not establish gains at 1–4 shots. The authors should report the full shot sweep (1, 2, 4, 8, 16, 32) for the 21-dataset average and for each adaptation method, and should qualify the 'few-shot' claim if the benefit is concentrated in many-shot regimes. This is load-bearing because the paper's central claim is that the learned context handling transfers from pretraining batch statistics to test-time support sets.
  2. [Table 2, §4.1] The comparison against optimization-based methods is confounded by the fact that SigLIxP is a newly pretrained model, while the cited baselines (e.g., Tip-Adapter-F, MaPLe, PromptSRC, DMN) are applied to a standard CLIP ViT-B/16. The caption's 'same backbone' refers to architecture, not to the same pretrained weights or pretraining data. Consequently, Table 2 does not establish that the training-free mechanism surpasses optimization-based adaptation on equal footing; it shows that a different pretraining objective plus a training-free classifier outperforms literature numbers for a different base model. To support the 'closes the gap' claim, the authors should either run the optimization-based baselines on the SigLIxP features, or clearly frame Table 2 as a cross-model comparison and move the gap-closing conclusion to a more modest statement.
minor comments (5)
  1. [§3.2, Eq. (8)] The definition of M = 1 − I∞ is confusing; please state explicitly that the diagonal entries of the attention matrix are set to zero before the softmax, and clarify the phrase 'ones-mask with −∞-diagonals'.
  2. [Table 2] The reference for 'Linear Probe' appears to be [92] (CasPL), which is not a linear-probe paper; please correct the citation and re-check all references in that table.
  3. [Appendix A, Table 3] The statement that 'we directly transfer the same hyperparameters' for CLIxP conflicts with the appendix, which specifies α=0.9 for 1.5B and 6B examples but α=0.95 for 15B runs; please clarify how α was chosen and whether the main claims are sensitive to this choice.
  4. [§4.1, Fig. 5] Please describe the regression model and the 'certain confounds (such as high base performance)' adjustment; the current p-value fit is not specified enough to interpret.
  5. [Abstract] The phrase 'average few-shot adaptation gains of over 5%' should specify the adaptation method and shot count (e.g., default Tip-Adapter at 32 shots on ViT-S/16), because Table 1 reports an average gain of +3.2% across all six methods.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the contextualized pretraining objective is a genuine training surrogate, not a fitted restatement of the test-time metric-based evaluation.

full rationale

The central derivation chain is Eq. (6) plus Eq. (8) producing improved few-shot adaptation as measured by Eqs. (3) and (4). The contextualization loss trains image representations to align with paired text after cross-attending over the current unlabeled batch, with buffer M_K = M_V = phi_I(B_I). The test-time methods are fixed, training-free, and operate on the base representation x_test together with the labeled support set X_spt; they are not optimized by the pretraining loss and are not identical to the training surrogate. The training buffer contains no labels, no class-balanced episodes, and no task structure, whereas the evaluation support sets are small, labeled, and class-balanced, so the reported gains are not forced by construction. The paper provides independent empirical grounding through 21 held-out datasets, six metric-based adaptation methods, several model scales, and both SigLIP and CLIP objectives, and it reports retained zero-shot performance. Self-citations such as [16] for training hyperparameters and [1] for related in-context work are not load-bearing: they do not define the objective, supply a uniqueness argument, or forbid alternative designs. The closest concern, that train-time batch context statistics differ from test-time support-set statistics, is a generalization and external-validity limitation rather than a circularity, and the paper's Fig. 5 candidly shows that relative gains grow with absolute support-set size. No step in the derivation reduces to its own input or to a fitted parameter renamed as a prediction.

Assumptions & free parameters 1 free parameters · 3 assumptions · 1 invented entities

The central claim rests on the proxy assumption that random in-batch images simulate test-time support sets, on standard contrastive pretraining assumptions, and on the representativeness of the 21 benchmark datasets. The main tuned free parameter is the loss weight α; learnable temperatures and fixed evaluation hyperparameters are not fitted to the reported results.

free parameters (1)
  • Loss weighting α = 0.9 for 1.5B/6B runs, 0.95 for 15B runs
    Controls the tradeoff between base contrastive loss and contextualized loss (Eq. 6); chosen from the ablation in Table 4b and slightly increased for the longest run, so the main results depend on this hand-tuned value.
assumptions (3)
  • ad hoc to paper The in-batch image context used during pretraining is a sufficient proxy for downstream support sets, so the learned contextualization transfers to test time.
    Section 3.2 describes the buffer as 'implicit per-iteration episodic training'; the paper does not demonstrate transfer across the gap between 32k unlabeled batch contexts and small labeled support sets.
  • domain assumption The SigLIP and CLIP contrastive objectives, with L2-normalized embeddings, are a valid foundation for training-free metric-based adaptation.
    Section 3.1 assumes cosine similarity is the right metric and uses standard pretraining recipes from Zhai et al. and Radford et al.
  • domain assumption WebLI pretraining followed by evaluation on 21 held-out datasets is representative of general few-shot transfer behavior.
    Section 4 uses WebLI and 21 datasets; no guarantee of transfer to other distributions, but standard practice.
invented entities (1)
  • Contextualization buffer (MK, MV) independent evidence
    purpose: Provides a key-value image memory during pretraining so that each image is contextualized by a softmax-weighted combination of batch neighbors, simulating test-time support sets.
    A new component introduced in Sec. 3.2; its effect is empirically measured on 21 downstream datasets and in buffer ablations (Tables 4e, 5), providing evidence outside the training objective itself.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Context-Aware Multimodal Pretraining." pith.science (2026). https://pith.science/paper/OSJHYCOF

@misc{pith2026241115099,
  author       = {Pith},
  title        = {Pith review of: Context-Aware Multimodal Pretraining},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OSJHYCOF}},
  note         = {Machine review of arXiv:2411.15099}
}
read the original abstract

Large-scale multimodal representation learning successfully optimizes for zero-shot transfer at test time. Yet the standard pretraining paradigm (contrastive learning on large amounts of image-text data) does not explicitly encourage representations to support few-shot adaptation. In this work, we propose a simple, but carefully designed extension to multimodal pretraining which enables representations to accommodate additional context. Using this objective, we show that vision-language models can be trained to exhibit significantly increased few-shot adaptation: across 21 downstream tasks, we find up to four-fold improvements in test-time sample efficiency, and average few-shot adaptation gains of over 5%, while retaining zero-shot generalization performance across model scales and training durations. In particular, equipped with simple, training-free, metric-based adaptation mechanisms, our representations easily surpass more complex and expensive optimization-based schemes, vastly simplifying generalization to new domains.

Figures

Figures reproduced from arXiv: 2411.15099 by the authors.

Figure 1
Figure 1. Context-aware multimodal pretraining facilitates few-shot transfer. Applying Tip-Adapter [101] on a ViT-S/16 pretrained with and without our contextualized pretraining objec￾tive (here modifying SigLIP [99]) showcases increases in test-time sample efficiency and overall few-shot performance while main￾taining the underlying zero-shot transfer performance. same time, representation learning for few-shot learning on s… view at source ↗
Figure 2
Figure 2. Dataset-level performance breakdown (32-shot, Tip￾Adapter [101]) for ViT-S/16 shows gains up to +16.2% on all 21 benchmarks; with each dataset improving by at least +1.0%. 4. Experiments Our pretraining pipeline follows the SigLIP [99] protocols for training ViT-(S/16, B/16, L/16) image encoders and cor￾respondingly sized BERT-{S,B,L} text encoders. To evalu￾ate both zero-shot as well as few-shot transfer capabiliti… view at source ↗
Figure 3
Figure 3. Significant gains across metric-based few-shot classifiers. Applying prototypical classification [76], Tip￾Adapters [101] and nearest neighbor classifiers [22, 54] on vision￾backbones using context-aware pretraining significantly boosts 32-shot results across the board (here ViT-S/16, 1.5B ex.). Model → ViT-S/16 → ViT-B/16 → ViT-L/16 Examples → 1.5B 6B 6B 15B 8B ZeroShot 46.9 52.1 60.3 62.5 64.1 +0.4 −0.2 −0.4 −0.5 … view at source ↗
Figures from the paper (2 more)
Figure 5
Figure 5. Figure 5: Contextualized pretraining particularly benefits many-shot transfer. For all 21 evaluation benchmarks, we plot the absolute number of examples (shots/class × #classes) against the relative gain when switching to SigLIxP. Results shown are for 32 shots/class. We find co…
Figure 6
Figure 6. Figure 6: Training Dynamics. (Left) Relation between the base SigLIP training objective LSigLIP and its contextualized counterpart L ctx SigLIP (here referring to the second term in Eq. (6)) for a ViT-B/16 model trained on 6B WebLI examples using SigLIxP. We jointly visualize th…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. How to Merge Your Multimodal Models Over Time?

    cs.LG 2024-12 conditional novelty 6.0 of 10

    A systematic study of temporal model merging shows that initialization and deployment choices matter far more than the merging technique, with EMA-style weight interpolation as the best practice.

Reference graph

Works this paper leans on

113 extracted references · 62 canonical work pages · cited by 1 Pith paper

  1. [1]

    Towards in-context scene understanding

    Ivana Balazevic, David Steiner, Nikhil Parthasarathy, Relja Arandjelovic, and Olivier J Henaff. Towards in-context scene understanding. In Thirty-seventh Conference on Neu- ral Information Processing Systems, 2023. 1

    2023

  2. [2]

    Food-101 – mining discriminative components with ran- dom forests

    Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. Food-101 – mining discriminative components with ran- dom forests. In European Conference on Computer Vision,

  3. [3]

    JAX: composable transformations of Python+NumPy programs, 2018

    James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne, and Qiao Zhang. JAX: composable transformations of Python+NumPy programs, 2018. 1

    2018

  4. [4]

    Emerging properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv ´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In Proceedings of the International Conference on Com- puter Vision (ICCV), 2021. 3, 1

    2021

  5. [5]

    A simple framework for contrastive learning of visual representations

    Ting Chen, Simon Kornblith, Mohammad Norouzi, and Ge- offrey Hinton. A simple framework for contrastive learning of visual representations. In Proceedings of the 37th In- ternational Conference on Machine Learning, pages 1597–

  6. [6]

    PaLI: A jointly- scaled multilingual language-image model

    Xi Chen, Xiao Wang, Soravit Changpinyo, AJ Piergio- vanni, Piotr Padlewski, Daniel Salz, Sebastian Goodman, Adam Grycner, Basil Mustafa, Lucas Beyer, Alexander Kolesnikov, Joan Puigcerver, Nan Ding, Keran Rong, Has- san Akbari, Gaurav Mishra, Linting Xue, Ashish V Thap- liyal, James Bradbury, Weicheng Kuo, Mojtaba Seyedhos- seini, Chao Jia, Burcu Karagol ...

  7. [7]

    Meta-baseline: Exploring simple meta- learning for few-shot learning

    Yinbo Chen, Zhuang Liu, Huijuan Xu, Trevor Darrell, and Xiaolong Wang. Meta-baseline: Exploring simple meta- learning for few-shot learning. In 2021 IEEE/CVF Inter- national Conference on Computer Vision (ICCV) , pages 9042–9051, 2021. 1, 2

    2021

  8. [8]

    Remote sens- ing image scene classification: Benchmark and state of the art

    Gong Cheng, Junwei Han, and Xiaoqiang Lu. Remote sens- ing image scene classification: Benchmark and state of the art. Proceedings of the IEEE, 105(10):1865–1883, 2017. 1, 2

    2017

Show all 113 references
  1. [9]

    Cimpoi, S

    M. Cimpoi, S. Maji, I. Kokkinos, S. Mohamed, and A. Vedaldi. Describing textures in the wild. In Proceedings of the IEEE Conf. on Computer Vision and Pattern Recog- nition (CVPR), 2014. 1, 2

    2014

  2. [10]

    Embedding arithmetic of multi- modal queries for image retrieval

    Guillaume Couairon, Matthijs Douze, Matthieu Cord, and Holger Schwenk. Embedding arithmetic of multi- modal queries for image retrieval. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pages 4950–4958, 2022. 1

    2022

  3. [11]

    J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. ImageNet: A Large-Scale Hierarchical Image Database. In CVPR09, 2009. 1, 2

    2009

  4. [12]

    Calibrated cache model for few- shot vision-language model adaptation

    Kun Ding, Qiang Yu, Haojian Zhang, Gaofeng Meng, and Shiming Xiang. Calibrated cache model for few- shot vision-language model adaptation. arXiv preprint arXiv:2410.08895, 2024. 2

    2024 arXiv

  5. [13]

    An im- age is worth 16x16 words: Transformers for image recog- nition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An im- age is worth 16x16 words: Transformers for image recog- nitio...

    2021

  6. [14]

    With a little help from my friends: Nearest-neighbor contrastive learning of vi- sual representations

    Debidatta Dwibedi, Yusuf Aytar, Jonathan Tompson, Pierre Sermanet, and Andrew Zisserman. With a little help from my friends: Nearest-neighbor contrastive learning of vi- sual representations. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision (ICCV), ...

    2021

  7. [15]

    Bad students make great teachers: Active learning acceler- ates large-scale visual understanding

    Talfan Evans, Shreya Pathak, Hamza Merzic, Jonathan Schwarz, Ryutaro Tanno, and Olivier J Henaff. Bad students make great teachers: Active learning acceler- ates large-scale visual understanding. arXiv preprint arXiv:2312.05328, 2023. 2

    2023 arXiv

  8. [16]

    Data curation via joint example selec- tion further accelerates multimodal learning

    Talfan Evans, Nikhil Parthasarathy, Hamza Merzic, and Olivier J Henaff. Data curation via joint example selec- tion further accelerates multimodal learning. arXiv preprint arXiv:2406.17711, 2024. 2, 1

    2024 arXiv

  9. [17]

    Data determines distributional robustness in contrastive lan- guage image pre-training (clip)

    Alex Fang, Gabriel Ilharco, Mitchell Wortsman, Yuhao Wan, Vaishaal Shankar, Achal Dave, and Ludwig Schmidt. Data determines distributional robustness in contrastive lan- guage image pre-training (clip). In ICML, pages 6216–

  10. [18]

    Cap- tion supervision enables robust learners

    Benjamin Feuer, Ameya Joshi, and Chinmay Hegde. Cap- tion supervision enables robust learners. arXiv preprint arXiv:2210.07396, 2022. 1

    2022 arXiv

  11. [19]

    Context-aware meta-learning

    Christopher Fifty, Dennis Duan, Ronald Guenther Junkins, Ehsan Amid, Jure Leskovec, Christopher Re, and Sebastian Thrun. Context-aware meta-learning. In The Twelfth Inter- national Conference on Learning Representations, 2024. 2

    2024

  12. [20]

    Model- agnostic meta-learning for fast adaptation of deep networks

    Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model- agnostic meta-learning for fast adaptation of deep networks. In Proceedings of the 34th International Conference on Ma- chine Learning - Volume 70 , page 1126–1135. JMLR.org,

  13. [21]

    Clip-adapter: Better vision-language models with feature adapters

    Peng Gao, Shijie Geng, Renrui Zhang, Teli Ma, Rongyao Fang, Yongfeng Zhang, Hongsheng Li, and Yu Qiao. Clip-adapter: Better vision-language models with feature adapters. Int. J. Comput. Vision, 132(2):581–595, 2023. 6

    2023

  14. [22]

    Towards flexible perception with visual memory

    Robert Geirhos, Priyank Jaini, Austin Stone, Sourabh Medapati, Xi Yi, George Toderici, Abhijit Ogale, and Jonathon Shlens. Towards flexible perception with visual memory. arXiv preprint arXiv:2408.08172, 2024. 1, 2, 3, 5

    2024 arXiv

  15. [23]

    Cyclip: Cyclic con- trastive language-image pretraining

    Shashank Goel, Hritik Bansal, Sumit Bhatia, Ryan Rossi, Vishwa Vinay, and Aditya Grover. Cyclip: Cyclic con- trastive language-image pretraining. In Advances in Neural Information Processing Systems, pages 6704–6719. Curran Associates, Inc., 2022. 2

    2022

  16. [24]

    kNN-CLIP: Retrieval enables training-free segmenta- tion on continually expanding large vocabularies

    Zhongrui Gui, Shuyang Sun, Runjia Li, Jianhao Yuan, Zhaochong An, Karsten Roth, Ameya Prabhu, and Philip Torr. kNN-CLIP: Retrieval enables training-free segmenta- tion on continually expanding large vocabularies. Transac- tions on Machine Learning Research, 2024. 1

    2024

  17. [25]

    Calip: zero-shot enhancement of clip with parameter-free attention

    Ziyu Guo, Renrui Zhang, Longtian Qiu, Xianzheng Ma, Xupeng Miao, Xuming He, and Bin Cui. Calip: zero-shot enhancement of clip with parameter-free attention. In Pro- ceedings of the Thirty-Seventh AAAI Conference on Artifi- cial Intelligence and Thirty-Fifth Conference on Innov...

    2023

  18. [26]

    Anchor-based robust finetuning of vision-language models

    Jinwei Han, Zhiwen Lin, Zhongyisun Sun, Yingguo Gao, Ke Yan, Shouhong Ding, Yuan Gao, and Gui-Song Xia. Anchor-based robust finetuning of vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 26919– 26928, 2024. 1

    2024

  19. [27]

    Dota: Dis- tributional test-time adaptation of vision-language models

    Zongbo Han, Jialong Yang, Junfan Li, Qinghua Hu, Qianli Xu, Mike Zheng Shou, and Changqing Zhang. Dota: Dis- tributional test-time adaptation of vision-language models. arXiv preprint arXiv:2409.19375, 2024. 2

    2024

  20. [28]

    Momentum contrast for unsu- pervised visual representation learning, 2019

    Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsu- pervised visual representation learning, 2019. cite arxiv:1911.05722Comment: CVPR 2020 camera-ready. Code: https://github.com/facebookresearch/moco. 2

    2019 arXiv

  21. [29]

    Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification

    Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12(7):2217–2226, 2019. 1, 2

    2019

  22. [30]

    Ross, and Alireza Fathi

    Ziniu Hu, Ahmet Iscen, Chen Sun, Zirui Wang, Kai-Wei Chang, Yizhou Sun, Cordelia Schmid, David A. Ross, and Alireza Fathi. Reveal: Retrieval-augmented visual- language pre-training with multi-source multimodal knowl- edge memory. In Proceedings of the IEEE/CVF Confer- ence on ...

    2023

  23. [31]

    Hughes and Marcel Salath ´e

    David P. Hughes and Marcel Salath ´e . An open access repository of images on plant health to enable the develop- ment of mobile disease diagnostics through machine learn- ing and crowdsourcing. CoRR, abs/1511.08060, 2015. 1, 2

    2015 arXiv

  24. [32]

    Retrieval-enhanced contrastive vision-text mod- els

    Ahmet Iscen, Mathilde Caron, Alireza Fathi, and Cordelia Schmid. Retrieval-enhanced contrastive vision-text mod- els. In The Twelfth International Conference on Learning Representations, 2024. 2

    2024

  25. [33]

    Scaling up visual and vision-language representation learning with noisy text supervision

    Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representation learning with noisy text supervision. In Pro- ceedings of the 38th International Conference on Machine ...

    2021

  26. [34]

    Billion- scale similarity search with gpus

    Jeff Johnson, Matthijs Douze, and Herv ´e J ´egou. Billion- scale similarity search with gpus. IEEE Transactions on Big Data, 7(3):535–547, 2021. 1

    2021

  27. [35]

    Multi-class texture analysis in colorectal cancer histology

    Jakob Nikolas Kather, Cleo-Aron Weis, Francesco Bian- coni, Susanne M Melchers, Lothar R Schad, Timo Gaiser, Alexander Marx, and Frank Gerrit Z”ollner. Multi-class texture analysis in colorectal cancer histology. Scientific reports, 6:27988, 2016. 1, 2

    2016

  28. [36]

    Maple: Multi-modal prompt learning

    Muhammad Uzair Khattak, Hanoona Rasheed, Muhammad Maaz, Salman Khan, and Fahad Shahbaz Khan. Maple: Multi-modal prompt learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 19113–19122, 2023. 6

    2023

  29. [37]

    Self-regulating prompts: Foundational model adaptation without forgetting

    Muhammad Uzair Khattak, Syed Talal Wasim, Muzam- mal Naseer, Salman Khan, Ming-Hsuan Yang, and Fa- had Shahbaz Khan. Self-regulating prompts: Foundational model adaptation without forgetting. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pa...

    2023

  30. [38]

    Datadream: Few-shot guided dataset generation

    Jae Myung Kim, Jessica Bader, Stephan Alaniz, Cordelia Schmid, and Zeynep Akata. Datadream: Few-shot guided dataset generation. In European Conference on Computer Vision, pages 252–268. Springer, 2025. 2

    2025

  31. [39]

    Kirchhof, K

    M. Kirchhof, K. Roth, Z. Akata, and E. Kasneci. A non-isotropic probabilistic take on proxy-based deep met- ric learning. In Computer Vision - ECCV 2022 - 17th Euro- pean Conference, Proceedings, Part XXVI, pages 435–454. Springer, 2022. 4

    2022

  32. [40]

    Wilds: A benchmark of in-the-wild distri- bution shifts

    Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, et al. Wilds: A benchmark of in-the-wild distri- bution shifts. InInternational conference on machine learn- ing...

    2021

  33. [41]

    3d object representations for fine-grained categorization

    Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine-grained categorization. In 4th International IEEE Workshop on 3D Representation and Recognition (3dRR-13), Sydney, Australia, 2013. 1, 2

    2013

  34. [42]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, 2009. 1, 2

    2009

  35. [43]

    SentencePiece: A sim- ple and language independent subword tokenizer and deto- kenizer for neural text processing

    Taku Kudo and John Richardson. SentencePiece: A sim- ple and language independent subword tokenizer and deto- kenizer for neural text processing. In Proceedings of the 2018 Conference on Empirical Methods in Natural Lan- guage Processing: System Demonstrations , pages 66–71, B...

    2018

  36. [44]

    Meta-learning with differentiable con- vex optimization

    Kwonjoon Lee, Subhransu Maji, Avinash Ravichandran, and Stefano Soatto. Meta-learning with differentiable con- vex optimization. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR),

  37. [45]

    Universal representation learning from multiple domains for few- shot classification

    Wei-Hong Li, Xialei Liu, and Hakan Bilen. Universal representation learning from multiple domains for few- shot classification. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision (ICCV), pages 9526–9535, 2021. 2

    2021

  38. [46]

    The devil is in the few shots: It- erative visual knowledge completion for few-shot learning

    Yaohui Li, Qifeng Zhou, Haoxing Chen, Jianbing Zhang, Xinyu Dai, and Hao Zhou. The devil is in the few shots: It- erative visual knowledge completion for few-shot learning. arXiv preprint arXiv:2404.09778, 2024. 2

    2024 arXiv

  39. [47]

    Decoupled weight de- cay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight de- cay regularization. In International Conference on Learn- ing Representations, 2019. 1

    2019

  40. [48]

    A closer look at few-shot classification again

    Xu Luo, Hao Wu, Ji Zhang, Lianli Gao, Jing Xu, and Jingkuan Song. A closer look at few-shot classification again. In Proceedings of the 40th International Conference on Machine Learning , pages 23103–23123. PMLR, 2023. 1, 2

    2023

  41. [49]

    Efficient and ro- bust approximate nearest neighbor search using hierarchi- cal navigable small world graphs

    Yury A Malkov and Dmitry A Yashunin. Efficient and ro- bust approximate nearest neighbor search using hierarchi- cal navigable small world graphs. IEEE transactions on pattern analysis and machine intelligence, 2018. 1

    2018

  42. [50]

    Visual classification via description from large language models

    Sachit Menon and Carl V ondrick. Visual classification via description from large language models. In The Eleventh International Conference on Learning Represen- tations, 2023. 1, 2

    2023

  43. [51]

    Understanding retrieval- augmented task adaptation for vision-language models

    Yifei Ming and Yixuan Li. Understanding retrieval- augmented task adaptation for vision-language models. In Proceedings of the 41st International Conference on Ma- chine Learning, pages 35719–35743. PMLR, 2024. 2

    2024

  44. [52]

    Slip: Self-supervision meets language-image pre- training

    Norman Mu, Alexander Kirillov, David Wagner, and Sain- ing Xie. Slip: Self-supervision meets language-image pre- training. In Computer Vision – ECCV 2022: 17th Euro- pean Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XXVI, page 529–544, Berlin, Heidelberg,

    2022

  45. [53]

    icassava 2019 fine- grained visual categorization challenge

    Ernest Mwebaze, Timnit Gebru, Andrea Frome, Solomon Nsumba, and Jeremy Tusubira. icassava 2019 fine- grained visual categorization challenge. arXiv preprint arXiv:1908.02900, 2019. 1, 2

    2019 arXiv

  46. [54]

    Revisiting knn- based image classification system with high-capacity stor- age

    Kengo Nakata, Youyang Ng, Daisuke Miyashita, Asuka Maki, Yu-Chieh Lin, and Jun Deguchi. Revisiting knn- based image classification system with high-capacity stor- age. In Computer Vision – ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Pro- ceeding...

    2022

  47. [55]

    On first-order meta-learning algorithms

    A Nichol. On first-order meta-learning algorithms. arXiv preprint arXiv:1803.02999, 2018. 1, 2

    2018 arXiv

  48. [56]

    CHiLS: Zero-shot image classifica- tion with hierarchical label sets

    Zachary Novack, Julian Mcauley, Zachary Chase Lipton, and Saurabh Garg. CHiLS: Zero-shot image classifica- tion with hierarchical label sets. In Proceedings of the 40th International Conference on Machine Learning, pages 26342–26362. PMLR, 2023. 2

    2023

  49. [57]

    Repre- sentation learning with contrastive predictive coding

    Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Repre- sentation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748, 2018. 2

    2018 arXiv

  50. [58]

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V . V o, Marc Szafraniec, Vasil Khalidov, Pierre Fer- nandez, Daniel HAZIZA, Francisco Massa, Alaaeldin El- Nouby, Mido Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Mich...

    2024

  51. [59]

    Svl-adapter: Self-supervised adapter for vision-language pretrained models

    Omiros Pantazis, Gabriel Brostow, Kate Jones, and Oisin Mac Aodha. Svl-adapter: Self-supervised adapter for vision-language pretrained models. In British Machine Vi- sion Conference (BMVC), 2022. 1, 2

    2022

  52. [60]

    O. M. Parkhi, A. Vedaldi, A. Zisserman, and C. V . Jawahar. Cats and dogs. In IEEE Conference on Computer Vision and Pattern Recognition, 2012. 1, 2

    2012

  53. [61]

    Moment matching for multi-source domain adaptation

    Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. Moment matching for multi-source domain adaptation. In Proceedings of the IEEE Interna- tional Conference on Computer Vision , pages 1406–1415,

  54. [62]

    Hieu Pham, Zihang Dai, Golnaz Ghiasi, Kenji Kawaguchi, Hanxiao Liu, Adams Wei Yu, Jiahui Yu, Yi-Ting Chen, Minh-Thang Luong, Yonghui Wu, Mingxing Tan, and Quoc V . Le. Combined scaling for zero-shot transfer learn- ing. Neurocomput., 555(C), 2023. 2

    2023

  55. [63]

    Online contin- ual learning without the storage constraint

    Ameya Prabhu, Zhipeng Cai, Puneet Dokania, Philip Torr, Vladlen Koltun, and Ozan Sener. Online contin- ual learning without the storage constraint. arXiv preprint arXiv:2305.09253, 2023. 1

    2023 arXiv

  56. [64]

    What does a platypus look like? generating customized prompts for zero-shot image classification

    Sarah Pratt, Ian Covert, Rosanne Liu, and Ali Farhadi. What does a platypus look like? generating customized prompts for zero-shot image classification. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 15691–15701, 2023. 2

    2023

  57. [65]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Proceedings of th...

    2021

  58. [66]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1–67, 2020. 1

    2020

  59. [67]

    Meta-learning with implicit gradients

    Aravind Rajeswaran, Chelsea Finn, Sham M Kakade, and Sergey Levine. Meta-learning with implicit gradients. In Advances in Neural Information Processing Systems . Cur- ran Associates, Inc., 2019. 2

    2019

  60. [68]

    Towards to- tal recall in industrial anomaly detection

    Karsten Roth, Latha Pemula, Joaquin Zepeda, Bernhard Sch¨olkopf, Thomas Brox, and Peter Gehler. Towards to- tal recall in industrial anomaly detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 14318–14328, 2022. 1

    2022

  61. [69]

    Sophia Koepke, Oriol Vinyals, Cordelia Schmid, and Zeynep Akata

    Karsten Roth, Jae Myung Kim, A. Sophia Koepke, Oriol Vinyals, Cordelia Schmid, and Zeynep Akata. Waffling around for performance: Visual classification with ran- dom words and broad concepts. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pa...

    2023

  62. [70]

    A practitioner’s guide to continual multimodal pre- training

    Karsten Roth, Vishaal Udandarao, Sebastian Dziadzio, Ameya Prabhu, Mehdi Cherti, Oriol Vinyals, Olivier H´enaff, Samuel Albanie, Matthias Bethge, and Zeynep Akata. A practitioner’s guide to continual multimodal pre- training. arXiv preprint arXiv:2408.14471, 2024. 1, 2

    2024 arXiv

  63. [71]

    Berg, and Li Fei-Fei

    Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recog- nition Challenge. International Journal of Computer Vision (I...

    2015

  64. [72]

    Rusu, Dushyant Rao, Jakub Sygnowski, Oriol Vinyals, Razvan Pascanu, Simon Osindero, and Raia Had- sell

    Andrei A. Rusu, Dushyant Rao, Jakub Sygnowski, Oriol Vinyals, Razvan Pascanu, Simon Osindero, and Raia Had- sell. Meta-learning with latent embedding optimization. In International Conference on Learning Representations ,

  65. [73]

    Is a caption worth a thou- sand images? a study on representation learning

    Shibani Santurkar, Yann Dubois, Rohan Taori, Percy Liang, and Tatsunori Hashimoto. Is a caption worth a thou- sand images? a study on representation learning. In The Eleventh International Conference on Learning Represen- tations, 2023. 1

    2023

  66. [74]

    Scott, Andrew C

    Tyler R. Scott, Andrew C. Gallagher, and Michael C. Mozer. von mises-fisher loss: An exploration of embed- ding geometries for supervised learning. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion (ICCV), pages 10612–10622, 2021. 4

    2021

  67. [75]

    statsmodels: Econo- metric and statistical modeling with python

    Skipper Seabold and Josef Perktold. statsmodels: Econo- metric and statistical modeling with python. In 9th Python in Science Conference, 2010. 6

    2010

  68. [76]

    Prototyp- ical networks for few-shot learning

    Jake Snell, Kevin Swersky, and Richard Zemel. Prototyp- ical networks for few-shot learning. In Proceedings of the 31st International Conference on Neural Information Pro- cessing Systems , page 4080–4090, Red Hook, NY , USA,

  69. [77]

    CLIP models are few-shot learners: Empirical stud- ies on VQA and visual entailment

    Haoyu Song, Li Dong, Weinan Zhang, Ting Liu, and Furu Wei. CLIP models are few-shot learners: Empirical stud- ies on VQA and visual entailment. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages 6088–6100,...

    2022

  70. [78]

    Momentum-based weight interpolation of strong zero- shot models for continual learning

    Zafir Stojanovski, Karsten Roth, and Zeynep Akata. Momentum-based weight interpolation of strong zero- shot models for continual learning. arXiv preprint arXiv:2211.03186, 2022. 2

    2022 arXiv

  71. [79]

    Torr, and Timothy M

    Flood Sung, Yongxin Yang, Li Zhang, Tao Xiang, Philip H.S. Torr, and Timothy M. Hospedales. Learning to compare: Relation network for few-shot learning. In 2018 IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 1199–1208, 2018. 2

    2018

  72. [80]

    A fistful of words: Learning transferable visual models from bag-of-words supervision

    Ajinkya Tejankar, Maziar Sanjabi, Bichen Wu, Sain- ing Xie, Madian Khabsa, Hamed Pirsiavash, and Hamed Firooz. A fistful of words: Learning transferable visual models from bag-of-words supervision. arXiv preprint arXiv:2112.13884, 2021. 2

    2021 arXiv

  73. [81]

    Reflecting on the state of rehearsal-free continual learning with pretrained models

    Lukas Thede, Karsten Roth, Olivier J H ´enaff, Matthias Bethge, and Zeynep Akata. Reflecting on the state of rehearsal-free continual learning with pretrained models. arXiv preprint arXiv:2406.09384, 2024. 1

    2024

  74. [82]

    Tenenbaum, and Phillip Isola

    Yonglong Tian, Yue Wang, Dilip Krishnan, Joshua B. Tenenbaum, and Phillip Isola. Rethinking few-shot image classification: A good embedding is all you need? In Com- puter Vision - ECCV 2020 - 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XIV, pag...

    2020

  75. [83]

    Learning a universal template for few-shot dataset generalization

    Eleni Triantafillou, Hugo Larochelle, Richard Zemel, and Vincent Dumoulin. Learning a universal template for few-shot dataset generalization. In Proceedings of the 38th International Conference on Machine Learning, pages 10424–10433. PMLR, 2021. 2

    2021

  76. [84]

    Sus-x: Training-free name-only transfer of vision-language models

    Vishaal Udandarao, Ankush Gupta, and Samuel Albanie. Sus-x: Training-free name-only transfer of vision-language models. In ICCV, 2023. 1, 2, 6

    2023

  77. [85]

    No ”zero-shot” without exponential data: Pretraining concept frequency determines multimodal model performance

    Vishaal Udandarao, Ameya Prabhu, Adhiraj Ghosh, Yash Sharma, Philip Torr, Adel Bibi, Samuel Albanie, and Matthias Bethge. No ”zero-shot” without exponential data: Pretraining concept frequency determines multimodal model performance. In The Thirty-eighth Annual Confer- ence on...

    2024

  78. [86]

    Gomez, Łukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Proceed- ings of the 31st International Conference on Neural Infor- mation Processing Systems , page 6000–6010, Red Hook,...

    2017

  79. [87]

    Matching networks for one shot learning

    Oriol Vinyals, Charles Blundell, Timothy Lillicrap, koray kavukcuoglu, and Daan Wierstra. Matching networks for one shot learning. In Advances in Neural Information Pro- cessing Systems. Curran Associates, Inc., 2016. 1, 2

    2016

  80. [88]

    Learning robust global representations by penalizing local predictive power

    Haohan Wang, Songwei Ge, Zachary Lipton, and Eric P Xing. Learning robust global representations by penalizing local predictive power. In Advances in Neural Information Processing Systems, pages 10506–10518, 2019. 1, 2

    2019

  81. [89]

    Simpleshot: Revisiting nearest- neighbor classification for few-shot learning.arXiv preprint arXiv:1911.04623, 2019

    Yan Wang, Wei-Lun Chao, Kilian Q Weinberger, and Lau- rens Van Der Maaten. Simpleshot: Revisiting nearest- neighbor classification for few-shot learning.arXiv preprint arXiv:1911.04623, 2019. 2

    1911 arXiv

  82. [90]

    A hard-to-beat baseline for training- free CLIP-based adaptation

    Zhengbo Wang, Jian Liang, Lijun Sheng, Ran He, Zilei Wang, and Tieniu Tan. A hard-to-beat baseline for training- free CLIP-based adaptation. In The Twelfth International Conference on Learning Representations, 2024. 1, 2

    2024

  83. [91]

    Welinder, S

    P. Welinder, S. Branson, T. Mita, C. Wah, F. Schroff, S. Be- longie, and P. Perona. Caltech-UCSD Birds 200. Technical Report CNS-TR-2010-001, California Institute of Technol- ogy, 2010. 1, 2

    2010

  84. [92]

    Cascade prompt learning for vision-language model adaptation

    Ge Wu, Xin Zhang, Zheng Li, Zhaowei Chen, Jiajun Liang, Jian Yang, and Xiang Li. Cascade prompt learning for vision-language model adaptation. In Computer Vision – ECCV 2024: 18th European Conference, Milan, Italy, September 29–October 4, 2024, Proceedings, Part L, page 304–32...

    2024

  85. [93]

    Yu, and Dahua Lin

    Zhirong Wu, Yuanjun Xiong, Stella X. Yu, and Dahua Lin. Unsupervised feature learning via non-parametric instance discrimination. In 2018 IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 3733–3742,

    2018

  86. [94]

    J. Xiao, J. Hays, K. A. Ehinger, A. Oliva, and A. Torralba. Sun database: Large-scale scene recognition from abbey to zoo. In 2010 IEEE Computer Society Conference on Com- puter Vision and Pattern Recognition , pages 3485–3492,

    2010

  87. [95]

    Ra-clip: Retrieval augmented contrastive language-image pre-training

    Chen-Wei Xie, Siyang Sun, Xiong Xiong, Yun Zheng, Deli Zhao, and Jingren Zhou. Ra-clip: Retrieval augmented contrastive language-image pre-training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 19265–19274, 2023. 2

    2023

  88. [96]

    MetaFun: Meta-learning with itera- tive functional updates

    Jin Xu, Jean-Francois Ton, Hyunjik Kim, Adam Kosiorek, and Yee Whye Teh. MetaFun: Meta-learning with itera- tive functional updates. In Proceedings of the 37th Inter- national Conference on Machine Learning , pages 10617– 10627. PMLR, 2020. 1, 2

    2020

  89. [97]

    Bag-of-visual-words and spatial extensions for land-use classification

    Yi Yang and Shawn Newsam. Bag-of-visual-words and spatial extensions for land-use classification. In ACM SIGSPATIAL International Conference on Advances in Ge- ographic Information Systems (ACM GIS), 2010. 1, 2

    2010

  90. [98]

    TapNet: Neural network augmented with task-adaptive projection for few-shot learning

    Sung Whan Yoon, Jun Seo, and Jaekyun Moon. TapNet: Neural network augmented with task-adaptive projection for few-shot learning. In Proceedings of the 36th Interna- tional Conference on Machine Learning, pages 7115–7123. PMLR, 2019. 2

    2019

  91. [99]

    Sigmoid loss for language image pre-training

    Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 11975–11986, 2023. 1, 2, 3, 4, 6, 7

    2023

  92. [100]

    Deepemd: Few-shot image classification with differen- tiable earth mover’s distance and structured classifiers

    Chi Zhang, Yujun Cai, Guosheng Lin, and Chunhua Shen. Deepemd: Few-shot image classification with differen- tiable earth mover’s distance and structured classifiers. In 2020 IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), pages 12200–12210, 2020. 2

    2020

  93. [101]

    Tip- adapter: Training-free adaption of clip for few-shot classi- fication

    Renrui Zhang, Wei Zhang, Rongyao Fang, Peng Gao, Kun- chang Li, Jifeng Dai, Yu Qiao, and Hongsheng Li. Tip- adapter: Training-free adaption of clip for few-shot classi- fication. In Computer Vision – ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, P...

    2022

  94. [102]

    Dual memory networks: A versatile adaptation approach for vision-language models

    Yabin Zhang, Wenjie Zhu, Hui Tang, Zhiyuan Ma, Kaiyang Zhou, and Lei Zhang. Dual memory networks: A versatile adaptation approach for vision-language models. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2024. 2, 6

    2024

  95. [103]

    Dual-image enhanced clip for zero-shot anomaly detection

    Zhaoxiang Zhang, Hanqiu Deng, Jinan Bao, and Xingyu Li. Dual-image enhanced clip for zero-shot anomaly detection. arXiv preprint arXiv:2405.04782, 2024. 1, 2

    2024 arXiv

  96. [104]

    1, 2, 3, 4, 5, 6

    Springer-Verlag. 1, 2, 3, 4, 5, 6

  97. [105]

    Conditional prompt learning for vision-language models

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Zi- wei Liu. Conditional prompt learning for vision-language models. In 2022 IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition (CVPR), pages 16795–16804,

    2022

  98. [106]

    Learning to prompt for vision-language models

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Zi- wei Liu. Learning to prompt for vision-language models. International Journal of Computer Vision (IJCV), 2022. 2

    2022

  99. [107]

    Places: A 10 million image database for scene recognition

    Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 million image database for scene recognition. IEEE Transactions on Pattern Anal- ysis and Machine Intelligence, 2017. 1, 2

    2017

  100. [108]

    Not all features mat- ter: Enhancing few-shot clip with adaptive prior refinement

    Xiangyang Zhu, Renrui Zhang, Bowei He, Aojun Zhou, Dong Wang, Bin Zhao, and Peng Gao. Not all features mat- ter: Enhancing few-shot clip with adaptive prior refinement. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 2605–2615, 2023. 2, 6

    2023

  101. [109]

    Enhancing zero-shot vision models by label-free prompt distribution learning and bias correct- ing

    Xingyu Zhu, Beier Zhu, Yi Tan, Shuo Wang, Yanbin Hao, and Hanwang Zhang. Enhancing zero-shot vision models by label-free prompt distribution learning and bias correct- ing. In The Thirty-eighth Annual Conference on Neural In- formation Processing Systems, 2024. 2

    2024

  102. [110]

    Test-time distribution normalization for contrastively learned visual-language models

    Yifei Zhou, Juntao Ren, Fengyu Li, Ramin Zabih, and Ser-Nam Lim. Test-time distribution normalization for contrastively learned visual-language models. In Thirty- seventh Conference on Neural Information Processing Sys- tems, 2023. 2

    2023

  103. [113]

    Aug- mented Entries

    Luisa Zintgraf, Kyriacos Shiarli, Vitaly Kurin, Katja Hof- mann, and Shimon Whiteson. Fast context adaptation via meta-learning. In Proceedings of the 36th Interna- tional Conference on Machine Learning, pages 7693–7702. PMLR, 2019. 2 Context-Aware Multimodal Pretraining Suppl...

    2019

  104. [2017]

    1, 2, 3, 5

    Curran Associates Inc. 1, 2, 3, 5

  105. [2022]

    Springer-Verlag. 1, 3, 5

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.