Pith. sign in

REVIEW 3 major objections 5 minor 2 cited by

CLIP meets DINO for Tuning Zero-Shot Classifier using Unlabeled Image Collections

T0 review · 3 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read NoLA tunes CLIP with no labels, beating LaFTer by 3.6%

desk verdict NoLA is a novel pipeline for label-free prompt tuning, but the headline 3.6% gain is unverified because k is selected on test labels. read the letter →

arxiv 2411.19346 v3 pith:JC7WLRMS submitted 2024-11-28 cs.CV cs.CLcs.LG

Mohamed Fazli Imam , Rufael Fedaku Marew , Jameel Hassan , Mustansar Fiaz , Alham Fikri Aji , Hisham Cholakkal This is my paper · ORCID
classification cs.CVcs.CLcs.LG
keywords label-freeprompttuningzero-shotclassificationCLIPDINOlargelanguagemodelsself-supervisedlearningvision-languagepseudo-labeling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that CLIP can be tuned for a fixed set of classes without any labeled images by combining two existing sources of knowledge: LLM-generated class descriptions and the visual features of a self-supervised model like DINO. The proposed NoLA method first builds a text classifier from LLM descriptions, then uses that classifier to pseudo-label the most confident training images and train a small alignment head on frozen DINO features, and finally uses the resulting DINO-based labelling network to supervise learnable visual prompts on CLIP's vision encoder. On 11 image classification datasets, the paper reports an average absolute gain of 3.6% over the previous label-free method LaFTer and an average 11.91% improvement over zero-shot CLIP, reaching state-of-the-art label-free accuracy on 9 of the 11 datasets. This matters because it suggests that human labels are not a hard requirement for adapting vision-language models to downstream classification tasks, and that the complementary strengths of SSL backbones and LLM text knowledge can be combined in a label-free pipeline.

What carries the argument

The carrying mechanism is the DINO-based Labelling (DL) network: a frozen DINO ViT-B/16 backbone $g_s$ plus a small trainable alignment head $h$ that maps DINO's visual features into the CLIP joint embedding space. This head is trained with smoothed cross-entropy on pseudo-labels produced by the class-description embedding (CDE) classifier, which averages the CLIP text embeddings of multiple LLM-generated descriptions per class. Once trained, the DL network serves as the teacher in the final prompt-learning stage, where learnable visual prompts are prepended to CLIP's vision encoder and optimized, together with the CDE classifier, so that CLIP's prediction on a strongly augmented view matches the DL network's prediction on the weakly augmented view (Eq. 4). The alignment head thus converts a self-supervised feature space into a vision-language embedding space without any human labels, which is what makes the transfer of DINO's visual quality to CLIP possible.

What would settle it

Take a benchmark dataset with ground-truth labels (e.g., CIFAR-100), train the alignment head $h$ on the exact same top-$k$ samples but with true labels instead of CDE pseudo-labels, and then run the full DINO-assisted prompt learning. If the oracle-aligned version does not outperform NoLA by a clear margin, the pseudo-label quality is not the bottleneck; if it improves substantially, the method's success depends on the CDE pseudo-labels being accurate, which would make the approach fragile on datasets where LLM descriptions are weak.

Watch

Extended reading notes

Core claim

The central claim is that a zero-shot CLIP classifier can be substantially improved for a closed set of classes using only unlabeled images, if one first enriches the text side with LLM-generated descriptions and then uses a self-supervised visual backbone as a teacher. The paper argues that CLIP's visual features are suboptimal for fine-grained discrimination, while DINO features are richer; the missing link is a mapping from DINO's feature space to CLIP's joint embedding space. NoLA obtains that mapping by training a lightweight alignment head $h$ on pseudo-labels produced by the class-description embedding (CDE) classifier, with a smoothed cross-entropy loss and a top-$k$ confident-sample selection. The aligned DINO network then acts as a pseudo-labeler for prompt-tuning CLIP's vision encoder using a FixMatch-style weak/strong augmentation consistency objective, jointly updating learnable visual prompts and the CDE classifier. The paper reports an average absolute gain of 3.6% over LaFTer across 11 datasets, an average 11.91% improvement over zero-shot CLIP, and state-of-the-art label-free accuracy on 9 of the 11 datasets, with ablations showing each of the three stages contributes to the final accuracy.

Load-bearing premise

The method assumes that a small alignment head, trained only on pseudo-labels from an LLM-built text classifier, can map DINO's visual features into CLIP's embedding space accurately enough that the resulting teacher's pseudo-labels are reliable for prompt tuning.

Editorial extensions

If this is right

  • NoLA's label-free accuracy exceeds CoOp with 1-shot or 5-shot labels on several of the 11 datasets, showing that unlabeled adaptation can partially close the gap to few-shot prompt learning.
  • The stage-wise ablation shows each component earns its keep: the CDE classifier raises the six-dataset average from 67.9% to 72.0%, the DL network to 73.9%, and the full DINO-assisted prompt learning to 80.5%.
  • The method requires no human labels, no caption or metadata for the target images, and no per-dataset prompt engineering beyond the LLM description prompts, which are generic per domain.
  • The trained DL network, with frozen DINO and a light alignment head, is the teacher for prompt tuning, so the final CLIP model retains its original weights apart from the learned prompts and the updated CDE classifier.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • It is natural to expect that the same alignment-and-teach recipe would transfer to other SSL backbones (e.g., SimCLR, MAE) and other vision-language models (e.g., SigLIP, ALIGN), because the alignment head only needs pseudo-labels to map the SSL space into the VLM space; this is not tested in the paper.
  • The top-$k$ confidence selection rule (20% of the estimated per-class mean, clamped to 16–512) is a heuristic; a confidence-threshold-based selection or an adaptive schedule could improve robustness on heavily long-tailed datasets, which the paper does not explore.
  • Since the CDE classifier is built from LLM descriptions, the method inherits the LLM's biases and may degrade on domains where the LLM has weak or inaccurate knowledge (e.g., specialist medical or remote-sensing categories); the paper's 11 datasets are all common object/scene/action benchmarks.
  • The paper reports gains on public benchmarks; a stress test on a heavily imbalanced or out-of-distribution dataset (e.g., iNaturalist or a medical image set) would clarify whether the average 3.6% gain over LaFTer is robust beyond the evaluated domains.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes NoLA, a label-free prompt-tuning method for CLIP that leverages LLM-generated class descriptions and DINO's self-supervised visual features. The pipeline has three stages: (i) a CDE classifier built from averaged CLIP text embeddings of LLM-generated class descriptions; (ii) a DINO-based labelling (DL) network, consisting of a frozen DINO backbone and a learned alignment module h, trained on top-k confident pseudo-labels from the CDE classifier; (iii) prompt-tuning of CLIP's vision encoder by training visual prompts to minimize a smoothed cross-entropy loss against the DL network's predictions, using FixMatch-style weak/strong augmentation. The paper reports an average absolute gain of 3.6% over the state-of-the-art label-free method LaFTer across 11 datasets, and an average 11.91% improvement over zero-shot CLIP, claiming state-of-the-art label-free performance on 9 of 11 datasets. The paper includes ablations showing each stage contributes, and an appendix describing the k-selection analysis.

Significance. The paper addresses a relevant problem: adapting CLIP to target datasets without labels. The idea of combining DINO's strong visual features with CLIP's text-aligned embedding space in a label-free manner is interesting, and the stage-wise ablations (Tables 3 and 4) provide some evidence that each component contributes. If the label-free claim were airtight, the method would be a practical contribution to test-time adaptation and label-efficient learning. However, the central claim is compromised by the choice of the k hyperparameter, as detailed below, and the self-training loop deserves careful consideration. The paper also provides a public code link, which is a positive for reproducibility.

major comments (3)
  1. [Supplementary, Analysis on k selection; Main text, 'DINO-based Labelling (DL) Network'] The choice of k = 20% of the average number of images per class is selected by inspecting the 'Top-1 Accuracy of trained DL network' curves in Figure 4 of the supplementary, which report accuracy on the test sets of ImageNet, CIFAR-100, UCF101, and Caltech101. This means test labels are used to select a hyperparameter of a supposedly label-free pipeline, leaking label information into the reported results. Because the final numbers in Table 1 depend on this choice, the average 3.6% gain over LaFTer is not a verifiable label-free result as presented. The authors must either (a) choose k using only unlabeled data (e.g., a held-out portion of the training set with pseudo-label confidence) and rerun all experiments, or (b) clearly report the results for a fixed rule without test-set tuning. This is load-bearing for the central claim.
  2. [Eq. (4) and 'DINO-based Labelling (DL) Network'] The final prompt-learning objective trains CLIP to match the DL network's predictions, but the DL network was trained on pseudo-labels produced by the CDE classifier, which is built directly from CLIP's own text encoder and LLM descriptions. This is a self-training loop in which the student (CLIP with learnable prompts) is taught by a teacher whose supervision originates from the same CLIP text encoder. While the ablation in Table 4 (replacing DINO with CLIP in the DL network) shows that DINO features contribute beyond CLIP, the paper should more explicitly analyze and discuss the risk of confirmation bias: the alignment module h may simply map DINO features into the CLIP text space without adding genuinely new visual information. To strengthen the claim, the authors could demonstrate that the DL network's pseudo-labels on a held-out unlabeled set are more accurate than the CDE classifier's own predictions, or otherwise quantify the added information from DINO.
  3. [Table 1 and Results] The reported results lack error bars or multiple seeds. The claimed gains over LaFTer are small on some datasets (e.g., +1.2% on ImageNet, +0.1% on UCF101, +1.0% on CIFAR100), and without variance estimates it is unclear whether these differences are statistically meaningful. The paper should report at least three random seeds and provide standard deviations for the main results, especially given that the k-selection rule itself adds a source of variance.
minor comments (5)
  1. [Throughout] There are numerous typographical errors and inconsistent terminologies, e.g., 'LaFter' vs 'LaFTer', 'unlabelled' vs 'unlabeled', 'performace', 'Morover', 'Inn order', 'propotional'. A thorough proofread is needed.
  2. [Table 1] The claim of 'state-of-the-art performance in 9 out of 11 datasets' is not accurate by the numbers in the table: EuroSAT (Ours 73.5 vs LaFTer 73.9) and CIFAR10 (Ours 94.9 vs LaFTer 95.8) are not the best, and Caltech101 is tied with WaffleCLIP at 94.8. Please clarify how the 9/11 count is computed.
  3. [Methodology, 'DINO-based Labelling (DL) Network'] The sentence 'Since our method is entirely label-free, we do not use information about the number of samples per class' is misleading: the average number of images per class is computed from the total number of training images and the number of classes, which is information about the dataset split but not per-class labels. Please rephrase to avoid confusion.
  4. [Implementation Details] The paper does not specify the architecture of the alignment module h (e.g., number of layers, hidden dimension), nor the exact smoothing parameter used in the smoothed cross-entropy loss. These details should be provided for reproducibility.
  5. [Equation (2)] In Eq. (2), the class index subscript is inconsistent: 'ϕC' is used in the first line and 'ϕC' in the second line, but the summation index is 'i'; the notation should be cleaned up.

Circularity Check

1 steps flagged · score 6.0 of 10

The label-free claim is partly produced by a hyperparameter (k) selected on test-set accuracy: the 20% rule used for all final results is chosen from Figure 4's Top-1 accuracy curves, so the reported 3.6% gain is not a fully independent label-free estimate.

  1. fitted input called prediction [Supplementary Material, 'Analysis on k selection', Figure 4; main text 'DINO-based Labelling (DL) Network' section (k selection paragraph)]
    "The choice of 20% as the optimal percentage is justified by empirical analysis, which we present in the supplementary material. ... As shown in Figure 4 (bottom), for smaller datasets, setting k to 16 yields better performance ... Empirically, setting k to around 20% of the average number of images per class achieves better accuracy (see Figure 4 - top). Thus, we adopt the following strategy: if 20% of the average number of images per class is less than 16, we select 16 confident samples. Otherwise, we select 20% of the confident samples, with a cap of 512 if the number exceeds this limit."

    The supplementary's Figure 4 reports 'Top-1 Accuracy of trained DL network with different values for k' on the paper's benchmark datasets, which is the same Top-1 accuracy convention used for the final test-set results in Table 1. The 20% rule is selected from these accuracy curves, and the same rule is then used to produce every final NoLA result in Table 1, including the claimed 3.6% average absolute gain over LaFTer and 9-of-11 state-of-the-art counts. No validation-based or unsupervised criterion for choosing k is provided. Thus a central hyperparameter of the supposedly label-free pipeline is fitted directly to target test labels; the reported label-free predictions are partly forced by the evaluation labels and the comparison is statistically optimistic.

full rationale

The only concrete circular/fitted element I can exhibit with the paper's own text is the selection of k via Top-1 accuracy in the supplementary, followed by use of that k in all final results. This is a genuine leak: the 'label-free' tuning pipeline has a hyperparameter chosen on test accuracy, so the headline 3.6% gain and 9/11 SOTA claim are not independent of the evaluation labels. I do not count the self-training loop (CLIP/LLM-derived CDE pseudo-labels training the DINO alignment module, whose outputs supervise the final prompt tuning) as formal circularity: DINO features are an external visual signal, the final evaluation is against held-out labels, and the objective in Eq. 4 is a learning loss rather than a definitional equivalence. However, the test-set k selection prevents a completely clean label-free verdict; hence the score is 6 rather than higher or near-zero.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

The central claim rests on the assumption that LLM-generated descriptions provide reliable pseudo-label anchors, that DINO features can be aligned to CLIP's embedding space with a small module, and that the 20% k rule tuned on benchmark test sets generalizes. The only tuned numerical parameter that directly affects all results is k; the 20% proportion, floor 16, and cap 512 are selected empirically on target datasets. No invented physical entities are introduced.

free parameters (3)
  • top-k selection percentage (k) = 20% of average images per class, floored at 16 and capped at 512
    The 20% proportion is chosen empirically from accuracy curves on the target datasets (Figure 4, supplementary); it is a hyperparameter fitted to benchmark performance rather than derived from first principles or a validation split.
  • visual prompt length = 16 learnable tokens
    The number of VPT tokens is a hand-set hyperparameter; no ablation on prompt length is shown.
  • alignment module training epochs = 50
    A fixed training budget for the alignment module h; sensitivity to this choice is not analyzed.
assumptions (6)
  • domain assumption DINO features are richer and better than CLIP vision features for fine-grained discrimination.
    Stated in the Motivation section; the method relies on this to justify using DINO as the pseudo-labeler.
  • domain assumption LLM-generated descriptions yield a more accurate zero-shot classifier than CLIP's default name prompts.
    The CDE classifier is built from GPT-3.5 descriptions (CuPL), and all downstream pseudo-labels inherit this assumption.
  • domain assumption A learned alignment module h can map DINO features into CLIP's joint embedding space using only CDE pseudo-labels.
    Section 'DINO-based Labelling Network'; this is the load-bearing assumption, with no theoretical guarantee, only empirical support.
  • domain assumption Top-k confident CDE predictions provide clean enough pseudo-labels to train h and the final prompts.
    Section 'DINO-based Labelling Network' and Eq. 4; if pseudo-label noise is too high, the prompt tuning degrades.
  • ad hoc to paper The 20% k rule tuned on benchmark test sets transfers to unseen datasets.
    Supplementary 'Analysis on k selection' chooses 20% based on accuracy curves on the target datasets; this is an ad hoc assumption for label-free generalization.
  • domain assumption FixMatch-style weak and strong augmentation consistency improves generalization for this setting.
    Section 'DINO-assisted prompt learning' adopts FixMatch motivation; no dedicated ablation isolates augmentation strength.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CLIP meets DINO for Tuning Zero-Shot Classifier using Unlabeled Image Collections." pith.science (2026). https://pith.science/paper/JC7WLRMS

@misc{pith2026241119346,
  author       = {Pith},
  title        = {Pith review of: CLIP meets DINO for Tuning Zero-Shot Classifier using Unlabeled Image Collections},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JC7WLRMS}},
  note         = {Machine review of arXiv:2411.19346}
}
read the original abstract

In the era of foundation models, CLIP has emerged as a powerful tool for aligning text & visual modalities into a common embedding space. However, the alignment objective used to train CLIP often results in subpar visual features for fine-grained tasks. In contrast, SSL-pretrained models like DINO excel at extracting rich visual features due to their specialized training paradigm. Yet, these SSL models require an additional supervised linear probing step, which relies on fully labeled data which is often expensive and difficult to obtain at scale. In this paper, we propose a label-free prompt-tuning method that leverages the rich visual features of self-supervised learning models (DINO) and the broad textual knowledge of large language models (LLMs) to largely enhance CLIP-based image classification performance using unlabeled images. Our approach unfolds in three key steps: (1) We generate robust textual feature embeddings that more accurately represent object classes by leveraging class-specific descriptions from LLMs, enabling more effective zero-shot classification compared to CLIP's default name-specific prompts. (2) These textual embeddings are then used to produce pseudo-labels to train an alignment module that integrates the complementary strengths of LLM description-based textual embeddings & DINO's visual features. (3) Finally, we prompt-tune CLIP's vision encoder through DINO-assisted supervision using the trained alignment module. This three-step process allows us to harness the best of visual & textual foundation models, resulting in a powerful and efficient approach that surpasses state-of-the-art label-free classification methods. Notably, our framework, NoLA (No Labels Attached), achieves an average absolute gain of 3.6% over the state-of-the-art LaFTer across 11 diverse image classification datasets. Our code & models can be found at https://github.com/fazliimam/NoLA.

Figures

Figures reproduced from arXiv: 2411.19346 by the authors.

Figure 1
Figure 1. Top-1 accuracy (%) comparison with recent label [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of proposed NoLA (No Labels Attached) method, (a) A set of templates and the class names are fed through an LLM to generate context-enriched text descriptions per class. The description embeddings obtained from the CLIP text encoder ft are averaged to compose the class description-based embedding (CDE) classifier ϕ. (b) Zero-shot inference is obtained for the training set by using the CLIP vision encoder fv… view at source ↗
Figure 3
Figure 3. TSNE projections comparison of EuroSAT embed [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Top-1 Accuracy of trained DL network with different values for k. The top row shows the perfor￾mance of different values for k in large datasets, namely, ImageNet (top-left) and CIFAR-100 (top-right). The bot￾tom row shows the performance of different values for k in s…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. AdaDINO: Context-Adaptive DINO-Distilled Vision Foundation Models for Efficient Open-Vocabulary Edge Inference

    cs.CV 2026-04 unverdicted novelty 6.0 of 10

    AdaVFM trains a DINOv2-distilled, CLIP-aligned NAS supernet and uses a cloud LLM to select the cheapest subnet per scene, reducing edge FLOPs by up to 77.9% at similar accuracy.

  2. Adapting Vision-Language Models Without Labels: A Comprehensive Survey

    cs.LG 2025-08 conditional novelty 5.0 of 10

    A survey that organizes unsupervised vision-language model adaptation by unlabeled-data availability into four paradigms: data-free transfer, domain transfer, episodic test-time, and online test-time adaptation.

Reference graph

Works this paper leans on

83 extracted references · 38 canonical work pages · cited by 2 Pith papers

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Akiva, P.; Purri, M.; and Leotta, M. 2022. Self-supervised material and texture representation learning for remote sensing tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8203--8215

    2022

  4. [4]

    Z.; Taha, T

    Alom, M. Z.; Taha, T. M.; Yakopcic, C.; Westberg, S.; Sidike, P.; Nasrin, M. S.; Van Esesn, B. C.; Awwal, A. A. S.; and Asari, V. K. 2018. The history began from alexnet: A comprehensive survey on deep learning approaches. arXiv preprint arXiv:1803.01164

    arXiv 2018

  5. [5]

    Assran, M.; Caron, M.; Misra, I.; Bojanowski, P.; Joulin, A.; Ballas, N.; and Rabbat, M. 2021. Semi-supervised learning of visual features by non-parametrically predicting view assignments with support samples. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 8443--8452

    work page 2021

  6. [6]

    M.; Mekhalfi, M

    Bazi, Y.; Al Rahhal, M. M.; Mekhalfi, M. L.; Al Zuair, M. A.; and Melgani, F. 2022. Bi-modal transformer-based approach for visual question answering in remote sensing imagery. IEEE Transactions on Geoscience and Remote Sensing, 60: 1--11

    work page 2022

  7. [7]

    Berthelot, D.; Carlini, N.; Goodfellow, I.; Papernot, N.; Oliver, A.; and Raffel, C. A. 2019. Mixmatch: A holistic approach to semi-supervised learning. Advances in neural information processing systems, 32

    2019

  8. [8]

    Caron, M.; Touvron, H.; Misra, I.; J \'e gou, H.; Mairal, J.; Bojanowski, P.; and Joulin, A. 2021. Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, 9650--9660

    2021

Show all 83 references
  1. [9]

    Chen, T.; Kornblith, S.; Norouzi, M.; and Hinton, G. 2020. A Simple Framework for Contrastive Learning of Visual Representations. arXiv:2002.05709

    2020 arXiv

  2. [10]

    Chen, X.; and He, K. 2021. Exploring simple siamese representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 15750--15758

    2021

  3. [11]

    Chen, Y.; Huang, J.; Li, X.; Xiong, S.; and Lu, X. 2023. Multiscale Salient Alignment Learning for Remote Sensing Image-Text Retrieval. IEEE Transactions on Geoscience and Remote Sensing

    2023

  4. [12]

    Cheng, G.; Han, J.; and Lu, X. 2017. Remote sensing image scene classification: Benchmark and state of the art. Proceedings of the IEEE, 105(10): 1865--1883

    2017

  5. [13]

    Cimpoi, M.; Maji, S.; Kokkinos, I.; Mohamed, S.; and Vedaldi, A. 2014. Describing textures in the wild. In Proceedings of the IEEE conference on computer vision and pattern recognition, 3606--3613

    2014

  6. [14]

    Cozzolino, D.; Poggi, G.; Corvi, R.; Nie ner, M.; and Verdoliva, L. 2024. Raising the Bar of AI-generated Image Detection with CLIP. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 4356--4366

    2024

  7. [15]

    Deng, J.; Dong, W.; Socher, R.; Li, L.-J.; Li, K.; and Fei-Fei, L. 2009. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, 248--255. Ieee

    2009

  8. [16]

    Dosovitskiy, A.; Beyer, L.; Kolesnikov, A.; Weissenborn, D.; Zhai, X.; Unterthiner, T.; Dehghani, M.; Minderer, M.; Heigold, G.; Gelly, S.; et al. 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929

    2020 arXiv

  9. [17]

    Eldele, E.; Ragab, M.; Chen, Z.; Wu, M.; Kwoh, C.-K.; Li, X.; and Guan, C. 2023. Self-supervised contrastive representation learning for semi-supervised time-series classification. IEEE Transactions on Pattern Analysis and Machine Intelligence

    2023

  10. [18]

    Fei-Fei, L.; Fergus, R.; and Perona, P. 2006. One-shot learning of object categories. IEEE transactions on pattern analysis and machine intelligence, 28(4): 594--611

    2006

  11. [19]

    Gao, P.; Geng, S.; Zhang, R.; Ma, T.; Fang, R.; Zhang, Y.; Li, H.; and Qiao, Y. 2024. Clip-adapter: Better vision-language models with feature adapters. International Journal of Computer Vision, 132(2): 581--595

    2024

  12. [20]

    Helber, P.; Bischke, B.; Dengel, A.; and Borth, D. 2019. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12(7): 2217--2226

    2019

  13. [21]

    S.; and You, X

    Hou, W.; Chen, S.; Chen, S.; Hong, Z.; Wang, Y.; Feng, X.; Khan, S.; Khan, F. S.; and You, X. 2024. Visual-Augmented Dynamic Semantic Prototype for Generative Zero-Shot Learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 23627--23637

    2024

  14. [22]

    J.; Naeem, M

    Hoyer, L.; Tan, D. J.; Naeem, M. F.; Van Gool, L.; and Tombari, F. 2023. SemiVL: Semi-Supervised Semantic Segmentation with Vision-Language Guidance. arXiv preprint arXiv:2311.16241

    2023 arXiv

  15. [23]

    Huang, T.; Chu, J.; and Wei, F. 2022. Unsupervised prompt learning for vision-language models. arXiv preprint arXiv:2204.03649

    2022 arXiv

  16. [24]

    Jia, C.; Yang, Y.; Xia, Y.; Chen, Y.-T.; Parekh, Z.; Pham, H.; Le, Q.; Sung, Y.-H.; Li, Z.; and Duerig, T. 2021. Scaling up visual and vision-language representation learning with noisy text supervision. In International conference on machine learning, 4904--4916. PMLR

    2021

  17. [25]

    Jia, M.; Tang, L.; Chen, B.-C.; Cardie, C.; Belongie, S.; Hariharan, B.; and Lim, S.-N. 2022. Visual prompt tuning. In ECCV, 709--727. Springer

    2022

  18. [26]

    K.; Vo, K.; Yamazaki, K.; and Le, N

    Joo, H. K.; Vo, K.; Yamazaki, K.; and Le, N. 2023. Clip-tsa: Clip-assisted temporal self-attention for weakly-supervised video anomaly detection. In 2023 IEEE International Conference on Image Processing (ICIP), 3230--3234. IEEE

    2023

  19. [27]

    U.; Naeem, M

    Khattak, M. U.; Naeem, M. F.; Naseer, M.; Van Gool, L.; and Tombari, F. 2024. Learning to Prompt with Text Only Supervision for Vision-Language Models. arXiv preprint arXiv:2401.02418

    2024 arXiv

  20. [28]

    U.; Rasheed, H.; Maaz, M.; Khan, S.; and Khan, F

    Khattak, M. U.; Rasheed, H.; Maaz, M.; Khan, S.; and Khan, F. S. 2023. Maple: Multi-modal prompt learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 19113--19122

    2023

  21. [29]

    P.; and Ba, J

    Kingma, D. P.; and Ba, J. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980

    2014 arXiv

  22. [30]

    T.; Hospedales, T

    Ko c yi g it, M. T.; Hospedales, T. M.; and Bilen, H. 2023. Accelerating Self-Supervised Learning via Efficient Training Strategies. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 5654--5664

    2023

  23. [31]

    Krizhevsky, A.; Hinton, G.; et al. 2009. Learning multiple layers of features from tiny images

    2009

  24. [32]

    Krizhevsky, A.; Sutskever, I.; and Hinton, G. E. 2012. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25

    2012

  25. [33]

    D.; Zhang, H.; Sohn, K.; and Carlini, N

    Kurakin, A.; Raffel, C.; Berthelot, D.; Cubuk, E. D.; Zhang, H.; Sohn, K.; and Carlini, N. 2020. Remixmatch: Semi-supervised learning with distribution matching and augmentation anchoring

    2020

  26. [34]

    Lee, Y.-L.; Tsai, Y.-H.; Chiu, W.-C.; and Lee, C.-Y. 2023. Multimodal Prompting with Missing Modalities for Visual Recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 14943--14952

    2023

  27. [35]

    Lester, B.; Al-Rfou, R.; and Constant, N. 2021. The power of scale for parameter-efficient prompt tuning. arXiv preprint arXiv:2104.08691

    2021 arXiv

  28. [36]

    Li, J.; Li, D.; Xiong, C.; and Hoi, S. 2022. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In International conference on machine learning, 12888--12900. PMLR

    2022

  29. [37]

    Li, M.; Li, Q.; and Wang, Y. 2023. Class Balanced Adaptive Pseudo Labeling for Federated Semi-Supervised Learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 16292--16301

    2023

  30. [38]

    Li, X.; Wen, C.; Hu, Y.; and Zhou, N. 2023 a . Rs-clip: Zero shot remote sensing scene classification via contrastive vision-language supervision. International Journal of Applied Earth Observation and Geoinformation, 124: 103497

    2023

  31. [39]

    Li, Z.; Qi, L.; Shi, Y.; and Gao, Y. 2023 b . IOMatch: Simplifying open-set semi-supervised learning with joint inliers and outliers utilization. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 15870--15879

    2023

  32. [40]

    Liang, F.; Wu, B.; Dai, X.; Li, K.; Zhao, Y.; Zhang, H.; Zhang, P.; Vajda, P.; and Marculescu, D. 2023. Open-vocabulary semantic segmentation with mask-adapted clip. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 7061--7070

    2023

  33. [41]

    Liu, C.; Zhang, W.; Lin, X.; Zhang, W.; Tan, X.; Han, J.; Li, X.; Ding, E.; and Wang, J. 2023. Ambiguity-Resistant Semi-Supervised Learning for Dense Object Detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 15579--15588

    2023

  34. [42]

    Liu, H.; Li, C.; Wu, Q.; and Lee, Y. J. 2024. Visual instruction tuning. Advances in neural information processing systems, 36

    2024

  35. [43]

    Lu, Y.; Liu, J.; Zhang, Y.; Liu, Y.; and Tian, X. 2022. Prompt distribution learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 5206--5215

    2022

  36. [44]

    J.; Karlinsky, L.; Lin, W.; Doveh, S.; ; Micorek, J.; Kozinski, M.; Kuhene, H.; and Possegger, H

    Mirza, M. J.; Karlinsky, L.; Lin, W.; Doveh, S.; ; Micorek, J.; Kozinski, M.; Kuhene, H.; and Possegger, H. 2024 a . Meta-Prompting for Automating Zero-shot Visual Recognition with LLMs . In Proceedings of the European Conference on Computer Vision (ECCV)

    2024

  37. [45]

    J.; Karlinsky, L.; Lin, W.; Possegger, H.; Kozinski, M.; Feris, R.; and Bischof, H

    Mirza, M. J.; Karlinsky, L.; Lin, W.; Possegger, H.; Kozinski, M.; Feris, R.; and Bischof, H. 2024 b . Lafter: Label-free tuning of zero-shot classifier using language and unlabeled image collections. Advances in Neural Information Processing Systems, 36

    2024

  38. [46]

    F.; Khan, M

    Naeem, M. F.; Khan, M. G. Z. A.; Xian, Y.; Afzal, M. Z.; Stricker, D.; Van Gool, L.; and Tombari, F. 2023 a . I2mvformer: Large language model generated multi-view document supervision for zero-shot image classification. In Proceedings of the IEEE/CVF Conference on Computer Vi...

    2023

  39. [47]

    F.; Xian, Y.; Zhai, X.; Hoyer, L.; Van Gool, L.; and Tombari, F

    Naeem, M. F.; Xian, Y.; Zhai, X.; Hoyer, L.; Van Gool, L.; and Tombari, F. 2023 b . Silc: Improving vision language pretraining with self-distillation. arXiv preprint arXiv:2310.13355

    2023 arXiv

  40. [48]

    Nguyen, K.-B.; and Yang, J.-S. 2023. Boosting Semi-Supervised Learning by bridging high and low-confidence predictions. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 1028--1038

    2023

  41. [49]

    Nilsback, M.-E.; and Zisserman, A. 2008. Automated flower classification over a large number of classes. In 2008 Sixth Indian conference on computer vision, graphics & image processing, 722--729. IEEE

    2008

  42. [50]

    Pan, C.; Yaman, B.; Velipasalar, S.; and Ren, L. 2024. Clip-bevformer: Enhancing multi-view image-based bev detector with ground truth flow. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 15216--15225

    2024

  43. [51]

    Pantazis, O.; Brostow, G.; Jones, K.; and Mac Aodha, O. 2022. Svl-adapter: Self-supervised adapter for vision-language pretrained models. arXiv preprint arXiv:2210.03794

    2022 arXiv

  44. [52]

    Park, S.; and Van Hentenryck, P. 2023. Self-supervised primal-dual learning for constrained optimization. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, 4052--4060

    2023

  45. [53]

    M.; Vedaldi, A.; Zisserman, A.; and Jawahar, C

    Parkhi, O. M.; Vedaldi, A.; Zisserman, A.; and Jawahar, C. 2012. Cats and dogs. In 2012 IEEE conference on computer vision and pattern recognition, 3498--3505. IEEE

    2012

  46. [54]

    P.; Wang, X.-Z.; and Wu, Q

    Pourpanah, F.; Abdar, M.; Luo, Y.; Zhou, X.; Wang, R.; Lim, C. P.; Wang, X.-Z.; and Wu, Q. J. 2022. A review of generalized zero-shot learning methods. IEEE transactions on pattern analysis and machine intelligence, 45(4): 4051--4070

    2022

  47. [55]

    Pratt, S.; Covert, I.; Liu, R.; and Farhadi, A. 2023. What does a platypus look like? generating customized prompts for zero-shot image classification. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 15691--15701

    2023

  48. [56]

    Qiu, C.; Yu, A.; Yi, X.; Guan, N.; Shi, D.; and Tong, X. 2022. Open Self-Supervised Features for Remote-Sensing Image Scene Classification Using Very Few Samples. IEEE Geoscience and Remote Sensing Letters, 20: 1--5

    2022

  49. [57]

    W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al

    Radford, A.; Kim, J. W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning, 8748--8763. PMLR

    2021

  50. [58]

    M.; Koepke, A.; Vinyals, O.; Schmid, C.; and Akata, Z

    Roth, K.; Kim, J. M.; Koepke, A.; Vinyals, O.; Schmid, C.; and Akata, Z. 2023. Waffling around for performance: Visual classification with random words and broad concepts. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 15746--15757

    2023

  51. [59]

    Saha, O.; Van Horn, G.; and Maji, S. 2024. Improved Zero-Shot Classification by Adapting VLMs with Text Descriptions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 17542--17552

    2024

  52. [60]

    C.; Rawat, Y

    Schiappa, M. C.; Rawat, Y. S.; and Shah, M. 2023. Self-supervised learning for videos: A survey. ACM Computing Surveys, 55(13s): 1--37

    2023

  53. [61]

    Shu, M.; Nie, W.; Huang, D.-A.; Yu, Z.; Goldstein, T.; Anandkumar, A.; and Xiao, C. 2022. Test-time prompt tuning for zero-shot generalization in vision-language models. Advances in Neural Information Processing Systems, 35: 14274--14289

    2022

  54. [62]

    A.; Cubuk, E

    Sohn, K.; Berthelot, D.; Carlini, N.; Zhang, Z.; Zhang, H.; Raffel, C. A.; Cubuk, E. D.; Kurakin, A.; and Li, C.-L. 2020. Fixmatch: Simplifying semi-supervised learning with consistency and confidence. Advances in neural information processing systems, 33: 596--608

    2020

  55. [63]

    R.; and Shah, M

    Soomro, K.; Zamir, A. R.; and Shah, M. 2012. UCF101: A dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv:1212.0402

    2012 arXiv

  56. [64]

    Stojnic, V.; and Risojevic, V. 2021. Self-supervised learning of remote sensing scene representations using contrastive multiview coding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 1182--1191

    2021

  57. [65]

    Szegedy, C.; Vanhoucke, V.; Ioffe, S.; Shlens, J.; and Wojna, Z. 2016. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2818--2826

    2016

  58. [66]

    N.; Kaiser, .; and Polosukhin, I

    Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, .; and Polosukhin, I. 2017. Attention is all you need. Advances in neural information processing systems, 30

    2017

  59. [67]

    Wei, T.; and Gan, K. 2023. Towards Realistic Long-Tailed Semi-Supervised Learning: Consistency Is All You Need. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 3469--3478

    2023

  60. [68]

    Wu, P.; Zhou, X.; Pang, G.; Zhou, L.; Yan, Q.; Wang, P.; and Zhang, Y. 2024. Vadclip: Adapting vision-language models for weakly supervised video anomaly detection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 6074--6082

    2024

  61. [69]

    Wysocza \'n ska, M.; Ramamonjisoa, M.; Trzci \'n ski, T.; and Sim \'e oni, O. 2024. Clip-diy: Clip dense inference yields open-vocabulary semantic segmentation for-free. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 1403--1413

    2024

  62. [70]

    A.; Hays, J.; Torralba, A.; and Oliva, A

    Xiao, J.; Ehinger, K. A.; Hays, J.; Torralba, A.; and Oliva, A. 2016. Sun database: Exploring a large collection of scene categories. International Journal of Computer Vision, 119: 3--22

    2016

  63. [71]

    Xu, W.; Xian, Y.; Wang, J.; Schiele, B.; and Akata, Z. 2020. Attribute prototype network for zero-shot learning. Advances in Neural Information Processing Systems, 33: 21969--21980

    2020

  64. [72]

    Yan, Z.; Wu, Y.; Qin, Y.; Han, X.; Cui, S.; and Li, G. 2024. Universal semi-supervised model adaptation via collaborative consistency training. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 872--882

    2024

  65. [73]

    Yao, L.; Huang, R.; Hou, L.; Lu, G.; Niu, M.; Xu, H.; Liang, X.; Li, Z.; Jiang, X.; and Xu, C. 2021. Filip: Fine-grained interactive language-image pre-training. arXiv preprint arXiv:2111.07783

    2021 arXiv

  66. [74]

    K.; Baek, W.; and Roh, B

    You, K.; Gu, J.; Ham, J.; Park, B.; Kim, J.; Hong, E. K.; Baek, W.; and Roh, B. 2023. Cxr-clip: Toward large scale chest x-ray language-image pre-training. In International Conference on Medical Image Computing and Computer-Assisted Intervention, 101--111. Springer

    2023

  67. [75]

    Yu, J.; Wang, Z.; Vasudevan, V.; Yeung, L.; Seyedhosseini, M.; and Wu, Y. 2022. Coca: Contrastive captioners are image-text foundation models. arXiv preprint arXiv:2205.01917

    2022 arXiv

  68. [76]

    Yuan, L.; Chen, D.; Chen, Y.-L.; Codella, N.; Dai, X.; Gao, J.; Hu, H.; Huang, X.; Li, B.; Li, C.; et al. 2021 a . Florence: A new foundation model for computer vision. arXiv preprint arXiv:2111.11432

    2021 arXiv

  69. [77]

    Yuan, Y.; Zhan, Y.; and Xiong, Z. 2023. Parameter-Efficient Transfer Learning for Remote Sensing Image-Text Retrieval. IEEE Transactions on Geoscience and Remote Sensing

    2023

  70. [78]

    Yuan, Z.; Zhang, W.; Rong, X.; Li, X.; Chen, J.; Wang, H.; Fu, K.; and Sun, X. 2021 b . A lightweight multi-scale crossmodal text-image retrieval method in remote sensing. IEEE Transactions on Geoscience and Remote Sensing, 60: 1--19

    2021

  71. [79]

    Zhang, X.; Xu, M.; Qiu, D.; Yan, R.; Lang, N.; and Zhou, X. 2024. MediCLIP: Adapting CLIP for Few-shot Medical Image Anomaly Detection. arXiv preprint arXiv:2405.11315

    2024 arXiv

  72. [80]

    Zhao, Z.; Liu, Y.; Wu, H.; Li, Y.; Wang, S.; Teng, L.; Liu, D.; Li, X.; Cui, Z.; Wang, Q.; et al. 2023. Clip in medical imaging: A comprehensive survey. arXiv preprint arXiv:2312.07353

    2023 arXiv

  73. [81]

    C.; and Liu, Z

    Zhou, K.; Yang, J.; Loy, C. C.; and Liu, Z. 2022 a . Conditional prompt learning for vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 16816--16825

    2022

  74. [82]

    C.; and Liu, Z

    Zhou, K.; Yang, J.; Loy, C. C.; and Liu, Z. 2022 b . Learning to prompt for vision-language models. International Journal of Computer Vision, 130(9): 2337--2348

    2022

  75. [83]

    Zhu, W.; Liu, J.; and Huang, Y. 2023. Hnssl: Hard negative-based self-supervised learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 4777--4786

    2023

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.