TIMO improves training-free CLIP few-shot classification by mutually guiding text and image features, and a tuned variant TIMO-S reports state-of-the-art accuracy with roughly 100x less time than training-required methods.
Prompt Distribution Learning
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
We present prompt distribution learning for effectively adapting a pre-trained vision-language model to address downstream recognition tasks. Our method not only learns low-bias prompts from a few samples but also captures the distribution of diverse prompts to handle the varying visual representations. In this way, we provide high-quality task-related content for facilitating recognition. This prompt distribution learning is realized by an efficient approach that learns the output embeddings of prompts instead of the input embeddings. Thus, we can employ a Gaussian distribution to model them effectively and derive a surrogate loss for efficient training. Extensive experiments on 12 datasets demonstrate that our method consistently and significantly outperforms existing methods. For example, with 1 sample per category, it relatively improves the average result by 9.1% compared to human-crafted prompts.
fields
cs.CV 1years
2024 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Text and Image Are Mutually Beneficial: Enhancing Training-Free Few-Shot Classification with CLIP
TIMO improves training-free CLIP few-shot classification by mutually guiding text and image features, and a tuned variant TIMO-S reports state-of-the-art accuracy with roughly 100x less time than training-required methods.