Pith. sign in

REVIEW 4 major objections 7 minor 38 references

MSGCoOp: Multiple Semantic-Guided Context Optimization for Few-Shot Learning

T0 review · 4 major / 7 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that training several parallel textual prompts on frozen CLIP, anchored by GPT-4-generated class descriptions and a diversity loss, improves few-shot generalization to unseen classes compared with single-prompt baselines…

desk verdict A clean, cheap prompt-ensemble recipe whose headline gain mostly rides on one EuroSAT outlier, and with seed statistics too thin to call it 'significant'. read the letter →

arxiv 2507.21786 v1 pith:JUI5FLJU submitted 2025-07-29 cs.CV

Zhaolong Wang , Tongfeng Sun , Mingzheng Du , Yachao Huang This is my paper
classification cs.CV
keywords few-shotlearningvision-languagemodelspromptCLIPLLMdescriptiongenerationensembledomaingeneralizationbase-to-novel
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to establish that a text-only prompt ensemble — several learnable context vectors trained in parallel on a frozen CLIP and fused at the logit level — can improve few-shot generalization to unseen classes without adding visual prompts or extra network layers. The ensemble is anchored by two regularizers: a semantic guidance loss that pulls the average prompt embedding toward CLIP-encoded, GPT-4-generated class descriptions, and a diversity loss that pushes the prompts' embeddings toward orthogonality. On the standard 11-dataset base-to-novel protocol, MSGCoOp reports an average harmonic mean of 78.10% against KgCoOp's 77.00%, and it also reports a small cross-domain robustness gain. The payoff, if correct, is a simple and cheap recipe for adapting CLIP to new tasks that generalizes better to classes never seen in training.

What carries the argument

The load-bearing object is the prompt ensemble: N independent learnable context vectors $v_n$ (length $M$, shared across classes) concatenated with the class-name token and encoded by the frozen CLIP text encoder, with predictions obtained by averaging the N logits. Two losses carry the semantic content: the semantic guidance loss $L_{sg} = \frac{1}{N_c}\sum_i (1 - \operatorname{sim}(\bar{w}^{\mathrm{soft}}_i, w^{\mathrm{sem}}_i))$ minimizes the cosine distance between the mean prompt embedding and the mean embedding of the top-k class descriptions; the diversity loss $L_{div}$ minimizes the average squared cosine similarity among the N prompt embeddings. The full objective is $L_{total} = L_{ce} + \lambda_{sg} L_{sg} + \lambda_{div} L_{div}$, optimized only over the context vectors.

What would settle it

Take a dataset such as EuroSAT and replace each class's selected descriptions with an equal number of sentences drawn randomly from other classes' descriptions, keeping the filtering step untouched; if the reported harmonic-mean gain over KgCoOp persists, the semantic content of the descriptions is not what drives the improvement. Alternatively, manually inspect the top-k descriptions for a few classes and count how many are generic or shared across classes, and check whether that count correlates with the per-dataset gains.

Watch

Extended reading notes

Core claim

The paper's central claim is that a single learnable prompt underrepresents a class, while several parallel prompts that are semantically guided and mutually orthogonal capture complementary discriminative aspects. MSGCoOp constructs N context vectors per class, encodes each with the frozen CLIP text encoder, and averages the resulting logits for classification. The objective adds a semantic guidance loss that aligns the mean prompt embedding with the mean embedding of filtered LLM descriptions, and a diversity loss that penalizes squared cosine similarity among prompts. The reported effect is an average harmonic mean of 78.10% versus 77.00% for KgCoOp, with the largest novel-class gains on EuroSAT and DTD, and a 0.30% cross-domain average gain over KgCoOp. The paper interprets this as evidence that diversity plus external semantic knowledge prevents the overfitting to base classes that hurts single-prompt methods.

Load-bearing premise

The method assumes that the GPT-4-generated class descriptions, once filtered by CLIP's own text-encoder self-similarity, are accurate and discriminative semantic anchors for every class and domain.

Editorial extensions

If this is right

  • If MSGCoOp's reported results hold, a purely textual prompt ensemble with LLM descriptions outperforms KgCoOp on base-to-novel generalization (HM 78.10 vs 77.00) and on cross-domain robustness (avg 60.41 vs 60.11) while adding no trainable parameters beyond the context vectors.
  • The gains are concentrated on novel classes (average +1.69%), especially in specialized domains like EuroSAT (+10.63%) and DTD (+1.65%), suggesting the recipe helps most where the zero-shot prior is weakest.
  • Ablation results attribute +0.82% HM to LLM descriptions over hand-crafted templates and +0.58% to moving from one to multiple prompts, implying both components contribute and are roughly additive.
  • In cross-dataset transfer, MSGCoOp at epoch 100 underperforms baselines (64.17 vs 65.51 for KgCoOp) but at epoch 5 leads all of them (65.92), which the paper reports as early-training generalization that later overfits to ImageNet.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The similarity filter that keeps the most mutually consistent descriptions likely selects generic consensus statements rather than the most class-discriminative ones; filtering instead by similarity to the class-name embedding, or to zero-shot CLIP image features of base classes, is a natural testable variant the paper does not try.
  • The EuroSAT result — a +10.63-point novel-class jump over KgCoOp — is far outside the pattern of the other ten datasets, and auditing the GPT-4 descriptions for EuroSAT would show whether the jump comes from unusually informative satellite-land-cover sentences or from an artifact of the filter.
  • The epoch-5/epoch-100 cross-dataset gap suggests the semantic-guidance and diversity losses are tuned for quick adaptation on the source domain, so a validation-based early-stopping rule or a decaying semantic-guidance weight might convert the early peak into a stable advantage.
  • Because the semantic guidance relies on a manually provided super-category, the recipe does not yet extend to open-vocabulary or hierarchical class sets; an automatic category-from-LLM step would be needed, and the paper leaves that unaddressed.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes MSGCoOp, a prompt-learning method for CLIP that combines N parallel learnable context vectors, a semantic-guidance loss based on GPT-4-generated class descriptions filtered by the frozen CLIP text encoder, and a diversity-regularization loss that penalizes pairwise squared cosine similarity among prompt embeddings. Only the context vectors are optimized; the image and text encoders are frozen. The method is evaluated under the standard 16-shot base-to-novel protocol on 11 datasets, on cross-domain generalization from ImageNet to four variants, and on cross-dataset transfer from ImageNet to 10 datasets. The headline claims are a +1.10 harmonic-mean gain over KgCoOp in base-to-novel generalization and a +0.30 average gain in cross-domain generalization. The paper also reports ablations on the number of prompts, the semantic-guidance loss, and the diversity weight, and it releases code.

Significance. If the reported gains are statistically reliable, MSGCoOp is a simple and parameter-efficient contribution: it adds only parallel context vectors, two scalar loss weights, and an offline LLM description step, and it improves over strong textual-prompt baselines on the standard 11-dataset protocol. The paper is also honest in reporting the failed cross-dataset result at 100 epochs (Section 4.4, Table 4), which is a useful negative result. The main weaknesses are statistical: 3-seed averages without error bars or significance tests, an average gain dominated by one dataset (EuroSAT), and a cross-domain gain of 0.30% that is likely within seed noise. These issues currently prevent the central claim of 'significant improvement' from being accepted at face value.

major comments (4)
  1. [Abstract; Section 4.2, Table 2] The claim that MSGCoOp 'significantly improves' base-to-novel generalization is not supported by the reported statistics. All results are averaged over only 3 seeds (Section 4.1), no standard deviations or significance tests are reported, and the +1.10 average HM gain is driven overwhelmingly by EuroSAT (+6.75 HM; Table 2j). Using the rounded values in Table 2, excluding EuroSAT reduces the average gain to roughly +0.6 percentage points, and StanfordCars is negative (-0.51 HM). Sub-percent gains on most datasets cannot be distinguished from seed noise at this scale. Please report per-dataset standard deviations, ideally per-seed results, and either significance tests or a clear statement that the gains are not statistically tested. The word 'significantly' should be removed or justified.
  2. [Section 4.4, Table 4] The cross-dataset experiment is an admitted failure at the standard 100-epoch setting: MSGCoOp reaches 64.17% average, below CLIP (65.12%), CoCoOp (65.74%), and KgCoOp (65.51%). The paper instead highlights the 5-epoch result (65.92%), but no validation-based early-stopping procedure is described, so it is unclear whether this is a protocol or a cherry-picked epoch. This result does not by itself invalidate the base-to-novel claim, but it directly limits the broader statement of 'enhanced robustness' and should be discussed as a limitation in the abstract and conclusion. Please specify how the early-stopping point would be chosen in practice, e.g., by a held-out validation set, and report results for the same protocol as the baselines.
  3. [Section 3.2, Eqs. (1)-(2); Section 3.4, Eq. (8)] The semantic-guidance mechanism depends on the quality of the LLM-generated descriptions, but the paper provides no inspection or failure analysis of the descriptions. The filtering criterion in Eqs. (1)-(2) selects the top-k descriptions by average self-similarity, which measures internal consistency among descriptions, not whether they are accurate or discriminative for the target class relative to other classes. Since Eq. (8) pulls the prompt embeddings toward the mean description embedding, inaccurate or generic descriptions could systematically bias the prompts. The aggregate gain in Table 5 is positive, but the mechanism remains a black box. Please include example descriptions for a few classes per dataset, especially for EuroSAT (the largest gain) and StanfordCars (the only negative dataset), and quantify how often the filter removes implausible descriptions.
  4. [Section 4.1; Sections 4.5.2-4.5.3] The hyperparameter selection is not fully specified. The paper states that λsg = 8.0 is set 'following the configuration of KgCoOp [33]', but KgCoOp does not have a semantic-guidance loss; this sentence is unclear. More importantly, the paper does not describe any validation split or tuning procedure for λsg, λdiv, N, k, and the number of seeds used for hyperparameter selection. If these were tuned on the reported test sets, the comparisons would be optimistic. Please clarify how each hyperparameter was chosen and whether the reported numbers come from the same configuration for all datasets.
minor comments (7)
  1. [Section 3.2, Eq. (1)] The notation is inconsistent: K is described as the total number of raw descriptions, but the paper never states the value of K, and k (top-k) is set to 4. Please state K explicitly and describe how the number of LLM prompts and generated descriptions relates to K.
  2. [Figure 2] The figure contains typographical and formatting issues: 'Instuction T' and the line 'a flower {} a pet {} a {} scene types of flowers types of pets types of scenes' are not readable. Please clean up the figure and make the template examples complete sentences.
  3. [Section 4.1; Section 4.3] The main base-to-novel experiments use N = 4 prompts, while the cross-domain experiments use N = 3 'due to resource constraints'. This inconsistency means Table 3 is not directly comparable with Table 2 in terms of model capacity, and the cross-domain gains over single-prompt baselines partly reflect an increased number of learnable vectors. Please justify the choice or report cross-domain results with the same N.
  4. [Section 4.5.1, Figure 3] The caption and text report HM values, but the text says 'Moving from N = 1 to N = 6 yields a +0.67% improvement in HM', while the figure shows a peak at N = 6 of 78.19% and N = 4 of 78.10%. Please clarify whether the figure shows HM or average accuracy and ensure the numbers are consistent.
  5. [Section 4.5.3, Figure 4] The text states that performance improves as λdiv increases from 0.0 to 1.0, but the x-axis of Figure 4 starts at 0.10 and does not include 0.0. Please include the λdiv = 0 point or adjust the text.
  6. [Section 5; Related Work] The paper claims the method keeps computational cost 'close to CoOp', but no runtime, FLOPs, or parameter count is reported for the N-prompt ensemble. Since the text encoder is run N times per class, a quantitative efficiency comparison would make this claim verifiable.
  7. [Throughout] There are several typos and style issues: 'avaliable' in the abstract, 'Instuction' in Figure 2, and the incomplete sentence 'While this manual approach may not discover optimal prompt variations' at the end of Section 2.4. These should be corrected.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the claimed base-to-novel and cross-domain gains are empirical benchmark measurements, not identities forced by the paper's equations or by self-citation.

full rationale

The paper's central claim is an empirical accuracy improvement over KgCoOp on standard 11-dataset and domain-generalization protocols. The semantic guidance mechanism (Eq. 8) is a regularizer that pulls learned prompt embeddings toward the mean of GPT-4-generated, CLIP-filtered class descriptions, but the final classification accuracy also depends on cross-entropy training on images (Eq. 6), diversity regularization (Eq. 9), and the frozen CLIP encoders. Nothing in the derivation defines the reported harmonic mean as equal to the semantic guidance loss or to any fitted parameter. The use of the same frozen CLIP text encoder for filtering descriptions (Eqs. 1-2) and for computing prompt embeddings (Eqs. 4-8) reflects a shared feature space, not a constructional equivalence: the learned prompts are not the descriptions by definition, and the loss does not force a particular accuracy value. Hyperparameters such as λsg = 8.0, λdiv = 1.0, k = 4, and N = 4 are chosen settings, not fitted quantities later reported as predictions. No load-bearing self-citation was found: the references to KgCoOp, CoCoOp, MaPLe, and related methods are prior external baselines, and no uniqueness theorem or prior result by this paper's authors is invoked to forbid alternatives. The absence of error bars and the sensitivity of the average gain to EuroSAT are statistical-support concerns rather than circularity, so they are outside the scope of this pass. The derivation chain is therefore self-contained with respect to the paper's empirical claims.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central contribution is an empirical tuning recipe: the method introduces two new loss terms with weights λsg and λdiv, an ensemble size N, and a filtering parameter k, all selected on the benchmark test metrics. No new theoretical objects are introduced. The main external dependency is GPT-4, whose output distribution is not controlled in the paper.

free parameters (5)
  • lambda_sg = 8.0
    Weight on the semantic guidance loss (Eq. 8); selected on the 11-dataset HM and stated to follow KgCoOp, but KgCoOp does not define this loss. No sensitivity analysis is shown.
  • lambda_div = 1.0
    Weight on the diversity loss (Eq. 9); selected from {0.1, 0.25, 0.5, 1.0, 2.0, 4.0, 8.0} in Figure 4 by maximizing the same test accuracy that is later reported.
  • N_prompts = 4 (base-to-novel), 3 (cross-domain)
    Number of parallel context vectors; chosen after Figure 3 showed N=6 only +0.09% HM over N=4. N=3 was used for cross-domain due to GPU memory limits, an inconsistency not discussed.
  • k_descriptions = 4
    Number of LLM descriptions retained per class after self-similarity filtering (Section 3.2); no ablation on k is reported.
  • num_llm_prompt_templates = 5
    Number of structured question templates used to query GPT-4; a design choice with no sensitivity analysis.
assumptions (4)
  • domain assumption The frozen CLIP ViT-B/16 text and image encoders provide a reliable shared embedding space for prompt optimization, description filtering, and classification.
    Every loss and similarity (Eqs. 1, 5, 8, 9) is computed in CLIP space; weak text-image alignment on a domain would degrade all components.
  • ad hoc to paper The top-k descriptions selected by CLIP self-similarity (Eqs. 1-2) are representative and semantically useful for every class in all 11 datasets.
    No human inspection, quality scores, or failure analysis of the GPT-4 descriptions is provided; the Lsg loss assumes these anchors are beneficial.
  • domain assumption The base-to-novel split with 16-shot training is a valid measure of few-shot generalization.
    This is the standard CoOp/KgCoOp protocol, taken without justification.
  • standard math Cosine similarity is an appropriate alignment metric for both description filtering and prompt-description guidance.
    Common in CLIP-based work; no alternative metrics are considered.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MSGCoOp: Multiple Semantic-Guided Context Optimization for Few-Shot Learning." pith.science (2026). https://pith.science/paper/JUI5FLJU

@misc{pith2026250721786,
  author       = {Pith},
  title        = {Pith review of: MSGCoOp: Multiple Semantic-Guided Context Optimization for Few-Shot Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JUI5FLJU}},
  note         = {Machine review of arXiv:2507.21786}
}
read the original abstract

Vision-language pre-trained models (VLMs) such as CLIP have demonstrated remarkable zero-shot generalization, and prompt learning has emerged as an efficient alternative to full fine-tuning. However, existing methods often struggle with generalization to novel classes, a phenomenon attributed to overfitting on seen classes and forgetting general knowledge. Furthermore, recent approaches that improve generalization often introduce complex architectures or heavy computational overhead. In this paper, we propose a Multiple Semantic-Guided Context Optimization (MSGCoOp) framework to enhance few-shot generalization while maintaining computational efficiency. Our approach leverages an ensemble of parallel learnable context vectors to capture diverse semantic aspects. To enrich these prompts, we introduce a semantic guidance mechanism that aligns them with comprehensive class descriptions automatically generated by a Large Language Model (LLM). Furthermore, a diversity regularization loss encourages the prompts to learn complementary and orthogonal features, preventing them from collapsing into redundant representations. Extensive experiments on 11 benchmark datasets show that MSGCoOp significantly improves performance on base-to-novel generalization, achieving an average harmonic mean improvement of 1.10\% over the strong KgCoOp baseline. Our method also demonstrates enhanced robustness in cross-domain generalization tasks. Our code is avaliable at: \href{https://github.com/Rain-Bus/MSGCoOp}{https://github.com/Rain-Bus/MSGCoOp}.

Figures

Figures reproduced from arXiv: 2507.21786 by the authors.

Figure 1
Figure 1. Overview of the MSGCoOp framework. Multi-prompt [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Pipeline for generating semantic-rich class descriptions [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Impact of ensemble size N on harmonic mean (HM). Our framework employs a multi-prompt ensemble strat￾egy to improve generalization. To evaluate its effectiveness and select a suitable configuration, we vary the number of parallel prompts N ∈ {1, 2, 3, 4, 5, 6} and report results in [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Impact of diversity regularization strength [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

38 extracted references · 22 canonical work pages

  1. [33]

    Visual- language prompt tuning with knowledge-guided context op- timization

    Hantao Yao, Rui Zhang, and Changsheng Xu. Visual- language prompt tuning with knowledge-guided context op- timization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6757–6767,

  2. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 ,

    arXiv

  3. [2]

    Flamingo: a visual language model for few-shot learning

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Men- sch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems, 35:23716–23736,

  4. [3]

    Food-101–mining discriminative components with random forests

    Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. Food-101–mining discriminative components with random forests. In ECCV, pages 446–461. Springer, 2014. 6

    work page 2014

  5. [4]

    Lan- guage models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Sub- biah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakan- tan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Lan- guage models are few-shot learners. Advances in neural in- formation processing systems, 33:1877–1901, 2020. 2

    1901

  6. [5]

    Describing textures in the wild

    Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. Describing textures in the wild. In CVPR, pages 3606–3613, 2014. 6

    work page 2014

  7. [6]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, pages 248–255. Ieee, 2009. 6

    work page 2009

  8. [7]

    Learning gener- ative visual models from few training examples: An incre- mental bayesian approach tested on 101 object categories

    Li Fei-Fei, Rob Fergus, and Pietro Perona. Learning gener- ative visual models from few training examples: An incre- mental bayesian approach tested on 101 object categories. In 2004 conference on computer vision and pattern recognition workshop, pages 178–178. IEEE, 2004. 6

    2004

Show all 38 references
  1. [8]

    Clip-adapter: Better vision-language models with feature adapters

    Peng Gao, Shijie Geng, Renrui Zhang, Teli Ma, Rongyao Fang, Yongfeng Zhang, Hongsheng Li, and Yu Qiao. Clip-adapter: Better vision-language models with feature adapters. International Journal of Computer Vision, 132(2): 581–595, 2024. 1

    2024

  2. [9]

    Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification

    Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12(7):2217–2226, 2019. 6

    2019

  3. [10]

    The many faces of robust- ness: A critical analysis of out-of-distribution generalization

    Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kada- vath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, et al. The many faces of robust- ness: A critical analysis of out-of-distribution generalization. In ICCV, pages 8340–8349, 2021. 6

    2021

  4. [11]

    Natural adversarial examples

    Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Stein- hardt, and Dawn Song. Natural adversarial examples. In CVPR, pages 15262–15271, 2021. 6

    2021

  5. [12]

    Scaling up visual and vision-language representa- tion learning with noisy text supervision

    Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representa- tion learning with noisy text supervision. In International Conference on Machine Learning, pages 4904–4916. PMLR,

  6. [13]

    10 Maple: Multi-modal prompt learning

    Muhammad Uzair Khattak, Hanoona Rasheed, Muham- mad Maaz, Salman Khan, and Fahad Shahbaz Khan. 10 Maple: Multi-modal prompt learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 19113–19122, 2023. 3, 5

    2023

  7. [14]

    Aapl: Adding attributes to prompt learning for vision-language models

    Gahyeon Kim, Sohee Kim, and Seokju Lee. Aapl: Adding attributes to prompt learning for vision-language models. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 1572–1582, 2024. 3

    2024

  8. [15]

    3d object representations for fine-grained categorization

    Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine-grained categorization. In ICCV, pages 554–561, 2013. 6

    2013

  9. [16]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In In- ternational conference on machine learning , pages 19730– 19742. PMLR, 2023. 2

    2023

  10. [17]

    Deepseek-v3 technical report

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024. 2

    2024 arXiv

  11. [18]

    Visual instruction tuning, 2023

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, 2023. 2

    2023

  12. [19]

    Learning customized visual models with retrieval-augmented knowledge

    Haotian Liu, Kilho Son, Jianwei Yang, Ce Liu, Jianfeng Gao, Yong Jae Lee, and Chunyuan Li. Learning customized visual models with retrieval-augmented knowledge. InProceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 15148–15158, 2023. 2

    2023

  13. [20]

    Fine-grained visual classi- fication of aircraft

    Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew Blaschko, and Andrea Vedaldi. Fine-grained visual classi- fication of aircraft. arXiv preprint arXiv:1306.5151 , 2013. 6

    2013 arXiv

  14. [21]

    Slip: Self-supervision meets language-image pre- training

    Norman Mu, Alexander Kirillov, David Wagner, and Sain- ing Xie. Slip: Self-supervision meets language-image pre- training. In European conference on computer vision, pages 529–544. Springer, 2022. 2

    2022

  15. [22]

    Automated flower classification over a large number of classes

    Maria-Elena Nilsback and Andrew Zisserman. Automated flower classification over a large number of classes. In 2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing, pages 722–729. IEEE, 2008. 6

    2008

  16. [23]

    Cats and dogs

    Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and CV Jawahar. Cats and dogs. In 2012 IEEE conference on computer vision and pattern recognition, pages 3498–3505. IEEE, 2012. 6

    2012

  17. [24]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. In International Conference on Machine Learning...

    2021

  18. [25]

    Do imagenet classifiers generalize to im- agenet? In International Conference on Machine Learning, pages 5389–5400

    Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do imagenet classifiers generalize to im- agenet? In International Conference on Machine Learning, pages 5389–5400. PMLR, 2019. 6

    2019

  19. [26]

    Exploiting cloze ques- tions for few shot text classification and natural language in- ference

    Timo Schick and Hinrich Sch ¨utze. Exploiting cloze ques- tions for few shot text classification and natural language in- ference. arXiv preprint arXiv:2001.07676, 2020. 3

    2001 arXiv

  20. [27]

    K-lite: Learning transferable visual models with external knowledge

    Sheng Shen, Chunyuan Li, Xiaowei Hu, Yujia Xie, Jianwei Yang, Pengchuan Zhang, Zhe Gan, Lijuan Wang, Lu Yuan, Ce Liu, et al. K-lite: Learning transferable visual models with external knowledge. Advances in Neural Information Processing Systems, 35:15558–15573, 2022. 2

    2022

  21. [28]

    Meta-adapter: An online few-shot learner for vision-language model

    Lin Song, Ruoyi Xue, Hang Wang, Hongbin Sun, Yixiao Ge, Ying Shan, et al. Meta-adapter: An online few-shot learner for vision-language model. Advances in Neural Information Processing Systems, 36:55361–55374, 2023. 1

    2023

  22. [29]

    Ucf101: A dataset of 101 human actions classes from videos in the wild

    Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah. Ucf101: A dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv:1212.0402, 2012. 6

    2012 arXiv

  23. [30]

    Learning robust global representations by penalizing local predictive power

    Haohan Wang, Songwei Ge, Zachary Lipton, and Eric P Xing. Learning robust global representations by penalizing local predictive power. In NeurIPS, 2019. 6

    2019

  24. [31]

    Sun database: Large-scale scene recognition from abbey to zoo

    Jianxiong Xiao, James Hays, Krista A Ehinger, Aude Oliva, and Antonio Torralba. Sun database: Large-scale scene recognition from abbey to zoo. In 2010 IEEE computer so- ciety conference on computer vision and pattern recognition, pages 3485–3492. IEEE, 2010. 6

    2010

  25. [32]

    Unified contrastive learning in image-text-label space

    Jianwei Yang, Chunyuan Li, Pengchuan Zhang, Bin Xiao, Ce Liu, Lu Yuan, and Jianfeng Gao. Unified contrastive learning in image-text-label space. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 19163–19173, 2022. 2

    2022

  26. [34]

    Filip: Fine-grained interactive language-image pre-training

    Lewei Yao, Runhui Huang, Lu Hou, Guansong Lu, Minzhe Niu, Hang Xu, Xiaodan Liang, Zhenguo Li, Xin Jiang, and Chunjing Xu. Filip: Fine-grained interactive language-image pre-training. arXiv preprint arXiv:2111.07783, 2021. 2

    2021 arXiv

  27. [35]

    Florence: A new foundation model for computer vision

    Lu Yuan, Dongdong Chen, Yi-Ling Chen, Noel Codella, Xiyang Dai, Jianfeng Gao, Houdong Hu, Xuedong Huang, Boxin Li, Chunyuan Li, et al. Florence: A new foundation model for computer vision. arXiv preprint arXiv:2111.11432, 2021. 2

    2021 arXiv

  28. [36]

    Conditional prompt learning for vision-language mod- els

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Conditional prompt learning for vision-language mod- els. In Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition , pages 16816–16825,

  29. [37]

    Learning to prompt for vision-language models

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision-language models. In- ternational Journal of Computer Vision, 130(9):2337–2348,

  30. [38]

    Prompt-aligned gradient for prompt tuning

    Beier Zhu, Yulei Niu, Yucheng Han, Yue Wu, and Hanwang Zhang. Prompt-aligned gradient for prompt tuning. In Pro- ceedings of the IEEE/CVF international conference on com- puter vision, pages 15659–15669, 2023. 3, 7 11

    2023

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.