Pith. sign in

REVIEW 4 major objections 4 minor 43 references

Does VLM Classification Benefit from LLM Description Semantics?

T0 review · 4 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read LLM description semantics boost VLM classification once selected, not raw.

desk verdict Useful new evaluation protocol, but the central claim is underdetermined by the missing random-pool control. read the letter →

arxiv 2412.11917 v3 pith:AINAOZI5 submitted 2024-12-16 cs.CV

Pingchuan Ma , Lennart Rietdorf , Dmytro Kotovenko , Vincent Tao Hu , Björn Ommer This is my paper · ORCID
classification cs.CV
keywords vision-languagemodelsCLIPLLMdescriptionsclassname-freeevaluationensemblingeffectdescriptionselectionzero-shotclassificationexplainability
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether the accuracy gains that vision-language models (VLMs) get from LLM-generated descriptions come from genuine semantics or from a noisy ensembling effect. It argues that previous evaluation setups cannot separate the two, because every prompt contains the class name and multiple modified prompts act as test-time augmentation. The authors introduce a classname-free setup in which only one prompt per class contains the class name, and all other prompts are descriptions without it. Under this setup, they show that raw LLM assignments barely beat random text, while descriptions selected by their algorithm produce consistent gains across seven datasets. Their conclusion is that VLM classification does benefit from LLM description semantics, but only when the descriptions are correctly selected.

What carries the argument

The load-bearing mechanism is the lookup similarity matrix $S$, built by averaging image-description similarities between $n$ labeled reference images per class and a global pool of classname-free LLM descriptions. For each test image, the method first retrieves the top-$k$ candidate classes using only the class-name prompt, forming a local label neighborhood $A(x_i)$. Within that neighborhood, it keeps only descriptions whose averaged similarity to the target class exceeds their averaged similarity to every other candidate class (the positive subset $S^+$), and then selects the top-$m$ descriptions by mean positive difference. These selected descriptions are used as classname-free prompts alongside a single class-name prompt, with a relative weight $w_{\text{cls}}$ controlling how much the class name counts.

What would settle it

Run the same classname-free evaluation with a few-shot baseline that uses the same labeled reference images to build class prototypes (e.g., averaged image embeddings) and classifies by nearest prototype, without any LLM descriptions or selection; if that baseline matches or beats the proposed method's accuracy, then the labeled reference data, not the description selection, carries the performance.

Watch

Extended reading notes

Core claim

The paper's central claim is that LLM-generated descriptions carry genuine semantic signal that improves VLM classification, but this signal is invisible under standard ensembling evaluation because the class name, repeated in every prompt, dominates the average. To expose it, the authors propose a classname-free evaluation: exactly one text prompt per class contains the class name, and the remaining prompts are descriptions without it. In this setup, raw LLM assignments perform barely better than random text, while descriptions selected by their method, which uses feedback from the VLM embedding space to pick descriptions that differentiate each class from its local competitors, improve accuracy on all seven datasets tested. The paper concludes that the benefit of LLM semantics is real but conditional on selection: the descriptions must be discriminative within the local neighborhood of ambiguous classes.

Load-bearing premise

The method needs a small set of labeled reference images from each dataset's training split to build the similarity table, so the measured gains could in principle come from that labeled data rather than from the description-selection step, since the paper compares only against zero-shot baselines.

Editorial extensions

If this is right

  • If the claim is correct, evaluation protocols for description-based VLM classification should report classname-free accuracy, not just ensembled accuracy.
  • Raw LLM assignments are not a reliable way to transfer semantic knowledge; selection based on VLM feedback is needed.
  • A handful of well-chosen descriptions (e.g., 5) can outperform methods using hundreds or thousands of prompts per class.
  • The method provides a way to audit whether a VLM's decisions are explainable through distinctive verbal descriptions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural next step is to feed the selection feedback back into the LLM in an agentic loop, letting the LLM refine the description pool; the paper notes this as future work.
  • The use of labeled reference images suggests that a fair comparison against true few-shot methods is needed to isolate the source of gain, and this comparison is not made in the paper.
  • The Pareto-like distribution of distinctiveness scores implies that even smaller description budgets ($m < 5$) might suffice on some datasets, and that the selection criterion could serve as a general pruning tool for description pools.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper asks whether VLM classification gains from LLM-generated descriptions reflect genuine description semantics or a noisy test-time ensembling effect. To separate these, the authors propose a classname-free evaluation setup in which, for each class, only one text prompt contains the class name and all other descriptions are classname-free; they also propose a selection algorithm (Algorithm 1) that, for each test image, picks m classname-free descriptions from a global pool that best separate the k candidate classes in CLIP embedding space, using a lookup similarity matrix S built from labeled reference images. Experiments on seven datasets with CLIP ViT-B/32 and ViT-L/14, plus additional ALIGN results, show accuracy gains for the selected descriptions over LLM assignments and random assignments, and the authors conclude that VLM classification benefits from LLM description semantics when descriptions are correctly selected.

Significance. The proposed classname-free protocol is a useful evaluation lens: it prevents the trivially ensembling-based gains demonstrated by Roth et al. and gives a concrete test for whether a description assignment is genuinely discriminative in the absence of the class name. The selection algorithm is simple, reproducible (code is released), and the paper includes welcome robustness checks across two CLIP backbones, two LLMs for pool construction, and a second VLM (ALIGN). If the central claim holds after the missing controls are added, the paper would be a solid contribution to explainable VLM classification. The main uncertainty is not whether the proposed selection improves accuracy (it does) but whether the improvement can be attributed to LLM description semantics rather than to the supervised selection of arbitrary text embeddings.

major comments (4)
  1. [Section 4.2, Tables 1 and 2] The paper's headline claim—that the gains come from LLM description semantics—is not isolated from the selection procedure, because the control that would separate the two factors is missing. Table 1 reports a 'Random Ours' condition (random description pool with the same selection heuristic) and it nearly matches 'DCLIP Ours' in the classname-free setup on 5 of 7 datasets (e.g., ImageNet 62.18 vs 62.57; Flowers 66.12 vs 66.37). Table 2, which is the strongest evidence for the semantic claim because it uses wcls=0, compares 'Ours' only against LLM assignments and random assignments; it does not include a random-pool version of 'Ours' under wcls=0. If the random-pool selection matches 'Ours' there as it does in Table 1, the accuracy is attributable to supervised selection of arbitrary text embeddings, not to LLM semantics. Please add this control (random P with Algorithm 1 and wcls=0) for all seven datasets, and report the gap between random-pool and LLM-pool selections.
  2. [Section 4.1, Algorithm 1] The method is described as training-free, but it uses labeled reference images from the standard training split to build S (Algorithm 1, line 1, and Section 4.1). This is a form of supervised model selection, and the zero-shot baselines (DCLIP, WaffleClip, FuDD) do not use such labels, making the comparison asymmetric. To isolate the contribution of description selection, add a few-shot baseline that uses the same n reference images per class and the same CLIP features (for example, nearest-class-mean or a linear probe). Note that such a baseline is not a substitute for the random-pool control of the previous comment: the few-shot baseline cannot distinguish LLM semantics from selection of arbitrary embeddings.
  3. [Appendix A.9, Table 11] For EuroSAT, the paper states that no train split was provided and that n=1000 was chosen arbitrarily. If the 1000 images per class used to build S include images from the evaluation set, the large EuroSAT gain (e.g., 49.89 vs 43.31 in Table 1) would be contaminated by leakage of test labels into the selection step. Please specify exactly where the EuroSAT reference images come from (e.g., a random subset of the original dataset disjoint from the test images, or the standard split used by prior work), and rerun the EuroSAT experiments with a properly disjoint selection set if necessary.
  4. [Table 1 caption; Section 4.2] Reporting 'the best-performing wcls of the respective assignment' means that a free parameter is tuned on the test set for each assignment. This can bias the reported peak accuracy and makes the comparison between assignments uncontrolled. Please report accuracy as a function of wcls for all compared assignments (as already done for the proposed method in Figure 3), or use a validation split to choose wcls for every method, and report the chosen values in the tables.
minor comments (4)
  1. [Appendix A.10, Table 12] The DTD value for 'Contrastive GPT3.5 w/ Our Selection' is 80.78, which is inconsistent with the DTD values around 48 in Tables 1, 3, and 6; this appears to be a typo and should be corrected.
  2. [Table 1] The term 'Random Ours' is not defined in the main text; please state explicitly whether it is a random description pool with Algorithm 1 selection or a random assignment of existing descriptions to classes, since the distinction is crucial for interpreting the control.
  3. [Section 3.2, Equation (5)] The sentence 'Weights of the classname-free descriptions are normalized to one' is ambiguous: the classname-free weights sum to one but the classname weight wcls is added on top, so the final weighting is not a convex combination. Please clarify the intended normalization.
  4. [Appendix A.7] For Flowers102, DCLIP descriptions were generated with GPT-3.5 using prompts from their codebase; please state how many descriptions per class were generated and how this differs from the original DCLIP distribution, since Table 3 compares against DCLIP on Flowers102.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the selection procedure uses labeled reference images as a supervised input, but the reported test accuracies are not equal to that input by construction.

full rationale

The paper's derivation chain is not circular. Algorithm 1 builds a lookup similarity matrix S from labeled training images and selects descriptions that best separate local candidate classes; the selected descriptions are then evaluated on test images. This is a supervised selection procedure, not a derivation whose output is fixed by its input: the same selection rule applied to a random description pool ('Random Ours' in Table 1) produces different accuracies, and in the wcls=0 scenario randomly assigned descriptions fail (Table 2), so the reported gains are not forced by construction. The central attribution claim—that the gains reflect LLM description semantics rather than the supervised selection of arbitrary text embeddings—is not fully isolated, because the paper does not report the wcls=0 'Random Ours' control and tunes wcls on the test set ('We use the best-performing wcls of the respective assignment to ensure a fair comparison'). These are soundness/validity concerns about a missing control and test-set hyperparameter selection, not circularity: no equation in Section 3 equates the output metric to the selection input, and the paper cites no self-authored results to justify its premises. The comparison against zero-shot baselines without a few-shot baseline using the same labeled references is another attribution gap, but it does not make the derivation equivalent to its inputs. Hence no circular step is identifiable under the required standard of exhibiting Eq. X = Eq. Y by construction or a fitted parameter renamed as a prediction.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The paper's contribution rests on availability of labeled reference images (n per class) and a good description pool. These are inputs from the dataset and LLM, not derived. The selection heuristic has no trainable parameters, but the test-set wcls tuning is a free parameter.

free parameters (4)
  • wcls = chosen per dataset as best-performing on test set
    Weight for the single classname prompt in Eq. 5, tuned on test data and reported as the peak accuracy.
  • k = 3
    Number of preliminary candidate labels per test image, fixed to 3 in the main experiments.
  • n = min class size or 1000
    Number of reference images per class used to build the similarity matrix S.
  • m = 5
    Number of descriptions selected per candidate class.
assumptions (3)
  • domain assumption Cosine similarity between image and text embeddings reflects semantic compatibility.
    The selection algorithm and evaluation rely on CLIP's embedding distances to identify descriptions that activate for one class and not another (Section 3.2).
  • domain assumption The top-k candidate class set A(x_i) obtained from classname-only retrieval contains the true class with high probability.
    The method only selects descriptions within this local neighborhood, so if the true class is not in A(x_i), classification will fail regardless of description quality (Algorithm 1).
  • domain assumption A global description pool P generated by LLMs contains enough useful, discriminative descriptions.
    Performance depends on the pool quality; the paper shows that a contrastive pool improves results, so the pool itself is a loaded input.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Does VLM Classification Benefit from LLM Description Semantics?." pith.science (2026). https://pith.science/paper/AINAOZI5

@misc{pith2026241211917,
  author       = {Pith},
  title        = {Pith review of: Does VLM Classification Benefit from LLM Description Semantics?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AINAOZI5}},
  note         = {Machine review of arXiv:2412.11917}
}
read the original abstract

Accurately describing images with text is a foundation of explainable AI. Vision-Language Models (VLMs) like CLIP have recently addressed this by aligning images and texts in a shared embedding space, expressing semantic similarities between vision and language embeddings. VLM classification can be improved with descriptions generated by Large Language Models (LLMs). However, it is difficult to determine the contribution of actual description semantics, as the performance gain may also stem from a semantic-agnostic ensembling effect, where multiple modified text prompts act as a noisy test-time augmentation for the original one. We propose an alternative evaluation scenario to decide if a performance boost of LLM-generated descriptions is caused by such a noise augmentation effect or rather by genuine description semantics. The proposed scenario avoids noisy test-time augmentation and ensures that genuine, distinctive descriptions cause the performance boost. Furthermore, we propose a training-free method for selecting discriminative descriptions that work independently of classname-ensembling effects. Our approach identifies descriptions that effectively differentiate classes within a local CLIP label neighborhood, improving classification accuracy across seven datasets. Additionally, we provide insights into the explainability of description-based image classification with VLMs.

Figures

Figures reproduced from arXiv: 2412.11917 by the authors.

Figure 1
Figure 1. Are the extra semantics provided by LLM truly useful? Our method first identifies candidate labels using only the [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. In the conventional setup (left), using CLIP with LLM-assigned class descriptions or even random strings can some￾times result in performance gains due to the added semantics or the smoothing ensemble effect. However, when the classname is removed, i.e. under the proposed classname-free setup (right), these descriptions will fail to perform well, as only meaningful descriptions w.r.t. the class are useful. In contra… view at source ↗
Figure 3
Figure 3. Overall Performance of all datasets in classname-free setup. For descriptions assigned by our method and an LLM, [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Distinctiveness scores of randomly chosen images obtained by the training-free approach presented in Section [PITH_FULL_IMAGE:figures/full_fig_p018_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

43 extracted references · 21 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Belinkov, Y.; and Bisk, Y. 2018. Synthetic and Natural Noise Both Break Neural Machine Translation. In ICLR

    work page 2018

  4. [4]

    D.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; et al

    Brown, T.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J. D.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33: 1877--1901

    2020

  5. [5]

    Chen, J.; Yang, Z.; and Yang, D. 2020. MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classification. In ACL

    work page 2020

  6. [6]

    Chiquier, M.; Mall, U.; and Vondrick, C. 2024. Evolving interpretable visual classifiers with large language models. In ECCV

    work page 2024

  7. [7]

    Cimpoi, M.; Maji, S.; Kokkinos, I.; Mohamed, S.; and Vedaldi, A. 2014. Describing Textures in the Wild. In CVPR

    work page 2014

  8. [8]

    M.; Rosenfeld, E.; and Kolter, J

    Cohen, J. M.; Rosenfeld, E.; and Kolter, J. Z. 2019. Certified adversarial robustness via randomized smoothing. arXiv preprint arXiv:1902.02918

    arXiv 2019

Show all 43 references
  1. [9]

    Deng, J.; Dong, W.; Socher, R.; Li, L.-J.; Li, K.; and Fei-Fei, L. 2009. Imagenet: A large-scale hierarchical image database. In CVPR

    2009

  2. [10]

    Dosovitskiy, A.; Beyer, L.; Kolesnikov, A.; Weissenborn, D.; Zhai, X.; Unterthiner, T.; Dehghani, M.; Minderer, M.; Heigold, G.; Gelly, S.; et al. 2021. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR

    2021

  3. [11]

    Esfandiarpoor, R.; and Bach, S. H. 2024. Follow-Up Differential Descriptions: Language Models Resolve Ambiguities for Image Classification. ICLR

    2024

  4. [12]

    Y.; Gangal, V.; Wei, J.; Chandar, S.; Vosoughi, S.; Mitamura, T.; and Hovy, E

    Feng, S. Y.; Gangal, V.; Wei, J.; Chandar, S.; Vosoughi, S.; Mitamura, T.; and Hovy, E. 2021. A survey of data augmentation approaches for NLP. arXiv preprint arXiv:2105.03075

    2021 arXiv

  5. [13]

    Feng, Z.; Bair, A.; and Kolter, J. Z. 2023. Text Descriptions are Compressive and Invariant Representations for Visual Learning. arXiv:2307.04317

    2023 arXiv

  6. [14]

    Hakimov, S.; and Schlangen, D. 2023. Images in Language Space: Exploring the Suitability of Large Language Models for Vision & Language Tasks. In Rogers, A.; Boyd-Graber, J.; and Okazaki, N., eds., Findings of the Association for Computational Linguistics: ACL 2023. Toronto, C...

    2023

  7. [15]

    Hao, X.; Zhu, Y.; Appalaraju, S.; Zhang, A.; Zhang, W.; Li, B.; and Li, M. 2023. Mixgen: A new multi-modal data augmentation. In WACV

    2023

  8. [16]

    Helber, P.; Bischke, B.; Dengel, A.; and Borth, D. 2017. EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing

    2017

  9. [17]

    V.; Sung, Y.; Li, Z.; and Duerig, T

    Jia, C.; Yang, Y.; Xia, Y.; Chen, Y.-T.; Parekh, Z.; Pham, H.; Le, Q. V.; Sung, Y.; Li, Z.; and Duerig, T. 2021. Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision. arXiv:2102.05918

    2021 arXiv

  10. [18]

    Jin, H.; Li, Z.; Tong, R.; and Lin, L. 2018. A deep 3D residual CNN for false-positive reduction in pulmonary nodule detection. Medical physics, 45(5): 2097--2107

    2018

  11. [19]

    Kobayashi, S. 2018. Contextual Augmentation: Data Augmentation by Words with Paradigmatic Relations. In NAACL-HLT

    2018

  12. [20]

    Lin, T.-Y.; Maire, M.; Belongie, S.; Hays, J.; Perona, P.; Ramanan, D.; Doll \'a r, P.; and Zitnick, C. L. 2014. Microsoft coco: Common objects in context. In Computer Vision--ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 1...

    2014

  13. [21]

    Menon, S.; and Vondrick, C. 2023. Visual Classification via Description from Large Language Models. ICLR

    2023

  14. [22]

    Nilsback, M.-E.; and Zisserman, A. 2008. Automated Flower Classification over a Large Number of Classes. In Indian Conference on Computer Vision, Graphics and Image Processing

    2008

  15. [23]

    Ouyang, L.; Wu, J.; Jiang, X.; Almeida, D.; Wainwright, C.; Mishkin, P.; Zhang, C.; Agarwal, S.; Slama, K.; Ray, A.; et al. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35: 27730--27744

    2022

  16. [24]

    Pratt, S.; Covert, I.; Liu, R.; and Farhadi, A. 2023. What does a platypus look like? Generating customized prompts for zero-shot image classification. arXiv:2209.03320

    2023 arXiv

  17. [25]

    W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al

    Radford, A.; Kim, J. W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al. 2021. Learning transferable visual models from natural language supervision. In ICML

    2021

  18. [26]

    Recht, B.; Roelofs, R.; Schmidt, L.; and Shankar, V. 2019. Do imagenet classifiers generalize to imagenet? In International conference on machine learning, 5389--5400. PMLR

    2019

  19. [27]

    M.; Koepke, A.; Vinyals, O.; Schmid, C.; and Akata, Z

    Roth, K.; Kim, J. M.; Koepke, A.; Vinyals, O.; Schmid, C.; and Akata, Z. 2023. Waffling around for performance: Visual classification with random words and broad concepts. In ICCV, 15746--15757

    2023

  20. [28]

    S ahin, G. G. 2022. To Augment or Not to Augment? A Comparative Study on Text Augmentation Techniques for Low-Resource NLP. Computational Linguistics

    2022

  21. [29]

    Shtedritski, A.; Rupprecht, C.; and Vedaldi, A. 2023. What does clip know about a red circle? visual prompt engineering for vlms. In ICCV, 11987--11997

    2023

  22. [30]

    Y.; and He, L

    Sun, L.; Xia, C.; Yin, W.; Liang, T.; Philip, S. Y.; and He, L. 2020. Mixup-Transformer: Dynamic Data Augmentation for NLP Tasks. In Computational Linguistics

    2020

  23. [31]

    Szegedy, C.; Liu, W.; Jia, Y.; Sermanet, P.; Reed, S.; Anguelov, D.; Erhan, D.; Vanhoucke, V.; and Rabinovich, A. 2015. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, 1--9

    2015

  24. [32]

    Touvron, H.; Martin, L.; Stone, K.; Albert, P.; Almahairi, A.; Babaei, Y.; Bashlykov, N.; Batra, S.; Bhargava, P.; Bhosale, S.; et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

    2023 arXiv

  25. [33]

    N.; Kaiser, .; and Polosukhin, I

    Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, .; and Polosukhin, I. 2017. Attention is all you need. Advances in neural information processing systems, 30

    2017

  26. [34]

    Vogel, F.; Shvetsova, N.; Karlinsky, L.; and Kuehne, H. 2022. VL-Taboo: An Analysis of Attribute-based Zero-shot Capabilities of Vision-Language Models. arXiv:2209.06103

    2022 arXiv

  27. [35]

    Wah, C.; Branson, S.; Welinder, P.; Perona, P.; and Belongie, S. 2011. The Caltech-UCSD Birds-200-2011 Dataset. Technical report, California Institute of Technology

    2011

  28. [36]

    Yan, A.; Wang, Y.; Zhong, Y.; Dong, C.; He, Z.; Lu, Y.; Wang, W.; Shang, J.; and McAuley, J. 2023. Learning Concise and Descriptive Attributes for Visual Recognition. arXiv:2308.03685

    2023 arXiv

  29. [37]

    Yang, Y.; Panagopoulou, A.; Zhou, S.; Jin, D.; Callison-Burch, C.; and Yatskar, M. 2023. Language in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classification. arXiv:2211.11158

    2023 arXiv

  30. [38]

    Young, P.; Lai, A.; Hodosh, M.; and Hockenmaier, J. 2014. From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions. Transactions of the Association for Computational Linguistics, 2: 67--78

    2014

  31. [39]

    Zang, Y.; Yun, T.; Tan, H.; Bui, T.; and Sun, C. 2024. Pre-trained Vision-Language Models Learn Discoverable Visual Concepts. arXiv:2404.12652

    2024 arXiv

  32. [40]

    Zhang, J.; Huang, J.; Jin, S.; and Lu, S. 2024. Vision-language models for vision tasks: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence

    2024

  33. [41]

    Zhao, T.; Zhang, T.; Zhu, M.; Shen, H.; Lee, K.; Lu, X.; and Yin, J. 2022. An Explainable Toolbox for Evaluating Pre-trained Vision-Language Models. In Che, W.; and Shutova, E., eds., Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing: Syste...

    2022

  34. [42]

    Zhou, B.; Lapedriza, A.; Khosla, A.; Oliva, A.; and Torralba, A. 2017. Places: A 10 million Image Database for Scene Recognition. T-PAMI

    2017

  35. [43]

    C.; and Liu, Z

    Zhou, K.; Yang, J.; Loy, C. C.; and Liu, Z. 2022. Learning to Prompt for Vision-Language Models. International Journal of Computer Vision, 130(9): 2337–2348

    2022

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.