Pith. sign in

REVIEW 4 major objections 3 minor 75 references

Constrained prompt enhancement improves zero-shot vision-language generalization by replacing incomplete text prompts and noisy image crops with semantically selected prompt sets.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Richer text prompts from LLM synonyms and cleaner image regions from activation maps improve zero-shot vision-language classification.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection A plausible and genuinely novel training-free prompt/region enhancement recipe for zero-shot VLMs, but the supplied PDF is corrupted, so the central accuracy claims can't be checked; the persistent-homology selection is the piece to probe. the 4 major comments →

arxiv 2508.17417 v1 pith:A2ZPCNJI submitted 2025-08-24 cs.CV

Constrained Prompt Enhancement for Improving Zero-Shot Generalization of Vision-Language Models

Xiaojie Yin , Qilong Wang , Qinghua Hu This is my paper
classification cs.CV
keywords zero-shot generalizationvision-language modelsprompt ensemblingpersistent homologytest-time adaptationoptimal transportdiscriminative region selectionsemantic alignment
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that zero-shot vision-language models fail mainly because their text prompts are incomplete and their visual prompts are noisy, and that both defects can be fixed from the semantic side without retraining. It proposes constrained prompt enhancement: generate a broad set of synonymous descriptions per class with a large language model, then prune them by semantic ambiguity and persistent homology to keep a comprehensive but compact text set; separately, select discriminative image regions using activation maps to replace random crops. The text set and region set are then matched as sets, either by test-time adaptation or optimal transport. If the method works as claimed, zero-shot classification accuracy on standard benchmarks improves beyond existing prompt-ensembling and region-alignment approaches.

Core claim

The central claim is that visual-textual alignment in vision-language models improves when both sides are treated as sets selected by semantic constraints. TGSSG builds a synonymous semantic set for each class with a large language model, then uses semantic ambiguity entropy and persistent homology to choose the most informative descriptions, producing comprehensive textual prompts. CADRS uses activation maps from a pre-trained vision encoder to pick compact discriminative regions, filtering out the noise that random cropping introduces. The final set-to-set matching, via test-time adaptation or optimal transport, aligns these prompt sets and improves zero-shot generalization.

What carries the argument

The two named mechanisms are TGSSG (Topology-Guided Synonymous Semantic Generation) and CADRS (Category-Agnostic Discriminative Region Selection). TGSSG generates synonymous descriptions per class, scores them by semantic ambiguity entropy, and applies persistent homology—a topological way of tracking when clusters of meanings appear and disappear—to select a comprehensive yet compact text set. CADRS derives activation maps from a frozen vision encoder and selects the discriminative regions, producing compact visual prompts. These sets are aligned with set-to-set matching, either by test-time adaptation or by optimal transport.

Load-bearing premise

The method assumes the LLM-generated set of synonyms for each class covers every important meaning of the class name, and that the topological pruning never drops a meaning that matters for the downstream task.

What would settle it

Run a benchmark with class names that have multiple distinct senses (for instance, 'seal', 'crane', 'bank') and compare CPE against a variant where a human manually adds the missing sense; if accuracy improves, the TGSSG set was incomplete. Also compare activation-map region selection against random crops on a dataset with small, off-center objects; if accuracy does not drop when regions are chosen randomly, the noise-filtering claim is not load-bearing.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Zero-shot classification accuracy on standard benchmarks should rise without any model fine-tuning, because both text and image side noise are reduced before matching.
  • The method should beat simple prompt ensembling because it targets missing class senses rather than just adding more descriptions.
  • Activation-map region selection should make visual prompts more robust than random cropping, particularly on datasets where object location and size vary.
  • Set-to-set matching with optimal transport provides a principled way to align text and image sets even when the two sets have different sizes.
  • The same two-stage design could be applied to other vision-language tasks that rely on matching a query to multiple candidate representations.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A natural extension is to apply the text-side pipeline alone to any vision-language model and the region-selection side alone to any cropping-based method; the paper's reported gains may decompose into two independent contributions.
  • Persistent homology may be a sufficient but not necessary selection tool; a cheaper clustering-based pruning might reproduce the same text sets, which would suggest the topological step is a means rather than the essential mechanism.
  • The approach could transfer to image-text retrieval or open-vocabulary detection, where aligning multiple text descriptions with multiple image regions is also the core operation.
  • A stress test that would expose the method's limits is a benchmark where class names have several distinct senses—for example 'seal' or 'crane'—because the assumption that the LLM-generated synonym set covers all relevant meanings is exactly what TGSSG relies on.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 3 minor

Summary. The paper proposes Constrained Prompt Enhancement (CPE), a method intended to improve zero-shot generalization of vision-language models. It consists of two components: TGSSG, which uses LLM-generated synonymous text prompts per category filtered by semantic ambiguity entropy and persistent homology, and CADRS, which selects discriminative image regions using activation maps. A set-to-set matching strategy through test-time adaptation and optimal transport is introduced. The abstract claims improved visual-textual alignment and zero-shot generalization. However, the supplied full text is severely corrupted and includes a header for an unrelated arXiv paper; no experimental results, benchmark tables, ablations, error bars, or implementation details are present. The central empirical claim is therefore not verifiable from the manuscript as provided.

Significance. If substantiated, the method addresses real limitations in VLM prompting: hand-crafted prompts can be semantically incomplete, and random crops introduce visual noise. The use of persistent homology for semantic text-prompt filtering is a novel angle, and the combination of LLM-generated synonyms with activation-map region selection is a reasonable pipeline. However, the manuscript supplies no evidence: there are no benchmarks, baselines, ablations, or error bars, and no code or data release is indicated. The significance of the contribution is entirely hypothetical at this stage. The paper's potential is visible only in the abstract, not in any demonstrable result.

major comments (4)
  1. [Overall/Abstract] The central claim — that CPE 'improves zero-shot generalization of VLMs' — is unsupported. The supplied manuscript contains no experimental results: no benchmark tables, no comparisons to existing prompt-based methods, no ablations, no error bars, and no hyperparameter settings. Since this is an empirical methods paper, the absence of validation is a load-bearing failure. Without experiments, the claimed improvement cannot be assessed.
  2. [Full Text] The body of the manuscript is corrupted (mojibake) and carries the header 'arXiv:2508.17418v2 [physics.chem-ph] 8 Jan 2026', which is inconsistent with the claimed paper identity. This prevents verification of any derivations, algorithm descriptions, figures, or references. It also raises a submission-integrity concern: if this file is the manuscript under review, the submission is not in a reviewable state.
  3. [TGSSG] The abstract asserts that TGSSG constructs 'comprehensive textual prompts' based on semantic ambiguity entropy and persistent homology. No theoretical or empirical support links persistent-homology persistence scale to semantic completeness. A rare but discriminative sense of a polysemous class (e.g., 'crane' as a machine vs. a bird) may not form a persistent cluster at the chosen threshold and could be pruned as noise. An ablation comparing PH-based selection with simpler filtering (e.g., clustering, entropy-only, or random sampling) is necessary to justify this design choice and its effect on zero-shot accuracy.
  4. [CADRS / set-to-set matching] Key implementation details are missing. The activation-map region selection threshold, the number of LLM-generated synonyms per category, the entropy and persistence thresholds, and the test-time adaptation and optimal-transport hyperparameters are not specified anywhere in the supplied text. Even if experiments existed, the method would not be reproducible without these details.
minor comments (3)
  1. [Abstract] The phrase 'and so improve zero-shot generalization' is grammatically awkward; consider 'thereby improving zero-shot generalization of VLMs.'
  2. [Full Text] The mathematical notation and equations are largely unreadable due to encoding corruption. The authors should ensure that the PDF is rendered with proper character encoding before resubmission.
  3. [Full Text] The manuscript lacks a limitations section and a broader-impact statement, which are expected in a complete submission. More importantly, no statement of data/availability or reproducibility plan is provided.

Circularity Check

0 steps flagged

No circularity found in the derivable content; the paper's components are generated externally and evaluated against benchmarks.

full rationale

The only usable in-scope content is the abstract; the supplied full text is corrupted (non-ASCII garbling and an arXiv header for 2508.17418v2 [physics.chem-ph]), so no equations, experiments, or ablations are available to inspect. From the abstract, the derivation chain is: TGSSG generates a synonymous semantic set for each category via large language models and then constructs comprehensive textual prompts using semantic ambiguity entropy and persistent homology; CADRS uses activation maps from a pre-trained vision model to select compact visual prompts; and set-to-set matching via test-time adaptation and optimal transport aligns the two prompt sets to improve zero-shot generalization. None of these steps defines its output in terms of the final zero-shot accuracy, no fitted parameter is renamed as a prediction, no self-citation or imported uniqueness theorem is load-bearing, and no known result is merely renamed. The skeptical concern that persistent-homology selection may prune rare class senses is a robustness/correctness risk, not circularity, because the method does not assume by construction that the selected prompt set preserves semantic completeness. Hidden test-set tuning would be a validity concern, but there is no quoted evidence of it. Under the hard rule requiring a quoted equation or explicit reduction, no circular step can be identified, so score 0 is appropriate.

Axiom & Free-Parameter Ledger

5 free parameters · 3 axioms · 0 invented entities

Abstract-only review: the supplied full text is corrupted, so the ledger entries are inferred from the component names in the abstract. No numerical values or detailed assumptions could be audited. The method introduces no new physical or conceptual entities; it combines existing tools.

free parameters (5)
  • Number of LLM-generated synonymous prompts per category
    TGSSG must decide how many synonyms to generate and keep per class; this count trades comprehensiveness against noise and is not stated in the abstract.
  • Semantic ambiguity entropy threshold
    The entropy measure filters synonyms; the cutoff is a free choice not specified in the abstract.
  • Persistent homology persistence threshold
    Persistent homology must select a subset of prompts based on topological persistence; the cutoff controls prompt set size.
  • Activation-map region selection threshold
    CADRS keeps regions whose activation exceeds some threshold; the threshold or number of regions is an unstated design choice.
  • Test-time adaptation hyperparameters
    The TTA matching strategy needs learning rate, update steps, and feature weights; none are given in the abstract.
axioms (3)
  • domain assumption LLM-generated synonyms enumerate the relevant semantic senses of each class name
    TGSSG depends on the completeness and correctness of LLM outputs; if the LLM omits a class sense, the text prompt set is incomplete.
  • domain assumption Persistent homology on semantic structure identifies genuinely informative synonyms
    The paper uses persistence analysis to select comprehensive prompts; this assumes topological persistence tracks semantic usefulness.
  • domain assumption Activation maps from a pre-trained vision model highlight class-discriminative regions without task-specific training
    CADRS relies on these maps to crop visual prompts; if the maps are noisy or class-agnostic, the compact visual prompts lose discriminative information.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Constrained Prompt Enhancement for Improving Zero-Shot Generalization of Vision-Language Models." pith.science (2026). https://pith.science/paper/A2ZPCNJI

@misc{pith2026250817417,
  author       = {Pith},
  title        = {Pith review of: Constrained Prompt Enhancement for Improving Zero-Shot Generalization of Vision-Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/A2ZPCNJI}},
  note         = {Machine review of arXiv:2508.17417}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Vision-language models (VLMs) pre-trained on web-scale data exhibit promising zero-shot generalization but often suffer from semantic misalignment due to domain gaps between pre-training and downstream tasks. Existing approaches primarily focus on text prompting with class-specific descriptions and visual-text adaptation via aligning cropped image regions with textual descriptions. However, they still face the issues of incomplete textual prompts and noisy visual prompts. In this paper, we propose a novel constrained prompt enhancement (CPE) method to improve visual-textual alignment by constructing comprehensive textual prompts and compact visual prompts from the semantic perspective. Specifically, our approach consists of two key components: Topology-Guided Synonymous Semantic Generation (TGSSG) and Category-Agnostic Discriminative Region Selection (CADRS). Textually, to address the issue of incomplete semantic expression in textual prompts, our TGSSG first generates synonymous semantic set for each category via large language models, and constructs comprehensive textual prompts based on semantic ambiguity entropy and persistent homology analysis. Visually, to mitigate the irrelevant visual noise introduced by random cropping, our CADRS identifies discriminative regions with activation maps outputted by a pre-trained vision model, effectively filtering out noisy regions and generating compact visual prompts. Given the comprehensive set of textual prompts and compact set of visual prompts, we introduce two set-to-set matching strategies based on test-time adaptation (TTA) and optimal transport (OT) to achieve effective visual-textual alignment, and so improve zero-shot generalization of VLMs.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

75 extracted references · 54 canonical work pages · 2 internal anchors

  1. [1]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, pages 8748--8763. PMLR, 2021

    2021

  2. [2]

    Laion-400m: Open dataset of clip-filtered 400 million image-text pairs

    Christoph Schuhmann, Richard Vencu, Romain Beaumont, Robert Kaczmarczyk, Clayton Mullis, Aarush Katta, Theo Coombes, Jenia Jitsev, and Aran Komatsuzaki. Laion-400m: Open dataset of clip-filtered 400 million image-text pairs. arXiv preprint arXiv:2111.02114, 2021

    Pith/arXiv arXiv 2021

  3. [3]

    Laion-5b: An open large-scale dataset for training next generation image-text models

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion-5b: An open large-scale dataset for training next generation image-text models. NeurIPS, 35: 0 25278--25294, 2022

    work page 2022

  4. [4]

    Scaling up visual and vision-language representation learning with noisy text supervision

    Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representation learning with noisy text supervision. In International Conference on Machine Learning, pages 4904--4916. PMLR, 2021

    work page 2021

  5. [5]

    Lit: Zero-shot transfer with locked-image text tuning

    Xiaohua Zhai, Xiao Wang, Basil Mustafa, Andreas Steiner, Daniel Keysers, Alexander Kolesnikov, and Lucas Beyer. Lit: Zero-shot transfer with locked-image text tuning. In CVPR, pages 18123--18133, 2022

    work page 2022

  6. [6]

    DALL-E for Detection: Language-driven Compositional Image Synthesis for Object Detection

    Yunhao Ge, Jiashu Xu, Brian Nlong Zhao, Neel Joshi, Laurent Itti, and Vibhav Vineet. Dall-e for detection: Language-driven compositional image synthesis for object detection. arXiv preprint arXiv:2206.09592, 2022

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  7. [7]

    Zero-shot text-to-image generation

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International Conference on Machine Learning, pages 8821--8831. PMLR, 2021

    2021

  8. [8]

    A survey on bias and fairness in machine learning

    Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. A survey on bias and fairness in machine learning. ACM computing surveys (CSUR), 54 0 (6): 0 1--35, 2021

    work page 2021

  9. [9]

    Task bias in vision-language models

    Sachit Menon, Ishaan Preetam Chandratreya, and Carl Vondrick. Task bias in vision-language models. arXiv preprint arXiv:2212.04412, 2022

    Pith/arXiv arXiv 2022

  10. [10]

    Evaluating clip: towards characterization of broader capabilities and downstream implications

    Sandhini Agarwal, Gretchen Krueger, Jack Clark, Alec Radford, Jong Wook Kim, and Miles Brundage. Evaluating clip: towards characterization of broader capabilities and downstream implications. arXiv preprint arXiv:2108.02818, 2021

    Pith/arXiv arXiv 2021

  11. [11]

    Visual classification via description from large language models

    Sachit Menon and Carl Vondrick. Visual classification via description from large language models. ICLR, 2022

    work page 2022

  12. [12]

    Just shift it: Test-time prototype shifting for zero-shot generalization with vision-language models

    Elaine Sui, Xiaohan Wang, and Serena Yeung-Levy. Just shift it: Test-time prototype shifting for zero-shot generalization with vision-language models. arXiv preprint arXiv:2403.12952, 2024

    Pith/arXiv arXiv 2024

  13. [13]

    Awt: Transferring vision-language models via augmentation, weighting, and transportation

    Yuhan Zhu, Yuyang Ji, Zhiyu Zhao, Gangshan Wu, and Limin Wang. Awt: Transferring vision-language models via augmentation, weighting, and transportation. NeurIPS, 2024

    work page 2024

  14. [14]

    The neglected tails in vision-language models

    Shubham Parashar, Zhiqiu Lin, Tian Liu, Xiangjue Dong, Yanan Li, Deva Ramanan, James Caverlee, and Shu Kong. The neglected tails in vision-language models. In CVPR, pages 12988--12997, 2024

    work page 2024

  15. [15]

    Visual-Text Cross Alignment: Refining the Similarity Score in Vision-Language Models

    Jinhao Li, Haopeng Li, Sarah Erfani, Lei Feng, James Bailey, and Feng Liu. Visual-text cross alignment: Refining the similarity score in vision-language models. arXiv preprint arXiv:2406.02915, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  16. [16]

    The claude 3 model family: Opus, sonnet, haiku, 2024

    Anthropic. The claude 3 model family: Opus, sonnet, haiku, 2024

    2024

  17. [17]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    Pith/arXiv arXiv 2023

  18. [18]

    Emerging properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv \'e J \'e gou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In ICCV, pages 9650--9660, 2021

    2021

  19. [19]

    Automated flower classification over a large number of classes

    Maria-Elena Nilsback and Andrew Zisserman. Automated flower classification over a large number of classes. In Indian Conference on Computer Vision, Graphics and Image Processing, pages 722--729, 2008

    work page 2008

  20. [20]

    Describing textures in the wild

    Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. Describing textures in the wild. In CVPR, pages 3606--3613, 2014

    work page 2014

  21. [21]

    Cats and dogs

    Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and CV Jawahar. Cats and dogs. In CVPR, pages 3498--3505, 2012

    work page 2012

  22. [22]

    3d object representations for fine-grained categorization

    Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine-grained categorization. In ICCV, pages 554--561, 2013

    work page 2013

  23. [23]

    Ucf101: A dataset of 101 human actions classes from videos in the wild

    K Soomro. Ucf101: A dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv:1212.0402, 2012

    Pith/arXiv arXiv 2012

  24. [24]

    Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories

    Li Fei-Fei, Rob Fergus, and Pietro Perona. Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. In CVPR Workshop, pages 178--178, 2004

    work page 2004

  25. [25]

    Food-101--mining discriminative components with random forests

    Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. Food-101--mining discriminative components with random forests. In ECCV, pages 446--461, 2014

    work page 2014

  26. [26]

    Sun database: Large-scale scene recognition from abbey to zoo

    Jianxiong Xiao, James Hays, Krista A Ehinger, Aude Oliva, and Antonio Torralba. Sun database: Large-scale scene recognition from abbey to zoo. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 3485--3492. IEEE, 2010

    work page 2010

  27. [27]

    Fine-grained visual classification of aircraft

    Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew Blaschko, and Andrea Vedaldi. Fine-grained visual classification of aircraft. arXiv preprint arXiv:1306.5151, 2013

    Pith/arXiv arXiv 2013

  28. [28]

    Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification

    Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12 0 (7): 0 2217--2226, 2019

    2019

  29. [29]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, pages 248--255. Ieee, 2009

    work page 2009

  30. [30]

    Natural adversarial examples

    Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adversarial examples. In CVPR, pages 15262--15271, 2021 a

    work page 2021

  31. [31]

    Do imagenet classifiers generalize to imagenet? In International Conference on Machine Learning, pages 5389--5400

    Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do imagenet classifiers generalize to imagenet? In International Conference on Machine Learning, pages 5389--5400. PMLR, 2019

    work page 2019

  32. [32]

    The many faces of robustness: A critical analysis of out-of-distribution generalization

    Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, et al. The many faces of robustness: A critical analysis of out-of-distribution generalization. In ICCV, pages 8340--8349, 2021 b

    work page 2021

  33. [33]

    Learning robust global representations by penalizing local predictive power

    Haohan Wang, Songwei Ge, Zachary Lipton, and Eric P Xing. Learning robust global representations by penalizing local predictive power. NeurIPS, 32, 2019

    work page 2019

  34. [34]

    Hmdb: a large video database for human motion recognition

    Hildegard Kuehne, Hueihan Jhuang, Est \' baliz Garrote, Tomaso Poggio, and Thomas Serre. Hmdb: a large video database for human motion recognition. In ICCV, pages 2556--2563. IEEE, 2011

    work page 2011

  35. [35]

    A short note about kinetics-600

    Joao Carreira, Eric Noland, Andras Banki-Horvath, Chloe Hillier, and Andrew Zisserman. A short note about kinetics-600. arXiv preprint arXiv:1808.01340, 2018

    Pith/arXiv arXiv 2018

  36. [36]

    Sigmoid loss for language image pre-training

    Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In ICCV, pages 11975--11986, 2023

    work page 2023

  37. [37]

    Eva-clip: Improved training techniques for clip at scale

    Quan Sun, Yuxin Fang, Ledell Wu, Xinlong Wang, and Yue Cao. Eva-clip: Improved training techniques for clip at scale. arXiv preprint arXiv:2303.15389, 2023

    Pith/arXiv arXiv 2023

  38. [38]

    Language-driven semantic segmentation

    Boyi Li, Kilian Q Weinberger, Serge Belongie, Vladlen Koltun, and Ren \'e Ranftl. Language-driven semantic segmentation. arXiv preprint arXiv:2201.03546, 2022

    Pith/arXiv arXiv 2022

  39. [39]

    Learning to prompt for vision-language models

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision-language models. IJCV, 130 0 (9): 0 2337--2348, 2022 a

    work page 2022

  40. [40]

    Conditional prompt learning for vision-language models

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Conditional prompt learning for vision-language models. In CVPR, pages 16816--16825, 2022 b

    work page 2022

  41. [41]

    Towards open-vocabulary detection using uncurated images

    C Feng, Y Zhong, Z Jie, X Chu, H Ren, X Wei, W Xie, and L Promptdet Ma. Towards open-vocabulary detection using uncurated images. In ECCV, pages 23--27, 2022

    work page 2022

  42. [42]

    Zero-shot detection via vision and language knowledge distillation

    Xiuye Gu, Tsung-Yi Lin, Weicheng Kuo, and Yin Cui. Zero-shot detection via vision and language knowledge distillation. arXiv preprint arXiv:2104.13921, 2 0 (3): 0 4, 2021

    Pith/arXiv arXiv 2021

  43. [43]

    Learning to prompt for open-vocabulary object detection with vision-language model

    Yu Du, Fangyun Wei, Zihe Zhang, Miaojing Shi, Yue Gao, and Guoqi Li. Learning to prompt for open-vocabulary object detection with vision-language model. In CVPR, pages 14084--14093, 2022

    work page 2022

  44. [44]

    Actionclip: A new paradigm for video action recognition

    Mengmeng Wang, Jiazheng Xing, and Yong Liu. Actionclip: A new paradigm for video action recognition. arXiv preprint arXiv:2109.08472, 2021

    Pith/arXiv arXiv 2021

  45. [45]

    Open-vclip: Transforming clip to an open-vocabulary video model via interpolated weight optimization

    Zejia Weng, Xitong Yang, Ang Li, Zuxuan Wu, and Yu-Gang Jiang. Open-vclip: Transforming clip to an open-vocabulary video model via interpolated weight optimization. In International Conference on Machine Learning, pages 36978--36989, 2023

    work page 2023

  46. [46]

    Froster: Frozen clip is a strong teacher for open-vocabulary action recognition

    Xiaohu Huang, Hao Zhou, Kun Yao, and Kai Han. Froster: Frozen clip is a strong teacher for open-vocabulary action recognition. arXiv preprint arXiv:2402.03241, 2024

    Pith/arXiv arXiv 2024

  47. [47]

    What does a platypus look like? generating customized prompts for zero-shot image classification

    Sarah Pratt, Ian Covert, Rosanne Liu, and Ali Farhadi. What does a platypus look like? generating customized prompts for zero-shot image classification. In ICCV, pages 15691--15701, 2023

    work page 2023

  48. [48]

    Language models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. NeurIPS, 33: 0 1877--1901, 2020

    work page 1901

  49. [49]

    Waffling around for performance: Visual classification with random words and broad concepts

    Karsten Roth, Jae Myung Kim, A Koepke, Oriol Vinyals, Cordelia Schmid, and Zeynep Akata. Waffling around for performance: Visual classification with random words and broad concepts. In ICCV, pages 15746--15757, 2023

    work page 2023

  50. [50]

    Meta-prompting for automating zero-shot visual recognition with llms

    M Jehanzeb Mirza, Leonid Karlinsky, Wei Lin, Sivan Doveh, Jakub Micorek, Mateusz Kozinski, Hilde Kuhene, and Horst Possegger. Meta-prompting for automating zero-shot visual recognition with llms. In ECCV, pages 1--30, 2024

    work page 2024

  51. [51]

    Test-time prompt tuning for zero-shot generalization in vision-language models

    Manli Shu, Weili Nie, De-An Huang, Zhiding Yu, Tom Goldstein, Anima Anandkumar, and Chaowei Xiao. Test-time prompt tuning for zero-shot generalization in vision-language models. NeurIPS, 35: 0 14274--14289, 2022

    work page 2022

  52. [52]

    Diverse data augmentation with diffusions for effective test-time prompt tuning

    Chun-Mei Feng, Kai Yu, Yong Liu, Salman Khan, and Wangmeng Zuo. Diverse data augmentation with diffusions for effective test-time prompt tuning. In ICCV, pages 2704--2714, 2023

    work page 2023

  53. [53]

    Align your prompts: Test-time prompting with distribution alignment for zero-shot generalization

    Jameel Abdul Samadh, Mohammad Hanan Gani, Noor Hussein, Muhammad Uzair Khattak, Muhammad Muzammal Naseer, Fahad Shahbaz Khan, and Salman H Khan. Align your prompts: Test-time prompting with distribution alignment for zero-shot generalization. NeurIPS, 36, 2024

    work page 2024

  54. [54]

    Test-time low rank adaptation via confidence maximization for zero-shot generalization of vision-language models

    Raza Imam, Hanan Gani, Muhammad Huzaifa, and Karthik Nandakumar. Test-time low rank adaptation via confidence maximization for zero-shot generalization of vision-language models. In 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 5449--5459. IEEE, 2025

    work page 2025

  55. [55]

    Siren's song in the ai ocean: a survey on hallucination in large language models

    Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, et al. Siren's song in the ai ocean: a survey on hallucination in large language models. arXiv preprint arXiv:2309.01219, 2023

    Pith/arXiv arXiv 2023

  56. [56]

    A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions

    Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, et al. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. arXiv preprint arXiv:2311.05232, 2023

    Pith/arXiv arXiv 2023

  57. [57]

    Computational homology

    K Mischaikow, T Kaczynski, and M Mrozek. Computational homology. Applied Mathematical Sciences, 157, 2004

    work page 2004

  58. [58]

    Persistent homology: An introduction and a new text representation for natural language processing

    Xiaojin Zhu. Persistent homology: An introduction and a new text representation for natural language processing. In IJCAI, pages 1953--1959, 2013

    work page 1953

  59. [59]

    Statistics notes: measurement error

    J Martin Bland and Douglas G Altman. Statistics notes: measurement error. Bmj, 312 0 (7047): 0 1654, 1996

    work page 1996

  60. [60]

    M \'e moire sur la th \'e orie des d \'e blais et des remblais

    Gaspard Monge. M \'e moire sur la th \'e orie des d \'e blais et des remblais. Mem. Math. Phys. Acad. Royale Sci., pages 666--704, 1781

    work page

  61. [61]

    On the translocation of masses

    Leonid V Kantorovich. On the translocation of masses. Journal of mathematical sciences, 133 0 (4), 2006

    work page 2006

  62. [62]

    Sinkhorn distances: Lightspeed computation of optimal transport

    Marco Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport. NeurIPS, 26, 2013

    work page 2013

  63. [63]

    What does bert look at? an analysis of bert's attention

    Kevin Clark, Urvashi Khandelwal, Omer Levy, and Christopher D Manning. What does bert look at? an analysis of bert's attention. arXiv preprint arXiv:1906.04341, 2019

    Pith/arXiv arXiv 1906

  64. [64]

    Maple: Multi-modal prompt learning

    Muhammad Uzair Khattak, Hanoona Rasheed, Muhammad Maaz, Salman Khan, and Fahad Shahbaz Khan. Maple: Multi-modal prompt learning. In CVPR, pages 19113--19122, 2023

    work page 2023

  65. [65]

    Plot: Prompt learning with optimal transport for vision-language models

    Guangyi Chen, Weiran Yao, Xiangchen Song, Xinyue Li, Yongming Rao, and Kun Zhang. Plot: Prompt learning with optimal transport for vision-language models. In ICLR, 2022 a

    work page 2022

  66. [66]

    Prompt pre-training with twenty-thousand classes for open-vocabulary visual recognition

    Shuhuai Ren, Aston Zhang, Yi Zhu, Shuai Zhang, Shuai Zheng, Mu Li, Alexander J Smola, and Xu Sun. Prompt pre-training with twenty-thousand classes for open-vocabulary visual recognition. NeurIPS, 36: 0 12569--12588, 2023

    work page 2023

  67. [67]

    Progressive visual prompt learning with contrastive feature re-formation

    Chen Xu, Yuhan Zhu, Haocheng Shen, Boheng Chen, Yixuan Liao, Xiaoxin Chen, and Limin Wang. Progressive visual prompt learning with contrastive feature re-formation. IJCV, 133 0 (2): 0 511--526, 2025

    work page 2025

  68. [68]

    Sus-x: Training-free name-only transfer of vision-language models

    Vishaal Udandarao, Ankush Gupta, and Samuel Albanie. Sus-x: Training-free name-only transfer of vision-language models. In ICCV, pages 2725--2736, 2023

    work page 2023

  69. [69]

    Expanding language-image pretrained models for general video recognition

    Bolin Ni, Houwen Peng, Minghao Chen, Songyang Zhang, Gaofeng Meng, Jianlong Fu, Shiming Xiang, and Haibin Ling. Expanding language-image pretrained models for general video recognition. In ECCV, pages 1--18, 2022

    work page 2022

  70. [70]

    Aim: Adapting image models for efficient video action recognition

    Taojiannan Yang, Yi Zhu, Yusheng Xie, Aston Zhang, Chen Chen, and Mu Li. Aim: Adapting image models for efficient video action recognition. In ICLR, 2022

    work page 2022

  71. [71]

    St-adapter: Parameter-efficient image-to-video transfer learning

    Junting Pan, Ziyi Lin, Xiatian Zhu, Jing Shao, and Hongsheng Li. St-adapter: Parameter-efficient image-to-video transfer learning. NeurIPS, 35: 0 26462--26477, 2022

    work page 2022

  72. [72]

    Vita-clip: Video and text adaptive clip via multimodal prompting

    Syed Talal Wasim, Muzammal Naseer, Salman Khan, Fahad Shahbaz Khan, and Mubarak Shah. Vita-clip: Video and text adaptive clip via multimodal prompting. In CVPR, pages 23034--23044, 2023

    work page 2023

  73. [73]

    Fine-tuned clip models are efficient video learners

    Hanoona Rasheed, Muhammad Uzair Khattak, Muhammad Maaz, Salman Khan, and Fahad Shahbaz Khan. Fine-tuned clip models are efficient video learners. In CVPR, pages 6545--6554, 2023

    work page 2023

  74. [74]

    Adaptformer: Adapting vision transformers for scalable visual recognition

    Shoufa Chen, Chongjian Ge, Zhan Tong, Jiangliu Wang, Yibing Song, Jue Wang, and Ping Luo. Adaptformer: Adapting vision transformers for scalable visual recognition. NeurIPS, 35: 0 16664--16678, 2022 b

    work page 2022

  75. [75]

    Reproducible scaling laws for contrastive language-image learning

    Mehdi Cherti, Romain Beaumont, Ross Wightman, Mitchell Wortsman, Gabriel Ilharco, Cade Gordon, Christoph Schuhmann, Ludwig Schmidt, and Jenia Jitsev. Reproducible scaling laws for contrastive language-image learning. In CVPR, pages 2818--2829, 2023

    work page 2023

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.