Pith. sign in

REVIEW 5 major objections 7 minor 51 references

Helping CLIP See Both the Forest and the Trees: A Decomposition and Description Approach

T0 review · 5 major / 7 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims CLIP's global bias blocks fine-grained descriptor use, and that random crops plus Earth Mover's Distance between image regions and LLM-generated descriptors fixes it.

desk verdict Plausible training-free CLIP booster, but the EMD mechanism needs a head-to-head against max-pooling before I'd believe the attribution. read the letter →

arxiv 2507.03458 v1 pith:Z4ZJM6S5 submitted 2025-07-04 cs.CV cs.AI

Leyan Xue , Zongbo Han , Guangyu Wang , Qinghua Hu , Mingyue Cheng , Changqing Zhang This is my paper · ORCID
classification cs.CVcs.AI
keywords CLIPzero-shotclassificationfine-graineddescriptorsrandomcroppingEarthMover'sDistanceoptimaltransportfew-shotlearningtest-timeadaptation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that CLIP's near-sightedness for local visual details, not missing text, is the bottleneck for description-based classification. It finds that CLIP answers label prompts far better than descriptor prompts and treats descriptors as weak context, then proposes D&D: decompose each image into random crops, describe each class with LLM-generated attributes, and match the two sets with Earth Mover's Distance. The authors report consistent gains over CLIP and CALIP in zero-shot classification and over Tip-Adapter and TDA in few-shot and test-time adaptation on 11 datasets. If the claim is right, the practical recipe for prompt engineering is to enrich both sides of the image-text comparison and to let optimal transport do the alignment.

What carries the argument

The central machinery is D&D, a plug-and-play inference procedure that pairs stochastic multi-crop augmentation with optimal transport. Each test image is split into $M$ random crops, each class is represented by $N$ LLM-generated descriptors, and CLIP embeds both sets; the class score is the minimal Earth Mover's Distance between the sets, with uniform marginal constraints $1/M$ and $1/N$ (Eq. 4). The crops do the work of constraining the model's receptive field so local features become global to the model, and EMD provides soft set-to-set alignment instead of averaging. For few-shot and test-time settings the same idea is transferred to cache construction: the crop feature closest to each descriptor is stored per class, and similarity is again computed with EMD.

What would settle it

On a fine-grained dataset such as FGVC, run D&D with crops drawn only from image margins and mostly background, and compare with crops drawn from the object's discriminative region; if background-crop accuracy does not fall well below object-crop accuracy, the claim that cropping constrains the receptive field onto the described local content is not what drives the reported gains.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that CLIP does not actually use localized textual descriptors when matching images; it leans on global category labels and treats descriptors as weak modifiers. That is supported by three observations: descriptor-only prompts drop sharply relative to label-only prompts, hybrid prompts that must jointly identify label and descriptor score near zero, and swapping a descriptor in a prompt moves similarity by only $0.23$ while swapping the label moves it by $5.91$. The positive half of the discovery is that random cropping reverses this: presenting partial image regions recalibrates CLIP's attention so regional visual patterns line up with descriptor text. D&D builds on this with Earth Mover's Distance, finding the cheapest way to transport mass from the set of $M$ random crops to the set of $N$ descriptors for each class.

Load-bearing premise

The load-bearing premise is that uniform random crops from an image reliably contain the local details the class descriptors describe, so that equal-weight EMD matching between crops and descriptors is a valid scoring rule rather than an arbitrary assignment of mass to background.

Editorial extensions

If this is right

  • Prompt enrichment alone is not enough: description-enhanced labels yield near-zero strict accuracy in the paper's motivation experiments, so effective descriptor use requires decomposing the image as well.
  • Random cropping plus EMD is a training-free, plug-and-play upgrade that can be layered onto CLIP, CALIP, Tip-Adapter, and TDA without modifying their weights.
  • Cache-based few-shot learning inherits the gain: Tip+D&D beats Tip-Adapter on all 11 datasets at every shot count from 1 to 16 in the paper's full results.
  • Test-time adaptation inherits the gain as well, with TDA+D&D ahead of TDA on the cross-domain benchmark and large improvements on tasks like Aircraft, DTD, and UCF101.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: if the mechanism is really receptive-field constraint, then crops guided by saliency or segmentation should beat uniform random crops on fine-grained classes, because the same budget of $M$ crops would be spent on discriminative parts.
  • Editorial inference: the same crop-and-match recipe could be tested on CLIP-based retrieval and attribute grounding, where local descriptors matter beyond classification; the paper does not run those experiments.
  • Editorial inference: the uniform marginal in Eq. 4 forces every descriptor to receive fixed weight $1/N$; a descriptor-weighted variant might be more stable on classes whose LLM descriptions are generic or redundant, a direction the paper leaves open.
  • Editorial inference: the largest reported gains cluster on hard fine-grained and texture datasets, suggesting the method's value grows with class similarity; a dedicated stress test on increasingly fine-grained splits would map where the advantage stops.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 7 minor

Summary. The paper proposes D&D, a plug-and-play inference-time method for CLIP classification. It first presents diagnostic experiments (Sec. 3.2) intended to show that CLIP is biased toward global image patterns and underuses localized textual descriptors, then introduces a scoring procedure in which an image is decomposed into random crops, a class is described by LLM-generated descriptors, and the match is computed as an Earth Mover's Distance between the two sets (Eq. 4). The method is evaluated in zero-shot, few-shot (Tip-Adapter), and test-time adaptation (TDA) settings across 11 datasets, with reported improvements over CLIP and CALIP in zero-shot, over Tip-Adapter in few-shot, and over TDA in TTA. Ablations compare against averaging crops with descriptors (CLIP+D+R).

Significance. If the reported gains are robust, the paper offers a simple and potentially useful way to force CLIP to exploit local visual features without retraining, and it contributes a systematic probe of CLIP's local-descriptor perception. The idea of constraining the receptive field via random cropping and scoring with set-level OT is intuitive and easy to reproduce. However, the current evidence is undermined by missing baseline comparisons, an internally inconsistent few-shot table, and the lack of an ablation that isolates the EMD matching mechanism from simple max/avg pooling. The central claim is plausible but not yet convincingly supported.

major comments (5)
  1. [Sec. 3.2, Figure 1, Table 1] The diagnostic experiments overstate the conclusion that CLIP 'cannot process localized visual descriptors.' The descriptor-only prompts (e.g., 'a red beak and striped wings') omit the class label entirely, which is not how descriptor-based classification methods (CuPL, [26]) use descriptors; those methods always combine the label with descriptors. The 'strict criterion' requiring the model to jointly predict both label and descriptor is also not part of CLIP's training objective. These experiments therefore do not establish that CLIP fails to use descriptors in the realistic prompt format, and the motivation for the proposed method rests on a weaker empirical basis than claimed.
  2. [Table 2, Sec. 5.2 (Zero-shot Classification)] The zero-shot comparison omits the main descriptor-based baseline CuPL, even though the paper's own descriptor generation follows [26] and CuPL is cited in the appendix (Table 7). Without CuPL (and ideally PLIP or similar) in Table 2, the comparison only shows gains over CLIP and CALIP, neither of which uses LLM-generated descriptors. The claim that D&D improves over descriptor-enhanced CLIP requires a direct comparison with CuPL on the same 11 datasets.
  3. [Eq. (4), Sec. 4.2, Table 5] The paper does not demonstrate that the uniform-marginal EMD scoring rule in Eq. (4) is the source of the gains. The ablation in Table 5 only compares against CLIP+D+R, which averages crop features; it does not test asymmetric alternatives such as max-pooling over crops per descriptor (score_c = (1/N) ∑_n max_m cos(v_m, d_n)) or max-pooling over the whole set. The inconsistency is highlighted by Sec. 4.2, where the cache construction itself selects the highest-similarity crop for each descriptor. Unless a max-based baseline is shown to be worse, the improvement could come from crops plus descriptors rather than from the OT mechanism.
  4. [Table 6, Sec. 5.2 (Few-shot Learning)] The text in Sec. 5.2 and Appendix A.2 claims 'consistent improvements on all 11 datasets' and 'consistently outperforms Tip across all datasets and shot settings,' but Table 6 contradicts this: for Flowers 4-shot, Tip achieves 83.80 vs. Tip+D&D 83.72, and for Flowers 8-shot, Tip achieves 87.98 vs. Tip+D&D 87.01. The claim of consistent improvement is therefore false as stated, and the few-shot evidence is weaker than advertised. Please correct the overstatement and discuss the variability across seeds, especially given that only the 16-shot case is highlighted in the main text.
  5. [Sec. 5.1 (Implementation details), Eq. (4)] The method has several free hyperparameters—M (number of crops), N (number of descriptors), crop scale range, α and β in Eqs. (7)-(8), and the Sinkhorn ϵ—but no sensitivity analysis is provided. The paper fixes M=9 and N=9 'for computational convenience' and gives no ablation showing how performance changes with these values or with the Sinkhorn approximation. Since the EMD solution and the final fused score depend on these choices, the robustness of the reported results is not established.
minor comments (7)
  1. [Sec. 5.2] The sentence 'maintains stronger generalization on mainstream benchmarks like ImageNet (59.68% vs 61.31%)' is inconsistent with Table 2, which lists CLIP ImageNet accuracy as 58.18 and D&D as 61.31; please correct the numbers.
  2. [Table 1] The similarity-difference values in Table 1 are reported without standard deviations or the number of prompts used; the conclusion that descriptor perturbations have 'relatively minor impact' should be accompanied by variance information.
  3. [Eq. (8)] The final prediction formula in Eq. (8) is ambiguous: it combines α·A_c with a negative EMD term, but the text says it combines the affinity score with the zero-shot textual similarity score. Please clarify how the textual similarity is incorporated and whether both terms are weighted.
  4. [Sec. 5.1] The random-cropping procedure is not fully specified: the text mentions a scale range of 10–75% but does not state how the aspect ratio, per-crop scale, or the number of crops sampled from that range are chosen for each of the M=9 patches.
  5. [Appendix A.1] The descriptor-generation prompt is described only in text; to ensure reproducibility, please provide the exact prompt template and an example of the LLM output, as in Figure 5.
  6. [Throughout] Several tabular results are reported as averages over three seeds, but no standard deviations or confidence intervals are given; adding these would strengthen the empirical claims.
  7. [Sec. 1] There is a grammatical typo in the contributions list: 'a extensive evaluation' should be 'an extensive evaluation.'

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: D&D's gains are evaluated against external baselines, and the EMD scoring rule is not fitted to the target outputs.

full rationale

The paper's derivation chain does not reduce to its own inputs. The motivating claim that CLIP is biased toward global patterns is supported by diagnostic prompt experiments in Sec. 3.2, which are separate from the construction of the final scorer; these diagnostics are not used as the testing criterion for D&D. The descriptor set is generated by an external LLM using a fixed template (Appendix A.1), and the random-crop image set is an inference-time augmentation with fixed hyperparameters (M = 9 crops, N = 9 descriptors), not fitted to dataset labels. The EMD rule in Eq. 4 is stated as an explicit scoring rule with uniform marginals, and no crop, descriptor, or transport plan is selected to match the test labels. Reported improvements over CLIP, CALIP, Tip-Adapter, and TDA are measured on held-out test sets across 11 standard datasets, giving the central claim independent empirical content. The ablation in Table 5 compares D&D against CLIP+D+R, a non-EMD averaging baseline, providing a direct test of the EMD component. The absence of a max-over-crops baseline is an evidence-completeness concern, not circularity: it does not show that the EMD result reduces to a fit or to a self-citation. There are no load-bearing self-citations or imported uniqueness theorems in the derivation. The paper is therefore not circular in any of the specified senses.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The method rests on several hyperparameters (M, N, crop scale, alpha, beta, Sinkhorn epsilon) that are chosen by hand and not reported in full, plus strong assumptions about descriptor quality and uniform matching. No new physical or model entities are introduced.

free parameters (5)
  • M, number of random crops per image = 9
    Chosen for computational convenience; not tuned per dataset; influences EMD transport size.
  • N, number of LLM-generated descriptors per class = 9
    Prompt to LLM requests 9 visual features; not justified beyond balancing M and N.
  • Crop scale range = 10% to 75% of image
    Range for random cropping; not reported how it was chosen.
  • alpha and beta in Eq. (7)-(8) = not reported
    Affinity and fusion weights adopted from Tip-Adapter/TDA but values not stated in the paper.
  • Sinkhorn epsilon for approximate EMD = not reported
    Not given, hindering exact replication.
assumptions (4)
  • domain assumption CLIP's text and image encoders produce comparable embeddings, so cosine similarity and EMD between them are meaningful.
    Used throughout Section 4; if embeddings are not aligned in a shared metric space, EMD costs based on cosine distance lose meaning.
  • domain assumption LLM-generated descriptors are accurate, class-discriminative, and visually grounded.
    Descriptors are built following Menon and Vondrick's prompt in Appendix A.1, with no verification of descriptor quality per dataset.
  • ad hoc to paper Uniform marginals on crops and descriptors are an appropriate prior for the matching problem.
    Equation (4) fixes each patch's supply at 1/M and each descriptor's demand at 1/N; no evidence that equal weighting is optimal.
  • standard math Standard mathematical properties of optimal transport and the Sinkhorn approximation hold.
    Invoked in Section 4.1 for EMD computation; standard background.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Helping CLIP See Both the Forest and the Trees: A Decomposition and Description Approach." pith.science (2026). https://pith.science/paper/Z4ZJM6S5

@misc{pith2026250703458,
  author       = {Pith},
  title        = {Pith review of: Helping CLIP See Both the Forest and the Trees: A Decomposition and Description Approach},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Z4ZJM6S5}},
  note         = {Machine review of arXiv:2507.03458}
}
read the original abstract

Vision-Language Models (VLMs) like CLIP achieve cross-modal semantic alignment through contrastive learning, exhibiting robust zero-shot generalization. Traditional prompt engineering, however, predominantly relies on coarse-grained category labels, neglecting fine-grained local semantics. Existing approaches assume that VLMs inherently recognize localized visual details and attempt to enhance classification by augmenting text prompts with attribute descriptors generated by large language models. However, our systematic experiments reveal critical limitations: CLIP's strong bias toward global image patterns hinders its ability to process localized visual descriptors. To address this fundamental constraint, we propose a simple, effective, and plug-and-play solution that enables CLIP to ``See Both the Forest and the Trees." Specifically, we employ stochastic multi-crop augmentation to activate CLIP's latent capacity for localized feature analysis. By cropping only partial regions, the approach effectively constrains the model's receptive field and recalibrates its attention mechanism, thereby mitigating its inherent bias. We evaluate the proposed method under zero-shot, few-shot, and test-time adaptation settings, and extensive experiments demonstrate that D&D achieves promising performance.

Figures

Figures reproduced from arXiv: 2507.03458 by the authors.

Figure 1
Figure 1. Motivation of the proposed method. (a) We provide a statistical analysis of the accuracy for three types of prompts. (b) We [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of our proposed method under zero-shot scenario. (a) Original CLIP utilizes cosine similarity between the image [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Overview of our proposed method under few-shot and test-time adaption scenarios. (a) Tip-Adapter and TDA constructs a [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Detailed workflow to build descriptors [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]
Figure 5
Figure 5. Figure 5: Examples of generated descriptors. A. Appendix A.1. Building Descriptors We follow the method proposed in [26] to build descriptors and the detailed workflow is illustrated in the [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

51 extracted references · 36 canonical work pages

  1. [26]

    Visual classification via description from large language models

    Sachit Menon and Carl V ondrick. Visual classification via description from large language models. In The Eleventh International Conference on Learning Representations. 1, 3, 12

    work page

  2. [1]

    Align your prompts: Test-time prompting with distribution align- ment for zero-shot generalization

    Jameel Abdul Samadh, Mohammad Hanan Gani, Noor Hus- sein, Muhammad Uzair Khattak, Muhammad Muzammal Naseer, Fahad Shahbaz Khan, and Salman H Khan. Align your prompts: Test-time prompting with distribution align- ment for zero-shot generalization. Advances in Neural Infor- mation Processing Systems, 36:80396–80413, 2023. 3, 8

    work page 2023

  3. [2]

    Wasserstein generative adversarial networks

    Martin Arjovsky, Soumith Chintala, and L ´eon Bottou. Wasserstein generative adversarial networks. In Interna- tional conference on machine learning , pages 214–223. PMLR, 2017. 3

    work page 2017

  4. [3]

    Food-101–mining discriminative components with random forests

    Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. Food-101–mining discriminative components with random forests. In Computer vision–ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, pro- ceedings, part VI 13, pages 446–461. Springer, 2014. 7

    2014

  5. [4]

    Lan- guage models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Sub- biah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakan- tan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Lan- guage models are few-shot learners. Advances in neural in- formation processing systems, 33:1877–1901, 2020. 1, 3

    work page 1901

  6. [5]

    Optimal transport for single- cell and spatial omics

    Charlotte Bunne, Geoffrey Schiebinger, Andreas Krause, Aviv Regev, and Marco Cuturi. Optimal transport for single- cell and spatial omics. Nature Reviews Methods Primers, 4 (1):58, 2024. 3

    work page 2024

  7. [6]

    Scal- able wasserstein gradient flow for generative modeling through unbalanced optimal transport

    Jaemoo Choi, Jaewoong Choi, and Myungjoo Kang. Scal- able wasserstein gradient flow for generative modeling through unbalanced optimal transport. In Proceedings of the 41st International Conference on Machine Learning , pages 8629–8650, 2024. 3

    work page 2024

  8. [7]

    Describing textures in the wild

    Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. Describing textures in the wild. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3606–3613, 2014. 7

    work page 2014

Show all 51 references
  1. [8]

    Optimal transport for domain adaptation

    Nicolas Courty, R ´emi Flamary, Devis Tuia, and Alain Rako- tomamonjy. Optimal transport for domain adaptation. IEEE transactions on pattern analysis and machine intelligence , 39(9):1853–1865, 2016. 3

    2016

  2. [9]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009. 7

    2009

  3. [10]

    De- coupling zero-shot semantic segmentation

    Jian Ding, Nan Xue, Gui-Song Xia, and Dengxin Dai. De- coupling zero-shot semantic segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11583–11592, 2022. 3

    2022

  4. [11]

    Unbalanced minibatch optimal transport; applica- tions to domain adaptation

    Kilian Fatras, Thibault S ´ejourn´e, R´emi Flamary, and Nicolas Courty. Unbalanced minibatch optimal transport; applica- tions to domain adaptation. In International Conference on Machine Learning, pages 3186–3197. PMLR, 2021. 3

    2021

  5. [12]

    Learning gener- ative visual models from few training examples: An incre- mental bayesian approach tested on 101 object categories

    Li Fei-Fei, Rob Fergus, and Pietro Perona. Learning gener- ative visual models from few training examples: An incre- mental bayesian approach tested on 101 object categories. In 2004 conference on computer vision and pattern recognition workshop, pages 178–178. IEEE, 2004. 7

    2004

  6. [13]

    Diverse data augmentation with diffusions for effective test-time prompt tuning

    Chun-Mei Feng, Kai Yu, Yong Liu, Salman Khan, and Wangmeng Zuo. Diverse data augmentation with diffusions for effective test-time prompt tuning. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 2704–2714, 2023. 8, 13

    2023

  7. [14]

    Clip-adapter: Better vision-language models with feature adapters

    Peng Gao, Shijie Geng, Renrui Zhang, Teli Ma, Rongyao Fang, Yongfeng Zhang, Hongsheng Li, and Yu Qiao. Clip-adapter: Better vision-language models with feature adapters. International Journal of Computer Vision, 132(2): 581–595, 2024. 3

    2024

  8. [15]

    Open-vocabulary object detection via vision and language knowledge distillation

    Xiuye Gu, Tsung-Yi Lin, Weicheng Kuo, and Yin Cui. Open-vocabulary object detection via vision and language knowledge distillation. In International Conference on Learning Representations. 3

  9. [16]

    Calip: Zero-shot en- hancement of clip with parameter-free attention

    Ziyu Guo, Renrui Zhang, Longtian Qiu, Xianzheng Ma, Xu- peng Miao, Xuming He, and Bin Cui. Calip: Zero-shot en- hancement of clip with parameter-free attention. InProceed- ings of the AAAI Conference on Artificial Intelligence, pages 746–754, 2023. 7, 13

    2023

  10. [17]

    Ow-detr: Open-world detection transformer

    Akshita Gupta, Sanath Narayan, KJ Joseph, Salman Khan, Fahad Shahbaz Khan, and Mubarak Shah. Ow-detr: Open-world detection transformer. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9235–9244, 2022. 3

    2022

  11. [18]

    Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification

    Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12(7):2217–2226, 2019. 7

    2019

  12. [19]

    Scaling up visual and vision-language representa- tion learning with noisy text supervision

    Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representa- tion learning with noisy text supervision. In International conference on machine learning, pages 4904–4916. PMLR,

  13. [20]

    Towards open world object de- tection

    KJ Joseph, Salman Khan, Fahad Shahbaz Khan, and Vi- neeth N Balasubramanian. Towards open world object de- tection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5830–5840,

  14. [21]

    Efficient test-time adaptation of vision-language models

    Adilbek Karmanov, Dayan Guan, Shijian Lu, Abdulmotaleb El Saddik, and Eric Xing. Efficient test-time adaptation of vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 14162–14171, 2024. 3, 6, 7, 8, 13

    2024

  15. [22]

    3d object representations for fine-grained categorization

    Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine-grained categorization. In Proceedings of the IEEE international conference on com- puter vision workshops, pages 554–561, 2013. 7 9

    2013

  16. [23]

    Language-driven semantic seg- mentation

    Boyi Li, Kilian Q Weinberger, Serge Belongie, Vladlen Koltun, and Rene Ranftl. Language-driven semantic seg- mentation. In International Conference on Learning Rep- resentations. 3

  17. [24]

    Fine-grained visual classi- fication of aircraft

    Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew Blaschko, and Andrea Vedaldi. Fine-grained visual classi- fication of aircraft. arXiv preprint arXiv:1306.5151 , 2013. 7

    2013 arXiv

  18. [25]

    Dou- bly right object recognition: A why prompt for visual ratio- nales

    Chengzhi Mao, Revant Teotia, Amrutha Sundar, Sachit Menon, Junfeng Yang, Xin Wang, and Carl V ondrick. Dou- bly right object recognition: A why prompt for visual ratio- nales. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 2722–2732,

  19. [27]

    M ´emoire sur la th ´eorie des d ´eblais et des remblais

    Gaspard Monge. M ´emoire sur la th ´eorie des d ´eblais et des remblais. Mem. Math. Phys. Acad. Royale Sci. , pages 666– 704, 1781. 3

  20. [28]

    Automated flower classification over a large number of classes

    Maria-Elena Nilsback and Andrew Zisserman. Automated flower classification over a large number of classes. In 2008 Sixth Indian conference on computer vision, graphics & im- age processing, pages 722–729. IEEE, 2008. 7

    2008

  21. [29]

    Cats and dogs

    Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and CV Jawahar. Cats and dogs. In 2012 IEEE conference on computer vision and pattern recognition, pages 3498–3505. IEEE, 2012. 7

    2012

  22. [30]

    Computational optimal transport: With applications to data science

    Gabriel Peyr ´e, Marco Cuturi, et al. Computational optimal transport: With applications to data science. Foundations and Trends® in Machine Learning, 11(5-6):355–607, 2019. 3

    2019

  23. [31]

    What does a platypus look like? generating customized prompts for zero-shot image classification

    Sarah Pratt, Ian Covert, Rosanne Liu, and Ali Farhadi. What does a platypus look like? generating customized prompts for zero-shot image classification. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 15691–15701, 2023. 1, 3, 13

    2023

  24. [32]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  25. [33]

    Waffling around for performance: Visual classification with random words and broad concepts

    Karsten Roth, Jae Myung Kim, A Koepke, Oriol Vinyals, Cordelia Schmid, and Zeynep Akata. Waffling around for performance: Visual classification with random words and broad concepts. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pages 15746–15757,

  26. [34]

    The earth mover’s distance as a metric for image retrieval

    Yossi Rubner, Carlo Tomasi, and Leonidas J Guibas. The earth mover’s distance as a metric for image retrieval. In- ternational journal of computer vision, 40:99–121, 2000. 3, 5

    2000

  27. [35]

    Accurate point cloud registration with robust optimal transport

    Zhengyang Shen, Jean Feydy, Peirong Liu, Ariel H Curiale, Ruben San Jose Estepar, Raul San Jose Estepar, and Marc Niethammer. Accurate point cloud registration with robust optimal transport. Advances in Neural Information Process- ing Systems, 34:5373–5389, 2021. 3

    2021

  28. [36]

    Test- time prompt tuning for zero-shot generalization in vision- language models

    Manli Shu, Weili Nie, De-An Huang, Zhiding Yu, Tom Goldstein, Anima Anandkumar, and Chaowei Xiao. Test- time prompt tuning for zero-shot generalization in vision- language models. Advances in Neural Information Process- ing Systems, 35:14274–14289, 2022. 8, 13

    2022

  29. [37]

    Ucf101: A dataset of 101 human actions classes from videos in the wild

    K Soomro. Ucf101: A dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv:1212.0402 ,

    arXiv

  30. [38]

    Aligning individual brains with fused unbalanced gromov wasserstein

    Alexis Thual, Quang Huy Tran, Tatiana Zemskova, Nico- las Courty, R´emi Flamary, Stanislas Dehaene, and Bertrand Thirion. Aligning individual brains with fused unbalanced gromov wasserstein. Advances in neural information pro- cessing systems, 35:21792–21804, 2022. 3

    2022

  31. [39]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth´ee Lacroix, Baptiste Rozi`ere, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023. 1, 3

    2023 arXiv

  32. [40]

    Sus-x: Training-free name-only transfer of vision-language models

    Vishaal Udandarao, Ankush Gupta, and Samuel Albanie. Sus-x: Training-free name-only transfer of vision-language models. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 2725–2736, 2023. 7

    2023

  33. [41]

    Sun database: Large-scale scene recognition from abbey to zoo

    Jianxiong Xiao, James Hays, Krista A Ehinger, Aude Oliva, and Antonio Torralba. Sun database: Large-scale scene recognition from abbey to zoo. In 2010 IEEE computer so- ciety conference on computer vision and pattern recognition, pages 3485–3492. IEEE, 2010. 7

    2010

  34. [42]

    Temporally consistent unbal- anced optimal transport for unsupervised action segmenta- tion

    Ming Xu and Stephen Gould. Temporally consistent unbal- anced optimal transport for unsupervised action segmenta- tion. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 14618–14627,

  35. [43]

    A simple baseline for open- vocabulary semantic segmentation with pre-trained vision- language model

    Mengde Xu, Zheng Zhang, Fangyun Wei, Yutong Lin, Yue Cao, Han Hu, and Xiang Bai. A simple baseline for open- vocabulary semantic segmentation with pre-trained vision- language model. In European Conference on Computer Vi- sion, pages 736–753. Springer, 2022. 3

    2022

  36. [44]

    Reliable weighted optimal transport for unsupervised domain adaptation

    Renjun Xu, Pelen Liu, Liyan Wang, Chao Chen, and Jindong Wang. Reliable weighted optimal transport for unsupervised domain adaptation. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 4394–4403, 2020. 3

    2020

  37. [45]

    When and why vision- language models behave like bags-of-words, and what to do about it? In The Eleventh International Conference on Learning Representations

    Mert Yuksekgonul, Federico Bianchi, Pratyusha Kalluri, Dan Jurafsky, and James Zou. When and why vision- language models behave like bags-of-words, and what to do about it? In The Eleventh International Conference on Learning Representations. 3

  38. [46]

    Deepemd: Few-shot image classification with differentiable earth mover’s distance and structured classifiers

    Chi Zhang, Yujun Cai, Guosheng Lin, and Chunhua Shen. Deepemd: Few-shot image classification with differentiable earth mover’s distance and structured classifiers. InProceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12203–12213, 2020. 3

    2020

  39. [47]

    Tip-adapter: Training-free clip-adapter for better vision- 10 language modeling

    Renrui Zhang, Rongyao Fang, Wei Zhang, Peng Gao, Kunchang Li, Jifeng Dai, Yu Qiao, and Hongsheng Li. Tip-adapter: Training-free clip-adapter for better vision- 10 language modeling. arXiv preprint arXiv:2111.03930, 2021. 3, 6, 7

    2021 arXiv

  40. [48]

    Neural topic model via optimal transport

    He Zhao, Dinh Phung, Viet Huynh, Trung Le, and Wray Buntine. Neural topic model via optimal transport. In In- ternational Conference on Learning Representations. 3

  41. [49]

    Conditional prompt learning for vision-language mod- els

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Conditional prompt learning for vision-language mod- els. In Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition , pages 16816–16825,

  42. [50]

    Learning to prompt for vision-language models

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision-language models. In- ternational Journal of Computer Vision, 130(9):2337–2348,

  43. [2022]

    Detailed workflow to build descriptors

    3 11 Figure 4. Detailed workflow to build descriptors. Figure 5. Examples of generated descriptors. A. Appendix A.1. Building Descriptors We follow the method proposed in [26] to build descriptors and the detailed workflow is illustrated in the Fig. 4. Specif- ically, our prom...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.