Pith. sign in

REVIEW 4 major objections 5 minor 3 cited by

KPL: Training-Free Medical Knowledge Mining of Vision-Language Models

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A training-free procedure, KPL, improves CLIP's zero-shot medical image classification by mining LLM-generated class descriptions and refining pseudo-labels to learn multimodal class proxies.

desk verdict Large transductive gains for CLIP on medical images, but the 'zero-shot' claim is overstated because the test set is used for adaptation; still a useful contribution and worth reviewing. read the letter →

arxiv 2501.11231 v1 pith:OOZBR36I submitted 2025-01-20 cs.CV

Jiaxiang Liu , Tianxiang Hu , Jiawei Du , Ruiyuan Zhang , Joey Tianyi Zhou , Zuozhu Liu This is my paper · ORCID
classification cs.CV
keywords CLIPzero-shotclassificationmedicalimagevision-languagemodelstextproxypseudo-labelrefinementSinkhornknowledge-enhancedbase
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether CLIP's zero-shot classifier is using the model's full capacity, and answers no for medical images. It introduces Knowledge Proxy Learning (KPL), a training-free inference procedure that first replaces each class name with a set of many LLM-generated visual descriptions, keeps only the descriptions most similar to the dataset's average image, and then learns class proxies in the shared vision-text space guided by pseudo-labels refined with a stabilized Sinkhorn-type algorithm. The paper reports that KPL consistently beats existing CLIP-based zero-shot methods, including description-based VCD and CMD and proxy-learning InMaP, on five medical datasets and in most configurations on four natural-image datasets, with large jumps such as malaria-cell accuracy rising from about 50% to above 80%. A sympathetic reader would care because the method shows that substantial accuracy can be extracted from a frozen CLIP model without additional training, which matters in medical settings where annotated data are scarce.

What carries the argument

The load-bearing object is the multimodal class proxy $w_j^{KPL}$, a learned class embedding in CLIP's shared vision-text space that serves as the classification template for class $j$. It is produced by minimizing the KL divergence between a pseudo-label distribution $Q_{F,G}$ and the softmax distribution $P_F(x, W)$ over image-proxy similarities. The pseudo-labels come from an entropy-regularized optimal transport problem whose cost matrix $M_{i,j}$ is the similarity between image $x_i$ and the averaged embeddings of the top-$k$ retrieved descriptions, and the matrix is solved by Stable Greenkhorn, a log-domain, greedily updated variant of Sinkhorn that the paper introduces to replace Sinkhorn for medical data.

What would settle it

Use KPL on Shenzhen and Montgomery, two chest X-ray datasets with the same tuberculosis classes but different acquisition conditions: adapt KPL's proxies on one dataset and evaluate on the other without re-running the retrieval or pseudo-label steps. If accuracy then falls back to CLIP or VCD levels, the reported gains come from transductive use of the test images rather than from a genuine zero-shot capability.

Watch

Extended reading notes

Core claim

The paper's central claim is that a frozen CLIP model can classify medical images much more accurately by optimizing its class proxies in two steps rather than using plain category-name embeddings. In the first step, Text Proxy Optimization, KPL builds a knowledge-enhanced base of at least fifty LLM-generated visual descriptions per class, then retrieves the top-$k$ descriptions whose text embeddings are closest to the dataset's mean image feature, and averages them to form semantic text proxies. In the second step, Multimodal Proxy Learning, KPL treats these text proxies as anchors for an entropy-regularized pseudo-label matrix $Q_{F,G}$, refines that matrix with the proposed Stable Greenkhorn (SG) algorithm instead of Sinkhorn, and then learns multimodal class proxies $W^{KPL}$ by minimizing the KL divergence between the pseudo-label distribution and the softmax of image-proxy similarities. The paper reports that this two-step procedure outperforms all four baselines on all five medical datasets with both tested CLIP backbones, with the largest improvement on MalariaCell rising from 50.83% to 80.86%, and that it also improves a medically pretrained BioMedCLIP model, including a Cataract gain from 16.47% to 51.25%.

Load-bearing premise

The method assumes that adapting the class proxies using the very images it will later classify, through the dataset-average image used for retrieval and the pseudo-labels computed from those same images, produces a classifier whose higher scores on those images reflect genuine zero-shot ability rather than in-sample fitting.

Editorial extensions

If this is right

  • On the five medical benchmarks in Table 1, KPL outperforms CLIP, VCD, CMD, and InMaP with both ViT-L/14 and ViT-L/14@336px backbones.
  • Applying KPL to BioMedCLIP, a domain-specific medical vision-language model, improves its zero-shot accuracy on all tested medical datasets, with Cataract rising from 16.47% to 51.25%.
  • The same procedure transfers to natural image datasets, where KPL with Sinkhorn beats CLIP, VCD, Hierarchical, and InMaP in most of the nine backbone-dataset configurations in Table 2.
  • Ablation results indicate that all three components matter: the LLM-generated knowledge-enhanced base beats VCD's descriptions, visual-based retrieval beats random description selection, and Stable Greenkhorn beats Sinkhorn for pseudo-label refinement.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reported accuracies are transductive: both description retrieval and pseudo-label estimation use the same unlabeled images that are later classified, so KPL is not a conventional out-of-sample zero-shot classifier; a fair deployment test would freeze the adaptation on one set and evaluate on a disjoint sample.
  • Because the largest gains occur on classes where CLIP's text-name baseline is near chance, such as MalariaCell, KPL may be surfacing discriminative visual knowledge CLIP already contains, meaning the method's ceiling is bounded by what CLIP's text encoder can express.
  • The method's dependence on LLM prompt quality and on the dataset-mean image for retrieval suggests its gains will likely vary across medical modalities and dataset homogeneity, which is testable on histopathology or ultrasound data.
  • A natural extension is to replace the global mean image with a small labeled support set for retrieval; if the gains persist, the same framework could become a few-shot adaptable inference procedure.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Knowledge Proxy Learning (KPL), a training-free method for CLIP-based image classification aimed primarily at medical datasets. KPL first uses an LLM to generate a large set of class descriptions, then ranks these descriptions by cosine similarity to the global mean of the test image features (Eq. 3) to form semantic text proxies. It then solves an entropic optimal-transport problem on the same test set to obtain pseudo-labels (Eq. 4), using a proposed 'Stable GreenkHorn' (SG) variant of Sinkhorn scaling, and fits multimodal proxies by minimizing KL divergence (Eq. 1). The final classifier is evaluated on the same images that were used for retrieval, pseudo-labeling, and proxy fitting. Experiments on five medical and four natural image datasets compare KPL with CLIP, VCD, CMD, and InMaP, reporting accuracy gains in most settings, with ablations on the KEB, visual-based retrieval, and SG components.

Significance. If the reported gains survive an honest evaluation protocol, KPL would be a practically useful extension of proxy-learning for medical image classification. The paper's strengths include a clearly stated two-stage method, release of code, experiments across multiple datasets and backbones, and ablation studies that attempt to isolate the contribution of each component. However, the current evaluation protocol is transductive: the unlabeled test set D is used in Eqs. 1, 3, and 4, and accuracy is then computed on the same D. The comparison against inductive baselines (CLIP, VCD, CMD) therefore conflates test-set adaptation with method quality, and the abstract's 'zero-shot' claim is overstated. The comparison with InMaP, which is also transductive, is more appropriate and does show gains, but hyperparameter selection by grid search on each target dataset without a described validation split introduces additional leakage. These issues are load-bearing for the central empirical claim and require substantive revision rather than mere copy-editing.

major comments (4)
  1. [§4.2, Eqs. 1, 3, 4] The method uses the full test set D to compute the global image mean for visual-based retrieval (Eq. 3), to construct the matrix M and solve the Sinkhorn/SG pseudo-label problem (Eq. 4), and to fit the multimodal proxies W in Eq. 1, and then evaluates accuracy on the same images in D. This is a transductive protocol, so the accuracies in Tables 1 and 2 are in-sample and do not demonstrate zero-shot out-of-sample generalization. The comparison with CLIP, VCD, and CMD, which do not see D, is therefore not apples-to-apples. I recommend either evaluating on a genuinely separate unseen test set or explicitly reframing the setting as transductive and restricting the headline comparison to other transductive methods such as InMaP.
  2. [§4.1, Experimental Setup] The paper states that k, Nmax, and τ are determined through grid search on each target dataset (Appendix Figures 1 and 2), but no labeled validation split is mentioned. If the grid search selects hyperparameters by the final accuracy on D, label information leaks into the evaluation. Please specify exactly what data is used for model selection, or report results across a range of hyperparameters with sensitivity analysis; otherwise the reported numbers cannot be taken as unbiased estimates of accuracy.
  3. [§4.2, Tables 1 and 2] The claim that KPL 'consistently and significantly outperforms' the baselines is not supported by any measure of variance, multiple runs, or significance tests; the tables report single-point accuracies. Either provide mean ± std over several runs with a significance test, or replace the word 'significantly' with a statement about observed improvements on the reported runs.
  4. [§4.3, Table 3] The ablation study does not include a variant that uses KEB text proxies with the original Sinkhorn pseudo-labeling, nor a variant that uses VCD descriptions with the proposed SG algorithm. Without such cross-condition ablations, the contribution of the knowledge-enhanced base and the contribution of SG relative to InMaP are not isolated, and the claim that both components are necessary is not fully supported.
minor comments (5)
  1. [§4.1, Experimental Setup] The text contains the typo 'Apendix Figure 1' and 'Apendix Figure 2'; these should read 'Appendix Figure 1' and 'Appendix Figure 2'.
  2. [References] The citation 'V ondrick' appears with a spurious space (e.g., in Menon and V ondrick 2023); please fix the LaTeX formatting.
  3. [§3.2 and Preliminaries] The proposed algorithm is called 'Stable GreenkHorn' in Section 3.2 but 'Greenkhorn' in the related-work paragraph; standardize the capitalization.
  4. [Table 2] The symbol 'KPLs' is used in Table 2 without a definition in the caption or the main text; please explain that it denotes KPL with Sinkhorn rather than SG.
  5. [Figure 6] The caption states 'k=3' while the text writes 'k = 3' in prose; unify the notation and define k in the caption.

Circularity Check

3 steps flagged · score 6.0 of 10

KPL's headline zero-shot gains are in-sample fits: Eq. (3) retrieves descriptions using the test-set image mean, Eq. (4) builds pseudo-labels from all test images, Eq. (1) fits multimodal proxies to those pseudo-labels, and accuracy is reported on the same D.

  1. fitted input called prediction [Methodology, Text Proxy Optimization, Eq. (3)]
    "C ′ j = {t σ(1) Cj , . . . , t σ(k) Cj }, (3) where {sσ(1), . . . , sσ(n)} = sort↓ ({s1, . . . , sn}) , sl = cos⟨ 1 |D| X x∈D F (x), G(t l Cj )⟩, ∀l = 1, ..., n."

    The visual-based retrieval that selects the text descriptions is scored against the global mean of the visual features of the very images in D that will later be classified. Eq. (3) therefore injects D into the construction of the text proxies, so the semantic side of the classifier is already tuned to the evaluation set before any classification is performed.

  2. fitted input called prediction [Methodology, Multimodal Proxy Learning, Eq. (4) and Eq. (1)]
    "QF,G(D, C) = arg max P :Pi,j ≥0,P j Pi,j = 1 |D| ,P i Pi,j =qj (< M, P >+τ H(P )) , Mi,j = F (xi) · 1 |C ′ j| X t∈C′ j G(t), ... ∀i = 1, . . . ,|D|, ∀j = 1, . . . ,|C| (4) ... WKP L(D, C) = arg min W Ex∼D[d(QF,G(x; D, C), PF (x, W))], (1) ... Finally, the classification is conducted on the image dataset D: ˆy = f (x) = argmax j∈{1,...,K} F (x) · wKP Lj , ∀x ∈ D."

    The same dataset D appears as both the fitting set and the evaluation set. M in Eq. (4) is constructed from F(x_i) for every x_i in D, and QF,G(D,C) is the pseudo-label target with row/column constraints over D. Eq. (1) then minimizes the KL divergence between Q and the model's predictions on D, and the final accuracy is computed on the same images. The reported 'zero-shot' accuracy is therefore a measure of fit to D, forced by construction to track Q(D), rather than an out-of-sample prediction. Comparisons with inductive CLIP, VCD, and CMD conflate test-set adaptation with method quality.

1 more flagged steps
  1. other [Experiments, Experimental Setup]
    "The parameters k, Nmax, and τ are determined through Grid Search (Apendix Figure 1)."

    As described, the grid search is conducted on the same target datasets whose accuracies are reported, with no held-out labeled validation split specified. If the reported accuracy is the selection criterion, this is an additional route by which D determines the final classifier, reinforcing the in-sample character of the results in Tables 1 and 2.

full rationale

KPL's derivation is not circular in the self-citation sense: the SG algorithm is an original modification of Sinkhorn, KEB is built from an external LLM (GPT-3.5-turbo), and the comparison against InMaP, which also uses D, is a fairer transductive comparison and still shows gains. However, the central empirical claim—'zero-shot image classification, outperforming all baselines'—is partially circular and statistically leaky because D is used at three points: Eq. (3) for retrieval, Eq. (4) for pseudo-label generation, and Eq. (1) for proxy fitting, with final accuracy measured on the same D. The gains over inductive CLIP, VCD, and CMD therefore do not establish zero-shot generalization; they demonstrate in-sample adaptation to the evaluation set. The absence of a described held-out validation split for the grid search adds a second leakage channel. Overall score 6: the 'prediction' partially reduces to fitting the evaluation set, but the method retains independent content in SG, KEB, and its improvements over the transductive InMaP baseline.

Assumptions & free parameters 4 free parameters · 5 assumptions · 1 invented entities

The method relies on CLIP's fixed embeddings, LLM-generated knowledge, a retrieval heuristic using the dataset mean, and a pseudo-labeling scheme all applied to the test set itself. The main free parameters are k, Nmax, tau, and the LLM description count; all are tuned per dataset, which further couples the method to the test data. No new physical entities are introduced.

free parameters (4)
  • k (number of retrieved descriptions per class) = Grid-searched; default 5 in main experiments (Table 3)
    The top-k descriptions from the KEB are retrieved per class (Eq. 3); k is selected by grid search per dataset (Appendix Figure 1), affecting text proxy quality.
  • Nmax (maximum SG iterations) = Grid-searched
    Maximum number of iterations in the Stable Greenkhorn algorithm; determined via grid search (Appendix Figure 1).
  • tau (temperature) = Grid-searched
    Temperature in the entropy-regularized optimal transport (Eq. 4) and the proxy learning objective (Eq. 1); tuned by grid search.
  • n_min (minimum LLM descriptions per class) = 50
    The LLM prompt asks for at least 50 descriptions per class; this hand-chosen threshold sets the size of the knowledge-enhanced base.
assumptions (5)
  • domain assumption CLIP's vision and text encoders provide a meaningful shared embedding space for medical images and disease descriptions.
    The entire method rests on CLIP's cross-modal alignment being useful for medical data, an assumption the paper itself notes is imperfect (modal gap discussion in Section 1).
  • domain assumption LLM-generated descriptions in the KEB accurately capture the visual features needed to distinguish medical categories.
    Eq. 2 relies on ChatGPT to produce at least 50 discriminative features per class. The paper only validates this internally via ablations, not against external medical knowledge sources.
  • domain assumption Cosine similarity between the dataset-mean image feature and a text description identifies image-relevant knowledge.
    Eq. 3 selects descriptions with highest sl = cos(1/|D| sum_i F(x_i), G(t_l)). This assumes the average feature of all images in D is a good probe for class-relevant descriptions, which is not theoretically justified.
  • domain assumption The pseudo-label matrix P computed from D (Eq. 4) provides a reliable guide for learning multimodal proxies.
    The method trusts the entropy-regularized optimal transport solution (computed via SG) as pseudo-labels; no theoretical guarantee is given that these pseudo-labels are accurate for medical images.
  • standard math Sinkhorn's theorem guarantees the unique solution form P = diag(u) A diag(v) for the optimization in Eq. 4.
    Proposition 1 cites Sinkhorn (1964) for the existence and uniqueness of the scaling; the paper does not prove this theorem itself.
invented entities (1)
  • Knowledge-Enhanced Base (KEB)
    purpose: A per-category database of LLM-generated visual descriptions, used as the source for text proxy retrieval in Text Proxy Optimization.
    The KEB is constructed for this paper using ChatGPT; the paper provides no external validation that the descriptions are accurate or complete, only internal comparisons to VCD descriptions. It is an invented resource specific to this work.

how reviews work

0 comments
Cite this review

Pith. "Pith review of KPL: Training-Free Medical Knowledge Mining of Vision-Language Models." pith.science (2026). https://pith.science/paper/OOZBR36I

@misc{pith2026250111231,
  author       = {Pith},
  title        = {Pith review of: KPL: Training-Free Medical Knowledge Mining of Vision-Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OOZBR36I}},
  note         = {Machine review of arXiv:2501.11231}
}
read the original abstract

Visual Language Models such as CLIP excel in image recognition due to extensive image-text pre-training. However, applying the CLIP inference in zero-shot classification, particularly for medical image diagnosis, faces challenges due to: 1) the inadequacy of representing image classes solely with single category names; 2) the modal gap between the visual and text spaces generated by CLIP encoders. Despite attempts to enrich disease descriptions with large language models, the lack of class-specific knowledge often leads to poor performance. In addition, empirical evidence suggests that existing proxy learning methods for zero-shot image classification on natural image datasets exhibit instability when applied to medical datasets. To tackle these challenges, we introduce the Knowledge Proxy Learning (KPL) to mine knowledge from CLIP. KPL is designed to leverage CLIP's multimodal understandings for medical image classification through Text Proxy Optimization and Multimodal Proxy Learning. Specifically, KPL retrieves image-relevant knowledge descriptions from the constructed knowledge-enhanced base to enrich semantic text proxies. It then harnesses input images and these descriptions, encoded via CLIP, to stably generate multimodal proxies that boost the zero-shot classification performance. Extensive experiments conducted on both medical and natural image datasets demonstrate that KPL enables effective zero-shot image classification, outperforming all baselines. These findings highlight the great potential in this paradigm of mining knowledge from CLIP for medical image classification and broader areas.

Figures

Figures reproduced from arXiv: 2501.11231 by the authors.

Figure 1
Figure 1. VCD utilizes a limited number of descriptions that [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. KPL is designed to leverage CLIP’s capabilities [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Overview of KPL. 1) Category texts (e.g., cataract) pass through a Knowledge-Enhanced Base and are encoded by [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: PCA Visualization on the IDRiD with CLIP: (a) [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Ablation studies with the KEB show that Multimodal Proxy Learning with KPL descriptions (KPL [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: (a) Ablation study for visual-based retrieval at [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. HSCR: Hierarchical Self-Contrastive Rewarding for Aligning Medical Vision Language Models

    cs.CV 2025-06 conditional novelty 6.0 of 10

    HSCR uses visual token dropout and logit contrast to construct self-generated dispreferred answers, then trains a medical VLM with explicit and implicit preference losses, improving zero-shot Rad-VQA, SLAKE, and PathV...

  2. Fair-MoE: Fairness-Oriented Mixture of Experts in Vision-Language Models

    cs.CV 2025-02 reject novelty 5.0 of 10

    Fair-MoE reports improved accuracy and fairness on Harvard-FairVLMed for some protected attributes by adding sparse mixture-of-experts layers and a variance-based fairness loss to CLIP, but the all-attribute improveme...

  3. Comparison of ConvNeXt and Vision-Language Models for Breast Density Assessment in Screening Mammography

    eess.IV 2025-06 conditional novelty 4.0 of 10

    Fine-tuned ConvNeXt achieves 0.73 accuracy and 0.78 F1, beating BioMedCLIP linear probe (0.64/0.63) and zero-shot (0.47/0.31) for BI-RADS breast density classification.

Reference graph

Works this paper leans on

52 extracted references · 40 canonical work pages · cited by 3 Pith papers

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    J.; Bagaria, V

    Abid, A.; Zhang, M. J.; Bagaria, V. K.; and Zou, J. 2018. Exploring patterns enriched in a dataset with contrastive principal component analysis. Nature communications, 9(1): 2134

    work page 2018

  4. [4]

    Ali, M.; and Khan, S. 2023. CLIP-Decoder: ZeroShot Multilabel Classification using Multimodal CLIP Aligned Representations. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 4675--4679

    work page 2023

  5. [5]

    Altschuler, J.; Niles-Weed, J.; and Rigollet, P. 2017. Near-linear time approximation algorithms for optimal transport via Sinkhorn iteration. Advances in neural information processing systems, 30

    work page 2017

  6. [6]

    Bacharach, M. 1965. Estimating nonnegative matrices from marginal data. International Economic Review, 6(3): 294--310

    work page 1965

  7. [7]

    Chang, S.; Qu, Z.; Leskovec, J.; and Ugander, J. 2023. Inferring Networks from Marginals Using Iterative Proportional Fitting. In The Second Learning on Graphs Conference

    work page 2023

  8. [8]

    Chen, X.; Lai, Z.; Ruan, K.; Chen, S.; Liu, J.; and Liu, Z. 2024. R-LLaVA: Improving Med-VQA Understanding through Visual Region of Interest. arXiv preprint arXiv:2410.20327

    arXiv 2024

Show all 52 references
  1. [9]

    Deng, J.; Dong, W.; Socher, R.; Li, L.-J.; Li, K.; and Fei-Fei, L. 2009. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, 248--255. Ieee

    2009

  2. [10]

    Eslami, S.; de Melo, G.; and Meinel, C. 2021. Does CLIP Benefit Visual Question Answering in the Medical Domain as Much as it Does in the General Domain? arXiv preprint arXiv:2112.13906

    2021 arXiv

  3. [11]

    Esmaeilpour, S.; Liu, B.; Robertson, E.; and Shu, L. 2022. Zero-shot out-of-distribution detection based on the pre-trained model clip. In Proceedings of the AAAI conference on artificial intelligence, volume 36, 6568--6576

    2022

  4. [12]

    Gai, X.; Zhou, C.; Liu, J.; Feng, Y.; Wu, J.; and Liu, Z. 2024. MedThink: Explaining Medical Visual Question Answering via Multimodal Decision-Making Rationale. arXiv preprint arXiv:2404.12372

    2024 arXiv

  5. [13]

    Guo, D.; Li, K.; Hu, B.; Zhang, Y.; and Wang, M. 2024. Benchmarking Micro-action Recognition: Dataset, Method, and Application. IEEE Transactions on Circuits and Systems for Video Technology, 34(7): 6238--6252

    2024

  6. [14]

    Y.; Hikal, N

    Hassan, E.; Shams, M. Y.; Hikal, N. A.; and Elmougy, S. 2022. A novel convolutional neural network model for malaria cell images classification. Computers, Materials & Continua, 72(3): 5889--5907

    2022

  7. [15]

    H.; Liang, T.; Sukerkar, P

    Hung, A. H.; Liang, T.; Sukerkar, P. A.; and Meade, T. J. 2013. High dynamic range processing for magnetic resonance imaging. PloS one, 8(11): e77883

    2013

  8. [16]

    Idel, M. 2016. A review of matrix scaling and Sinkhorn's normal form for matrices and positive maps. arXiv preprint arXiv:1609.06349

    2016 arXiv

  9. [17]

    J.; Lu, P.-X.; and Thoma, G

    Jaeger, S.; Candemir, S.; Antani, S.; W \'a ng, Y.-X. J.; Lu, P.-X.; and Thoma, G. 2014. Two public chest X-ray datasets for computer-aided screening of pulmonary diseases. Quantitative imaging in medicine and surgery, 4(6): 475

    2014

  10. [18]

    Jeong, J.; Zou, Y.; Kim, T.; Zhang, D.; Ravichandran, A.; and Dabeer, O. 2023. Winclip: Zero-/few-shot anomaly classification and segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 19606--19616

    2023

  11. [19]

    Jiao, S.; Wei, Y.; Wang, Y.; Zhao, Y.; and Shi, H. 2023. Learning mask-aware clip representations for zero-shot segmentation. Advances in Neural Information Processing Systems, 36: 35631--35653

    2023

  12. [20]

    S.; Yin, S.; Nathan, A.; and Rowlands, J

    Karim, K. S.; Yin, S.; Nathan, A.; and Rowlands, J. A. 2004. High-dynamic-range pixel architectures for diagnostic medical imaging. In Medical Imaging 2004: Physics of Medical Imaging, volume 5368, 657--667. SPIE

    2004

  13. [21]

    Kruithof, J. 1937. Telefoonverkeersrekening. De Ingenieur, 52: 15--25

    1937

  14. [22]

    Lai, H.; Yao, Q.; Jiang, Z.; Wang, R.; He, Z.; Tao, X.; and Zhou, S. K. 2024. CARZero: Cross-Attention Alignment for Radiology Zero-Shot Classification. arXiv preprint arXiv:2402.17417

    2024 arXiv

  15. [23]

    H.; Nickisch, H.; and Harmeling, S

    Lampert, C. H.; Nickisch, H.; and Harmeling, S. 2013. Attribute-based classification for zero-shot visual object categorization. IEEE transactions on pattern analysis and machine intelligence, 36(3): 453--465

    2013

  16. [24]

    W.; Zhang, Y.; Kwon, Y.; Yeung, S.; and Zou, J

    Liang, V. W.; Zhang, Y.; Kwon, Y.; Yeung, S.; and Zou, J. Y. 2022. Mind the gap: Understanding the modality gap in multi-modal contrastive representation learning. Advances in Neural Information Processing Systems, 35: 17612--17625

    2022

  17. [25]

    Lin, T.; Ho, N.; and Jordan, M. 2019. On efficient optimal transport: An analysis of greedy and accelerated mirror descent algorithms. In International Conference on Machine Learning, 3982--3991. PMLR

    2019

  18. [26]

    Lin, T.; Ho, N.; and Jordan, M. I. 2022. On the efficiency of entropic regularized algorithms for optimal transport. Journal of Machine Learning Research, 23(137): 1--42

    2022

  19. [27]

    Liu, J.; Hu, T.; Xiong, H.; Du, J.; Feng, Y.; Wu, J.; Zhou, J.; and Liu, Z. 2024 a . VPL: Visual Proxy Learning Framework for Zero-Shot Medical Image Diagnosis. In Findings of the Association for Computational Linguistics: EMNLP 2024, 9978--9992

    2024

  20. [28]

    Liu, J.; Hu, T.; Zhang, Y.; Feng, Y.; Hao, J.; Lv, J.; and Liu, Z. 2023 a . Parameter-Efficient Transfer Learning for Medical Visual Question Answering. IEEE Transactions on Emerging Topics in Computational Intelligence

    2023

  21. [29]

    Liu, J.; Hu, T.; Zhang, Y.; Gai, X.; FENG, Y.; and Liu, Z. 2023 b . A ChatGPT Aided Explainable Framework for Zero-Shot Medical Image Diagnosis. In ICML 3rd Workshop on Interpretable Machine Learning in Healthcare (IMLH)

    2023

  22. [30]

    Liu, J.; Wang, Y.; Du, J.; Zhou, J.; and Liu, Z. 2024 b . MedCoT: Medical Chain of Thought via Hierarchical Expert. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 17371--17389

    2024

  23. [31]

    L \'o pez-Cifuentes, A.; Escudero-Vinolo, M.; Besc \'o s, J.; and Garc \' a-Mart \' n, \'A . 2020. Semantic-aware scene recognition. Pattern Recognition, 102: 107256

    2020

  24. [32]

    Mahapatra, D.; Bozorgtabar, B.; and Ge, Z. 2021. Medical image classification using generalized zero shot learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 3344--3353

    2021

  25. [33]

    Mahapatra, D.; Ge, Z.; and Reyes, M. 2022. Self-supervised generalized zero shot learning for medical image classification using novel interpretable saliency maps. IEEE Transactions on Medical Imaging, 41(9): 2443--2456

    2022

  26. [34]

    Menon, S.; and Vondrick, C. 2023. Visual Classification via Description from Large Language Models. In The Eleventh International Conference on Learning Representations

    2023

  27. [35]

    C.; and Garg, S

    Novack, Z.; McAuley, J.; Lipton, Z. C.; and Garg, S. 2023. Chils: Zero-shot image classification with hierarchical label sets. In International Conference on Machine Learning, 26342--26362. PMLR

    2023

  28. [36]

    Porwal, P.; Pachade, S.; Kamble, R.; Kokare, M.; Deshmukh, G.; Sahasrabuddhe, V.; and Meriaudeau, F. 2018. Indian diabetic retinopathy image dataset (IDRiD): a database for diabetic retinopathy screening research. Data, 3(3): 25

    2018

  29. [37]

    Qian, Q.; Xu, Y.; and Hu, J. 2024. Intra-Modal Proxy Learning for Zero-Shot Visual Categorization with CLIP. Advances in Neural Information Processing Systems, 36

    2024

  30. [38]

    W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al

    Radford, A.; Kim, J. W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al. 2021. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, 8748--8763. PMLR

    2021

  31. [39]

    Ren, Z.; Su, Y.; and Liu, X. 2024. ChatGPT-Powered Hierarchical Comparisons for Image Classification. Advances in Neural Information Processing Systems, 36

    2024

  32. [40]

    Rokhana, R.; Herulambang, W.; and Indraswari, R. 2022. Classification of ocular diseases on fundus images using weighted MobileNetV2. In 2022 International Electronics Symposium (IES), 570--575. IEEE

    2022

  33. [41]

    Shi, L.; Fan, J.; and Yan, J. 2024. OT-CLIP: Understanding and Generalizing CLIP via Optimal Transport. In Forty-first International Conference on Machine Learning

    2024

  34. [42]

    Sinkhorn, R. 1964. A relationship between arbitrary positive matrices and doubly stochastic matrices. The annals of mathematical statistics, 35(2): 876--879

    1964

  35. [43]

    Stephan, F. F. 1942. An iterative method of adjusting sample frequency tables when expected marginal totals are known. The Annals of Mathematical Statistics, 13(2): 166--178

    1942

  36. [44]

    Wah, C.; Branson, S.; Welinder, P.; Perona, P.; and Belongie, S. 2011. The caltech-ucsd birds-200-2011 dataset

    2011

  37. [45]

    Wang, H.; Li, Y.; Yao, H.; and Li, X. 2023. Clipn for zero-shot ood detection: Teaching clip to say no. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 1802--1812

    2023

  38. [46]

    Wang, Z.; Codella, N.; Chen, Y.-C.; Zhou, L.; Yang, J.; Dai, X.; Xiao, B.; You, H.; Chang, S.-F.; and Yuan, L. 2022 a . Clip-td: Clip targeted distillation for vision-language tasks. arXiv preprint arXiv:2201.05729

    2022 arXiv

  39. [47]

    Wang, Z.; Wu, Z.; Agarwal, D.; and Sun, J. 2022 b . MedCLIP: Contrastive Learning from Unpaired Medical Images and Text. In 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022

    2022

  40. [48]

    Xian, Y.; Lorenz, T.; Schiele, B.; and Akata, Z. 2018. Feature generating networks for zero-shot learning. In Proceedings of the IEEE conference on computer vision and pattern recognition, 5542--5551

    2018

  41. [49]

    Yule, G. U. 1912. On the Methods of Measuring Association Between Two Attributes. Journal of the Royal Statistical Society, 75(6): 579--652

    1912

  42. [50]

    Y.; and Xiu, N

    Zhang, H.; Zhou, S.; Li, G. Y.; and Xiu, N. 2022. 0/1 deep neural networks via block coordinate descent. arXiv preprint arXiv:2206.09379

    2022 arXiv

  43. [51]

    Zhang, S.; Xu, Y.; Usuyama, N.; Bagga, J.; Tinn, R.; Preston, S.; Rao, R.; Wei, M.; Valluri, N.; Wong, C.; et al. 2023. Large-scale domain-specific pretraining for biomedical vision-language processing. arXiv preprint arXiv:2303.00915, 2(3): 6

    2023 arXiv

  44. [52]

    Zhou, Z.; Lei, Y.; Zhang, B.; Liu, L.; and Liu, Y. 2023. Zegclip: Towards adapting clip for zero-shot semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 11175--11185

    2023

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.