Pith. sign in

REVIEW 3 major objections 6 minor 29 references

MAKE: Multi-Aspect Knowledge-Enhanced Vision-Language Pretraining for Zero-shot Dermatological Assessment

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read MAKE pretrains a vision-language model on 403,563 dermatology image-text pairs decomposed into multiple knowledge aspects, and claims state-of-the-art zero-shot skin disease classification, concept annotation, and cross-modal retrieval…

desk verdict Useful combination of known VLP techniques for dermatology, but the zero-shot claim depends on unverified dataset hygiene—dedup analysis is the missing piece. read the letter →

arxiv 2505.09372 v1 pith:2VLRZNML submitted 2025-05-14 cs.CV

Siyuan Yan , Xieji Li , Ming Hu , Yiwen Jiang , Zhen Yu , Zongyuan Ge This is my paper · ORCID
classification cs.CV
keywords vision-languagepretrainingdermatologyzero-shotclassificationcontrastivelearningclinicaltextdecompositionconceptannotationcross-modalretrievalknowledgeenhancement
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Dermatology images come with long, unstructured clinical narratives, and standard vision-language models truncate those texts at a fixed token length, losing diagnostic detail. MAKE claims that decomposing each narrative into multiple knowledge aspects—the raw text, a disease-aspect text, a concept-aspect text, and individual sentences—and aligning each aspect with image patches during pretraining yields a model that transfers better to unseen skin conditions. Pretrained on 403,563 dermatological image-text pairs from public educational sources, MAKE reports state-of-the-art zero-shot results across eight datasets, including a 7.58% accuracy gain over the best baseline on DermNet and 3.95% on Fitzpatrick17K. The framework matters because it offers a way to use long clinical language in contrastive vision-language models without extending token limits, and it suggests that how text is structured during pretraining, not just its content, drives zero-shot generalization.

What carries the argument

The central object is the multi-aspect text decomposition: each image-text pair is expanded into a knowledge set of three texts (raw, disease-aspect, concept-aspect) and a subtext set of K sentences, producing K+3 embeddings that all serve as positive pairs in a multi-positive contrastive loss. Two additional mechanisms carry the argument: a fine-grained alignment that weights image patches by dot-product similarity to the raw text and pools them into a knowledge-enhanced visual embedding, and a diagnosis-guided weighting that scales each sub-caption's contribution by its cosine similarity to the disease-aspect embedding. Together the losses are $\mathcal{L}_{\text{total}} = \hat{w}\,\mathcal{L}_{\text{mkcl}} + \lambda \hat{w}\,\mathcal{L}_{\text{slra}}$, where the weights $\hat{w}$ come from the diagnosis similarity. This lets the model learn which aspects of clinical language matter for which visual features, without needing annotations of those aspects.

What would settle it

Search DermNet image URLs or near-duplicate images within Derm1M; if even a modest fraction of DermNet test images appear in pretraining, retrain on a deduplicated corpus and check whether the 7.58% DermNet accuracy gain over the strongest baseline shrinks or disappears.

Watch

Extended reading notes

Core claim

The central claim is that a vision-language model tailored to dermatology can beat general-purpose and domain-agnostic medical models on zero-shot skin tasks if it is trained to associate not one caption but a set of knowledge-decomposed sub-captions with each image. MAKE encodes the raw text plus two LLM-derived aspects (disease terminology and clinical concept descriptors) plus sentence-level sub-captions, and trains with three losses: a multi-positive contrastive loss aligning all K+3 texts with the image, a fine-grained alignment loss matching each sub-caption to a knowledge-enhanced pooled visual embedding, and a weighting term that up-weights sub-captions most similar to the disease-aspect text. The paper reports that this setup outperforms strong baselines on zero-shot disease classification, concept annotation, and cross-modal retrieval, with the largest gains on datasets that require broad clinical vocabulary.

Load-bearing premise

The evaluation assumes that none of the test images or captions, especially from DermNet, appear in the 403,563-image pretraining corpus, since Derm1M was crawled from public sources that could overlap with the test atlases.

Editorial extensions

If this is right

  • The framework's success in dermatology suggests the same decomposition approach will transfer to other medical specialties with long unstructured reports, such as pathology and ophthalmology.
  • The finding that knowledge-augmented captions hurt conventional VLMs but help MAKE implies that the benefit comes from the multi-aspect alignment structure, not from the augmented text alone, so future work can focus on structuring, not just enriching, medical captions.
  • Zero-shot concept annotation and retrieval improve alongside classification, so the model's internal alignments are finer-grained than label-level supervision, which may support interpretability tools that ground predictions in clinical concepts.
  • The diagnosis-guided weighting offers a principled way to inject clinical priors into contrastive pretraining, and it could be extended to patient history or demographic context as additional weighting signals.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A concrete risk not resolved by the paper: the pretraining corpus Derm1M draws on PubMed, textbooks, YouTube, and Twitter, while evaluation datasets like DermNet are public atlases; if any DermNet images or captions leaked into pretraining, the 7.58% DermNet gain would reflect memorization. A clean held-out evaluation on a corpus curated after removing all evaluation-set images would settle this.
  • The method's success suggests that sentence-level decomposition acts as a soft form of data augmentation for the text side; one testable extension is whether random sentence sampling with a similar multi-positive loss reproduces part of the gain without LLM-generated knowledge aspects.
  • Because the weighting uses cosine similarity to the disease-aspect text, the framework implicitly assumes the disease is named in or inferable from the raw text; in fully unlabeled clinical narratives without diagnosis mentions, this weighting signal would need an alternative source.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes MAKE, a vision-language pretraining framework for dermatology that decomposes clinical captions into multiple knowledge aspects via LLMs, aligns them with image and patch embeddings through several contrastive losses, and weights them by similarity to a disease-aspect embedding. The model is pretrained on Derm1M (403,563 image-text pairs) and evaluated zero-shot on eight datasets covering disease classification, concept annotation, and cross-modal retrieval. MAKE is reported to outperform CLIP, SigLIP, CoCa, BiomedCLIP, PMC-CLIP, and MONET on most benchmarks, with ablations attributing gains to the three proposed components.

Significance. If the results are robust and free of data overlap, MAKE would be a useful contribution: it addresses text length limits by structured knowledge decomposition, and the fine-grained patch-text alignment and diagnosis-guided weighting are reasonable, clearly described additions to standard CLIP-style training. The paper also compares several baselines under matched pretraining data and includes ablations. However, the absence of a dataset-overlap analysis and the lack of statistical validation mean that the headline zero-shot gains are not yet established.

major comments (3)
  1. [§3 Experimental details, Tables 1–2] The central zero-shot claim rests on the assumption that the 403,563 Derm1M pretraining pairs are disjoint from the evaluation images, but the paper provides no evidence for this. Derm1M is assembled from PubMed, textbooks, YouTube, and Twitter following the pipelines of [13] and [9], while DermNet and Fitzpatrick17k are public atlases whose images can easily appear in such sources. Exact- or near-duplicate removal against the eight evaluation sets is never reported, and Derm1M construction is only cited to the companion paper [22], so the overlap status cannot be checked from this manuscript. If overlap exists, the reported gains (e.g., 7.58% on DermNet and 3.95% on F17K) measure memorization rather than zero-shot generalization. Please report a deduplication analysis and either restrict Derm1M by removing overlapping sources or validate on a held-out set known to be disjoint.
  2. [§4 Results, Tables 1–3] All results are single point estimates with no error bars, confidence intervals, or significance tests, despite the abstract and text using the phrase "significantly outperforms." Given that several reported differences are small (e.g., 0.68% and 0.85% in the Table 3 ablation), the improvements may not be stable across training runs. Please report means and standard deviations over at least three pretraining seeds and apply an appropriate statistical test to support the significance claim.
  3. [§3 Implementation Details] The statement "we use the final checkpoint and conduct extensive hyperparameter tuning to find the optimal model" does not specify the selection criterion or whether any held-out validation set was used. If hyperparameters were selected on the test sets of Tables 1–2, the comparisons are overoptimistic; if not, the protocol should be described. Please clarify the tuning and validation procedure and report the chosen hyperparameters for every model.
minor comments (6)
  1. [§2.4, Eq. (8)] The loss weighting notation is ambiguous: Eq. (7) produces a vector of K subtext weights, while the knowledge-set embeddings have default weights of 1, but Eq. (8) writes w_mkcl and w_slra as scalars multiplying scalar losses. Please specify the elementwise application of these weight vectors.
  2. [§3 Experimental details] It is unclear whether Derm1M† supplies the three text types (raw, disease, concept) separately, as used by MAKE, or as a single prepended string, as stated for baselines; please clarify the input format for each model.
  3. [§2.1 Encoding Stage] The LLM used for knowledge extraction is not specified; please provide the model name, prompt template, and sampling settings, since the generated sub-texts are a core component and affect reproducibility.
  4. [§4 Results] The zero-shot classification and concept-annotation protocols are underspecified: no class-name templates are given for classification and no description of how per-concept AUROC is computed; please add this information.
  5. [§1 Introduction] The contribution statement claiming "the first vision-language pretraining framework for dermatology" overstates novelty given MONET [13] and other dermatology-oriented VLP efforts; please rephrase to claim the first multi-aspect knowledge-enhanced framework or similar.
  6. [Table 3 caption] The caption contains a typo: "without spitted text" should read "without split text."

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: MAKE's losses are derived from image-text inputs and LLM-augmented subtexts, not from downstream benchmarks; the self-cited Derm1M dataset supplies data resources rather than a conclusion.

full rationale

The paper's derivation chain is an empirical pretraining objective, not a theorem. Equations (1)-(8) define multi-aspect contrastive learning, fine-grained local alignment, and diagnosis-guided weighting entirely in terms of the raw image-text pairs, LLM-extracted disease/concept texts, sentence-decomposed subtexts, and patch embeddings. No loss or evaluation quantity is defined in terms of the downstream test sets or their labels, so the zero-shot results are not forced by construction. The ablation study compares loss variants on the same pretraining data, and the main comparison is against independently pretrained baselines. Derm1M is cited from the authors' companion work (ref 22), but that citation provides input data rather than an unverified theorem or fitted parameter, so it does not create a circular justification. The most serious integrity risk is possible train/test overlap between Derm1M's crawled education resources and public benchmarks such as DermNet and Fitzpatrick17K, since the paper reports no deduplication; however, that is a data-contamination and external-validity concern, not a circularity of the derivation, and the paper itself does not assert or demonstrate the overlap needed to make it circular. Overall, no step reduces by construction to its inputs or to a self-citation chain.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The ledger captures the hand-set hyperparameters and domain assumptions that the reported results depend on. The most consequential free parameters are the loss weight lambda and the training hyperparameters, which are chosen without reported sensitivity analysis. The central domain assumptions are the accuracy of LLM-generated knowledge text, the quality of the Derm1M corpus, and the absence of train/evaluation overlap.

free parameters (6)
  • lambda = 0.7
    Loss weighting factor between mkcl and slra in Eq. 8; chosen by hand, no sensitivity study.
  • learning_rate = 1e-4
    Set for all models in Section 3; no grid reported.
  • weight_decay = 0.1
    Optimization hyperparameter from Section 3.
  • batch_size = 2048
    Batch size for pretraining in Section 3.
  • warmup_steps = 1500
    Warmup length in Section 3.
  • epochs = 15
    Pretraining duration in Section 3.
assumptions (4)
  • domain assumption LLM-generated disease-aspect and concept-aspect texts are clinically accurate and free of hallucinated findings.
    Section 2.1 uses LLMs to expand each raw text with disease and concept aspects, but no manual verification or filtering is described.
  • domain assumption The Derm1M image-text pairs are correctly matched and representative of the dermatology domain.
    Section 3 relies on Derm1M with pairs from PubMed, textbooks, YouTube, and Twitter; no cleaning or quality control is reported in this paper.
  • domain assumption No evaluation dataset overlaps with the Derm1M pretraining corpus.
    Section 4 evaluates on DermNet and SkinCon, which are public atlases; the paper does not analyze crawl leakage from dermnet.com or other sources.
  • domain assumption Cosine similarity between subtext and disease-aspect embeddings is a valid proxy for clinical diagnostic significance.
    Eq. 7 uses this similarity to weight all sub-captions; no clinical validation or ablation of this weighting proxy is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MAKE: Multi-Aspect Knowledge-Enhanced Vision-Language Pretraining for Zero-shot Dermatological Assessment." pith.science (2026). https://pith.science/paper/2VLRZNML

@misc{pith2026250509372,
  author       = {Pith},
  title        = {Pith review of: MAKE: Multi-Aspect Knowledge-Enhanced Vision-Language Pretraining for Zero-shot Dermatological Assessment},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2VLRZNML}},
  note         = {Machine review of arXiv:2505.09372}
}
read the original abstract

Dermatological diagnosis represents a complex multimodal challenge that requires integrating visual features with specialized clinical knowledge. While vision-language pretraining (VLP) has advanced medical AI, its effectiveness in dermatology is limited by text length constraints and the lack of structured texts. In this paper, we introduce MAKE, a Multi-Aspect Knowledge-Enhanced vision-language pretraining framework for zero-shot dermatological tasks. Recognizing that comprehensive dermatological descriptions require multiple knowledge aspects that exceed standard text constraints, our framework introduces: (1) a multi-aspect contrastive learning strategy that decomposes clinical narratives into knowledge-enhanced sub-texts through large language models, (2) a fine-grained alignment mechanism that connects subcaptions with diagnostically relevant image features, and (3) a diagnosis-guided weighting scheme that adaptively prioritizes different sub-captions based on clinical significance prior. Through pretraining on 403,563 dermatological image-text pairs collected from education resources, MAKE significantly outperforms state-of-the-art VLP models on eight datasets across zero-shot skin disease classification, concept annotation, and cross-modal retrieval tasks. Our code will be made publicly available at https: //github.com/SiyuanYan1/MAKE.

Figures

Figures reproduced from arXiv: 2505.09372 by the authors.

Figure 1
Figure 1. Comparison between training strategies. Our framework utilizes CLIP with multi-knowledge enhanced learning, addressing long caption modeling limitations. It enables dynamic knowledge modeling between each image and its multiple corre￾sponding captions, each capturing diverse aspects of crucial dermatological knowledge. timodal nature of dermatological knowledge. This multimodal complexity ne￾cessitates more sophisti… view at source ↗
Figure 2
Figure 2. Overview of our MAKE framework. (a) Encoding multi-aspect clinical knowledge. (b) The process of aligning visual embeddings with multiple positive text embeddings. Knowledge 1: disease-aspect text; Knowledge 2: concept-aspect text. (c) A fine-grained alignment process, which matches each subtext embedding with knowledge-enhanced visual embeddings. (d) Diagnosis similarity-based weights that modulate alignment betwee… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

29 extracted references · 11 canonical work pages

  1. [13]

    Nature Medicine30(4), 1154–1165 (2024)

    Kim, C., Gadgil, S.U., DeGrave, A.J., Omiye, J.A., Cai, Z.R., Daneshjou, R., Lee, S.I.: Transparent medical image ai via an image–text foundation model grounded in medical literature. Nature Medicine30(4), 1154–1165 (2024). https://doi.org/10.1038/s41591-024-02887-x, https://doi.org/10.1038/s41591- 024-02887-x

    doi:10.1038/s41591-024-02887-x 2024

  2. [9]

    Advances in neural information processing systems36, 37995– 38017 (2023)

    Ikezogwo, W., Seyfioglu, S., Ghezloo, F., Geva, D., Sheikh Mohammed, F., Anand, P.K., Krishna, R., Shapiro, L.: Quilt-1m: One million image-text pairs for histopathology. Advances in neural information processing systems36, 37995– 38017 (2023)

    work page 2023

  3. [22]

    arXiv preprint arXiv:2503.14911 (2025)

    Yan,S.,Hu,M.,Jiang,Y.,Li,X.,Fei,H.,Tschandl,P.,Kittler,H.,Ge,Z.:Derm1m: A million-scale vision-language dataset aligned with clinical ontology knowledge for dermatology. arXiv preprint arXiv:2503.14911 (2025)

    arXiv 2025

  4. [1]

    Advances in Neural Information Processing Systems35, 18157–18167 (2022)

    Daneshjou, R., Yuksekgonul, M., Cai, Z.R., Novoa, R., Zou, J.Y.: Skincon: A skin disease dataset densely annotated by domain experts for fine-grained debugging and analysis. Advances in Neural Information Processing Systems35, 18157–18167 (2022)

    work page 2022

  5. [2]

    Dermnet: Dermnet (2023),https://dermnet.com/

    work page 2023

  6. [3]

    arXiv preprint arXiv:2010.11929 (2020)

    Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al.: An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)

    arXiv 2020

  7. [4]

    nature 542(7639), 115–118 (2017)

    Esteva, A., Kuprel, B., Novoa, R.A., Ko, J., Swetter, S.M., Blau, H.M., Thrun, S.: Dermatologist-level classification of skin cancer with deep neural networks. nature 542(7639), 115–118 (2017)

    2017

  8. [5]

    Advances in Neural Information Processing Systems36, 35544– 35575 (2023)

    Fan, L., Krishnan, D., Isola, P., Katabi, D., Tian, Y.: Improving clip training with language rewrites. Advances in Neural Information Processing Systems36, 35544– 35575 (2023)

    2023

Show all 29 references
  1. [6]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Groh, M., Harris, C., Soenksen, L., Lau, F., Han, R., Kim, A., Koochek, A., Badri, O.: Evaluating deep neural networks trained on clinical images in dermatology with the fitzpatrick 17k dataset. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognit...

    2021

  2. [7]

    https://doi.org/10.6084/m9.figshare.6454973.v12

    Han, S.S.: SNU dataset + Quiz (3 2019). https://doi.org/10.6084/m9.figshare.6454973.v12

    2019 doi

  3. [8]

    arXiv preprint arXiv:2411.15421 (2024)

    Hu, M., Yuan, K., Shen, Y., Tang, F., Xu, X., Zhou, L., Li, W., Chen, Y., Xu, Z., Peng, Z., et al.: Ophclip: Hierarchical retrieval-augmented learning for ophthalmic surgical video-language pretraining. arXiv preprint arXiv:2411.15421 (2024)

    2024 arXiv

  4. [10]

    In: Proceedings of the AAAI conference on artificial intelligence

    Irvin, J., Rajpurkar, P., Ko, M., Yu, Y., Ciurea-Ilcus, S., Chute, C., Marklund, H., Haghgoo, B., Ball, R., Shpanskaya, K., et al.: Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison. In: Proceedings of the AAAI conference on artificial in...

    2019

  5. [11]

    Scientific data 6(1), 317 (2019) 10 Authors Suppressed Due to Excessive Length

    Johnson, A.E., Pollard, T.J., Berkowitz, S.J., Greenbaum, N.R., Lungren, M.P., Deng, C.y., Mark, R.G., Horng, S.: Mimic-cxr, a de-identified publicly available database of chest radiographs with free-text reports. Scientific data 6(1), 317 (2019) 10 Authors Suppressed Due to E...

    2019

  6. [12]

    IEEE Journal of Biomedical and Health Informatics 23(2), 538–546 (mar 2019)

    Kawahara, J., Daneshvar, S., Argenziano, G., Hamarneh, G.: Seven-point check- list and skin lesion classification using multitask multimodal neural nets. IEEE Journal of Biomedical and Health Informatics 23(2), 538–546 (mar 2019). https://doi.org/10.1109/JBHI.2018.2824327

    2019

  7. [14]

    In: International Conference on Medical Image Computing and Computer-Assisted Intervention

    Lin,W.,Zhao,Z.,Zhang,X.,Wu,C.,Zhang,Y.,Wang,Y.,Xie,W.:Pmc-clip:Con- trastive language-image pre-training using biomedical documents. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 525–536. Springer (2023)

    2023

  8. [15]

    Nature medicine26(6), 900–908 (2020)

    Liu,Y.,Jain,A.,Eng,C.,Way,D.H.,Lee,K.,Bui,P.,Kanada,K.,deOliveiraMar- inho, G., Gallegos, J., Gabriele, S., et al.: A deep learning system for differential diagnosis of skin diseases. Nature medicine26(6), 900–908 (2020)

    2020

  9. [16]

    Data in Brief32, 106221 (2020)

    Pacheco, A.G., et al.: Pad-ufes-20: A skin lesion dataset composed of patient data and clinical images collected from smartphones. Data in Brief32, 106221 (2020). https://doi.org/https://doi.org/10.1016/j.dib.2020.106221

    2020

  10. [17]

    In: International conference on machine learning

    Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: International conference on machine learning. pp. 8748–8763. PmLR (2021)

    2021

  11. [18]

    OpenAI blog1(8), 9 (2019)

    Radford,A.,Wu,J.,Child,R.,Luan,D.,Amodei,D.,Sutskever,I.,etal.:Language models are unsupervised multitask learners. OpenAI blog1(8), 9 (2019)

    2019

  12. [19]

    NPJ Digital Medicine7(1), 28 (2024)

    Shen, Y., Li, H., Sun, C., Ji, H., Zhang, D., Hu, K., Tang, Y., Chen, Y., Wei, Z., Lv, J.: Optimizing skin disease diagnosis: harnessing online community data with contrastive learning and clustering techniques. NPJ Digital Medicine7(1), 28 (2024)

    2024

  13. [20]

    In: Leibe, B., Matas, J., Sebe, N., Welling, M

    Sun, X., Yang, J., Sun, M., Wang, K.: A benchmark for automatic visual classifi- cation of clinical skin disease images. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) Computer Vision – ECCV 2016. pp. 206–222. Springer International Publishing, Cham (2016)

    2016

  14. [21]

    Xie, Y., Chen, Q., Wang, S., To, M.S., Lee, I., Khoo, E.W., Hendy, K., Koh, D., Xia, Y., Wu, Q.: Pairaug: What can augmented image-text pairs do for radiology? In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 11652–11661 (2024)

    2024

  15. [23]

    arXiv preprint arXiv:2410.15038 (2024)

    Yan, S., Yu, Z., Primiero, C., Vico-Alonso, C., Wang, Z., Yang, L., Tschandl, P., Hu, M., Tan, G., Tang, V., et al.: A general-purpose multimodal foundation model for dermatology. arXiv preprint arXiv:2410.15038 (2024)

    2024 arXiv

  16. [24]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Yan, S., Yu, Z., Zhang, X., Mahapatra, D., Chandra, S.S., Janda, M., Soyer, P., Ge, Z.: Towards trustable skin cancer diagnosis via rewriting model’s decision. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 11568–11577 (2023)

    2023

  17. [25]

    arXiv preprint arXiv:2205.01917 (2022) Title Suppressed Due to Excessive Length 11

    Yu, J., Wang, Z., Vasudevan, V., Yeung, L., Seyedhosseini, M., Wu, Y.: Coca: Contrastive captioners are image-text foundation models. arXiv preprint arXiv:2205.01917 (2022) Title Suppressed Due to Excessive Length 11

    2022 arXiv

  18. [26]

    In: Proceedings of the IEEE/CVF international conference on computer vision

    Zhai, X., Mustafa, B., Kolesnikov, A., Beyer, L.: Sigmoid loss for language im- age pre-training. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 11975–11986 (2023)

    2023

  19. [27]

    arXiv preprint arXiv:2303.00915 (2023)

    Zhang, S., Xu, Y., Usuyama, N., Xu, H., Bagga, J., Tinn, R., Preston, S., Rao, R., Wei, M., Valluri, N., et al.: Biomedclip: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs. arXiv preprint arXiv:2303.00915 (2023)

    2023 arXiv

  20. [28]

    In: European Conference on Computer Vision

    Zheng, K., Zhang, Y., Wu, W., Lu, F., Ma, S., Jin, X., Chen, W., Shen, Y.: Dream- lip: Language-image pre-training with long captions. In: European Conference on Computer Vision. pp. 73–90. Springer (2024)

    2024

  21. [29]

    arXiv preprint arXiv:2405.18004 (2024)

    Zhou, J., Sun, L., Xu, Y., Liu, W., Afvari, S., Han, Z., Song, J., Ji, Y., He, X., Gao, X.: Skincap: A multi-modal dermatology dataset annotated with rich medical captions. arXiv preprint arXiv:2405.18004 (2024)

    2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.