MaskedCLIP: Bridging the Masked and CLIP Space for Semi-Supervised Medical Vision-Language Pre-training

Jun Zhou; Lei Zhu; Rick Siow Mong Goh; Yong Liu

REVIEW 3 major objections 5 minor 41 references

MaskedCLIP: Bridging the Masked and CLIP Space for Semi-Supervised Medical Vision-Language Pre-training

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Semi-supervised pre-training that merges masked reconstruction and CLIP alignment beats retinal foundation models.

desk verdict A plausible semi-supervised mix of MAE and CLIP for retinal imaging with consistent gains, but the key ablation conflates added capacity with the claimed feature-bridging mechanism. read the letter →

arxiv 2507.17239 v1 pith:6CKSOQSY submitted 2025-07-23 cs.CV

Lei Zhu , Jun Zhou , Rick Siow Mong Goh , Yong Liu This is my paper

classification cs.CV

keywords semi-supervisedvision-languagepre-trainingmaskedimagemodelingcontrastivelanguage-imageretinalanalysismedicalfoundationmodelknowledgedistillationlabelefficiency

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces semi-supervised vision-language pre-training, in which a medical foundation model is trained on both paired image-text data and unpaired images at once. It proposes MaskedCLIP, which combines masked image modeling with CLIP-style contrastive language-image pre-training so that a single image encoder learns from both data types. The paper argues that the two objectives produce incompatible feature spaces, and that a bridge transformer plus a masked knowledge distillation loss lets each space support the other. On seven retinal image analysis tasks, MaskedCLIP reports higher average ROC and PRC than CLIP, MAE, DINOv2, RETFound, and FLAIR, with especially large gains when only 10% of downstream labels are available. The point of the work is that using all available images, paired or not, yields a more generalizable and label-efficient medical foundation model.

What carries the argument

Two mechanisms carry the argument: a bridge transformer and a masked knowledge distillation loss. The bridge transformer sits between the image encoder's masked-reconstruction branch and the CLIP contrastive head, mapping the general masked feature space into the more semantic CLIP feature space. A feature decoder predicts masked patch features in CLIP space from visible-patch latent features, and a momentum encoder supplies the distillation targets. Their joint role is to let CLIP benefit from unpaired images while letting masked reconstruction benefit from semantic supervision, without forcing the two spaces into one incompatible shared space.

What would settle it

Hold total trainable parameters fixed across two runs—MAE+CLIP with a shared encoder, and MAE+CLIP plus the bridge transformer implemented by shrinking the image encoder—and compare average 10%-label ROC. If the gap disappears, the paper's feature-incompatibility explanation is refuted; if the gap persists, it is confirmed.

Watch

Extended reading notes

Core claim

The central claim is that one pre-training objective can jointly supervise an image encoder with both contrastive language-image alignment and masked reconstruction, provided the two feature spaces are explicitly bridged rather than naively shared. On the paper's retinal datasets, MaskedCLIP's average ROC across seven tasks is 82.1 with 10% training labels and 89.0 with the full training data, beating the strongest prior foundation models (RETFound at 78.0 and 87.2 respectively). The mutually interactive design—CLIP-style semantic features refined by general masked features, and masked features guided by CLIP semantic knowledge through distillation—is what the paper credits for the gains.

Load-bearing premise

The method assumes that masked-image modeling and CLIP create fundamentally incompatible feature spaces when forced to share one image encoder, and that the bridge transformer cures this incompatibility rather than simply adding model capacity.

Editorial extensions

If this is right

Pre-training on both paired and unpaired images produces better downstream medical image classifiers than pre-training on either data type alone.
The pre-trained encoder is more label-efficient: with 10% training labels, MaskedCLIP's average ROC is 82.1, a 4.1-point gain over the best previous model RETFound.
Because categorical labels can be converted into text prompts, the framework can absorb paired image-label datasets that lack free-text descriptions.
Unpaired images can be funneled into a vision-language model without needing captions, which matters for medical domains where paired reports are scarce.
The reported gains span diabetic retinopathy grading, glaucoma detection, and multi-disease diagnosis, suggesting the benefit is not task-specific.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A controlled comparison that holds total parameters fixed between MAE+CLIP and MAE+CLIP+Bridge would tell whether the bridge's small 0.8-point average ROC gain comes from resolving feature incompatibility or from added capacity.
The same semi-supervised recipe could be tested on other medical modalities with paired reports and unpaired scans, such as chest X-rays and pathology slides, where unpaired images vastly outnumber paired examples.
If the incompatibility story is right, its effect should grow as the semantic gap between unpaired and paired data widens; training on strongly domain-shifted unpaired images would be a direct test.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Punchline: the paper delivers a workable recipe for mixing paired image-text and unpaired image data in medical pre-training, with consistent gains across seven retinal tasks. The catch is that the ablation meant to isolate the bridge transformer actually adds a 4-block transformer and a feature decoder, so the central mechanism is not pinned down.

What's new: the task framing—semi-supervised vision-language pre-training—is a real gap, and the loss combination (MAE reconstruction, label-guided CLIP, masked feature distillation) is clearly specified. The experimental effort is substantial: 348K images, seven downstream tasks, label-efficiency curves, and comparisons against RETFound, FLAIR, MAE, DINOv2, and CLIP. On 10% labels the method beats the next best by roughly 5 points average ROC; at 100% it is competitive or better. That is an honest contribution to medical foundation models.

Soft spots: the main one is the one I opened with. Table 2 goes from MAE+CLIP (78.2 avg ROC) to +Bridge Transformer (79.0), but the bridge adds parameters, inserts a new pathway into the contrastive branch, and brings an EMA and feature decoder. The improvement is consistent, but you cannot attribute it to 'resolving feature incompatibility' without a capacity-matched control. The masked distillation row adds yet another change, so the full gain is not decomposed. Also missing: error bars or significance tests, code/data release, and citation of prior MIM-CLIP distillation work (EVA, BEiT-v2). The claims of 'significantly' better are not statistically supported, and the premise that MIM and CLIP feature spaces are fundamentally incompatible is asserted rather than demonstrated.

These are fixable. The core direction—jointly using both data types with distillation—is plausible and supported by the aggregate trend. The paper would be stronger with a controlled ablation (equal-capacity bridge) and per-task variance reporting.

Who it's for: groups building retinal or medical foundation models, or anyone thinking about semi-supervised vision-language pre-training. It deserves a serious referee; the empirical pattern is strong enough that the mechanism question should be resolved in revision rather than by desk rejection.

Referee Report

3 major / 5 minor

Summary. The paper introduces MaskedCLIP, a semi-supervised vision-language pre-training framework that combines masked image modeling (MAE) and CLIP-style contrastive learning using a shared image encoder, a bridge transformer, and a masked knowledge distillation loss. Paired image-text data and unpaired image data are jointly used for pre-training, and the resulting ViT-Large encoder is evaluated by fine-tuning on seven retinal image analysis tasks under 10% and 100% label settings. The paper reports that MaskedCLIP outperforms or matches CLIP, MAE, DINOv2, FLAIR, RETFound, and ImageNet21K baselines on average ROC/PRC, and includes an ablation study in Table 2 isolating the contributions of the bridge transformer and the masked knowledge distillation loss.

Significance. If the central mechanism is validated, the paper addresses a practical and under-explored problem: leveraging both paired and unpaired medical image data in one pre-training pipeline. The strengths include a clearly specified objective in Eqs. (1)-(6), a large assembled pre-training corpus of 348,481 fundus images, comparison against strong external foundation models (FLAIR and RETFound), and a label-efficiency analysis. The claim that a bridge transformer resolves a feature-space incompatibility between masked and CLIP features is, however, the load-bearing contribution, and the evidence for that specific claim is currently insufficient because the ablation that should isolate the bridge transformer also changes model capacity and the contrastive pathway. The paper would be strengthened substantially by a capacity-matched control and by uncertainty quantification for the reported averages.

major comments (3)

[Section 2.1 and Table 2] The central claim that masked and CLIP feature spaces are fundamentally incompatible and that the bridge transformer resolves this incompatibility is not actually isolated by the provided ablation. The comparison between 'MAE+CLIP' and '+Bridge Transformer' in Table 2 simultaneously (a) adds a 4-block transformer, (b) inserts a feature transformation into the contrastive path, and (c) appears to introduce the feature decoder and momentum-encoder machinery used by the subsequent MaskedCLIP row. The observed improvement from 78.2 to 79.0 average ROC could therefore be explained by added model capacity or by the architectural change alone, rather than by 'resolving feature incompatibility.' I request a capacity-matched control: for example, a same-size transformer inserted in parallel or in a bypass configuration, or a variant where the bridge is present but ablated in a way that keeps parameters and the contrastive input pathway unchanged. A feature-space diagnostic (e.g., CKA between masked and CLIP features before and after bridging) would also directly test the asserted incompatibility.
[Section 3, Comparison with SoTA Methods] The claim that MaskedCLIP 'significantly' outperforms baselines is not supported by statistical evidence. Table 1 and Fig. 2 report point estimates only, with no error bars, no standard deviations, no confidence intervals, and no significance tests. Because several per-task differences are small (e.g., APTOS 94.8 vs 95.0 ROC in the 100% setting, Retina 72.8 vs 68.1 PRC in the 10% setting), the average improvements may not be robust across seeds. I request multiple random seeds (at least three) for the main comparisons and ablations, or explicit reporting of variance and significance tests, especially for the label-efficiency claims in Fig. 2.
[Section 2.2, Eq. (6)] The overall objective in Eq. (6) is written as 'Lmin' but the masked image modeling loss is defined as 'Lmim' in Eq. (1). This typo affects a core equation and should be corrected. More substantively, the two balancing weights are stated to be 'empirically tuned based on their magnitudes' and set to 0.01; the paper should report the sensitivity of the final results to these weights, since the contribution of the CLIP loss and the distillation loss depends directly on them.

minor comments (5)

[Section 2.1, Eqs. (2)-(3)] The text states that a lightweight projection head is used after the bridge transformer and the text encoder, but the equations write the similarity as B(EI(x))^T ET(t) without showing the projection heads. Please make the notation consistent.
[Section 2.1, Discussion] 'Additional, for paired image-label data' should read 'Additionally.'
[Section 1] The phrase 'a principally designed framework' should be 'a principled design' or 'a principled framework.'
[Introduction] The claim that a naive shared-encoder combination 'will result in poor performance' is stated without a reference or supporting experiment; this is related to major comment 1 and should be backed by either a diagnostic or a controlled ablation.
[Tables] Tables 1 and 2 would be easier to read if average ROC/PRC values were accompanied by standard deviations or if statistically significant differences from the second-best method were marked explicitly.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the method is evaluated against external baselines and the components are drawn from prior literature, so the central claim is self-contained.

full rationale

MaskedCLIP's central claim is an empirical result: combining masked image modeling and contrastive language-image pre-training with a bridge transformer and masked knowledge distillation outperforms existing methods on downstream retinal tasks. The derivation chain is not circular. The pre-training objectives in Eqs. (1)-(6) are defined from MAE-style reconstruction, CLIP-style contrastive learning, and a knowledge-distillation term; none of these definitions presupposes the downstream ROC/PRC results. The comparison in Table 1 is against external methods (CLIP, MAE, DINOv2, RETFound, FLAIR) and external foundation models, so the claimed improvement is not a renaming of an input or a fitted parameter renamed as a prediction. Hyperparameters such as the loss weights are set by loss magnitudes and stated directly (lambda values 0.01), not tuned on downstream labels. The only qualifier is that the ablation in Table 2 confounds the bridge transformer with added capacity and with a changed contrastive pathway, making the specific mechanistic attribution to 'feature incompatibility' under-supported; however, that is a question of experimental control and evidential strength, not a circular derivation. No load-bearing self-citation, imported uniqueness theorem, or ansatz-by-citation is present. Therefore the circularity score is 0.

Assumptions & free parameters 6 free parameters · 3 assumptions · 2 invented entities

The free parameters are training hyperparameters chosen by hand; none are fit to downstream labels. The axioms are domain assumptions about feature space compatibility and the quality of the assembled data. The bridge transformer and feature decoder are novel architectural components without independent external evidence.

free parameters (6)

lambda_lg_clip = 0.01
Balancing weight for label-guided contrastive loss, set after empirical tuning based on loss magnitudes (Section 2.3).
lambda_mfd = 0.01
Balancing weight for masked knowledge distillation loss, set with lambda_lg_clip (Section 2.3).
mask_ratio = 0.75
Proportion of image patches masked, taken from MAE [17]; chosen by hand.
EMA_momentum = 0.999
Exponential moving average decay for the momentum encoder, standard in MoCo/DINO-style methods.
bridge_and_feature_decoder_depth = 4 transformer blocks
Number of transformer blocks in the bridge transformer and feature decoder; a design choice.
pre_training_epochs = 200
Number of pre-training epochs; chosen to balance compute and performance.

assumptions (3)

domain assumption There exists a natural semantic hierarchical structure across the masked and CLIP feature spaces, and directly sharing an image encoder between the two learning paradigms causes feature incompatibility that harms performance.
This premise motivates the bridge transformer but is asserted in Section 2.1 without direct measurement; the ablation adds parameters, so the improvement does not isolate this effect.
domain assumption The paired image-text data constructed from ODIR-5K, AIROGS, and EYEPACS with expert-encoded text (following FLAIR [35]) is sufficient to teach semantic CLIP features that transfer to downstream tasks.
The text supervision relies on the FLAIR encoding scheme and the assumption that these three datasets capture relevant medical semantics.
domain assumption The momentum encoder EMA(B o E_I) provides a stable and meaningful target for masked feature distillation.
The EMA update is standard in self-supervised learning, but the interaction between the bridge transformer and image encoder co-adaptation is not analyzed.

invented entities (2)

Bridge transformer
purpose: Connects the masked feature space to the CLIP feature space, taking image encoder outputs and mapping them to a shared embedding for contrastive learning.
The bridge transformer is introduced in this paper; its effectiveness is only demonstrated through internal ablations, with no external falsifiable handle outside the paper's own experiments.
Feature decoder
purpose: Predicts masked image features from visible patches for the masked knowledge distillation loss.
Introduced in this paper; evaluated only through downstream task performance, not independently verified.

how reviews work

0 comments

Cite this review

Pith. "Pith review of MaskedCLIP: Bridging the Masked and CLIP Space for Semi-Supervised Medical Vision-Language Pre-training." pith.science (2026). https://pith.science/paper/6CKSOQSY

@misc{pith2026250717239,
  author       = {Pith},
  title        = {Pith review of: MaskedCLIP: Bridging the Masked and CLIP Space for Semi-Supervised Medical Vision-Language Pre-training},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6CKSOQSY}},
  note         = {Machine review of arXiv:2507.17239}
}

read the original abstract

Foundation models have recently gained tremendous popularity in medical image analysis. State-of-the-art methods leverage either paired image-text data via vision-language pre-training or unpaired image data via self-supervised pre-training to learn foundation models with generalizable image features to boost downstream task performance. However, learning foundation models exclusively on either paired or unpaired image data limits their ability to learn richer and more comprehensive image features. In this paper, we investigate a novel task termed semi-supervised vision-language pre-training, aiming to fully harness the potential of both paired and unpaired image data for foundation model learning. To this end, we propose MaskedCLIP, a synergistic masked image modeling and contrastive language-image pre-training framework for semi-supervised vision-language pre-training. The key challenge in combining paired and unpaired image data for learning a foundation model lies in the incompatible feature spaces derived from these two types of data. To address this issue, we propose to connect the masked feature space with the CLIP feature space with a bridge transformer. In this way, the more semantic specific CLIP features can benefit from the more general masked features for semantic feature extraction. We further propose a masked knowledge distillation loss to distill semantic knowledge of original image features in CLIP feature space back to the predicted masked image features in masked feature space. With this mutually interactive design, our framework effectively leverages both paired and unpaired image data to learn more generalizable image features for downstream tasks. Extensive experiments on retinal image analysis demonstrate the effectiveness and data efficiency of our method.

Figures

Figures reproduced from arXiv: 2507.17239 by the authors.

**Figure 1.** Architecture and dataflow of our proposed MaskedCLIP framework. Our framework consists of five modules, namely an image encoder, a bridge transformer, a text encoder, an image decoder, and a feature decoder to process both image and text data for synergistic masked image modeling and contrastive language-image pre-training. We employ the bridge transformer to connect the masked and CLIP feature space to resolve the… view at source ↗

**Figure 2.** Label efficiency analysis on exemplary downstream tasks. The X axis shows the training data proportion and the Y axis shows the ROC score. image data for foundation model learning, leading to significantly better performance. The experiment results highlight that rather than focusing exclusively on either paired or unpaired image data, a unified approach that leverages all available image data is key to develop mo… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

41 extracted references · 29 canonical work pages

[1]

PloS one13(11), e0207982 (2018)

Ahn, J.M., Kim, S., Ahn, K.S., Cho, S.H., Lee, K.B., Kim, U.S.: A deep learn- ing model for the detection of both advanced and early glaucoma using fundus photography. PloS one13(11), e0207982 (2018)

2018
[2]

Data in brief (2021)

Benítez, V.E.C., Matto, I.C., Román, J.C.M., Noguera, J.L.V., García-Torres, M., Ayala, J., Pinto-Roa, D.P., Gardel-Sotomayor, P.E., Facon, J., Grillo, S.A.: Dataset from fundus images for the study of diabetic retinopathy. Data in brief (2021)

work page 2021
[3]

arXiv preprint arXiv:2108.07258 (2021)

Bommasani, R., Hudson, D.A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., et al.: On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258 (2021)

arXiv 2021
[4]

International journal of biomedical imaging (2013)

Budai, A., Bock, R., Maier, A., Hornegger, J., Michelson, G.: Robust vessel seg- mentation in fundus images. International journal of biomedical imaging (2013)

work page 2013
[5]

Advances in neural information processing systems33, 9912–9924 (2020)

Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., Joulin, A.: Unsupervised learning of visual features by contrasting cluster assignments. Advances in neural information processing systems33, 9912–9924 (2020)

work page 2020
[6]

In: CVPR (2021)

Caron, M., Touvron, H., Misra, I., Jégou, H., Mairal, J., Bojanowski, P., Joulin, A.: Emerging properties in self-supervised vision transformers. In: CVPR (2021)

2021
[7]

Nature communications (2021)

Cen, L.P., Ji, J., Lin, J.W., Ju, S.T., Lin, H.J., Li, T.P., Wang, Y., Yang, J.F., Liu, Y.F., Tan, S., et al.: Automatic detection of 39 fundus diseases and conditions in retinal photographs using deep neural networks. Nature communications (2021)

work page 2021
[8]

In: ICML (2020)

Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for con- trastive learning of visual representations. In: ICML (2020)

2020

Show all 41 references

[9]

In: CVPR (2021)

Chen, X., Xie, S., He, K.: An empirical study of training self-supervised vision transformers. In: CVPR (2021)

2021
[10]

IEEE transactions on medical imaging (2023)

De Vente, C., Vermeer, K.A., Jaccard, et al.: Airogs: Artificial intelligence for robust glaucoma screening challenge. IEEE transactions on medical imaging (2023)

2023
[11]

Image Analysis & Stereology (2014)

Decencière, E., Zhang, X., Cazuguel, G., Lay, B., Cochener, B., Trone, C., Gain, P., Ordóñez-Varela, J.R., Massin, P., E.A., et al.: Feedback on a publicly distributed image database: the messidor database. Image Analysis & Stereology (2014)

2014
[12]

In: CVPR (2009)

Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: CVPR (2009)

2009
[13]

Biomedical engineering online18, 1–19 (2019)

Diaz-Pinto, A., Morales, S., Naranjo, V., Köhler, T., Mossi, J.M., Navea, A.: Cnns for automatic glaucoma assessment using fundus images: an extensive validation. Biomedical engineering online18, 1–19 (2019)

2019
[14]

arXiv preprint arXiv:2010.11929 (2020)

Dosovitskiy, A.: An image is worth 16x16 words: Transformers for image recogni- tion at scale. arXiv preprint arXiv:2010.11929 (2020)

2020 arXiv
[15]

EmilyAlsentzer:(2019),https://huggingface.co/emilyalsentzer/Bio_ClinicalBERT

2019
[16]

Hamamci, I.E., Er, S., Almas, F., Simsek, A.G., Esirgun, S.N., Dogan, I., Dasdelen, M.F.,Durugol,O.F.,Wittmann,B.,Amiranashvili,T.,etal.:Developinggeneralist foundation models from a multimodal dataset for 3d computed tomography (2024)

2024
[17]

In: CVPR (2022)

He, K., Chen, X., Xie, S., Li, Y., Dollár, P., Girshick, R.: Masked autoencoders are scalable vision learners. In: CVPR (2022)

2022
[18]

TMI (2000)

Hoover, A., Kouznetsova, V., Goldbaum, M.: Locating blood vessels in retinal images by piecewise threshold probing of a matched filter response. TMI (2000)

2000
[19]

Scientific Data (2023)

Huang, X., Kong, X., et al.: Grape: A multi-modal dataset of longitudinal follow-up visual field and fundus images for glaucoma management. Scientific Data (2023)

2023
[20]

Neurips (2024) MaskedCLIP for Semi-Supervised Medical Vision-Language Pre-training 11

Ikezogwo, W., Seyfioglu, S., et al.: Quilt-1m: One million image-text pairs for histopathology. Neurips (2024) MaskedCLIP for Semi-Supervised Medical Vision-Language Pre-training 11

2024
[21]

Scientific data 9(1), 475 (2022)

Jin, K., Huang, X., Zhou, J., Li, Y., Yan, Y., Sun, Y., Zhang, Q., Wang, Y., Ye, J.: Fives: A fundus image dataset for artificial intelligence based vessel segmentation. Scientific data 9(1), 475 (2022)

2022
[22]

Kaggle: (2015), https://www.kaggle.com/c/diabetic-retinopathy-detection

2015
[23]

Kaggle: (2019), https://www.kaggle.com/c/aptos2019-blindness-detection

2019
[24]

Kaggle: (2022), https://www.kaggle.com/datasets/jr2ngb/cataractdataset

2022
[25]

Scientific data10(1), 70 (2023)

Kumar, J.H., Seelamantula, C.S., Gagan, J., Kamath, Y.S., Kuzhuppilly, N.I., Vivekanand, U., Gupta, P., Patil, S.: Cháks.u: A glaucoma specific fundus image database. Scientific data10(1), 70 (2023)

2023
[26]

In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

Li, L., Xu, M., Wang, X., Jiang, L., Liu, H.: Attention based glaucoma detection: A large-scale database and cnn model. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 10571–10580 (2019)

2019
[27]

Scientific Data7(1), 409 (2020)

Lin, L., Li, M., Huang, Y., Cheng, P., Xia, H., Wang, K., Yuan, J., Tang, X.: The sustech-sysu dataset for automated exudate detection and diabetic retinopathy grading. Scientific Data7(1), 409 (2020)

2020
[28]

Medical image analysis (2017)

Litjens, G., Kooi, T., Bejnordi, B.E., et al.: A survey on deep learning in medical image analysis. Medical image analysis (2017)

2017
[29]

Patterns (2022)

Liu, R., Wang, X., Wu, Q., Dai, L., Fang, X., et al.: Deepdrid: Diabetic retinopa- thy—grading and image quality estimation challenge. Patterns (2022)

2022
[30]

ODIR 2019 Grand Challenge: (2019), https://odir2019.grand-challenge.org/

2019
[31]

arXiv preprint arXiv:2304.07193 (2023)

Oquab, M., Darcet, T., et al.: Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193 (2023)

2023 arXiv
[32]

Medical image analysis (2020)

Orlando, J.I., Fu, H., Breda, J.B., Van Keer, K., et al.: Refuge challenge: A unified frameworkforevaluatingautomatedmethodsforglaucomaassessmentfromfundus photographs. Medical image analysis (2020)

2020
[33]

Medical image analysis59, 101561 (2020)

Porwal, P., Pachade, S., Kokare, M., Deshmukh, G., Son, J., Bae, W., Liu, L., Wang, J., Liu, X., Gao, L., et al.: Idrid: Diabetic retinopathy–segmentation and grading challenge. Medical image analysis59, 101561 (2020)

2020
[34]

In: ICML (2021)

Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: ICML (2021)

2021
[35]

Medical Image Analysis99, 103357 (2025)

Silva-Rodriguez, J., Chakor, H., Kobbi, R., Dolz, J., Ayed, I.B.: A foundation language-image model of the retina (flair): Encoding expert knowledge in text supervision. Medical Image Analysis99, 103357 (2025)

2025
[36]

In: ISBI (2014)

Sivaswamy, J., Krishnadas, S., Joshi, G.D., Jain, M., Tabish, A.U.S.: Drishti-gs: Retinal image dataset for optic nerve head (onh) segmentation. In: ISBI (2014)

2014
[37]

arXiv preprint arXiv:2210.10163 (2022)

Wang, Z., Wu, Z., Agarwal, D., Sun, J.: Medclip: Contrastive learning from un- paired medical images and text. arXiv preprint arXiv:2210.10163 (2022)

2022 arXiv
[38]

In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

Yang, J., Li, C., Zhang, P., Xiao, B., Liu, C., Yuan, L., Gao, J.: Unified contrastive learning in image-text-label space. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 19163–19173 (2022)

2022
[39]

In: EMBC (2010)

Zhang, Z., Yin, F.S., et al.: Origa-light: An online retinal fundus image database for glaucoma analysis and research. In: EMBC (2010)

2010
[40]

arXiv preprint arXiv:2111.07832 (2021)

Zhou, J., Wei, C., Wang, H., Shen, W., Xie, C., Yuille, A., Kong, T.: ibot: Image bert pre-training with online tokenizer. arXiv preprint arXiv:2111.07832 (2021)

2021 arXiv
[41]

Nature (2023)

Zhou, Y., Chia, M.A., Wagner, S.K., Ayhan, M.S., Williamson, D.J., Struyven, R.R., Liu, T., Xu, M., Lozano, M.G., Woodward-Court, P., et al.: A foundation model for generalizable disease detection from retinal images. Nature (2023)

2023

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

PloS one13(11), e0207982 (2018)

Ahn, J.M., Kim, S., Ahn, K.S., Cho, S.H., Lee, K.B., Kim, U.S.: A deep learn- ing model for the detection of both advanced and early glaucoma using fundus photography. PloS one13(11), e0207982 (2018)

2018

[2] [2]

Data in brief (2021)

Benítez, V.E.C., Matto, I.C., Román, J.C.M., Noguera, J.L.V., García-Torres, M., Ayala, J., Pinto-Roa, D.P., Gardel-Sotomayor, P.E., Facon, J., Grillo, S.A.: Dataset from fundus images for the study of diabetic retinopathy. Data in brief (2021)

work page 2021

[3] [3]

arXiv preprint arXiv:2108.07258 (2021)

Bommasani, R., Hudson, D.A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., et al.: On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258 (2021)

arXiv 2021

[4] [4]

International journal of biomedical imaging (2013)

Budai, A., Bock, R., Maier, A., Hornegger, J., Michelson, G.: Robust vessel seg- mentation in fundus images. International journal of biomedical imaging (2013)

work page 2013

[5] [5]

Advances in neural information processing systems33, 9912–9924 (2020)

Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., Joulin, A.: Unsupervised learning of visual features by contrasting cluster assignments. Advances in neural information processing systems33, 9912–9924 (2020)

work page 2020

[6] [6]

In: CVPR (2021)

Caron, M., Touvron, H., Misra, I., Jégou, H., Mairal, J., Bojanowski, P., Joulin, A.: Emerging properties in self-supervised vision transformers. In: CVPR (2021)

2021

[7] [7]

Nature communications (2021)

Cen, L.P., Ji, J., Lin, J.W., Ju, S.T., Lin, H.J., Li, T.P., Wang, Y., Yang, J.F., Liu, Y.F., Tan, S., et al.: Automatic detection of 39 fundus diseases and conditions in retinal photographs using deep neural networks. Nature communications (2021)

work page 2021

[8] [8]

In: ICML (2020)

Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for con- trastive learning of visual representations. In: ICML (2020)

2020