The reviewed record of science sign in
Pith

arxiv: 2405.14137 · v2 · pith:EAVHU77G · submitted 2024-05-23 · cs.CV

RET-CLIP: A Retinal Image Foundation Model Pre-trained with Clinical Diagnostic Reports

Jiawei Du , Jia Guo , Weihang Zhang , Shengzhu Yang , Hanruo Liu , Huiqi Li , Ningli Wang This is my paper

Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel pith:EAVHU77Grecord.jsonopen to challenge →

classification cs.CV
keywords modelfoundationret-clipclinicaldemonstratediagnosticimagemultiple
0
0 comments X
read the original abstract

The Vision-Language Foundation model is increasingly investigated in the fields of computer vision and natural language processing, yet its exploration in ophthalmology and broader medical applications remains limited. The challenge is the lack of labeled data for the training of foundation model. To handle this issue, a CLIP-style retinal image foundation model is developed in this paper. Our foundation model, RET-CLIP, is specifically trained on a dataset of 193,865 patients to extract general features of color fundus photographs (CFPs), employing a tripartite optimization strategy to focus on left eye, right eye, and patient level to reflect real-world clinical scenarios. Extensive experiments demonstrate that RET-CLIP outperforms existing benchmarks across eight diverse datasets spanning four critical diagnostic categories: diabetic retinopathy, glaucoma, multiple disease diagnosis, and multi-label classification of multiple diseases, which demonstrate the performance and generality of our foundation model. The sourse code and pre-trained model are available at https://github.com/sStonemason/RET-CLIP.

This paper has not been read by Pith yet.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Bridging Restoration and Diagnosis: A Comprehensive Benchmark for Retinal Fundus Enhancement

    cs.CV 2026-04 unverdicted novelty 7.0

    EyeBench-V2 is a new benchmark that evaluates retinal fundus enhancement models using downstream clinical tasks, generalization tests, and structured expert assessments to measure real diagnostic utility.

  2. REVEAL: Multimodal Vision-Language Alignment of Retinal Morphometry and Clinical Risks for Incident AD and Dementia Prediction

    cs.CV 2026-04 unverdicted novelty 6.0

    REVEAL uses vision-language alignment of retinal morphometry and clinical risk narratives plus group contrastive learning to predict AD and dementia about 8 years early.

  3. REVEAL++: Differentiable Phenotypic Grouping for Vision-Language Retinal Modeling of Alzheimer's Disease Risk

    cs.AI 2026-06 unverdicted novelty 5.0

    REVEAL++ replaces discrete phenotypic groups with differentiable soft multi-positive weighting derived from intra-modality embeddings in contrastive learning, outperforming prior discrete and baseline methods on UK Bi...

  4. Cross-Modal Semantic-Enhanced Diffusion Framework for Diabetic Retinopathy Grading

    eess.IV 2026-05 unverdicted novelty 5.0

    CGSD framework reaches 87.5% accuracy and 0.731 macro F1 on APTOS 2019 by conditioning diffusion denoising on dot-product vectors from image features and DR-grade text descriptions.