Pith. sign in

REVIEW 4 major objections 4 minor 46 references

MM-Retinal V2: Transfer an Elite Knowledge Spark into Fundus Vision-Language Pretraining

T0 review · 4 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read MM-Retinal V2 claims that pretraining on a few thousand elite fundus image-text pairs, injected into public categorical datasets, matches models trained on hundreds of thousands of private clinical pairs.

desk verdict Worth refereeing, but the authors must rule out pretraining/evaluation overlap before the headline claim can be taken at face value. read the letter →

arxiv 2501.15798 v1 pith:EB4N25U4 submitted 2025-01-27 cs.CV

Ruiqi Wu , Na Su , Chenran Zhang , Tengfei Ma , Tao Zhou , Zhiting Cui , Nianfeng Tang , Tianyu Mao
show 5 more authors
Yi Zhou Wen Fan Tianxing Wu Shenqi Jing Huazhu Fu
This is my paper · ORCID
classification cs.CV
keywords fundusimageanalysisvision-languagepretrainingknowledgeinjectionmulti-modalitydatasetCFPFFAOCTzero-shotclassificationcontrastivelearningvectorquantization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tackles a data bottleneck in fundus vision-language pretraining: expert image-text pairs are scarce, while large public datasets only carry category labels. The authors build MM-Retinal V2, a public dataset of about 17,000 high-quality image-text pairs across CFP, FFA, and OCT, spanning over 96 diseases. They then train KeepFIT V2, which transfers these elite captions into the public categorical data through a hybrid knowledge-injection scheme, so the model learns from far fewer image-text pairs than prior private-data models. The claim, backed by zero-shot, few-shot, and linear-probing comparisons, is that this small elite spark yields performance competitive with models pretrained on 190K to 450K private image-text pairs. If true, it would mean public categorical data plus a modest amount of expert captions can substitute for large private clinical text collections.

What carries the argument

The load-bearing mechanism is the hybrid image-text knowledge injection module. A semantics-oriented branch uses multi-head cross-attention, with public image features as query and MM-Retinal V2 image features as key, to reweight MM-Retinal V2 text features as expert knowledge; an appearance-oriented branch repeats this over discrete tokens produced by index backpropagation quantization (IBQ), a vector-quantization scheme that turns visual feature maps into codebook tokens preserving low-level lesion appearance. Two expert-knowledge refinement losses (MSE between retrieved expert text and public text features) then pull the public dataset's template text toward the expert captions, so the categorical data effectively learns from the elite captions during contrastive pretraining. A preliminary masked-language-modeling pass on the MM-Retinal-Text corpus adapts the text encoder to ophthalmic terminology before any vision-language training.

What would settle it

Measure per-category image-retrieval precision between MM-Retinal V2 and each public pretraining dataset using the model's own cross-attention scores, then on low-precision categories keep the injection but swap in random expert captions; if performance does not drop, the refinement loss is not transferring the claimed ophthalmic knowledge.

Watch

Extended reading notes

Core claim

The central claim is that knowledge from a small, high-quality image-text dataset can be distilled into a vision-language model through hybrid injection of global semantic and local appearance information, to the point that downstream performance matches models trained on over an order of magnitude more private data. KeepFIT V2 first pretrains its text encoder on ophthalmic text via masked language modeling, then for each public categorical image retrieves expert captions from MM-Retinal V2 based on both contrastive semantic features and vector-quantized appearance tokens, and refines the public text features toward the retrieved expert text with MSE losses. Trained jointly on MM-Retinal V2 and public categorical datasets, the model achieves top results on several CFP benchmarks and strong gains on FFA and OCT, despite using roughly 1% of the image-text pairs used by RET-CLIP, ViLRef, or RetiZero.

Load-bearing premise

The method assumes that MM-Retinal V2 images are visually similar to the public categorical images and cover nearly all the common retinal disease categories; if that overlap is weak, the retrieval will pull in irrelevant expert captions and the refinement loss will push the public text features toward unhelpful targets.

Editorial extensions

If this is right

  • KeepFIT V2 attains the best zero-shot average on REFUGE among all compared models and competitive or better few-shot and linear-probing results, despite using about 1% of the image-text pairs used by RET-CLIP, ViLRef, or RetiZero.
  • The hybrid semantic-plus-appearance injection outperforms either branch alone in the CFP ablation, so both global and local matching contribute to the knowledge transfer.
  • The same pretraining recipe transfers to FFA and OCT, with the largest relative gains appearing on FFA benchmarks such as MPOS.
  • Because the scheme relies on public categorical data rather than private clinical text, it offers a reproducible route for other medical imaging domains that face the same scarcity of image-text pairs.
  • The public release of MM-Retinal V2 and the pretrained weights gives the community a reproducible alternative to the closed, large-scale private corpora used by competing models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable corollary, not stated in the paper: the gain from knowledge injection should scale with how well MM-Retinal V2's category coverage and visual appearance overlap the public pretraining set; on categories absent from the elite set, injection could even hurt.
  • The appearance-oriented branch may be doing more than adding detail: quantized token matching could act as a regularizer that prevents the contrastive branch from collapsing onto template text, an effect that could be isolated by ablating the codebook with random assignment.
  • One could extend the recipe to other scarce-text medical modalities, such as chest X-ray or dermoscopy, whenever a small corpus of diagram-book style expert captions exists; the paper argues this generality but does not demonstrate it.
  • Another implicit implication is that if a few thousand expert captions suffice, the marginal value of ever-larger private image-text collections may be lower than recent scaling efforts assume, at least for disease categories well covered by public data.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper presents MM-Retinal V2, a public fundus image-text dataset covering CFP, FFA, and OCT modalities with roughly 5K pairs per modality plus a 452K-utterance ophthalmic text corpus, and proposes KeepFIT V2, a CLIP-style vision-language pretraining model. KeepFIT V2 is pretrained on categorical public datasets together with the small elite image-text dataset. Knowledge transfer is implemented by a semantically oriented cross-attention module and an appearance-oriented VQ-based cross-attention module, whose outputs refine the public-dataset text features through MSE losses (Eqs. 20-21). The authors evaluate zero-shot, few-shot, and linear probing performance across CFP, FFA, and OCT, reporting that KeepFIT V2 is competitive with fundus VLP models trained on hundreds of thousands of private image-text pairs.

Significance. If the claims hold, the work is practically significant: it suggests that a few thousand carefully curated public image-text pairs plus large categorical public datasets can substitute for large private clinical image-text corpora in fundus VLP, and it releases both data and model. The paper's strengths include the breadth of the evaluation (three modalities and three settings), the release of the MM-Retinal V2 dataset, the ablation study in Table VII, and the explicit goal of reducing dependence on private data. The central claim, however, rests on two assumptions that are not yet substantiated: that the pretraining and evaluation data do not overlap, and that MM-Retinal V2 is visually and categorically close enough to the public data for the knowledge-injection retrieval to be relevant.

major comments (4)
  1. [Section V-A1 and Section V-D1] The manuscript does not rule out overlap between the CFP pretraining data and the evaluation datasets. Section V-A1 states that pretraining uses FLAIR's public datasets (over 190K images) plus an additional 'over 80K data samples from public categorical retinal datasets' that are not itemized, while Section V-D1 evaluates on REFUGE, ODIR200×3, iChallenge-AMD, Retina, FIVES, and APTOS—public collections that are plausibly contained in FLAIR's 37-dataset aggregation or in the additional 80K samples. Because zero-shot classification here is computed from image-to-text similarity (Eqs. 2-3) and few-shot/linear probing use frozen features, memorized or near-duplicate pretraining images would directly inflate the reported parity with private-data models. Please provide an explicit exclusion statement, enumerate the additional 80K public categorical datasets, and include a near-duplicate/hash-based deduplication analysis between all pretraining images and each evaluation set.
  2. [Section V-A3, Tables V-VI] The eleven OCT public pretraining datasets are not enumerated. Section V-A3 says only that 'we also collect eleven OCT public datasets with only category labels for KeepFIT V2 pretraining, totaling over 181K images,' and the evaluation uses OCTID and OCTDL. Since OCTID and OCTDL are themselves public OCT datasets, it is unclear whether they were excluded from pretraining. Please list all eleven pretraining datasets and explicitly state that the two evaluation datasets (and their source collections) were excluded, or the OCT results cannot be attributed to generalization from the proposed method.
  3. [Section IV-C1, Eqs. (7), (18), (20)-(21)] The load-bearing premise that 'MM-Retinal V2 exhibits a high degree of similarity with the public ones and almost covers all the common retinal disease categories' is asserted but not quantitatively validated. The cross-attention retrieval in Eqs. (7) and (18) and the refinement losses in Eqs. (20)-(21) assume that the retrieved elite captions are semantically and visually relevant to the public images; if the similarity or category coverage is poor, the MSE losses will push public text features toward unhelpful targets. Please provide quantitative evidence for this assumption, for example nearest-neighbor retrieval precision against category labels, a per-disease coverage table comparing MM-Retinal V2 with each public pretraining dataset, or distributions of image-similarity scores. Without such validation, the mechanism's contribution to the reported gains is not established.
  4. [Section V-C, Tables II, IV, VI] The manuscript states that all metrics are averaged across five cross-validation folds, but no standard deviations, confidence intervals, or significance tests are reported anywhere in the tables. Several headline comparisons involve small margins (for example, Table II REFUGE Tip-adapter 10-shot ACC of 91.4 for KeepFIT V2 versus 94.4 for KeepFIT V2L, and Table VI OCTDL few-shot differences around 1-2 points). Because the central claim is about competitiveness with private-data models, please report variance across folds and, where possible, significance tests or paired comparisons for the key zero-shot and few-shot results.
minor comments (4)
  1. [Section V-B] There is a typo: 'KeppFIT V L' should read 'KeepFIT V2L'.
  2. [Section II-A and Section IV] The word 'pertaining' appears to be a typo for 'pretraining' in 'restricting their application to foundational pertaining' and 'general vision-language pertaining'.
  3. [Tables I-VI] The model name is written inconsistently as 'KeepFITV1', 'KeepFITV2', and 'KeepFIT V2L'; please standardize the spacing for readability.
  4. [Section V-A3] For reproducibility, please provide the exact names and licenses of the eleven OCT public pretraining datasets and the additional CFP/FFA public categorical datasets, not only the aggregate image counts.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: KeepFIT V2's knowledge injection is an image-similarity distillation into public categorical texts, evaluation is on held-out public datasets, and the self-citations to MM-Retinal V1 are baseline references rather than load-bearing evidence.

full rationale

I find no circular step in the claimed derivation chain. The central claim is that KeepFIT V2, pretrained on roughly 5K elite image-text pairs per modality plus public categorical data, is competitive with models trained on hundreds of thousands of private image-text pairs. This claim is not defined in terms of its inputs. The hybrid knowledge injection (Eqs. 5-19) retrieves MM-Retinal V2 text by image-to-image similarity with public images and then refines the public text features through Eqs. 20-21; this is a distillation mechanism, not a fitting of the downstream evaluation labels. The downstream evaluations use separate public datasets (REFUGE, ODIR200×3, Retina, iChallenge-AMD, FIVES, APTOS, MPOS, AngioReport, OCTID, OCTDL), and the claimed competitiveness is measured against external private-data models rather than being read off from the pretraining objective. The self-citations to MM-Retinal V1 [40] are used for baseline comparison, architecture continuity, and dataset expansion; they are not used to justify the central new result. The paper's weakest assumption—that MM-Retinal V2 visually overlaps and categorically covers the public datasets—is an unvalidated premise, and the lack of an explicit statement excluding evaluation datasets from the FLAIR-derived 190K plus additional 80K public pretraining images is a data-leakage risk. However, neither issue makes the derivation circular: no equation reduces to the target result, and no fitted parameter is renamed as a prediction. Therefore the appropriate finding is no significant circularity, with a low score reflecting only minor self-citation and unresolved data-hygiene concerns.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The method introduces no new physical or conceptual entities. It relies on standard deep learning building blocks (CLIP, attention, VQ-IBQ) and on domain assumptions about the coverage and similarity between MM-Retinal V2 and public datasets. The only hand-chosen free parameters are the two loss weights, whose selection is not tied to a documented validation split.

free parameters (2)
  • loss weight λ1 = 100
    Set to achieve best performance in Eq. (22); no validation split documented.
  • loss weight λ2 = 1×10^4
    Set to achieve best performance in Eq. (22); no validation split documented.
assumptions (3)
  • domain assumption MM-Retinal V2 images are visually similar to public dataset images and almost cover all their disease categories.
    Section IV-C1 states this observation, which justifies using MM-Retinal V2 as the knowledge source for public datasets. If the category coverage or visual similarity is poor, the injected knowledge is irrelevant to public texts.
  • ad hoc to paper Pushing public text features toward MM-Retinal V2-derived text features via MSE refinement improves downstream vision-language alignment.
    The expert knowledge refinement losses (Eqs. 20-21) define the transfer mechanism. The assumption that this refinement direction is beneficial is a design choice validated only by ablations without statistical tests.
  • domain assumption The IBQ visual tokenizer trained on the same data produces appearance tokens that capture lesion-level details useful for cross-modal retrieval.
    The appearance-oriented module relies on a frozen codebook trained separately on the same images; the paper assumes these tokens capture lesion appearance, but does not independently validate the tokenizer.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MM-Retinal V2: Transfer an Elite Knowledge Spark into Fundus Vision-Language Pretraining." pith.science (2026). https://pith.science/paper/EB4N25U4

@misc{pith2026250115798,
  author       = {Pith},
  title        = {Pith review of: MM-Retinal V2: Transfer an Elite Knowledge Spark into Fundus Vision-Language Pretraining},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EB4N25U4}},
  note         = {Machine review of arXiv:2501.15798}
}
read the original abstract

Vision-language pretraining (VLP) has been investigated to generalize across diverse downstream tasks for fundus image analysis. Although recent methods showcase promising achievements, they significantly rely on large-scale private image-text data but pay less attention to the pretraining manner, which limits their further advancements. In this work, we introduce MM-Retinal V2, a high-quality image-text paired dataset comprising CFP, FFA, and OCT image modalities. Then, we propose a novel fundus vision-language pretraining model, namely KeepFIT V2, which is pretrained by integrating knowledge from the elite data spark into categorical public datasets. Specifically, a preliminary textual pretraining is adopted to equip the text encoder with primarily ophthalmic textual knowledge. Moreover, a hybrid image-text knowledge injection module is designed for knowledge transfer, which is essentially based on a combination of global semantic concepts from contrastive learning and local appearance details from generative learning. Extensive experiments across zero-shot, few-shot, and linear probing settings highlight the generalization and transferability of KeepFIT V2, delivering performance competitive to state-of-the-art fundus VLP models trained on large-scale private image-text datasets. Our dataset and model are publicly available via https://github.com/lxirich/MM-Retinal.

Figures

Figures reproduced from arXiv: 2501.15798 by the authors.

Figure 1
Figure 1. The construction pipeline of MM-Retinal V2 and randomly selected examples from MM-Retinal V2. For CFP and FFA modalities, we propose a semi [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Statistical overview of MM-Retinal V2. (a) highlights a part of the [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The architecture of the proposed KeepFIT V2. KeepFIT V2 generally complies with the vision-language pretraining paradigm and introduces four [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Linear probing classification performance in CFP modality across [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Linear probing classification performance in FFA and OCT modalities [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 34 canonical work pages

  1. [1]

    Retina dataset, https://www.kaggle.com/datasets/jr2ngb/cataractdataset/ data

    work page

  2. [2]

    Aptos 2019 blindness detection (2019), https://www.kaggle.com/c/ aptos2019-blindness-detection

    work page 2019

  3. [3]

    arXiv preprint arXiv:1904.03323 (2019)

    Alsentzer, E., Murphy, J.R., Boag, W., Weng, W.H., Jin, D., Naumann, T., McDermott, M.: Publicly available clinical bert embeddings. arXiv preprint arXiv:1904.03323 (2019)

    arXiv 2019

  4. [4]

    In: European conference on computer vision

    Boecking, B., Usuyama, N., Bannur, S., Castro, D.C., Schwaighofer, A., Hyland, S., Wetscherek, M., Naumann, T., Nori, A., Alvarez-Valle, J., et al.: Making the most of text semantics to improve biomedical vision– language processing. In: European conference on computer vision. pp. 1–21. Springer (2022)

    work page 2022

  5. [5]

    IEEE Transactions on Neural Networks and Learning Systems (2023)

    Chen, D., Wu, Z., Liu, F., Yang, Z., Zheng, S., Tan, Y ., Zhou, E.: Protoclip: Prototypical contrastive language image pretraining. IEEE Transactions on Neural Networks and Learning Systems (2023)

    work page 2023

  6. [6]

    Image Analysis & Stereology pp

    Decenci `ere, E., Zhang, X., Cazuguel, G., Lay, B., Cochener, B., Trone, C., Gain, P., Ord ´o˜nez-Varela, J.R., Massin, P., Erginay, A., et al.: Feed- back on a publicly distributed image database: the messidor database. Image Analysis & Stereology pp. 231–234 (2014)

    work page 2014

  7. [7]

    In: International Conference on Medical Image Computing and Computer-Assisted Intervention

    Du, J., Guo, J., Zhang, W., Yang, S., Liu, H., Li, H., Wang, N.: Ret-clip: A retinal image foundation model pre-trained with clinical diagnostic reports. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 709–719. Springer (2024)

    work page 2024

  8. [8]

    Eslami, S., Meinel, C., De Melo, G.: Pubmedclip: How much does clip benefit visual question answering in the medical domain? In: Findings of the Association for Computational Linguistics: EACL 2023. pp. 1181– 1193 (2023)

    work page 2023

Show all 46 references
  1. [9]

    https://doi.org/10.21227/dt4f-rt59, https://dx.doi.org/10.21227/dt4f-rt59

    Fu, H., Li, F., Orlando, J.I., Bogunovi ´c, H., Sun, X., Liao, J., Xu, Y ., Zhang, S., Zhang, X.: Adam: Automatic detection challenge on age- related macular degeneration (2020). https://doi.org/10.21227/dt4f-rt59, https://dx.doi.org/10.21227/dt4f-rt59

    2020 doi

  2. [10]

    International Journal of Computer Vision 132(2), 581–595 (2024)

    Gao, P., Geng, S., Zhang, R., Ma, T., Fang, R., Zhang, Y ., Li, H., Qiao, Y .: Clip-adapter: Better vision-language models with feature adapters. International Journal of Computer Vision 132(2), 581–595 (2024)

    2024

  3. [11]

    Computers & Electrical Engineering 81, 106532 (2020)

    Gholami, P., Roy, P., Parthasarathy, M.K., Lakshminarayanan, V .: Octid: Optical coherence tomography image database. Computers & Electrical Engineering 81, 106532 (2020)

    2020

  4. [12]

    In: Proceedings of the IEEE conference on computer vision and pattern recognition

    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770–778 (2016)

    2016

  5. [13]

    In: International Conference on Medical Image Computing and Computer-Assisted Intervention

    He, X., Zhou, Y ., Wang, B., Cui, S., Shao, L.: Dme-net: Diabetic macular edema grading by auxiliary task learning. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 788–796. Springer (2019)

    2019

  6. [14]

    In: Proceedings of the IEEE/CVF winter conference on applications of computer vision

    Huang, J.H., Yang, C.H.H., Liu, F., Tian, M., Liu, Y .C., Wu, T.W., Lin, I., Wang, K., Morikawa, H., Chang, H., et al.: Deepopht: medical report generation for retinal images via deep models and visual explanation. In: Proceedings of the IEEE/CVF winter conference on applicati...

    2021

  7. [15]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Huang, Z., Zeng, Z., Huang, Y ., Liu, B., Fu, D., Fu, J.: Seeing out of the box: End-to-end pre-training for vision-language representation learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 12976–12985 (2021)

    2021

  8. [16]

    Scientific data 9(1), 475 (2022)

    Jin, K., Huang, X., Zhou, J., Li, Y ., Yan, Y ., Sun, Y ., Zhang, Q., Wang, Y ., Ye, J.: Fives: A fundus image dataset for artificial intelligence based vessel segmentation. Scientific data 9(1), 475 (2022)

    2022

  9. [17]

    Scientific Data 11(1), 365 (2024)

    Kulyabin, M., Zhdanov, A., Nikiforova, A., Stepichev, A., Kuznetsova, A., Ronkin, M., Borisov, V ., Bogachev, A., Korotkich, S., Constable, P.A., et al.: Octdl: Optical coherence tomography dataset for image- based deep learning methods. Scientific Data 11(1), 365 (2024)

    2024

  10. [18]

    In: Forty-first International Conference on Machine Learning

    Lavoie, S., Kirichenko, P., Ibrahim, M., Assran, M., Wilson, A.G., Courville, A., Ballas, N.: Modeling caption diversity in contrastive vision-language pretraining. In: Forty-first International Conference on Machine Learning

  11. [19]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Li, H., Zhu, J., Jiang, X., Zhu, X., Li, H., Yuan, C., Wang, X., Qiao, Y ., Wang, X., Wang, W., et al.: Uni-perceiver v2: A generalist model for large-scale vision and vision-language tasks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition....

    2023

  12. [20]

    Advances in neural information processing systems 34, 9694–9705 (2021)

    Li, J., Selvaraju, R., Gotmare, A., Joty, S., Xiong, C., Hoi, S.C.H.: Align before fuse: Vision and language representation learning with momen- tum distillation. Advances in neural information processing systems 34, 9694–9705 (2021)

    2021

  13. [21]

    In: Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2) (2021)

    Li, M., Cai, W., Liu, R., Weng, Y ., Zhao, X., Wang, C., Chen, X., Liu, Z., Pan, C., Li, M., et al.: Ffa-ir: Towards an explainable and reliable medical report generation benchmark. In: Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Tr...

    2021

  14. [22]

    arXiv preprint arXiv:2408.02865 (2024)

    Li, Z., Song, D., Yang, Z., Wang, D., Li, F., Zhang, X., Kina- han, P.E., Qiao, Y .: Visionunite: A vision-language foundation model for ophthalmology enhanced with clinical knowledge. arXiv preprint arXiv:2408.02865 (2024)

    2024 arXiv

  15. [23]

    In: The Eleventh International Conference on Learning Representations (2022)

    Lu, J., Clark, C., Zellers, R., Mottaghi, R., Kembhavi, A.: Unified-io: A unified model for vision, language, and multi-modal tasks. In: The Eleventh International Conference on Learning Representations (2022)

    2022

  16. [24]

    Medical image analysis 59, 101570 (2020)

    Orlando, J.I., Fu, H., Breda, J.B., Van Keer, K., Bathula, D.R., Diaz- Pinto, A., Fang, R., Heng, P.A., Kim, J., Lee, J., et al.: Refuge challenge: A unified framework for evaluating automated methods for glaucoma assessment from fundus photographs. Medical image analysis 59, ...

    2020

  17. [25]

    Data 6(2), 14 (2021)

    Pachade, S., Porwal, P., Thulkar, D., Kokare, M., Deshmukh, G., Sahasrabuddhe, V ., Giancardo, L., Quellec, G., M ´eriaudeau, F.: Retinal fundus multi-disease image dataset (rfmid): a dataset for multi-disease detection research. Data 6(2), 14 (2021)

    2021

  18. [26]

    Pelka, O., Koitka, S., R ¨uckert, J., Nensa, F., Friedrich, C.M.: Radiology objects in context (roco): a multimodal image dataset. In: Intravascular Imaging and Computer Assisted Stenting and Large-Scale Annotation of Biomedical Data and Expert Label Synthesis: 7th Joint Inter...

    2018

  19. [27]

    IEEE Transactions on Medical Imaging 40(7), 1750–1762 (2021)

    Peng, Y ., Zhu, W., Chen, Z., Wang, M., Geng, L., Yu, K., Zhou, Y ., Wang, T., Xiang, D., Chen, F., et al.: Automatic staging for retinopathy of prematurity with deep feature fusion and ordinal classification strat- egy. IEEE Transactions on Medical Imaging 40(7), 1750–1762 (2021)

    2021

  20. [28]

    Medical image analysis 59, 101561 (2020)

    Porwal, P., Pachade, S., Kokare, M., Deshmukh, G., Son, J., Bae, W., Liu, L., Wang, J., Liu, X., Gao, L., et al.: Idrid: Diabetic retinopathy– segmentation and grading challenge. Medical image analysis 59, 101561 (2020)

    2020

  21. [29]

    arXiv preprint arXiv:2310.04992 (2023)

    Qiu, J., Wu, J., Wei, H., Shi, P., Zhang, M., Sun, Y ., Li, L., Liu, H., Liu, H., Hou, S., et al.: Visionfm: a multi-modal multi-task vision foundation model for generalist ophthalmic artificial intelligence. arXiv preprint arXiv:2310.04992 (2023)

    2023 arXiv

  22. [30]

    In: International conference on machine learning

    Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transfer- able visual models from natural language supervision. In: International conference on machine learning. pp. 8748–8763. PMLR (2021)

    2021

  23. [31]

    arXiv preprint arXiv:2409.06644 (2024)

    Shi, D., Zhang, W., Yang, J., Huang, S., Chen, X., Yusufu, M., Jin, K., Lin, S., Liu, S., Zhang, Q., et al.: Eyeclip: A visual-language foundation model for multi-modal ophthalmic image analysis. arXiv preprint arXiv:2409.06644 (2024)

    2024 arXiv

  24. [32]

    arXiv preprint arXiv:2412.02692 (2024)

    Shi, F., Luo, Z., Ge, Y ., Yang, Y ., Shan, Y ., Wang, L.: Taming scalable visual tokenizer for autoregressive image generation. arXiv preprint arXiv:2412.02692 (2024)

    2024 arXiv

  25. [33]

    arXiv preprint arXiv:2308.07898 (2023)

    Silva-Rodriguez, J., Chakor, H., Kobbi, R., Dolz, J., Ayed, I.B.: A foundation language-image model of the retina (flair): Encoding expert knowledge in text supervision. arXiv preprint arXiv:2308.07898 (2023)

    2023 arXiv

  26. [34]

    University, P.: Peking university international competition on ocular dis- ease intelligent recognition (2019), https://odir2019.grand-challenge.org/

    2019

  27. [35]

    Advances in neural information processing systems 30 (2017)

    Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017)

    2017

  28. [36]

    In: Proceedings of the 1st International Workshop on Multimedia Computing for Health and Medicine

    Wang, H., Xing, Z., Wu, W., Yang, Y ., Tang, Q., Zhang, M., Xu, Y ., Zhu, L.: Non-invasive to invasive: Enhancing ffa synthesis from cfp with a benchmark dataset and a novel network. In: Proceedings of the 1st International Workshop on Multimedia Computing for Health and Medic...

    2024

  29. [37]

    arXiv preprint arXiv:2406.09317 (2024) JOURNAL OF LATEX CLASS FILES, VOL

    Wang, M., Lin, T., Lin, A., Yu, K., Peng, Y ., Wang, L., Chen, C., Zou, K., Liang, H., Chen, M., et al.: Common and rare fundus diseases identification using vision-language foundation model with knowledge of over 400 diseases. arXiv preprint arXiv:2406.09317 (2024) JOURNAL OF...

    2024 arXiv

  30. [38]

    JAMA ophthalmology 138(2), 190– 198 (2020)

    Wang, M., Tichelaar, J., Pasquale, L.R., Shen, L.Q., Boland, M.V ., Wellik, S.R., De Moraes, C.G., Myers, J.S., Ramulu, P., Kwon, M., et al.: Characterization of central visual field loss in end-stage glaucoma by unsupervised artificial intelligence. JAMA ophthalmology 138(2),...

    2020

  31. [39]

    arXiv preprint arXiv:2403.10823 (2024)

    Wei, H., Liu, B., Zhang, M., Shi, P., Yuan, W.: Visionclip: An med-aigc based ethical language-image foundation model for generalizable retina image analysis. arXiv preprint arXiv:2403.10823 (2024)

    2024 arXiv

  32. [40]

    In: International Conference on Medical Image Computing and Computer-Assisted Intervention

    Wu, R., Zhang, C., Zhang, J., Zhou, Y ., Zhou, T., Fu, H.: Mm-retinal: Knowledge-enhanced foundational pretraining with fundus image-text expertise. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 722–732. Springer (2024)

    2024

  33. [41]

    arXiv e-prints pp

    Yang, S., Du, J., Guo, J., Zhang, W., Liu, H., Li, H., Wang, N.: Vilref: A chinese vision-language retinal foundation model. arXiv e-prints pp. arXiv–2408 (2024)

    2024

  34. [42]

    In: European Conference on Computer Vision

    Zhang, R., Zhang, W., Fang, R., Gao, P., Li, K., Dai, J., Qiao, Y ., Li, H.: Tip-adapter: Training-free adaption of clip for few-shot classification. In: European Conference on Computer Vision. pp. 493–510. Springer (2022)

    2022

  35. [43]

    arXiv preprint arXiv:2303.00915 (2023)

    Zhang, S., Xu, Y ., Usuyama, N., Xu, H., Bagga, J., Tinn, R., Preston, S., Rao, R., Wei, M., Valluri, N., et al.: Biomedclip: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs. arXiv preprint arXiv:2303.00915 (2023)

    2023 arXiv

  36. [44]

    medRxiv pp

    Zhang, W., Chotcomwongse, P., Chen, X., Chung, F.H., Song, F., Zhang, X., He, M., Shi, D., Ruamviboonsuk, P.: Angiographic report generation for the 3rd aptos’s competition: Dataset and baseline methods. medRxiv pp. 2023–11 (2023)

    2023

  37. [45]

    IEEE Transactions on Neural Networks and Learning Systems (2024)

    Zhang, X., Xiao, Z., Wu, X., Chen, Y ., Zhao, J., Hu, Y ., Liu, J.: Pyramid pixel context adaption network for medical image classification with supervised contrastive learning. IEEE Transactions on Neural Networks and Learning Systems (2024)

    2024

  38. [46]

    Nature 622(7981), 156–163 (2023)

    Zhou, Y ., Chia, M.A., Wagner, S.K., Ayhan, M.S., Williamson, D.J., Struyven, R.R., Liu, T., Xu, M., Lozano, M.G., Woodward-Court, P., et al.: A foundation model for generalizable disease detection from retinal images. Nature 622(7981), 156–163 (2023)

    2023

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.