A BERT-Style Self-Supervised Learning CNN for Disease Identification from Retinal Images

Amal Youssef; Oana M. Dumitrascu; Peijie Qiu; Wenhui Zhu; Xin Li; Yalin Wang

REVIEW 3 major objections 4 minor 31 references

A BERT-Style Self-Supervised Learning CNN for Disease Identification from Retinal Images

T0 review · 3 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A BERT-style masked-image pre-training scheme lets a lightweight CNN identify Parkinson's disease from retinal images with 99.38% accuracy, matching or beating transformer-based foundation models while using far less data.

desk verdict SparK-style pre-training on a lightweight CNN shows real gains on a disjoint retinal benchmark, but the headline AD/PD results are compromised by participant overlap between pre-training and test sets. read the letter →

arxiv 2504.18049 v1 pith:M6RA46GZ submitted 2025-04-25 cs.CV cs.AI

Xin Li , Wenhui Zhu , Peijie Qiu , Oana M. Dumitrascu , Amal Youssef , Yalin Wang This is my paper · ORCID

classification cs.CVcs.AI

keywords self-supervisedlearningpre-trainingconvolutionalneuralnetworksretinalfundusimagesmaskedimagemodelingsparseconvolutionBERT-stylediseaseidentification

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a lightweight convolutional network can receive the same kind of benefit from masked-image pre-training that transformer-based foundation models receive, without the heavy compute and data appetite. Its method randomly masks 60% of a retinal fundus image, reconstructs the hidden regions through a hierarchical encoder-decoder using sparse convolution, and pre-trains on 178,803 unlabeled images. After pre-training, the same CNN reaches 99.33% accuracy for Alzheimer's disease identification and 99.38% for Parkinson's disease, and improves grading of myopic maculopathy by 0.0232 in quadratic-weighted kappa over the non-pre-trained network. A sympathetic reader would care because it suggests that label scarcity in medical imaging can be addressed with modest computing resources, using CNNs that already dominate clinical image analysis.

What carries the argument

The load-bearing mechanism is sparse convolution in a hierarchical masked-image reconstruction pipeline. Sparse convolution computes the convolution operation only at unmasked, non-zero pixel positions, so masked regions cannot leak into feature computations; the paper extends this to self-supervised pre-training by randomly masking 60% of each image and feeding only visible pixels into the encoder. The encoder produces feature maps at five downsampled resolutions, a projection layer adapts their channels, and a lightweight UNet-style decoder with four upsampling blocks reconstructs the masked area, with the loss being mean squared error on the masked region only. This combination lets a MobileNetV2-derived CNN behave like a masked autoencoder while preserving the spatial hierarchy and localization that make CNNs attractive for medical images.

What would settle it

Pre-train the same CNN on 178,803 unlabeled images drawn only from participants who never appear in the AD or PD evaluation sets, fine-tune on the same labelled splits, and compare the resulting accuracy and kappa to the reported 99.38% PD accuracy and 0.9876 kappa. A substantial drop would show that patient overlap between pre-training and evaluation, not the pre-training method alone, drives the reported improvement.

Watch

Extended reading notes

Core claim

The central claim is that BERT-style self-supervised pre-training, implemented on a CNN through sparse convolution, significantly improves downstream disease identification from retinal images. The authors argue that prior masked-image modelling worked for transformers because transformers process variable-length patch sequences, whereas ordinary convolutions sweep over masked pixels and corrupt the signal; sparse convolution removes that obstruction by computing only over unmasked pixels. During pre-training, a random 60% mask is applied, hierarchical feature maps at five resolutions are built, and a lightweight UNet decoder reconstructs the masked regions under an MSE loss. After pre-training, the network matches or beats comparisons including a foundation model pre-trained on 1.6 million retinal images, using about one tenth of that data, and produces attention maps consistent with known retinal vessel changes in Alzheimer's disease.

Load-bearing premise

The load-bearing premise is that using unlabeled images from the same Alzheimer's and Parkinson's patients in pre-training does not inflate downstream accuracy; if the pre-training corpus overlaps with the evaluation cohort, some of the reported gains could come from having effectively seen those patients' images before, rather than from learning general retinal representations.

Editorial extensions

If this is right

If the central claim holds, masked-image pre-training is no longer tied to transformer architectures; lightweight CNNs can be pre-trained with the same BERT-style recipe and then fine-tuned on small labelled sets.
The reported gains on myopic maculopathy grading imply that the pre-training improves multi-class, multi-level disease classification, not just binary case-control tasks.
Because the pre-trained network reaches parity with a foundation model trained on roughly nine times more images, the method points toward a lower-data and lower-compute route to medical foundation-model-style performance.
The same architecture could be retrained and evaluated on other imaging modalities such as optical coherence tomography or magnetic resonance imaging, which the paper names as future work.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: a decisive control experiment would pre-train on a participant-disjoint set of unlabeled images and re-run the AD and PD fine-tuning; the paper does not report this, so its headline gains could overstate generalization if the pre-training set contains images from the same patients used for evaluation.
Editorial inference: the method's components are not intrinsically tied to this specific backbone, so the same sparse-convolution masking recipe could plausibly transfer to other efficient CNNs, such as ResNet or EfficientNet variants; testing that would show whether the gain is a property of the recipe or of the chosen backbone.
Editorial inference: because the reconstruction target is the original image's pixels, the pre-training also learns background and imaging artifacts; a version that reconstructs only anatomical structures, such as vessels, might learn representations that generalize better across acquisition devices.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

I read the paper with the stress-test note in hand, and I think the note lands. The paper does one useful thing and then weakens its own headline. The useful thing is the MMAC result: a SparK/ConvNeXt V2 style masked autoencoder applied to nn-MobileNet gives a genuinely disjoint benchmark gain—kappa +0.023, F1 +0.041, accuracy +0.021. That is modest but plausible, and it supports the claim that lightweight CNNs can get BERT-style pre-training benefits without ViT-scale data. The method section is honest about provenance; SparK and ConvNeXt V2 are cited plainly, and the novelty is framed as a new backbone and domain, which is fair.

The soft spot is the AD/PD evaluation, and it is load-bearing. The pre-training corpus is 178,803 UK Biobank images from 87,245 participants, and the AD/PD test sets come from the same UK Biobank cohort. The paper's defense—that labels are unused and that this is common practice—does not address the actual risk: the model has seen the same participants' eyes, both eyes, during pre-training. At 99.38% PD accuracy, even a small identity cue could explain the gap over the 98.76% baseline. The stress-test note is right that the correct fix is a disjoint-participant pre-training run. Also, there are no per-fold variances, so we cannot tell if the PD gain is noise.

Minor issues: the text says kappa improved by 0.232 but the table says 0.0232; the AD AUC in the text is 0.9997 while the table says 1.0; no code release. All fixable. The MMAC experiment should be replicated and reported with error bars.

Who is this for? People working on label-scarce medical imaging who want a lightweight CNN alternative to ViT foundation models. The MMAC result alone deserves a look. But the AD/PD headline should not be trusted until the overlap is resolved. I would not desk-reject; a serious referee should ask for the disjoint pre-training experiment and variance estimates. If the MMAC gains survive, the paper is a useful contribution even with the AD/PD claims trimmed.

Referee Report

3 major / 4 minor

Summary. The paper proposes a BERT-style masked-image self-supervised pre-training scheme for the lightweight CNN nn-MobileNet. The method masks 60% of each input, applies sparse convolution to process only visible pixels, and reconstructs masked regions using a hierarchical encoder-decoder with an MSE loss. The model is pre-trained on 178,803 unlabeled UK Biobank fundus images and then evaluated on Alzheimer's disease (AD), Parkinson's disease (PD), and MMAC myopic maculopathy classification. The authors report that pre-training improves PD accuracy and MMAC metrics, and that the pre-trained model is on par with or better than strong baselines including RETFound.

Significance. If the evaluation is valid, the paper makes a useful empirical contribution: it shows that a lightweight CNN can obtain masked-image pre-training benefits with roughly one-tenth of the data used by ViT-based foundation models, while retaining CNN locality and computational efficiency. The method itself is sensible and builds on existing SparK/ConvNeXt-v2 ideas, and the MMAC experiment provides a genuinely disjoint evaluation in which pre-training yields plausible gains (Kappa +0.0232, Weighted F1 +0.0412). The main limitation is that the headline AD/PD result is compromised by participant overlap between pre-training and downstream evaluation, and by the absence of variance estimates, so the central claim of 'significant improvement' is not yet established.

major comments (3)

[Section 3, Fig. 4, Table 1] The AD/PD evaluation is contaminated by the fact that the unlabeled pre-training corpus contains the same UK Biobank participants whose images are used downstream. The paragraph beginning 'Although we use the fundus images derived from patients with a diagnosis of AD and PD from the UK Biobank' argues that pre-training does not use labels and is consistent with common self-supervised practice, but this is not evidence that participant overlap does not inflate performance. Because the PD gain over the non-pre-trained nn-MobileNet baseline is only +0.0062 accuracy (+0.0124 kappa), overlap or near-duplicate images from the same participants could plausibly account for the gap. Please re-run pre-training with all AD/PD participants excluded from the pre-training corpus and compare downstream performance, or otherwise provide a direct test that overlap does not affect the reported gains.
[Table 1, Section 3] No error bars, per-fold results, or significance tests are reported for the AD/PD experiments, so the abstract's claim of 'significantly improves performance' is not statistically supported. On AD, the pre-trained model yields exactly the same metrics as nn-MobileNet (ACC 0.9933, AUC 1.0, Kappa 0.9866), so pre-training shows no measurable gain there; on PD, the difference between 0.9938 and 0.9876 could easily be within run-to-run noise given the small dataset (415 PD images). Please provide per-fold statistics with standard deviations and a paired significance test across folds.
[Section 3, Table 2] The MMAC experiment is the strongest evidence for the method, but it also reports only 5-fold means without variance or significance testing. The claims that Kappa, Accuracy, and F1 'significantly improved' should be accompanied by per-fold results or confidence intervals, especially with only 1143 images and four imbalanced classes. In addition, the text states that 'Kappa value improved by 0.232' and 'Accuracy increased by 0.0238', which disagree with Table 2's +0.0232 and +0.0208; these numbers should be reconciled.

minor comments (4)

[Section 3] The text after Table 1 states that the AD result has 'an AUC of 0.9997', but Table 1 reports AD AUC as 1.0; please correct the inconsistency.
[Section 3, paragraph 1] The description 'we randomly divided the data into training and validation sets in the ratio of 8:2. We employed 5-fold stratified cross-validation' is ambiguous; 5-fold cross-validation and a single 80/20 split are different procedures, so please clarify which evaluation protocol was used.
[Fig. 4] The figure shows both a 'Normal Control group' with 176,392 images / 86,069 participants and an 'NC group with GOOD quality images' with 389 images / 282 participants; it is unclear how many control subsets are used for AD versus PD, and whether the same control images are used in both tasks.
[Abstract and Section 4] The phrase 'significantly improve performance in the downstream tasks' is stronger than the evidence in Table 1, where AD metrics are unchanged by pre-training; please soften the claim or add supporting statistics.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation found: the SSL benefit claim is empirical, and the closest concern (pre-training/test participant overlap) is a data-leakage risk rather than a definitional reduction.

full rationale

The paper makes no formal derivation claim that would permit definitional circularity. The pre-training objective is image reconstruction (MSE between the masked and original regions), and downstream disease identification is a separate fine-tuned classification task; no downstream label, metric, or test-set quantity appears in the pre-training loss, and no fitted parameter is renamed as a prediction. The only load-bearing assumption is stated in Section 3: 'Although we use the fundus images derived from patients with a diagnosis of AD and PD from the UK Biobank, we argue that our pre-training 1) does not use the image labels, and 2) primarily focuses on learning mask regions to build the feature maps. Our practice is consistent with common self-supervised approaches [27].' This passage identifies a genuine evaluation-design threat: the same UK Biobank cohort contributes both unlabeled pre-training images and AD/PD downstream test images, so the reported gains could be inflated by participant-level leakage. That is an experimental validity concern, not a circularity of the kind this pass evaluates: the improvement is not equivalent to the model's input by construction, and no equation or fitted quantity reduces to the claimed result. The method also shows gains on the external, disjoint MMAC benchmark (kappa +0.0232, AUC +0.0039, accuracy +0.0208, weighted F1 +0.0412), which independently supports the central claim that BERT-style pre-training improves CNN performance. Citations to the authors' own nn-MobileNet [28] supply the backbone architecture but do not carry the self-supervised-learning claim, which is adopted from external works (SparK [21], ConvNeXt V2 [22], sparse convolution [10]). No self-citation chain forces the conclusion, and no uniqueness theorem or ansatz is smuggled in via prior work. Accordingly, the appropriate finding is no significant circularity.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claim rests on the choice of masking hyperparameters, the transferability of SparK to retinal images, and the comparability of the stated evaluation protocol; none of these are validated with ablations or released artifacts.

free parameters (3)

Pre-training mask ratio = 0.6 (60%)
The paper fixes 60% masking without reporting sensitivity; downstream gains may depend on this.
Mask size = 7x7 pixels
Derived from 224x224 input and downsampling ratio 32; not tuned or justified.
Decoder depth = 4 UNet blocks
SparK-style decoder chosen without ablation; reconstruction quality is not measured.

assumptions (3)

domain assumption SparK-style sparse masked modeling transfers from natural images to retinal fundus images.
The method is adopted from [21] without a validation study on the target domain.
domain assumption Pre-training on UK Biobank unlabeled fundus images benefits downstream AD, PD, and MMAC classification.
The paper assumes transferability from the pre-training distribution to the downstream tasks.
domain assumption The reported 5-fold cross-validation protocol was followed consistently for all baselines.
The paper states this but provides no logs, seeds, or code to verify comparability across methods.

how reviews work

0 comments

Cite this review

Pith. "Pith review of A BERT-Style Self-Supervised Learning CNN for Disease Identification from Retinal Images." pith.science (2026). https://pith.science/paper/M6RA46GZ

@misc{pith2026250418049,
  author       = {Pith},
  title        = {Pith review of: A BERT-Style Self-Supervised Learning CNN for Disease Identification from Retinal Images},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/M6RA46GZ}},
  note         = {Machine review of arXiv:2504.18049}
}

read the original abstract

In the field of medical imaging, the advent of deep learning, especially the application of convolutional neural networks (CNNs) has revolutionized the analysis and interpretation of medical images. Nevertheless, deep learning methods usually rely on large amounts of labeled data. In medical imaging research, the acquisition of high-quality labels is both expensive and difficult. The introduction of Vision Transformers (ViT) and self-supervised learning provides a pre-training strategy that utilizes abundant unlabeled data, effectively alleviating the label acquisition challenge while broadening the breadth of data utilization. However, ViT's high computational density and substantial demand for computing power, coupled with the lack of localization characteristics of its operations on image patches, limit its efficiency and applicability in many application scenarios. In this study, we employ nn-MobileNet, a lightweight CNN framework, to implement a BERT-style self-supervised learning approach. We pre-train the network on the unlabeled retinal fundus images from the UK Biobank to improve downstream application performance. We validate the results of the pre-trained model on Alzheimer's disease (AD), Parkinson's disease (PD), and various retinal diseases identification. The results show that our approach can significantly improve performance in the downstream tasks. In summary, this study combines the benefits of CNNs with the capabilities of advanced self-supervised learning in handling large-scale unlabeled data, demonstrating the potential of CNNs in the presence of label scarcity.

Figures

Figures reproduced from arXiv: 2504.18049 by the authors.

**Figure 1.** The ViT on the left panel can process non-masked patches without any changes since it can process variable-length sequences, while the CNN on the right panel cannot skip masks for convolutions. Simply adopting masking in CNN may lead to performance degradation. This work adopts a novel solution to address this problem. The main contributions of this paper include: 1). By adopting the sparse convolution, we propose … view at source ↗

**Figure 2.** The detailed architecture of the nn-MobileNet and its ILRB design. The nnMobileNet achieved superior results in retinal imaging research [28]. The current selfsupervised learning scheme further enhances its performance. Dropout. In order to solve the overfitting problem, the nn-MobileNet attempts to add Dropout modules at various locations within the network to identify the optimal placement for them as shown in… view at source ↗

**Figure 3.** Illustration of our pre-training workflow. We start by masking all the images randomly (Step 1). Next, we mask the feature maps adapted to different resolutions for the CNN encoder and decoder (Step 2). Finally, we perform sparse convolution on the masked image and restore the image through the decoder (Step 3). Hierarchical Structure. Hierarchy is widely recognized as the gold standard for vision representation sy… view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: Illustration of the quality control pipeline as we filter fundus image data of AD patients, PD patients, and normal control subjects from the UK Biobank dataset. Loss Function: Our loss function utilizes a method of comparing the mean squared error (MSE) between the ma…

**Figure 5.** Figure 5: Heat maps for AD (the first row) and PD (the second row) demonstrate our work achieves valuable biomarkers consistent with prior research [4]. outperform other state-of-the-art methods [27,7]. It is worth noting the RETFound model [27] was pre-trained with over 1.6M r…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

31 extracted references · 14 canonical work pages

[1]

Advances in neural information processing systems33, 1877–1901 (2020)

Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J.D., Dhariwal, P., Nee- lakantan, A., Shyam, P., Sastry, G., Askell, A., et al.: Language models are few-shot learners. Advances in neural information processing systems33, 1877–1901 (2020)

2020
[2]

arXiv preprint arXiv:1810.04805 (2018)

Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidirec- tional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)

arXiv 2018
[3]

arXiv preprint arXiv:2010.11929 (2020)

Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al.: An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)

arXiv 2020
[4]

Cells10(11) (Oct 2021)

Dumitrascu, O.M., Rosenberry, R., Sherman, D.S., Khansari, M.M., Sheyn, J., Tor- bati, T., Sherzai, A., Sherzai, D., Johnson, K.O., Czeszynski, A.D., Verdooner, S., Black, K.L., Frautschy, S., Lyden, P.D., Shi, Y., Cheng, S., Koronyo, Y., Koronyo- Hamaoui, M.: Retinal Venular Tortuosity Jointly with Retinal Amyloid Burden Correlates with Verbal Memory Los...

work page 2021
[5]

com/jacobgil/pytorch-grad-cam(2021)

Gildenblat, J., contributors: Pytorch library for cam methods.https://github. com/jacobgil/pytorch-grad-cam(2021)

work page 2021
[6]

In: Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition

Han, D., Yun, S., Heo, B., Yoo, Y.: Rethinking channel dimensions for efficient model design. In: Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition. pp. 732–741 (2021)

work page 2021
[7]

In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

He, K., Chen, X., Xie, S., Li, Y., Doll´ ar, P., Girshick, R.: Masked autoencoders are scalable vision learners. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 16000–16009 (2022)

2022
[8]

arXiv preprint arXiv:2006.08217 (2020)

Heo, B., Chun, S., Oh, S.J., Han, D., Yun, S., Kim, G., Uh, Y., Ha, J.W.: Adamp: Slowing down the slowdown for momentum optimizers on scale-invariant weights. arXiv preprint arXiv:2006.08217 (2020)

arXiv 2020

Show all 31 references

[9]

arXiv preprint arXiv:2104.05707 (2021)

Li, Y., Zhang, K., Cao, J., Timofte, R., Van Gool, L.: Localvit: Bringing locality to vision transformers. arXiv preprint arXiv:2104.05707 (2021)

2021 arXiv
[10]

In: Proceedings of the IEEE conference on computer vision and pattern recognition

Liu, B., Wang, M., Foroosh, H., Tappen, M., Pensky, M.: Sparse convolutional neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 806–814 (2015)

2015
[11]

In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

Liu, Z., Mao, H., Wu, C.Y., Feichtenhofer, C., Darrell, T., Xie, S.: A convnet for the 2020s. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 11976–11986 (2022)

2022
[12]

upsaclay.fr/competitions/12441(2023) 10 Authors Suppressed Due to Excessive Length

MICCAI: Myopic maculopathy analysis challenge 2023.https://codalab.lisn. upsaclay.fr/competitions/12441(2023) 10 Authors Suppressed Due to Excessive Length

2023
[13]

In: Proceedings of the IEEE conference on computer vision and pattern recognition

Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., Efros, A.A.: Context en- coders: Feature learning by inpainting. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 2536–2544 (2016)

2016
[14]

Science China Technological Sciences63(10), 1872–1897 (2020)

Qiu, X., Sun, T., Xu, Y., Shao, Y., Dai, N., Huang, X.: Pre-trained models for nat- ural language processing: A survey. Science China Technological Sciences63(10), 1872–1897 (2020)

2020
[15]

In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015

Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomed- ical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015. pp. 234–241. Springer (2015)

2015
[16]

In: Proceedings of the IEEE conference on computer vision and pattern recognition

Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: Mobilenetv2: In- verted residuals and linear bottlenecks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 4510–4520 (2018)

2018
[17]

Evol Intell15(1), 1–22 (2022)

Sarvamangala, D.R., Kulkarni, R.V.: Convolutional neural networks in medical image understanding: a survey. Evol Intell15(1), 1–22 (2022)

2022
[18]

In: Proceedings of the IEEE international conference on computer vision

Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad- cam: Visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE international conference on computer vision. pp. 618–626 (2017)

2017
[19]

arXiv preprint arXiv:1409.1556 (2014)

Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

2014 arXiv
[20]

PLoS Med12(3), e1001779 (Mar 2015)

Sudlow, C., Gallacher, J., Allen, N., et al.: UK biobank: an open access resource for identifying the causes of a wide range of complex diseases of middle and old age. PLoS Med12(3), e1001779 (Mar 2015)

2015
[21]

arXiv preprint arXiv:2301.03580 (2023)

Tian, K., Jiang, Y., Diao, Q., Lin, C., Wang, L., Yuan, Z.: Designing bert for convolutional networks: Sparse and hierarchical masked modeling. arXiv preprint arXiv:2301.03580 (2023)

2023 arXiv
[22]

In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

Woo, S., Debnath, S., Hu, R., Chen, X., Liu, Z., Kweon, I.S., Xie, S.: Convnext v2: Co-designing and scaling convnets with masked autoencoders. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 16133–16142 (2023)

2023
[23]

In: Medical Image Computing and Computer Assisted Intervention– MICCAI 2021

Yu, S., Ma, K., Bi, Q., Bian, C., Ning, M., He, N., Li, Y., Liu, H., Zheng, Y.: Mil-vt: Multiple instances learning enhanced vision transformer for fundus image classification. In: Medical Image Computing and Computer Assisted Intervention– MICCAI 2021. pp. 45–54. Springer (2021)

2021
[24]

In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

Zhang, C., Lin, G., Liu, F., Yao, R., Shen, C.: Canet: Class-agnostic segmentation networks with iterative refinement and attentive few-shot learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 5217–5226 (2019)

2019
[25]

In: Proceedings of the IEEE conference on computer vision and pattern recognition

Zhang, R., Isola, P., Efros, A.A.: Split-brain autoencoders: Unsupervised learning by cross-channel prediction. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 1058–1067 (2017)

2017
[26]

In: Proceedings of the IEEE/CVF conference on computer vision and pattern recog- nition

Zhou, Y., He, X., Huang, L., Liu, L., Zhu, F., Cui, S., Shao, L.: Collaborative learning of semi-supervised segmentation and classification for medical images. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recog- nition. pp. 2079–2088 (2019)

2019
[27]

Nature622(7981), 156–163 (2023) A BERT-Style Self-Supervised Learning CNN 11

Zhou, Y., Chia, M.A., Wagner, S.K., Ayhan, M.S., Williamson, D.J., Struyven, R.R., Liu, T., Xu, M., Lozano, M.G., Woodward-Court, P., et al.: A foundation model for generalizable disease detection from retinal images. Nature622(7981), 156–163 (2023) A BERT-Style Self-Supervise...

2023
[28]

In: International Conference on Medical Image Computing and Computer-Assisted Intervention

Zhu, W., Qiu, P., Chen, X., Li, H., Wang, H., Lepore, N., Dumitrascu, O.M., Wang, Y.: Beyond mobilenet: An improved mobilenet for retinal diseases. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 56–65. Springer (2023)

2023
[29]

In: International Conference on Information Processing in Medical Imaging

Zhu, W., Qiu, P., Dumitrascu, O.M., Sobczak, J.M., Farazi, M., Yang, Z., Nan- dakumar, K., Wang, Y.: Otre: Where optimal transport guided unpaired image-to- image translation meets regularization by enhancing. In: International Conference on Information Processing in Medical I...

2023
[30]

arXiv preprint arXiv:2302.02991 (2023)

Zhu, W., Qiu, P., Farazi, M., Nandakumar, K., Dumitrascu, O.M., Wang, Y.: Optimal transport guided unsupervised learning for enhancing low-quality retinal images. arXiv preprint arXiv:2302.02991 (2023)

2023 arXiv
[31]

arXiv preprint arXiv:1810.07810 (2018)

Zhuang, J.: Laddernet: Multi-path networks based on u-net for medical image segmentation. arXiv preprint arXiv:1810.07810 (2018)

2018 arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.

[1] [1]

Advances in neural information processing systems33, 1877–1901 (2020)

Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J.D., Dhariwal, P., Nee- lakantan, A., Shyam, P., Sastry, G., Askell, A., et al.: Language models are few-shot learners. Advances in neural information processing systems33, 1877–1901 (2020)

2020

[2] [2]

arXiv preprint arXiv:1810.04805 (2018)

Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidirec- tional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)

arXiv 2018

[3] [3]

arXiv preprint arXiv:2010.11929 (2020)

Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al.: An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)

arXiv 2020

[4] [4]

Cells10(11) (Oct 2021)

Dumitrascu, O.M., Rosenberry, R., Sherman, D.S., Khansari, M.M., Sheyn, J., Tor- bati, T., Sherzai, A., Sherzai, D., Johnson, K.O., Czeszynski, A.D., Verdooner, S., Black, K.L., Frautschy, S., Lyden, P.D., Shi, Y., Cheng, S., Koronyo, Y., Koronyo- Hamaoui, M.: Retinal Venular Tortuosity Jointly with Retinal Amyloid Burden Correlates with Verbal Memory Los...

work page 2021

[5] [5]

com/jacobgil/pytorch-grad-cam(2021)

Gildenblat, J., contributors: Pytorch library for cam methods.https://github. com/jacobgil/pytorch-grad-cam(2021)

work page 2021

[6] [6]

In: Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition

Han, D., Yun, S., Heo, B., Yoo, Y.: Rethinking channel dimensions for efficient model design. In: Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition. pp. 732–741 (2021)

work page 2021

[7] [7]

In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

He, K., Chen, X., Xie, S., Li, Y., Doll´ ar, P., Girshick, R.: Masked autoencoders are scalable vision learners. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 16000–16009 (2022)

2022

[8] [8]

arXiv preprint arXiv:2006.08217 (2020)

Heo, B., Chun, S., Oh, S.J., Han, D., Yun, S., Kim, G., Uh, Y., Ha, J.W.: Adamp: Slowing down the slowdown for momentum optimizers on scale-invariant weights. arXiv preprint arXiv:2006.08217 (2020)

arXiv 2020