Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

Addressing High Class Imbalance in Multi-Class Diabetic Retinopathy Severity Grading with Augmentation and Transfer Learning

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A transfer-learning pipeline with class-balanced augmentation to 20,000 samples per class reaches 98.9% binary and 84.6% five-class accuracy on the APTOS 2019 diabetic retinopathy grading benchmark.

desk verdict A competent but overclaimed benchmark: binary results are fine, but the five-class 'state-of-the-art' claim is not supported by the paper's own macro metrics. read the letter →

arxiv 2507.17121 v2 pith:WOLSO3M2 submitted 2025-07-23 cs.CV cs.LG

Faisal Ahmed This is my paper
classification cs.CVcs.LG
keywords DiabeticRetinopathyDeepLearningTransferDataAugmentationClassImbalanceMedicalImagingAPTOS2019EfficientNet
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to show that class imbalance, not architecture, is the main obstacle to automated diabetic retinopathy grading. By augmenting every severity class in the APTOS 2019 training set to 20,000 images and fine-tuning pretrained CNNs, the author reports 98.9% accuracy and 99.4% AUC for binary normal-versus-DR screening, and 84.6% accuracy with 94.1% AUC for five-class severity grading. If these numbers generalize, the approach offers a simple, scalable template for imbalanced medical imaging: balance the data, fine-tune a pretrained network, and screen reliably without specialized loss functions or architectures. The paper also positions EfficientNet-B0 and ResNet34 as the best accuracy-efficiency trade-offs, and notes that its own five-class precision (74.1%) and recall (63.8%) trail the headline accuracy.

What carries the argument

The central mechanism is class-balanced stochastic augmentation: a composition $T$ of nine transforms (horizontal and vertical flips, $\pm 25^\circ$ rotation, color jitter, random-resized crop, affine distortion, Gaussian blur, sharpness, and perspective) applied iteratively until every severity class reaches a target $M = 20{,}000$ samples, producing a training set $\bigcup_c (X^c_{\mathrm{train}} \cup \widetilde{X}^c_{\mathrm{train}})$. Training then uses cross-entropy loss, Adam with learning rate $10^{-4}$, and early stopping on macro F1 over up to 500 epochs. This machinery moves the imbalance correction into the data itself, so the model trains on a fully balanced proxy of the original distribution.

What would settle it

Run the identical pipeline on the same APTOS split with augmentation removed; if the control reaches roughly the same 84.6% five-class accuracy, augmentation is not causing the reported gain. Also test the trained model on a second fundus dataset collected by a different institution: a large accuracy drop would show that the 20,000-per-class copies overfit APTOS-specific appearance rather than learning general severity features.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that heavy class-balanced augmentation neutralizes the skew in DR severity labels enough that a standard transfer-learning pipeline beats or matches published baselines on both tasks. The author reports binary accuracy of 98.9% (precision 98.6%, recall 99.3%, AUC 99.4%) and five-class accuracy of 84.6% (macro precision 74.1%, macro recall 63.8%, AUC 94.1%) on APTOS 2019 under an 85:15 stratified split. The gain is attributed to the augmentation regime, where each of the five classes is stochastically expanded to 20,000 training images, combined with fine-tuning pretrained ResNet and EfficientNet backbones. The paper further claims that this pipeline outperforms prior methods including Topo-CNN, Tofi-ML, SCL, DRISTI, and C-DNN, while Section 7 concedes that augmented samples may not capture the full diversity of real clinical data.

Load-bearing premise

The load-bearing premise is that 20,000 stochastically varied copies of each severity class represent the true class distributions well enough that a model trained on them generalizes to real fundus images; the paper itself notes in Section 7 that augmentation may not replicate the diversity and complexity of real-world clinical data.

Editorial extensions

If this is right

  • A binary screener at 98.9% accuracy and 99.3% recall would miss very few true DR cases, making automated pre-screening a realistic triage step.
  • Five-class grading at 84.6% accuracy suggests that a single fine-tuned CNN can separate adjacent severity levels once the training data are balanced.
  • The augmentation recipe transfers to other imbalanced medical imaging problems because it requires no architecture changes or class-weighted losses.
  • ResNet34 and EfficientNet-B0 deliver the reported accuracies at moderate compute, so replication does not require the largest models tested.
  • The five-class macro recall of 63.8% means the 84.6% accuracy overstates how reliably severe and proliferative cases are caught.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not test on external data, and APTOS 2019 comes from a single eye hospital in India; a reader should treat the reported numbers as potentially inflated by site-specific appearance until cross-dataset validation appears.
  • Because the augmentation target is fixed at 20,000 for every class, minority classes are repeatedly copied hundreds of times more than the majority class; this repetition weighting can reshape the decision boundary in ways aggregate accuracy and AUC do not reveal.
  • A natural extension the author leaves implicit is per-class confusion analysis: reporting which adjacent severity pairs are confused would tell clinicians where the five-class model needs human review, especially given macro recall of 63.8%.
  • If the recipe transfers, a practical deployment pattern would be binary screening to rule out DR, followed by five-class grading of positives, with low-confidence severe or proliferative calls escalated to a human grader.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The manuscript proposes a transfer-learning pipeline combined with heavy class-balanced augmentation (20,000 synthetic images per class) for binary and five-class diabetic retinopathy severity grading on the APTOS 2019 dataset. It reports binary classification results of 98.9% accuracy and 99.4% AUC using ResNet34, and five-class results of 84.6% accuracy and 94.1% AUC using EfficientNet-B0, claiming state-of-the-art binary performance and five-class performance that 'outperforms or matches' leading methods. The paper compares 13 pretrained backbones and makes code publicly available. The main methodological contribution is the augmentation strategy, while the central empirical claim is that this strategy yields state-of-the-art or competitive results on both tasks.

Significance. If the binary result is reproducible, it provides a useful data point that standard augmentation plus transfer learning reaches near-ceiling performance on the binary APTOS task. The five-class result, however, is not state-of-the-art on the paper's own balanced metrics: Table 6 shows that Topo-CNN achieves higher macro precision (82.2 vs 74.1), macro recall (78.2 vs 63.8), and AUC (95.5 vs 94.1). The value of the paper therefore lies more in the systematic comparison of backbones and in the public release of code than in the stated SOTA claim. The manuscript's empirical framing, public code, and disclosure of the augmentation limitation are positive aspects that facilitate verification and further study.

major comments (4)
  1. [§5.2, Table 6] The claim that the method 'outperforms or matches current leading methods' in five-class grading is not supported by the paper's own metrics. In Table 6, Topo-CNN attains macro precision 82.2 vs 74.1, macro recall 78.2 vs 63.8, and one-vs-rest AUC 95.5 vs 94.1. The 4.6-point accuracy advantage (84.6 vs 80.0) does not establish superiority on an imbalanced task; the implied macro-F1 is 68.5 for the proposed model versus 80.1 for Topo-CNN. Please either restrict the claim to 'higher overall accuracy with lower balanced metrics' or supply additional evidence (e.g., class-wise confusion matrices, statistical comparison) to support the stronger claim.
  2. [§3.2, §7] The augmentation strategy is the paper's central methodological contribution, but the experiments never isolate its effect. There is no comparison against the same backbones trained on the original imbalanced data, against class-weighted loss, or against a simpler oversampling baseline. Consequently, the reader cannot attribute the reported results to augmentation. Moreover, Section 7 concedes that augmentation 'may not fully replicate the diversity and complexity of real-world clinical data'; this is exactly the load-bearing assumption behind the 20,000-sample-per-class training set, and it is not tested, e.g., by evaluating on an imbalanced holdout drawn from the original distribution.
  3. [§4.2, §5.2] All results come from a single stratified 85:15 split with no repeated seeds, error bars, or confidence intervals. The five-class test set contains only 29 severe and 44 proliferative images (Table 2), so the 84.6 vs 80.0 accuracy difference is within the range of split-to-split variance. Reporting mean ± std over at least three seeds or bootstrap confidence intervals would substantially strengthen the binary and five-class comparisons.
  4. [§3.2, §3.4] The method details are underspecified for reproducibility. The exact parameter values for the augmentation transforms (e.g., the ranges for ColorJitter and RandomAffine) are not stated, and the validation set is not described: Section 3.4 refers to validation-based early stopping and model selection, but Section 4.1 only describes an 85:15 train/test split. Please report the validation split, its size, and the epoch or criterion used for selecting the final model.
minor comments (6)
  1. [Keywords] The keyword list contains 'Data Augmentation,,' with a double comma; please correct the typo.
  2. [Tables 5 and 6] The 'Nor:Abn' column heading is appropriate only for binary classification; in Table 6 it should be replaced by the actual class distribution or removed to avoid confusion.
  3. [§4.1] The sentence 'The total number of training and test samples in the dataset were 3662 and 1928 respectively' is confusing because the paper later uses only the 3662 labeled images; please rephrase to clarify that the original competition test labels are unavailable.
  4. [§3.5] The Precision and Recall definitions are given per class; please explicitly state that macro averaging is used for the multiclass metrics, as the current text is ambiguous.
  5. [References] Reference [7] is incomplete ('A. Unknown', 'arXiv preprint arXiv:???'); please complete or remove it.
  6. [§5.2] The statement that Tofi-ML 'lagged behind in accuracy (75.6%)' uses the same numerical value as its recall; please clarify whether this is a coincidence or a typo.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper is an empirical benchmark whose reported metrics do not reduce to any fitted parameter, self-citation, or definitional identity.

full rationale

This paper is an empirical transfer-learning study with no formal derivation chain to examine. The central claims are measured accuracies, AUCs, and comparisons on the APTOS 2019 dataset. The data augmentation target of 20,000 samples per class is a hand-selected experimental design choice (Section 3.2), not a parameter fitted to the test labels; the augmentation equations define class sizes, not performance metrics. No 'prediction' is derived from a fitted quantity, and no metric is defined in terms of the training input in a way that forces the reported outcome. The only self-referential element is the use of the author's own prior published models, Topo-CNN [10] and Tofi-ML [11], as comparison baselines. These baselines were published independently and their reported numbers are external evidence, not outputs of the present paper's pipeline; citing them does not make the comparison circular. The skeptical concern about macro recall and AUC being lower than Topo-CNN in the five-class task is a correctness or interpretation question, not a circularity question. Likewise, the acknowledged limitation that augmentation 'may not fully replicate the diversity and complexity of real-world clinical data' (Section 7) concerns generalization robustness, not circular reasoning. The paper is self-contained as an empirical benchmark, and no load-bearing step reduces to its own inputs.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The claim rests on standard assumptions of the empirical transfer-learning literature plus one hand-chosen augmentation target; no new entities are introduced.

free parameters (3)
  • Per-class target augmentation size M = 20,000
    Hand-chosen in Section 3.2 with no sensitivity analysis; directly controls the training distribution and thus all reported metrics.
  • Augmentation transform hyperparameters = flip p=0.5, rotation +-25 degrees, crop scale [0.7,1.0], blur sigma [0.1,2.0], perspective 0.2
    Defined in Section 3.2; no ablation or tuning procedure is reported, so the contribution of each transform cannot be assessed.
  • Training hyperparameters = learning rate 1e-4, batch size 32, max 500 epochs, patience 50
    Fixed choices in Section 4.2; no search is described, and early stopping depends on an unspecified validation split.
assumptions (3)
  • domain assumption APTOS 2019 expert grades are accurate ground truth
    The competition labels from Aravind Eye Hospital are treated as correct without adjudication (Section 4.1). This is standard practice for benchmark papers.
  • domain assumption Augmented synthetic images are representative of real class distributions
    The balancing strategy in Section 3.2 requires this; the paper itself expresses uncertainty in Section 7.
  • domain assumption ImageNet-pretrained features transfer to fundus images
    Standard transfer-learning assumption invoked in Section 3.3 and not validated independently.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Addressing High Class Imbalance in Multi-Class Diabetic Retinopathy Severity Grading with Augmentation and Transfer Learning." pith.science (2026). https://pith.science/paper/WOLSO3M2

@misc{pith2026250717121,
  author       = {Pith},
  title        = {Pith review of: Addressing High Class Imbalance in Multi-Class Diabetic Retinopathy Severity Grading with Augmentation and Transfer Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WOLSO3M2}},
  note         = {Machine review of arXiv:2507.17121}
}
read the original abstract

Diabetic retinopathy (DR) is a leading cause of vision loss worldwide, and early diagnosis through automated retinal image analysis can significantly reduce the risk of blindness. This paper presents a robust deep learning framework for both binary and five-class DR classification, leveraging transfer learning and extensive data augmentation to address the challenges of class imbalance and limited training data. We evaluate a range of pretrained convolutional neural network architectures, including variants of ResNet and EfficientNet, on the APTOS 2019 dataset. For binary classification, our proposed model achieves a state-of-the-art accuracy of 98.9%, with a precision of 98.6%, recall of 99.3%, F1-score of 98.9%, and an AUC of 99.4%. In the more challenging five-class severity classification task, our model obtains a competitive accuracy of 84.6% and an AUC of 94.1%, outperforming several existing approaches. Our findings also demonstrate that EfficientNet-B0 and ResNet34 offer optimal trade-offs between accuracy and computational efficiency across both tasks. These results underscore the effectiveness of combining class-balanced augmentation with transfer learning for high-performance DR diagnosis. The proposed framework provides a scalable and accurate solution for DR screening, with potential for deployment in real-world clinical environments.

Figures

Figures reproduced from arXiv: 2507.17121 by the authors.

Figure 1
Figure 1. Architecture of the proposed model. Overview of the methodology, including preprocessing, augmentation, training, and evaluation [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. RepViT-CXR: A Channel Replication Strategy for Vision Transformers in Chest X-ray Tuberculosis and Pneumonia Classification

    cs.CV 2025-09 reject novelty 2.0 of 10

    Applying a standard channel-replication trick to a pretrained Vision Transformer yields very high reported accuracy for TB and pneumonia classification on three chest X-ray datasets.

Reference graph

Works this paper leans on

22 extracted references · 11 canonical work pages · cited by 1 Pith paper

  1. [1]

    W. H. Organization, Blindness and vision im- pairment, 2023. URL: https://www.who. int/news-room/fact-sheets/detail/ blindness-and-visual-impairment , accessed: 2025-05-02

    2023

  2. [2]

    Ting, et al., Deep learning for diabetic retinopathy analysis: A review, Journal of

    D. Ting, et al., Deep learning for diabetic retinopathy analysis: A review, Journal of ... (2019)

    work page 2019

  3. [3]

    Li, et al., Applications of machine learning in retinal imag- ing, Medical Image Analysis (2021)

    X. Li, et al., Applications of machine learning in retinal imag- ing, Medical Image Analysis (2021)

    work page 2021

  4. [4]

    Sarhan, et al., Machine learning in retinal image analysis, Pattern Recognition Letters (2020)

    M. Sarhan, et al., Machine learning in retinal image analysis, Pattern Recognition Letters (2020)

    work page 2020

  5. [5]

    B. Oltu, B. K. Karaca, H. Erdem, A. ¨Ozg¨ur, A systematic re- view of transfer learning based approaches for diabetic retinopa- thy detection, arXiv preprint arXiv:2105.13793 (2021). Review covering many pretrained models including ResNet, Inception, EfficientNet in DR classification

    work page Pith review arXiv 2021

  6. [6]

    Diabetic Retinopathy Grading System Based on Transfer Learning

    E. AbdelMaksoud, S. Barakat, M. Elmogy, Diabetic retinopa- thy grading system based on transfer learning, arXiv preprint arXiv:2012.12515 (2020). Uses EfficientNet in grading system context

    work page Pith review arXiv 2020

  7. [7]

    Unknown, Aptos 2019 blindness detection database: five-class diabetic retinopathy grading with severe and pro- liferative under-representation, arXiv preprint arXiv:??? (2019)

    A. Unknown, Aptos 2019 blindness detection database: five-class diabetic retinopathy grading with severe and pro- liferative under-representation, arXiv preprint arXiv:??? (2019). Describes class distribution: severe and proliferative under-represented (3662 images, class3 = 193, class4 = 295)

    work page 2019

  8. [8]

    Gangwar, K

    S. Gangwar, K. Kotecha, Balancing data through data augmen- tation improves the generality of transfer learning for diabetic retinopathy classification, Applied Sciences 12 (2022) 5363. doi:10.3390/app12115363, augmentation applied on AP- TOS to address imbalance and transfer learning with ResNet etc

    work page doi:10.3390/app12115363 2022

Show all 22 references
  1. [9]

    of VR-FuseNet, Vr-fusenet: A fusion of heterogeneous fun- dus data and explainable deep network for diabetic retinopathy classification, arXiv preprint arXiv:2504.21464 (2025)

    A. of VR-FuseNet, Vr-fusenet: A fusion of heterogeneous fun- dus data and explainable deep network for diabetic retinopathy classification, arXiv preprint arXiv:2504.21464 (2025). Applies SMOTE oversampling across five-class DR datasets, including APTOS

    2025

  2. [10]

    Ahmed, M

    F. Ahmed, M. A. N. Bhuiyan, B. Coskunuzer, Topo-cnn: Reti- nal image analysis with topological deep learning, Journal of Imaging Informatics in Medicine (2025) 1–17

    2025

  3. [11]

    Ahmed, B

    F. Ahmed, B. Coskunuzer, Tofi-ml: Retinal image screening with topological machine learning, in: Annual Conference on Medical Image Understanding and Analysis, Springer, 2023, pp. 281–297

    2023

  4. [12]

    Bodapati, et al., Composite dnn approach for diabetic retinopathy detection, Medical Image Computing and

    A. Bodapati, et al., Composite dnn approach for diabetic retinopathy detection, Medical Image Computing and ... (2021)

    2021

  5. [13]

    Ahmed, B

    F. Ahmed, B. Nuwagira, F. Torlak, B. Coskunuzer, Topo-CXR: Chest X-ray TB and Pneumonia Screening with Topological Machine Learning, in: Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, 2023, pp. 2326–2336

    2023

  6. [14]

    Ahmed, Topological Machine Learning in Medical Image Analysis, Ph.D

    F. Ahmed, Topological Machine Learning in Medical Image Analysis, Ph.D. thesis, The University of Texas at Dallas, 2023

    2023

  7. [15]

    Gupta, et al., Augmentation strategies in dr severity grading, Medical Imaging 2023 (2023)

    R. Gupta, et al., Augmentation strategies in dr severity grading, Medical Imaging 2023 (2023)

    2023

  8. [16]

    https://www.kaggle.com/c/ aptos2019-blindness-detection

    APTOS, Asia Pacific Tele-Ophthalmology So- ciety (APTOS) 2019 Blindness Detection Dataset, 2019. https://www.kaggle.com/c/ aptos2019-blindness-detection

    2019

  9. [17]

    M. R. Islam, et al., Applying supervised contrastive learning for the detection of DR and its severity levels from fundus images, Computers in Biology and Medicine 146 (2022) 105602

    2022

  10. [18]

    Kumar, S

    G. Kumar, S. Chatterjee, C. Chattopadhyay, Dristi: a hybrid deep neural network for diabetic retinopathy diagnosis, Signal, Image and Video Processing 15 (2021) 1679–1686

    2021

  11. [19]

    J. D. Bodapati, et. al., Blended multi-modal deep convnet fea- tures for diabetic retinopathy severity prediction, Electronics 9 (2020) 914

    2020

  12. [20]

    S. S. Chaturvedi, K. Gupta, V . Ninawe, P. S. Prasad, Automated diabetic retinopathy grading using deep convolutional neural network, arXiv preprint arXiv:2004.06334 (2020)

    2020 arXiv

  13. [21]

    Macsik, J

    P. Macsik, J. Pavlovicova, J. Goga, S. Kajan, Local binary cnn for diabetic retinopathy classification on fundus images, Acta Polytech. Hung. 19 (2022) 27–45

    2022

  14. [22]

    F. Long, H. Xiong, J. Sang, A classification method for diabetic retinopathy based on self-supervised learning, in: International Conference on Intelligent Computing, Springer, 2024, pp. 347– 357. 9

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.