{"id":"853493a2-cfe0-4e6d-bc29-49db9de5def6","arxiv_id":"2501.12048","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"A two-stage fine-tuning method, called CELD, reports 91% accuracy for three-class fundus image classification, but the claimed anti-forgetting benefit is not demonstrated.","lead":"This paper describes a two-stage training method for classifying eye fundus images into healthy, diabetic retinopathy, and glaucoma, reporting 91% accuracy on a pooled public dataset. The authors frame it as a way to add new disease classes with limited data, but the experiments do not show it beats simpler baselines.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim that CELD prevents catastrophic forgetting is unsupported: the second stage fine-tunes on the full expanded dataset τT with τS⊂τT, so old-class data is re-trained and no forgetting metric or replay-free baseline is reported.","rationale":"The reader's weakest assumption identifies the same load-bearing concern: the CELD experimental design retrains on all old-class data, so it never tests catastrophic forgetting in the scenario where old data becomes unavailable. This is not a minor omission; it is the core of the claimed contribution. The paper frames CELD as an incremental class-extension method and explicitly claims it prevents catastrophic forgetting and handles limited data, but Equation (4) trains on the entire pooled dataset, and the evaluation reports only the final three-class accuracy. There is no forgetting metric, no comparison to a model that sees only the new class in the second stage, and no ablation that isolates the contribution of the two-stage scheme from simple fine-tuning on the full dataset. The reported 0.9100 accuracy could equally arise from ordinary joint training on all samples. The reader's REJECT verdict is therefore justified: the central claim is not established by the presented evidence, and the paper currently reads as a standard fine-tuning study with an unsupported incremental-learning interpretation.","tokens_in":8018,"tokens_out":2921,"duration_ms":33003,"concrete_test":"Fine-tune the two-class DenseNet121 (ω*S) using only the new-class Glaucoma training samples, with no Healthy or DR images in the second stage, and evaluate per-class F1 on the held-out test set. Compare Healthy and DR F1 before and after this update and against the current CELD result. If Healthy/DR F1 drops substantially (or if CELD's advantage over this replay-free variant disappears), the catastrophic-forgetting claim is not supported. Additionally report the same comparison for a standard fine-tuning baseline and a from-scratch three-class model.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's claimed contribution (Section 1, bullet 2) is that CELD 'prevents catastrophic forgetting of previous learning, while leveraging existing knowledge to learn new classes in the presence of limited data.' The experimental protocol cannot test this claim. In Section 2.1, Eq. (4) minimizes the loss over all N samples of τT, and the text states τS ⊂ τT. Thus, the fine-tuning stage sees every Healthy and DR image from the source task again, together with the new Glaucoma images. This is essentially joint training on the expanded dataset, not an incremental-learning update from new-class data alone. No before/after measurement of accuracy on the original two-class task is provided, no forgetting metric (e.g., drop in per-class F1 for Healthy/DR) is computed, and no comparison is made to a model fine-tuned only on Glaucoma samples or to a standard fine-tuning baseline. The 0.9100 overall accuracy in Section 3.4 therefore demonstrates only that fine-tuning on all data reaches a certain accuracy; it does not establish the claimed ability to add a class while retaining old knowledge under data scarcity. If the intended use case is updating a deployed model without access to prior training data, the central claim collapses.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a two-stage framework, CELD, for retinal fundus image classification. A DenseNet121 classifier is first trained to distinguish Healthy from Diabetic Retinopathy (DR) images, and is then fine-tuned on an expanded dataset that also includes Glaucoma images, pooled from Messidor2, Chaksu, and LES-AV. The authors report an overall three-class accuracy of 0.9100 and a perturbation-based explainability analysis. The central claim is that CELD enables class extension with limited new data while preventing catastrophic forgetting.","tokens_in":8293,"tokens_out":3760,"duration_ms":43015,"significance":"If the claimed behavior were established, the framework would be relevant to incremental medical image classification under data scarcity. The use of public datasets and the inclusion of perturbation-based explainability are positive aspects. However, the core claim is not supported by the experimental design: the second stage fine-tunes on the full expanded dataset containing all old-class data, so the setup does not test catastrophic forgetting. In addition, all accuracy numbers come from a single train/validation/test split with no confidence intervals, multiple runs, or per-class test-set sizes, and the backbone is selected on the same pooled data used for final evaluation. As it stands, the contribution is not established.","major_comments":[{"comment":"The experimental protocol cannot support the claim that CELD 'prevents catastrophic forgetting of previous learning, while leveraging existing knowledge to learn new classes in the presence of limited data.' Since the manuscript states τS ⊂ τT and Eq. (4) minimizes the loss over all N samples of τT, the second stage retrains on every Healthy and DR image from the source task. The model is therefore never updated from new-class data alone, and no forgetting can occur by construction; no scenario is tested in which old-class data is unavailable. To support the claim, the second stage should be trained only on Glaucoma samples (or a small replay set), and the paper should report per-class accuracy on the original two classes before and after the second stage, together with a standard fine-tuning baseline. Without such experiments, the main contribution is unsupported.","section":"2.1, Eq. (4)"},{"comment":"All reported accuracies, including the headline 0.9100, come from a single train/validation/test split with no confidence intervals and no multiple runs. With only 199 Glaucoma images in the pooled dataset, the test split contains roughly 20 Glaucoma samples, so the reported Glaucoma F1-score of 0.6667 has very wide uncertainty. The paper should report per-class test-set sizes, confidence intervals or standard deviations over multiple seeds and splits, and ideally a bootstrap or permutation-based assessment of the accuracy difference between CELD and the baselines.","section":"3.4, Fig. 3 and Table 2"},{"comment":"DenseNet121 was selected as the backbone because it achieved the highest two-class accuracy on the same pooled dataset that is later used to report the CELD result. This is model selection on the evaluation distribution and can inflate the final reported accuracy. Architecture selection should be performed on a separate validation split, or the entire comparison should be repeated with a fixed, pre-specified backbone, in order to avoid circularity in the comparison with SeResNet101 and ViT.","section":"2.2 and 3.4"},{"comment":"The perturbation experiments are described qualitatively: confusion matrices and F1-scores are shown, but there are no error bars, significance tests, or repeated perturbation instantiations. Statements such as 'significantly decreased performance' and 'highly depends on the optic disc' are therefore not statistically supported. At minimum, the authors should report the mean and standard deviation of each metric over multiple perturbation runs and state the number of test images used for each class and perturbation type.","section":"3.4, perturbation analysis"}],"minor_comments":[{"comment":"The F1-score equation contains a typo: 'Precsion' should be 'Precision'.","section":"3.3"},{"comment":"The three-class results are presented only in Fig. 3 without a corresponding numerical table; a table with precision, recall, and F1 per class would improve readability and reproducibility.","section":"3.4"},{"comment":"The split description gives only percentages; the paper should report the exact number of training, validation, and test images per class and per source dataset, since the class imbalance is substantial.","section":"3.1"},{"comment":"The statement that CELD is 'unlike transfer learning' is not substantiated, because the proposed procedure is itself a fine-tuning method; the distinction should be clarified.","section":"1"},{"comment":"Reference [8] has a typo in the title: 'Rethinking ImageNet pre-training' should be 'Rethinking ImageNet Pre-training'.","section":"References"},{"comment":"No code or trained models are released, which limits reproducibility; the authors should state availability or provide a public implementation.","section":"general"}],"recommendation":"reject","confidential_remarks":"The reviewer agrees with the reader's assessment: the main claimed contribution, prevention of catastrophic forgetting, is not tested by the protocol because the second stage includes all old-class data. The paper is also statistically weak (single split, no confidence intervals, no per-class test sizes, backbone selection on the evaluation distribution). These are load-bearing issues for a paper whose novelty is the incremental learning claim, and they cannot be resolved by minor edits; the current version does not establish the proposed framework as an advance over standard fine-tuning on the full expanded dataset."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is a two-stage fine-tuning pipeline: train DenseNet121 on Healthy vs. DR, then fine-tune on the full expanded three-class dataset. The central claim—that CELD 'prevents catastrophic forgetting'—is not supported by the protocol. Equation (4) trains on all N samples of τT, which includes every source-task image, so old data is re-seen. There is no forgetting metric, no before/after comparison on the original two-class task, and no baseline like fine-tuning only on the new glaucoma samples. In fact, the method is joint training with warm-start initialization, which is standard transfer learning, not class-incremental learning. The 0.9100 accuracy therefore demonstrates only that fine-tuning on all data reaches a certain accuracy.\n\nWhat is worth credit: the authors pool three public fundus datasets and split them per source to avoid leakage, which is careful. The perturbation study is the most interesting part—it tests clinically motivated hypotheses (green channel importance, optic disc occlusion) and shows the model degrades in ways that align with ophthalmology. That is genuinely useful qualitative evidence for model behavior in screening applications, even if it isn't a novel explainability method.\n\nThe soft spots are substantial. Single train/test/validation split, no confidence intervals, no per-class test sizes, and only one run. Glaucoma has only 199 total samples, so the 0.6667 F1 is fragile. The backbone DenseNet121 is selected because it performed best on the same pooled data used for evaluation, which is a mild form of selection bias. The framing as a new framework is overblown; there is no new algorithm or architecture. The paper also doesn't cite any incremental-learning baseline (e.g., EWC, replay methods), which would be necessary to support the forgetting claim.\n\nWho gets value from this? Someone building a fundus screening classifier might read the perturbation results as sanity checks. Anyone working on continual learning will see a textbook example of how not to evaluate catastrophic forgetting. It deserves a serious referee only if the authors reframe it as an application study and add proper evaluation: repeated splits, confidence intervals, a real forgetting metric, and a replay-free baseline. In current form, I wouldn't send it to peer review; I'd desk reject and invite a resubmission after major revision.","headline":"Two-stage fine-tuning dressed as incremental learning; the forgetting claim isn't tested, but the perturbation analysis shows some clinical thought.","tokens_in":8806,"tokens_out":1913,"would_cite":false,"duration_ms":24114,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims a two-stage fine-tuning scheme, CELD, lets a fundus classifier add Glaucoma as a third class without forgetting Healthy and Diabetic Retinopathy knowledge, reaching an overall accuracy of 0.9100 on pooled public datasets.","keywords":["fundus images","class extension","limited data","catastrophic forgetting","diabetic retinopathy","glaucoma classification","DenseNet121","explainability"],"falsifier":"Train the second stage on Glaucoma images only, or on a target set with the Healthy and DR images removed, then measure Healthy and DR accuracy on the original test set; if accuracy drops well below the reported 0.8729 two-class level, the claim that CELD prevents catastrophic forgetting is contradicted.","tokens_in":7841,"feed_emoji":"👁️","tokens_out":4633,"duration_ms":45675,"temperature":0.7,"pith_summary":"This paper proposes a two-stage training scheme, Class Extension with Limited Data (CELD), for classifying retinal fundus images. A DenseNet121 network is first trained to separate Healthy from Diabetic Retinopathy (DR) images, and its optimized weights are then used to initialize a second network that is fine-tuned on an expanded dataset also containing Glaucoma images. The authors claim this lets the model pick up the new class without catastrophic forgetting of the original two classes, even when labeled data for the new class is scarce. On pooled public fundus datasets the three-class CELD model reaches an overall accuracy of 0.9100, outperforming direct three-class training of the same backbone models, and perturbation experiments show the model leans on the green channel and the optic disc.","feed_headline":"Two-stage training adds glaucoma to eye screening at 91% accuracy","feed_subtitle":"Fine-tuning a healthy-vs-DR model on a small glaucoma set beats three-class training from scratch.","key_machinery":"The load-bearing mechanism is sequential fine-tuning with parameter reuse: a DenseNet121 classifier $C_S$ is trained on the two-class source domain, and its optimized weights initialize a three-class classifier $C_T$ that is fine-tuned on $\\tau_T$ with cross-entropy loss. DenseNet121's dense connectivity supplies strong gradient flow and feature reuse, which the paper argues reduces overfitting on imbalanced data. The framework's formal condition $\\tau_S \\subset \\tau_T$ is what lets the target task keep the source task's images in the training set, so the model is never required to reconstruct old knowledge from memory alone. The explainability component uses six controlled perturbations—reducing the green channel, randomly removing green segments, reducing contrast, adding Gaussian noise, edge sharpening, and optic disc occlusion—to probe which image features drive the model's decisions.","core_discovery":"The central claim is that a classifier can be extended from two classes (Healthy, DR) to three classes (Healthy, DR, Glaucoma) by fine-tuning from the source classifier's weights on an expanded dataset, rather than retraining from scratch. The paper formalizes this with source data $\\tau_S$ and target data $\\tau_T$ satisfying $\\tau_S \\subset \\tau_T$, trains DenseNet121 on the two-class source task, then reuses the optimized weights $\\omega_S^*$ to initialize classifier $C_T$ and minimizes cross-entropy on the three-class target task. The reported result is an overall accuracy of 0.9100 with noticeably higher F1-scores for DR and Glaucoma than direct three-class training with SeResNet101, DenseNet121, or ViT. The perturbation analysis is part of the same contribution: degrading the green channel or occluding the optic disc degrades DR and Glaucoma classification, indicating which input features the model relies on.","pith_inferences":["A stronger test of the forgetting claim would be to fine-tune on Glaucoma images alone, or on a target set that excludes most Healthy and DR images; the current protocol keeps all source-task data in $\\tau_T$, so catastrophic forgetting is not truly measured.","The perturbation results could be turned into a prospective clinical hypothesis: a fundus screening tool that emphasizes green-channel contrast and optic-disc structure should transfer to other camera types better than one using full-color texture only.","The same two-stage extension pattern could be tried for other scarce ocular conditions, such as age-related macular degeneration, with the caveat that the paper's Glaucoma F1-score of 0.6667 remains the weakest, so rare-class performance will likely still lag.","A direct extension would be to add a replay buffer or regularization penalty when old data cannot be stored, which would turn CELD into a genuinely incremental learner rather than a two-stage fine-tuner."],"forward_implications":["A screening model can grow from two to three disease classes by fine-tuning on a small batch of new-class images, without retraining from scratch.","The 0.9100 accuracy on pooled public data suggests the approach is usable for DR and Glaucoma screening where labeled Glaucoma images are scarce.","If the perturbation findings hold, the model's decisions are driven by clinically sensible features: the green channel for DR lesions and the optic disc for Glaucoma.","DenseNet121 is a suitable backbone for this two-stage extension, outperforming SeResNet101 and ViT in the paper's comparisons.","The same weight-transfer recipe could be applied as new ocular disease classes are added over time in a deployed screening system."],"supporting_citations":[{"why":"Supplies the DenseNet121 classifier architecture whose dense connectivity is claimed to reduce overfitting and enable the two-stage fine-tuning.","marker":"[10]"},{"why":"Defines the catastrophic forgetting problem that CELD is designed to solve and motivates the incremental learning setup.","marker":"[19]"},{"why":"Grounds the claim that same-domain source data avoids the degradation seen when fine-tuning ImageNet models on medical images.","marker":"[8]"},{"why":"Provides the Chaksu Glaucoma and Healthy fundus images used in the target task.","marker":"[11]"},{"why":"Adds a small set of Glaucoma and Healthy images from a different acquisition setting, the LES-AV dataset.","marker":"[17]"},{"why":"Supplies the clinical basis for the green-channel perturbation experiments used to study DR lesion detection.","marker":"[20]"},{"why":"Serves as a state-of-the-art DR classifier whose performance the CELD framework is compared against.","marker":"[14]"},{"why":"One of the baseline architectures directly trained on three classes and outperformed by the CELD framework.","marker":"[9]"}],"fun_headline_variants":["Fine-tune two-class model to spot glaucoma too, hits 91%","From DR to glaucoma: adaptive class learning hits 91% accuracy","Small data? Extend model to new classes, 91% on eye scans","Two-step training detects glaucoma in retinal images at 91%","Class extension with limited data: 91% accuracy on fundus"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The framework's claim of preventing catastrophic forgetting is only tested when the expanded training set still contains all the original Healthy and DR images; if the intended use case is adding a class without access to the old data, the forgetting claim is not actually demonstrated.","fun_headline_variants_meta":{"raw":{"variants":["Fine-tune two-class model to spot glaucoma too, hits 91%","From DR to glaucoma: adaptive class learning hits 91% accuracy","Small data? Extend model to new classes, 91% on eye scans","Two-step training detects glaucoma in retinal images at 91%","Class extension with limited data: 91% accuracy on fundus"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000651,"raw_usage":{"total_tokens":2957,"prompt_tokens":889,"completion_tokens":2068,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":505,"completion_tokens_details":{"reasoning_tokens":1983}},"tokens_in":505,"tokens_out":2068,"duration_ms":14854,"temperature":1.0,"reasoning_tokens":1983,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T17:32:37.644742+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the second stage on Glaucoma images only, or on a target set with the Healthy and DR images removed, then measure Healthy and DR accuracy on the original test set; if accuracy drops well below the reported 0.8729 two-class level, the claim that CELD prevents catastrophic forgetting is contradicted.","supporting_citations":[{"cited_title":"In: Proceed- ingsoftheIEEEconferenceonComputerVisionandPatternRecognition(CVPR)","cited_arxiv_id":null,"evidence_quote":"Supplies the DenseNet121 classifier architecture whose dense connectivity is claimed to reduce overfitting and enable the two-stage fine-tuning."},{"cited_title":": Three types of incremental learning","cited_arxiv_id":null,"evidence_quote":"Defines the catastrophic forgetting problem that CELD is designed to solve and motivates the incremental learning setup."},{"cited_title":": Rethinking ImageNet pre-training","cited_arxiv_id":null,"evidence_quote":"Grounds the claim that same-domain source data avoids the degradation seen when fine-tuning ImageNet models on medical images."},{"cited_title":"Scientific Data10, 70 (2023)","cited_arxiv_id":null,"evidence_quote":"Provides the Chaksu Glaucoma and Healthy fundus images used in the target task."},{"cited_title":"In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2018: 21st International Confer- ence, Granada, Spain, September 16-20, 2018, Proceedings, Part II 11","cited_arxiv_id":null,"evidence_quote":"Adds a small set of Glaucoma and Healthy images from a different acquisition setting, the LES-AV dataset."},{"cited_title":"Medical Image Analysis11, 555–566 (2007)","cited_arxiv_id":null,"evidence_quote":"Supplies the clinical basis for the green-channel perturbation experiments used to study DR lesion detection."},{"cited_title":"Computers and Electrical Engineering 117, 109243 (2024)","cited_arxiv_id":null,"evidence_quote":"Serves as a state-of-the-art DR classifier whose performance the CELD framework is compared against."},{"cited_title":"In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (June 2018)","cited_arxiv_id":null,"evidence_quote":"One of the baseline architectures directly trained on three classes and outperformed by the CELD framework."}],"review_version":1}