{"id":"2f22cf8d-a8a5-4209-a036-8dc7d52884ac","arxiv_id":"2501.07750","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":9,"one_line_summary":"A semi-supervised framework with domain-specific augmentations and an improved U2Net segments the sclera well with as few as four labeled training images.","lead":"This paper presents a semi-supervised learning framework for segmenting the sclera, the white part of the eye, using very few labeled images and many unlabeled ones. It also introduces a new eye-diagnosis dataset and reports competitive segmentation accuracy on two public benchmarks, which matters for medical diagnosis and biometric identification.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Claimed SSL advantage is never isolated: the only comparison to the original SSL uses a different, weaker backbone, and no supervised-only baseline is reported.","rationale":"The reader's verdict is CONDITIONAL, and I agree that the paper's evidence is insufficient as presented. However, the reader's formal 'weakest assumption' pins the concern on pseudo-label reliability in Eq. (6). While that is a real mechanism-level risk, the more load-bearing issue is that the experimental design never isolates the SSL contribution. The paper's Section 4.3 openly acknowledges that the RITnet-based comparison found neither method effective, so the subsequent switch to an improved U2Net means the original SSL is never tested with the final backbone. Additionally, the absence of a supervised-only baseline means the unlabeled-data losses could be irrelevant or even harmful. This is not a question of internal inconsistency in the consistency assumption; it is a missing control that directly prevents the central claim from being evaluated. The concrete test proposed (a 2x2 ablation) would definitively separate the backbone effect from the SSL effect. If the ablation is provided and shows a genuine SSL gain, the paper's central claim could stand; otherwise the 'superiority over original SSL' collapses to 'a stronger backbone helps'. Thus I keep the reader's CONDITIONAL verdict, since the issue is addressable with additional experiments, but I strengthen the rationale by identifying exactly which comparison is missing.","tokens_in":15922,"tokens_out":8302,"duration_ms":78883,"concrete_test":"Run a controlled 2x2 ablation on the proposed dataset, holding fixed the label subsets Xl = {4, 12, 24, 48, 72, 96}, optimizer, and training schedule: (a) original SSL (Chaudhary et al., 2021) with the improved U2Net; (b) proposed SSL-SS with the improved U2Net; (c) supervised-only improved U2Net trained on the same labeled images with no unlabeled loss; (d) proposed SSL-SS with the standard U2Net. If (a) achieves mIoU close to (b), or (c) achieves mIoU close to (b), the reported gains come from the backbone or from labeled data alone, not from the SSL contributions.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that the proposed semi-supervised method 'boosts' sclera segmentation is not supported by the experimental design, because the effect of the SSL machinery is confounded with the change of segmentation network. In Section 4.3, 'Evaluation of our semi-supervised method', the proposed SSL method is compared to the original SSL (Chaudhary et al., 2021) only when both use RITnet as the backbone. The authors explicitly state that 'neither method effectively segmented the sclera' and that they therefore abandoned RITnet in favor of their enhanced U2Net. However, the original SSL framework is never run with the improved U2Net. Consequently, the large gains shown in Table 2 between 'U2Net' and 'Proposed Method' could be entirely attributable to the deeper 7-level U2Net architecture, not to the SSLD/SSL-SS pseudo-labeling or the domain-specific augmentations. Moreover, no fully supervised baseline is reported: the paper never trains the improved U2Net on only the labeled images (with the same Xl values) while omitting the unsupervised losses Lu and Lss. Without such a control, the contribution of the unlabeled data—the defining element of the claimed SSL advantage—cannot be isolated. The paper's own narrative admits that with RITnet neither SSL variant worked, which further undermines the attribution of success to the SSL framework rather than the backbone choice.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a semi-supervised learning framework for sclera segmentation that combines domain-specific augmentations (CLAHE, gamma correction), a two-stage SSL procedure (SSLD and SSL-SS) based on consistency across augmentations and inverse spatial transformations, and an enhanced seven-level U2Net backbone. The authors introduce a new eye-diagnosis dataset of roughly 800-1000 manually annotated images and evaluate the method on this dataset, UBIRIS.v2, and SBVPI using only 4 to 96 labeled images. The central claim is that the proposed SSL framework achieves accurate sclera segmentation with very few labels, outperforming the original SSL method and most prior fully supervised sclera segmentation models.","tokens_in":16221,"tokens_out":2822,"duration_ms":29142,"significance":"If the claims are supported, the work would be a useful contribution to low-label medical image segmentation and sclera biometrics: the new dataset is a potential community resource, the domain-specific augmentations are sensible, and the experiments span three datasets with systematically varied label counts. The paper also gives credit for attempting a real-world diagnostic dataset rather than relying only on public benchmarks. However, the experimental design currently conflates the SSL machinery with the choice of segmentation backbone, and it lacks the supervised-only and same-backbone controls needed to attribute the reported gains to semi-supervised learning. The significance of the method is therefore not yet established by the evidence presented.","major_comments":[{"comment":"The claimed SSL advantage is never isolated. The comparison to the original SSL method (Chaudhary et al., 2021) uses RITnet as the backbone for both methods, and the authors state that 'neither method effectively segmented the sclera' with RITnet. The proposed method is then evaluated with the enhanced U2Net, but the original SSL framework is never run with the enhanced U2Net. Consequently, the large gap between 'U2Net' and 'Proposed Method' in Table 2 could be due to the deeper seven-level architecture rather than to the SSLD/SSL-SS pseudo-labeling or the domain-specific augmentations. A supervised-only baseline is also missing: there is no experiment training the enhanced U2Net on the same labeled images while omitting the unsupervised losses Lu and Lss. Without these two controls, the contribution of the unlabeled data, which is the defining element of the claimed SSL boost, cannot be assessed.","section":"§4.3, Table 2 and Fig. 6-8"},{"comment":"The comparisons against prior work are not conducted under identical training conditions. Table 3 reports results of fully supervised methods trained with larger label counts (e.g., 120 labeled UBIRIS.v2 images for ScleraSegNet, 734 for SBVPI), while Table 4 reports the proposed method at various Xl values. The claim that Xl=72 on UBIRIS.v2 'surpasses the performance of most models listed in Table 3' is therefore not a head-to-head comparison, since the backbone, training schedule, loss terms, and number of labels all differ. The paper should provide same-protocol comparisons, at least for the main baselines, and should also report results over multiple random seeds with standard deviations or error bars, since the reported improvements are often small (e.g., 87.94 vs 86.45 mIoU for Xl=4 in Table 2).","section":"§4.3, Table 4 and Table 3"},{"comment":"The dataset description is internally inconsistent and this is load-bearing for the evaluation. The abstract and contributions state 'approximately 800 images from over 100 patients,' but §4.1 says 'about 1000 images were used,' and Table 1 reports 700 training, 200 validation, and 100 test images, which sums to exactly 1000. The authors should clarify the true dataset size, report the number of subjects and images precisely, and state whether all experiments use the same train/validation/test split. In addition, since all evaluation depends on manual annotations, details on annotator expertise and inter-annotator agreement should be provided.","section":"§4.1, Table 1 and Abstract/Contributions"},{"comment":"Several hyperparameters required to reproduce Equations (5), (6), and (8) are missing or incompletely specified. The number of augmented copies k in Eqs. (5)-(6) is not defined; the unsupervised loss schedules are described only by slopes (0.02 and 0.002 per epoch) without their initial values and saturation behavior; and the sentence describing the augmentation probabilities ends with 'with probabilities of 50' in §4.2, leaving p1 and p2 unspecified. These values are needed for any independent verification of the reported results, and they are among the free parameters listed in the method.","section":"§4.2 and §3.3.1"}],"minor_comments":[{"comment":"There are numerous typos and citation errors, including 'paer' in §4.1, 'n the initial phase' in §3.2, and 'Rot et al.(Ronneberger et al., 2015)' and 'Lucio et al.(Ronneberger et al., 2015)' in Section 1.2, which cite the wrong reference.","section":"Throughout"},{"comment":"The subcaptions in Figures 6-8 all use the same label '(a)' for different numbers of labeled images; each panel should be uniquely identified.","section":"§4.3, Figures 6-8"},{"comment":"The notation k is introduced as the number of augmented copies but is never explicitly defined in the text; please define it before Eq. (5) and state its value in the experiments.","section":"§3.3.1, Eqs. (5)-(6)"},{"comment":"The text 'is the result of the segmentation of ground true' should read 'ground truth', and the recall formula should be written with explicit set intersection/union or TP/FP/FN notation consistently.","section":"§4.1, Eq. (10)"},{"comment":"The sentence about the T transform ends with 'with probabilities of 50' and is incomplete; the values of p1 and p2 and the exact rotation/translation ranges should be given.","section":"§4.2"}],"recommendation":"major_revision","confidential_remarks":"The central issue is experimental: the authors must either add the missing controls (same backbone for the original SSL, supervised-only U2Net, same-protocol comparisons, multi-seed variability) or substantially weaken the claims. If such experiments cannot be added within the scope of a revision, the manuscript should not be accepted, because the current evidence does not demonstrate that semi-supervised learning is the source of the improvement. I would also suggest asking the authors to make the new dataset, or at least its statistics and annotation protocol, clearly described, since it is a key contribution of the paper."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe headline: this paper promises a big practical win—sclera segmentation with as few as four labeled images—but the experiment never isolates the SSL contribution from the backbone change. The improved U2Net does most of the heavy lifting, and without a supervised-only control, the claim that unlabeled data is what helps is not supported.\n\nWhat is genuinely new: a new manually annotated eye-diagnosis dataset (about 800 images per the text, though the experiment uses \"about 1000\"; Table 1 splits 700/200/100, so the description needs fixing), a reasonable extension of Chaudhary et al.'s SSL framework with domain-specific CLAHE/gamma augmentation and an extra self-supervised consistency stage, and an enhanced 7-level U2Net. The evaluation spans three datasets (their own, UBIRIS.v2, SBVPI) and shows consistent monotone gains as labeled data increases—that internal consistency is a point in their favor. They also honestly report that with RITnet neither their method nor the original SSL segmented the sclera well, which is why they moved to U2Net.\n\nThe soft spots are real. The stress-test captures the main one: the only head-to-head with the original SSL uses RITnet, which they then abandon. The \"Proposed Method\" in Table 2 is compared against U2Net within their own SSL framework, not against the original Chaudhary framework on the same backbone. So the 1-3 point mIoU gains could just be the deeper network. No supervised-only baseline appears anywhere, so the contribution of the unlabeled data—the core of the paper—is never measured. There are also no error bars or repeated-seed statistics; differences of 1-2 mIoU points are easily within noise. Minor issues: the dataset count inconsistency, some citation errors in related work (Lucio et al. cited as Ronneberger et al.), and no code/data release, which makes the numbers hard to verify.\n\nI don't think the method is wrong or the idea is trivial. The self-supervised consistency stage and domain-specific augmentations are sensible additions, and the new dataset is a useful resource. But the paper as written overclaims its evidence. It deserves a serious referee, because the subfield would benefit from a low-label sclera segmentation baseline and a new dataset. The authors should be asked to run the original SSL with their improved U2Net, add a supervised-only control (same network, same labels, no unsupervised losses), report variance across at least three seeds, and sort out the dataset description. If those changes land, this becomes a solid subfield contribution. For now, treat the \"SSL superiority\" claim with caution.\n\nWould I cite it? Not until I see the controls. Would I bring it to reading group? Maybe, as a case study in how backbone choice can confound SSL claims.\n\nRecommendation: send to peer review, but with a clear request for the missing baseline experiments.","headline":"The SSL advantage is never isolated from the backbone change, and without a supervised-only control the paper's central claim is not supported; still, the new dataset and sensible SSL extension make it worth reviewing.","tokens_in":16760,"tokens_out":3324,"would_cite":false,"duration_ms":32345,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A two-stage semi-supervised framework with domain-specific and spatial-transform augmentations attains accurate sclera segmentation from as few as four labeled images, beating the original SSL method and most prior models.","keywords":["sclera segmentation","semi-supervised learning","consistency regularization","data augmentation","CLAHE","U2Net","medical image segmentation","label-efficient learning"],"falsifier":"During training, compute the agreement between the guessed labels and the held-out ground truth on a small labeled subset the model never trains on. If that agreement falls below the accuracy of a model trained on the supervised loss alone while the unsupervised loss keeps decreasing, the pseudo-label loop is teaching systematic errors; likewise, corrupting the ground-truth masks with a known pixel shift and observing whether the unsupervised losses amplify the shift faster than the supervised loss corrects it would isolate the same failure.","tokens_in":15725,"feed_emoji":"👁️","tokens_out":8232,"duration_ms":67762,"temperature":0.7,"pith_summary":"The paper claims that sclera segmentation, a step needed for eye-disease diagnosis and biometric identification, can be trained from a handful of labeled images instead of hundreds. Its framework pairs semi-supervised learning with two kinds of augmentation: domain-specific adjustments (CLAHE and gamma correction) that stabilize predictions under lighting variation, and spatial transforms (rotation and translation) whose inverse-averaged predictions serve as self-supervised targets for unlabeled data. The segmentation backbone is an enlarged U2Net with seven encoder and six decoder stages. On the authors' new eye-diagnosis dataset, the method reports 87.94% mIoU with only four labeled examples; on UBIRIS.v2 and SBVPI it matches or beats prior models that were trained on many more labels. If correct, this makes accurate eye segmentation accessible where expert annotations are scarce.","feed_headline":"Four labeled images yield accurate sclera segmentation","feed_subtitle":"A two-stage semi-supervised pipeline beats models trained on hundreds of labels, cutting the cost of expert annotation.","key_machinery":"The load-bearing machinery is the two-stage consistency framework together with the enlarged U2Net. In the first stage (SSLD), each image is augmented $k$ times with CLAHE and gamma correction; the model's softmax predictions on the augmented copies are averaged to form guessed labels for unlabeled data, and an L2 loss ($L_u$) enforces agreement with those guesses. In the second stage (SSL-SS), a spatial transform $T$ (small rotations and translations) is applied before prediction, and the inverse transforms $T^{-1}$ are applied to the predictions before averaging per Eq. (6), so the guessed labels are consistent under geometric perturbation. The supervised loss $L_s$ combines cross-entropy, boundary-aware, dice, and surface losses with a schedule that weights dice first and surface loss later. The segmentation backbone is U2Net expanded from six to seven encoder levels and five to six decoder levels, with RSU blocks and dilated RSU-4F in the deepest stages and a Saliency Graph Fusion Module that fuses seven side maps into the final saliency map. This combination is what the paper credits for the rapid convergence and label efficiency.","core_discovery":"The paper's central claim is that its SSL framework, combining domain-specific augmentations in a first stage (SSLD), a second stage (SSL-SS) that enforces consistency across spatial transformations by averaging inverse-transformed predictions, and an improved U2Net backbone, attains high segmentation accuracy with as few as four labeled images, where the original SSL method stays underfitted within the same 100 training epochs. The authors report that with the same RITnet segmentation network, their method's test mIoU at four labels already exceeded the original method's mIoU at 96 labels. They also state that neither SSL variant with RITnet segmented the sclera effectively, which is why they moved to an enlarged U2Net as the backbone. On their dataset, the proposed method reaches 87.94% mIoU at $X_l = 4$ and 89.90% at $X_l = 96$; on UBIRIS.v2, 72 labeled examples yield 84.60% mIoU, surpassing most earlier models trained with 120 labeled examples; on SBVPI, 500 labeled examples yield 91.77% mIoU, above ScleraSegNet's result with 734 labels. The paper additionally contributes a manually annotated eye-diagnosis dataset of about 800 images from over 100 patients.","pith_inferences":["The consistency mechanism in Eq. (6) is a generic spatial regularizer with no sclera-specific parts, so the same two-stage recipe could be lifted to other anatomical segmentation tasks such as iris, pupil, or conjunctival vessels.","Because the guessed labels are unthresholded softmax averages, a natural extension the paper does not explore is confidence weighting or thresholding on pseudo-labels to suppress low-quality guesses.","The dataset's multiple gaze directions suggest evaluating segmentation under extreme gaze and eyelid occlusion separately; the paper reports only aggregate metrics, leaving per-condition performance open.","Ablating the SSLD and SSL-SS stages separately would reveal which stage carries the label efficiency, since the paper reports only the combined method."],"forward_implications":["Sclera segmentation becomes feasible where expert annotations are scarce: four labeled images plus unlabeled data yield 87.94% mIoU on the authors' dataset.","Label efficiency transfers across image domains: at 72 labeled examples the method reaches 84.60% mIoU on UBIRIS.v2, exceeding most prior methods trained on 120 labeled examples.","On the high-resolution SBVPI dataset, 500 labeled examples give 91.77% mIoU, surpassing ScleraSegNet, which used 734 labeled examples.","The spatial-consistency stage (SSL-SS) appears to be what lifts training out of the underfitting that the original SSL method showed within 100 epochs.","The new eye-diagnosis dataset of roughly 800 manually annotated images provides a benchmark for label-scarce sclera segmentation, with gaze directions that stress occlusion and illumination."],"supporting_citations":[{"why":"The SSL framework for eye image segmentation that this paper builds on; supplies the base two-stage paradigm and the guessed-labels mechanism.","marker":"(Chaudhary et al., 2021)"},{"why":"U2Net, the salient-object-detection architecture that the paper enlarges into its seven-level encoder and six-level decoder backbone.","marker":"(Qin et al., 2020)"},{"why":"RITnet, the prior eye-segmentation network used as baseline and as source of the CLAHE/gamma augmentation idea and the loss schedule.","marker":"(Chaudhary et al., 2019)"},{"why":"Temporal ensembling, the consistency-assumption source behind averaging predictions and the L2 unsupervised loss.","marker":"(Laine and Aila, 2016)"},{"why":"Surface loss, one component of the supervised loss that penalizes stray regions in the final training schedule.","marker":"(Kervadec et al., 2019)"},{"why":"U-Net, the architecture family that the boundary-aware loss term and the later comparison networks come from.","marker":"(Ronneberger et al., 2015)"},{"why":"CLAHE, the contrast-limiting histogram equalization used as the domain-specific augmentation.","marker":"(Zuiderveld, 1994)"},{"why":"ScleraSegNet, the prior state-of-the-art on SBVPI that the proposed method outperforms at $X_l = 500$.","marker":"(Wang et al., 2019)"},{"why":"UBIRIS.v2, one of the two public benchmark datasets used for cross-domain evaluation.","marker":"(Proença et al., 2009)"},{"why":"SBVPI, the other public benchmark dataset, whose sclera-annotated images the authors use.","marker":"(Rot et al., 2018)"}],"fun_headline_variants":["Sclera segmentation from just 4 labeled images","Four labels outperform hundreds in sclera segmentation","Semi-supervised sclera segmentation with minimal labels","Accurate sclera segmentation from four labeled samples","Fewer labels, high accuracy: SSL sclera segmentation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that the pseudo-labels formed by averaging the model's own predictions across augmentations and inverse-transformed views are reliable enough to train on; if those guesses carry systematic errors, the unsupervised losses will reinforce them instead of correcting them.","fun_headline_variants_meta":{"raw":{"variants":["Sclera segmentation from just 4 labeled images","Four labels outperform hundreds in sclera segmentation","Semi-supervised sclera segmentation with minimal labels","Accurate sclera segmentation from four labeled samples","Fewer labels, high accuracy: SSL sclera segmentation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000319,"raw_usage":{"total_tokens":1827,"prompt_tokens":998,"completion_tokens":829,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":614,"completion_tokens_details":{"reasoning_tokens":751}},"tokens_in":614,"tokens_out":829,"duration_ms":7719,"temperature":1.0,"reasoning_tokens":751,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:35:52.363966+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"During training, compute the agreement between the guessed labels and the held-out ground truth on a small labeled subset the model never trains on. If that agreement falls below the accuracy of a model trained on the supervised loss alone while the unsupervised loss keeps decreasing, the pseudo-label loop is teaching systematic errors; likewise, corrupting the ground-truth masks with a known pixel shift and observing whether the unsupervised losses amplify the shift faster than the supervised loss corrects it would isolate the same failure.","supporting_citations":[{"cited_title":"U-net: Convolutional networks for biomed- ical image segmentation","cited_arxiv_id":null,"evidence_quote":"U-Net, the architecture family that the boundary-aware loss term and the later comparison networks come from."},{"cited_title":"Semi-supervised learn- ing by disentangling and self-ensembling over stochastic latent space","cited_arxiv_id":null,"evidence_quote":"ScleraSegNet, the prior state-of-the-art on SBVPI that the proposed method outperforms at $X_l = 500$."},{"cited_title":"Deep multi-class eye segmentation for ocular biometrics","cited_arxiv_id":null,"evidence_quote":"SBVPI, the other public benchmark dataset, whose sclera-annotated images the authors use."}],"review_version":1}