{"id":"c398c110-516e-4b56-9b04-36484af45e50","arxiv_id":"2504.12132","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"CroCo, a two-branch method with bag-level and instance-level cross-consistency, improves whole slide image classification accuracy when only a small portion of slides are labeled.","lead":"This paper introduces CroCo, a method for classifying whole slide images (giant pathology scans) when only a small fraction of slides have labels and many are unlabeled. It uses two complementary classifiers that supervise each other at both the slide level and the patch level, and reports better accuracy than adapted semi-supervised baselines on four datasets.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The instance-level pseudo-labels for positive bags are unverified: the mutual attention/instance loop can entrench 'lazy attention' rather than correct it, and the paper's own Section 4.5 and Conclusion concede the mechanism.","rationale":"The reader's weakest assumption is also my own, and the manuscript's internal evidence supports it: Section 4.5 documents lazy attention at high positive-instance ratios, and the Conclusion concedes the absence of pseudo-label filtering. The central mechanism of CroCo is the two-level cross-consistency loop, and its instance-level component is only as good as the attention-derived targets. If those targets are biased toward easy positives, the mutual supervision can amplify rather than correct the bias, which would directly undermine the claimed instance-classification superiority. A secondary reproducibility issue is the Algorithm 1 flow, where the unlabeled bag-level consistency loss appears inside the 'if Wi has true label' branch; this is a defect in the published description but not the main logical soft spot. The controlled hard-positive experiment proposed above would settle the concern because it isolates exactly the failure mode the paper itself identifies. Since the reader already issued CONDITIONAL and my concern is addressable rather than disproving, the verdict should remain UNCHANGED.","tokens_in":17923,"tokens_out":4789,"duration_ms":54398,"concrete_test":"Construct a controlled CIFAR-10-MIL variant at a higher positive-instance ratio (e.g., 50% or 70%) with two disjoint positive subclasses: an 'easy' subclass (e.g., trucks with distinctive color/background, easily separable from negatives) and a 'hard' subclass (e.g., low-contrast or occluded trucks). Train CroCo exactly as in the paper at 10% and 50% labeled-bag ratios. Report (i) bag AUC, (ii) overall instance AUC, and (iii) instance AUC restricted to hard-positive instances. If hard-positive AUC is near chance while bag AUC is high, the mutual attention/instance pseudo-label loop entrenches lazy attention rather than resolving it, and the instance-classification claim is not supported. If hard-positive AUC tracks overall AUC, the concern does not land.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.4 (Eqs. 3-4) uses normalized attention scores from the bag branch as instance targets for the lower branch, and lower-branch predictions as targets for the attention module, for both labeled positive bags and all unlabeled bags. This is the only source of instance-level supervision for positive bags. The bag-level CE loss (Eq. 6) never requires the attention distribution to match the true instance distribution; a bag with many positives can be classified correctly by attending to one easy positive. Section 4.5 documents exactly this degeneracy: at 50%/70% positive-instance ratios, full-data bag AUC is 1.0 while instance AUC drops to 0.8224/0.7935 (Table 1), a 'lazy' attention pattern. CroCo's claimed fix is that unlabeled consistency 'mitigates' laziness, but the mitigation uses the same attention scores as pseudo-labels, unfiltered as the Conclusion explicitly admits. If attention has locked onto easy positives, the mutual CE loop trains the instance branch to reproduce that biased ranking; no mechanism in the loss breaks the symmetry or corrects the bias. The ablations in Table 5 show that adding instance consistency helps on average, but they do not measure whether the pseudo-label targets are accurate for hard positives, or whether errors amplify over training. Thus the central claim that two-level cross consistency yields reliable instance classification rests on an unverified assumption.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces a new problem setting called Weakly Semi-supervised Whole slide image Classification (WSWC), in which only a small fraction of bags are labeled at the bag level while the remaining bags are unlabeled. The authors propose CroCo, a dual-branch architecture with a bag-based attention classifier and an instance-based classifier that share an encoder. CroCo enforces bag-level and instance-level cross-consistency between the two branches, using attention scores as instance pseudo-labels and instance predictions as attention pseudo-labels. The paper reports bag- and instance-level AUC on one synthetic MIL dataset (CIFAR-10-MIL) and three real datasets (Camelyon16, TCGA-Lung, Clinical Cervical), comparing against adapted Mean Teacher, FixMatch, FreeMatch, and a labeled-only baseline. The main empirical claim is that CroCo outperforms these methods across labeled-data ratios of 10–50% and positive-instance ratios of 1–70%.","tokens_in":18229,"tokens_out":4019,"duration_ms":41667,"significance":"If the empirical claims hold, the WSWC formulation is a useful and clinically motivated problem, and CroCo is a simple, strong baseline for it. The paper provides a reasonably broad evaluation across four datasets and ablates several design choices, including the two consistency levels, the bi-directional supervision, the heterogeneous branches, true-negative instance use, and the shared encoder. These ablations give partial support to the central design. However, the significance is tempered by three issues: the instance-level pseudo-label loop is not validated against ground truth, results are reported as single AUC values without error bars, and the comparison set omits prior semi-supervised MIL/WSI methods that are closest to the proposed setting.","major_comments":[{"comment":"The instance-level supervision for positive and unlabeled bags is entirely self-generated: normalized attention scores from the bag branch are used as pseudo-labels for the instance branch, and instance-branch predictions are fed back to the attention module (Section 3.4, Eqs. (3)–(4), Algorithm 1). The bag-level CE loss in Eq. (6) does not require the attention distribution to match the true instance distribution, and Table 1 shows that the resulting 'lazy attention' regime is real: at PIR 50% and 70% with full data, bag AUC is 1.0 while instance AUC drops to 0.8224 and 0.7935. The paper argues in Section 4.5 that unlabeled consistency mitigates this laziness, but the mitigation uses the same unfiltered attention scores as targets, and the Conclusion explicitly admits that no pseudo-label filtering is used. The central claim of reliable instance classification therefore rests on an unverified assumption. Please report instance-level pseudo-label AUC against ground truth during training on CIFAR-10-MIL and Camelyon16, and compare CroCo with a variant that filters or thresholds the attention-based pseudo-labels.","section":"§3.4, Eqs. (3)–(4), Algorithm 1, Table 1, §4.5, Conclusion"},{"comment":"All tables report a single AUC per method and setting, with no error bars, confidence intervals, or significance tests. Many reported gaps are small, for example Table 3(a) at 50% labeled data shows CroCo at 0.9462 versus FreeMatch at 0.9415, a difference that could easily fall within run-to-run variability. In addition, the comparison set in Section 4.3 contains only adapted natural-image SSL methods (Mean Teacher, FixMatch, FreeMatch) and the labeled-only baseline; there is no prior semi-supervised MIL or WSI method, such as the works cited as Refs. [58–62], adapted to the WSWC setting. Please add multiple-seed results with standard deviations or confidence intervals, and include at least one existing WSI semi-supervised or MIL-based method as a baseline.","section":"Tables 1–4 and §4.3"},{"comment":"Tables 3 and 4 report a Full-data column only for CroCo, not for the labeled-only baseline or the adapted SSL methods. The text in Section 4.7 uses the closeness to Full-data as evidence of CroCo's effectiveness, but without Full-data results for the other methods the reader cannot tell whether CroCo closes the label-efficiency gap better than, for example, FreeMatch would at 50% labeled data. Please either add the missing Full-data rows/columns for all comparison methods or remove the comparative 'close to Full-data' claims.","section":"Tables 3 and 4, §4.7"}],"minor_comments":[{"comment":"The implementation details state that 'Entropy loss is used for model training', while Algorithm 1 and Section 3 define the losses as cross-entropy; please make the terminology consistent.","section":"§4.4 and Algorithm 1"},{"comment":"The caption contains a typo, 'T able 2', and the table is difficult to read because the column boundaries for the labeled-data-ratio groups are not visually separated.","section":"Table 2 caption"},{"comment":"The text first says that 'Algorithm 1 in the Supplementary Materials presents the pseudocode', and then later says 'Algorithm 1 presents the pseudocode' with the algorithm appearing in the main text; please reconcile these references.","section":"§3.4"},{"comment":"The SVM/linear evaluation in Figure 3C is not described in enough detail; please specify which features are extracted, how the SVM is trained, and whether the instance labels used for training the SVM are the ground-truth labels or the model's pseudo-labels.","section":"Figure 3C and §4.8"},{"comment":"The unsupervised loss weight tau is a free parameter, but its value or selection procedure across datasets is not reported; since the total loss in Algorithm 1 depends linearly on tau, please report the chosen values and the grid used.","section":"Algorithm 1, total loss"},{"comment":"In the ablation table, the 'bag cls.→ins cls.' and 'ins cls.→bag cls.' rows are described as applying at both instance and bag levels, which makes the labels ambiguous; please clarify whether the arrow indicates one-directional supervision at both levels, and what is supervised at each level.","section":"Table 5"}],"recommendation":"major_revision","confidential_remarks":"The paper's novelty claim of being 'the first' WSWC formulation is made against a very brief dismissal of prior semi-supervised MIL works (Refs. [58–62]); since this novelty is part of the contribution, the authors should engage with those methods more concretely, ideally by adapting at least one of them as a baseline. There is also a notable concentration of self-citations to the WENO method [30] for dataset construction and preprocessing; this is not improper, but the dependence on that prior work for experimental design should be acknowledged more explicitly."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: this is a decent empirical paper, not a breakthrough. It introduces a semi-supervised MIL setting for whole slide images where only a fraction of bags are labeled, and proposes CroCo, a dual-branch consistency method that improves bag and instance AUC across four datasets. The central empirical claim survives my reading; the novelty and the instance-labeling mechanism are the weak spots.\n\nWhat's genuinely useful: the WSWC setting is practically motivated and will resonate with anyone who has tried to get pathologists to label slides. CroCo is a clean extension of the authors' own WENO work: a bag-level attention branch and an instance-level branch, with cross-consistency at both levels for unlabeled bags. The experiments are reasonably extensive (CIFAR-MIL, Camelyon16, TCGA-Lung, an in-house cervical dataset), with two backbones, and the ablations in Table 5 show the two-level consistency and heterogeneous branches contribute. The discussion of lazy attention at high positive-instance ratios is honest and useful.\n\nThe soft spots are real but addressable. The novelty claim is overstated: semi-supervised MIL for histology goes back at least to Lu et al. [62], which the paper dismisses as requiring instance annotations; that is not accurate. The comparison lacks any prior semi-supervised MIL baseline, and all tables report single AUC values with no error bars, so we cannot judge whether the gains are meaningful. The pseudocode in Algorithm 1 appears to place the unlabeled-bag loss inside the labeled-bag block; that needs fixing.\n\nThe more substantive concern, also flagged by the stress test, is the instance pseudo-label loop. For positive bags, the only instance-level supervision is the normalized attention score from the bag branch, and the instance branch's predictions feed back to the attention module. Nothing forces the attention to identify all positives; bag-level CE can be satisfied by one easy positive. The paper's own Section 4.5 shows exactly this degeneracy at high PIRs. The claim that unlabeled consistency mitigates laziness is plausible, and the empirical instance AUC improvements support it, but there is no analysis of whether pseudo-labels for hard positives are accurate or whether errors amplify. The conclusion admits there is no filtering. This is not a fatal flaw, but it is a missing piece of evidence for the mechanism.\n\nBottom line: this paper deserves peer review. The empirical contribution is useful for the computational pathology community, and the weaknesses are fixable. A serious referee should ask for error bars, a comparison to [62] or similar, a cleaned-up pseudocode, and an analysis of pseudo-label quality. I would not cite it as a methodological breakthrough, but I would cite it as a comparison in WSI work.","headline":"A practically motivated semi-supervised MIL method for WSIs with solid empirical gains, an overstated novelty claim, and an under-analyzed instance pseudo-label loop.","tokens_in":18735,"tokens_out":4514,"would_cite":true,"duration_ms":43720,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":null,"created_at":"2026-08-16T12:37:24.361619+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":null,"supporting_citations":[],"review_version":1}