{"id":"0edfa41a-a64d-480f-a0e3-e572707cf964","arxiv_id":"2512.22666","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"INTERACT-CMIL combines shared features, selective head supervision, and a dependence loss to improve multi-criteria CMIL grading, reaching macro F1 of 0.76-0.88 on a 486-patch dataset.","lead":"The paper introduces a multi-head deep learning framework that jointly predicts five histopathological grading axes for conjunctival melanocytic intraepithelial lesions (CMIL), reporting relative macro F1 gains up to 55.1% over baselines. It also presents a new multi-center dataset of 486 expert-annotated biopsy patches as a computational benchmark.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The three-component attribution is not supported: ablations are cumulative, and the temperature-scaling component is never defined in the method section.","rationale":"The reader's verdict of CONDITIONAL is appropriate. The strongest claim—that INTERACT-CMIL outperforms the two baselines on all five tasks—is supported by the reported macro-F1 values, which are separated by more than the fold standard deviations, although formal significance testing is absent. However, the second part of the central claim, that the three introduced components are responsible for the gains, is not established by the paper's experiments. The cumulative ablation design and the undefined temperature-scaling component are concrete, localizable gaps: the paper cannot currently support the statement in Sec. 3.3 that 'each component provides additive gains.' This is a reproducibility and attribution problem, not a fatal flaw; the underlying method may still work, and a leave-one-out ablation could validate the claim. I therefore do not change the verdict, but I would make the reported component-wise percentages conditional on a proper factorial ablation and a full definition of the temperature-scaling mechanism. I partially agree with the reader's weakest_assumption: the dependence-loss noise is a valid concern about one component, but the more load-bearing issue is that the ablation architecture prevents any single-component attribution at all, and one component lacks even a definition.","tokens_in":6732,"tokens_out":6424,"duration_ms":71549,"concrete_test":"Run a leave-one-out ablation from the full INTERACT-CMIL model: (1) remove only the dependence loss, (2) remove only temperature scaling, (3) remove only selective supervision, keeping all other components and hyperparameters fixed, and report macro-F1 for all five tasks. Also write down the temperature-scaling operation and its temperature value in the method section before running the ablation. If each single-component removal produces a drop comparable to the paper's claimed ~3–4% per component, the additive attribution survives; if, for example, removing only temperature scaling changes macro-F1 by less than the reported ~4%, the cumulative ablation path overstates (or misattributes) that component's contribution.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim includes not just the overall gain over baselines but the assertion that the gain comes from three components: shared selective supervision, inter-dependence loss, and temperature scaling (Sec. 3.3, Table 1). That component-level attribution is the least secure part of the argument. The ablation table removes components cumulatively: full model → –dep. → –dep.–temp. → –dep.–temp.–sel. Under this design, the effect of each component is measured only after earlier components have already been removed. For example, the reported ~4% contribution of temperature scaling is computed by comparing 0.7397 (WHO4, –dep.) with 0.6924 (–dep.–temp.), and the ~4% contribution of selective supervision compares 0.6924 with 0.6120 (–dep.–temp.–sel.). This does not tell us what happens if only one component is removed from the full model; interactions among the three components could account for the observed drops. The paper states that 'each component provides additive gains across all five diagnostic tasks,' but the evidence only supports a claim about one particular removal order. In addition, temperature scaling is never defined: Sec. 2.4 gives L_total = L_cls + λL_dep with no temperature parameter, and no equation, default value, or training-time description of 'temp.' appears anywhere. Thus the ablation labeled '–temp.' tests an unspecified modification, and the exact contribution attributed to it is not reproducible.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents INTERACT-CMIL, a multi-head classifier for five CMIL grading axes (WHO4, WHO5, horizontal spread, vertical spread, and cytologic atypia) built on frozen CHIEF embeddings and a shared 256-D projection. The training objective combines combinatorial partial supervision (three of five heads active per iteration), a KL-based inter-dependence loss that matches the predicted joint distribution over the active heads to the empirical joint distribution of the batch's one-hot labels, and, according to the ablation study, a temperature-scaling step that is not actually defined in the Methods. The model is evaluated with 5-fold cross-validation on a newly collected 486-patch dataset from three German university hospitals. Table 1 reports mean macro F1 of 0.76/0.88/0.71/0.76/0.70 for the five axes, corresponding to relative gains over a frozen-CHIEF single-task baseline of 55.1/24.7/15.2/25.0/12.5%. The paper claims that these gains arise additively from the three components.","tokens_in":7114,"tokens_out":4908,"duration_ms":52724,"significance":"If the reported results are reproducible, the contribution is a clinically aligned, multi-criteria benchmark for a rare ocular lesion and a sensible use of pathology foundation-model features. Strengths include patient-disjoint splits, multi-center annotation with expert adjudication, and head-to-head comparison against both CNN and foundation-model baselines. The full-model improvements over BaseCHIEF are large and consistent across all five axes, and the qualitative examples illustrate plausible predictions. The main significance is as a proof-of-concept for dependency-aware multi-task grading in a data-scarce pathology setting. The work is much less persuasive as a methodological contribution until the undefined temperature-scaling component is specified and the ablation evidence is made non-cumulative.","major_comments":[{"comment":"Temperature scaling is listed as one of the three key components and is removed in the ablation ('–dep.–temp.'), but it is never defined. Sec. 2.4 defines the objective as L_total = L_cls + λ L_dep and contains no temperature parameter; no equation, default value, or training-time description appears anywhere in the paper. Therefore the ~4% contribution attributed to 'temperature scaling' is not a reproducible finding. Please specify how temperature is applied (e.g., logit scaling before softmax, with initialization/scheduling) and report the value(s) used. If this component was inherited from a previous version, it needs to be fully integrated into the Methods and Figure 2.","section":"Sec. 2.4, Sec. 3.3.3"},{"comment":"The ablation removes components in one cumulative order (full → –dep. → –dep.–temp. → –dep.–temp.–sel.), so the individual effect of each component is confounded with removal order. For example, the temperature-scaling effect is estimated by comparing 0.7397 (WHO4, –dep.) with 0.6924 (–dep.–temp.), after the dependency loss is already removed; the selective-supervision effect is estimated by comparing 0.6924 with 0.6120. These differences do not identify the contribution of each component in the full model and do not support the statement that 'each component provides additive gains across all five diagnostic tasks' (Sec. 3.3). Please run leave-one-out ablations (full model minus one component) and, if interactions are expected, a small factorial design.","section":"Sec. 3.3, Table 1"},{"comment":"All performance claims rest on 5-fold cross-validation on 486 patches, but the paper reports no significance tests, confidence intervals for the differences, or paired comparisons. With only five folds, the mean/median and standard deviation in Table 1 are insufficient to establish that the gains over BaseCHIEF are statistically reliable, especially for horizontal spread (+0.0932 macro F1) where the reported standard deviations overlap. Please report per-fold paired differences with a Wilcoxon signed-rank test or bootstrap CIs, and state how many patients are involved in addition to the 486 patches, confirming that the patient-disjoint split is strictly enforced.","section":"Table 1, Sec. 3.2"},{"comment":"The dependence loss uses as its target P, the empirical joint distribution of the one-hot labels within each training batch. This is not an independent clinical constraint—it summarizes the training labels themselves, so the loss enforces internal consistency with dataset statistics rather than an external pathological prior. Given the small, class-imbalanced dataset, batch-level co-occurrence estimates may be noisy (e.g., the predominance of high-grade lesions). The paper does not analyze sensitivity to batch size or class imbalance, nor does it justify the KL direction used here. Please add such an analysis and clarify the clinical source of the target distribution, or temper the claim that the loss aligns predictions with 'established WHO and C-MIN standards.'","section":"Sec. 2.3, Eq. (2)"}],"minor_comments":[{"comment":"There are two obvious typos: 'Conjunctiv al' in the title and 'Conjunctiv al' in the abstract. Also, the abstract says 'WHO4, WHO5, horizontal spread, vertical spread, and cytologic atypia' but the figure uses 'AP' for atypia; please unify the abbreviations and define them at first use.","section":"Title/Abstract"},{"comment":"ROC curves are shown without numeric AUC values either in the figure or the text. Since AUC is listed as a secondary metric, report the AUC values in Table 1 or in the figure caption.","section":"Fig. 3"},{"comment":"The dataset description 'WHO5=2, WHO4=3' is unclear; it probably refers to the most frequent grade, but the sentence is ambiguous. Please specify the class distributions for all five tasks, including the number of classes per task and per-class counts.","section":"Sec. 3.1.1"},{"comment":"The paper calls the method 'reproducible' but provides no code, data, or model-checkpoint availability statement. Given the undefined temperature-scaling component and the lack of data release, the reproducibility claim is currently not supported.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The full-model results are promising and the dataset is a useful contribution, but the missing definition of temperature scaling and the cumulative ablation design are load-bearing issues that prevent publication in the current form. I would not reject on dataset size alone, but the authors need to deliver a specifiable method, proper ablations, and at least a minimal statistical comparison before this is ready."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a genuinely new application — first deep-learning framework for grading conjunctival melanocytic intraepithelial lesions, on a new multi-center dataset of 486 expert-annotated patches. The overall gain over frozen CHIEF and ResNet baselines is real on this dataset: macro F1 up 55% on WHO4 and 25% on vertical spread, consistent across all five axes, and patient-disjoint cross-validation. That is worth something, especially given the rarity of the disease.\n\nWhat the paper does well: the problem is well-motivated, the multi-task framing maps directly onto clinical criteria, and the authors are honest about label imbalance and the small dataset. The shared-feature setup with selective supervision is a reasonable way to regularize with few samples. The qualitative examples are helpful.\n\nSoft spots, in order of severity. First, the temperature-scaling component is never defined. Section 2.4 gives L_total = L_cls + λL_dep with no temperature parameter anywhere; the ablation '–temp.' tests something the reader cannot identify. That alone makes the 4% attribution unverifiable. Second, the ablations are cumulative, not leave-one-out. The reported contributions of each component depend on the removal order; the paper claims 'additive gains' but only shows one path. Third, the dependence loss uses the empirical co-occurrence of the training labels as the target for consistency. That's a summary of the labels, not an independent constraint; on 486 imbalanced samples, batch-level co-occurrence could encode sampling bias. The gains on held-out folds suggest it helps here, but there's no sensitivity analysis to batch size or class imbalance. Fourth, no code or data release, no significance tests for the relative gains, and the 'first DL framework' claim would need a more careful literature check (there may be prior work on PAM or C-MIN classification). These are addressable rather than fatal.\n\nThe stress-test note got the main issue right: the three-component attribution is not supported as stated because of the cumulative ablation design, and temperature scaling is undefined. The central claim of overall improvement over baselines is better supported.\n\nWho is this for? Researchers working on computational pathology for rare ocular lesions, and anyone thinking about multi-task regularization in low-data regimes. It deserves a serious referee — the dataset and first benchmark are useful — but the authors should be asked to define temperature scaling, run proper leave-one-out ablations, and release code/data. I'd take it for review, but expect revision.","headline":"A plausible first benchmark for CMIL grading, but the component attribution and the temperature-scaling component are under-supported; worth a careful referee.","tokens_in":7529,"tokens_out":1713,"would_cite":false,"duration_ms":17256,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A shared five-task grading model that enforces the experts' known label co-occurrence lifts F1 by up to 55% over single-task baselines on a new 486-patch dataset.","keywords":["conjunctival melanocytic intraepithelial lesions","multi-task learning","inter-dependence loss","digital pathology","foundation model","selective supervision","grading criteria","macro F1"],"falsifier":"One decisive check: within each training batch, shuffle the one-hot labels before computing the co-occurrence prior (preserving each task's marginal distribution but destroying genuine inter-task correlations); if the model's macro F1 stays at the reported level or the dependency loss shows no effect, the claim that cross-task dependency structure drives the gains is refuted.","tokens_in":6716,"feed_emoji":"🔬","tokens_out":10725,"duration_ms":94258,"temperature":0.7,"pith_summary":"The paper tries to establish that CMIL grading is best modeled as one multi-task problem with explicit inter-task consistency, not as five independent classifications. The proposed framework uses a shared feature representation, five task-specific heads, a training scheme that updates only three heads per iteration in rotation, and a regularizer that pushes the model's predicted joint distribution of labels toward the co-occurrence statistics observed in the data. On a newly collected, multi-center dataset of 486 expert-annotated biopsy patches, it improves macro F1 over a frozen pathology foundation-model baseline by double-digit relative margins on four of five axes, with the largest gain on the most subtle criteria. If these results hold, pathologists gain a reproducible, interpretable second reader for a rare but potentially lethal disease, and the field gains a benchmark for digital ocular pathology.","feed_headline":"Joint five-axis grading beats single-task models by up to 55%","feed_subtitle":"A 486-patch multi-center study: modeling correlations among five grading criteria improves detection.","key_machinery":"The central mechanism is Shared Feature Learning with Combinatorial Partial Supervision (SFCS) plus an Inter-Dependence Loss. Five task heads share one encoder-derived representation; each training step updates only three of them, cycling through all three-head combinations. The Inter-Dependence Loss aligns the predicted joint distribution — the batch-average outer product of the active heads' probability vectors — with the empirical joint distribution of the expert one-hot labels, via KL divergence, added to the classification loss. This enforces the clinically known co-occurrence of criteria (e.g., severe atypia with vertical proliferation) while selective supervision stops any one task fr","core_discovery":"The central claim is that enforcing inter-task consistency across five interrelated grading criteria — WHO4, WHO5, horizontal spread, vertical spread, and cytologic atypia — yields substantially better predictions than predicting each criterion alone. The authors show that a multi-head architecture built on a frozen pathology encoder, trained with combinatorial partial supervision (cycling through all three-head subsets) and an inter-dependence loss that aligns the predicted joint distribution with the empirical joint distribution of expert labels, achieves mean macro F1 scores of 0.76, 0.88, 0.71, 0.76, and 0.70 across the five axes, with relative gains up to 55.1% over the strongest single","pith_inferences":["The inter-dependence loss is computed from batch-level empirical co-occurrence, so its usefulness likely depends on the training distribution matching the target population; on datasets with different grade prevalence, the enforced dependencies could become misleading rather than helpful.","Selective supervision (three of five heads per step) may also function as implicit task-level ensembling or dropout; comparing it against task-dropout or gradient-blending baselines would clarify its mechanism beyond regularization.","If the co-occurrence prior reflects a property of the disease rather than one center's staining protocol, the same loss could serve as a domain-adaptation anchor when transferring across institutions with different class distributions.","Given the dataset's small size and high-grade predominance, the strongest reported gain (WHO4, +55%) should be validated on a larger external cohort before drawing clinical conclusions; the paper itself does not claim such validation."],"forward_implications":["If the reported gains hold, computational CMIL grading becomes feasible with a reproducible multi-center benchmark that other labs can extend.","The largest gains cluster on vertical spread and cytologic atypia, suggesting that explicit inter-criterion consistency is the key to detecting the most subtle, high-risk features.","The ablation numbers imply a practical recipe: a frozen pathology encoder plus selective multi-head supervision plus a dependency regularizer yields roughly 10% higher macro F1 than the strongest single-task baseline, with each component contributing additively.","The framework offers a template for other multi-criteria histopathology grading tasks where diagnostic axes are known to be correlated, not just for ocular lesions.","Because the model outputs coherent multi-criteria predictions, it could serve as a decision-support tool that flags internally inconsistent gradings for pathologist review."],"fun_headline_variants":["Five-axis consistency lifts CMIL grading 55% over single-task","Joint five-criteria model beats single-task by 55% in CMIL","Inter-task consistency boosts CMIL F1 scores by up to 55%","Multi-center study: shared five-axis learning improves CMIL grading 55%"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that batch-level co-occurrence statistics of one-hot labels in this 486-sample, class-imbalanced dataset are a stable, unbiased estimate of true inter-criterion relationships; if they are noisy or skewed by grade prevalence, the dependency loss will enforce spurious correlations and the claimed gains could vanish.","fun_headline_variants_meta":{"raw":{"variants":["Five-axis consistency lifts CMIL grading 55% over single-task","Joint five-criteria model beats single-task by 55% in CMIL","Inter-task consistency boosts CMIL F1 scores by up to 55%","Multi-center study: shared five-axis learning improves CMIL grading 55%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000453,"raw_usage":{"total_tokens":2102,"prompt_tokens":720,"completion_tokens":1382,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":464,"completion_tokens_details":{"reasoning_tokens":1301}},"tokens_in":464,"tokens_out":1382,"duration_ms":13328,"temperature":1.0,"reasoning_tokens":1301,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T13:47:48.576770+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"One decisive check: within each training batch, shuffle the one-hot labels before computing the co-occurrence prior (preserving each task's marginal distribution but destroying genuine inter-task correlations); if the model's macro F1 stays at the reported level or the dependency loss shows no effect, the claim that cross-task dependency structure drives the gains is refuted.","supporting_citations":[],"review_version":1}