{"id":"e26fba7b-d702-43ba-bf44-9490627a46f5","arxiv_id":"2501.19048","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"Graph-based WSI classifiers match or outperform plain MIL under domain shift, but interventional training does not help graph models, and the paper's claim that graphs alone beat interventional training is contradicted by its own Camelyon17 results.","lead":"This paper tests whether graph-based models or causal 'interventional training' make whole-slide cancer image classifiers more robust to domain shifts between hospitals. It reports that interventional training does not improve graph-based models, but a non-graph model with interventional training matches or beats them on the Camelyon17 dataset.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Central claim 'graph-based models alone outperform models enhanced with interventional training' is not supported by Table 2: the key differences are within one standard deviation, no significance tests are reported, and on Camelyon17 ABMIL+IT (0.852) beats the best graph-only model (0.837).","rationale":"The paper is a useful comparative study of graph construction methods, MIL aggregators, and interventional training, and the code release is a genuine contribution. However, the central claim is an empirical comparative statement that depends entirely on Table 2, and that table is not statistically equipped to support it. The AUC differences between PatchGAT-ABMIL with and without interventional training are 0.013 and 0.014, far smaller than the reported standard deviations, and no significance testing is reported. On Camelyon17, the same table shows ABMIL with interventional training outperforming the best graph-only model, which directly contradicts the unqualified version of the claim. The paper's own cluster-purity analysis, where K-means centroids are cleanly separated into normal and tumor classes, further suggests that the interventional module may be adjusting for class structure rather than staining confounders, but the statistical weakness is sufficient and prior. The reader's verdict of REJECT is appropriate because the headline claim is not supported by the evidence as presented; the remedy is a revised, more modest claim rather than dismissal of the entire comparative study.","tokens_in":16371,"tokens_out":4190,"duration_ms":41356,"concrete_test":"Using the released code, rerun the configurations in Table 2 on Camelyon16 and Camelyon17, recording per-fold AUCs. Compute paired significance tests for the three decisive comparisons: (a) PatchGAT-ABMIL vs PatchGAT-ABMIL w/IT, (b) PatchGAT-ABMIL vs ABMIL w/IT, and (c) ABMIL vs ABMIL w/IT. Use DeLong tests for paired ROC curves or paired bootstrap/permutation over the five folds, and report p-values and 95% confidence intervals for the AUC differences. If comparison (a) is non-significant and comparison (b) favors ABMIL w/IT, the headline conclusion 'graph-based models alone outperform models enhanced with interventional training' must be withdrawn or revised to a claim about the absence of benefit from this particular interventional module on graph models.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The conclusion in the abstract and Section 6 rests entirely on the comparisons in Table 2, but that table does not establish the claimed superiority. For the decisive contrast, PatchGAT-ABMIL versus PatchGAT-ABMIL w/IT, the AUC differences are only 0.013 on Camelyon16 (0.923 vs 0.910) and 0.014 on Camelyon17 (0.837 vs 0.823), while the reported standard deviations are 0.06/0.04 and 0.04/0.07 respectively. No paired significance test, confidence interval, or multi-seed analysis is provided, so these differences are statistically indistinguishable from noise. Furthermore, the unqualified statement that graph-based models outperform models enhanced with interventional training is contradicted by the same table on Camelyon17, where ABMIL w/IT reaches AUC 0.852±0.03, numerically higher than the best graph-only model, PatchGAT-ABMIL at 0.837±0.04. On Camelyon16, ABMIL w/IT (0.920±0.04) and PatchGAT-ABMIL (0.923±0.06) are effectively tied. The data support at most the weaker claim that interventional training did not help graph-based models in these experiments; they do not support the causal-sounding conclusion that graph structure alone provides the generalization anticipated from interventional training.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies whole-slide image classification with MIL, graph-based MIL, and interventional training via backdoor adjustment. It introduces GMIL-IT, a pipeline that combines feature extraction, graph construction (patch, region, centroid graphs), GNN backbones (GCN, GAT), MIL aggregators (ABMIL, DSMIL), and an IBMIL-style interventional training stage. Experiments on Camelyon16 and Camelyon17 compare these configurations under explicit domain shifts and report that patch graphs outperform other graph constructions, that graph-based models generally improve over plain MIL, and that interventional training harms graph-based models while helping plain MIL. The paper concludes that graph-based models alone achieve the generalization initially expected from interventional training, and that graph structure provides inherent robustness to confounders. Code is publicly available.","tokens_in":16719,"tokens_out":6286,"duration_ms":59464,"significance":"The question addressed is relevant: whether spatial graph structure in WSI classifiers can replace or reduce the need for causal interventional training is a useful empirical question for computational pathology. The paper contributes a broad comparison of graph construction methods and a reproducible implementation of IBMIL-style backdoor adjustment. The Camelyon17 evaluation is explicitly set up as a leave-one-center-out domain-shift test, which is a strength. Code release and the use of standard public datasets support reproducibility. However, the headline claim is not supported by the reported numbers: interventional training substantially improves plain ABMIL, and on Camelyon17 ABMIL with interventional training outperforms the best graph-only model. The central contribution therefore requires substantial re-framing rather than a small local fix.","major_comments":[{"comment":"The central claim that \"graph-based models alone outperform models enhanced with interventional training\" is contradicted by the paper's own Table 2. On Camelyon17, ABMIL with interventional training reaches AUC 0.852±0.03, higher than the best graph-only model, PatchGAT-ABMIL, at 0.837±0.04; on Camelyon16 the two are effectively tied (0.920±0.04 vs 0.923±0.06). The data support at most the weaker statement that interventional training did not improve graph-based models in these experiments. The abstract, contribution list, and Section 6 need to be rewritten to state that weaker claim.","section":"Abstract; Section 6; Table 2"},{"comment":"No statistical significance testing is reported for the key comparisons. The differences between PatchGAT-ABMIL and PatchGAT-ABMIL w/IT are 0.013 on Camelyon16 and 0.014 on Camelyon17, with standard deviations between 0.04 and 0.07, and only 5 cross-validation folds are used. These differences are within one standard deviation and cannot support the conclusion of a \"consistent decline\" caused by interventional training. Paired tests, confidence intervals, or multi-seed results are needed before claiming that interventions harm graph-based models.","section":"Table 2; Section 5.3"},{"comment":"The confounder dictionary is built by K-means on the model's own bag embeddings, and the paper reports cluster purity 1.0 with centroids corresponding to normal and tumor classes. This indicates that the clusters encode class structure, not visual confounders such as stain variation, so the backdoor adjustment in Eqs. (7)–(9) may not implement a valid causal intervention. Without evidence that the clusters correspond to genuine confounders (for example, by comparing cluster membership against hospital or staining labels), the causal interpretation of the negative results for graph models is not established. Please validate the confounder clusters or explicitly limit the conclusions to the observed behavior of this particular IBMIL-style training procedure.","section":"Section 3.3; Section 5.3; Figures 4/5"}],"minor_comments":[{"comment":"The first equality in Eq. (6) is not correct as written: P(Y|do(X)) = P(Y|X) Σ_i P(c_i) simplifies to P(Y|X), not to Σ_i P(Y|X,c_i)P(c_i). The correct backdoor adjustment is the second expression, and the text should be corrected to avoid a formal error in the central formula.","section":"Eq. (6)"},{"comment":"The text refers to \"this dissertation\" and \"this thesis,\" but the manuscript is presented as a conference paper; please use consistent terminology such as \"this paper.\"","section":"Abstract; Section 7"},{"comment":"There are typos: \"explanability\" should be \"explainability,\" and \"strataci\" should be \"strata.\"","section":"Section 3.2; Section 3.3"},{"comment":"The statement that \"standard deviations for MIL w\\IT are generally larger\" is ambiguous because Table 2 shows ABMIL w/IT has smaller standard deviations than ABMIL without IT (e.g., 0.04 vs 0.21 for Camelyon16 AUC); please specify the comparison group clearly.","section":"Section 5.3"},{"comment":"Several rows of Table 1 report identical Accuracy, Recall, and AUC values (e.g., the Region-GCN and Centroid-GCN rows). Please clarify whether this is a rounding artifact or a consequence of the evaluation setup, since identical values across three distinct metrics are otherwise surprising.","section":"Table 1"},{"comment":"The caption of Figure 6 refers to GCN + ABMIL, while Section 5.4 and the surrounding text refer to GMIL and GAT-based models; please align the labels in the figure and text.","section":"Figure 6"}],"recommendation":"major_revision","confidential_remarks":"The MMIST-ccRCC validation reported in Section 7 cites Mota et al. (2024), a paper that includes two of the current co-authors. This is not necessarily improper, but the validation dataset is not independent of the author group, so the claim that it \"reinforces the original claims\" should be treated with caution. The deeper issue is that the paper's main conclusion overturns its own data; a revision that reframes the central claim as \"interventional training did not help graph-based models in these experiments\" would be honest and publishable, but the current framing is not."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: this is a systematic comparison that is worth having, but the advertised conclusion is not supported by the paper's own numbers. The paper compares Patch-, Region-, and Centroid-Graphs with GCN/GAT backbones and ABMIL/DSMIL pooling, with and without IBMIL-style backdoor adjustment, on Camelyon16 and Camelyon17 with an enforced domain shift. That is genuine work, the code is released, and the result that patch-graphs beat compact graph representations by a wide margin is credible.\n\nThe problem is the claim in the abstract and conclusion: graph-based models alone outperform models enhanced with interventional training. Table 2 does not establish that. On Camelyon16, PatchGAT-ABMIL with and without IT differ by 0.013 AUC with overlapping standard deviations; on Camelyon17 the difference is 0.014. No significance test, confidence interval, or multi-seed analysis is reported. Worse, the broad wording is contradicted on Camelyon17, where ABMIL+IT reaches 0.852 AUC, numerically higher than the best graph-only model (PatchGAT-ABMIL at 0.837). On Camelyon16 the same comparison is a tie. The data support at most the weaker claim that interventional training did not help graph-based models in these experiments. They do not support the causal-sounding conclusion that graph structure alone provides the generalization anticipated from interventional training.\n\nThe more interesting finding is actually the opposite one: interventional training substantially helped the non-graph baselines (ABMIL AUC +0.209 on Camelyon16, +0.062 on Camelyon17), while slightly hurting the graph models. That deserves to be the headline, but the paper does not frame it that way.\n\nThe confounder dictionary is also under-validated. The authors admit the causal graph is unknown and that the K-means clusters 'likely represent visual biases,' but their own t-SNE and purity analysis shows the clusters separate classes and have purity 1.0, with centroids labeled normal and tumor. That suggests the backdoor adjustment is conditioning on class-related structure, not stain artifacts, which makes the negative result for graph models hard to interpret. The MMIST-ccRCC validation in the Limitations section is mentioned without detail; those results should be in the main text if they are being used as supporting evidence.\n\nFor whom is this useful? Practitioners choosing graph construction strategies for WSI classification will get a reasonable comparison. Researchers working on causal or interventional MIL should read it as a cautionary example, not as a negative verdict on interventional training.\n\nRecommendation: send it to a serious referee, but expect major revision. The authors need to soften the central claim, add paired significance tests or repeated seeds, and either validate the confounder construction or reinterpret the results in light of their own class-separated clusters.","headline":"Useful graph-MIL comparison undermined by an overreaching central claim that Table 2 does not support; the more defensible result is that interventional training helps non-graph MIL but not graph models.","tokens_in":17260,"tokens_out":3482,"would_cite":true,"duration_ms":36115,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that graph-based MIL models alone match or exceed the generalization gains that interventional training provides, and that adding the interventional module to graph models usually reduces performance.","keywords":["Whole Slide Images","Multiple Instance Learning","Graph Neural Networks","Interventional Training","Backdoor Adjustment","Domain Shift","Camelyon17","Breast Cancer Metastasis Detection"],"falsifier":"Compute the purity of the K-means confounder clusters against slide-level labels and against hospital or scanner provenance on both datasets, since the paper already reports class purity 1.0 for GAT-ABMIL; if clusters align with labels rather than scanners or stain groups, rerun interventional training with confounders defined by stain statistics or scanner identity, and if graph models still beat interventional training the claim stands, while if interventional training then improves, the claim fails.","tokens_in":16151,"feed_emoji":"🔬","tokens_out":8067,"duration_ms":75834,"temperature":0.7,"pith_summary":"Whole-slide cancer images are classified from thousands of patches, and models are vulnerable to spurious visual biases such as staining and scanner differences. This paper tries to show that giving a multiple-instance model an explicit spatial graph of the slide, where nodes are patches and edges connect neighboring tissue, provides the domain-shift stability that interventional (causal) training was designed to add. Comparing graph construction methods, GNN backbones, and MIL pooling on Camelyon16 and Camelyon17, it finds that graph-based models alone match or beat their counterparts augmented with interventional training, and that added interventions usually lower graph-model performance and spread attention. If that holds, simpler graph-MIL pipelines are sufficient for cross-center generalization, and the extra confounder-modeling stage may be unnecessary for spatially structured models.","feed_headline":"Graphs alone beat interventional training for WSI classification","feed_subtitle":"Spatial graph structure gives the domain-shift stability that causal deconfounding was meant to add, in Camelyon16/17 tests.","key_machinery":"The load-bearing object is the patch-graph: each patch of the WSI is a node carrying a 1024-dimensional ResNet-50 embedding, and edges connect spatially adjacent patches, defined by patch coordinates rather than feature similarity. On top of this graph sit GCN or GAT layers, a MIL pooling (ABMIL or DSMIL) that aggregates node embeddings into one bag vector, and, in GMIL-IT, an interventional stage that clusters bag embeddings with K-means into a confounder dictionary $C$ and applies the backdoor adjustment $P(Y|\\mathrm{do}(X)) = \\sum_i P(Y|X, h(X,c_i))P(c_i)$ with an attention function $h$ and fusion $B \\oplus \\sum_i \\alpha_i c_i P(c_i)$. The graph is what carries the argument: it preserves spatial context, and the paper argues that its attention mechanism filters spurious features so thoroughly that the confounder dictionary ends up encoding class separation instead of adding robustness.","core_discovery":"On the paper's own terms, the central discovery is that spatial structure, not the interventional module, is what buys robustness: PatchGAT-ABMIL without interventional training reaches AUC 0.923 on Camelyon16 and 0.837 on Camelyon17, while adding interventional training lowers those numbers to 0.910 and 0.823. The paper reports that for the GAT-based graph models interventional training decreases performance across nearly all evaluated metrics and datasets, with GCN results mixed, while the same intervention substantially helps the non-graph ABMIL baseline, raising its Camelyon16 AUC from 0.711 to 0.920. It interprets this as graph attention acting like an information bottleneck, filtering irrelevant visual features so that the K-means confounder clusters align with classes rather than adding useful causal adjustment.","pith_inferences":["Because the reported cluster purity is 1.0, meaning confounder clusters equal classes, the K-means dictionary built from bag embeddings may be encoding the target label rather than stain or scanner confounders; if so, the negative result is about confounder misspecification, not about graph models being immune to all interventions.","A decisive extension would build the confounder dictionary from stain statistics, scanner identity, or hospital provenance instead of bag embeddings; if graph models still outperform interventional training under those true confounders, the claim that spatial structure substitutes for causal adjustment is much stronger.","Frontdoor adjustment, which does not require observing confounders, becomes an obvious next test for graph-MIL, since graph models may already learn invariant features and could avoid the misspecification that backdoor adjustment suffers here.","These results concern spatial artifacts such as staining and scanner differences in breast-cancer lymph-node slides; non-spatial biases like marker pen or air bubbles may still need interventional training even in graph models, because graph adjacency cannot filter what is not localized spatially."],"forward_implications":["Graph-based MIL alone can give the cross-center generalization that motivated interventional training, so the confounder dictionary and backdoor module can be dropped for graph models without sacrificing AUC.","Patch-level graphs outperform region- and centroid-level graphs, so graph construction that keeps original patch information is preferable for WSI classification.","A domain-shift protocol that holds out entire medical centers, as in Camelyon17, exposes robustness differences that standard shuffled cross-validation on Camelyon16 hides.","Interventional training still helps non-graph MIL, especially ABMIL, showing that deconfounding is architecture-dependent rather than universally beneficial.","Attention heatmaps indicate interventions blur spatial focus, so causal adjustment may actively disrupt the spatial reasoning graph models use."],"supporting_citations":[{"why":"Supplies the IBMIL-style backdoor-adjustment interventional training framework that the paper adopts for its confounder dictionary.","marker":"Lin et al., 2023"},{"why":"Provides the ABMIL attention pooling used as a MIL aggregator and as a baseline model.","marker":"Ilse et al., 2018"},{"why":"Provides the DSMIL dual-stream pooling used as a second MIL aggregator and baseline model.","marker":"Li et al., 2021"},{"why":"Defines patch-graph adjacency by patch coordinates, the Patch-Graph construction the paper adopts.","marker":"Chen et al., 2021"},{"why":"Defines the GCN propagation used in the graph experiments.","marker":"Kipf and Welling, 2017"},{"why":"Defines the GAT attention layers and is invoked to explain graph robustness through an information-bottleneck argument.","marker":"Veličković et al., 2018"},{"why":"Defines the patching scheme, magnification, and ResNet-50 feature extraction used for all compared models.","marker":"Lu et al., 2021"},{"why":"Provides the Camelyon16 dataset and labels used for the first evaluation setting.","marker":"Bejnordi et al., 2017"},{"why":"Provides the Camelyon17 dataset with its multi-center, multi-scanner domain shifts used for the second evaluation setting.","marker":"Bándi et al., 2019; Litjens et al., 2018"}],"fun_headline_variants":["Graph structure, not causal training, drives WSI robustness","Spatial graphs beat causal deconfounding in WSI classification","Graph-MIL: why spatial links trump interventional training","For WSI, graph attention alone handles domain shift best","Interventional training backfires on graph-based WSI models"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Interventional training is valid only if the K-means clusters of bag embeddings really capture visual biases like stain variation; if those clusters instead capture tumor class or other true signal, the backdoor adjustment is not a genuine intervention and the comparison says little about whether causal deconfounding helps graph models.","fun_headline_variants_meta":{"raw":{"variants":["Graph structure, not causal training, drives WSI robustness","Spatial graphs beat causal deconfounding in WSI classification","Graph-MIL: why spatial links trump interventional training","For WSI, graph attention alone handles domain shift best","Interventional training backfires on graph-based WSI models"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000608,"raw_usage":{"total_tokens":2830,"prompt_tokens":941,"completion_tokens":1889,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":557,"completion_tokens_details":{"reasoning_tokens":1806}},"tokens_in":557,"tokens_out":1889,"duration_ms":12356,"temperature":1.0,"reasoning_tokens":1806,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T21:31:00.962470+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the purity of the K-means confounder clusters against slide-level labels and against hospital or scanner provenance on both datasets, since the paper already reports class purity 1.0 for GAT-ABMIL; if clusters align with labels rather than scanners or stain groups, rerun interventional training with confounders defined by stain statistics or scanner identity, and if graph models still beat interventional training the claim stands, while if interventional training then improves, the claim fails.","supporting_citations":[{"cited_title":"Tomczak, and Max Welling","cited_arxiv_id":null,"evidence_quote":"Provides the ABMIL attention pooling used as a MIL aggregator and as a baseline model."},{"cited_title":"Van Der Laak, Meyke Hermsen, Quirine F","cited_arxiv_id":null,"evidence_quote":"Provides the Camelyon16 dataset and labels used for the first evaluation setting."}],"review_version":1}