REVIEW 4 major objections 5 minor 48 references
Integrating Implicit and Explicit Relational Biases through Graph-Based Multiple Instance Learning: A Case Study in Skin Lesion Diagnosis
T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Combining self-supervised patch features with graph attention outperforms whole-image CNNs and attention-only multiple instance learning for skin lesion diagnosis, reaching 79.27% balanced accuracy on ISIC-2018.
desk verdict Systematic comparison of graph topologies for patch-based MIL, but a load-bearing ambiguity about whether SSL pretraining included official test images makes the numbers uninterpretable as evidence for relational modelling. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the bag-of-patches graph. Each image is cut into 14×14 = 196 non-overlapping 16×16 pixel patches, each encoded by a frozen convolutional masked autoencoder (ConvMAE) into a 768-dimensional vector. These vectors act as nodes in a graph whose edges come from one of three topologies: random, grid (4- or 8-neighbourhood spatial adjacency), or k-nearest neighbours in feature space. Message passing via graph attention networks refines node features, and the same attention-based MIL pooling used in the baseline aggregates them into an image-level representation. This design lets the authors attribute any improvement to the explicit relational step rather than to a different aggregation mechanism.
What would settle it
Retrain the ConvMAE encoder on only the ISIC-2018 and ISIC-2019 training splits, leaving the official test images out of pretraining, and re-run the AMIL and graph-AMIL evaluations on the same test sets; if the balanced-accuracy gap between graph and non-graph models shrinks materially, the reported gains came from test-set exposure rather than relational modelling. Alternatively, give the graph models the same hyperparameter tuning budget as the baseline and see whether the 2–3 point gap persists.
Extended reading notes
Core claim
The paper claims that a two-stage relational pipeline—self-supervised patch representations followed by explicit graph-based message passing—outperforms both a whole-image convolutional baseline and a patch-based attention MIL model that ignores inter-patch structure. On ISIC-2018, the best configuration (a 4-neighbourhood grid graph processed by a graph attention network) reaches 79.27% balanced accuracy, up from 76.17% for EfficientNet-B3 and 77.12% for attention MIL on ConvMAE embeddings. On ISIC-2019, a k-nearest-neighbour graph with GAT reaches 60.67% versus 59.84% for attention MIL alone. The authors interpret the large drop to 34.28% when GNN weights are frozen as evidence that learned message passing, not added capacity, drives the gain.
Load-bearing premise
The whole comparison hinges on the assumption that the official test images were never seen during self-supervised pretraining—the paper excludes label leakage but not image-level leakage—and that the graph models' extra hyperparameter tuning did not create the observed accuracy gap.
Editorial extensions
If this is right
- A whole-image convolutional baseline can be improved simply by switching to self-supervised patch embeddings with attention MIL (77.12% versus 76.17% on ISIC-2018).
- Adding explicit graph structure yields a further gain, with the best ISIC-2018 model (grid graph, 4-neighbourhood, GAT) reaching 79.27% balanced accuracy.
- The graph topology that works best differs by dataset: grid structure wins on ISIC-2018, while a k-nearest-neighbour graph in feature space wins on ISIC-2019 (60.67% versus 59.84%).
- Frozen message passing drops balanced accuracy to roughly 34% on the grid configuration, indicating that the relational gain requires learned rather than fixed graph propagation.
- The single-model graph results are comparable to some ensemble methods on both benchmarks, suggesting that relational modelling can substitute for part of the gain achieved by averaging many networks.
Reading between the lines
- One consequence the authors leave implicit is a test-set leakage concern: the masked autoencoder was pretrained on images from both datasets, and the paper only excludes duplicate images and label use, not the official test images, so the reported accuracies could be inflated by transductive feature learning rather than relational structure.
- Because the graph-based models were tuned over additional hyperparameters (graph type, neighbourhood size, number of layers) beyond the baseline's shared set, part of the observed gap could reflect tuning budget rather than relational bias; a matched-budget comparison would settle this.
- A natural extension is to use the same two-stage recipe on other spatially structured medical images, with an anatomical or physical grid defining the graph edges instead of a learned kNN graph.
- The choice between grid and kNN topology could be made adaptive per image, using spatial edges where local texture matters and feature-space edges where distant semantic similarity matters, rather than committing to one topology for an entire dataset.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper compares three approaches to skin lesion classification on the ISIC-2018 and ISIC-2019 benchmarks: a whole-image EfficientNetB3 baseline; a patch-based attention MIL (AMIL) model using frozen self-supervised ConvMAE embeddings; and graph-augmented AMIL variants that explicitly propagate information over random, grid, or kNN graphs via GNNs. The reported balanced accuracies on the official test sets are 76.17% for the CNN baseline, 77.12% for AMIL, and 79.27% for the best grid-GAT configuration on ISIC-2018; on ISIC-2019 the paper reports 59.84% for AMIL and 60.67% for a kNN-GAT model. The central claim is that adding explicit relational modelling to implicit patch representations yields further gains over both whole-image CNNs and attention-only MIL.
Significance. If the empirical results are trustworthy, the paper provides a useful case study on combining self-supervised patch features with graph-based relational biases in a medical imaging setting. The exploration of multiple graph construction strategies and the inclusion of a frozen-GNN control are constructive design choices, and the use of two public benchmarks is a strength. However, the reported gains are modest (around 1–2 percentage points) and, as detailed below, the experimental protocol has not yet ruled out a data-split contamination risk and lacks statistical significance testing. The contribution is therefore best characterized as a preliminary empirical report rather than a definitive demonstration; its value depends critically on resolving the methodological ambiguities.
major comments (4)
- [Section 2.1] The statement that "images from both datasets were combined and used jointly to train the masked autoencoder" is ambiguous, because the preceding sentence identifies each dataset as including a dedicated test set. The natural reading is that the official ISIC-2018 and ISIC-2019 test images were included in the unlabeled ConvMAE pretraining. If so, the frozen encoder has reconstructed the exact test images during pretraining, and the downstream classifiers are evaluated on features that were adapted to those images. The sentence "no label leakage occurs" addresses only annotation leakage, not image-level leakage, so it is not responsive to this concern. Please state explicitly whether the official test images were excluded from the SSL pretraining. If they were not excluded, all reported test accuracies, including the 77.12→79.27 and 59.84→60.67 improvements, are potentially inflated and do not support the relational-modelling claim. This is the most load-bearing issue in the manuscript.
- [Section 2.6 and Table 1] No statistical significance tests are reported. The ISIC-2019 gain (59.84±1.50 vs 60.67±0.68) is within the reported standard deviations, and even the ISIC-2018 gain (77.12±1.49 vs 79.27±1.38) may not be significant across the five CV folds. Because the graph models were tuned over a much larger hyperparameter space (GNN type, number of layers, hidden dimensions, graph construction strategy, connectivity variants, r, k), the reported best configuration is a selected result, not an independently tested hypothesis. Please report the number of configurations explored, perform paired per-fold significance tests (e.g., Wilcoxon signed-rank) or provide confidence intervals, and state whether any multiple-comparison correction was applied.
- [Section 2.5 and Table 1] The frozen-GNN ablation (34.28±3.39) is interpreted as evidence that "learned relational modelling, rather than augmented capacity, is a primary factor contributing to performance," but this interpretation is not supported. Random-weight message passing is expected to destroy the features regardless of whether relational structure is beneficial, and the frozen configuration was evidently not tuned with the same freedom as the trained graph models. To attribute the observed gain to explicit relational structure, the paper should include a matched-capacity non-relational control (e.g., an MLP or additional linear layer applied to the patch embeddings before attention pooling) and should compare against the random-graph result (78.24±0.86) with a significance test. The current ablation only shows that learned message passing matters for these features, not that structured message passing is the cause of the gain.
- [Sections 2.1 and 3 / Table 1] The protocol description for ISIC-2019 states that models are trained on the training split and evaluated on the dedicated test set, which implies a single random split, yet Table 1 reports mean ± standard deviation for ISIC-2019 entries (59.84±1.50 and 60.67±0.68). This is inconsistent. If the results are averaged over multiple seeds or a repeated hold-out procedure, please describe the procedure and the number of runs; if they are from a single run, the ± values should be removed or replaced with an appropriate uncertainty estimate.
minor comments (5)
- [Section 2.5] The text says the evaluation included "graph convolutional networks (GCN), graph attention networks (GAT), and graph transformer-based architectures," but Table 1 reports only GAT configurations. Please either report results for all tested GNN families or remove the unsupported mention.
- [Section 2.2] The term "convolutional masked autoencoder" is imprecise for the architecture in [4], which is a hybrid ViT/convolutional model; consider using the exact model name (ConvMAE) consistently.
- [Table 1 caption] The caption refers to P, I, and E relational indicators but does not define them. Please expand the caption or add a table note clarifying the notation and identifying which rows are from prior work.
- [References] Reference [6] cites only the GCN paper, but the manuscript also uses graph attention networks and graph transformers; these architectures require their own citations (e.g., Veličković et al., 2018 and Shi et al., 2021).
- [Miscellaneous] The running head on the title page appears as "INTEGRATINGIMPLICIT ANDEXPLICITRELATIONALBIASES" without spaces; this is presumably a formatting artifact but should be corrected in the camera-ready version.
Circularity Check
No circular derivation: the pipeline is an empirical comparison with held-out official test sets; the main validity threat is possible image-level leakage in SSL pretraining, which is a data-hygiene risk rather than circularity.
full rationale
The paper's comparison chain is empirical rather than deductive. A frozen ConvMAE encoder produces 196 patch embeddings, and the AMIL baseline differs from Graph-AMIL only by explicit message passing before the same attention-based MIL pooling; the reported differences therefore isolate the relational mechanism under study. No reported number is obtained by definition from a fitted parameter: the encoder is selected by linear probing, but all downstream accuracies are measured on the official ISIC-2018 and ISIC-2019 dedicated test sets, and hyperparameters, including graph topology, are chosen through five-fold cross-validation on training splits. The statement that grid-GAT and kNN-GAT perform best is a model-selection outcome, not an independent prediction forced by construction; this is normal empirical reporting, though readers should not treat the best configuration as a confirmed general claim. Section 2.1's statement that 'images from both datasets were combined and used jointly to train the masked autoencoder' is a genuine data-hygiene concern if the official test sets were included in unlabeled pretraining, and the paper's response ('no label leakage occurs') does not address image-level leakage. That threat undermines external validity but is not circularity, because no equation or fitted quantity reduces the claimed accuracy gain to its own input. There are no load-bearing self-citations and no imported uniqueness theorems. The result is therefore self-contained as an empirical study, and the circularity score is 0.
Assumptions & free parameters
free parameters (6)
- ConvMAE masking ratio =
0.50
- Random graph degree r =
2
- kNN neighbourhood size k =
4
- Grid graph connectivity =
4-neighbourhood, no diagonal
- GNN family and hidden dimensions =
GAT (values not reported)
- Attention MIL hidden projection dimensions =
Not reported
assumptions (5)
- domain assumption ISIC-2018 and ISIC-2019 class labels are accurate and balanced accuracy is the appropriate metric for comparing methods under class imbalance.
- ad hoc to paper Splitting each image into a fixed 14x14 grid of 16x16 patches preserves diagnostically relevant structure.
- ad hoc to paper The self-supervised pretraining set may include all images from both datasets; if it includes test images, evaluation remains valid.
- domain assumption The graph topologies tested (random, grid, kNN) adequately span the space of useful inter-patch relations.
- standard math Attention-based MIL pooling and GNN message passing equations (Eqs. 1-4) are correctly and consistently implemented.
Cite this review
Pith. "Pith review of Integrating Implicit and Explicit Relational Biases through Graph-Based Multiple Instance Learning: A Case Study in Skin Lesion Diagnosis." pith.science (2026). https://pith.science/paper/LF3QWPKD
@misc{pith2026260806037,
author = {Pith},
title = {Pith review of: Integrating Implicit and Explicit Relational Biases through Graph-Based Multiple Instance Learning: A Case Study in Skin Lesion Diagnosis},
year = {2026},
howpublished = {\url{https://pith.science/paper/LF3QWPKD}},
note = {Machine review of arXiv:2608.06037}
}
read the original abstract
Relational inductive biases are essential for capturing structural dependencies among data. This study investigates a dual-level relational framework for image classification, bridging the gap between implicit representation learning and explicit structural modelling. We begin by establishing a baseline using an EfficientNetB3 architecture. To move beyond standard convolutional biases, we adopt a patch-based strategy, employing a convolutional masked autoencoder to learn implicit inter-patch relationships through self-supervised reconstruction. We then extend this approach by incorporating explicit relational modelling, organizing the learned embeddings into various graph topologies, including grid-based, random, and k-nearest neighbour structures. Experimental results on the ISIC-2018 and ISIC-2019 skin lesion diagnosis benchmarks show that combining implicit inter-patch modelling with explicit graph-based message passing yields the best performance. On the ISIC-2018 test set, the baseline model achieves a balanced accuracy of 76.17%, which improves to 77.12% with implicit patch-based relational modelling. The fully integrated grid-structured Graph Attention Network further increases performance to 79.27%. Similarly, on ISIC-2019, the implicit approach reaches 59.84% balanced accuracy, while the combination of implicit and explicit modelling yields 60.67%.
Figures
Reference graph
Works this paper leans on
-
[1]
Self-supervised learning to increase the performance of skin lesion classification , author=. Electronics , volume=. 2020 , publisher=
work page 2020
-
[2]
Self-supervised multi-modality learning for multi-label skin lesion classification , journal =
Hao Wang and Euijoon Ahn and Lei Bi and Jinman Kim , keywords =. Self-supervised multi-modality learning for multi-label skin lesion classification , journal =. 2025 , issn =. doi:https://doi.org/10.1016/j.cmpb.2025.108729 , url =
arXiv 2025
-
[3]
International conference on machine learning , pages=
A simple framework for contrastive learning of visual representations , author=. International conference on machine learning , pages=. 2020 , organization=
2020
-
[4]
Asian Pacific Journal of Cancer Prevention: APJCP , volume=
Contrastive self-supervised ensemble transfer learning for robust skin cancer classification and early detection , author=. Asian Pacific Journal of Cancer Prevention: APJCP , volume=
-
[5]
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
Masked autoencoders are scalable vision learners , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
-
[6]
Advances in Neural Information Processing Systems , volume=
Mcmae: Masked convolution meets masked autoencoders , author=. Advances in Neural Information Processing Systems , volume=
-
[7]
Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=
Contrastive learning relies more on spatial inductive bias than supervised learning: an empirical study , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=
-
[8]
Applied Soft Computing , volume=
Medical supervised masked autoencoder: Crafting a better masking strategy and efficient fine-tuning schedule for medical image classification , author=. Applied Soft Computing , volume=. 2025 , publisher=
work page 2025
Show all 48 references
-
[9]
IEEE Transactions on Medical Imaging , year=
Advancing Volumetric Medical Image Segmentation via Global-Local Masked Autoencoders , author=. IEEE Transactions on Medical Imaging , year=
-
[10]
IEEE Transactions on Medical Imaging , year=
Mim: Mask in mask self-supervised pre-training for 3d medical image analysis , author=. IEEE Transactions on Medical Imaging , year=
-
[11]
Pattern Recognition , volume=
Global contrast-masked autoencoders are powerful pathological representation learners , author=. Pattern Recognition , volume=. 2024 , publisher=
2024
-
[12]
International Conference on Machine Learning , pages=
Attention-based deep multiple instance learning , author=. International Conference on Machine Learning , pages=. 2018 , organization=
2018
-
[13]
Pattern Recognition , volume=
Revisiting multiple instance neural networks , author=. Pattern Recognition , volume=. 2018 , publisher=
2018
-
[14]
Advances in Neural Information Processing Systems , volume=
Transmil: Transformer based correlated multiple instance learning for whole slide image classification , author=. Advances in Neural Information Processing Systems , volume=
-
[15]
Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=
Dual-stream multiple instance learning network for whole slide image classification with self-supervised contrastive learning , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=
-
[16]
arXiv preprint arXiv:1906.04881 , year=
Multiple instance learning with graph neural networks , author=. arXiv preprint arXiv:1906.04881 , year=
1906 arXiv
-
[17]
arXiv preprint arXiv:2501.19048 , year=
The Role of Graph-based MIL and Interventional Training in the Generalization of WSI Classifiers , author=. arXiv preprint arXiv:2501.19048 , year=
-
[18]
Aggregation: Evaluating Patient-Level Survival Prediction Strategies Using Graph-Based Learning , author=
MIL vs. Aggregation: Evaluating Patient-Level Survival Prediction Strategies Using Graph-Based Learning , author=. Iberian Conference on Pattern Recognition and Image Analysis , pages=. 2025 , organization=
2025
-
[19]
Advances in neural information processing systems , volume=
Imagenet classification with deep convolutional neural networks , author=. Advances in neural information processing systems , volume=
-
[20]
arXiv preprint arXiv:1609.02907 , year=
Semi-supervised classification with graph convolutional networks , author=. arXiv preprint arXiv:1609.02907 , year=
-
[21]
arXiv preprint arXiv:1710.10903 , year=
Graph attention networks , author=. arXiv preprint arXiv:1710.10903 , year=
-
[22]
arXiv preprint arXiv:1409.0473 , year=
Neural machine translation by jointly learning to align and translate , author=. arXiv preprint arXiv:1409.0473 , year=
-
[23]
Advances in neural information processing systems , volume=
Attention is all you need , author=. Advances in neural information processing systems , volume=
-
[24]
Scientific Reports , volume=
An interpretable multiple-instance approach for the detection of referable diabetic retinopathy in fundus images , author=. Scientific Reports , volume=. 2021 , publisher=
2021
-
[25]
Cancers , volume=
Attention-based deep learning system for classification of breast lesions—multimodal, weakly supervised approach , author=. Cancers , volume=. 2023 , publisher=
2023
-
[26]
Advances in neural information processing systems , volume=
Graph transformer networks , author=. Advances in neural information processing systems , volume=
-
[27]
Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=
Simmim: A simple framework for masked image modeling , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=
-
[28]
Bert: Pre-training of deep bidirectional transformers for language understanding , author=. Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers) , pages=
2019
-
[29]
Computers in biology and medicine , volume=
Data augmentation for medical imaging: A systematic literature review , author=. Computers in biology and medicine , volume=. 2023 , publisher=
2023
-
[30]
Scientific Reports , volume=
An optimized multi-task contrastive learning framework for HIFU lesion detection and segmentation , author=. Scientific Reports , volume=. 2025 , publisher=
2025
-
[31]
arXiv preprint arXiv:1902.03368 , year=
Skin lesion analysis toward melanoma detection 2018: A challenge hosted by the international skin imaging collaboration (isic) , author=. arXiv preprint arXiv:1902.03368 , year=
2018 arXiv
-
[32]
Tschandl, Philipp and Rosendahl, Cliff and Kittler, Harald , month = aug, year =. The. Scientific Data , publisher =. doi:10.7910/DVN/DBW86T , language =
-
[33]
arXiv preprint arXiv:1806.01261 , year=
Relational inductive biases, deep learning, and graph networks , author=. arXiv preprint arXiv:1806.01261 , year=
-
[34]
Journal of the American Academy of Dermatology , volume=
The ABCD rule of dermatoscopy: high prospective value in the diagnosis of doubtful melanocytic skin lesions , author=. Journal of the American Academy of Dermatology , volume=. 1994 , publisher=
1994
-
[35]
Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops , pages=
Representation learning of histopathology images using graph neural networks , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops , pages=
-
[36]
Advances in Neural Information Processing Systems , volume=
Self-supervised relational reasoning for representation learning , author=. Advances in Neural Information Processing Systems , volume=
-
[37]
2020 33rd SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI) , pages=
Superpixel image classification with graph attention networks , author=. 2020 33rd SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI) , pages=. 2020 , organization=
2020
-
[38]
Proceedings of the IEEE international conference on computer vision , pages=
Unsupervised visual representation learning by context prediction , author=. Proceedings of the IEEE international conference on computer vision , pages=
-
[39]
European conference on computer vision , pages=
Unsupervised learning of visual representations by solving jigsaw puzzles , author=. European conference on computer vision , pages=. 2016 , organization=
2016
-
[40]
arXiv preprint arXiv:1808.03426 , year=
WonDerM: Skin lesion classification with fine-tuned neural networks , author=. arXiv preprint arXiv:1808.03426 , year=
-
[41]
Medical image analysis , volume=
Analysis of the ISIC image datasets: Usage, benchmarks and recommendations , author=. Medical image analysis , volume=. 2022 , publisher=
2022
-
[42]
Pattern Recognition , volume=
Explainable skin lesion diagnosis using taxonomies , author=. Pattern Recognition , volume=. 2021 , publisher=
2021
-
[43]
Computer Methods and Programs in Biomedicine , volume=
Transfer learning using a multi-scale and multi-network ensemble for skin lesion classification , author=. Computer Methods and Programs in Biomedicine , volume=. 2020 , publisher=
2020
-
[44]
arXiv preprint arXiv:2010.11929 , year=
An image is worth 16x16 words: Transformers for image recognition at scale , author=. arXiv preprint arXiv:2010.11929 , year=
2010 arXiv
-
[45]
IEEE Access , volume=
Accuracy is Not Enough: Multi-Head Gated Attention Multiple Instance Learning Model for Explainable Breast Cancer Diagnosis , author=. IEEE Access , volume=. 2025 , publisher=
2025
-
[46]
2025 29th International Conference on Methods and Models in Automation and Robotics (MMAR) , pages=
Multi-Head Attention Multiple Instance Learning Deep Neural Classifier Enhanced with Model Uncertainty Quantification , author=. 2025 29th International Conference on Methods and Models in Automation and Robotics (MMAR) , pages=. 2025 , organization=
2025
-
[47]
IEEE Transactions on Pattern Analysis and Machine Intelligence , volume=
A survey on self-supervised learning: Algorithms, applications, and future trends , author=. IEEE Transactions on Pattern Analysis and Machine Intelligence , volume=. 2024 , publisher=
2024
-
[48]
IEEE Access , year=
An ensemble classifier based on diverse convolutional neural networks for skin lesions classification , author=. IEEE Access , year=
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.