REVIEW 5 major objections 5 minor 31 references
Hypergraph Tversky-Aware Domain Incremental Learning for Brain Tumor Segmentation with Missing Modalities
T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read The paper claims that treating each MRI modality as an incremental-learning domain, with replay and Tversky-aware contrastive alignment, yields state-of-the-art brain tumor segmentation under missing-modality conditions.
desk verdict Plausible DIL+hypergraph combination for missing-modality segmentation, but the SOTA claim rests on an uncontrolled test-time batch-dependent inference. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
Three mechanisms carry the argument. The first is the domain-incremental replay schedule: each MRI modality becomes a new training stage, weights carry over from the previous stage, and a replay buffer built from the median-loss samples of each stage keeps earlier modalities from being forgotten. The second is the Tversky-Aware Contrastive (TAC) loss, which defines positive and negative pairs across the replay queue and the current-modality queue and measures similarity with the Tversky index (a Dice-like overlap measure with separate false-positive and false-negative weights) instead of cosine similarity; with $\beta>\alpha$ it penalizes false negatives more heavily, counteracting the small size of tumor regions. The third is the Cross-Patient Hypergraph module, in which flattened encoder features become vertices of a KNN-built hypergraph, a hypergraph convolution propagates high-order associations among patients, and the updated features are fused back into the decoder. The Tversky-based Dice and Focal losses supply the pixel-level intra-modality regularization, and $\omega=0$ for the first modality followed by $\omega=1$ later switches on the contrastive term only once a second modality exists.
What would settle it
Retrain the five baselines (mmFormer, UNet-MFI, M3AE, PASSION, M2FTrans) under ReHyDIL's exact protocol—2D slices at 224x224, same 8:1:1 patient split, same optimizer and epoch schedule—and compare mean Dice on BraTS2019. If their numbers match previously published 3D results instead of the table, the claimed 41-of-45 superiority is not established. A second check: measure per-stage DSC on earlier modalities after each new modality is added; if the replay buffer prevents forgetting, old-modality scores should not drop more than a small threshold.
Extended reading notes
Core claim
The central claim is that Replay-based Hypergraph Domain Incremental Learning makes missing-modality brain tumor segmentation more accurate than methods that assume all four MRI modalities are present during training. Modalities are introduced as separate domains in the order T1, T2, FLAIR, T1CE; after each stage the 10% of training samples closest to the median loss are stored in a replay buffer, and training on later modalities mixes those replayed samples with the current ones under the Tversky-Aware Contrastive loss. The loss treats predictions from different patients and different modalities but the same tumor region as positive pairs, using Tversky similarity with a stronger penalty on false negatives, and it is supplemented by a Tversky-based Dice loss and a Tversky-based Focal loss. The Cross-Patient Hypergraph Segmentation Network, a U-Net whose two deepest encoder stages are augmented by hypergraph convolutions over patient-derived vertices, supplies the segmentation backbone. The authors report state-of-the-art mean Dice on BraTS2019 and note that the replay buffer adds memory overhead and that the framework is currently limited to four MRI modalities with similar image characteristics.
Load-bearing premise
The superiority claim holds only if the five baselines were trained and evaluated under the same 2D, 224x224, 8:1:1 patient-split protocol as ReHyDIL, which the paper does not explicitly state.
Editorial extensions
If this is right
- If the reported numbers hold, a single incrementally trained CHSNet can segment brain tumors from any subset of the four modalities at test time without retraining for each missing pattern.
- The largest relative gains are in tumor core and enhancing tumor, the smallest and most imbalanced regions, which is consistent with the TAC loss and Tversky regularization targeting false negatives.
- Because the replay buffer stores 10% of each stage's training data, memory overhead grows with the number of modalities, a trade-off the paper explicitly acknowledges.
- The comparison implies that methods trained on all modalities at once and then masked at test time leave performance on the table relative to a model that has seen modalities arrive sequentially.
Reading between the lines
- Inference: the headline comparison is only as fair as the baseline protocol; rerunning mmFormer, UNet-MFI, M3AE, PASSION, and M2FTrans under ReHyDIL's exact 2D, 224x224, 8:1:1 split would settle whether the reported gains are protocol effects rather than method effects.
- Inference: if the hypergraph's cross-patient edges are doing real work, then changing the batch constraint so that samples can come from the same patient, or shuffling patients between stages, should degrade segmentation, especially for tumor core and enhancing tumor.
- Inference: the beta/alpha imbalance is tuned on BraTS2019; a similar method applied to other small-structure segmentation tasks would need to re-tune this ratio, since the paper itself shows beta=1.6 collapses performance.
- Inference: the same 'modality as domain' treatment could extend beyond MRI contrasts to sequential follow-up scans or other imaging protocols, but that requires testing beyond four modalities.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes ReHyDIL, a replay-based domain-incremental learning framework for brain tumor segmentation with missing MRI modalities. It combines a U-Net with cross-patient hypergraph modules (CHSNet), a Tversky-based contrastive loss (TAC), and Tversky focal/Dice losses. On BraTS2019, the authors report average DSC of 87.78% (WT), 79.49% (TC), and 66.16% (ET) across 15 missing-modality combinations, claiming superiority over four baselines in 41 of 45 cases with p<0.05 in 39 cases.
Significance. If the comparison is fair and the test-time protocol is well specified, this is a practically valuable contribution: it recasts missing-modality segmentation as a domain-incremental learning problem, provides an open-source implementation, and offers a clear ablation study of the proposed components. The framing and the Tversky-aware contrastive loss are interesting and go beyond the common fixed-modality-set assumption. However, the current manuscript does not establish that the comparison is controlled, primarily because of the batch-dependent hypergraph at inference and the ambiguity about baseline retraining. The significance of the reported state-of-the-art numbers therefore cannot yet be assessed.
major comments (5)
- [Section 2.2 and Section 3 (Quantitative Results)] The Cross-Patient Hypergraph is constructed from the flattened encoder output of the entire batch, with N = B×H×W vertices and KNN-based hyperedges that connect vertices across patients. The paper states a constraint that training batches contain different patients, but it never specifies how test batches are formed. Consequently, a given test patient's segmentation depends on which other patients are co-batched, i.e., the method is transductive at inference. None of the compared baselines have this property, so the Table 1 comparison is uncontrolled even if all models were retrained under the same protocol. Please specify the test-time batch construction (e.g., single-patient batches or a fixed batch composition) and either demonstrate invariance to batch composition or rerun the comparison under a protocol that matches the baselines.
- [Section 3 (Experimental Setup vs Quantitative Results)] The manuscript says 'All models were trained using PyTorch...' but it is not stated explicitly that mmFormer, UNet-MFI, M3AE, PASSION, and M2FTrans were all retrained under the identical 2D-slice, 224×224, 8:1:1 patient-split protocol. If baseline numbers are taken from prior publications that use different pipelines (e.g., 3D volumes, other splits, or different preprocessing), the reported gains—3.13% over MFI for WT, 2.53% over M3 for TC, 7.16% over M+P for ET—are not meaningful. Please state unambiguously which numbers come from re-runs under the same protocol and make the evaluation pipeline public for all methods.
- [Section 3 (Ablation Study) and Table 2] The hyperparameters α, β, γ, and the number of CPH layers are selected via the ablation study in Table 2, but the text does not state whether this selection was performed on the validation split or on the test set. If the test set (5,270 slices) was used for model selection, the paired t-test p-values reported in Table 1 are not valid significance tests. Please clarify the hyperparameter selection procedure and, if necessary, re-evaluate the final configuration on a properly held-out test set.
- [Abstract and Section 1] The abstract claims 'an improvement of over 2% in the Dice Similarity Coefficient across various tumor regions,' but the introduction reports 1.05% for WT, 2.53% for TC, and 1.52% for ET. These numbers are inconsistent: the abstract overstates the actual improvements for WT and ET. Please correct the abstract so that it matches the reported results.
- [Section 2.1 and Section 3 (Experimental Setup)] The DIL procedure trains each stage on a single modality plus replay samples from previous stages, so the final model never sees multi-modal input during training. Nevertheless, Table 1 includes multi-modal input combinations (e.g., T2+T1c+T1, T2+T1c+FLAIR). The paper does not explain how multi-modal inputs are represented or why the model is expected to generalize to inputs that are out-of-distribution relative to its training. This makes the multi-modal rows of Table 1 difficult to interpret and weakens the claim of robustness across arbitrary missing-modality patterns. Please describe the input representation (e.g., zero-padding missing channels) and, if needed, justify or retrain under a protocol that includes multi-modal examples.
minor comments (5)
- [Section 2.2] The abstract and introduction say that 'each patient is represented as a vertex,' but the method in Section 2.2 defines vertices as flattened pixel-level feature vectors (N = B×H×W). Please reconcile this terminology: the hypergraph appears to connect pixel-level features across patients, not patient-level representations.
- [Section 2.2] The citation for HGNN is given as reference [12], which is the M3AE paper by Liu et al. This appears to be a citation error; the intended hypergraph neural network reference is missing.
- [Section 2.3, Eq. (4)] In Eq. (4), the symbols g and u are said to denote 'two different predictions,' but their roles in Eq. (3) are not made explicit. Please define them clearly, especially which argument is treated as the anchor and which as the paired sample.
- [Section 3 (Experimental Setup)] The description of the paired t-test is terse: it says the method is compared with the second-best method, but does not state whether the test is over patients, slices, or multiple runs. Please specify the unit of analysis and the number of samples used in the test.
- [Section 3 (Experimental Setup)] The paper sets ω=0 for D1 and ω=1 for all subsequent modalities, meaning the TAC loss is absent in the first training stage. This choice is not justified; since the replay buffer is built from the first stage, the rule used to select replay samples in later stages may depend on it. Please add a brief justification or ablation.
Circularity Check
No significant circularity: ReHyDIL's improvements are empirical results from a self-contained training and evaluation pipeline, with no derivation step that reduces to its assumptions.
full rationale
This is an empirical engineering paper, not a formal derivation, and I found no load-bearing step that is circular by the paper's own equations or by self-citation. The method's components (replay-based DIL, hypergraph feature update in Eq. 2, Tversky-aware contrastive loss in Eq. 3, and the combined loss in Eq. 6) are defined independently of the reported DSC numbers; none of these formulas assumes the outcome it is used to explain. The final performance claim (Table 1) is an experimental measurement on BraTS2019, not a quantity forced by construction. The authors' prior hypergraph work [10] is cited only as general background for deep-learning-based segmentation and is not used to justify the central architecture or to forbid alternatives. The only circularity-adjacent concern is that hyperparameters such as α=0.7 and β=1.5 are selected via the ablation Table 2(b) without an explicit statement that selection was confined to the validation split; that is an overfitting or reproducibility risk, not a circular derivation. The baseline comparison may also be difficult to reconstruct because the paper does not explicitly state that baselines were retrained under the exact 2D-slice protocol, but again that is a fairness-of-comparison issue, not a case where a prediction reduces to a fitted input. Since no step exhibits the required quote-and-reduction pattern, the honest finding is no significant circularity.
Assumptions & free parameters
free parameters (7)
- alpha (Tversky FP weight) =
0.7
- beta (Tversky FN weight) =
1.5
- gamma (focal exponent) =
1.2
- P (replay buffer percentage) =
10%
- omega (TAC loss weight) =
0 for D1, 1 for later stages
- temperature tau =
1
- number of CPH layers =
2 (5th and 4th encoders)
assumptions (4)
- domain assumption 2D slices cropped to 224x224 along the z-axis are a valid proxy for 3D MRI volumes in segmentation evaluation.
- domain assumption Zero-filling missing modality channels is a valid simulation of missing modalities at inference.
- ad hoc to paper The hypergraph constructed from flattened encoder features captures patient-level associations useful for segmentation.
- domain assumption Baselines were evaluated under the same preprocessing and split protocol as ReHyDIL.
Cite this review
Pith. "Pith review of Hypergraph Tversky-Aware Domain Incremental Learning for Brain Tumor Segmentation with Missing Modalities." pith.science (2026). https://pith.science/paper/BDXIBMQ7
@misc{pith2026250516809,
author = {Pith},
title = {Pith review of: Hypergraph Tversky-Aware Domain Incremental Learning for Brain Tumor Segmentation with Missing Modalities},
year = {2026},
howpublished = {\url{https://pith.science/paper/BDXIBMQ7}},
note = {Machine review of arXiv:2505.16809}
}
read the original abstract
Existing methods for multimodal MRI segmentation with missing modalities typically assume that all MRI modalities are available during training. However, in clinical practice, some modalities may be missing due to the sequential nature of MRI acquisition, leading to performance degradation. Furthermore, retraining models to accommodate newly available modalities can be inefficient and may cause overfitting, potentially compromising previously learned knowledge. To address these challenges, we propose Replay-based Hypergraph Domain Incremental Learning (ReHyDIL) for brain tumor segmentation with missing modalities. ReHyDIL leverages Domain Incremental Learning (DIL) to enable the segmentation model to learn from newly acquired MRI modalities without forgetting previously learned information. To enhance segmentation performance across diverse patient scenarios, we introduce the Cross-Patient Hypergraph Segmentation Network (CHSNet), which utilizes hypergraphs to capture high-order associations between patients. Additionally, we incorporate Tversky-Aware Contrastive (TAC) loss to effectively mitigate information imbalance both across and within different modalities. Extensive experiments on the BraTS2019 dataset demonstrate that ReHyDIL outperforms state-of-the-art methods, achieving an improvement of over 2% in the Dice Similarity Coefficient across various tumor regions. Our code is available at https://github.com/reeive/ReHyDIL.
Figures
Reference graph
Works this paper leans on
-
[1]
arXiv preprint arXiv:1811.02629 (2018)
Bakas, S., Reyes, M., Jakab, A., Bauer, S., Rempfler, M., Crimi, A., Shinohara, R.T., Berger, C., Ha, S.M., Rozycki, M., et al.: Identifying the best machine learn- ing algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the brats challenge. arXiv preprint arXiv:1811.02629 (2018)
arXiv 2018
- [2]
-
[3]
Expert Systems with Applications251, 123783 (2024)
Cong, C., Liu, S., Rana, P., Pagnucco, M., Di Ieva, A., Berkovsky, S., Song, Y.: Adaptive unified contrastive learning with graph-based feature aggregator for imbalanced medical image classification. Expert Systems with Applications251, 123783 (2024)
work page 2024
- [4]
- [5]
-
[6]
Neuro-oncology17(9), 1188–1198 (2015)
Ellingson, B.M., Bendszus, M., Boxerman, J., Barboriak, D., Erickson, B.J., Smits, M., Nelson, S.J., Gerstner, E., Alexander, B., Goldmacher, G., et al.: Consensus recommendations for a standardized brain tumor imaging protocol in clinical trials. Neuro-oncology17(9), 1188–1198 (2015)
work page 2015
- [7]
-
[8]
Fan, L., Sowmya, A., Meijering, E., Song, Y.: Fast ff-to-ffpe whole slide image translation via laplacian pyramid and contrastive learning. In: MICCAI. pp. 409–
Show all 31 references
-
[9]
IEEE TPAMI44(5), 2548–2566 (2020)
Gao, Y., Zhang, Z., Lin, H., Zhao, X., Du, S., Zou, C.: Hypergraph learning: Methods and practices. IEEE TPAMI44(5), 2548–2566 (2020)
2020
-
[10]
Pattern Recognition165, 111544 (2025)
Jing, W., Wang, J., Di, D., Li, D., Song, Y., Fan, L.: Multi-modal hypergraph contrastive learning for medical image segmentation. Pattern Recognition165, 111544 (2025)
2025
-
[11]
In: 2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC)
Kalb, T., Mauthe, B., Beyerer, J.: Improving replay-based continual semantic seg- mentation with smart data selection. In: 2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC). pp. 1114–1121. IEEE (2022)
2022
-
[12]
In: AAAI
Liu, H., Wei, D., Lu, D., Sun, J., Wang, L., Zheng, Y.: M3ae: multimodal represen- tation learning for brain tumor segmentation with missing modalities. In: AAAI. vol. 37, pp. 1657–1665 (2023)
2023
-
[13]
Current opin- ion in neurology16(6), 643–650 (2003)
Rees, J.: Advances in magnetic resonance imaging of brain tumours. Current opin- ion in neurology16(6), 643–650 (2003)
2003
-
[14]
In: MICCAI
Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomed- ical image segmentation. In: MICCAI. pp. 234–241. Springer (2015)
2015
-
[15]
In: CVPR
Ross, T.Y., Dollár, G.: Focal loss for dense object detection. In: CVPR. pp. 2980– 2988 (2017)
2017
-
[16]
Insights into imaging11, 1–19 (2020)
Sawlani, V., Patel, M.D., Davies, N., Flintham, R., Wesolowski, R., Ughratdar, I., Pohl, U., Nagaraju, S., Petrik, V., Kay, A., et al.: Multiparametric mri: practical approach and pictorial review of a useful tool in the evaluation of brain tumours and tumour-like lesions. Ins...
2020
-
[17]
In: ACM International Conference on Multimedia
Shi, J., Shang, C., Sun, Z., Yu, L., Yang, X., Yan, Z.: Passion: Towards effective in- complete multi-modal medical image segmentation with imbalanced missing rates. In: ACM International Conference on Multimedia. pp. 456–465 (2024)
2024
-
[18]
IEEE Journal of Biomedical and Health Informatics (2023)
Shi, J., Yu, L., Cheng, Q., Yang, X., Cheng, K.T., Yan, Z.: M2ftrans: Modality- masked fusion transformer for incomplete multi-modality brain tumor segmenta- tion. IEEE Journal of Biomedical and Health Informatics (2023)
2023
-
[19]
In: CVPR
Tang, Q., Fan, L., Pagnucco, M., Song, Y.: Prototype-based image prompting for weakly supervised histopathological image segmentation. In: CVPR. pp. 30271– 30280 (2025)
2025
-
[20]
Psychological review84(4), 327 (1977)
Tversky, A.: Features of similarity. Psychological review84(4), 327 (1977)
1977
-
[21]
Nature Machine Intelligence4(12), 1185–1197 (2022)
VandeVen,G.M.,Tuytelaars,T.,Tolias,A.S.:Threetypesofincrementallearning. Nature Machine Intelligence4(12), 1185–1197 (2022)
2022
-
[22]
In: MICCAI
Wang, Z., Hong, Y.: A2fseg: Adaptive multi-modal fusion network for medical image segmentation. In: MICCAI. pp. 673–681. Springer (2023)
2023
-
[23]
In: Topological, Algebraic and Geometric Learning Workshops 2022
Yi, K., Chen, J., Wang, Y.G., Zhou, B., Lio, P., Fan, Y., Hamann, J.: Approximate equivariance so (3) needlet convolution. In: Topological, Algebraic and Geometric Learning Workshops 2022. pp. 189–198. PMLR (2022)
2022
-
[24]
IEEE TPAMI (2024)
Yuan, B., Zhao, D.: A survey on continual semantic segmentation: Theory, chal- lenge, method and application. IEEE TPAMI (2024)
2024
-
[25]
In: MICCAI
Zeng, Z., Peng, Z., Yang, X., Shen, W.: Missing as masking: Arbitrary cross-modal feature reconstruction for incomplete multimodal brain tumor segmentation. In: MICCAI. pp. 424–433. Springer (2024)
2024
-
[26]
In: MICCAI
Zhang, Y., He, N., Yang, J., Li, Y., Wei, D., Huang, Y., Zhang, Y., He, Z., Zheng, Y.: mmformer: Multimodal medical transformer for incomplete multimodal learn- ing of brain tumor segmentation. In: MICCAI. pp. 107–117. Springer (2022)
2022
-
[27]
arXiv preprint arXiv:2408.13733 (2024) Title Suppressed Due to Excessive Length 11
Zhang, Z., Liu, X., Chen, Z., Zhang, Y., Yue, H., Ou, Y., Sun, X.: Anatomical consistency distillation and inconsistency synthesis for brain tumor segmentation with missing modalities. arXiv preprint arXiv:2408.13733 (2024) Title Suppressed Due to Excessive Length 11
2024 arXiv
-
[28]
In: AAAI
Zhang, Z., Yang, G., Zhang, Y., Yue, H., Liu, A., Ou, Y., Gong, J., Sun, X.: Tmformer: Token merging transformer for brain tumor segmentation with missing modalities. In: AAAI. vol. 38, pp. 7414–7422 (2024)
2024
-
[29]
In: MICCAI
Zhao, Z., Yang, H., Sun, J.: Modality-adaptive feature interaction for brain tumor segmentation with missing modalities. In: MICCAI. pp. 183–192. Springer (2022)
2022
-
[30]
IEEE Transactions on Image Processing30, 4263–4274 (2021)
Zhou, T., Canu, S., Vera, P., Ruan, S.: Latent correlation representation learning for brain tumor segmentation with missing mri modalities. IEEE Transactions on Image Processing30, 4263–4274 (2021)
2021
-
[419]
Wang et al
Springer (2022) 10 J. Wang et al
2022
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.