REVIEW 5 major objections 6 minor 34 references
Automated MRI Tumor Segmentation using hybrid U-Net with Transformer and Efficient Attention
T0 review · 5 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read This paper reports that a hybrid U-Net with a transformer bottleneck and efficient attention reaches 0.7636 Dice and 0.7357 IoU on a local hospital MRI tumor dataset of 6,080 images, arguing that site-specific, small clinical datasets can…
desk verdict Interesting architecture integration and a useful local data pipeline, but the reported Dice/IoU are training metrics, so the generalization claim is unsupported. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the hybrid encoder-bottleneck-decoder stack. The encoder is a ResNet-50 pretrained on ImageNet, with Squeeze-and-Excitation and CBAM added to recalibrate channel and spatial features. The deepest 32 by 32 by 1024 feature map is flattened into 1024 tokens and fed through four transformer blocks with 8-head self-attention to capture global context. In the decoder, CBAM refines skip connections, an efficient attention module computes per-channel gates with linear complexity, and ResNeXt blocks with grouped convolutions fuse multi-scale patterns; the network is trained end-to-end with binary cross-entropy plus soft Dice loss.
What would settle it
Run the trained model on a held-out set of slices from patients not used in training, using the same preprocessing and augmentation; if the Dice score on that split falls substantially below 0.7636, the central claim of competitive generalizable segmentation is not supported.
Extended reading notes
Core claim
The paper reports that the complete hybrid model—a ResNet-50 encoder with Squeeze-and-Excitation and CBAM, a four-block transformer bottleneck, and a decoder with efficient attention, SE, CBAM, and ResNeXt blocks—reaches 0.7636 Dice and 0.7357 IoU on the local MRI dataset, compared with 0.7190 and 0.6818 for the plain U-Net baseline and 0.7286 and 0.7018 for a U-Net with only a transformer bottleneck. The authors take this as evidence that each added module contributes and that the combined design balances local feature extraction with long-range context. They interpret the result as competitive with published models such as HTTU-Net, H-DenseUNet, and MM-BiFPN even though those used far larger public datasets.
Load-bearing premise
The load-bearing premise is that the reported Dice and IoU numbers, which are shown as training curves, describe how well the model will segment MRI slices it has not seen before; the paper describes no held-out test set or patient-level split.
Editorial extensions
If this is right
- If the reported figures hold on held-out scans, the same architecture could be transferred to other local hospital datasets with modest GPU budgets.
- The module-wise comparison shows that adding the transformer bottleneck and attention modules raises Dice by roughly 0.04 over the baseline, suggesting measurable gains from combining local convolutional features with global context.
- Pretrained ImageNet weights plus checkpointed training offer a practical recipe for resource-limited sites that cannot train large models from scratch.
- The comparison against BraTS- and LiTS-trained models implies that dataset provenance matters as much as architecture when the goal is clinical deployment.
Reading between the lines
- Extending the paper's argument, the reported metrics are shown as training curves, so a fairer test of the clinical-deployment claim would require a held-out patient-level split; this is my inference, not a claim the paper makes.
- The augmentation pipeline expands roughly 1,000 images into 6,080, but augmented copies share label structure, so the model may be overfit to six patients; testing on truly unseen patients would clarify this.
- A testable extension is to run the same hybrid architecture on a public benchmark under a matched small-data regime to isolate whether the gains come from the architecture itself or from characteristics of the local dataset.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a hybrid U-Net architecture for automated MRI tumor segmentation, combining a ResNet-50 encoder with a transformer bottleneck and several attention modules (efficient attention, SE, CBAM, ResNeXt). The model is trained on a local clinical dataset of six patients (~1,050 original MRI slices, augmented to 6,080 images) using Kaggle GPUs with checkpointing. The authors report a Dice score of 0.7636 and IoU of 0.7357, and they argue that these results demonstrate competitive performance despite limited data. The paper includes a comparison with published methods on BraTS/LiTS and an ablation of three model variants.
Significance. If the reported results were obtained on a proper held-out test set, the paper would be a useful feasibility study on site-specific model development for resource-constrained clinical settings. The data pipeline, privacy considerations, and use of pretrained encoders and checkpointing are practical contributions. However, the central quantitative claim is currently unsupported because the reported Dice and IoU values are explicitly described as training-performance metrics. The lack of any out-of-sample evaluation, patient-level split, cross-validation, or error bars means the paper does not establish generalization, which is the core premise of the study. The architectural novelty is also incremental, as it combines existing modules (transformer bottleneck, SE, CBAM, ResNeXt, efficient attention) without a systematic analysis of their individual contributions beyond a small ablation.
major comments (5)
- [Section 3, Figure 5] The only reported quantitative results (Dice 0.7636, IoU 0.7357, precision 0.9736, recall 0.9756) are presented in Figure 5, whose caption reads 'Training performance metrics over epochs.' The text states that 'by epoch 29, these metrics improved' without ever describing a held-out test set, a validation split, a patient-level split, or cross-validation. Training-set metrics do not support the abstract's claim of 'competitive performance' on a local dataset, because they can reflect memorization rather than generalization. This is a load-bearing issue: the paper's central claim is empirical, and the evidence for it is absent.
- [Section 2.1 and Section 3] The dataset consists of MRI scans from six patients, with approximately 1,050 original slices. The augmentation pipeline expands this to 6,080 images, and Section 3 states that the model was 'trained end-to-end on a clinically sourced MRI dataset comprising 6,080 images.' The paper does not explain how the data were partitioned into training, validation, and test sets. If the 6,080 augmented images were used for both training and the reported evaluation, augmented copies of the same original slice could appear in both partitions, causing data leakage. A patient-level split is essential with only six patients, and no such split is described.
- [Table 2] The comparison in Table 2 is not meaningful for the claims made. HTTU-Net, H-DenseUNet, and MM-BiFPN are evaluated on BraTS or LiTS datasets with 58,000–221,400 slices, while the proposed hybrid model is evaluated on 6,080 local slices (and apparently on training metrics only). These comparisons involve different datasets, different modalities (CT for LiTS), and different evaluation protocols, so the reported Dice/IoU values cannot support the conclusion that the proposed model is 'competitive' or that it demonstrates a 'generalization gap.'
- [Table 3] The ablation in Table 3 compares 'U-Net (Baseline),' 'U-Net with Transformer Bottleneck,' and the 'Final Hybrid Model,' but no evaluation methodology is described for these variants. Since the paper never mentions a test split, all values in Table 3 likely reflect training performance. The improvements from 0.7190 to 0.7286 to 0.7636 Dice could be due to increased model capacity fitting the training set better, rather than improved generalization. Additionally, with only six patients, reporting a single point estimate without error bars or patient-level cross-validation is insufficient to establish that the architectural modifications are beneficial.
- [Section 2.3, Eq. (3)] The composite loss in Eq. (3) includes a weight λ that is described as 'empirically optimized,' but its value is never reported. The paper also omits details such as the optimizer, learning rate, learning-rate schedule, number of training epochs (29 appears only in Section 3), batch size details beyond 'maximum batch size of 8,' and any early-stopping criterion. These omissions substantially reduce the reproducibility of the experiments, which is especially important given that the central evaluation is already under-specified.
minor comments (6)
- [Section 2.1] The sentence 'All images were acquired at a voxel resolution of acquired at a voxel resolution of 0.84 × 0.84 × 2.00mm' contains a duplicated phrase; it should be corrected to a single 'acquired at a voxel resolution of 0.84 × 0.84 × 2.00 mm.'
- [Figure 5 and Section 3] The in-text references to Figure 5 are inconsistent. Section 3 says 'precision and recall increased ... as shown in Figure 5b' and 'the hybrid loss also steadily decreased ... as depicted in Figure 5c,' but the figure caption lists (a) Dice/IoU, (b) Precision/Recall, and (c) Loss. The text and caption should be reconciled.
- [Section 2.2] In the description of the transformer bottleneck, the text says the 32×32×1024 tensor 'is reshaped into a sequence of 1024 tokens (i.e., 1024 × 1024).' The parenthetical is confusing: a 32×32 tensor has 1024 spatial positions, each with 1024 channels, so the token sequence should be 1024 tokens of dimension 1024. The notation '1024 × 1024' should be clarified.
- [References] References [8], [9], and [12] are authored or co-authored by Asifullah Khan, and the Introduction leans on these preprints. Self-citation is not inherently problematic, but the authors should consider citing published peer-reviewed versions if they exist, to strengthen the literature basis.
- [Table 2] The naming is inconsistent: the text in Section 2.2 refers to 'Two-Track UNet' [24], while Table 2 calls it 'HTTU-Net.' Please use the same name consistently, matching the cited paper.
- [Abstract and Section 3] The abstract reports Dice '76.4%' and IoU '73.6%,' while the body uses 0.7636 and 0.7357. Use one consistent format (either proportions or percentages) throughout.
Circularity Check
Reported Dice/IoU are training metrics, so the central performance claim reduces to the optimized training objective; self-citations are not load-bearing.
-
fitted input called prediction
[Section 3, Figure 5 caption; Tables 2 and 3; Abstract]
"As depicted in Figure 5a, the model exhibited consistent convergence over 29 epochs with a batch size of 8. The initial Dice Score and IoU were 0.4548 and 0.3519, respectively. By epoch 29, these metrics improved to 0.7636 and 0.7357, respectively. Figure 5: Training performance metrics over epochs."
The only reported outcome metrics (Dice 0.7636, IoU 0.7357) are the final-epoch values from Figure 5, whose caption explicitly says 'Training performance metrics over epochs.' Section 3 describes no held-out test set, validation split, patient-level split, or cross-validation, so these numbers are computed on the same augmented 6,080-image data used to train the model. Furthermore, Eq. (3) trains the network with L_overall = L_BCE + lambda*L_Dice, meaning the soft Dice on the training set is the very quantity being optimized by gradient descent. Reporting final training Dice/IoU as 'competitive performance despite limited data' is therefore reporting the fitted value of the training objective, not an independent prediction of generalization.
full rationale
The paper is an empirical pipeline rather than a derivation chain, and its architectural pieces (SE, CBAM, ResNeXt, efficient attention, transformer bottleneck) are standard modules defined independently of the target result. The self-citations in references [8], [9], and [12] are used for motivation and for naming existing hybrid models; none of them supplies the reported Dice/IoU or forces the architecture choice in a way that reduces the result to those citations. However, the central performance claim is circular in a narrower sense: the reported Dice/IoU are explicitly training metrics from Figure 5, and no out-of-sample evaluation is described. Because the training loss directly optimizes BCE plus soft Dice on the same augmented dataset, the reported values are the optimized training objective, not a held-out prediction. This is a fitted-input-called-prediction issue and makes the central generalization claim unsupported; the remainder of the paper does not exhibit additional circular steps.
Assumptions & free parameters
free parameters (3)
- loss weighting lambda =
not reported
- SE reduction ratio r =
16 (standard, not tuned)
- transformer blocks and heads =
4 blocks, 8 heads
assumptions (3)
- domain assumption The reported Dice and IoU on the training set are valid proxies for clinical segmentation performance on unseen patients.
- domain assumption The augmented local dataset is representative of clinical heterogeneity and does not introduce leakage between patients.
- standard math Standard softmax attention and U-Net operations are accepted as background.
Cite this review
Pith. "Pith review of Automated MRI Tumor Segmentation using hybrid U-Net with Transformer and Efficient Attention." pith.science (2026). https://pith.science/paper/JGVUTASI
@misc{pith2026250615562,
author = {Pith},
title = {Pith review of: Automated MRI Tumor Segmentation using hybrid U-Net with Transformer and Efficient Attention},
year = {2026},
howpublished = {\url{https://pith.science/paper/JGVUTASI}},
note = {Machine review of arXiv:2506.15562}
}
read the original abstract
Cancer is an abnormal growth with potential to invade locally and metastasize to distant organs. Accurate auto-segmentation of the tumor and surrounding normal tissues is required for radiotherapy treatment plan optimization. Recent AI-based segmentation models are generally trained on large public datasets, which lack the heterogeneity of local patient populations. While these studies advance AI-based medical image segmentation, research on local datasets is necessary to develop and integrate AI tumor segmentation models directly into hospital software for efficient and accurate oncology treatment planning and execution. This study enhances tumor segmentation using computationally efficient hybrid UNet-Transformer models on magnetic resonance imaging (MRI) datasets acquired from a local hospital under strict privacy protection. We developed a robust data pipeline for seamless DICOM extraction and preprocessing, followed by extensive image augmentation to ensure model generalization across diverse clinical settings, resulting in a total dataset of 6080 images for training. Our novel architecture integrates UNet-based convolutional neural networks with a transformer bottleneck and complementary attention modules, including efficient attention, Squeeze-and-Excitation (SE) blocks, Convolutional Block Attention Module (CBAM), and ResNeXt blocks. To accelerate convergence and reduce computational demands, we used a maximum batch size of 8 and initialized the encoder with pretrained ImageNet weights, training the model on dual NVIDIA T4 GPUs via checkpointing to overcome Kaggle's runtime limits. Quantitative evaluation on the local MRI dataset yielded a Dice similarity coefficient of 0.764 and an Intersection over Union (IoU) of 0.736, demonstrating competitive performance despite limited data and underscoring the importance of site-specific model development for clinical deployment.
Figures
Reference graph
Works this paper leans on
-
[1]
Global cancer observatory: cancer today
Jacques Ferlay et al. “Global cancer observatory: cancer today”. In: Lyon: International agency for research on cancer20182020 (2020)
work page 2020
-
[2]
M. Essig et al. “MR imaging of neoplastic central nervous system lesions: review and recom- mendations for current practice”. In: AJNR. American Journal of Neuroradiology33.5 (2012), pp. 803–817. doi: 10.3174/ajnr.A2640. url: https://doi.org/10.3174/ajnr.A2640
-
[3]
Jr. Adams H. P. et al. “Classification of subtype of acute ischemic stroke: Definitions for use in a multicenter clinical trial. TOAST. Trial of Org 10172 in Acute Stroke Treatment”. In: Stroke 24.1 (1993), pp. 35–41. doi: 10.1161/01.STR.24.1.35 . url: https://doi.org/10. 1161/01.STR.24.1.35. 13
-
[4]
Simon K. Warfield, Kelly H. Zou, and William M. Wells. “Simultaneous Truth and Performance Level Estimation (STAPLE): an algorithm for the validation of image segmentation”. In:IEEE Transactions on Medical Imaging23.7 (2004), pp. 903–921. doi: 10.1109/TMI.2004.828354
arXiv 2004
-
[5]
A survey on deep learning in medical image analysis
Geert Litjens et al. “A survey on deep learning in medical image analysis”. In: Medical Image Analysis 42 (2017), pp. 60–88. doi: 10.1016/j.media.2017.07.005
-
[6]
The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS)
Bjoern H Menze et al. “The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS)”. In: IEEE Transactions on Medical Imaging34.10 (2015), pp. 1993–2024
work page 2015
-
[7]
U-net: Convolutional networks for biomedical image segmentation
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. “U-net: Convolutional networks for biomedical image segmentation”. In:Medical image computing and computer-assisted intervention– MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceed- ings, part III 18. Springer. 2015, pp. 234–241
work page 2015
-
[8]
A survey of the self supervised learning mechanisms for vision trans- formers
Asifullah Khan et al. “A survey of the self supervised learning mechanisms for vision trans- formers”. In: arXiv preprint arXiv:2408.17059(2024)
arXiv 2024
Show all 34 references
-
[9]
A recent survey of vision transformers for medical image segmentation
Asifullah Khan et al. “A recent survey of vision transformers for medical image segmentation”. In: arXiv preprint arXiv:2312.00634(2023)
2023 arXiv
-
[10]
Domain Adaptation for Medical Image Analysis: A Survey
H. Guan and M. Liu. “Domain Adaptation for Medical Image Analysis: A Survey”. In: IEEE Transactions on Biomedical Engineering69.3 (Mar. 2022). Epub 2022 Feb 18. PMID: 34606445; PMCID: PMC9011180, pp. 1173–1185. doi: 10.1109/TBME.2021.3117407
2022
-
[11]
Official Journal of the European Union, L 119, 4 May 2016, pp
Regulation (EU) 2016/679 of the European Parliament (General Data Protection Regulation). Official Journal of the European Union, L 119, 4 May 2016, pp. 1–88. http://data.europa. eu/eli/reg/2016/679/oj. 2016
2016
-
[12]
MaxViT-UNet: Multi-axis attention for medical image segmentation
Abdul Rehman Khan and Asifullah Khan. “MaxViT-UNet: Multi-axis attention for medical image segmentation”. In: arXiv preprint arXiv:2305.08396(2023)
2023 arXiv
-
[13]
nnU-Net: A Self-Configuring Method for Deep Learning-Based Biomed- ical Image Segmentation
Fabian Isensee et al. “nnU-Net: A Self-Configuring Method for Deep Learning-Based Biomed- ical Image Segmentation”. In: Nature Methods 18.2 (2021), pp. 203–211
2021
-
[14]
UNETR: Transformers for 3D Medical Image Segmentation
Amir Hatamizadeh et al. “UNETR: Transformers for 3D Medical Image Segmentation”. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2022, pp. 9312–9321
2022
-
[15]
The Liver Tumor Segmentation Benchmark (LiTS)
Patrick Bilic et al. “The Liver Tumor Segmentation Benchmark (LiTS)”. In: arXiv preprint arXiv:1901.04056 (2019)
2019 arXiv
-
[16]
H-DenseUNet: Hybrid Densely Connected UNet for Liver and Tumor Segmentation From CT Volumes
Xiaomeng Li et al. “H-DenseUNet: Hybrid Densely Connected UNet for Liver and Tumor Segmentation From CT Volumes”. In: IEEE Transactions on Medical Imaging37.12 (2018), pp. 2663–2674. doi: 10.1109/TMI.2018.2845918
2018
-
[17]
pydicom: An Open Source DICOM Library
pydicom Contributors. pydicom: An Open Source DICOM Library. https://pydicom.github. io. Accessed: Feb. 2025. 2025
2025
-
[18]
A Survey on Image Data Augmentation for Deep Learning
Connor Shorten and Taghi M. Khoshgoftaar. “A Survey on Image Data Augmentation for Deep Learning”. In: Journal of Big Data6.1 (2019), p. 60. 14
2019
-
[19]
Attention is all you need
A Vaswani. “Attention is all you need”. In: Advances in Neural Information Processing Systems (2017)
2017
-
[20]
A Stacked Multi-Connection Simple Reducing Net for Brain Tumor Segmen- tation
Yi Ding et al. “A Stacked Multi-Connection Simple Reducing Net for Brain Tumor Segmen- tation”. In: IEEE AccessPP (July 2019), pp. 1–1. doi: 10.1109/ACCESS.2019.2926448
2019
-
[21]
Transunet: Transformers make strong encoders for medical image seg- mentation
Jieneng Chen et al. “Transunet: Transformers make strong encoders for medical image seg- mentation”. In: arXiv preprint arXiv:2102.04306(2021)
2021 arXiv
-
[22]
An image is worth 16x16 words: Transformers for image recognition at scale
Alexey Dosovitskiy et al. “An image is worth 16x16 words: Transformers for image recognition at scale”. In: arXiv preprint arXiv:2010.11929(2020)
2020 arXiv
-
[23]
HUT: Hybrid UNet transformer for brain lesion and tumour segmentation
Wei Kwek Soh, Hing Yee Yuen, and Jagath C Rajapakse. “HUT: Hybrid UNet transformer for brain lesion and tumour segmentation”. In: Heliyon 9.12 (2023)
2023
-
[24]
HTTU-Net: Hybrid Two Track U-Net for Automatic Brain Tumor Segmentation
Nagwa M. Aboelenein et al. “HTTU-Net: Hybrid Two Track U-Net for Automatic Brain Tumor Segmentation”. In: IEEE Access 8 (2020), pp. 101406–101415. doi: 10.1109/ACCESS.2020. 2998601
2020 doi
-
[25]
Deep residual learning for image recognition
Kaiming He et al. “Deep residual learning for image recognition”. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2016, pp. 770–778
2016
-
[26]
Segmentation Models
Pavel Yakubovskiy. Segmentation Models. https : / / github . com / qubvel / segmentation _ models. Accessed: 2025-06-09. 2019
2025
-
[27]
Imagenet: A large-scale hierarchical image database
Jia Deng et al. “Imagenet: A large-scale hierarchical image database”. In: 2009 IEEE confer- ence on computer vision and pattern recognition. Ieee. 2009, pp. 248–255
2009
-
[28]
Squeeze-and-excitation networks
Jie Hu, Li Shen, and Gang Sun. “Squeeze-and-excitation networks”. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2018, pp. 7132–7141
2018
-
[29]
Cbam: Convolutional block attention module
Sanghyun Woo et al. “Cbam: Convolutional block attention module”. In: Proceedings of the European conference on computer vision (ECCV). 2018, pp. 3–19
2018
-
[30]
Efficient attention: Attention with linear complexities
Zhuoran Shen et al. “Efficient attention: Attention with linear complexities”. In: Proceedings of the IEEE/CVF winter conference on applications of computer vision. 2021, pp. 3531–3539
2021
-
[31]
Aggregated residual transformations for deep neural networks
Saining Xie et al. “Aggregated residual transformations for deep neural networks”. In: Pro- ceedings of the IEEE conference on computer vision and pattern recognition. 2017, pp. 1492– 1500
2017
-
[32]
V-Net: Fully Convolutional Neu- ral Networks for Volumetric Medical Image Segmentation
Fausto Milletari, Nassir Navab, and Seyed-Ahmad Ahmadi. “V-Net: Fully Convolutional Neu- ral Networks for Volumetric Medical Image Segmentation”. In: 2016 Fourth International Conference on 3D Vision (3DV). 2016, pp. 565–571. doi: 10.1109/3DV.2016.79
2016 doi
-
[33]
Unet++: A nested u-net architecture for medical image segmentation
Zongwei Zhou et al. “Unet++: A nested u-net architecture for medical image segmentation”. In: Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support: 4th International Workshop, DLMIA 2018, and 8th International Workshop, ML- CDS 2018, He...
2018
-
[34]
MM-BiFPN: multi-modality fu- sion network with Bi-FPN for MRI brain tumor segmentation
Nur Suriza Syazwany, Ju-Hyeon Nam, and Sang-Chul Lee. “MM-BiFPN: multi-modality fu- sion network with Bi-FPN for MRI brain tumor segmentation”. In: IEEE Access 9 (2021), pp. 160708–160720. 16
2021
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.