REVIEW 3 major objections 5 minor 19 references
An Exploratory Approach Towards Investigating and Explaining Vision Transformer and Transfer Learning for Brain Disease Detection
T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read The paper claims a Vision Transformer reaches 94.39% accuracy on a 37-class brain MRI dataset, beating four transfer-learning CNN baselines, and applies five explainability methods to the strongest CNN.
desk verdict Routine ViT comparison on a Bangladesh MRI dataset, undone by a leakage-prone preprocessing pipeline and best-of-five reporting. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The carrying mechanism is the contrast between two learning regimes. ViT splits each 128x128 MRI into 64x64 patches, projects the patches into a 64-dimensional embedding space, adds positional embeddings, and runs them through a transformer encoder with eight layers and four attention heads per layer, ending in an MLP head; its self-attention can relate any patch to any other patch directly. The four transfer-learning baselines start from convolutional backbones pretrained on a large natural-image corpus, with a flatten layer, two dense layers (1024 and 512 units), dropout, and a softmax head, fine-tuned on the MRI data. The paper also uses SMOTE to balance the 37 classes and an augmentation pipeline (flip, rotation, zoom) to expand the training data, and it takes the reported numbers from a ten-percent held-out split.
What would settle it
Run the same five models with SMOTE and augmentation applied only after an 80:10:10 split of the original images, holding all hyperparameters fixed, and compare test accuracies; if ViT's 94.39% lead over ResNet50V2 shrinks or disappears, the original ranking was an artifact of preprocessing order.
Extended reading notes
Core claim
On the paper's own terms, the central claim is that a Vision Transformer trained for 400 epochs on a 37-class Bangladeshi brain MRI dataset reaches 94.39% accuracy, 96.51% precision, 96.44% recall, and a 96.38% F1 score, ahead of every transfer-learning CNN tested (ResNet50V2 91.41%, MobileNetV2 88.56%, VGG19 89.05%, VGG16 88.44%). The authors attribute the margin to the transformer's patch-based self-attention, which captures long-range relationships in the image that convolutional filters with local receptive fields miss. Because gradient-based explanation methods do not map cleanly onto the transformer's architecture, the paper applies GradCAM, GradCAM++, LayerCAM, ScoreCAM, and FasterScoreCAM to ResNet50V2 instead, and reports that the heatmaps consistently point to the same brain regions, which it reads as a sign the model is attending to clinically meaningful structures.
Load-bearing premise
The result depends on the test split containing only genuinely unseen images, but this pipeline applies SMOTE and augmentation before the 80:10:10 split, so versions of the same image can appear in both training and test data.
Editorial extensions
If this is right
- If the reported ranking is correct, ViT should be the model of choice for this 37-class brain MRI task, with ResNet50V2 as the strongest CNN fallback.
- The roughly three-percentage-point accuracy gap and four-point F1 gap between ViT and ResNet50V2 imply that long-range, patch-level context contributes real signal for distinguishing the 37 lesion classes.
- The consistent XAI heatmaps across five gradient-based methods on ResNet50V2 imply that the best CNN model is making decisions from localized, reproducible image regions, which is a useful property for clinical review.
- All five models clear 88% accuracy, so even if ViT is unavailable in a resource-constrained hospital, the transfer-learning baselines still provide a usable diagnostic aid on this dataset.
Reading between the lines
- ViT was run for 400 epochs with a batch size of 256 while the CNN curves shown stop at about 80 epochs; an ablation that matches training budget across architectures would separate the transformer effect from the extra optimization time.
- The explainability analysis covers only the best CNN (ResNet50V2); applying attention-based explanation methods to the ViT itself would test whether the transformer's higher accuracy comes from the same or different image regions.
- The dataset's 37 classes include many closely related lesions, so a per-class confusion analysis, beyond overall accuracy, would show whether ViT's advantage comes from rare or morphologically similar classes.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript compares a Vision Transformer (ViT) trained from scratch with four ImageNet-pretrained transfer-learning models (VGG16, VGG19, ResNet50V2, MobileNetV2) on a 37-class brain MRI dataset collected in Bangladesh, reporting that ViT reaches 94.39% accuracy and that five gradient-based XAI methods provide interpretability. The central empirical claims are supported by Table 1, Figure 6, and qualitative XAI visualizations in Figure 7. The paper concludes that ViT significantly outperforms the CNN baselines and that XAI improves model transparency.
Significance. If the evaluation were sound, the study would be a useful benchmark for a local clinical MRI dataset and a practical comparison of ViT against transfer learning under class imbalance, with visual explanations from five XAI methods. The use of a publicly available dataset and the explicit comparative design are strengths, as is the attempt to include interpretability. However, the experimental protocol contains a data-leakage flaw and a best-run selection procedure that invalidate the reported accuracy numbers; without code, checkpoints, or corrected experiments, the main comparative claim is not verifiable in its current form.
major comments (3)
- [Section 3.2] SMOTE and the TensorFlow augmentation pipeline (resizing, random flip, rotation, zoom) are applied to the full dataset before the 80:10:10 train/validation/test split, as stated in the final sentence of Section 3.2. This creates data leakage: augmented copies or SMOTE-generated synthetic versions of the same original image can appear in both the training partition and the test partition. Consequently, the test accuracies in Table 1, including the headline 94.39% for ViT, do not measure generalization to unseen scans; they partly measure recognition of near-duplicate images. The split must be performed first, with SMOTE fit and augmentation applied only to the training partition, before any accuracy claim can be accepted.
- [Section 4.1] Section 4.1 states that each model was executed five times 'with the optimal results chosen for analysis.' Selecting the best of five runs inflates the reported point estimate and discards variance information, and Table 1 reports no standard deviations, confidence intervals, or significance tests. The statement in Section 4.2 that ViT 'significantly outperformed' the CNN models is therefore not statistically supported. The authors should report the mean and standard deviation over runs, or otherwise justify a non-selected evaluation protocol.
- [Section 5 and Abstract] The abstract and conclusion credit the integration of XAI methods with enhancing model transparency, but Section 5 applies the five XAI techniques only to ResNet50V2, not to the ViT model that is the paper's main contribution, and Figure 7 provides only qualitative heatmaps. No quantitative evaluation of explanation quality (e.g., localization, faithfulness, or expert assessment) is presented, so the claim that XAI improves reliability or transparency is not demonstrated. This should be restated as a qualitative exploration or supported by an appropriate evaluation.
minor comments (5)
- [Section 3.3] The paragraph beginning 'This study utilizes a range of pre-trained models...' is duplicated, and the architecture description repeats 'The task is effectively modified for brain disease' in consecutive sentences; one copy should be removed.
- [Figure 5] The diagram label '16 Patches' is inconsistent with the stated input size of 128x128 and patch size of 64x64, which yields four patches; please correct the figure or the text.
- [Figure 7 caption] The caption contains the typo 'Gradinat Based XAI' and should read 'Gradient Based XAI'; similarly, Figure 6 uses the misspelling 'Accurcay'.
- [Section 3.2] The phrase 'aver aging the complete dataset' contains a spacing error and should read 'averaging'.
- [References] Reference [2] is formatted inconsistently with the other references (author name in all caps, thesis title not in a standard style), and the model name appears variously as 'MobileNetV2' and 'MobilenetV2' throughout the text and Table 1.
Circularity Check
No circular derivation; the reported results are empirical measurements, with separate validity concerns.
full rationale
The central claim (Section 4.2, Table 1) is an empirical measurement, not a derivation. ViT and transfer-learning accuracies are obtained by training on the dataset and evaluating on a split; no equation in the paper defines the reported accuracy in terms of the inputs it is supposed to predict. The only self-citation with author overlap is [17] (Shifat Islam), used in Section 5 as a generic GradCAM/VQA reference; it is not load-bearing. The preprocessing order (SMOTE/augmentation before the 80:10:10 split) and the 'optimal results chosen' from five runs are real threats to the validity of the accuracy numbers, but they are not circular reductions: the test accuracy is not equal by construction to the training accuracy, nor is a fitted parameter renamed as a prediction. The model comparison is internally computed from the same pipeline for all five models. Accordingly, no step in the paper's claimed chain reduces to its own input.
Assumptions & free parameters
free parameters (7)
- ViT learning rate =
1e-4
- Transfer learning learning rate =
1e-5
- ViT batch size =
256
- ViT patch size and embedding dimension =
64x64 patches, 64-dimensional embedding
- ViT transformer layers and attention heads =
8 layers, 4 heads
- SMOTE oversampling target =
mean sample count across classes
- Dropout rates =
0.1 in transformer layers, 0.5 before classification
assumptions (3)
- domain assumption ImageNet-pretrained weights provide useful features for brain MRI classification.
- domain assumption SMOTE generates synthetic samples that are valid representatives of the minority classes.
- domain assumption Data augmentation (flip, rotation, zoom) preserves class labels.
Cite this review
Pith. "Pith review of An Exploratory Approach Towards Investigating and Explaining Vision Transformer and Transfer Learning for Brain Disease Detection." pith.science (2026). https://pith.science/paper/B4LHZRMX
@misc{pith2026250516039,
author = {Pith},
title = {Pith review of: An Exploratory Approach Towards Investigating and Explaining Vision Transformer and Transfer Learning for Brain Disease Detection},
year = {2026},
howpublished = {\url{https://pith.science/paper/B4LHZRMX}},
note = {Machine review of arXiv:2505.16039}
}
read the original abstract
The brain is a highly complex organ that manages many important tasks, including movement, memory and thinking. Brain-related conditions, like tumors and degenerative disorders, can be hard to diagnose and treat. Magnetic Resonance Imaging (MRI) serves as a key tool for identifying these conditions, offering high-resolution images of brain structures. Despite this, interpreting MRI scans can be complicated. This study tackles this challenge by conducting a comparative analysis of Vision Transformer (ViT) and Transfer Learning (TL) models such as VGG16, VGG19, Resnet50V2, MobilenetV2 for classifying brain diseases using MRI data from Bangladesh based dataset. ViT, known for their ability to capture global relationships in images, are particularly effective for medical imaging tasks. Transfer learning helps to mitigate data constraints by fine-tuning pre-trained models. Furthermore, Explainable AI (XAI) methods such as GradCAM, GradCAM++, LayerCAM, ScoreCAM, and Faster-ScoreCAM are employed to interpret model predictions. The results demonstrate that ViT surpasses transfer learning models, achieving a classification accuracy of 94.39%. The integration of XAI methods enhances model transparency, offering crucial insights to aid medical professionals in diagnosing brain diseases with greater precision.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
Amar Y Jaffar. Combining local and global feature extraction for brain tumor classification: A vision transformer and iresnet hybrid model.Engineering, Technology & Applied Science Research, 14(5):17011–17018, 2024
work page 2024
-
[2]
PhD thesis, tilburg university
IVO V AN DONGEN.COMPARISON OF INDIVIDUAL VISION TRANSFORMERS AND MODEL SOUPS FOR BRAIN TUMOR CLASSIFICATION ON MAGNETIC RESONANCE IMAGES. PhD thesis, tilburg university
-
[3]
Combining the transformer and convolution for effective brain tumor classification using mri images
Mohammed Aloraini, Asma Khan, Suliman Aladhadh, Shabana Habib, Mohammed F Alsharekh, and Muhammad Islam. Combining the transformer and convolution for effective brain tumor classification using mri images. Applied Sciences, 13(6):3680, 2023
work page 2023
-
[4]
Palani Thanaraj Krishnan, Pradeep Krishnadoss, Mukund Khandelwal, Devansh Gupta, Anupoju Nihaal, and T Sunil Kumar. Enhancing brain tumor detection in mri with a rotation invariant vision transformer.Frontiers in Neuroinformatics, 18:1414925, 2024
work page 2024
-
[5]
Ramy A Zeineldin, Mohamed E Karar, Ziad Elshaer, · Jan Coburger, Christian R Wirtz, Oliver Burgert, and Franziska Mathis-Ullrich. Explainability of deep neural networks for mri analysis of brain tumors.International journal of computer assisted radiology and surgery, 17(9):1673–1683, 2022
work page 2022
-
[6]
Deep learning-based model with xai for brain tumor classification and segmentation using mri images
Sakibul Hasan, Maisha Musarrat Nabila, Rijjet Bin Anis, and Raqeebir Rab. Deep learning-based model with xai for brain tumor classification and segmentation using mri images. In2023 IEEE 9th International Women in Engineering (WIE) Conference on Electrical and Computer Engineering (WIECON-ECE), pages 1–6. IEEE, 2023
work page 2023
-
[7]
Exploring alzheimer’s disease prediction with xai in various neural network models
Hamza Ahmed Shad, Quazi Ashikur Rahman, Nashita Binte Asad, Atif Zawad Bakshi, SM Faiaz Mursalin, Md Tanzim Reza, and Mohammad Zavid Parvez. Exploring alzheimer’s disease prediction with xai in various neural network models. InTENCON 2021-2021 IEEE Region 10 Conference (TENCON), pages 720–725. IEEE, 2021
work page 2021
-
[8]
Tanjim Mahmud, Koushick Barua, Sultana Umme Habiba, Nahed Sharmen, Mohammad Shahadat Hossain, and Karl Andersson. An explainable ai paradigm for alzheimer’s diagnosis using deep transfer learning.Diagnostics, 14(3):345, 2024
work page 2024
Show all 19 references
-
[9]
Deep transfer learning for brain magnetic resonance image multi-class classification.arXiv preprint arXiv:2106.07333, 2021
Yusuf Brima, Mossadek Hossain Kamal Tushar, Upama Kabir, and Tariqul Islam. Deep transfer learning for brain magnetic resonance image multi-class classification.arXiv preprint arXiv:2106.07333, 2021
2021 arXiv
-
[10]
Brain mri dataset
Yusuf Brima, Mossadek Hossain Kamal Tushar, Upama Kabir, and Tariqul Islam. Brain mri dataset. https: //doi.org/10.6084/m9.figshare.14778750.v2, 2021
2021 doi
-
[11]
Smote: synthetic minority over-sampling technique.Journal of artificial intelligence research, 16:321–357, 2002
Nitesh V Chawla, Kevin W Bowyer, Lawrence O Hall, and W Philip Kegelmeyer. Smote: synthetic minority over-sampling technique.Journal of artificial intelligence research, 16:321–357, 2002. 10 APREPRINT- SEPTEMBER22, 2025
2002
-
[12]
A study of cnn and transfer learning in medical imaging: Advantages, challenges, future scope.Sustainability, 15(7):5930, 2023
Ahmad Waleed Salehi, Shakir Khan, Gaurav Gupta, Bayan Ibrahimm Alabduallah, Abrar Almjally, Hadeel Alsolai, Tamanna Siddiqui, and Adel Mellit. A study of cnn and transfer learning in medical imaging: Advantages, challenges, future scope.Sustainability, 15(7):5930, 2023
2023
-
[13]
Classification of brain tumor from magnetic resonance imaging using vision transformers ensembling.Current Oncology, 29(10):7498–7511, 2022
Sudhakar Tummala, Seifedine Kadry, Syed Ahmad Chan Bukhari, and Hafiz Tayyab Rauf. Classification of brain tumor from magnetic resonance imaging using vision transformers ensembling.Current Oncology, 29(10):7498–7511, 2022
2022
-
[14]
Keras.https://keras.io, 2015
François Chollet et al. Keras.https://keras.io, 2015
2015
-
[15]
Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunat...
2015
-
[16]
Multi-class brain lesion classification using deep transfer learning with mobilenetv3.IEEE Access, 2024
Ahmed Firas Majeed, Pedram Salehpour, Leili Farzinvash, and Saeid Pashazadeh. Multi-class brain lesion classification using deep transfer learning with mobilenetv3.IEEE Access, 2024
2024
-
[17]
A deep learning-based bengali visual question answering system
Mahamudul Hasan Rafi, Shifat Islam, SM Hasan Imtiaz Labib, SM Sajid Hasan, Faisal Muhammad Shah, and Sifat Ahmed. A deep learning-based bengali visual question answering system. In2022 25th International Conference on Computer and Information Technology (ICCIT), pages 114–119....
2022
-
[18]
Fatema Tuj Johora Faria, Mukaffi Bin Moin, Pronay Debnath, Asif Iftekher Fahim, and Faisal Muhammad Shah. Explainable convolutional neural networks for retinal fundus classification and cutting-edge segmentation models for retinal blood vessels from fundus images.arXiv preprin...
2024 arXiv
-
[19]
Explainability and evaluation of vision transformers: An in-depth experimental study.Electronics, 13(1):175, 2023
Sédrick Stassin, Valentin Corduant, Sidi Ahmed Mahmoudi, and Xavier Siebert. Explainability and evaluation of vision transformers: An in-depth experimental study.Electronics, 13(1):175, 2023. 11
2023
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.