CoMViT: An Efficient Vision Backbone for Supervised Classification in Medical Imaging

Aon Safdar; Mohamed Saadeldin

REVIEW 4 major objections 5 minor 21 references

CoMViT: An Efficient Vision Backbone for Supervised Classification in Medical Imaging

T0 review · 4 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A 4.5-million-parameter vision transformer matches or beats far larger models across twelve medical imaging datasets.

desk verdict A plausible but thinly evidenced lightweight ViT for MedMNIST—reviewable, but the headline claims lean on unmatched baselines and a tuned-on-benchmark architecture. read the letter →

arxiv 2510.27442 v1 pith:ISTFAI4S submitted 2025-10-31 cs.CV cs.AI

Aon Safdar , Mohamed Saadeldin This is my paper

classification cs.CVcs.AI

keywords compactvisiontransformermedicalimagingMedMNISTconvolutionaltokenizersequencepoolinglow-resourcelearningefficientclassificationself-attention

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a deliberately compact vision transformer can deliver high-accuracy medical image classification without scaling up parameters. CoMViT combines a convolutional tokenizer, diagonally masked attention with a learned temperature, and sequence pooling to embed spatial priors that large ViTs normally lack. Across all twelve 2D MedMNIST datasets it reaches about 84.5% average top-1 accuracy with 4.55M parameters, matching or beating much larger CNN and ViT baselines and landing near the accuracy-versus-parameters Pareto frontier. If the claims hold, resource-constrained clinical settings can use a lightweight, interpretable transformer rather than large pretrained models.

What carries the argument

The convolutional tokenizer replaces rigid patch splitting with learned 7x7 conv filters followed by 3x3 max pooling, building local spatial priors before attention. Diagonal masking (setting M_ii = -inf) suppresses each token attending to itself, and a learnable temperature scales the attention logits, together keeping attention localized and stable. Learnable sequence pooling, a learned weighted sum over tokens, replaces the CLS token for final classification.

What would settle it

Run CoMViT and each baseline from scratch under CoMViT's exact training recipe (AdamW, 300 epochs, cosine decay, RandAugment, Mixup/CutMix, label smoothing, batch 512) on the same MedMNIST2D splits. If a larger baseline then matches or beats CoMViT's 84.5% average, the central claim that scale can be traded for design would need revision.

Watch

Extended reading notes

Core claim

CoMViT's central claim is that for medical imaging, architectural design can substitute for scale: a 7-layer transformer with 256-dim hidden width, 4 heads, a two-layer 7x7 convolutional tokenizer, a diagonal mask that suppresses self-attention, learnable temperature scaling, and learned sequence pooling reaches 84.5% average top-1 accuracy on MedMNIST2D with 4.55M parameters. It reports best or second-best accuracy on 8 of 12 datasets, outperforming ResNet-50 (82.1% with 23.5M parameters) and matching a comparable compact transformer baseline (84.0% with 10.2M parameters), with up to 5-20x parameter reduction. Heatmap visualizations show the model attending to clinically relevant structures

Load-bearing premise

The baseline accuracy numbers for ResNet, MedViT, DeiT, and similar models are quoted from earlier publications with different training protocols, so the claimed parity and 5-20x parameter advantage assume those numbers are directly comparable to CoMViT's own 300-epoch, heavily augmented run.

Editorial extensions

If this is right

On the MedMNIST2D suite, CoMViT achieves 84.5% average accuracy with 4.55M parameters, a 5-20x parameter reduction over common CNN and ViT baselines at similar or better accuracy.
It reaches best or second-best accuracy on 8 of the 12 datasets, spanning X-ray, OCT, dermoscopy, and microscopy modalities without per-dataset tuning.
On TissueMNIST it leads the tiny-model group (69.8%) while using the fewest parameters, and rivals small and large models.
The design removes the CLS token and relies on sequence pooling, making the architecture adaptable to varying lesion locations and sizes.
Qualitative heatmaps indicate the compact model concentrates on pathology-relevant image regions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A fair head-to-head re-benchmark that trains every baseline with CoMViT's exact 300-epoch augmentation and regularization schedule could shrink or enlarge the reported 5-20x advantage; the paper compares against published numbers rather than matched runs.
The components are modular: an ablation study could reveal which one (convolutional tokenizer, diagonal masking, temperature scaling, or sequence pooling) carries the accuracy gain, since the paper reports the combination only.
Because the gains are attributed to architecture rather than scale, the same recipe could be tested on other small-data vision domains such as satellite or manufacturing-defect imagery, where larger transformers overfit.
The tokenizer and pooling likely transfer to dense tasks like segmentation or detection in low-resource medical settings, though the paper evaluates classification only.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The paper you'll want to know about: CoMViT is a 4.55M-parameter ViT for MedMNIST2D classification, built from four published components (convolutional tokenizer and sequence pooling from CCT, diagonal masking from SPT-Swin, learnable temperature from Lee et al.). The empirical claim is that it matches or beats much larger CNNs/ViTs at 5–20x fewer parameters. That claim is plausible, but as it stands the evidence is thinner than the abstract suggests.

What the paper does well: it is honest about borrowing components, provides code, evaluates on all twelve MedMNIST2D datasets, and reports efficiency metrics. The Grad-CAM qualitative results are a nice addition, and the authors resist overselling interpretability. The specific configuration (7 layers, 256 hidden, 4 heads) is not in prior work, so the empirical result is new in a narrow sense.

The problems, in order of severity. First, the baselines come from prior publications (mainly MedViT, ref [14]) and were not retrained under CoMViT's recipe: 300 epochs, RandAugment, Mixup/CutMix, label smoothing, drop-path. The decisive margins are small—CoMViT's 84.5% average vs. MedViT-T's 84.0% and MedViT-S's 85.1%. Without matched training or error bars across seeds, the "matches or outperforms" claim is not stable. The baselines may well have used weaker augmentation, which would inflate CoMViT's relative advantage. Second, the architecture is described as "empirically tuned" on the benchmark with no validation-based model-selection protocol. That raises a tuning-on-test risk: if the configuration was chosen after seeing MedMNIST test accuracies, the numbers are circular. The paper must state the selection procedure explicitly. Third, there are no ablations showing each component's contribution—without them, the "systematic architectural optimization" claim is unsupported. Fourth, minor: the model name alternates between CoMViT and ComViT.

These are all addressable. The central idea—that locality-aware attention plus a convolutional stem can make a small ViT competitive on medical imaging—is reasonable, and the paper is written straightforwardly without spin. It deserves serious refereeing, though likely with requests for matched baselines, error bars, ablations, and transparency about model selection. For a reading group, it's a maybe: a useful data point on MedMNIST efficiency, not a conceptual advance.

Referee Report

4 major / 5 minor

Summary. The paper introduces CoMViT, a compact Vision Transformer backbone (~4.55M parameters) for MedMNIST2D classification. The architecture combines a shallow convolutional tokenizer, a 7-layer encoder with diagonal masking and learnable temperature scaling, and learnable sequence pooling in place of a CLS token. On twelve 2D MedMNIST datasets the authors report 84.5% average top-1 accuracy, claiming that CoMViT matches or outperforms larger CNNs and ViTs while using 5–20x fewer parameters, and they present Grad-CAM visualizations as evidence of clinically relevant attention. The central claim is evaluated mainly against baselines taken from prior publications rather than retrained under the same protocol.

Significance. If the accuracy-efficiency comparison is substantiated, CoMViT would be a practically useful lightweight medical-imaging backbone: it evaluates on all twelve MedMNIST2D datasets, reports a model with 4.55M parameters that reaches roughly 84.5% average accuracy, and makes code available. The qualitative Grad-CAM analysis is a useful addition. However, the current evidence is not sufficient to support the strength of the claimed parity/outperformance, because several decisive margins are below 1 percentage point, no uncertainty quantification is provided, and the baselines are not trained under a matched protocol.

major comments (4)

[§4.3, Tables 2–4] The central claim relies on cross-paper baseline comparisons. CoMViT is trained with a specific 300-epoch, heavily regularized recipe (AdamW, cosine decay, RandAugment, Mixup 0.8, CutMix 1.0, label smoothing, drop-path), whereas the baseline numbers for ResNet, MedViT, DeiT, Swin, etc. are taken from prior publications with no evidence of equivalent training protocols. The decisive margins in Table 3 are only 0.5 pp (CoMViT 84.5 vs MedViT-T 84.0) and −0.6 pp vs MedViT-S (85.1), which are smaller than typical protocol-induced shifts. Please either retrain all baselines under the same protocol or substantially hedge the comparison claims.
[Tables 2 and 3] No error bars or significance tests are reported; every number appears to be a single run. With margins of 0.5–0.6 pp on a benchmark as small as MedMNIST, the difference between 'matches' and 'trails' could be seed noise. Report mean ± std over at least three random seeds for CoMViT and the re-trained baselines, and use an appropriate test (e.g., paired bootstrap or Wilcoxon) for the aggregate claim.
[§5, Table 2] The abstract and conclusion claim CoMViT offers parameter reduction 'without sacrificing accuracy,' but the paper's own per-dataset table contradicts this on several datasets: vs MedViT-Tiny, CoMViT is 91.08 vs 95.6 on PathMNIST, 83.97 vs 89.6 on BreastMNIST, and 92.14 vs 94.9 on PneumoniaMNIST. The average is higher, but a model that loses by >4 pp on some tasks does not support an unqualified 'without sacrificing accuracy.' Please qualify the claim by reporting which datasets are traded off and how large those losses are.
[§3, Table 1 caption] Table 1 describes CoMViT's configuration as 'empirically tuned lightweight setup optimized for medical imaging,' and §3 says the settings were 'further optimized.' No validation-based model-selection protocol is reported (e.g., how many configurations were tried, which validation split was used, whether the test set was seen during tuning). This creates a selection-risk issue for the reported average. Please specify the model-selection procedure and, ideally, freeze the architecture before evaluating on the test sets.

minor comments (5)

[Throughout] The model name is inconsistent: CoMViT, ComViT, and CaMViT are all used (e.g., 'ComViT' in §3, 'CaMViT' in §5). Please unify.
[Table 2 caption] The caption says 'Red is best, blue is second-best,' but the table as rendered contains no color. Either add the highlighting or remove the caption claim.
[§4.3 / Table 4] FLOPs are reported without specifying how they were measured (resolution, whether convolution/attention FLOPs include batch norm, etc.). State the protocol or cite the source for baseline FLOPs.
[References [8,9]] References [8] and [9] are the same work (Hassani et al., 'Escaping the Big Data Paradigm with Compact Transformers'). Please merge or distinguish.
[§5] The Grad-CAM discussion is qualitative. It would strengthen the paper to include a quantitative interpretability measure (e.g., localization accuracy against segmentation masks where available) or to soften claims about 'clinically relevant regions.'

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation found: CoMViT's accuracy and efficiency claims are empirical measurements against an external benchmark, not consequences of fitted inputs or self-citations.

full rationale

The paper's central claims are empirical: CoMViT is trained and evaluated on MedMNIST's fixed train/test splits, and the accuracy numbers in Tables 2-4 are measured results, not quantities derived from the model's definitions by construction. The baseline numbers (ResNet, MedViT, DeiT, Swin, etc.) are quoted from external publications, so they do not depend on any parameter fitted by this paper. No equation in Section 3 defines a target quantity in terms of the same target quantity; the design choices (convolutional tokenizer, diagonal masking, learnable temperature, sequence pooling) are introduced by citing independent external work rather than by a self-citation chain. The closest concern is Table 1's caption: 'ComViT adopts an empirically tuned lightweight setup optimized for medical imaging,' and the abstract's 'systematic architectural optimization.' If the architecture or training recipe were selected using the MedMNIST test labels, the results would be partly in-sample; however, the paper does not state that, and hyperparameter tuning on validation splits is standard practice. Similarly, the comparison with previously published baselines may suffer from different training protocols and missing error bars, but that is a correctness or validity risk, not a circular reduction. No fitted parameter is renamed as a prediction, and no uniqueness or ansatz is imported from the authors' own prior work. Therefore the paper has no significant circularity.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central empirical claim rests on benchmark comparability, tuning choices, and the proxy validity of MedMNIST; these are unstated assumptions rather than derived facts.

free parameters (2)

Architecture configuration (L=7, D=256, MLP=512, heads=4, conv tokenizer 7x7, maxpool 3x3) = 7 layers / 256 hidden / 512 MLP / 4 heads
Chosen by hand/empirically ('empirically tuned', Table 1 caption); no search or ablation reported, and no statement that selection used only validation splits.
Training hyperparameters (300 epochs, lr 1.1e-4, batch 512, mixup/cutmix/label smoothing/drop-path) = 300 epochs, lr 1.1e-4, batch 512, α_mixup=0.8, α_cutmix=1.0
Set in §4.3 to a common small-data ViT recipe; no sensitivity analysis, and baselines from prior papers likely used different schedules.

assumptions (3)

domain assumption Baseline accuracy numbers from prior publications are comparable to CoMViT's runs even though training protocols differ.
Section 4.3 specifies CoMViT-specific training, while Tables 2–4 borrow baseline numbers from other papers; if baselines were retrained under the same protocol, the parameter-efficiency advantage could shrink.
domain assumption MedMNIST test accuracy is a valid proxy for real-world low-resource clinical performance.
The paper motivates clinical deployment (abstract, §5) but evaluates only on the standardized MedMNIST benchmark, with no clinical or distribution-shift validation.
domain assumption Architecture and hyperparameters were not selected using MedMNIST test labels.
Table 1 caption says the config was 'empirically tuned ... optimized for medical imaging'; the paper does not describe a validation-based selection protocol, so test-set leakage cannot be ruled out.

how reviews work

0 comments

Cite this review

Pith. "Pith review of CoMViT: An Efficient Vision Backbone for Supervised Classification in Medical Imaging." pith.science (2026). https://pith.science/paper/ISTFAI4S

@misc{pith2026251027442,
  author       = {Pith},
  title        = {Pith review of: CoMViT: An Efficient Vision Backbone for Supervised Classification in Medical Imaging},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ISTFAI4S}},
  note         = {Machine review of arXiv:2510.27442}
}

read the original abstract

Vision Transformers (ViTs) have demonstrated strong potential in medical imaging; however, their high computational demands and tendency to overfit on small datasets limit their applicability in real-world clinical scenarios. In this paper, we present CoMViT, a compact and generalizable Vision Transformer architecture optimized for resource-constrained medical image analysis. CoMViT integrates a convolutional tokenizer, diagonal masking, dynamic temperature scaling, and pooling-based sequence aggregation to improve performance and generalization. Through systematic architectural optimization, CoMViT achieves robust performance across twelve MedMNIST datasets while maintaining a lightweight design with only ~4.5M parameters. It matches or outperforms deeper CNN and ViT variants, offering up to 5-20x parameter reduction without sacrificing accuracy. Qualitative Grad-CAM analyses show that CoMViT consistently attends to clinically relevant regions despite its compact size. These results highlight the potential of principled ViT redesign for developing efficient and interpretable models in low-resource medical imaging settings.

Figures

Figures reproduced from arXiv: 2510.27442 by the authors.

**Figure 1.** CoMViT Architecture Overview MedViT [14] introduces local convolutional attention and depthwise MLPs for efficient context modeling. These methods improve locality but add architectural complexity. Fine-tuning large pretrained ViTs [6] yields competitive results but often suffers from domain mismatch and negative knowledge transfer [4]. Other works explore ViT robustness and domain alignment. PyramidAT [10] enhance… view at source ↗

**Figure 2.** Overview of the MedMNIST Dataset Family. Modality, Classification Task, and split (Train / Val / Test) are mentioned. Best viewed in color and zoomed in. reproducibility. Unlike task-specific datasets, MedMNIST provides a modalityagnostic benchmark for evaluating generalizable tokenization and attention strategies. It supports multi-class, binary, and ordinal tasks, spanning textureand structure-rich modalities un… view at source ↗

**Figure 3.** Grad-CAM visualizations across six MedMNIST datasets highlighting the regions of interest that contribute most to the model’s predictions. CoMViT accurately localizes relevant regions across diverse modalities. Despite a compact footprint, CoMViT remains competitive with SOTA models [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: CoMViT appears near the Pareto frontier, striking a balance where [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

**Figure 4.** Figure 4: Model Size vs. Accuracy on TissueMNIST. CoMViT achieves strong accuracy with minimal parameters, illustrating excellent efficiency. marization of the token sequence without needing a rigid [CLS] vector, improving adaptation to lesions of varying shapes and locations. T…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

21 extracted references · 2 canonical work pages

[14]

Computers in Biology and Medicine157, 106791 (May 2023)

Manzari, O.N., Ahmadabadi, H., Kashiani, H., Shokouhi, S.B., Ayatollahi, A.: MedViT: A Robust Vision Transformer for Generalized Medical Im- age Classification. Computers in Biology and Medicine157, 106791 (May 2023). https://doi.org/10.1016/j.compbiomed.2023.106791,http://arxiv.org/ abs/2302.09462, arXiv:2302.09462 [cs]

arXiv 2023
[1]

Journal of Imaging Informatics in Medicine pp

Aburass, S., Dorgham, O., Al Shaqsi, J., Abu Rumman, M., Al-Kadi, O.: Vision Transformers in Medical Imaging: a Comprehensive Review of Advancements and Applications Across Multiple Diseases. Journal of Imaging Informatics in Medicine pp. 1–44 (Mar 2025). https://doi.org/10.1007/s10278-025-01481-y,https://link. springer.com/article/10.1007/s10278-025-0148...

doi:10.1007/s10278-025-01481-y 2025
[2]

In: 2021 IEEE/CVF International Conference on Computer Vision (ICCV)

Caron, M., Touvron, H., Misra, I., Jegou, H., Mairal, J., Bojanowski, P., Joulin, A.: Emerging Properties in Self-Supervised Vision Transformers. In: 2021 IEEE/CVF International Conference on Computer Vision (ICCV). pp. 9630–9640. IEEE, Mon- treal, QC, Canada (Oct 2021). https://doi.org/10.1109/ICCV48922.2021.00951, https://ieeexplore.ieee.org/document/9709990/

arXiv 2021
[3]

Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., Houlsby, N.: An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (Jun 2021),http://arxiv.org/abs/2010.11929, arXiv:2010.11929 [cs]

arXiv 2021
[4]

https://doi.org/10.1007/978-3-031-43901-8_43,https://link.springer

Du, S., Bayasi, N., Hamarneh, G., Garbi, R.: MDViT: Multi-domain Vi- sion Transformer for Small Medical Image Segmentation Datasets (Oct 2023). https://doi.org/10.1007/978-3-031-43901-8_43,https://link.springer. com/chapter/10.1007/978-3-031-43901-8_43

doi:10.1007/978-3-031-43901-8_43 2023
[5]

IEEE Access12, 117617–117626 (2024)

Ferdous, G.J., Sathi, K.A., Hossain, M.A., Dewan, M.A.A.: SPT-Swin: A Shifted Patch Tokenization Swin Transformer for Image Classification. IEEE Access12, 117617–117626 (2024). https://doi.org/10.1109/ACCESS.2024.3448304,https:// ieeexplore.ieee.org/document/10643534

arXiv 2024
[6]

Scientific Reports 14(1), 12567 (May 2024)

Halder, A., Gharami, S., Sadhu, P., Singh, P.K., Woźniak, M., Ijaz, M.F.: Imple- menting vision transformer for classifying 2D biomedical images. Scientific Reports 14(1), 12567 (May 2024). https://doi.org/10.1038/s41598-024-63094-9,https: //www.nature.com/articles/s41598-024-63094-9, publisher: Nature Publishing Group

work page doi:10.1038/s41598-024-63094-9 2024
[7]

IEEE Transactions on Pattern Analysis and Machine Intelligence45(1), 87–110 (Jan 2023)

Han, K., Wang, Y., Chen, H., Chen, X., Guo, J., Liu, Z., Tang, Y., Xiao, A., Xu, C., Xu, Y., Yang, Z., Zhang, Y., Tao, D.: A Survey on Vision Transformer. IEEE Transactions on Pattern Analysis and Machine Intelligence45(1), 87–110 (Jan 2023). https://doi.org/10.1109/TPAMI.2022.3152247,https://ieeexplore. ieee.org/abstract/document/9716741, conference Name...

arXiv 2023

Show all 21 references

[8]

Hassani, A., Walton, S., Shah, N., Abuduweili, A., Li, J., Shi, H.: Escaping the big data paradigm with compact transformers. CoRRabs/2104.05704(2021), https://arxiv.org/abs/2104.05704 2 For the purpose of Open Access, the author has applied a CC BY public copyright licence to...

2021 arXiv
[9]

https://doi.org/10.48550/arXiv.2104.05704,http://arxiv.org/abs/2104.05704, arXiv:2104.05704 [cs]

Hassani, A., Walton, S., Shah, N., Abuduweili, A., Li, J., Shi, H.: Es- caping the Big Data Paradigm with Compact Transformers (Jun 2022). https://doi.org/10.48550/arXiv.2104.05704,http://arxiv.org/abs/2104.05704, arXiv:2104.05704 [cs]

2022 doi arXiv
[10]

Herrmann, C., Sargent, K., Jiang, L., Zabih, R., Chang, H., Liu, C., Krishnan, D., Sun, D.: CVPR 2022 Open Access Repository.https: //openaccess.thecvf.com/content/CVPR2022/html/Herrmann_Pyramid_ Adversarial_Training_Improves_ViT_Performance_CVPR_2022_paper.html

2022
[11]

IEEE Access10, 123212–123224 (2022)

Lee, S., Lee, S., Song, B.C.: Improving Vision Transformers to Learn Small-Size Dataset From Scratch. IEEE Access10, 123212–123224 (2022). https://doi.org/10.1109/ACCESS.2022.3224044,https://ieeexplore.ieee.org/ document/9957006/

2022
[12]

Medical Image Analysis85, 102762 (Apr 2023)

Li, J., Chen, J., Tang, Y., Wang, C., Landman, B.A., Zhou, S.K.: Transform- ing medical imaging with Transformers? A comparative review of key proper- ties, current progresses, and future perspectives. Medical Image Analysis85, 102762 (Apr 2023). https://doi.org/10.1016/j.medi...

2023
[13]

In: 2022 Interna- tional Joint Conference on Neural Networks (IJCNN)

Liu, J., Li, Y., Cao, G., Liu, Y., Cao, W.: Feature Pyramid Vi- sion Transformer for MedMNIST Classification Decathlon. In: 2022 Interna- tional Joint Conference on Neural Networks (IJCNN). pp. 1–8 (Jul 2022). https://doi.org/10.1109/IJCNN55064.2022.9892282,https://ieeexplore....

2022
[15]

In: 2021 15th International Conference on Advanced Computing and Applications (ACOMP)

Nguyen, N.Q., Le, T.S.: A Semi-Supervised Learning Method to Rem- edy the Lack of Labeled Data. In: 2021 15th International Conference on Advanced Computing and Applications (ACOMP). pp. 78–84 (Nov 2021). https://doi.org/10.1109/ACOMP53746.2021.00017,https://ieeexplore. ieee.o...

2021
[16]

https://doi.org/10.48550/arXiv.2106.10270,http://arxiv.org/abs/ 2106.10270, arXiv:2106.10270 [cs]

Steiner, A., Kolesnikov, A., Zhai, X., Wightman, R., Uszkoreit, J., Beyer, L.: How totrainyourViT?Data,Augmentation,andRegularizationinVisionTransformers (Jun 2022). https://doi.org/10.48550/arXiv.2106.10270,http://arxiv.org/abs/ 2106.10270, arXiv:2106.10270 [cs]

2022 doi arXiv
[17]

In: Pro- ceedings of the 38th International Conference on Machine Learning

Touvron, H., Cord, M., Douze, M., Massa, F., Sablayrolles, A., Jegou, H.: Train- ing data-efficient image transformers & distillation through attention. In: Pro- ceedings of the 38th International Conference on Machine Learning. pp. 10347– 10357. PMLR (Jul 2021),https://procee...

2021
[18]

In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Has- sner, T

Wang, W., Zhang, J., Cao, Y., Shen, Y., Tao, D.: Towards Data-Efficient Detec- tion Transformers. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Has- sner, T. (eds.) Computer Vision – ECCV 2022, vol. 13669, pp. 88–105. Springer Nature Switzerland, Cham (2022). https:...

2022 doi
[19]

https://doi.org/10.5281/zenodo.4414861, https://github.com/huggingface/pytorch-image-models CoMViT: Efficient Vision Backbone for Medical Imaging 11

Wightman, R.: PyTorch Image Models. https://doi.org/10.5281/zenodo.4414861, https://github.com/huggingface/pytorch-image-models CoMViT: Efficient Vision Backbone for Medical Imaging 11

doi
[20]

IEEE Transactions on Medical Imaging41(8), 2130–2143 (Aug 2022)

Xu, M., Zhang, T., Zhang, D.: MedRDF: A Robust and Retrain-Less Diagnostic Framework for Medical Pretrained Models Against Adversar- ial Attack. IEEE Transactions on Medical Imaging41(8), 2130–2143 (Aug 2022). https://doi.org/10.1109/TMI.2022.3156268,https://ieeexplore.ieee. o...

2022
[21]

Scientific Data10(1), 41 (Jan 2023)

Yang, J., Shi, R., Wei, D., Liu, Z., Zhao, L., Ke, B., Pfister, H., Ni, B.: MedMNIST v2 - A large-scale lightweight benchmark for 2D and 3D biomedical image classifi- cation. Scientific Data10(1), 41 (Jan 2023). https://doi.org/10.1038/s41597-022- 01721-8,https://www.nature.co...

2023 doi

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.

[1] [14]

Computers in Biology and Medicine157, 106791 (May 2023)

Manzari, O.N., Ahmadabadi, H., Kashiani, H., Shokouhi, S.B., Ayatollahi, A.: MedViT: A Robust Vision Transformer for Generalized Medical Im- age Classification. Computers in Biology and Medicine157, 106791 (May 2023). https://doi.org/10.1016/j.compbiomed.2023.106791,http://arxiv.org/ abs/2302.09462, arXiv:2302.09462 [cs]

arXiv 2023

[2] [1]

Journal of Imaging Informatics in Medicine pp

Aburass, S., Dorgham, O., Al Shaqsi, J., Abu Rumman, M., Al-Kadi, O.: Vision Transformers in Medical Imaging: a Comprehensive Review of Advancements and Applications Across Multiple Diseases. Journal of Imaging Informatics in Medicine pp. 1–44 (Mar 2025). https://doi.org/10.1007/s10278-025-01481-y,https://link. springer.com/article/10.1007/s10278-025-0148...

doi:10.1007/s10278-025-01481-y 2025

[3] [2]

In: 2021 IEEE/CVF International Conference on Computer Vision (ICCV)

Caron, M., Touvron, H., Misra, I., Jegou, H., Mairal, J., Bojanowski, P., Joulin, A.: Emerging Properties in Self-Supervised Vision Transformers. In: 2021 IEEE/CVF International Conference on Computer Vision (ICCV). pp. 9630–9640. IEEE, Mon- treal, QC, Canada (Oct 2021). https://doi.org/10.1109/ICCV48922.2021.00951, https://ieeexplore.ieee.org/document/9709990/

arXiv 2021

[4] [3]

Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., Houlsby, N.: An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (Jun 2021),http://arxiv.org/abs/2010.11929, arXiv:2010.11929 [cs]

arXiv 2021

[5] [4]

https://doi.org/10.1007/978-3-031-43901-8_43,https://link.springer

Du, S., Bayasi, N., Hamarneh, G., Garbi, R.: MDViT: Multi-domain Vi- sion Transformer for Small Medical Image Segmentation Datasets (Oct 2023). https://doi.org/10.1007/978-3-031-43901-8_43,https://link.springer. com/chapter/10.1007/978-3-031-43901-8_43

doi:10.1007/978-3-031-43901-8_43 2023

[6] [5]

IEEE Access12, 117617–117626 (2024)

Ferdous, G.J., Sathi, K.A., Hossain, M.A., Dewan, M.A.A.: SPT-Swin: A Shifted Patch Tokenization Swin Transformer for Image Classification. IEEE Access12, 117617–117626 (2024). https://doi.org/10.1109/ACCESS.2024.3448304,https:// ieeexplore.ieee.org/document/10643534

arXiv 2024

[7] [6]

Scientific Reports 14(1), 12567 (May 2024)

Halder, A., Gharami, S., Sadhu, P., Singh, P.K., Woźniak, M., Ijaz, M.F.: Imple- menting vision transformer for classifying 2D biomedical images. Scientific Reports 14(1), 12567 (May 2024). https://doi.org/10.1038/s41598-024-63094-9,https: //www.nature.com/articles/s41598-024-63094-9, publisher: Nature Publishing Group

work page doi:10.1038/s41598-024-63094-9 2024

[8] [7]

IEEE Transactions on Pattern Analysis and Machine Intelligence45(1), 87–110 (Jan 2023)

Han, K., Wang, Y., Chen, H., Chen, X., Guo, J., Liu, Z., Tang, Y., Xiao, A., Xu, C., Xu, Y., Yang, Z., Zhang, Y., Tao, D.: A Survey on Vision Transformer. IEEE Transactions on Pattern Analysis and Machine Intelligence45(1), 87–110 (Jan 2023). https://doi.org/10.1109/TPAMI.2022.3152247,https://ieeexplore. ieee.org/abstract/document/9716741, conference Name...

arXiv 2023