A generalizable 3D framework and model for self-supervised learning in medical imaging

Anne L. Martel; Anthony Rinaldi; Chris Anderson; Maged Goubran; Rahul G. Krishnan; Sepehr Hosseini; Tony Xu

REVIEW 3 major objections 4 minor 4 cited by

A generalizable 3D framework and model for self-supervised learning in medical imaging

T0 review · 3 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A self-supervised 3D vision transformer pretrained on about 99,000 unlabeled medical volumes is claimed to transfer across MRI, CT, PET, unseen organs, and unseen modalities, beating state-of-the-art and matched baselines on most…

desk verdict A solid 3D SSL recipe and pretraining corpus, but the headline method-vs-method comparison is confounded by compute and resolution; it deserves a serious referee rather than a desk reject. read the letter →

arxiv 2501.11755 v2 pith:RBBZNLHJ submitted 2025-01-20 eess.IV cs.CV

Tony Xu , Sepehr Hosseini , Chris Anderson , Anthony Rinaldi , Rahul G. Krishnan , Anne L. Martel , Maged Goubran This is my paper · ORCID

classification eess.IVcs.CV

keywords self-supervisedlearning3Dmedicalimagingvisiontransformerfoundationmodeltransferimagesegmentationclassificationmultimodalpretraining

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to show that a single self-supervised model can be a general-purpose representation learner for 3D medical images rather than a specialist for one organ or one modality. It introduces 3DINO, a 3D adaptation of an efficient self-distillation and masked-patch-prediction pipeline, and uses it to pretrain 3DINO-ViT on roughly 98,815 unlabeled MRI, CT, and PET volumes from more than ten organs. Across segmentation and classification benchmarks, including an unseen organ and an unseen modality, the pretrained model is reported to outperform state-of-the-art and controlled baselines on most metrics and most labeled-data budgets. If this holds, downstream medical imaging models could be built from shared frozen features with far fewer annotations and without training a separate pretrained model for each task.

What carries the argument

The central object is 3DINO-ViT, a ViT-Large with patches of 16x16x16 pretrained on the 98,815-volume multimodal corpus. The mechanism that carries the argument is 3DINO's two-objective pretraining: an image-level self-distillation loss between a student network and an exponential-moving-average teacher on two global crops, plus a patch-level masked-prediction loss on eight local crops; a high-resolution adaptation stage refines the weights at 112x112x112 input size. For segmentation, a 3D ViT-Adapter built on 3D multi-scale deformable attention injects spatial inductive bias into the frozen transformer, and the pretrained representations are evaluated by finetuning that adapter plus a decoder or by linear probing.

What would settle it

Run a controlled pretraining comparison in which 3DINO and a vanilla-ViT baseline pretrained with the competing method use the same batch size, the same frozen-encoder plus 3D adapter setup, the same finetuning iterations, and the same crop augmentation; if the reported segmentation and classification advantages on the brain-tumor MRI, abdominal CT, brain-age, and lung-CT tasks disappear or fall below significance, the claimed superiority of 3DINO's representation learning is not supported.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that a 3D self-supervised vision transformer can learn representations that serve both image-level and patch-level tasks across multiple imaging modalities at once. The authors state that 3DINO-ViT is the first 3D self-supervised medical imaging model able to extract salient features and generalize across modalities simultaneously, and they support this with transfer results on brain-tumor MRI segmentation, abdominal CT organ segmentation, brain-age classification, lung-CT disease classification, and out-of-distribution left-atrium MRI and breast-ultrasound segmentation. With the encoder frozen and a 3D adapter added for dense tasks, the pretrained weights beat random initialization, the previous state-of-the-art pretrained baseline, and a matched vanilla-ViT baseline pretrained on the same data, with the largest gains in low-label regimes.

Load-bearing premise

The load-bearing premise is that the anisotropic crop strategy, random-resized cropping in the two in-plane axes with the cross-slice extent sampled independently, creates usable training views across a corpus whose volumes range from about 30 slices to about 1,000 slices.

Editorial extensions

If this is right

A single frozen 3DINO-ViT encoder can replace organ- or modality-specific pretraining, removing the need to pretrain separate models for each downstream task.
In low-label regimes the gains are largest: for example, training on 10% of the brain-tumor MRI labels or 25% of the abdominal CT labels reaches results comparable to baselines trained on 100%.
Transfer to an unseen organ (left atrium MRI) and an unseen modality (3D breast ultrasound) means the pretrained weights can initialize tasks outside the pretraining distribution.
Patch-level features are strong enough that a two-layer linear decoder yields large segmentation improvements over other initializations, enabling lightweight dense prediction without a full decoder.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If corpus diversity is the main driver of these gains, adding more PET, ultrasound, or underrepresented organs to pretraining should further improve out-of-distribution transfer; the current corpus is dominated by MRI and CT.
The anisotropic crop rule, random-resized cropping in-plane with independent cross-slice sampling, may itself act as an implicit augmentation that teaches invariance to slice thickness, and an ablation swapping it for isotropic resampling would separate that effect from the pretext objectives.
The 3D ViT-Adapter and the channel-mixing procedure are separable components whose contribution to downstream performance could be measured independently of the pretrained weights.
Because the paper notes that its pretraining used a batch size of 512 while the matched vanilla-ViT baseline used 64, a batch-size-matched comparison is needed before attributing all of the gap to the pretext objectives.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Here's my take. Read this as an engineering-plus-empirics paper, not as a proof that the 3DINO objective beats its predecessor. The stress-test note is correct, and it lands on a load-bearing claim. The Methods section contradicts the 'differing only in pretraining methodology' sentence: 3DINO pretrains with batch size 512 for 125k iterations plus a 12.5k high-resolution adaptation stage at 112^3; MONAI-ViT uses batch size 64 for 100k iterations at 96^3 and no adaptation. That is roughly 8x the samples seen and a different downstream input resolution. The gains over MONAI-ViT could easily come from compute and resolution, not from the pretext task. The Discussion half-admits this by crediting 'the computational efficiency of 3DINO' alongside the pretext quality. So the controlled comparison is not controlled.

What is genuinely new and useful: the 3D adaptation of DINOv2 is careful and concrete—anisotropic RandomResizedCrop to handle variable slice counts, 3D MSDA with octant-based offset initialization, 3D ViT-Adapter for dense tasks. The pretraining corpus of ~98,815 volumes from 35 sources is an order of magnitude larger and broader than Tang et al.'s CT-only set. The downstream evaluation is real: BraTS, BTCV, ICBM, COVID-CT-MD, LA-SEG, and TDSC-ABUS, all external, with per-scan statistics, bootstrapped CIs, and a sensible effort to remove TCGA overlap from BraTS. The out-of-distribution ultrasound result is the most persuasive part of the paper. I also credit the authors for explicitly listing limitations, including the unbalanced organ/modality composition and the missing ViT-Giant experiments.

Soft spots, in proportion. The ablation gap is the main one: no experiment isolates the anisotropic crop, the high-res adaptation, the batch size, or the pretext objective. That is a lot of free parameters for a paper whose headline is a superiority claim. The 'first 3D SSL model to generalize across multiple modalities' claim is broader than the data: pretraining is mostly MRI and CT, with 0.6% PET. That claim should be softened. Weights are promised only upon acceptance, so the reproducibility artifacts are incomplete; code without a commit hash and no weights makes verification harder. None of this makes the paper unserious.

Who this is for: anyone working on 3D medical SSL foundations. You get a well-documented recipe, a large corpus list, and a strong benchmark suite. It deserves peer review, not desk rejection. A serious referee should ask for a matched-compute or at least batch-size-matched ablation, and for the weights to be released before acceptance. I'd cite it as a data point; I'd probably not bring it to reading group until the ablation question is answered.

Referee Report

3 major / 4 minor

Summary. This manuscript presents 3DINO, a 3D adaptation of the DINOv2 self-supervised learning pipeline, and 3DINO-ViT, a ViT-Large model pretrained on approximately 98,815 unlabeled MRI, CT, and PET volumes assembled from 35 public and internal sources. The method combines image-level self-distillation with masked patch reconstruction, uses a custom anisotropic random-resized crop for 3D inputs, and introduces a 3D ViT-Adapter with multi-scale deformable attention for downstream segmentation. The authors evaluate frozen-encoder transfer on BraTS, BTCV, LA-SEG, and TDSC-ABUS segmentation and linear probing on ICBM and COVID-CT-MD classification, across multiple labeled-data fractions, comparing against random initialization, a Swin Transformer baseline, and a same-architecture MONAI-ViT baseline pretrained on the same data. They report that 3DINO-ViT outperforms these baselines on most metrics, especially at low label fractions and on out-of-distribution tasks, and claim it is the first 3D SSL medical imaging model that generalizes across multiple modalities simultaneously.

Significance. The resource contribution is substantial: an unusually large multimodal 3D pretraining corpus, a planned public code and weight release, and six external benchmark tasks evaluated with per-scan bootstrap confidence intervals and multiple label fractions. The inclusion of a MONAI-ViT baseline with the same ViT-Large architecture and same pretraining data is the right way to isolate the pretext objective, and the removal of TCGA-GBM/LGG from the BraTS finetuning split shows care about data leakage. If the reported gains are truly attributable to the 3DINO objective rather than to compute and input resolution, the work would be a useful step toward general-purpose 3D medical imaging foundation models. The current manuscript, however, does not yet establish that attribution because of a major confound in the key controlled comparison, and it overstates the out-of-distribution claim for cardiac data.

major comments (3)

[Methods: 3DINO and MONAI-ViT pretraining details; 3DINO segmentation] The MONAI-ViT control is confounded by pretraining compute and downstream input resolution. 3DINO uses a total batch size of 512 for 125,000 iterations plus 12,500 high-resolution adaptation iterations at crop size 112^3, while MONAI-ViT uses a total batch size of 64 for 100,000 iterations at 96^3; 3DINO therefore sees roughly 10 times more pretraining sample presentations and is evaluated for segmentation at 112^3 versus 96^3. Because the headline claim is that the 3DINO objective extracts more salient features than the MONAI-ViT objective, these differences are load-bearing. The Discussion's own statement that gains are attributable 'jointly to the quality of the pretext task and the computational efficiency of 3DINO' concedes the confound. To support the central claim, the authors should either train MONAI-ViT with matched batch size and iterations and a matched high-resolution stage, or explicitly ablate the high-resolution adaptation and evaluate both models at a common input size.
[LA-SEG dataset and Performance on unseen organs and modalities] The paper twice describes left atrium MRI segmentation as an 'unseen organ', but the LA-SEG Methods paragraph states that 'the heart makes up a very small subset of the pretraining dataset'. This is a direct contradiction. If any cardiac volumes were in the 98,815-volume pretraining set, LA-SEG is a low-resource in-distribution organ rather than a completely unseen one. The out-of-distribution claim should be restricted to TDSC-ABUS (ultrasound), and the authors should quantify any cardiac overlap or explicitly exclude heart volumes from pretraining before claiming an unseen-organ result.
[Methods: 3DINO RandomResizedCrop adaptation] The anisotropic crop strategy is a key novel design choice but is not validated or ablated. The Methods rely on the expectation that 'using a large variety of pretraining datasets will allow the model to learn to generalize to various volume sizes', yet no experiment varies the crop scheme, and the corpus spans roughly 1,000 slices in whole-body CT to about 30 slices in fastMRI knee volumes. At minimum, the authors should report the distribution of actual crop aspect ratios produced during pretraining and, ideally, compare against an isotropic or depth-aligned crop on one or two downstream tasks.

minor comments (4)

[Within and out-of-distribution downstream tasks] The spelling '3DINO-VIT' appears in the Results section and should be corrected to '3DINO-ViT'.
[MONAI-ViT pretraining implementation details] The sentence 'The Swin ViT pretraining code was taken from the original work 3' cites reference 3, which is a general deep-learning review, rather than reference 21 (Tang et al.); this citation should be corrected.
[Methods: loss definitions] The equations for the DINO image-level loss and the iBOT patch-level loss are not rendered in the provided manuscript; they should appear in the final version because the patch-level masking objective is central to the method.
[Introduction and Discussion] The 'first' claims, such as 'the first 3D SSL-based medical imaging model that can extract salient features and generalize across multiple modalities simultaneously', should be qualified by a more complete comparison with prior 3D SSL methods, including Taleb et al. and Chen et al., which are cited elsewhere in the paper.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: downstream evaluations are external transfer benchmarks, not quantities defined by the pretraining objective.

full rationale

The paper's central empirical claims are transfer evaluations on external benchmarks (BraTS, BTCV, LA-SEG, TDSC-ABUS, ICBM, COVID-CT-MD) that were not used to define the pretraining loss or to fit the frozen 3DINO-ViT weights. The 3DINO objective is a DINOv2-style combination of image-level self-distillation and patch-level masked-image modeling; none of these terms is defined by downstream Dice, AUC, or F1, so no reported metric is forced by construction. The comparison against MONAI-ViT is not perfectly controlled (batch size 512 vs 64, 125k vs 100k iterations, plus a 112^3 high-resolution adaptation stage), and the authors themselves attribute gains 'jointly to the quality of the pretext task and the computational efficiency of 3DINO'; this is a confound in the controlled comparison, not a circular reduction, because the downstream results still come from held-out benchmarks rather than from the pretraining recipe. The manuscript also explicitly acknowledges limitations, including the MRI/CT imbalance in pretraining and the lack of ViT-Giant experiments, and these admissions do not reveal a definitional loop. No load-bearing self-citation appears: DINOv2 and Tang et al. are cited as external sources, and the 'first 3D SSL-based medical imaging model' claim is a stated priority assertion rather than a result derived from the authors' own prior theorems. One transparency caveat is that the full pretraining corpus is only summarized in Extended Data Table 1, so exact overlap with downstream datasets cannot be audited from this text alone; that is a leakage-risk concern, not an established circularity.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central empirical claim depends mostly on standard ML hyperparameters and design choices inherited from DINOv2; none are fitted to the downstream benchmarks, so the circularity burden is low. No new physical or conceptual entities are introduced. The main unverified premises are the transferability of DINOv2's 2D recipe to anisotropic 3D medical volumes and the cleanliness of the pretraining and finetuning data separation.

free parameters (6)

Pretraining batch size = 512 standard, 256 high-resolution adaptation
Chosen for four A100 80GB GPUs; differs from MONAI-ViT's batch size of 64, confounding attribution of downstream gains to the pretraining objective.
Global and local crop sizes = 96^3 and 48^3; high-resolution 112^3 and 64^3
Hand-picked from the DINOv2 recipe; crop geometry is central to the SSL view-generation process.
Learning rate and EMA schedule = LR 0.002 and 0.001, EMA 0.992 to 1.000 cosine
Inherited from DINOv2; no sensitivity analysis is provided in the paper.
Pretraining length = 125,000 iterations plus 12,500 high-resolution iterations
Fixed by an approximate 11-day compute budget; convergence is not demonstrated.
Patch size = 16x16x16
Only one patch size is evaluated; it determines token resolution and interacts with the adapter scales.
3D ViT-Adapter capacity = feature size 256, 8 MSDA heads, scales 1/4, 1/8, 1/16
Chosen to control compute; no ablation isolates the adapter's contribution to segmentation improvements.

assumptions (5)

domain assumption DINOv2 objectives and regularizers transfer from 2D natural images to 3D medical volumes.
3DINO is a port of DINOv2, reference 28; the paper provides downstream results but no isolated evidence that each transferred design choice remains effective in 3D.
domain assumption Anisotropic crops with independent cross-slice resizing produce coherent 3D views.
In Methods, 3DINO augmentations use 2D RandomResizedCrop in-plane and independent cross-slice sampling; the paper only states it 'expects' a variety of pretraining data will teach generalization to volume sizes.
domain assumption No hidden patient overlap exists between pretraining and downstream test sets beyond the TCGA cases removed from BraTS.
Only the BraTS and TCGA-GBM/TCGA-LGG overlap is handled explicitly; overlap between other brain MRI pretraining sources such as ADNI, OASIS, and HCP and the ICBM test set is assumed absent.
ad hoc to paper The 3D MSDA octant-based offset initialization is a valid substitute for the 2D angular initialization.
In Methods, 3D ViT-Adapter, initial offsets point to the eight 3D octants because a full 3D angular separation would require an intractable number of heads; no ablation or theoretical justification is offered.
domain assumption The DINOv2 high-resolution adaptation stage transfers to 3D data.
The 112^3 adaptation stage is copied from the 2D DINOv2 recipe and assumed to give comparable benefits in 3D without a dedicated study.

how reviews work

0 comments

Cite this review

Pith. "Pith review of A generalizable 3D framework and model for self-supervised learning in medical imaging." pith.science (2026). https://pith.science/paper/RBBZNLHJ

@misc{pith2026250111755,
  author       = {Pith},
  title        = {Pith review of: A generalizable 3D framework and model for self-supervised learning in medical imaging},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RBBZNLHJ}},
  note         = {Machine review of arXiv:2501.11755}
}

read the original abstract

Current self-supervised learning methods for 3D medical imaging rely on simple pretext formulations and organ- or modality-specific datasets, limiting their generalizability and scalability. We present 3DINO, a cutting-edge SSL method adapted to 3D datasets, and use it to pretrain 3DINO-ViT: a general-purpose medical imaging model, on an exceptionally large, multimodal, and multi-organ dataset of ~100,000 3D medical imaging scans from over 10 organs. We validate 3DINO-ViT using extensive experiments on numerous medical imaging segmentation and classification tasks. Our results demonstrate that 3DINO-ViT generalizes across modalities and organs, including out-of-distribution tasks and datasets, outperforming state-of-the-art methods on the majority of evaluation metrics and labeled dataset sizes. Our 3DINO framework and 3DINO-ViT will be made available to enable research on 3D foundation models or further finetuning for a wide range of medical imaging applications.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

The MAMA-MIA Challenge: Advancing Generalizability and Fairness in Breast MRI Tumor Segmentation and Treatment Response Prediction
cs.CV 2026-03 conditional novelty 6.0 of 10

In a cross-continental benchmark, breast tumor segmentation generalized reasonably across institutions, but predicting pathologic complete response from pre-treatment DCE-MRI alone was no better than random for nearly...
Live(r) Die: Predicting Survival in Colorectal Liver Metastasis
cs.CV 2025-09 conditional novelty 6.0 of 10

A fully automated pre/post-contrast MRI framework, combining prompt-based segmentation with autoencoder multiple-instance survival analysis, improves CRLM post-surgery survival prediction over clinical and genomic bio...
Revisiting 2D Foundation Models for Scalable 3D Medical Image Classification
cs.CV 2025-12 conditional novelty 5.0 of 10

A frozen 2D vision foundation model with lightweight LoRA adapters and attention-based slice fusion achieves state-of-the-art 3D medical image classification across 12 tasks with about 1M trainable parameters per task.
Multi-Scale Feature Fusion with Image-Driven Spatial Integration for Left Atrium Segmentation from Cardiac MRI Images
cs.CV 2025-02 conditional novelty 4.0 of 10

A DINOv2 encoder with a UNet decoder, learned multi-scale feature weights, and input image integration improves left atrium segmentation over nnUNet on the LAScarQS 2022 dataset.

Reference graph

Works this paper leans on

64 extracted references · 55 canonical work pages · cited by 4 Pith papers

[1]

Barragán-Montero, A. et al. Artificial intelligence and machine learning for medical imaging: A technology review. Phys. Med. 83 , 242–256 (2021)

work page 2021
[2]

Bi, W. L. et al. Artificial intelligence in cancer imaging: Clinical challenges and applications. CA Cancer J. Clin. 69 , 127–157 (2019)

work page 2019
[3]

& Zequera-Diaz, M

Anaya-Isaza, A., Mera-Jiménez, L. & Zequera-Diaz, M. An overview of deep learning in medical imaging. Informatics in Medicine Unlocked 26 , 100723 (2021)

work page 2021
[4]

& Singh Verma, V

Wadhwa, A., Bhardwaj, A. & Singh Verma, V. A review on brain tumor segmentation of MRI images. Magn. Reson. Imaging 61 , 247–259 (2019)

work page 2019
[5]

Zhao, X. et al. Deep learning–based fully automated detection and segmentation of lymph nodes on multiparametric-mri for rectal cancer: A multicentre study. eBioMedicine 56 , (2020)

work page 2020
[6]

Iqbal, S. et al. Prostate Cancer Detection Using Deep Learning and Traditional Techniques. IEEE Access 9 , 27085–27100 (2021)

work page 2021
[7]

O., Linsen, P

Martins Jarnalo, C. O., Linsen, P. V. M., Blazís, S. P., van der Valk, P. H. M. & Dickerscheid, D. B. M. Clinical evaluation of a deep-learning-based computer-aided detection system for the detection of pulmonary nodules in a large teaching hospital. Clin. Radiol. 76 , 838–845 (2021)

work page 2021
[8]

M., Elshennawy, N

Ibrahim, D. M., Elshennawy, N. M. & Sarhan, A. M. Deep-chest: Multi-classification deep learning model for diagnosing COVID-19, pneumonia, and lung cancer chest diseases. Comput. Biol. Med. 132 , 104348 (2021)

work page 2021

Show all 64 references

[9]

Adweb, K. M. A., Cavus, N. & Sekeroglu, B. Cervical Cancer Diagnosis Using Very Deep Networks Over Different Activation Functions. IEEE Access 9 , 46612–46625 (2021)

2021
[10]

Urushibara, A. et al. Diagnosing uterine cervical cancer on a single T2-weighted image: Comparison between deep learning versus radiologists. Eur. J. Radiol. 135 , 109471 (2021). 30

2021
[11]

De Fauw, J. et al. Clinically applicable deep learning for diagnosis and referral in retinal disease. Nat. Med. 24 , 1342–1350 (2018)

2018
[12]

Chan, H.-P., Hadjiiski, L. M. & Samala, R. K. Computer-aided diagnosis in the era of deep learning. Med. Phys. 47 , e218–e227 (2020)

2020
[13]

L., Gómez-Pulido, J

Caballé-Cervigón, N., Castillo-Sequera, J. L., Gómez-Pulido, J. A., Gómez-Pulido, J. M. & Polo-Luque, M. L. Machine Learning Applied to Diagnosis of Human Diseases: A Systematic Review. NATO Adv. Sci. Inst. Ser. E Appl. Sci. 10 , 5135 (2020)

2020
[14]

Wang, C. et al. Deep learning for predicting subtype classification and survival of lung adenocarcinoma on computed tomography. Transl. Oncol. 14 , 101141 (2021)

2021
[15]

Current trends in the use of machine learning for diagnostics and/or risk stratification in cardiovascular disease

Maurovich-Horvat, P. Current trends in the use of machine learning for diagnostics and/or risk stratification in cardiovascular disease. Cardiovasc. Res. 117 , e67–e69 (2021)

2021
[16]

Cè, M. et al. Artificial intelligence in breast cancer imaging: risk stratification, lesion detection and classification, treatment planning and prognosis-a narrative review. Explor Target Antitumor Ther 3 , 795–816 (2022)

2022
[17]

M., Lee, K

Kim, H., Goo, J. M., Lee, K. H., Kim, Y. T. & Park, C. M. Preoperative CT-based Deep Learning Model for Predicting Disease-Free Survival in Patients with Lung Adenocarcinomas. Radiology 296 , 216–224 (2020)

2020
[18]

Jin, C. et al. Predicting treatment response from longitudinal images using multi-task deep learning. Nat. Commun. 12 , 1–11 (2021)

2021
[19]

Taleb, A. et al. 3d self-supervised methods for medical imaging. Adv. Neural Inf. Process. Syst. 33 , 18158–18172 (2020)

2020
[20]

Chen, Z. et al. Masked image modeling advances 3d medical image analysis. in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision 1970–1980 (2023)

2023
[21]

Tang, Y. et al. Self-supervised pre-training of swin Transformers for 3D medical image analysis. Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. 20698–20708 (2021) doi:10.1109/CVPR52688.2022.02007. 31

2021
[22]

& Hinton, G

Chen, T., Kornblith, S., Norouzi, M. & Hinton, G. A simple framework for contrastive learning of visual representations. in International conference on machine learning 1597–1607 (PMLR, 2020)

2020
[23]

Grill, J.-B. et al. Bootstrap your own latent-a new approach to self-supervised learning. Adv. Neural Inf. Process. Syst. 33 , 21271–21284 (2020)

2020
[24]

Caron, M. et al. Emerging Properties in Self-Supervised Vision Transformers. in Proceedings of the International Conference on Computer Vision (ICCV) (2021)

2021
[25]

He, K. et al. Masked autoencoders are scalable vision learners. in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition 16000–16009 (2022)

2022
[26]

Zhou, J. et al. ibot: Image bert pre-training with online tokenizer. arXiv preprint arXiv:2111. 07832 (2021)

2021
[27]

Avesta, A. et al. Comparing 3D, 2.5D, and 2D Approaches to Brain Image Auto-Segmentation. Bioengineering (Basel) 10 , (2023)

2023
[28]

Oquab, M. et al. DINOv2: Learning Robust Visual Features without Supervision. arXiv:2304. 07193 Preprint at (2023)

2023
[29]

Dosovitskiy, A. et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010. 11929 (2020)

2020
[30]

Chen, Z. et al. Vision Transformer Adapter for Dense Predictions. arXiv [cs.CV] (2022)

2022
[31]

Zhu, X. et al. Deformable DETR: Deformable Transformers for End-to-End Object Detection. arXiv [cs.CV] (2020)

2020
[32]

Baid, U. et al. The rsna-asnr-miccai brats 2021 benchmark on brain tumor segmentation and radiogenomic classification. arXiv preprint arXiv:2107. 02314 (2021)

2021
[33]

Landman, B. et al. Miccai multi-atlas labeling beyond the cranial vault--workshop and challenge. in Proc. MICCAI Multi-Atlas Labeling Beyond Cranial Vault—Workshop Challenge vol. 5 12 (2015)

2015
[34]

Xiong, Z. et al. A global benchmark of algorithms for segmenting the left atrium from late gadolinium-enhanced cardiac magnetic resonance imaging. Med. Image Anal. 67 , 101832 (2021). 32

2021
[35]

Wang, K. et al. Tumor Detection, Segmentation and Classification Challenge on Automated 3D Breast Ultrasound (ABUS) 2023. Tdsc-Abus2023 - Grand Challenge https://tdsc-abus2023.grand-challenge.org/

2023
[36]

Mazziotta, J. et al. A probabilistic atlas and reference system for the human brain: International Consortium for Brain Mapping (ICBM). Philos. Trans. R. Soc. Lond. B Biol. Sci. 356 , 1293–1322 (2001)

2001
[37]

Afshar, P. et al. COVID-CT-MD, COVID-19 computed tomography scan dataset applicable in machine learning and deep learning. Scientific Data 8 , 1–8 (2021)

2021
[38]

Wang, H. et al. SAM-Med3D. arXiv [cs.CV] (2023)

2023
[39]

Kirillov, A. et al. Segment Anything. ICCV 3992–4003 (2023) doi:10.1109/ICCV51070.2023.00371

2023
[40]

Knoll, F. et al. fastMRI: A publicly available raw k-space and DICOM dataset of knee images for accelerated MR image reconstruction using machine learning. Radiology: Artificial Intelligence 2 , e190007 (2020)

2020
[41]

Zbontar, J. et al. fastMRI: An open dataset and benchmarks for accelerated MRI. arXiv preprint arXiv:1811. 08839 (2018)

2018
[42]

Flanders, A. E. et al. Construction of a machine learning dataset through collaboration: the RSNA 2019 brain CT hemorrhage challenge. Radiology: Artificial Intelligence 2 , e190211 (2020)

2020
[43]

Data from the National Lung Screening Trial (NLST) [Data set]

National Lung Screening Trial Research Team. Data from the National Lung Screening Trial (NLST) [Data set]. The Cancer Imaging Archive https://doi.org/10.7937/TCIA.HMQ8-J677 (2013)

2013 doi
[44]

The National Lung Screening Trial: overview and study design

National Lung Screening Trial Research Team et al. The National Lung Screening Trial: overview and study design. Radiology 258 , 243–253 (2011)

2011
[45]

Hugo, G. D. et al. A longitudinal four-dimensional computed tomography and cone beam computed tomography dataset for image-guided radiation therapy research in lung cancer. Med. Phys. 44 , 762–771 (2017)

2017
[46]

Hugo, G. D. et al. Data from 4D lung imaging of NSCLC patients. The Cancer Imaging 33 Archive 10 , K9 (2016)

2016
[47]

Boone, L. et al. ROOD-MRI: Benchmarking the robustness of deep learning segmentation models to out-of-distribution and corrupted data in MRI. Neuroimage 278 , 120289 (2023)

2023
[48]

Caron, M. et al. Unsupervised learning of visual features by contrasting cluster assignments. Adv. Neural Inf. Process. Syst. abs/2006.09882 , (2020)

2020 arXiv
[49]

& Jégou, H

Sablayrolles, A., Douze, M., Schmid, C. & Jégou, H. Spreading vectors for similarity search. arXiv preprint arXiv:1806. 03198 (2018)

2018
[50]

& Wei, F

Bao, H., Dong, L., Piao, S. & Wei, F. Beit: Bert pre-training of image transformers. arXiv preprint arXiv:2106. 08254 (2021)

2021
[51]

Wagner, D. et al. On the Importance of Hyperparameters and Data Augmentation for Self-Supervised Learning. arXiv [cs.LG] (2022)

2022
[52]

Selfridge, A. R. et al. Low-Dose CT Images of Healthy Cohort (Healthy-Total-Body-CTs) (Version 1) [Data set]. The Cancer Imaging Archive (2023) doi:10.7937/NC7Z-4F76

2023 doi
[53]

Sundar, L. K. S. et al. Fully automated, semantic segmentation of whole-body 18F-FDG PET/CT images based on data-centric artificial intelligence. J. Nucl. Med. 63 , 1941–1948 (2022)

2022
[54]

Liu, Z. et al. Swin Transformer: Hierarchical vision Transformer using shifted windows. ICCV 9992–10002 (2021) doi:10.1109/ICCV48922.2021.00986

2021
[55]

Scarpace, L. et al. The Cancer Genome Atlas Glioblastoma Multiforme Collection (TCGA-GBM)(Version 4)[Data set]. Cancer Imaging Arch. Published online (2016) doi:10.7937/K9/TCIA.2016.RNYFUYE9

2016 doi
[56]

Pedano, N. et al. The cancer genome atlas low grade glioma collection (TCGA-LGG)(version 3)[Data set]. The Cancer Imaging Archive (2016) doi:10.7937/K9/TCIA.2016.L4LTD3TK

2016 doi
[57]

Ntiri, E. E. et al. Improved Segmentation of the Intracranial and Ventricular Volumes in Populations with Cerebrovascular Lesions and Atrophy Using 3D CNNs. Neuroinformatics 19 , 597–618 (2021). 34

2021
[58]

Steiner, A. et al. How to train your ViT? Data, Augmentation, and Regularization in Vision Transformers. arXiv [cs.CV] (2021)

2021
[59]

Brown, T. B. et al. Language Models are Few-Shot Learners. Adv. Neural Inf. Process. Syst. abs/2005.14165 , (2020)

2020 arXiv
[60]

& Brox, T

Ronneberger, O., Fischer, P. & Brox, T. U-Net: Convolutional Networks for Biomedical Image Segmentation. in Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015 234–241 (Springer International Publishing, 2015). doi:10.1007/978-3-319-24574-4_28

2015 doi
[61]

& Gimpel, K

Hendrycks, D. & Gimpel, K. Gaussian error linear units (gelus). arXiv preprint arXiv:1606. 08415 (2016)

2016
[62]

& Hutter, F

Loshchilov, I. & Hutter, F. Decoupled Weight Decay Regularization. arXiv [cs.LG] (2017)

2017
[63]

Hatamizadeh, A. et al. Unetr: Transformers for 3d medical image segmentation. in Proceedings of the IEEE/CVF winter conference on applications of computer vision 574–584 (2022)

2022
[64]

Clark, K. et al. The Cancer Imaging Archive (TCIA): maintaining and operating a public information repository. J. Digit. Imaging 26 , 1045–1057 (2013). 35

2013

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

Barragán-Montero, A. et al. Artificial intelligence and machine learning for medical imaging: A technology review. Phys. Med. 83 , 242–256 (2021)

work page 2021

[2] [2]

Bi, W. L. et al. Artificial intelligence in cancer imaging: Clinical challenges and applications. CA Cancer J. Clin. 69 , 127–157 (2019)

work page 2019

[3] [3]

& Zequera-Diaz, M

Anaya-Isaza, A., Mera-Jiménez, L. & Zequera-Diaz, M. An overview of deep learning in medical imaging. Informatics in Medicine Unlocked 26 , 100723 (2021)

work page 2021

[4] [4]

& Singh Verma, V

Wadhwa, A., Bhardwaj, A. & Singh Verma, V. A review on brain tumor segmentation of MRI images. Magn. Reson. Imaging 61 , 247–259 (2019)

work page 2019

[5] [5]

Zhao, X. et al. Deep learning–based fully automated detection and segmentation of lymph nodes on multiparametric-mri for rectal cancer: A multicentre study. eBioMedicine 56 , (2020)

work page 2020

[6] [6]

Iqbal, S. et al. Prostate Cancer Detection Using Deep Learning and Traditional Techniques. IEEE Access 9 , 27085–27100 (2021)

work page 2021

[7] [7]

O., Linsen, P

Martins Jarnalo, C. O., Linsen, P. V. M., Blazís, S. P., van der Valk, P. H. M. & Dickerscheid, D. B. M. Clinical evaluation of a deep-learning-based computer-aided detection system for the detection of pulmonary nodules in a large teaching hospital. Clin. Radiol. 76 , 838–845 (2021)

work page 2021

[8] [8]

M., Elshennawy, N

Ibrahim, D. M., Elshennawy, N. M. & Sarhan, A. M. Deep-chest: Multi-classification deep learning model for diagnosing COVID-19, pneumonia, and lung cancer chest diseases. Comput. Biol. Med. 132 , 104348 (2021)

work page 2021