CoralBay: A Self-Supervised CT Foundation Model

Fei Tang; Ioannis Gatopoulos; Nicolas K\"anzig; Sebastian Ot\'alora

CoralBay: A Self-Supervised CT Foundation Model

T0 review · 1 major / 0 minor · reviewed 2026-06-28 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read CoralBay adapts self-distillation to 3D CT volumes using a Swin backbone to learn transferable representations.

desk verdict CoralBay is a direct 3D Swin + multi-scale DINO extension for CT with a useful leaderboard addition but zero experimental results shown. read the letter →

arxiv 2606.03888 v1 pith:4VUY74K5 submitted 2026-06-02 cs.CV cs.LG

Ioannis Gatopoulos , Nicolas Känzig , Sebastian Otálora , Fei Tang This is my paper

classification cs.CVcs.LG

keywords self-supervisedlearningCTimaging3DrepresentationsSwintransformerself-distillationvolumetricmedicalradiologicaltasksfoundationmodel

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces a self-supervised pre-training method for CT scans that works directly on three-dimensional volumes rather than slices. It extends an existing self-distillation approach by feeding concatenated multi-scale features from a hierarchical 3D Swin backbone into the student-teacher process. This setup is intended to encode both overall anatomy and local tissue details that are specific to volumetric CT data, such as spatial continuity and Hounsfield unit intensities. A reader would care because most current pre-training relies on 2D natural images that miss these properties, limiting how well models handle real clinical scans. The authors also release a unified evaluation suite to compare volumetric methods on multiple radiological tasks.

What carries the argument

Self-distillation on concatenated multi-scale features from a hierarchical 3D Swin backbone, which processes volumetric CT data to capture spatial and intensity information.

What would settle it

If models trained with CoralBay show no consistent gains over 2D pre-trained baselines or random initialization when evaluated on a standardized set of CT segmentation and classification tasks spanning multiple body regions, the transfer claim would be falsified.

Watch

Extended reading notes

Core claim

CoralBay extends DINO by using a hierarchical 3D Swin backbone and applying self-distillation to concatenated multi-scale features, enabling data-efficient self-supervised learning of rich spatial representations that encode both global semantics and fine-grained local structure for CT scans.

Load-bearing premise

That self-distillation on multi-scale 3D Swin features from CT volumes will capture spatial continuity, organ anatomy, and tissue intensity properties sufficiently to produce transferable representations.

Editorial extensions

If this is right

CoralBay produces representations that transfer effectively to a wide range of downstream radiological tasks.
Performance remains strong and consistent across diverse anatomical targets.
The method supports data-efficient learning without large labeled CT datasets.
A public, reproducible 3D radiology leaderboard unifies multiple datasets for standardized evaluation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The multi-scale concatenation step could be tested on other 3D medical volumes such as MRI to check whether the same adaptation works beyond CT.
If the learned features prove robust, they might reduce the amount of task-specific labeled data needed for training clinical segmentation or detection models.
The approach implies that 3D-specific pre-training is required rather than relying on transferred 2D weights for volumetric modalities.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

read the letter

The paper's main move is to take the DINO self-distillation recipe, swap in a hierarchical 3D Swin backbone, and distill on concatenated multi-scale features instead of single-scale ones. That combination for CT volumes is not in the prior DINO literature, so the named framework and the specific architectural choices count as new.

It does a clear job laying out the differences between natural 2D images and CT (spatial continuity, organ scale, Hounsfield intensities) and why those matter for pre-training. The decision to contribute a unified 3D radiology leaderboard inside the eva framework is also concrete and reusable; anyone running volumetric SSL experiments will appreciate having a single place to compare.

The obvious gap is that the abstract supplies no training details, no downstream numbers, no ablations, and no error bars. The claim that CoralBay "transfers effectively" and shows "strong and consistent performance" therefore sits on an untested assumption that the multi-scale 3D setup will actually capture tissue properties better than existing 3D methods. That is an empirical question, not a logical flaw, but it leaves the central performance argument unsupported.

The work is aimed at labs already doing self-supervised learning on medical volumes who need either a new baseline or a standardized benchmark. A reader who wants a verified foundation model or reproducible gains will have to wait for the full experiments. I would send the paper to peer review once the results section is filled in, because the setup is reasonable and the leaderboard piece is independently useful.

Referee Report

1 major / 0 minor

Summary. The paper introduces CoralBay, a self-supervised CT foundation model that extends the DINO framework using a hierarchical 3D Swin backbone and applies self-distillation to concatenated multi-scale features. It claims this enables data-efficient learning of spatial representations for CT scans that transfer effectively to a wide range of downstream radiological tasks across diverse anatomical targets. Additionally, it contributes a public, reproducible 3D radiology leaderboard to the eva framework.

Significance. If the results hold, this would represent a meaningful advance in self-supervised learning for volumetric medical imaging by adapting 2D methods to 3D CT data, potentially improving performance on tasks requiring understanding of spatial continuity and tissue properties. The open leaderboard is a positive contribution for benchmarking in the field.

major comments (1)

[Abstract] Abstract: The central claim that 'CoralBay transfers effectively to a wide range of downstream radiological tasks, demonstrating strong and consistent performance across diverse anatomical targets' is presented without any supporting experiments, results, tables, or figures in the manuscript. This makes the primary contribution unassessable.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for their review. We address the single major comment below.

read point-by-point responses

Referee: [Abstract] Abstract: The central claim that 'CoralBay transfers effectively to a wide range of downstream radiological tasks, demonstrating strong and consistent performance across diverse anatomical targets' is presented without any supporting experiments, results, tables, or figures in the manuscript. This makes the primary contribution unassessable.

Authors: We agree that the abstract claim requires supporting evidence for the contribution to be assessable. The submitted manuscript version does not contain the experimental results, tables, or figures on downstream tasks. We will revise the manuscript to add the missing Experiments section with quantitative results across the claimed radiological tasks and anatomical targets. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity identified

full rationale

The paper describes an architectural extension of the DINO self-distillation framework to a hierarchical 3D Swin backbone with multi-scale feature concatenation for CT volumes. No equations, derivations, or parameter-fitting steps are present in the abstract or described construction. Performance claims on downstream tasks are framed as empirical outcomes rather than logical necessities derived from the model definition itself. No self-citation load-bearing arguments, uniqueness theorems, or ansatz smuggling appear. The derivation chain is self-contained as a standard model proposal whose validity rests on external evaluation rather than internal reduction to inputs.

Assumptions & free parameters 0 free parameters · 0 assumptions · 0 invented entities

Abstract-only; no free parameters, axioms, or invented entities are specified or derivable from the text.

how reviews work

0 comments

Cite this review

Pith. "Pith review of CoralBay: A Self-Supervised CT Foundation Model." pith.science (2026). https://pith.science/paper/4VUY74K5

@misc{pith2026260603888,
  author       = {Pith},
  title        = {Pith review of: CoralBay: A Self-Supervised CT Foundation Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4VUY74K5}},
  note         = {Machine review of arXiv:2606.03888}
}

read the original abstract

Self-supervised learning has enabled large-scale pre-training on 2D natural images, producing general-purpose visual representations that transfer effectively across tasks. However, many medical imaging modalities, such as CT scans, are inherently three-dimensional and differ fundamentally from natural images in both structure and semantics. Volumetric modalities capture spatial continuity, organ anatomy, and intensity-based tissue properties (e.g., Hounsfield Units), which are not adequately modeled by 2D pre-training. To bridge this gap, we introduce CoralBay, a self-distillation framework that extends DINO by using a hierarchical 3D Swin backbone and applying self-distillation to concatenated multi-scale features, enabling data-efficient self-supervised learning of rich spatial representations that encode both global semantics and fine-grained local structure. As a result, CoralBay transfers effectively to a wide range of downstream radiological tasks, demonstrating strong and consistent performance across diverse anatomical targets. In addition, we contribute to the open-source \eva framework by introducing a public, reproducible 3D radiology leaderboard that unifies multiple datasets and establishes a standardized benchmark for evaluating volumetric representation learning methods.

Figures

Figures reproduced from arXiv: 2606.03888 by the authors.

**Figure 1.** Challenges in 3D CT Data Representation. Left: The impact of HU windowing on anatomical visibility; narrow windows enhance soft tissue but clip highdensity information. Center: Variability in slice thickness leads to the partial volume effect, where thinner slices provide higher spatial resolution while thicker slices increase the signal-to-noise ratio at the cost of blurring. Right: The three standard orthogonal p… view at source ↗

**Figure 2.** Self-supervised training and downstream inference pipeline. Top (Training): A DINO-based distillation framework processes global (963 ) and local (483 ) crops through student and EMA-updated teacher 3D SwinTransformer backbones to minimize distribution cross-entropy loss. Bottom (Inference): A sliding window technique processes 963 ROI crops of the scan through the encoder. These downsampled intermediate features a… view at source ↗

**Figure 3.** Ablation studies. Top Left: Scaling model & pre-training dataset size. Top right: 2D vs. 3D encoder-decoder configurations (frozen encoder). Bottom: Label efficiency across tasks of increasing difficulty (from left → right). 2D vs 3D Modelling Replacing the standard DINO ViT-B/16 2D encoder with a Swin UNETR-B 2D backbone pretrained using DINO within the CoralBay framework yields only a marginal gain when paired wi… view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

OrganLens: Organ-Specific Representation Learning for CT Foundation Models
cs.CV 2026-07 conditional novelty 6.0 of 10

OrganLens conditions a shared CT encoder on an organ identity and uses mask-supervised pooling to produce 11 organ-specific representations from the same volume, with organ-matched representations improving downstream...

Reference graph

Works this paper leans on

19 extracted references · 3 canonical work pages · cited by 1 Pith paper

[1]

On the Opportunities and Risks of Foundation Models

Bommasani, R., Hudson, D.A., Adeli, E., et al.: On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258 (2021)

work page Pith review arXiv 2021
[2]

Proceedings of the IEEE/CVF International Conference on Computer Vision (2021)

Caron, M., Touvron, H., Misra, I., et al.: Emerging properties in self-supervised vision transformers. Proceedings of the IEEE/CVF International Conference on Computer Vision (2021)

2021
[3]

In: International Conference on Machine Learning (ICML) (2020)

Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for con- trastive learning of visual representations. In: International Conference on Machine Learning (ICML) (2020)

2020
[4]

In: Medical Imaging with Deep Learning (2024)

Gatopoulos,I.,Känzig,N.,Moser,R.,Otálora,S.,etal.:eva:Evaluationframework for pathology foundation models. In: Medical Imaging with Deep Learning (2024)

2024
[5]

Advances in Neural Information Processing Systems (NeurIPS) (2020)

Grill, J.B., Strub, F., Altché, F., et al.: Bootstrap your own latent: A new approach to self-supervised learning. Advances in Neural Information Processing Systems (NeurIPS) (2020)

2020
[6]

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2022)

Hatamizadeh, A., Tang, Y., Nath, V., et al.: Swin unetr: Swin transformers for se- mantic segmentation of brain tumors in mri images. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2022)

2022
[7]

In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

He, K., Fan, H., Wu, Y., Xie, S., Girshick, R.: Momentum contrast for unsupervised visual representation learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 9729–9738 (2020)

2020
[8]

IEEE Transactions on Pattern Analysis and Machine Intelligence 43(11), 4037–4058 (2020)

Jing, L., Tian, Y.: Self-supervised visual feature learning with deep neural net- works: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence 43(11), 4037–4058 (2020)

2020

Show all 19 references

[9]

Li, W., Yuille, A., Zhou, Z.: How well do supervised 3d models transfer to medical imaging tasks? (2025),https://arxiv.org/abs/2501.11253

2025
[10]

In: 2023 IEEE/CVF International Conference on Computer Vi- sion (ICCV)

Liu, J., Zhang, Y., Chen, J.N., Xiao, J., Lu, Y., Landman, B.A., Yuan, Y., Yuille, A., Tang, Y., Zhou, Z.: Clip-driven universal model for organ segmentation and tumor detection. In: 2023 IEEE/CVF International Conference on Computer Vi- sion (ICCV). pp. 21095–21107. IEEE (Oct...

2023 doi
[11]

In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)

Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., Guo, B.: Swin trans- former: Hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). pp. 10012– 10022 (2021)

2021
[12]

Oquab, M., et al.: Dinov2: Learning robust visual features without supervision (2024)

2024
[13]

In: International Conference on Machine Learn- ing (ICML)

Radford, A., Kim, J.W., Hallacy, C., et al.: Learning transferable visual models from natural language supervision. In: International Conference on Machine Learn- ing (ICML). pp. 8748–8763 (2021)

2021
[14]

Annual Review of Biomedical Engineering19, 221–248 (2017)

Shen, D., Wu, G., Suk, H.I.: Deep learning in medical image analysis. Annual Review of Biomedical Engineering19, 221–248 (2017)

2017
[15]

Advances in Neural Information Processing Systems (NeurIPS) (2021)

Taleb, A., Lippert, C., Klein, T., Nabi, M.: 3d self-supervised methods for medical imaging. Advances in Neural Information Processing Systems (NeurIPS) (2021)

2021
[16]

In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Tang, Y., Yang, D., Li, W., Roth, H.R., Landman, B.A., Xu, D., Nath, V., Hatamizadeh, A.: Self-supervised pre-training of swin transformers for 3d medi- cal image analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 20730–...

2022
[17]

IEEE Transactions on Pattern Analysis and Machine Intelligence (2025) 10 Ioannis Gatopoulos, Nicolas Känzig, Sebastian Otálora, and Fei Tang

Wu, L., Zhuang, J., Chen, H.: Large-scale 3d medical image pre-training with geometric context priors. IEEE Transactions on Pattern Analysis and Machine Intelligence (2025) 10 Ioannis Gatopoulos, Nicolas Känzig, Sebastian Otálora, and Fei Tang

2025
[18]

Medical Image Analysis (2022)

Zhang, Y., et al.: Self-supervised pretraining of 3d medical image models by learn- ing region-aware representations. Medical Image Analysis (2022)

2022
[19]

Medical Image Analysis67, 101840 (2021)

Zhou, Z., Sodha, V., Pang, J., Gotway, M.B., Liang, J.: Models genesis: Generic autodidactic models for 3d medical image analysis. Medical Image Analysis67, 101840 (2021)

2021

Pith tools

Reviewed June 28, 2026 · model on record in the stance chip above.

[1] [1]

On the Opportunities and Risks of Foundation Models

Bommasani, R., Hudson, D.A., Adeli, E., et al.: On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258 (2021)

work page Pith review arXiv 2021

[2] [2]

Proceedings of the IEEE/CVF International Conference on Computer Vision (2021)

Caron, M., Touvron, H., Misra, I., et al.: Emerging properties in self-supervised vision transformers. Proceedings of the IEEE/CVF International Conference on Computer Vision (2021)

2021

[3] [3]

In: International Conference on Machine Learning (ICML) (2020)

Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for con- trastive learning of visual representations. In: International Conference on Machine Learning (ICML) (2020)

2020

[4] [4]

In: Medical Imaging with Deep Learning (2024)

Gatopoulos,I.,Känzig,N.,Moser,R.,Otálora,S.,etal.:eva:Evaluationframework for pathology foundation models. In: Medical Imaging with Deep Learning (2024)

2024

[5] [5]

Advances in Neural Information Processing Systems (NeurIPS) (2020)

Grill, J.B., Strub, F., Altché, F., et al.: Bootstrap your own latent: A new approach to self-supervised learning. Advances in Neural Information Processing Systems (NeurIPS) (2020)

2020

[6] [6]

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2022)

Hatamizadeh, A., Tang, Y., Nath, V., et al.: Swin unetr: Swin transformers for se- mantic segmentation of brain tumors in mri images. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2022)

2022

[7] [7]

In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

He, K., Fan, H., Wu, Y., Xie, S., Girshick, R.: Momentum contrast for unsupervised visual representation learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 9729–9738 (2020)

2020

[8] [8]

IEEE Transactions on Pattern Analysis and Machine Intelligence 43(11), 4037–4058 (2020)

Jing, L., Tian, Y.: Self-supervised visual feature learning with deep neural net- works: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence 43(11), 4037–4058 (2020)

2020