Self-Supervised Contrastive Learning for Cardiac MR Sequence Classification

Cheng Ting Lin; Dongshen Peng; Haoyue Guan; Harrison Bai; Hyewon Jung; Ihab Kamel; Jing Wu; Joao Lima; Yoko Kato; Yuli Wang

REVIEW 3 major objections 1 minor 25 references

Self-Supervised Contrastive Learning for Cardiac MR Sequence Classification

T0 review · 3 major / 1 minor · reviewed 2026-06-30 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Self-supervised contrastive learning adapts Vision Transformers to classify cardiac MR sequences with AUC above 0.75 and generalizes to external datasets.

desk verdict Standard contrastive ViT adaptation on cardiac MR with a generalization claim that tests on brain datasets instead. read the letter →

arxiv 2605.24789 v1 pith:6FPKOGQO submitted 2026-05-24 cs.CV eess.IV

Yuli Wang , Hyewon Jung , Dongshen Peng , Yuwei Dai , Jing Wu , Haoyue Guan , Yoko Kato , Zhicheng Jiao

show 5 more authors

Yu Sun Ihab Kamel Joao Lima Cheng Ting Lin Harrison Bai

This is my paper

classification cs.CVeess.IV

keywords cardiacMRVisionTransformerself-supervisedlearningcontrastivesequenceclassificationdomainadaptationmedicalimaging

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper shows that standard pretrained Vision Transformer models fail to transfer effectively to cardiac magnetic resonance images because they lack domain-specific knowledge. It proposes an adaptation method that applies image-based self-supervised contrastive learning on an in-house cardiac MR dataset, which outperforms conventional supervised training. The resulting model achieves classification AUC exceeding 0.75 on the four most common cardiac MR sequences and demonstrates strong generalization when evaluated on external MR datasets such as BraTS and ADNI. Ablation experiments examine how batch size and dataset scale affect results. This establishes a practical route for domain adaptation in medical imaging without relying solely on labeled data.

What carries the argument

Image-based self-supervised contrastive learning used to adapt pretrained Vision Transformer (ViT) models to an in-house cardiac MR dataset for sequence classification.

What would settle it

Performance on a previously unseen external cardiac MR dataset dropping below AUC 0.75 or equaling that of a supervised baseline would indicate the adaptation did not achieve genuine generalization.

Watch

Extended reading notes

Core claim

Pretrained ViT features do not transfer well to the cardiac MR domain, but an adaptation strategy based on image-based self-supervised contrastive learning produces better performance than supervised training and enables the model to generalize to other MR datasets including BraTS and ADNI, reaching AUC over 0.75 on the four most common cardiac MR sequences.

Load-bearing premise

The in-house cardiac MR dataset together with the chosen contrastive learning setup produces genuine domain adaptation instead of fitting that only works on the tested sets.

Editorial extensions

If this is right

The adapted model classifies the four most common cardiac MR sequences with AUC exceeding 0.75.
The same model generalizes to external MR datasets such as BraTS and ADNI.
Increasing batch size and dataset scale during contrastive pretraining improves downstream classification performance according to the ablation results.
Self-supervised contrastive adaptation outperforms traditional supervised training on this medical imaging task.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The same contrastive adaptation procedure could be tested on other medical imaging modalities such as CT or ultrasound to check whether it reduces reliance on large labeled medical datasets.
Combining the adapted ViT features with 3D volumetric inputs rather than 2D slices might further improve sequence classification accuracy.
The observed sensitivity to batch size suggests that hardware constraints could limit how well this approach scales to very large unlabeled medical archives.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

read the letter

The paper takes a pretrained ViT, runs image-level contrastive self-supervision on an in-house cardiac MR collection, and reports better sequence classification than plain supervised training, with AUC above 0.75 on the four main cardiac sequences. They also run ablations on batch size and dataset size. That is the core of it.

The method itself is not new; contrastive adaptation of ViTs is established, so this is a domain-transfer experiment rather than a new framework. The ablations are straightforward and give some practical guidance on scaling.

The main weakness is the generalization section. The abstract states strong transfer to BraTS and ADNI, but those are brain MR datasets with T1, T2, FLAIR sequences whose labels and anatomy have little overlap with cardiac cine, tagging, or perfusion. If the external test is not actually measuring cardiac sequence classification, the result does not directly support the cardiac-domain claim and could just reflect generic MR feature quality. The abstract also gives no baselines, statistical tests, or error bars, so the performance numbers cannot be judged from the text alone.

This is for people already working on ViT adaptation inside cardiac MR or similar narrow medical imaging tasks. A reader in that subfield might extract the batch-size and scale findings, but the paper does not reorganize broader practice.

I would send it to peer review so the external validation and full tables can be checked, but the current generalization argument needs tightening.

Referee Report

3 major / 1 minor

Summary. The manuscript proposes adapting Vision Transformers (ViT) for cardiac MR sequence classification via image-based self-supervised contrastive learning on an in-house dataset. It claims that pretrained ViT features do not transfer well to the cardiac domain, that the contrastive adaptation outperforms traditional supervised training, that the adapted model achieves classification AUC exceeding 0.75 on the four most common cardiac MR sequences, and that it exhibits strong generalization to external MR datasets such as BraTS and ADNI. Ablation studies examine effects of batch size and dataset scale.

Significance. If the quantitative claims hold with proper baselines and the external evaluation directly supports cardiac-domain adaptation, the work would offer a practical approach to domain-specific ViT adaptation in medical imaging where general pretraining fails. The contrastive learning strategy and ablation results on scale could inform similar efforts in other specialized imaging modalities.

major comments (3)

[Abstract] Abstract: the claim that the adapted ViT 'exhibits strong generalization to external MR datasets such as BraTS and ADNI' is load-bearing for the central contribution yet does not specify whether the external evaluation uses cardiac sequence labels, remaps to brain sequence labels (T1/T2/FLAIR), or measures a different downstream task; BraTS and ADNI contain brain rather than cardiac anatomy, so this does not directly corroborate improved cardiac MR sequence classification.
[Abstract] Abstract: the assertion of 'superior performance compared to traditional supervised training approaches' and 'classification AUC exceeding 0.75' supplies no quantitative baselines, statistical tests, ablation details, error bars, or dataset splits, preventing evaluation of whether the contrastive adaptation produces genuine gains rather than dataset-specific fitting.
[Abstract] Abstract: the weakest assumption—that the in-house cardiac MR dataset and chosen contrastive setup suffice for domain adaptation that generalizes beyond tested external sets—is not supported by any reported cross-validation or negative controls that would rule out generic MR feature quality rather than cardiac-specific adaptation.

minor comments (1)

[Abstract] Abstract: the phrase 'four most common cardiac MR sequences' is not enumerated, making it impossible to assess coverage of the claimed task.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive feedback. We address each major comment below and agree that the abstract requires revision for clarity and completeness. Details supporting the claims are present in the main text, but we will update the abstract accordingly.

read point-by-point responses

Referee: [Abstract] Abstract: the claim that the adapted ViT 'exhibits strong generalization to external MR datasets such as BraTS and ADNI' is load-bearing for the central contribution yet does not specify whether the external evaluation uses cardiac sequence labels, remaps to brain sequence labels (T1/T2/FLAIR), or measures a different downstream task; BraTS and ADNI contain brain rather than cardiac anatomy, so this does not directly corroborate improved cardiac MR sequence classification.

Authors: We agree the abstract is imprecise. The evaluation applies the cardiac-adapted model to BraTS and ADNI for classification of their native brain MR sequences (T1, T2, FLAIR) using the datasets' own labels. This tests transfer of the learned features to a different anatomy and sequence classification task, rather than cardiac labels on external cardiac data. We will revise the abstract to explicitly state the evaluation protocol, the use of brain sequence labels, and its role as supporting evidence for feature generalization rather than direct corroboration of cardiac performance. revision: yes
Referee: [Abstract] Abstract: the assertion of 'superior performance compared to traditional supervised training approaches' and 'classification AUC exceeding 0.75' supplies no quantitative baselines, statistical tests, ablation details, error bars, or dataset splits, preventing evaluation of whether the contrastive adaptation produces genuine gains rather than dataset-specific fitting.

Authors: The abstract is a high-level summary. The full manuscript reports AUC values with error bars from repeated runs, statistical comparisons (e.g., paired tests) against supervised ViT baselines, ablation results on batch size and dataset scale, and explicit train/validation/test splits. We will revise the abstract to include key quantitative results (specific AUCs and performance deltas) and reference the supporting analyses and controls in the main text. revision: yes
Referee: [Abstract] Abstract: the weakest assumption—that the in-house cardiac MR dataset and chosen contrastive setup suffice for domain adaptation that generalizes beyond tested external sets—is not supported by any reported cross-validation or negative controls that would rule out generic MR feature quality rather than cardiac-specific adaptation.

Authors: We acknowledge the abstract does not highlight these elements. The manuscript contains ablation studies varying dataset scale and batch size that partially address overfitting concerns. However, we agree that explicit cross-validation results and negative controls (e.g., comparison against models pretrained on non-cardiac MR) would strengthen the cardiac-specific claim. We will add a dedicated discussion section on this point and include any additional cross-validation metrics in the revision. revision: partial

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; empirical results with no derivation chain

full rationale

The paper reports an empirical adaptation of ViT via self-supervised contrastive learning on an in-house cardiac MR dataset, with performance measured by AUC on cardiac sequences and external sets (BraTS, ADNI). No equations, derivations, fitted parameters renamed as predictions, or self-citation chains for uniqueness theorems appear in the abstract or described claims. Results are presented as experimental outcomes rather than reductions to inputs by construction, making the work self-contained against external benchmarks with no load-bearing circular steps.

Assumptions & free parameters 0 free parameters · 1 assumptions · 0 invented entities

Abstract-only review supplies no derivations, fitted constants, or new entities. The sole visible assumption is that general pretraining fails on cardiac MR and that contrastive adaptation fixes it.

assumptions (1)

domain assumption Pretrained ViT features do not effectively transfer to the cardiac MR domain
Stated directly in the abstract as motivation for the adaptation strategy.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Self-Supervised Contrastive Learning for Cardiac MR Sequence Classification." pith.science (2026). https://pith.science/paper/6FPKOGQO

@misc{pith2026260524789,
  author       = {Pith},
  title        = {Pith review of: Self-Supervised Contrastive Learning for Cardiac MR Sequence Classification},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6FPKOGQO}},
  note         = {Machine review of arXiv:2605.24789}
}

read the original abstract

Vision Transformer (ViT) models, utilizing self-attention mechanisms, have demonstrated robust generalization capabilities across various vision tasks, including image classification. However, these models, typically pretrained on general public datasets, often lack the specialized domain knowledge necessary for medical imaging applications. In this study, we investigate the adaptation of ViT models, specifically for cardiac magnetic resonance (MR) images, using an in-house dataset. We found that pretrained ViT features do not effectively transfer to the cardiac MR domain. To overcome this limitation, we introduce an adaptation strategy that utilizes image-based self-supervised contrastive learning, demonstrating superior performance compared to traditional supervised training approaches. Moreover, our adapted ViT model exhibits strong generalization to external MR datasets such as BraTS and ADNI. Through ablation studies, we further investigate the impact of batch size and dataset scale on performance. Ultimately, our adapted model achieves classification AUC exceeding 0.75 across the four most common cardiac MR sequences.

Figures

Figures reproduced from arXiv: 2605.24789 by the authors.

**Figure 2.** Example of cardiac MR images acquired from different MR sequences. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** Examples of latent space visualizations from SimSiam during pretraining: [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Results for batch size ablation studies and dataset size effects. (a) Batch [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

25 extracted references · 25 canonical work pages

[1]

Remote Sensing15(7), 1860 (2023)

Aleissaee, A.A., Kumar, A., Anwer, R.M., Khan, S., Cholakkal, H., Xia, G.S., Khan, F.S.: Transformers in remote sensing: A survey. Remote Sensing15(7), 1860 (2023)

work page 2023
[2]

Bordes, F., Garrido, Q., Kao, J., Williams, A., Rabbat, M., and Dupoux, E

Beyer, L., Zhai, X., Kolesnikov, A.: Better plain vit baselines for imagenet-1k. arXiv preprint arXiv:2205.01580 (2022)

work page arXiv 2022
[3]

In: Proceedings of the IEEE/CVF interna- tional conference on computer vision

Chen, C.F.R., Fan, Q., Panda, R.: Crossvit: Cross-attention multi-scale vision transformer for image classification. In: Proceedings of the IEEE/CVF interna- tional conference on computer vision. pp. 357–366 (2021)

work page 2021
[4]

In: International conference on machine learning

Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for con- trastive learning of visual representations. In: International conference on machine learning. pp. 1597–1607. PmLR (2020)

work page 2020
[5]

Chen,X.,He,K.:Exploringsimplesiameserepresentationlearning.In:Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 15750–15758 (2021)

work page 2021
[6]

Advances in Neural Information Processing Systems36, 2252–2274 (2023)

Dehghani, M., Mustafa, B., Djolonga, J., Heek, J., Minderer, M., Caron, M., Steiner, A., Puigcerver, J., Geirhos, R., Alabdulmohsin, I.M., et al.: Patch n’pack: Navit, a vision transformer for any aspect ratio and resolution. Advances in Neural Information Processing Systems36, 2252–2274 (2023)

work page 2023
[7]

In: 2009 IEEE conference on computer vision and pattern recognition

Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: A large- scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition. pp. 248–255. Ieee (2009)

work page 2009
[8]

An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al.: An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)

work page Pith review arXiv 2010

Show all 25 references

[9]

arXiv preprint arXiv:2601.12174 (2026)

Guo, H., Li, C.Y., Wang, Y., Wang, R., Dai, Y., Peng, Q., Cao, D., Zhong, Z., Vu, T., Zhao, L., et al.: A multitask framework for automated interpretation of multi-frame right upper quadrant ultrasound in clinical decision support. arXiv preprint arXiv:2601.12174 (2026)

2026
[10]

Neuroimage48(1), 138–149 (2009)

Hinrichs, C., Singh, V., Mukherjee, L., Xu, G., Chung, M.K., Johnson, S.C., Ini- tiative, A.D.N., et al.: Spatially augmented lpboosting for ad classification with evaluations on the adni dataset. Neuroimage48(1), 138–149 (2009)

2009
[11]

Radiology316(2), e243412 (2025)

Hsu, W.C., Wang, Y., Wu, Y.F., Chen, R., Afyouni, S., Liu, J., Vin, S., Shi, V., Imami, M., Chotiyanonta, J.S., et al.: Mri-based ovarian lesion classification via a foundation segmentation model and multimodal analysis: a multicenter study. Radiology316(2), e243412 (2025)

2025
[12]

In: Proceedings of the IEEE/CVF international conference on computer vision

Kamran, S.A., Hossain, K.F., Tavakkoli, A., Zuckerbrod, S.L., Baker, S.A.: Vtgan: Semi-supervised retinal image synthesis and disease prediction using vision trans- formers. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 3235–3245 (2021)

2021
[13]

IEEE transactions on medical imaging 34(10), 1993–2024 (2014)

Menze, B.H., Jakab, A., Bauer, S., Kalpathy-Cramer, J., Farahani, K., Kirby, J., Burren, Y., Porz, N., Slotboom, J., Wiest, R., et al.: The multimodal brain tumor image segmentation benchmark (brats). IEEE transactions on medical imaging 34(10), 1993–2024 (2014)

1993
[14]

arXiv preprint arXiv:1807.03748 (2018)

Oord, A.v.d., Li, Y., Vinyals, O.: Representation learning with contrastive predic- tive coding. arXiv preprint arXiv:1807.03748 (2018)

2018 arXiv
[15]

In: Proceedings of the IEEE/CVF international conference on computer vision

Strudel, R., Garcia, R., Laptev, I., Schmid, C.: Segmenter: Transformer for seman- tic segmentation. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 7262–7272 (2021) Self-Supervised Contrastive Learning for Cardiac MR Sequence Classification 11

2021
[16]

Knowledge-Based Systems232, 107471 (2021)

Su, J., Liu, Z., Zhang, J., Sheng, V.S., Song, Y., Zhu, Y., Liu, Y.: Dv-net: Accurate liver vessel segmentation via dense connection model with d-bce loss function. Knowledge-Based Systems232, 107471 (2021)

2021
[17]

arXiv preprint arXiv:2507.06937 (2025)

Wang, Y., Shi, V.R., Zhou, L., Chin, R., Dai, Y., Hu, Y., Li, C.Y., Guan, H., Cheng, J., Sun, Y., et al.: Dataset and benchmark for enhancing critical retained foreign object detection. arXiv preprint arXiv:2507.06937 (2025)

2025
[18]

Patterns6(3) (2025)

Wang,Z.,Santa-Maria,C.A.,Popel,A.S.,Sulam,J.:Bi-levelgraphlearningunveils prognosis-relevant tumor microenvironment patterns in breast multiplexed digital pathology. Patterns6(3) (2025)

2025
[19]

IEEE transactions on medical imaging41(12), 3952–3968 (2022)

Wang, Z., Saoud, C., Wangsiricharoen, S., James, A.W., Popel, A.S., Sulam, J.: Label cleaning multiple instance learning: Refining coarse annotations on sin- gle whole-slide images. IEEE transactions on medical imaging41(12), 3952–3968 (2022)

2022
[20]

arXiv preprint arXiv:2601.12233 (2026)

Wang, Z., Zhou, Z., Wen, Z., Kook, J.H., Wojcik, J.B., Kang, J.: Diffu- sionqc: Artifact detection in histopathology via diffusion model. arXiv preprint arXiv:2601.12233 (2026)

2026
[21]

npj Artificial Intelligence1(1), 17 (2025)

Wu, J., Wang, Y., Zhong, Z., Liao, W., Trayanova, N., Jiao, Z., Bai, H.X.: Vision- language foundation model for 3d medical imaging. npj Artificial Intelligence1(1), 17 (2025)

2025
[22]

In: Proceedings of the IEEE conference on computer vision and pattern recognition

Wu, Z., Xiong, Y., Yu, S.X., Lin, D.: Unsupervised feature learning via non- parametric instance discrimination. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 3733–3742 (2018)

2018
[23]

Radiotherapy and Oncology214, 111265 (2026)

Zhao, L., Imami, M.R., Wang, Y., Mao, Y., Hsu, W.C., Chen, R., Mena, E., Li, Y., Tang, J., Wu, J., et al.: Artificial intelligence-based lesion characterization and outcome prediction of prostate cancer on [18f] dcfpyl psma imaging. Radiotherapy and Oncology214, 111265 (2026)

2026
[24]

In: MICCAI Work- shop on Data Engineering in Medical Imaging

Zhou, L., Wang, Y., Shi, V., Feng, J., Zhao, L.m., Atagu, N., Madhu, P., Mehta, T., Trivedi, P., Lin, C.T., et al.: A dataset and benchmark for enhancing retained foreign object detection through physics-based image synthesis. In: MICCAI Work- shop on Data Engineering in Medic...

2025
[25]

PloS one15(12), e0244378 (2020)

Zhou, P., Liu, Z., Wu, H., Wang, Y., Lei, Y., Abbaszadeh, S.: Automatically detect- ing bregma and lambda points in rodent skull anatomy images. PloS one15(12), e0244378 (2020)

2020

Pith tools

Reviewed June 30, 2026 · model on record in the stance chip above.

[1] [1]

Remote Sensing15(7), 1860 (2023)

Aleissaee, A.A., Kumar, A., Anwer, R.M., Khan, S., Cholakkal, H., Xia, G.S., Khan, F.S.: Transformers in remote sensing: A survey. Remote Sensing15(7), 1860 (2023)

work page 2023

[2] [2]

Bordes, F., Garrido, Q., Kao, J., Williams, A., Rabbat, M., and Dupoux, E

Beyer, L., Zhai, X., Kolesnikov, A.: Better plain vit baselines for imagenet-1k. arXiv preprint arXiv:2205.01580 (2022)

work page arXiv 2022

[3] [3]

In: Proceedings of the IEEE/CVF interna- tional conference on computer vision

Chen, C.F.R., Fan, Q., Panda, R.: Crossvit: Cross-attention multi-scale vision transformer for image classification. In: Proceedings of the IEEE/CVF interna- tional conference on computer vision. pp. 357–366 (2021)

work page 2021

[4] [4]

In: International conference on machine learning

Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for con- trastive learning of visual representations. In: International conference on machine learning. pp. 1597–1607. PmLR (2020)

work page 2020

[5] [5]

Chen,X.,He,K.:Exploringsimplesiameserepresentationlearning.In:Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 15750–15758 (2021)

work page 2021

[6] [6]

Advances in Neural Information Processing Systems36, 2252–2274 (2023)

Dehghani, M., Mustafa, B., Djolonga, J., Heek, J., Minderer, M., Caron, M., Steiner, A., Puigcerver, J., Geirhos, R., Alabdulmohsin, I.M., et al.: Patch n’pack: Navit, a vision transformer for any aspect ratio and resolution. Advances in Neural Information Processing Systems36, 2252–2274 (2023)

work page 2023

[7] [7]

In: 2009 IEEE conference on computer vision and pattern recognition

Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: A large- scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition. pp. 248–255. Ieee (2009)

work page 2009

[8] [8]

An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al.: An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)

work page Pith review arXiv 2010