Pith. sign in

REVIEW 2 major objections 5 minor 21 references

Multi-view Echo alignment produces small ECG features that beat larger models at structural heart phenotyping.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-15 12:32 UTC pith:KF2T5AAN

load-bearing objection Solid multi-view ECG–Echo CLIP extension with clean kNN wins and code; private pretrain corpus is the real soft spot, not a fatal flaw. the 2 major comments →

arxiv 2603.08505 v2 pith:KF2T5AAN submitted 2026-03-09 cs.LG cs.AI

Echo2ECG: Enhancing ECG Representations with Cardiac Morphology from Multi-View Echos

Michelle Espranita Liman , \"Ozg\"un Turgut , Alexander M\"uller , Eimo Martens , Daniel Rueckert , Philip M\"uller This is my paper
classification cs.LG cs.AI
keywords ECGechocardiographymultimodal self-supervised learningcardiac morphologyLVEF classificationcontrastive learningmulti-view aggregationfeature extraction
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

ECG is cheap and ubiquitous for electrical problems, but structural measures such as ejection fraction usually need echocardiography. Prior self-supervised work tried to transfer morphology by pairing ECGs with single Echo views, which only show restricted anatomy. This paper argues that the right target is a complete multi-view Echo study: freeze a strong Echo encoder, pool its views with attention into one study embedding, and train a lightweight ECG encoder with CLIP-style contrastive loss so that global electrical signals learn the full morphological picture. The resulting ECG features, used with a simple kNN, outperform larger unimodal and multimodal baselines on LVEF classification across three datasets and on structural heart disease detection even with very little labeled data, and they also retrieve morphologically similar Echo studies from ECG queries. A model 18 times smaller than the largest baseline therefore becomes a practical feature extractor for morphology-aware ECG tasks.

Core claim

Aligning ECGs to multi-view Echo studies, rather than to single views or text, transfers comprehensive cardiac morphology into ECG representations. Those representations, extracted by a 12.5 M-parameter encoder, yield higher AUROC than stronger unimodal and multimodal baselines on LVEF and structural heart disease classification and enable accurate phenotype-aware ECG-to-Echo retrieval.

What carries the argument

Echo2ECG: frozen multi-view Echo encoder whose view embeddings are attention-pooled into a study-level embedding, then aligned with a trainable ECG encoder via bidirectional CLIP contrastive loss in a shared latent space.

Load-bearing premise

ECG and multi-view Echo studies acquired within a seven-day window, when pooled by a frozen Echo encoder, form a faithful enough morphological target that the transferred ECG features will generalize beyond the private pre-training site.

What would settle it

On a large external cohort of ECG-Echo pairs never seen during pre-training, freeze the Echo2ECG ECG encoder and measure whether a kNN on its features still beats the same baselines on LVEF and structural heart disease classification, and whether ECG-to-Echo retrieval precision remains above random for volumes and ejection fraction.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. Echo2ECG proposes a multimodal self-supervised framework that aligns 12-lead ECG embeddings with study-level multi-view echocardiography embeddings via CLIP-style contrastive learning. A frozen EchoPrime MViTv2 encodes individual Echo views; attention pooling aggregates them into a single study embedding that serves as the morphological target for a trainable OTiS-initialized ECG encoder (plus lightweight projections). The resulting ECG features are evaluated non-parametrically with kNN on LVEF classification (internal, EchoNext, UK Biobank) and SHD classification (EchoNext, including low-data regimes), and on phenotype-aware ECG-to-Echo retrieval (EDV, ESV, SV, EF). Across these tasks the method reports consistent gains over strong unimodal (OTiS, xECG, ECGFounder) and multimodal (EchoingECG, PTACL) baselines while remaining substantially smaller than the largest baseline. Ablations compare single- versus multi-view alignment and pooling strategies; code is released.

Significance. If the multi-view morphological transfer claim holds under broader scrutiny, the work supplies a practical, lightweight ECG feature extractor that improves structural phenotype prediction without requiring Echo or text at inference. The combination of multi-dataset kNN evaluation with bootstrap CIs, low-data SHD scaling curves, phenotype-aware retrieval metrics, and an explicit single- versus multi-view ablation is stronger than typical contrastive ECG papers. Public code and the decision to freeze a strong Echo encoder while training only a small ECG pathway are concrete strengths that aid reproducibility and clinical deployability. The result is therefore of clear interest to the medical multimodal-learning community, provided the private-pretraining and frozen-target assumptions can be stress-tested.

major comments (2)
  1. The central claim that multi-view Echo alignment yields generalizable morphological ECG features rests on a private ECG–Echo corpus (7-day pairing window, Table 1) and a frozen EchoPrime encoder whose own training involved clinical reports. Neither pairing quality, view-selection policy, nor residual report leakage inside EchoPrime can be audited. Table 5 shows multi-view + attention pooling helps, yet does not isolate genuine morphology transfer from better matching of the private institution’s acquisition/label distribution. The UKB second-place result (behind PTACL, which used CMR on the same cohort) already indicates that target modality and cohort match matter. Without a controlled public multi-view Echo–ECG experiment or release of pairing statistics, the “robust, powerful feature extractor” claim cannot be fully stress-tested for selection bias or domain shift.
  2. §4.2 and Figure 2 evaluate SHD with a fixed kNN (k=20) on frozen features. While this design cleanly isolates representation quality, it leaves open whether the reported low-data advantage persists under linear probes or light fine-tuning that other baselines could also receive. A short controlled comparison (same probe protocol for all models) would strengthen the claim that the multi-view pre-training itself, rather than the evaluation protocol, drives the gains.
minor comments (5)
  1. Table 1 and the surrounding text leave the exact patient counts, view-count distribution, and exclusion criteria for the private pre-training set underspecified; a short supplementary table would help readers gauge scale and selection.
  2. Equation (1) and the subsequent text do not state the temperature schedule or the relative weighting of the two directional losses; these free parameters should be listed for reproducibility.
  3. Figure 1 caption and §3.2 refer to “up to 128 views” without clarifying how missing or low-quality views are handled at training time; a one-sentence policy would remove ambiguity.
  4. The limitations paragraph (§6) correctly notes the lack of beat-/phase-level synchronization; a brief quantitative estimate of residual temporal misalignment (e.g., mean ECG–Echo acquisition lag) would make the limitation more concrete.
  5. Minor typographic inconsistencies appear in the rendered tables (garbled headers in Tables 1–3 of the arXiv PDF); these should be cleaned for the camera-ready version.

Circularity Check

0 steps flagged

No significant circularity: multi-view CLIP alignment produces ECG features whose quality is measured by independent held-out kNN classification and phenotype retrieval.

full rationale

The paper's load-bearing claims are empirical: ECG embeddings obtained by freezing an EchoPrime encoder, attention-pooling its multi-view outputs into a study embedding, and optimizing the standard bidirectional CLIP loss (Eq. 1) against an OTiS-initialized ECG encoder, then feeding the resulting ECG features into a non-parametric kNN (k=20) or cosine-ranked retrieval. Downstream labels (LVEF bins, SHD presence/absence, continuous EDV/ESV/SV/EF) are clinical/Echo-derived phenotype values external to the contrastive objective; AUROC, Prec@1 and MnR are therefore not algebraic rearrangements of the pre-training loss or of any free parameters fitted on the evaluation sets. Ablation Table 5 and the multi-view versus single-view comparison are controlled empirical checks, not definitional identities. Initialization from OTiS/EchoPrime and comparison to the authors' prior PTACL are ordinary transfer-learning and baseline practices; they do not force the reported gains by construction, nor is any uniqueness theorem or ansatz imported to declare the architecture inevitable. Private pre-training data raises reproducibility questions but does not create a circular derivation. The chain is therefore self-contained against the stated evaluation protocol.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 1 invented entities

The central empirical claim rests on standard contrastive multimodal learning plus domain choices about pairing, frozen teachers, and multi-view pooling—not on new physical entities. Free parameters are ordinary training/design knobs; the load-bearing domain assumptions are the 7-day pairing validity and that attention-pooled multi-view EchoPrime embeddings are the right morphological teacher for ECG.

free parameters (4)
  • ECG–Echo pairing time window
    Pairs are defined as same patient within 7 days; this hand-chosen window defines the positive pairs the contrastive objective sees.
  • CLIP temperature τ
    Temperature in the contrastive softmax (Eq. 1) scales similarity sharpness; standard free hyperparameter of the training objective.
  • Projection and pooling dimensions / architecture
    View embeddings projected 512→1024, study embedding and joint space 512-d, attention pooling design—chosen design knobs that shape the teacher signal.
  • Echo clip sampling (16 frames, stride 2) and training schedule
    Frame sampling, zero-padding, 50 epochs, and related optimization settings are fitted/chosen for this pipeline and affect learned ECG features.
axioms (4)
  • domain assumption CLIP-style bidirectional contrastive alignment transfers morphological information from imaging embeddings into ECG embeddings useful for phenotype prediction.
    Core training objective in §3; standard in multimodal medical SSL but not mathematically guaranteed for ECG morphology.
  • domain assumption ECG and Echo studies from the same patient within 7 days are valid positive pairs for morphological alignment.
    Stated in Experimental Setup §4.1; if morphology or clinical state changes inside the window, positives are noisy.
  • ad hoc to paper A frozen EchoPrime MViTv2 encoder plus attention pooling over up to 128 views yields a study-level embedding that represents complete cardiac morphology relevant to ECG.
    Method §3 freezes Echo encoder and introduces multi-view aggregation; ablation supports it empirically but it is a paper-specific modeling premise.
  • domain assumption kNN on frozen ECG embeddings is a fair probe of morphological content for comparing feature extractors.
    Downstream protocol §4; common SSL practice, but linear probes or fine-tuning could reorder models.
invented entities (1)
  • Echo2ECG multi-view study-level Echo embedding (attention-pooled views as ECG alignment target) no independent evidence
    purpose: Replace single-view Echo targets with a single study representation intended to capture whole-heart morphology for contrastive transfer into ECG.
    Design construct introduced to fix the stated representational mismatch; validated only via this paper’s ablations and downstream tasks, not an independently measured physical object.

pith-pipeline@v1.1.0-grok45 · 13956 in / 3121 out tokens · 33068 ms · 2026-07-15T12:32:24.861211+00:00 · methodology

0 comments
read the original abstract

Electrocardiography (ECG) is a low-cost, widely used modality for diagnosing electrical abnormalities like atrial fibrillation by capturing the heart's electrical activity. However, it cannot directly measure cardiac morphological phenotypes, such as left ventricular ejection fraction (LVEF), which typically require echocardiography (Echo). Predicting these phenotypes from ECG would enable early, accessible health screening. Existing self-supervised methods suffer from a representational mismatch by aligning ECGs to single-view Echos, which only capture local, spatially restricted anatomical snapshots. To address this, we propose Echo2ECG, a multimodal self-supervised learning framework that enriches ECG representations with the heart's morphological structure captured in multi-view Echos. We evaluate Echo2ECG as an ECG feature extractor on two clinically relevant tasks that fundamentally require morphological information: (1) classification of structural cardiac phenotypes across three datasets, and (2) retrieval of Echo studies with similar morphological characteristics using ECG queries. Our extracted ECG representations consistently outperform those of state-of-the-art unimodal and multimodal baselines across both tasks, despite being 18x smaller than the largest baseline. These results demonstrate that Echo2ECG is a robust, powerful ECG feature extractor. Our code is accessible at https://github.com/michelleespranita/Echo2ECG.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

21 extracted references · 4 canonical work pages

  1. [1]

    Circulation: Arrhythmia and Electrophysiology18(10), e013734 (2025).https://doi.org/10.1161/CIRCEP.125.013734,https://www

    Brant, L.C., Ribeiro, A.H., Eromosele, O.B., Pinto-Filho, M.M., Barreto, S.M., Duncan, B.B., Larson, M.G., Benjamin, E.J., Ribeiro, A.L., Lin, H.: Predic- tion of atrial fibrillation from the ecg in the community using deep learning: A multinational study. Circulation: Arrhythmia and Electrophysiology18(10), e013734 (2025).https://doi.org/10.1161/CIRCEP.1...

    work page doi:10.1161/circep.125.013734 2025

  2. [2]

    European Heart Jour- nal42(30), 2885–2896 (03 2021).https://doi.org/10.1093/eurheartj/ehab153, https://doi.org/10.1093/eurheartj/ehab153

    Cohen-Shelly, M., Attia, Z.I., Friedman, P.A., Ito, S., Essayagh, B.A., Ko, W.Y., Murphree, D.H., Michelena, H.I., Enriquez-Sarano, M., Carter, R.E., Johnson, P.W., Noseworthy, P.A., Lopez-Jimenez, F., Oh, J.K.: Electrocardiogram screen- ing for aortic valve stenosis using artificial intelligence. European Heart Jour- nal42(30), 2885–2896 (03 2021).https:...

    work page doi:10.1093/eurheartj/ehab153 2021

  3. [3]

    13026/3ykd-bf14,https://doi.org/10.13026/3ykd-bf14, version 1.1.0

    Elias, P., Finer, J.: EchoNext: A Dataset for Detecting Echocardiogram-Confirmed StructuralHeartDiseasefromECGs.PhysioNet(Sep2025).https://doi.org/10. 13026/3ykd-bf14,https://doi.org/10.13026/3ykd-bf14, version 1.1.0

    doi:10.13026/3ykd-bf14

  4. [4]

    JACC80(6), 613– 626 (2022).https://doi.org/10.1016/j.jacc.2022.05.029,https://www.jacc

    Elias, P., Poterucha, T.J., Rajaram, V., et al.: Deep learning electrocardiographic analysis for detection of left-sided valvular heart disease. JACC80(6), 613– 626 (2022).https://doi.org/10.1016/j.jacc.2022.05.029,https://www.jacc. org/doi/abs/10.1016/j.jacc.2022.05.029

    doi:10.1016/j.jacc.2022.05.029 2022

  5. [5]

    In: proceedings of Medical Image Computing and Computer Assisted Intervention – MICCAI 2025

    Gao, Y., Kim, S., McIntosh, C.: EchoingECG: An Electrocardiogram Cross-Modal Model for Echocardiogram Tasks . In: proceedings of Medical Image Computing and Computer Assisted Intervention – MICCAI 2025. vol. LNCS 15964. Springer Nature Switzerland (September 2025)

    2025

  6. [6]

    PhysioNet (Sep 2023).https://doi.org/10

    Gow, B., Pollard, T., Nathanson, L.A., Johnson, A., Moody, B., Fernandes, C., Greenbaum, N., Waks, J.W., Eslami, P., Carbonati, T., Chaudhari, A., Herbst, E., Moukheiber, D., Berkowitz, S., Mark, R., Horng, S.: MIMIC-IV-ECG: Diagnostic Electrocardiogram Matched Subset. PhysioNet (Sep 2023).https://doi.org/10. 13026/4nqg-sb35,https://doi.org/10.13026/4nqg-...

    doi:10.13026/4nqg-sb35 2023

  7. [7]

    PhysioNet (Oct 2024).https://doi.org/10.13026/ kpb9-mt58,https://doi.org/10.13026/kpb9-mt58, version 3.1

    Johnson, A., Bulgarelli, L., Pollard, T., Gow, B., Moody, B., Horng, S., Celi, L.A., Mark, R.: MIMIC-IV. PhysioNet (Oct 2024).https://doi.org/10.13026/ kpb9-mt58,https://doi.org/10.13026/kpb9-mt58, version 3.1

    doi:10.13026/kpb9-mt58 2024

  8. [8]

    JACC75(7), 722–733 (2020).https: //doi.org/10.1016/j.jacc.2019.12.030,https://www.jacc.org/doi/abs/10

    Ko, W.Y., Siontis, K.C., Attia, Z.I., Carter, R.E., Kapa, S., Ommen, S.R., Demuth, S.J., Ackerman, M.J., Gersh, B.J., Arruda-Olson, A.M., Geske, J.B., Asirvatham, S.J., Lopez-Jimenez, F., Nishimura, R.A., Friedman, P.A., Nose- worthy, P.A.: Detection of hypertrophic cardiomyopathy using a convolutional neural network-enabled electrocardiogram. JACC75(7), ...

    work page doi:10.1016/j.jacc.2019.12.030 2020

  9. [9]

    NEJM AI2(7), AIoa2401033 (2025)

    Li, J., Aguirre, A.D., Junior, V.M., Jin, J., Liu, C., Zhong, L., Sun, C., Clifford, G., Brandon Westover, M., Hong, S.: An electrocardiogram foundation model built on over 10 million recordings. NEJM AI2(7), AIoa2401033 (2025)

    2025

  10. [10]

    In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion(CVPR).pp.4794–4804(2022).https://doi.org/10.1109/CVPR52688.2022

    Li, Y., Wu, C.Y., Fan, H., Mangalam, K., Xiong, B., Malik, J., Feichtenhofer, C.: Mvitv2: Improved multiscale vision transformers for classification and detec- tion. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion(CVPR).pp.4794–4804(2022).https://doi.org/10.1109/CVPR52688.2022. 00476

    doi:10.1109/cvpr52688.2022 2022

  11. [11]

    Lunelli, R., Nicolson, A., Pröll, S.M., Reinstadler, S.J., Bauer, A., Dlaska, C.: Benchecg and xecg: a benchmark and baseline for ecg foundation models (2025), https://arxiv.org/abs/2509.10151

    Pith/arXiv arXiv 2025

  12. [12]

    European Heart Journal42(36), 3599–3726 (08 2021).https://doi.org/10.1093/eurheartj/ehab368,https://doi.org/10

    McDonagh, T.A., Metra, M., Adamo, M., et al.: 2021 esc guidelines for the di- agnosis and treatment of acute and chronic heart failure: Developed by the task force for the diagnosis and treatment of acute and chronic heart failure of the european society of cardiology (esc) with the special contribution of the heart failure association (hfa) of the esc. E...

    doi:10.1093/eurheartj/ehab368 2021

  13. [13]

    Nature644(8075), 221–230 (Aug 2025).https://doi.org/10.1038/s41586-025-09227-0,https://doi.org/10

    Poterucha, T.J., Jing, L., Ricart, R.P., et al.: Detecting structural heart disease from electrocardiograms using ai. Nature644(8075), 221–230 (Aug 2025).https://doi.org/10.1038/s41586-025-09227-0,https://doi.org/10. 1038/s41586-025-09227-0

    doi:10.1038/s41586-025-09227-0 2025

  14. [14]

    In: Meila, M., Zhang, T

    Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., Sutskever, I.: Learning transferable visual models from natural language supervision. In: Meila, M., Zhang, T. (eds.) Proceedings of the 38th International Conference on Machine Learning. Proceed- ings of Machine Learning Res...

    2021

  15. [15]

    Ribeiro, A.L.P., Paixão, G.M.M., Gomes, P.R., Ribeiro, M.H., Ribeiro, A.H., Canazart, J.A., Oliveira, D.M., Ferreira, M.P., Lima, E.M., Moraes, J.L.d., Castro, N., Ribeiro, L.B., Macfarlane, P.W.: Tele-electrocardiography and bigdata: The CODE (clinical outcomes in digital electrocardiography) study. J. Electrocardiol. 57S, S75–S78 (Nov 2019)

    2019

  16. [16]

    In: proceedings of Medical Image Computing and Computer Assisted Intervention – MICCAI 2025

    Selivanov, A., Müller, P., Turgut, Ö., Stolt-Ansó, N., Rueckert, D.: Global and Local Contrastive Learning for Joint Representations from Cardiac MRI and ECG . In: proceedings of Medical Image Computing and Computer Assisted Intervention – MICCAI 2025. vol. LNCS 15960. Springer Nature Switzerland (September 2025)

    2025

  17. [17]

    PLoS Med.12(3), e1001779 (Mar 2015)

    Sudlow, C., Gallacher, J., Allen, N., Beral, V., Burton, P., Danesh, J., Downey, P., Elliott, P., Green, J., Landray, M., Liu, B., Matthews, P., Ong, G., Pell, J., Silman, A., Young, A., Sprosen, T., Peakman, T., Collins, R.: UK biobank: an open access Echo2ECG 11 resource for identifying the causes of a wide range of complex diseases of middle and old ag...

    2015

  18. [18]

    Medical Image Analysis101, 103451 (2025)

    Özgün Turgut, Müller, P., Hager, P., Shit, S., Starck, S., Menten, M.J., Martens, E., Rueckert, D.: Unlocking the diagnostic potential of electrocardiograms through information transfer from cardiac mag- netic resonance imaging. Medical Image Analysis101, 103451 (2025). https://doi.org/https://doi.org/10.1016/j.media.2024.103451,https: //www.sciencedirect...

    work page doi:10.1016/j.media.2024.103451 2025

  19. [19]

    Özgün Turgut, Müller, P., Menten, M.J., Rueckert, D.: Towards generalisable time series understanding across domains (2025),https://arxiv.org/abs/2410.07299

    Pith/arXiv arXiv 2025

  20. [20]

    In: Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R

    Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L.u., Polosukhin, I.: Attention is all you need. In: Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. (eds.) Advances in Neural Information Processing Systems. vol. 30. Curran Associates, Inc. (2017),https://proceedings.neurips....

    2017

  21. [21]

    Nature (Nov 2025).https://doi.org/10.1038/ s41586-025-09850-x,https://doi.org/10.1038/s41586-025-09850-x

    Vukadinovic, M., Chiu, I.M., Tang, X., Yuan, N., Chen, T.Y., Cheng, P., Li, D., Cheng, S., He, B., Ouyang, D.: Comprehensive echocardiogram evaluation with view primed vision language ai. Nature (Nov 2025).https://doi.org/10.1038/ s41586-025-09850-x,https://doi.org/10.1038/s41586-025-09850-x

    doi:10.1038/s41586-025-09850-x 2025