REVIEW 4 major objections 5 minor 2 cited by
Sensing Cardiac Health Across Scenarios and Devices: A Multi-Modal Foundation Model Pretrained on Heterogeneous Data from 1.7 Million Individuals
T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read A single transformer pretrained on heterogeneous ECG, PPG, and text from 1.7 million people learns channel-agnostic cardiac representations that outperform bespoke one-task models across diagnosis, demographics, blood pressure, outcomes…
desk verdict A credible multi-modal cardiac foundation model whose headline 'consistently outperforms' is directly contradicted by its own reported macro-F1 numbers in the CVD diagnosis section. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is masked modeling along both time and channel dimensions. During pretraining, random segments of the input waveform—and, for multi-channel data, random channels—are obscured and the transformer must reconstruct them from the remaining context, so the model learns to represent cardiac activity without assuming a fixed channel set. This is what the paper calls being channel-agnostic, and it is the property that enables the same checkpoint to be fine-tuned on 12-lead, 6-lead, 2-lead, or single-lead inputs and on ECG or PPG without reinitializing input-specific layers. A dense regression head can be added on top of the pretrained transformer for waveform-level tasks such as blood-pressure reconstruction and cross-modality signal generation.
What would settle it
Take the released checkpoint and fine-tune it on a lead configuration that was not emphasized in pretraining—for example, a three-lead montage such as leads I, aVF, and V3—using the paper's standard pipeline. If performance collapses unless the input projection is reinitialized or the lead ordering is permuted, the channel-agnostic claim is falsified; a simpler check is whether the tokenizer or positional indexing assumes a fixed maximum channel count.
Extended reading notes
Core claim
The central discovery is that a transformer pretrained generatively on cardiac signals with channel-wise and temporal masking learns representations that are not locked to any particular lead count or sensor type. The paper reports that CSFM embeddings, used either by fine-tuning or as frozen features with a classical classifier, outperform conventional convolutional, recurrent, and autoencoder baselines trained from scratch across the five evaluated scenarios. It further reports that the model transfers from 12-lead pretraining to 6-, 2-, and 1-lead settings without reinitializing input layers, that it works with ECG alone, PPG alone, or both together, and that it can reconstruct PPG-based ECGs and single-lead-to-12-lead ECGs, generating synthetic waveforms that support transferable classification. The authors interpret this as evidence that heterogeneous, multi-modal pretraining yields a versatile cardiac sensing backbone.
Load-bearing premise
The entire transfer claim rests on the assumption that the channel-masking pretraining makes the transformer genuinely channel-agnostic, meaning any subset of leads or either modality can be fed in at inference time without reinitializing any input-specific layer; the paper asserts this property but provides no architecture-level specification or ablation that proves it.
Editorial extensions
If this is right
- A single deployed checkpoint could serve a 12-lead diagnostic ECG at a hospital and a single-lead wearable at home, because the input layer does not need to be rebuilt for each device.
- New sites with very little labelled data could fine-tune from the pretrained backbone; the paper finds fewer-lead fine-tuning with 10% of the training set reaches performance comparable to conventional models trained on 100%.
- The embeddings work as a frozen feature extractor, so lightweight deployment is possible by feeding them to a classical classifier rather than fine-tuning the whole transformer.
- Cross-modality generation means PPG-only devices could be used to synthesize ECG waveforms for atrial-fibrillation screening, and single-lead devices could synthesize 12-lead waveforms for diagnosis.
- The same pretrained representations support long-term mortality prediction and short-term ICU false-alarm detection, suggesting one model can cover both risk-stratification and acute-care monitoring.
Reading between the lines
- If the channel-agnostic property is real, then varying which channels appear during pretraining is itself a form of augmentation for lead invariance; an ablation that randomizes the visible channel set on each training step would test this directly.
- The paper's result that text-paired pretraining helps single-lead performance even when it shrinks the signal-only dataset suggests clinical text carries cross-lead knowledge; a controlled comparison with text removed from pretraining would isolate that effect.
- The same masked-channel recipe could apply to other multi-channel physiological recordings, such as EEG montages or multiparameter ICU waveforms, where device heterogeneity currently forces per-montage retraining.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces CSFM, a family of transformer-based cardiac sensing foundation models pretrained with masked modeling on heterogeneous ECG, PPG, and text data from roughly 1.7 million individuals across MIMIC-III-WDB, MIMIC-IV-ECG, and CODE. The authors claim that CSFM consistently outperforms conventional one-modal-one-task models across five downstream scenarios: cardiac disease diagnosis, demographic information recognition, vital sign measurement, clinical outcome prediction, and ECG question answering, while also transferring across ECG lead configurations (12-, 6-, 2-, 1-lead) and across ECG-only, PPG-only, and combined modalities. The paper further reports that CSFM embeddings outperform hand-crafted features and features from prior ECG/PPG foundation models, and that the model supports cross-modality waveform reconstruction and augmentation. The central claim is that a single pretrained backbone can serve multiple tasks and input configurations without task- or device-specific architectural changes.
Significance. If the central claims were fully supported, the work would be a substantial contribution: a large-scale, multi-modal, channel-agnostic foundation model for cardiac signals with broad downstream applicability would be of high practical value, especially for settings with limited lead configurations or wearable-only data. The paper's strengths include its very large pretraining corpus, the breadth of downstream evaluations against multiple baselines, and the explicit comparison with existing foundation models and hand-crafted features. However, the paper's own reported numbers in the cardiovascular disease diagnosis section directly contradict the headline claim of consistent superior performance, and the CODE-15 mortality evaluation appears to involve a subset of the pretraining data. These issues are load-bearing because they concern the central claim of the paper, and they must be resolved before the findings can be accepted.
major comments (4)
- [Results, Cardiovascular Disease Diagnosis (Wearable ECG, PPG, 12-Lead ECG)] The reported macro-F1 values for CSFM are lower than the best conventional baseline on all three diagnostic datasets: CinC17 0.634 (95% CI [0.558, 0.710]) versus 0.677 ([0.656, 0.699]), PTB-XL 0.328 ([0.296, 0.361]) versus 0.357 ([0.338, 0.377]), and SimBand 0.357 ([0.324, 0.391]) versus 0.398 ([0.279, 0.516]). The text immediately following states that 'In most cases, our CSFM model series substantially outperforms conventional learning strategies,' and the Abstract and Conclusion claim that CSFM 'consistently outperforms traditional one-modal-one-task approaches.' This is a direct self-contradiction: the confidence intervals for CinC17 and PTB-XL do not overlap, so the comparison is statistically unambiguous. The authors must either correct the reported numbers or substantially temper the claims; as written, the flagship diagnostic evidence fails the paper's central claim.
- [Results, Clinical Outcome Prediction (ECG, PPG)] The evaluation of 1-year mortality prediction on CODE-15 is potentially circular because CODE-15 is described as 'public small version of CODE-Full' and CODE-Full is one of the pretraining datasets. If CODE-15 is a subset of CODE-Full, then the pretraining data contains the very recordings (or subjects) used for downstream validation. The sentence 'we ensured that no training subjects in CODE-Full is available in validation/testing subset of CODE-15' is unclear and grammatically ambiguous; if the intended meaning is that CODE-15 subjects were excluded from pretraining, this needs a precise explanation of how that was possible given that CODE-15 is a subset of CODE-Full. Without such clarification, the reported AUC of up to 0.844 on CODE-15 cannot be interpreted as an external validation result.
- [Transfer from 12-Lead to Fewer-Lead Settings] The claim that CSFM is 'channel-agnostic, enabling direct transfer learning without the need to reinitialize input-specific layers' is load-bearing for the entire multi-lead and multi-modality generalization story, yet the paper provides no architectural specification of how the model accepts variable channel counts or arbitrary subsets of channels. The main text does not describe how leads are tokenized, how channel masking is applied at inference time, or how positional encodings handle different numbers of channels. An explicit description of the mechanism, or an ablation demonstrating that the model does not degrade simply because of input-size handling, is needed to support the transfer claims in Figure 4 and Table 2.
- [Table 2, transfer learning results] The text in the 'Transfer from 12-Lead to Fewer-Lead Settings' paragraph states that 'CSFMs consistently outperform conventional approaches,' but Table 2 shows that CSFM-Tiny is not consistently superior: for example, in the 6-lead 100% condition CSFM-Tiny achieves macro-F1 0.268, which is below ResNet1d18's 0.290, and in the 2-lead 100% condition CSFM-Tiny's 0.267 is below ResNet1d18's 0.272. The claim of consistent superiority is therefore not supported by the table's own numbers, even setting aside the diagnostic results in Figure 3a. The authors should either revise the claim or report the comparison in a way that is faithful to the full set of results.
minor comments (5)
- [Figure 1 caption] The word 'reocgnition' appears in the sentence 'demographic information reocgnition' and should be corrected to 'recognition.'
- [Results, Demographic Information Recognition] The phrase 'measured by meas absolute error' should read 'measured by mean absolute error (MAE).'
- [Results, ECG Question Answering] There is a duplicated phrase in the sentence 'This was compared with that of the Fusion Transformer model introduced in Ohet al.32, as well as with as well as with LLaVA'; one 'as well as with' should be removed.
- [Results, Clinical Outcome Prediction] The text refers to 'as shown in Figure 3d' and 'as shown in Figure 3e' with a missing closing parenthesis after the second reference, and the sentence structure should be repaired for clarity.
- [Methods and captions] The reference 'Ranftlet al.26' is missing a space, and the phrase 'with R-squared value calculated' appears in the Figure 3 caption in a stilted manner; a fuller description of the metric would be clearer.
Circularity Check
The CODE-15 mortality prediction is evaluated on data that, by the paper's own description, is a subset of the CODE-Full pretraining set, so that central 'prediction' partially reduces to pretraining exposure.
-
fitted input called prediction
[Results, 'Clinical Outcome Prediction' paragraph (Figure 3d/e); also Figure 4 and Figure 2a]
"It should be noted that CODE-15 is public small version of CODE-Full10, and in experimental settings we ensured that no training subjects in CODE-Full is available in validation/testing subset of CODE-15."
The paper states that CODE-15 is a 'public small version of CODE-Full' and that pretraining used CODE-Full (1,558,748 segments; Figure 2a). If CODE-15 is a subset of CODE-Full, then CODE-15's validation/test subjects are contained in CODE-Full by definition, making it impossible for 'no training subjects in CODE-Full' to be absent from those splits. The reported CODE-15 1-year mortality AUC (CSFM up to 0.844 vs 0.816 for conventional models) is therefore not a held-out generalization result: the model was pretrained on records that include the evaluation set. The improvement can be at least partially attributed to memorization/recognition of pretraining data rather than to transferable representations.
full rationale
The CODE-15 mortality experiment is the clearest circular step: the paper's own text identifies CODE-15 as a small version of CODE-Full, and CODE-Full is a pretraining corpus. Evaluating on a subset of the pretraining data without a demonstrated exclusion makes that 'prediction' partially an evaluation of training-data exposure. This affects one of the central outcome-prediction claims. Most other evaluations are on external public datasets not used in pretraining (PTB-XL, CinC17, SimBand, VitalDB, VTaC, ECG-QA), so the overall contribution still has independent content. The internal inconsistency in the CVD diagnosis paragraph, where the reported CSFM macro-F1 values are below the best baselines on CinC17, PTB-XL, and SimBand despite the text claiming substantial outperformance, is a correctness and reporting issue rather than a circularity issue; it does not reduce the derivation to its inputs. No other load-bearing self-citation chain or definitional equivalence was found: the transformer architecture, masked pretraining, and downstream fine-tuning are not circularly defined in terms of the claimed results.
Assumptions & free parameters
free parameters (2)
- Channel and temporal masking ratios during pretraining
- Model width/depth for CSFM-Tiny/Base/Large
assumptions (3)
- domain assumption Disjointness of CODE-15 test subjects from CODE-Full pretraining subjects
- domain assumption Correct pairing of ECG/PPG signals with text reports in MIMIC-III and MIMIC-IV
- ad hoc to paper Channel-agnostic representation from masked channel training
Cite this review
Pith. "Pith review of Sensing Cardiac Health Across Scenarios and Devices: A Multi-Modal Foundation Model Pretrained on Heterogeneous Data from 1.7 Million Individuals." pith.science (2026). https://pith.science/paper/YSRLPC7V
@misc{pith2026250701045,
author = {Pith},
title = {Pith review of: Sensing Cardiac Health Across Scenarios and Devices: A Multi-Modal Foundation Model Pretrained on Heterogeneous Data from 1.7 Million Individuals},
year = {2026},
howpublished = {\url{https://pith.science/paper/YSRLPC7V}},
note = {Machine review of arXiv:2507.01045}
}
read the original abstract
Cardiac biosignals, such as electrocardiograms (ECG) and photoplethysmograms (PPG), are of paramount importance for the diagnosis, prevention, and management of cardiovascular diseases, and have been extensively used in a variety of clinical tasks. Conventional deep learning approaches for analyzing these signals typically rely on homogeneous datasets and static bespoke models, limiting their robustness and generalizability across diverse clinical settings and acquisition protocols. In this study, we present a cardiac sensing foundation model (CSFM) that leverages advanced transformer architectures and a generative, masked pretraining strategy to learn unified representations from vast, heterogeneous health records. Our model is pretrained on an innovative multi-modal integration of data from multiple large-scale datasets (including MIMIC-III-WDB, MIMIC-IV-ECG, and CODE), comprising cardiac signals and the corresponding clinical or machine-generated text reports from approximately 1.7 million individuals. We demonstrate that the embeddings derived from our CSFM not only serve as effective feature extractors across diverse cardiac sensing scenarios, but also enable seamless transfer learning across varying input configurations and sensor modalities. Extensive evaluations across diagnostic tasks, demographic information recognition, vital sign measurement, clinical outcome prediction, and ECG question answering reveal that CSFM consistently outperforms traditional one-modal-one-task approaches. Notably, CSFM exhibits robust performance across multiple ECG lead configurations from standard 12-lead systems to single-lead setups, and in scenarios where only ECG, only PPG, or a combination thereof is available. These findings highlight the potential of CSFM as a versatile and scalable solution, for comprehensive cardiac monitoring.
Figures
Figures from the paper (5 more)
Forward citations
Cited by 2 Pith papers
-
Pretraining EHR Foundation Models with Patient-Aware Sampling
Patient-aware sampling of pretraining windows, with patients weighted by a tunable exponent, improves downstream AUROC/AUPRC over a global token-stream baseline in autoregressive EHR models.
-
Physical activities enable scalable foundation modelling for broad-spectrum health prediction
A 3.4M-parameter foundation model pre-trained on step-count data alone achieves best AUROC on 20 of 21 health risk prediction tasks across multiple devices, regions, and diseases.
Reference graph
Works this paper leans on
-
[1]
Kaptoge, S.et al.World health organization cardiovascular disease risk charts: revised models to estimate risk in 21 global regions.The Lancet global health7, e1332–e1345 (2019)
work page 2019
-
[2]
Bayoumy, K.et al.Smart wearable devices in cardiovascular care: where we are and how to move forward.Nat. Rev. Cardiol.18, 581–599 (2021)
work page 2021
- [3]
-
[4]
Steinhubl, S. R.et al.Effect of a home-based wearable continuous ecg monitoring patch on detection of undiagnosed atrial fibrillation: the mstops randomized clinical trial.Jama320, 146–155 (2018). 5.Gu, X.et al.Beyond supervised learning for pervasive healthcare.IEEE Rev. Biomed. Eng.(2023)
work page 2018
-
[6]
Clifford, G. D.et al.Af classification from a short single lead ecg recording: The physionet/computing in cardiology challenge 2017. In2017 Computing in Cardiology (CinC), 1–4 (IEEE, 2017). 7.Wagner, P.et al.Ptb-xl, a large publicly available electrocardiography dataset.Sci. data7, 1–15 (2020)
work page 2020
-
[8]
Shashikumar, S. P., Shah, A. J., Li, Q., Clifford, G. D. & Nemati, S. A deep learning approach to monitoring and detecting atrial fibrillation using wearable technology. In2017 IEEE EMBS international conference on biomedical & health informatics (BHI), 141–144 (Ieee, 2017)
work page 2017
-
[9]
Lehman, L.-w.et al.Vtac: a benchmark dataset of ventricular tachycardia alarms from icu monitors.Adv. Neural Inf. Process. Syst.36(2024)
work page 2024
-
[10]
M.et al.Deep neural network-estimated electrocardiographic age as a mortality predictor.Nat
Lima, E. M.et al.Deep neural network-estimated electrocardiographic age as a mortality predictor.Nat. communications 12, 5117 (2021). 11.Ribeiro, A. H.et al.Automatic diagnosis of the 12-lead ecg using a deep neural network.Nat. communications11, 1760 (2020)
work page 2021
Show all 25 references
-
[12]
communications13, 1583 (2022)
Sangha, V.et al.Automated multilabel diagnosis on electrocardiographic images and signals.Nat. communications13, 1583 (2022)
2022
-
[13]
Y.et al.Cardiologist-level arrhythmia detection and classification in ambulatory electrocardiograms using a deep neural network.Nat
Hannun, A. Y.et al.Cardiologist-level arrhythmia detection and classification in ambulatory electrocardiograms using a deep neural network.Nat. medicine25, 65–69 (2019). 14.Bommasani, R.et al.On the opportunities and risks of foundation models.arXiv preprint arXiv:2108.07258(2...
2019 arXiv
-
[17]
InProceedings of the IEEE/CVF International Conference on Computer Vision, 4015–4026 (2023)
Kirillov, A.et al.Segment anything. InProceedings of the IEEE/CVF International Conference on Computer Vision, 4015–4026 (2023). 18.Dosovitskiy, A.et al.An image is worth 16x16 words: Transformers for image recognition at scale.ICLR(2021)
2023
-
[19]
Medicine6, 108 (2023)
Vaid, A.et al.A foundational vision transformer improves diagnostic performance for electrocardiograms.NPJ Digit. Medicine6, 108 (2023). 20.Vaswani, A. Attention is all you need.Adv. Neural Inf. Process. Syst.(2017)
2023
-
[21]
& Clifton, D
Xu, P., Zhu, X. & Clifton, D. A. Multimodal learning with transformers: A survey.IEEE Transactions on Pattern Analysis Mach. Intell.45, 12113–12132 (2023). 22.Johnson, A. E.et al.Mimic-iii, a freely accessible critical care database.Sci. data3, 1–9 (2016). 23.Gow, B.et al.Mimi...
2023
-
[24]
Lu, L.et al.Decoding 2.3 million ecgs: interpretable deep learning for advancing cardiovascular diagnosis and mortality risk stratification.Eur. Hear. Journal-Digital Heal.5, 247–259 (2024)
2024
-
[25]
Data9, 279 (2022)
Lee, H.-C.et al.Vitaldb, a high-fidelity multi-parameter vital signs database in surgical patients.Sci. Data9, 279 (2022)
2022
-
[26]
& Koltun, V
Ranftl, R., Bochkovskiy, A. & Koltun, V. Vision transformers for dense prediction. InProceedings of the IEEE/CVF international conference on computer vision, 12179–12188 (2021)
2021
-
[27]
Ismail Fawaz, H.et al.Inceptiontime: Finding alexnet for time series classification.Data Min. Knowl. Discov.34, 1936–1962 (2020). 14/15
2020
-
[28]
Commun.14, 3741 (2023)
Lai, J.et al.Practical intelligent diagnostic algorithm for wearable 12-lead ecg via self-supervised learning on large-scale dataset.Nat. Commun.14, 3741 (2023)
2023
-
[29]
& Yang, G.-Z
Gu, X., Guo, Y., Deligianni, F., Lo, B. & Yang, G.-Z. Cross-subject and cross-modal transfer for generalized abnormal gait pattern recognition.IEEE Transactions on Neural Networks Learn. Syst.32, 546–560 (2020)
2020
-
[30]
Wang, W., Mohseni, P., Kilgore, K. L. & Najafizadeh, L. Pulsedb: A large, cleaned dataset based on mimic-iii and vitaldb for benchmarking cuff-less blood pressure estimation methods.Front. Digit. Heal.4, 1090854 (2023)
2023
-
[31]
D.et al.The physionet/computing in cardiology challenge 2015: reducing false arrhythmia alarms in the icu
Clifford, G. D.et al.The physionet/computing in cardiology challenge 2015: reducing false arrhythmia alarms in the icu. In2015 Computing in Cardiology Conference (CinC), 273–276 (IEEE, 2015)
2015
-
[32]
& Choi, E
Oh, J., Lee, G., Bae, S., Kwon, J.-m. & Choi, E. Ecg-qa: A comprehensive question answering dataset combined with electrocardiogram. In Oh, A.et al.(eds.)Advances in Neural Information Processing Systems, vol. 36, 66277–66288 (Curran Associates, Inc., 2023)
2023
-
[33]
A.et al.Issues in the automated classification of multilead ecgs using heterogeneous labels and populations
Reyna, M. A.et al.Issues in the automated classification of multilead ecgs using heterogeneous labels and populations. Physiol. measurement43, 084001 (2022)
2022
-
[34]
& Clifton, D
Kiyasseh, D., Zhu, T. & Clifton, D. A clinical deep learning framework for continually learning from cardiac signals across diseases, time, modalities, and institutions.Nat. Commun.12, 4221 (2021). 35.McKeen, K.et al.Ecg-fm: An open electrocardiogram foundation model.arXiv pre...
2021 arXiv
-
[36]
& Malekzadeh, M
Pillai, A., Spathis, D., Kawsar, F. & Malekzadeh, M. Papagei: Open foundation models for optical physiological signals. arXiv preprint arXiv:2410.20542(2024). 15/15
2024 arXiv
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.