{"id":"8bb38fd9-7007-4d07-9bbc-c52dd24d9ffe","arxiv_id":"2411.17711","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"AnyECG, a two-stage self-supervised ECG foundation model with a vector-quantized rhythm codebook and sparse attention, reports state-of-the-art numbers on four cardiac tasks, but its evaluation is weakened by in-domain testing and missing data.","lead":"AnyECG is a self-supervised deep learning model that reads electrocardiogram (ECG) signals from many device types and turns them into reusable rhythm codes for heart diagnostics. A generalist would read it to judge whether a single pretrained AI model can truly replace task-specific ECG classifiers across wearables, Holter monitors, and hospital machines.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim is not testable as reported: Section IV-A says all seven datasets used in pre-training were 'mixed together' for downstream tasks, and Table I lists the 'Undisclosed Database' test set as one of those seven pre-training sources, so reported gains may reflect data leakage rather…","rationale":"The reader's stated weakest_assumption concerns resampling to 300 Hz and fixed 1-second patches, which is a real but secondary risk: even if that preprocessing is diagnostically lossy, it would weaken only the 'any real-world ECG' breadth claim, not the benchmark comparisons themselves. The reader's rationale, by contrast, already identifies the more serious issue of pre-training and downstream data overlap, and I agree that this is the load-bearing concern. The paper never specifies a subject-level or record-level split between the pre-training corpus and the downstream evaluation sets, and the text in Section IV-A explicitly says all datasets were mixed for downstream tasks while Table I lists the same seven datasets, including the purported 'Undisclosed Database' test set, as pre-training sources. That makes the core SOTA claim unfalsifiable from the manuscript alone, independent of whether the architecture is sound. I am not objecting to the architectural ideas, and the ablation studies provide some internal support for the two-stage pretraining design. But a foundation-model claim requires evidence of transfer to unseen data, and the submitted evaluation cannot provide that evidence without a contamination audit. The identical numbers in Tables III and IV are additional evidence that the tables require verification, though that is secondary to the leakage concern. Because the existing REJECT verdict already follows from this issue, my read leaves the verdict unchanged.","tokens_in":18582,"tokens_out":3079,"duration_ms":31452,"concrete_test":"Obtain from the authors the exact record/patient IDs used for pre-training and for downstream train/test splits for each of the seven sources, then recompute Tables II through V with all pre-training records removed from the test sets. For the 'Undisclosed Database,' replace it with a public held-out ECG dataset not used in pre-training (e.g., Chapman-Shaoxing or CODE-15) and repeat the four tasks. If performance on pre-training-excluded data falls to baseline levels, the central claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's SOTA claim (abstract; Section IV-D) depends on evaluating AnyECG on data that was not seen during pre-training. However, Section IV-A states that the evaluation used 'a comprehensive set of ECG datasets that include all available unlabeled data during pre-training' and that 'for various downstream tasks, we mixed all datasets together.' Table I lists the same seven databases, including the 10,000-record 'Undisclosed Database' explicitly described as a test set, as the pre-training corpus. No subject-level or record-level split is stated anywhere. Unless downstream test records were excluded from pre-training, the benchmark compares a model that may have memorized test subjects against baselines with no such exposure. This would invalidate the 'any real-world ECG data' generalization claim even if every Table II through Table V number is correct. The identical entries in Tables III and IV (e.g., DENS-ECG and ContraWR rows) are a further internal red flag, but the unresolved pretraining/test overlap is the load-bearing issue because it undermines the main contribution, not just a single result.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces AnyECG, a family of ECG foundation models (254M–1.7B parameters) that use a two-stage self-supervised pretraining: a Rhythm Quantizer that maps fixed-duration ECG patches to discrete codebook tokens via morphology, wavelet-frequency, and demographic proxy tasks, followed by masked modeling of token sequences. The authors claim that jointly pretraining on seven heterogeneous ECG datasets yields a model that, after fine-tuning, outperforms specialized baselines on anomaly detection, arrhythmia classification, corrupted lead generation, and ultra-long ECG analysis, with an average improvement of 6%.","tokens_in":18732,"tokens_out":7746,"duration_ms":63043,"significance":"AnyECG addresses a timely and important problem: constructing a single ECG encoder that works across devices, sampling rates, lead counts, and noise levels. The proposed components (Cardio-Sparse Attention, multi-view quantizer) are technically interesting, and the paper reports ablations supporting the contribution of each loss term. If the empirical results were trustworthy, this would be a valuable contribution to the ECG foundation model literature. However, the evaluation protocol has a load-bearing data-leakage problem, and several tables are internally inconsistent, so the significance cannot be assessed from the current manuscript.","major_comments":[{"comment":"The central generalization claim is not supported because the downstream evaluation appears to use the same data as pretraining. Section IV-A states that the evaluation used 'a comprehensive set of ECG datasets that include all available unlabeled data during pre-training' and that 'for various downstream tasks, we mixed all datasets together.' Section IV-C states that pretraining was conducted on seven different sources, and Table I lists exactly seven datasets, including the 'Undisclosed Database,' which is described in the text as a 'geographically distinct test set.' No patient-level or record-level split between pretraining and downstream test sets is described anywhere. Therefore, the reported results may be inflated by memorization of test subjects, and the claimed generalization to real-world data is not demonstrated. The authors must re-run the evaluation on a held-out set that was never used in pretraining, or explicitly state and justify the split if one exists.","section":"Section IV-A, Table I, Section IV-C"},{"comment":"Tables III and IV contain numerically identical rows for multiple baselines on two different tasks. For example, DENS-ECG, ContraWR, CNN-Transformer, FFCL, and ST-Transformer have the same four numbers in both tables, yet the column labels differ: Table III reports AUROC and Weighted F1 Score as the third and fourth columns, while Table IV reports Weighted F1 Score and Precision. Either the experiments were accidentally run on the same task and mislabeled, or the tables were copied incorrectly. This makes it impossible to verify the task-specific baseline comparisons and undermines the credibility of the experimental results.","section":"Tables III and IV"},{"comment":"The abstract claims 'an average performance improvement of 6% across four critical tasks,' but the manuscript never defines how this average is computed or which numbers are averaged. More importantly, many of the claimed gains are within the reported standard deviations (e.g., Table III, AnyECG-XL vs. RNN1D for AUROC; Table IV, AnyECG-XL vs. DENS-ECG for accuracy). No statistical significance tests are reported, and the arrhythmia detection accuracies are very low for all methods (≤0.35), which raises questions about the task setup. The authors should provide a clear aggregation rule, error bars on the aggregate, and significance tests, and should discuss the practical relevance of the absolute performance.","section":"Abstract and Section IV-D"}],"minor_comments":[{"comment":"The statement that '300 Hz is considered sufficient for diagnosing most cardiac conditions based on the Nyquist-Shannon sampling theorem' is an overgeneralization; the Nyquist theorem applies to bandlimited signals, and the filter cutoff is 75 Hz, so resampling to 300 Hz may discard high-frequency content relevant for some diagnoses. A sensitivity analysis or a citation for the sufficiency claim would strengthen the paper.","section":"Section III-A"},{"comment":"The notation is inconsistent: P is used both for the number of patches (Section III-A) and for the patch size (Section IV-C: 'we set the patch size P = 300'). This can confuse readers and should be corrected.","section":"Section III-A and Section IV-C"},{"comment":"Hyperparameters such as codebook size K, masking ratio r, CSA mask width, and DWT levels Lw are mentioned but never specified; a hyperparameter table would aid reproducibility.","section":"Section IV-C and Appendix"},{"comment":"Figure 3 would benefit from axis labels, time scales, and voltage units; a quantitative comparison with the original signal traces would be more informative than the current visual display.","section":"Figure 3"},{"comment":"Minor typos include 'AnyECGs' in the introduction (Section I) and 'Therefore, We proposed' in Section IV-D(d).","section":"Throughout"}],"recommendation":"reject","confidential_remarks":"The identical numerical entries in Tables III and IV, combined with the unresolved pretraining/test overlap, are serious concerns. I recommend that the editor ask the authors to provide the raw evaluation protocols and results; even with major revision, the central claim of generalization would require a new evaluation on data that is provably held out from pretraining."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things up front: the architecture is a coherent engineering combination, and the evaluation as written does not support the headline claim. The paper's own text suggests the supposedly independent test set was part of pretraining.\n\nWhat is actually new and worth credit: the two-stage pretraining idea is sensible. A vector-quantized codebook with three decoding views (morphology, wavelet frequency, demographic attributes) followed by masked modeling with a cardio-sparse attention pattern is a reasonable and fairly novel combination for ECG. The model family scales to 1.7B parameters and is applied to four distinct tasks. The ablations show that each loss term and each pretraining stage contributes to downstream performance, which is real internal evidence that the design choices matter.\n\nThe soft spots are load-bearing. Section IV-A says the evaluation used \"all available unlabeled data during pre-training\" and mixed all datasets together for downstream tasks. Table I lists the \"Undisclosed Database\" as one of seven datasets, and Section IV-C says pretraining used seven sources. That same database is described as a \"geographically distinct test set.\" No subject-level or record-level split is stated anywhere. So the central generalization claim is not testable from the paper; the model may have memorized test subjects during pretraining. This is not a minor issue, it undermines the main contribution.\n\nSecond, Tables III and IV contain identical numbers for several baselines (DENS-ECG, ContraWR, CNN-Transformer, FFCL, ST-Transformer) across two different tasks. That looks like a copy-paste error and further erodes trust in the benchmark.\n\nThird, the abstract claims AnyECG \"significantly outperforms state-of-the-art methods in each of these tasks,\" but in corrupted lead generation AnyECG has worse MAE than CGAN. The paper acknowledges this in the text, but the blanket claim remains overstated.\n\nFourth, no code, weights, or key hyperparameters (codebook size, masking ratio, commitment loss weight, CSA mask width) are provided. Independent replication is impossible as it stands.\n\nThe paper's own limitation section only discusses model scale and data availability, not these evaluation problems.\n\nMy recommendation: this deserves a serious referee, but not acceptance. The architecture has merit and the fix is tractable: rerun with a clean pretraining/evaluation split or a genuinely external dataset, release code and weights, correct the duplicate table entries, and temper the claims. If the authors do that, this could become a useful foundation model paper. I would send it to peer review with a request for major revision, not desk reject.","headline":"Plausible architecture, but the evaluation as written does not support the 'any real-world ECG' claim because the test set appears to be inside the pretraining corpus.","tokens_in":19395,"tokens_out":2704,"would_cite":false,"duration_ms":27069,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"AnyECG is a pretrained ECG foundation model that turns any real-world recording into discrete rhythm codes and, after fine-tuning, outperforms task-specific state-of-the-art models on four cardiac tasks.","keywords":["ECG foundation model","self-supervised pretraining","rhythm codebook","cardio-sparse attention","masked modeling","arrhythmia classification","ultra-long ECG analysis","signal denoising"],"falsifier":"Compare AnyECG's anomaly detection on a database of subtle ST-segment and T-wave abnormalities recorded at a native sampling rate above 300 Hz, running one version on signals resampled to 300 Hz and another on native-resolution signals. If the downsampled version loses materially on these cases, the paper's any-real-world-ECG-data promise is falsified.","tokens_in":18293,"feed_emoji":"🫀","tokens_out":7186,"duration_ms":68008,"temperature":0.7,"pith_summary":"AnyECG is a family of pretrained models built to handle electrocardiogram (ECG) recordings from any device, length, sampling rate, lead count, or noise level. The paper's central claim is that a single fine-tuned AnyECG backbone outperforms task-specific state-of-the-art methods on anomaly detection, arrhythmia classification, corrupted lead generation, and ultra-long ECG analysis, with an average improvement of 6%. The key move is to convert noisy raw ECG patches into discrete, clinically meaningful rhythm codes during a first pretraining stage, then teach the model how rhythm codes link into cardiac events during a second masked-modeling stage. A sparse attention mechanism keeps the transformer efficient enough for very long recordings. If the claim holds, one shared model could replace several specialized ECG classifiers.","feed_headline":"Pretrained ECG model lifts four cardiac tasks by 6 percent on average","feed_subtitle":"One model learns discrete rhythm codes from noisy heart signals, then fine-tunes to beat specialized baselines.","key_machinery":"The load-bearing object is the Rhythm Codebook, a vector-quantized lookup table of normalized codewords that maps each fixed-duration ECG patch to its nearest code, turning continuous, noisy waveforms into discrete tokens. It is trained with a multi-view synergistic decoder: a morphology decoder reconstructs the raw time-domain signal, a frequency decoder reconstructs discrete wavelet transform coefficients, and a demography decoder predicts patient attributes, with codebook and commitment losses stabilizing quantization. The second stage runs masked modeling over these code indices, and the Cardio-Sparse Attention mask (each patch attends only to patches from the same lead or the same temporal position, within a tolerance window) is the mechanism that keeps ultra-long sequences computationally feasible.","core_discovery":"The paper argues that ECG analysis does not need a separate specialized model per task or per device. AnyECG first trains a tokenizer whose vector-quantized codebook indexes each one-second ECG patch to a rhythm code; three decoding objectives—waveform reconstruction, wavelet coefficient reconstruction, and demographic attribute prediction—force those codes to carry morphology, frequency, and patient-context information while discarding noise. The full model is then pretrained by masked modeling: random patches are replaced by a mask token and the model must predict the original rhythm-code index from the surrounding unmasked patches, which teaches it rhythm-event associations. Throughout, Cardio-Sparse Attention restricts each patch to attend only to same-lead and same-time-position patches with a small tolerance, cutting computation on uninformative segments. Across four downstream tasks, the AnyECG-B, AnyECG-L, and AnyECG-XL variants beat the non-pretrained baselines and the prior open ECG foundation model, with the average gain reported as 6%.","pith_inferences":["The strongest untested boundary of the paper is preprocessing: a direct ablation of native sampling rates versus 300 Hz would reveal whether the claim of universal coverage depends on that choice.","The rhythm codebook could be reused as a discrete token vocabulary for tasks the paper does not explore, such as generating natural-language descriptions of an ECG or retrieving similar clinical cases.","A more demanding test than the four reported tasks would be a truly unseen wearable device with a low native sampling rate and heavy motion artifact, since pretraining still includes mostly hospital-style high-rate recordings."],"forward_implications":["One fine-tuned AnyECG backbone can replace separate specialized models for anomaly detection, arrhythmia classification, corrupted lead generation, and ultra-long ECG analysis.","Because the rhythm codebook is trained on heterogeneous, noisy data, downstream fine-tuning on a new device or population should need less task-specific data than training from scratch.","Cardio-Sparse Attention makes transformer-based processing of ultra-long recordings practical, so the architecture can scale toward 24-hour Holter-style inputs.","The reported average gain of 6% is the quantitative claim that independent runs on the same four tasks would either confirm or refute."],"supporting_citations":[{"why":"Supplies the largest labeled 12-lead corpus, used for both pretraining and as the source of benchmark baseline models.","marker":"[34]"},{"why":"Supplies the two 12-lead corpora used in pretraining and used to evaluate anomaly and arrhythmia detection.","marker":"[31]"},{"why":"Supplies 30-minute Holter-derived recordings at a lower sampling rate, exercising varied sampling rates and long inputs.","marker":"[32]"},{"why":"Provides the db6 wavelet denoising step in the preprocessing pipeline.","marker":"[25]"},{"why":"Prior open ECG foundation model used as the pretrained comparison baseline that AnyECG must beat.","marker":"[37]"},{"why":"Non-pretrained ECG analysis baseline used in the anomaly and arrhythmia comparisons.","marker":"[35]"},{"why":"Benchmark 1D convolutional baseline for ECG classification used in anomaly, arrhythmia, and ultra-long comparisons.","marker":"[42]"},{"why":"Conditional GAN baseline for the corrupted lead generation task.","marker":"[44]"},{"why":"Wasserstein GAN baseline for the corrupted lead generation task.","marker":"[45]"}],"fun_headline_variants":["AnyECG lifts four cardiac tasks by 6% on average","ECG foundation model beats specialized baselines by 6%","One ECG model for four tasks: 6% average improvement","AnyECG: real-world ECG, one model, 6% gain on four tasks","Pretrained on noisy ECGs, AnyECG averages 6% task gain"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Resampling every ECG to 300 Hz and cutting it into fixed one-second patches preserves all diagnostically relevant waveform detail, including subtle ST-T changes and low-amplitude pathological features.","fun_headline_variants_meta":{"raw":{"variants":["AnyECG lifts four cardiac tasks by 6% on average","ECG foundation model beats specialized baselines by 6%","One ECG model for four tasks: 6% average improvement","AnyECG: real-world ECG, one model, 6% gain on four tasks","Pretrained on noisy ECGs, AnyECG averages 6% task gain"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000759,"raw_usage":{"total_tokens":3420,"prompt_tokens":1044,"completion_tokens":2376,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":660,"completion_tokens_details":{"reasoning_tokens":2279}},"tokens_in":660,"tokens_out":2376,"duration_ms":16663,"temperature":1.0,"reasoning_tokens":2279,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T18:52:53.572458+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compare AnyECG's anomaly detection on a database of subtle ST-segment and T-wave abnormalities recorded at a native sampling rate above 300 Hz, running one version on signals resampled to 300 Hz and another on native-resolution signals. If the downsampled version loses materially on these cases, the paper's any-real-world-ECG-data promise is falsified.","supporting_citations":[{"cited_title":"An open access database for evaluating the algorithms of electrocardiogram rhythm and morphology abnormality detection,","cited_arxiv_id":null,"evidence_quote":"Supplies the two 12-lead corpora used in pretraining and used to evaluate anomaly and arrhythmia detection."},{"cited_title":"St petersburg incart 12-lead arrhythmia database,","cited_arxiv_id":null,"evidence_quote":"Supplies 30-minute Holter-derived recordings at a lower sampling rate, exercising varied sampling rates and long inputs."},{"cited_title":"An effective data enhancement method for classification of ecg arrhythmia,","cited_arxiv_id":null,"evidence_quote":"Provides the db6 wavelet denoising step in the preprocessing pipeline."},{"cited_title":"Dens-ecg: A deep learning approach for ecg signal delineation,","cited_arxiv_id":null,"evidence_quote":"Non-pretrained ECG analysis baseline used in the anomaly and arrhythmia comparisons."},{"cited_title":"Deep learning for ecg analysis: Benchmarks and insights from ptb-xl,","cited_arxiv_id":null,"evidence_quote":"Benchmark 1D convolutional baseline for ECG classification used in anomaly, arrhythmia, and ultra-long comparisons."},{"cited_title":"Banach wasserstein gan,","cited_arxiv_id":null,"evidence_quote":"Wasserstein GAN baseline for the corrupted lead generation task."}],"review_version":1}