{"id":"169cbf60-af97-4c49-9021-1ced2f212cf0","arxiv_id":"2608.12695","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"On Icentia11k, self-supervised ECG models trained on 5-10 minute windows with continuous CNN patch embeddings outperform 16-second and vector-quantized models on AFib/AFL detection and patient retrieval.","lead":"This paper tests whether ECG models trained on longer recordings and continuous signal embeddings learn better patterns than models trained on short snapshots or discretized tokens. The results provide concrete design rules for building ECG foundation models for rhythm diagnosis and patient-level search.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Retrieval Recall@k is reported at native window lengths; random baselines shift from 0.009 to 0.078, so the temporal-context retrieval claim may conflate context length with test-time signal.","rationale":"The paper's practical guidance has two pillars: longer context and continuous encoders. The classification results for temporal context are on a matched evaluation protocol (probability aggregation over 592s/600s), so they are reasonably secure. The retrieval results, however, are not described under that matched protocol. The random baseline shift from 0.009 to 0.078 Rec@1 between 16s and 10min is a red flag that the retrieval tasks differ in difficulty; any claim about 'higher retrieval accuracy' from longer pretraining needs to separate the benefit of more test-time signal from the benefit of longer context. This is more load-bearing than the reader's VQ concern because the VQ comparison is scoped in the abstract to 'fixed vector-quantized tokens' and the continuous-vs-discrete gap appears consistently across all settings; the retrieval confound directly affects the headline temporal-context claim and is not acknowledged in the paper. The reader's VQ concern is still valid for the generalization to learned VQ codebooks, but it is a secondary limitation. Since the classification evidence for longer context remains intact, the correct verdict stays CONDITIONAL: the temporal-context retrieval claim needs reanalysis, but the overall design recommendation is plausible.","tokens_in":8572,"tokens_out":10146,"duration_ms":103908,"concrete_test":"Recompute Recall@1/5 for the 16s, 1min, and 5min frozen SSL and random encoders on matched 10-minute windows: embed each non-overlapping sub-window, mean-pool the L2-normalized embeddings into one 10-minute embedding (or average pairwise similarities), and rerun retrieval against the 10-minute encoder's native embeddings. Also recompute Random+CNN/Frozen baselines under the same matching. If the SSL+CNN 16s-vs-10min Rec@1 gap (0.170) collapses to roughly the random-baseline gap (0.069), the retrieval-based support for the temporal-context claim fails; if the gap remains large after matching, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Patient-level retrieval is not evaluated under the same 10-minute protocol used for classification. Section 'Downstream Evaluation Protocol' states that 'each window was embedded independently' and Table I reports Rec@1/Rec@5 for the 16-second and 10-minute encoders without an aggregation column, while the unified 592s/600s aggregation is described only for classification probabilities. The random baselines reveal the task-difficulty shift: Random+CNN (Frozen) reaches Rec@1 0.009 at 16s but 0.078 at 10min and Rec@5 0.026 vs 0.184, even though no pretraining has occurred. A 10-minute window contains about 37x more samples per retrieval example, so patient identity is easier to recover from raw statistics alone. Thus the headline 'longer context yields higher retrieval accuracy' (SSL+CNN Frozen 0.737 vs 0.907) conflates pretraining context length with the amount of test-time signal. The classification results are not affected because both encoders are evaluated on fixed 10-minute windows via probability averaging.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript reports a controlled study of self-supervised ECG representation learning on the Icentia11k dataset. It varies the input context length (16 s, 1 min, 5 min, 10 min) and the front-end representation (continuous 1D CNN patch embeddings vs. a fixed offline K-means VQ codebook with K=256), while keeping the Transformer backbone, SSL objective, and training hyperparameters fixed. Representations are evaluated by downstream AFib/AFL vs. normal rhythm classification (linear probing and fine-tuning) and by patient-level retrieval Recall@1/Recall@5. The main claims are that longer temporal context improves both transfer and retrieval, with the best results at 5 and 10 minutes, and that continuous embeddings outperform discretized tokens across all tested settings.","tokens_in":8792,"tokens_out":5362,"duration_ms":57233,"significance":"If the results hold, the paper provides practical design guidance for ECG foundation models: multi-minute context windows and continuous, non-quantized encoders are preferable for rhythm classification and patient-consistent retrieval. The strengths of the study are the controlled experimental design, use of a public dataset, release of code and pretrained models, consistent architecture and training protocol across conditions, and large reported effect sizes, e.g., frozen 10-minute CNN AUPRC 0.877 vs. 0.610 for VQ, and Recall@1 0.907 vs. 0.351. The downstream AFib/AFL classification is an external clinical task and is therefore not circular with the pretraining objective. However, the retrieval evidence for the temporal-context claim is confounded by the amount of test-time signal, and all numerical comparisons are single-run point estimates, so the conclusions are not yet as strong as the abstract states.","major_comments":[{"comment":"The patient-retrieval comparison at native window lengths conflates pretraining context length with the amount of test-time signal available per retrieval example. In Table I, the random baselines shift dramatically without any pretraining: Random+CNN (Frozen) Recall@1 rises from 0.009 at 16 s to 0.078 at 10 min and Recall@5 from 0.026 to 0.184, and Random+VQ (Frozen) Recall@1 rises from 0.022 to 0.148. A 10-minute window contains roughly 37 times more samples than a 16-second window, so raw statistics alone make patient identity easier to recover. The abstract and §IV therefore overstate the claim that longer pretraining context 'yields higher retrieval accuracy'; the current evidence supports only that longer test windows make retrieval easier. To support the representation-quality claim, the retrieval evaluation should be run under a matched-duration protocol, for example by aggregating sub-window embeddings from shorter-context encoders on fixed 10-minute inputs, with the random-baseline shift reported explicitly.","section":"§III 'Downstream Evaluation Protocol'; Table I"},{"comment":"All quantitative comparisons are single-run point estimates from one train/validation/test split with a fixed seed, and no confidence intervals, error bars, or significance tests are reported anywhere. This matters specifically for the claimed ordering of the 1-minute, 5-minute, and 10-minute conditions in Fig. 2a and for the intermediate-context retrieval trend, because those claims rest on the relative magnitudes of small or unquantified differences rather than on a single large gap. The paper should either report multiple seeds with means and variances, or clearly state that the intermediate-context conclusions are qualitative and not statistically tested.","section":"§III 'Experimental Setup'; §IV 'Impact of Temporal Context' and Fig. 2a"},{"comment":"The conclusion that 'quantization can discard clinically relevant waveform detail' is presented as a general property of discretized tokenization, but the experiments only test one specific VQ recipe: an offline K-means codebook with K=256 centroids, trained on roughly 563,000 patches and held fixed during SSL training. No variation of codebook size, codebook training procedure (e.g., end-to-end learned VQ as in HeartLang), patch size, or patch resolution is reported, so the continuous-versus-discrete gap may be specific to this configuration. The claim should either be restricted to the tested VQ configuration, or the robustness of the gap should be demonstrated across at least one alternative quantizer or codebook size.","section":"§III 'Temporal Context and Tokenization'; §V Conclusion"}],"minor_comments":[{"comment":"The figure is confusing: panel labels '(a)' and '(b)' appear twice in the caption, and the text refers to 'Fig. 2b and 2c' while the caption labels the t-SNE panels as '(b)' and '(c)'. Please renumber the panels consistently.","section":"Fig. 2 caption and panels"},{"comment":"For Fig. 2a, the caption says retrieval metrics were computed using fixed 10-minute windows with shorter-context models aggregated over sub-windows, but the text says each retrieval window was 'embedded independently' and Table I has no aggregation columns for Recall@k. Please state explicitly how retrieval embeddings were aggregated for shorter-context encoders, or clarify that Fig. 2a uses a different retrieval protocol than Table I.","section":"§III 'Downstream Evaluation Protocol'"},{"comment":"The description of the 'Random + VQ' baseline is underspecified: the random encoder has not seen the VQ codebook during pretraining, so it is unclear how the VQ token assignments are generated at evaluation time. Please clarify what is frozen and what is randomly initialized in this baseline.","section":"§III 'Temporal Context and Tokenization'"},{"comment":"Per-window z-score normalization before patch extraction is applied to windows of very different lengths (16 s vs. 10 min). Because this normalizes each window by its own mean and standard deviation, it may remove long-timescale amplitude or baseline information that is part of what a longer context could capture. Please justify this choice or discuss its effect on the temporal-context comparison.","section":"§III 'Dataset and Preprocessing'"},{"comment":"The equation for probability aggregation, p_+ = (1/K) Σ softmax(ℓ_j)_+, uses a subscript '+' that is not defined; please define it as the AFib/AFL class probability and clarify the notation.","section":"§IV 'Results & Discussion'"},{"comment":"The table is dense and the column alignment is unclear: '(Agg.)' applies only to the 16-second encoder's AUC/AUPRC, while the 10-minute encoder columns have no such marker, and the retrieval columns have no aggregation marker at all. Adding explicit column group headers would improve readability.","section":"Table I"}],"recommendation":"major_revision","confidential_remarks":"The paper is a well-scoped controlled study with a clear practical message and a public code release. The main technical risk is the retrieval confound: the temporal-context retrieval claim is not supported by the current native-window protocol, and the lack of any uncertainty quantification makes the quantitative ordering of conditions fragile. I would ask the authors to either add a matched-duration retrieval experiment or substantially weaken the retrieval claim in the abstract and conclusions. The continuous-vs-discrete conclusion is also more conditional than the abstract suggests, given that only one VQ configuration was tested. These are fixable with additional experiments or a careful rewriting, so rejection is not warranted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: if you build ECG foundation models, this paper gives you a concrete, reproducible comparison of 16s vs 1/5/10min contexts and continuous CNN vs fixed VQ tokenization that you did not have before. The classification half of the study is solid and supports the main design guidance. The retrieval half is shakier than the abstract suggests.\n\nWhat the paper does well: fixed Transformer backbone, same patient-level splits, same hyperparameters; public code and pretrained models. Classification is evaluated on fixed 10-min windows via probability averaging, so the AFib/AFL comparison between 16s and 10min is apples-to-apples. The result that SSL + CNN at 10min reaches AUPRC 0.960 and linear-probing AUPRC 0.877 is a useful datapoint. The controlled contrast between CNN patches and VQ tokens under one backbone is real.\n\nSoft spots. The retrieval comparison in Table I is not apples-to-apples: 16s and 10min encoders are evaluated on windows of their own native lengths. Random+CNN frozen retrieval goes from Rec@1 0.009 at 16s to 0.078 at 10min with no pretraining at all, so a large part of the \"longer context improves retrieval\" effect is just more signal per query window. The paper does not describe how short-context retrieval would be aggregated to 10min; Fig 2a's caption says fixed 10-min windows, but the methods only describe aggregation for classification probabilities. That needs to be fixed or clarified. Once you account for that, the temporal-context retrieval claim is much weaker.\n\nSecond, all numbers are single-run point estimates from one split. No error bars, no significance tests. For a benchmark meant to give design rules, that is a real gap, particularly when some gaps are small (e.g., CNN FT 16s AUPRC 0.946 vs 10min 0.960).\n\nThird, the continuous-vs-discrete conclusion rests on one VQ recipe: offline K-means with K=256 centroids fixed during SSL. The effect is large, but the paper hasn't varied codebook size or training. Saying \"quantization can discard clinically relevant waveform detail\" beyond this recipe is a hypothesis, not a demonstrated general law.\n\nNone of these are fatal. The classification result and the controlled setup are worth publishing as a benchmark. The authors should add repeats, clarify or repair the retrieval protocol, and tone down the general claims about quantization. I would send it out for review.","headline":"A clean controlled benchmark of context length and tokenization for ECG SSL, with genuinely useful classification results, but the retrieval claim is confounded by window size and all numbers are single-run.","tokens_in":9320,"tokens_out":3422,"would_cite":true,"duration_ms":34306,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Longer ECG context and continuous encoders beat short windows and tokenized ECGs.","keywords":["ECG","self-supervised learning","temporal context","vector quantization","contrastive learning","representation learning","ambulatory ECG","Icentia11k"],"falsifier":"Retrain the same SSL pipeline with an end-to-end learned VQ codebook or with codebooks of sizes 512, 1024, and 4096, then compare 10-minute AFib/AFL AUPRC and Recall@1 against the continuous CNN encoder; if the gap narrows to near zero or reverses, the paper's central claim about discretization fails.","tokens_in":8356,"feed_emoji":"❤️","tokens_out":4483,"duration_ms":40318,"temperature":0.7,"pith_summary":"Self-supervised ECG models are often trained on a few seconds of signal, sometimes after quantizing the waveform into discrete tokens. This paper asks whether both choices discard information needed for rhythm diagnosis and for recognizing the same patient across days. Using a controlled comparison on ambulatory single-lead recordings, it reports that extending the input window to five or ten minutes improves downstream atrial fibrillation and atrial flutter detection and patient retrieval, and that continuous patch embeddings beat a fixed vector-quantized codebook at every horizon. If the result holds, ECG foundation models should be built around multi-minute context and continuous encoders rather than short snapshots and token vocabularies.","feed_headline":"ECG AI needs minutes of context, not 16-second snapshots","feed_subtitle":"A controlled study shows 5- and 10-minute windows plus continuous encoders beat short, tokenized training.","key_machinery":"The controlled pipeline is the machinery: one shared 6-layer Transformer encoder with hidden size 256 and 8 heads, trained with an InfoNCE contrastive loss that pairs two windows drawn from different recording segments of the same patient. The only systematic variations are input horizon (16 seconds, 1 minute, 5 minutes, 10 minutes) and front-end tokenization (continuous 1D-CNN patch embeddings versus a fixed offline K-means codebook with 256 centroids, each patch covering 160 samples). Patient-level splits, fixed evaluation windows, and a convergence-based training budget isolate the two factors. The same fixed 10-minute evaluation windows are used across models, with shorter-context encoders aggregated over non-overlapping sub-windows, so the comparison reflects representation quality rather than input duration during evaluation.","core_discovery":"The paper's central discovery is that, holding the Transformer backbone, contrastive objective, and training protocol fixed, the two design choices that dominate current ECG foundation models both hurt performance. Models pretrained on 16-second windows reach a frozen-encoder AUPRC of 0.712 for AFib/AFL versus normal rhythm and a patient-level Recall@1 of 0.737; models pretrained on 10-minute windows reach 0.877 AUPRC and 0.907 Recall@1, with the largest gains appearing at 5- and 10-minute contexts. Replacing continuous 1D-CNN patch embeddings with a fixed K-means VQ codebook of 256 centroids drops the 10-minute frozen AUPRC from 0.877 to 0.610 and Recall@1 from 0.907 to 0.351, and the same loss appears at every horizon. The authors interpret this as evidence that long context captures slow rhythm dynamics and patient-specific structure, while quantization discards clinically relevant waveform detail.","pith_inferences":["The quantization gap may shrink or reverse with a learned end-to-end codebook or a much larger codebook; the paper only tests one fixed offline K-means recipe with 256 centroids, so its 'discretization hurts' conclusion is conditional on that recipe.","If longer context works because it exposes the model to more rhythm-state transitions, then context length beyond 10 minutes may show diminishing returns; a 20- or 30-minute comparison would test this.","The strong patient retrieval result suggests the same embeddings could serve as a biometric or a stratification tool for rare disease discovery, but the paper only evaluates AFib/AFL, so extension to other labels remains open.","The aggregation protocol means shorter-context models are evaluated only through averaging over sub-windows; a multiple-instance or attention-based aggregation could narrow the context gap and is a natural next test."],"forward_implications":["ECG self-supervised pretraining should move from 16-second snapshots to at least 5-minute contexts, since the largest classification and retrieval gains appear at 5 and 10 minutes.","Continuous patch embeddings should be preferred over fixed VQ tokenization when building ECG foundation models, because the discrete bottleneck degraded both clinical metrics and patient retrieval at every horizon tested.","Pretrained ECG representations can support cross-session patient retrieval (Recall@1 near 0.91), enabling similarity-based applications such as cohort stratification and longitudinal monitoring.","Fine-tuning the pretrained encoder improves AFib/AFL classification but lowers patient retrieval, so deployment choices should weigh task-specific accuracy against identity-preserving structure."],"supporting_citations":[{"why":"Supplies the Icentia11k ambulatory single-lead ECG dataset with long, multi-segment, patient-level recordings used in all experiments.","marker":"[12]"},{"why":"Establishes the short-window SSL baseline and evaluation approach on clinical ECGs that this study extends and contrasts with.","marker":"[5]"},{"why":"Introduces patient-consistent contrastive learning, the idea that same-patient windows form positive pairs across time and leads.","marker":"[13]"},{"why":"Defines the InfoNCE objective that drives the self-supervised pretraining in this paper.","marker":"[14]"},{"why":"Motivates the mean-aggregation protocol for rhythm-like labels in ECG foundation models and supports the evaluation design.","marker":"[6]"},{"why":"Represents the VQ-based ECG tokenization trend that the paper's discrete setting is compared against.","marker":"[11]"},{"why":"Represents another quantized tokenization approach (HeartBERT) cited as part of the discretization trend.","marker":"[10]"},{"why":"Supports the observed trade-off that fine-tuning can distort pretrained features and reduce patient-identity preservation.","marker":"[17]"}],"fun_headline_variants":["ECG AI: longer context beats short snapshots","Continuous ECG encoders surpass tokenized versions","Minutes of ECG context boost AI rhythm detection","ECG foundation models: context length and encoding matter","Long, continuous ECG signals improve self-supervised AI"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The conclusion that quantization discards clinically relevant detail rests on a single VQ implementation, a fixed offline K-means codebook with 256 centroids, so the result may not generalize to learned or larger codebooks.","fun_headline_variants_meta":{"raw":{"variants":["ECG AI: longer context beats short snapshots","Continuous ECG encoders surpass tokenized versions","Minutes of ECG context boost AI rhythm detection","ECG foundation models: context length and encoding matter","Long, continuous ECG signals improve self-supervised AI"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00027,"raw_usage":{"total_tokens":1652,"prompt_tokens":999,"completion_tokens":653,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":615,"completion_tokens_details":{"reasoning_tokens":581}},"tokens_in":615,"tokens_out":653,"duration_ms":6289,"temperature":1.0,"reasoning_tokens":581,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T04:22:04.440489+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain the same SSL pipeline with an end-to-end learned VQ codebook or with codebooks of sizes 512, 1024, and 4096, then compare 10-minute AFib/AFL AUPRC and Recall@1 against the continuous CNN encoder; if the gap narrows to near zero or reverses, the paper's central claim about discretization fails.","supporting_citations":[{"cited_title":"Self-supervised representation learning from 12-lead ecg data,","cited_arxiv_id":null,"evidence_quote":"Establishes the short-window SSL baseline and evaluation approach on clinical ECGs that this study extends and contrasts with."},{"cited_title":"Clocs: Contrastive learning of cardiac signals across space, time, and patients,","cited_arxiv_id":null,"evidence_quote":"Introduces patient-consistent contrastive learning, the idea that same-patient windows form positive pairs across time and leads."},{"cited_title":"Reading your heart: Learning ECG words and sentences via pre-training ECG language model,","cited_arxiv_id":null,"evidence_quote":"Represents the VQ-based ECG tokenization trend that the paper's discrete setting is compared against."},{"cited_title":"Fine-tuning can distort pretrained features and underperform out-of-distribution,","cited_arxiv_id":null,"evidence_quote":"Supports the observed trade-off that fine-tuning can distort pretrained features and reduce patient-identity preservation."}],"review_version":1}