{"id":"9c608ce9-cb36-443b-9115-faf4cb3398c4","arxiv_id":"2505.05538","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":2.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"The paper proposes Cardioformer, a Medformer-style multi-granularity ECG classifier with residual blocks, and reports modest AUROC gains on MIMIC-IV, PTB-XL, and PTB under subject-independent splits.","lead":"Cardioformer is a transformer model for ECG classification that adds residual blocks to a Medformer-style multi-granularity patch encoder. It reports small AUROC gains on three public datasets, but its main claims are contradicted by its own tables and its cross-dataset evaluation is not fully specified.","discovery_kind":"incremental","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Cross-dataset generalization is uninterpretable: no label or lead mapping from MIMIC-IV (4-class, 12-lead) to PTB (2-class, 15-lead) or PTB-XL (5-class) is specified, and the reported 49.18% PTB AUROC is near chance.","rationale":"The reader's weakest assumption identifies the same load-bearing concern: the cross-dataset evaluation lacks an explicit label and channel mapping, so the headline generalization numbers are not interpretable. I agree this is the primary issue because the within-dataset AUROC results do support a consistent, if modest, improvement over the four baselines (e.g., MIMIC-IV AUROC 96.34 vs. Medformer 95.76; PTB-XL 89.99 vs. 88.93), whereas the cross-dataset claim is a distinct advertised contribution and is invalidated by the missing protocol. A secondary internal inconsistency exists: Section 4's statement that Cardioformer achieves the highest performance in all metrics on PTB is false in Table 3, where Reformer's accuracy (93.87) and precision (88.15) exceed Cardioformer's (92.84 and 85.23). That contradiction further weakens the paper's framing, but the cross-dataset flaw is more central because it undermines a standalone abstract claim. Since the reader already recommended REJECT and this analysis reinforces that decision, the verdict remains unchanged.","tokens_in":8475,"tokens_out":7656,"duration_ms":78195,"concrete_test":"Run the released Cardioformer code (github.com/KMobin555/Cardioformer) to reproduce Table 4 with an explicit protocol: (i) choose a 12-lead subset of PTB (e.g., the 12 leads shared with MIMIC-IV) and verify the embedding forward pass accepts it; (ii) fix a deterministic mapping from MIMIC-IV's four output logits to PTB's two classes and PTB-XL's five classes; (iii) recompute AUROC on PTB and PTB-XL. If the reported 49.18% and 68.41% cannot be reproduced under any reasonable mapping, or if no mapping is implemented in the repository, the cross-dataset claim should be withdrawn.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central generalization claim rests on comparing a model trained on MIMIC-IV's four-class, 12-lead format with PTB's two-class, 15-lead format and PTB-XL's five-class, 12-lead format. The method fixes the embedding dimension in Eq. (1) as W(i) ∈ R^{(L_i·C)×D}, so C = 12 for MIMIC-IV. Evaluating on PTB therefore requires an explicit selection of 12 of 15 leads and an explicit mapping of PTB's binary labels into the four MIMIC-IV classes; evaluating on PTB-XL requires extending or collapsing the five-class label space into four classes. No such mapping or lead selection is described in Section 4, Table 4, or Appendix B/C. Without it, the AUROC values in Table 4 are not well-defined, and the abstract's '49.18% AUROC on PTB' cannot be interpreted as a generalization measurement. Moreover, 49.18% is at chance level for binary classification, so calling this 'strong cross-dataset generalization' is contradicted by the paper's own number. Because the cross-dataset result is a headline contribution, this missing protocol is load-bearing for the paper's central claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Cardioformer, a hybrid CNN-transformer for ECG classification that combines cross-channel multi-granularity patch embedding, a ResNet patch projector, and a two-stage intra-/inter-granularity self-attention mechanism. The authors evaluate Cardioformer against PatchTST, Reformer, Transformer, and Medformer on MIMIC-IV, PTB-XL, and PTB under subject-independent splits with three seeds, and report the highest AUROC on all three datasets (96.34, 89.99, and 95.59). The paper further claims strong cross-dataset generalization for a MIMIC-IV-trained model, reporting 49.18% AUROC on PTB and 68.41% AUROC on PTB-XL, and states that source code is released.","tokens_in":8748,"tokens_out":13836,"duration_ms":124922,"significance":"If the in-domain AUROC results hold, Cardioformer is a credible incremental contribution: it attains the best AUROC among the five compared models on three public ECG benchmarks, with standard deviations reported over three seeds and a stated subject-independent split, and the code release is a positive reproducibility step. The significance is substantially weakened, however, by three problems: the claim of winning 'all metrics' is directly contradicted by Table 3; the headline cross-dataset result is methodologically undefined (no label or lead mapping is given) and the PTB transfer number is at chance; and the architectural novelty relative to Medformer is thin, since Section 3 reproduces Medformer's patch embedding, router embeddings, and two-stage attention. The paper also acknowledges patch-length sensitivity in its limitations, which is relevant because the multi-granularity patch list is a dataset-specific free choice.","major_comments":[{"comment":"The claim that 'Cardioformer achieves the highest performance in all metrics on PTB, PTB-XL, and MIMIC-IV' is contradicted by the paper's own table. On PTB, Reformer reports higher Accuracy (93.87±0.27 vs 92.84±0.92) and higher Precision (88.15±0.75 vs 85.23±1.59); on PTB-XL, PatchTST reports higher Precision (68.09±1.41 vs 65.28±0.73); on MIMIC-IV, PatchTST reports higher Accuracy (85.00±0.10 vs 84.71±0.24) and higher Precision (82.17±0.24 vs 81.96±0.28). The defensible empirical claim is that Cardioformer attains the highest AUROC on all three datasets; the text as written overstates the results and must be corrected.","section":"Section 4, Table 3"},{"comment":"The cross-dataset experiment lacks a defined evaluation protocol. A model trained on MIMIC-IV outputs four classes from 12-lead inputs (Eq. (1) fixes the input channels at C=12), while PTB is a 2-class, 15-lead dataset and PTB-XL is a 5-class, 12-lead dataset. The paper never specifies how the four MIMIC-IV labels are mapped to PTB's binary classes or to PTB-XL's five classes, nor how 12 of the 15 PTB leads are selected or harmonized. Without this specification, the AUROC values in Table 4 are not well-defined and the transfer experiment cannot be reproduced or interpreted.","section":"Section 4, Table 4, Appendix B"},{"comment":"The abstract and Section 4 describe 49.18% AUROC on PTB as 'strong cross-dataset generalization' and state that the model 'performs well' there, but for PTB's binary classification this value is at chance level (50%) and is below the Reformer baseline's 52.05±11.29 under the same protocol. No correction to the mapping protocol can turn this number into evidence of strong transfer; the PTB transfer result must be reported as a negative finding or removed from the claims.","section":"Abstract and Section 4, Table 4"},{"comment":"Tables 3 and 4 report inconsistent in-domain MIMIC-IV numbers for what should be the same experiment: Cardioformer's MIMIC-IV Accuracy is 84.71±0.24 with AUROC 96.34±0.11 in Table 3 but 88.28±0.39 with AUROC 94.91±0.05 in Table 4. Since Table 4's caption states that all models are trained exclusively on MIMIC-IV, the MIMIC-IV rows there should match Table 3 under the same data splits and seeds. The discrepancy is unexplained and must be resolved or explicitly justified.","section":"Tables 3 and 4"},{"comment":"The architecture as described is very close to Medformer [9]: the multi-granularity patch embedding with granularity embeddings, router embeddings u(i), and the intra-/inter-granularity two-stage self-attention in Eqs. (1)-(5) reproduce Medformer's design, and the only structural difference stated is the use of a ResNet for the patch projection. The reported AUROC gains over Medformer are modest (1.10 on PTB, 1.06 on PTB-XL, 0.58 on MIMIC-IV). The paper should explicitly enumerate which components are new relative to Medformer; otherwise the 'novel' claim in the abstract is not supportable.","section":"Section 3 vs Medformer"}],"minor_comments":[{"comment":"Reference [16] (Liu et al., iTransformer) is cited for the 'Transformer' baseline; the plain Transformer baseline should cite Vaswani et al. [4].","section":"References"},{"comment":"The first manuscript page contains the placeholder 'DOI: 0', which must be removed or filled in.","section":"Title page"},{"comment":"The equations in Section 3 are typeset with broken superscripts (e.g., W(i) ∈ R(Li·C)×D); the exponent should be rendered as R^{(L_i·C)×D} and the patchified input x_p^{(i)} should be defined precisely.","section":"Section 3"},{"comment":"The augmentation description in Appendix C is internally inconsistent: it says augmentations are drawn from 'none, jitter, scale, and mask,' but Appendix A defines channel shuffling, temporal masking, frequency masking, jittering, and dropout; 'scale' and 'mask' are not defined as standalone methods there.","section":"Appendix C"},{"comment":"There are several grammatical and typographical errors, including 'demonstrate that model consistently outperforms' in the abstract, 'performance ofCardioformer' in the Table 4 caption, and the garbled phrase 'while Intra-Granularity Self-AttentionConcatenate' in Section 3.","section":"Abstract and tables"},{"comment":"The introduction reports 'AUPRC of 87.65' for MIMIC-IV; Table 3 reports 87.65±0.42, so the standard deviation should be included for consistency with the other reported values.","section":"Introduction"}],"recommendation":"reject","confidential_remarks":"The paper's overlap with Medformer (arXiv:2405.19363) is substantial: the multi-granularity patching, granularity embeddings, router embeddings, and intra/inter-granularity attention are the same design, and the only architectural change is the ResNet patch projector. If the authors resubmit, the novelty claim must be narrowed accordingly. The revealed mismatch between the abstract's headline claims and the paper's own tables is the main reason for my recommendation; the 'DOI: 0' placeholder and the inconsistent augmentation description in Appendix C add to the impression of hasty preparation. The in-domain AUROC results could plausibly support a revised submission that drops or honestly reframes the cross-dataset transfer claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: Cardioformer is basically Medformer with ResNet blocks in the patch embedding. Code is public, and the within-dataset evaluation is done with subject-independent splits, three seeds, and a common library, so the numbers are honest in that narrow sense. But the paper's own tables contradict its headline claims. 'Highest performance in all metrics' is false: Reformer beats it on PTB accuracy and precision; PatchTST beats it on MIMIC-IV accuracy. The gains over Medformer are small (0.2–1.3 AUROC points) and often within one standard deviation, so the 'advancing AI in ECG' rhetoric is not supported.\n\nThe cross-dataset generalization section is worse than unsupported. Models trained on MIMIC-IV's four-class, 12-lead format are evaluated on PTB's two-class, 15-lead and PTB-XL's five-class data, with no description of lead selection or label mapping. The reported 49.18% AUROC on PTB is therefore not a well-defined number. And even if it were, 0.49 is at chance for a binary problem; calling it 'strong cross-dataset generalization' is an internal contradiction.\n\nWhat's genuinely here: a clean implementation, honest citation of Medformer, and a plausible if small within-dataset improvement on PTB-XL and MIMIC-IV. The method is a delta, not a new architecture, and the authors acknowledge hyperparameter sensitivity. What's missing is a truthful statement of results and any protocol for cross-dataset transfer.\n\nThis is a paper for a reader who wants a reproducible ECG baseline, not for someone looking for state-of-the-art. I would not bring it to reading group, and I would not cite it. A serious editor would be right to desk reject; if the authors fix the cross-dataset protocol and tone down the claims, it could be a workshop-level contribution. Recommend reject.","headline":"Cardioformer is a thin Medformer variant whose small gains don't match its claims, and the cross-dataset result is uninterpretable as reported.","tokens_in":9287,"tokens_out":4371,"would_cite":false,"duration_ms":45193,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Cardioformer, a multi-granularity ECG transformer with two-stage self-attention and residual blocks, claims the highest AUROC among five models on the PTB, PTB-XL, and MIMIC-IV benchmarks, and reports a MIMIC-IV-trained model transferring…","keywords":["ECG Classification","Transformer","ResNet","Multi-Granularity","Cross-Channel Self-Attention","Time-Series Analysis","Patch Embedding","Cross-Dataset Generalization"],"falsifier":"Test the transfer claim by evaluating the MIMIC-IV-trained Cardioformer on PTB after truncating PTB from 15 leads to the 12 leads shared with MIMIC-IV and mapping PTB's binary labels into the four-class scheme; if AUROC remains at or below 50%, the cross-dataset result is an artifact of the mismatched protocol rather than evidence of generalization.","tokens_in":8241,"feed_emoji":"🫀","tokens_out":11383,"duration_ms":105015,"temperature":0.7,"pith_summary":"The paper proposes Cardioformer, an ECG classifier that splits each lead into patches at many time scales, refines them within and across scales using two stages of self-attention, and feeds the fused tokens through residual blocks. The central claim is that this multi-granularity design captures both local waveform morphology and long-range temporal context, which the four comparison transformers do not fully combine. On subject-independent splits of PTB, PTB-XL, and MIMIC-IV, the paper reports that Cardioformer achieves the highest area under the ROC curve (AUROC) among the five models (95.59, 89.99, and 96.34), and the best F1 on MIMIC-IV. It also claims that a model trained only on MIMIC-IV transfers to PTB-XL with 68.41% AUROC and to PTB with 49.18% AUROC, which it reads as evidence of cross-dataset robustness.","feed_headline":"Cardioformer beats four transformer baselines on three ECG datasets","feed_subtitle":"Multi-scale patching plus two-stage attention lifts AUROC to 96.34% and transfers across datasets.","key_machinery":"The load-bearing mechanism is the two-stage multi-granularity self-attention with router tokens, supported by cross-channel patch embedding and residual blocks. A router token is a per-scale summary vector that carries that granularity's state into the cross-scale attention stage. ECG leads are split into non-overlapping patches whose lengths range from 2 to 32 time steps; each scale's patches are embedded through 1x1 convolutional residual blocks, and a fixed positional embedding plus a learnable granularity embedding marks each token's place in time and scale. Intra-granularity attention refines patch and router tokens within each scale; inter-granularity attention updates each scale's router against the concatenation of all routers, reducing the self-attention cost from quadratic in the total number of patches to $\\mathcal{O}(nD^2)$ with $n$ granularities and embedding dimension $D$. The residual blocks act as the local-feature extractor that preserves morphological detail before the transformer's global stages.","core_discovery":"In the paper's own account, the discovery is that combining cross-channel multi-granularity patching with hierarchical residual learning and a two-stage self-attention mechanism lets a single ECG model beat four time-series transformers on standard benchmarks. The input signal is divided into non-overlapping patches at granularities from 2 to 32 time steps; each scale is embedded with residual blocks, augmented, and given a router token. Intra-granularity attention refines the patches and router inside each scale; inter-granularity attention then lets the routers exchange information across scales. The refined representations are concatenated and classified. The paper reports that this yields the highest AUROC on all three datasets and the best F1 on MIMIC-IV, with the strongest within-dataset result at 96.34% AUROC on MIMIC-IV.","pith_inferences":["An ablation that compares the full multi-granularity patch list against a single patch length would isolate whether the reported gains come from scale fusion or simply from more parameters.","The PTB cross-dataset AUROC of 49.18% sits at the chance level for a binary task; a rerun with PTB channels restricted to the same 12 leads as MIMIC-IV and with labels mapped into the four-class scheme would tell whether the transfer evidence survives a fair protocol.","The patch-length list is fixed by the authors; an adaptive choice of granularities based on heart rate or signal quality is a natural next experiment."],"forward_implications":["If the AUROC rankings are reproduced, Cardioformer becomes a reference point for ECG transformer comparisons on these three benchmarks.","Subject-independent splits mean the reported ranking is not an artifact of patient overlap; new patients recorded with the same protocols should see similar relative performance.","The PTB-XL transfer result (68.41% AUROC from a MIMIC-IV-only model) suggests a single pretrained model could serve smaller sites that cannot collect their own labeled ECGs.","Because inter-granularity attention cost grows with the number of granularities rather than the total number of patches, the architecture can be applied to longer recordings without quadratic attention blow-up."],"supporting_citations":[{"why":"Provides the patch-based Transformer baseline whose AUROC Cardioformer must beat in the comparison.","marker":"[7]"},{"why":"Provides the efficient Transformer baseline used to test whether attention speed alone competes with multi-granularity features.","marker":"[8]"},{"why":"Defines the multi-granularity cross-channel patching and two-stage attention design that Cardioformer extends into a hybrid with residual blocks.","marker":"[9]"},{"why":"Serves as the standard Transformer baseline in the experiments, standing for canonical attention without multi-granularity patches.","marker":"[16]"},{"why":"Supplies the PTB benchmark used for the binary myocardial-infarction evaluation and the cross-dataset transfer target.","marker":"[19]"},{"why":"Supplies the PTB-XL five-class benchmark and the second cross-dataset transfer target.","marker":"[20]"},{"why":"Supplies the MIMIC-IV-ECG four-class benchmark and the training source for the transfer experiments.","marker":"[21]"}],"fun_headline_variants":["Cardioformer: multi-granularity patching beats four transformers","Cross-channel patches and two-stage attention lift ECG AUROC to 96.34%","Cardioformer's multi-scale tokens outperform on three ECG benchmarks","ECG hybrid model with cross-granularity attention wins on three datasets"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a model trained on MIMIC-IV's four-class, 12-lead heartbeats can be evaluated directly on PTB's two-class, 15-lead heartbeats and PTB-XL's five-class, 12-lead heartbeats without an explicit mapping of output labels or input channels.","fun_headline_variants_meta":{"raw":{"variants":["Cardioformer: multi-granularity patching beats four transformers","Cross-channel patches and two-stage attention lift ECG AUROC to 96.34%","Cardioformer's multi-scale tokens outperform on three ECG benchmarks","ECG hybrid model with cross-granularity attention wins on three datasets"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001202,"raw_usage":{"total_tokens":4973,"prompt_tokens":987,"completion_tokens":3986,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":603,"completion_tokens_details":{"reasoning_tokens":3905}},"tokens_in":603,"tokens_out":3986,"duration_ms":30524,"temperature":1.0,"reasoning_tokens":3905,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T23:04:42.391884+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Test the transfer claim by evaluating the MIMIC-IV-trained Cardioformer on PTB after truncating PTB from 15 leads to the 12 leads shared with MIMIC-IV and mapping PTB's binary labels into the four-class scheme; if AUROC remains at or below 50%, the cross-dataset result is an artifact of the mismatched protocol rather than evidence of generalization.","supporting_citations":[{"cited_title":"A time series is worth 64 words: Long-term forecasting with transformers","cited_arxiv_id":null,"evidence_quote":"Provides the patch-based Transformer baseline whose AUROC Cardioformer must beat in the comparison."},{"cited_title":"Reformer: The efficient transformer","cited_arxiv_id":null,"evidence_quote":"Provides the efficient Transformer baseline used to test whether attention speed alone competes with multi-granularity features."},{"cited_title":"iTransformer: Inverted transformers are effective for time series forecasting","cited_arxiv_id":null,"evidence_quote":"Serves as the standard Transformer baseline in the experiments, standing for canonical attention without multi-granularity patches."},{"cited_title":"PTB-XL, a large publicly available electrocardiography dataset (version 1.0.3)","cited_arxiv_id":null,"evidence_quote":"Supplies the PTB-XL five-class benchmark and the second cross-dataset transfer target."},{"cited_title":"Gow et al","cited_arxiv_id":null,"evidence_quote":"Supplies the MIMIC-IV-ECG four-class benchmark and the training source for the transfer experiments."}],"review_version":1}