Pith. sign in

REVIEW 4 major objections 7 minor 57 references

ECG classifier failures become design evidence: an offline LLM refines models from measurements and cases, then freezes a deployable predictor.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-31 15:24 UTC pith:VGSK36BO

load-bearing objection Real multi-dataset gains and a usable offline design loop, but the ablations undercut the claim that case-level evidence is what drives most of the 10% lift. the 4 major comments →

arxiv 2607.24419 v1 pith:VGSK36BO submitted 2026-07-27 cs.AI

Failures Reveal What Metrics Miss: An Evidence-Driven Agent for Recursive Refinement of ECG Classifiers

Jinliang Deng , Yiming Niu , Yibo Pan , Zhiqi Shao , Qin Luo , Yongxin Tong This is my paper
classification cs.AI
keywords ECG AnalysisTime-Series ClassificationLarge Language ModelsAutonomous Model RefinementEvidence GroundingLLM-as-DesignerFailure-driven revision
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Deep ECG classifiers still improve mainly when humans inspect failures and redesign the model. Metric-only AI designers can raise scores but rarely explain why a particular recording fails or what to change. RecursiveECG treats an LLM as an offline designer: it turns curated clinical ECG criteria into deterministic measurement functions, reviews failed cases against similar correct ones using waveforms, measurements, and model outputs, proposes code-level revisions, and keeps only those that improve validation under a fixed protocol. The final classifier is frozen—no LLM at deployment—and each accepted change is linked to the evidence that motivated it. On three public 12-lead benchmarks the method beats strong baselines by about 10% relative macro-F1 on average, with ablations showing that failure evidence and compiled measurements both matter.

Core claim

When ECG classifier refinement is driven by case-level, reference-backed measurements and contrastive failure review—not only aggregate metrics—an offline LLM designer can recursively produce stronger, deployable classifiers with an audit trail from each revision to the failures that justified it, yielding consistent gains across PTB-XL, Georgia, and CPSC2018.

What carries the argument

RecursiveECG’s evidence loop: Criteria-to-Measurement Compilation (curated criteria → validated deterministic measurement functions) plus Evidence-Grounded Failure Review (failed vs comparator cases with waveforms, measurements, and outputs) feeding executable classifier revisions under a fixed problem contract, with only evidence-supported, validation-improving updates retained.

Load-bearing premise

The method assumes that curated clinical criteria, once compiled into deterministic measurements, are trustworthy enough case-level evidence to diagnose real model flaws and guide useful redesign—not incomplete or noisy cues that can steer revisions the wrong way.

What would settle it

On the same fixed splits and problem contract, disable evidence-grounded failure review (or measurement execution) and check whether macro-F1 and the paper’s claimed average ~10% relative gain over the strongest baselines disappear or reverse across PTB-XL, Georgia, and CPSC2018.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • ECG model improvement can be automated offline while keeping inference as a frozen non-LLM classifier with a revision audit trail.
  • Aggregate validation metrics alone are insufficient design feedback for subtle ECG criteria; case-level measurements and comparators are needed to target revisions.
  • Architectures can evolve dataset-adaptively (e.g., lead-aware dual-branch attention) from failure evidence rather than from fixed hand-designed templates.
  • The same evidence-driven designer pattern is positioned to extend to other medical time-series where domain criteria can be compiled into executable measurements.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Labels with sparse support or brittle morphology (e.g., rare hypertrophy subtypes, some conduction blocks) may remain weak points unless failure selection and measurement libraries are specialized further.
  • The audit trail plus frozen predictor suggests a practical path for regulated model updates: promote only revisions that replay on cited failures and pass regression gates.
  • Compiling guidelines into deterministic tools may transfer to other criteria-heavy signals (EEG, PPG, polysomnography) where metric-only agents currently stall.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper presents RecursiveECG, an "LLM-as-Designer" framework in which an offline LLM agent iteratively refines a 12-lead ECG classifier. Two mechanisms anchor the design: Criteria-to-Measurement Compilation (CMC), which converts curated ECG criteria into validated deterministic measurement functions, and Evidence-Grounded Failure Review (EGFR), which contrasts failed validation cases against correctly classified comparators using waveforms, measurements, and model outputs to diagnose limitations and propose code-level revisions. Candidates are retrained and promoted only on validation improvement under a fixed problem contract; the final predictor is frozen, requires no LLM at inference, and carries an audit trail. On PTB-XL, Georgia, and CPSC2018, the system reports average relative macro-F1 gains of 10.0% over the strongest of seven baselines, with five-run means ± std, mechanism ablations, backbone sensitivity, transfer experiments, per-label breakdowns, and publicly released code.

Significance. If the results hold, this is a useful contribution at the intersection of agentic ML engineering and medical time series: a frozen, deployable classifier with a revision-to-evidence audit trail is a meaningful step beyond metric-only AutoML loops, and the CMC idea of compiling textual clinical criteria into deterministic, reference-backed, validated measurement functions has independent value. The experimental package is unusually complete for this genre: three public datasets with fixed splits, held-out test evaluation, five-run statistics that re-execute the entire design trajectory rather than just reseeding training, strong baselines spanning deep/pretrained/agentic paradigms, per-label F1 and AUC tables that include unfavorable rows (CPSC2018 RBBB, PTB-XL RVH/SEHYP), a refinement trajectory table, and released implementation. The negative-result reporting in Sec. B.4 is a genuine strength and raises confidence in the rest. That said, the significance is bounded by an attribution problem the paper's own ablations expose: the "evidence-driven" framing is load-bearing on only one of three datasets, and the comparison budget is asymmetric, so what is currently demonstrated is more

major comments (4)
  1. [§5.3 / Tab. 17] Tab. 17 (Sec. C.1) vs. the framing in the title, abstract, and §1: the causal claim that evidence-grounded failure review is what beats the baselines is supported on only one of three datasets. On Georgia, w/o EGFR (metric-driven refinement in the same loop) reaches 0.8735 vs full 0.8826 and best baseline MERL 0.8130 — i.e., ~87% of the gain over the strongest baseline persists with no failure review at all, and w/o Measurement Execution is effectively neutral (delta −0.0003). On CPSC2018, w/o EGFR retains ~63% of the gain (0.7616 vs 0.8027 vs 0.6931). Only on PTB-XL does removing EGFR drop the system below the best baseline (0.7040 < xLSTM-ECG 0.7250). The honest, data-supported claim is: 'the governed iterative LLM-driven search outperforms baselines on all three datasets; EGFR/CMC add further gains whose size varies, decisively on PTB-XL.' The title 'Failures Reveal What Metrics Miss'
  2. [§5.1 / §D.3] §5.1 and §D.3: the compute comparison is asymmetric and is the most plausible alternative explanation for the headline gaps. RecursiveECG consumes 5 iterations × k=3 candidates = 15 full architecture retrains per run, at ~44.9 min agent time per iteration (Tab. 20, D.3), while baselines receive only 'key hyperparameters independently tuned on the validation set' with no stated tuning budget. For an architecture-search claim, the standard control is a compute-matched search: e.g., random search or an evidence-free LLM agent given the same number of retrains. The w/o EGFR variant partially fills this role but is not analyzed as a budget-matched control, and Direct LLM Design uses a different (weaker) workflow. Please report development compute for all methods and add at least one budget-matched search baseline; without it, 'RecursiveECG outperforms X' conflates search budget with method on
  3. [§B.4 / Tab. 13 / Tab. 9] Tab. 13 and Sec. B.4 (CPSC2018 RBBB): the full system (F1 0.5556) is not only far below MERL (0.9084) but below every ablated variant, including w/o EGFR. Combined with Tab. 9 — which shows the CMC measurement functions produce heuristic/indeterminate outputs, a 3/16 abnormal burden on the PTB-XL NORM control, P-wave and axis heuristics flagged as unreliable on Georgia, and 100 Hz resolution limits — this directly challenges the assumption (§4.2–4.3) that compiled criteria are reliable enough evidence to steer revisions. The transparency of Sec. B.4 is commendable, but the proposed remedies (label-wise regression checks, failure-selection quotas) appear necessary for the method's soundness rather than optional future work. At minimum, quantify how often evidence-grounded revisions are accepted and later prove counterproductive at label level, and strengthen the promotion gate so that agg
  4. [§4.4 / Tab. 20] §4.4 and §A.4: the promotion rule selects candidates on validation improvement across 15 adaptive proposals per run, and the failure-review itself is validation-driven. Test is held out and used once, so the headline test numbers are not biased, but the paper should characterize validation-to-test transfer: report the validation score of the frozen predictor alongside test, and discuss how validation adaptivity (including label-wise thresholds ν tuned on validation) interacts with the large across-run variance observed for agentic runs (e.g., Qwen3.5-27B on PTB-XL: 0.7338±0.0415, roughly 5× the std of the best baseline). A validation/test gap trajectory across the five refinement rounds (extending Tab. 20) would address this cheaply.
minor comments (7)
  1. [Tab. 11] Tab. 11, PMI row: support is 2 test cases and RecursiveECG reports F1 0.9200 where all baselines score 0; presenting this in the main fine-grained table inflates the per-label narrative. The Sec. B.4 caveat for RVH/SEHYP should be extended to all support<20 rows, ideally flagged directly in the table.
  2. [Tab. 1] Tab. 1: define 'Impr (%)' in the caption (relative gain over the strongest non-RecursiveECG baseline per metric); also note the claim 'consistently achieves the best performance across all datasets and evaluation metrics' holds only when counting both RecursiveECG backbones jointly — e.g., on PTB-XL Hamming Acc the Qwen variant (0.9118) exceeds the DeepSeek variant (0.8929), which is fine but should be stated precisely.
  3. [§5.1] §5.1: specify the baseline hyperparameter tuning protocol (number of trials, search space, selection criterion). 'Key hyperparameters independently tuned' is not sufficient to assess comparison fairness.
  4. [§C.1] §C.1: clarify that w/o EGFR retains access to the CMC measurement functions — the variant name suggests full removal of evidence, but it is 'metric-driven refinement with measurement tools available,' which matters for interpreting Tab. 17.
  5. [Tab. 7] Tab. 7, Georgia row: the case-review trace notes 'extractor unavailable in this trace' — clarify what evidence EGFR actually used for Georgia and whether CMC measurements contributed at all there, given that w/o Measurement Execution is neutral on this dataset (Tab. 17).
  6. [Front matter / references] The ACM template is unmodified ('Conference acronym ’XX', 'Make sure to enter the correct conference title', placeholder DOI/ISBN). Also verify forward-dated references ([3] DeepSeek-V4, [7] Co-Scientist Nature 655, [29] Qwen3.5) resolve to citable artifacts.
  7. [Fig. 5 / Fig. 7 / Tab. 1] Fig. 5 legend and Fig. 7 dual-panel layout are hard to parse at print size; since Tab. 17 duplicates the ablation numbers, add a cross-reference in the Fig. 5 caption. Georgia UniTS AUC (0.7149, Tab. 1) is a strong outlier relative to its other metrics — worth a footnote confirming it is not an evaluation artifact.

Circularity Check

0 steps flagged

No significant circularity: empirical agentic refinement with held-out tests; gains are measured, not derived by construction from inputs.

full rationale

RecursiveECG is an empirical systems paper, not a first-principles derivation. The load-bearing claim is comparative test performance under fixed splits and a frozen predictor. Criteria-to-Measurement Compilation operationalizes external ECG references into deterministic functions; those functions are not fitted to the reported test metrics, and promotion requires executable validation improvement under a fixed problem contract with the test set inaccessible during refinement. Ablations, transfer, and label-level breakdowns are reported against external baselines rather than restating fitted quantities as predictions. Mild methodological risk (iterative validation-driven architecture search can overfit the validation pool) is ordinary ML selection bias, not definitional circularity: Eq.-level reductions of outputs to inputs, uniqueness theorems imported from the same authors, or fitted parameters renamed as predictions are absent. Self-citations are not load-bearing for the central empirical claim. Score 0 is therefore appropriate.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 3 invented entities

The central empirical claim rests on standard supervised multi-label ECG evaluation plus several workflow choices: that curated clinical criteria can be compiled into useful deterministic measurements; that validation-selected failures plus comparators diagnose fixable model limits; that LLM-proposed executable revisions under a locked contract are an adequate search space; and that a handful of free process knobs (candidate width, iteration cap, backbone, thresholds) do not dominate the reported gains. No new physical entities are postulated; the invented pieces are methodological modules.

free parameters (5)
  • candidate budget k = 3
    Number of executable revisions proposed per iteration; default k=3 chosen as accuracy–cost operating point (Tab. 19).
  • maximum refinement iterations = 5
    Hard stop on the design loop; default 5 iterations in main experiments (Tab. 4, Tab. 20).
  • label-wise decision thresholds ν = validation-tuned per label/dataset
    Binary decisions use validation-selected per-label thresholds; operating point affects F1/Hamming and can flip rare-label outcomes.
  • failure-review case budgets and strata = dataset-specific small fixed budgets
    Fixed small review quotas and influence-weighted strata (e.g., PTB-XL 5 bad/2 controls) shape which limitations the LLM sees (Sec. A.4, Tab. 7).
  • LLM backbone and sampling behavior = DeepSeek-V4-Pro default
    Design trajectory quality depends on the offline controller; DeepSeek-V4-Pro is default, Qwen3.5-27B is weaker (Tab. 18).
axioms (5)
  • domain assumption Held-out multi-label ECG benchmarks with fixed splits and no test leakage are a valid external measure of classifier quality.
    Problem formulation §3 and protocol §5.1/A.1 treat official or fixed splits and locked evaluation contracts as ground truth for promotion and final reporting.
  • domain assumption Standard ECG interpretation references can be operationalized as deterministic measurement functions that provide useful, non-speculative case evidence when indeterminate states are allowed.
    CMC §4.2 and A.3–A.5 bind measurements to AHA/ACCF/HRS-style references and accept heuristic/indeterminate outputs as development evidence, not clinical diagnosis.
  • ad hoc to paper Contrastive review of failed vs correctly classified cases with comparable evidence profiles identifies systematic classifier limitations rather than case idiosyncrasies.
    EGFR §4.3 defines comparator retrieval and batch-level diagnosis as the bridge from errors to revision hypotheses.
  • ad hoc to paper Executable code-level architecture/training edits proposed by an LLM, retained only on validation improvement plus regression checks, constitute a sufficient refinement operator over ECG classifiers.
    §4.4 and promotion rules in Tab. 4 make validation-gated program mutation the optimization mechanism.
  • standard math Aggregate and label-wise F1/AUC/Hamming on imbalanced multi-label tasks are appropriate objectives for comparing revisions.
    Standard multi-label metrics; Macro F1 emphasized due to imbalance (§5.1).
invented entities (3)
  • Criteria-to-Measurement Compilation (CMC) no independent evidence
    purpose: Turn curated ECG criteria into validated deterministic measurement functions used as case-level evidence.
    Method module introduced in §4.2; independent_evidence is partial via public datasets and stated validation reports, but not external clinical assay validation.
  • Evidence-Grounded Failure Review (EGFR) no independent evidence
    purpose: Select failures, build measurement-backed evidence, contrast with comparators, and emit a classifier limitation for revision.
    Core control loop in §4.3; value is shown mainly by ablations inside this paper.
  • RecursiveECG governed refinement contract / audit trail no independent evidence
    purpose: Keep label space, splits, metrics, and leakage constraints fixed while logging (limitation, revision, before/after scores).
    Process scaffold enabling offline designer claims and deploy-time freeze; not an external physical entity.

pith-pipeline@v1.2.0-grok45-kimik3 · 43301 in / 4085 out tokens · 99460 ms · 2026-07-31T15:24:11.637430+00:00 · methodology

0 comments
read the original abstract

Deep models have substantially advanced 12-lead ECG classification, yet their refinement still relies heavily on human experts to inspect failures and iteratively revise classifier designs. Recent LLM-based agents have demonstrated the potential for automated model design, but when guided only by aggregate performance metrics, they lack insight into why individual cases fail and how the classifier should be revised. We present RecursiveECG, an evidence-driven LLM-as-Designer framework in which an LLM serves as an offline model designer that refines ECG classifiers based on concrete failures and objective ECG evidence. To ground failure diagnosis in executable evidence, Criteria-to-Measurement Compilation converts curated ECG criteria into validated deterministic functions that produce reproducible, reference-backed measurements for individual ECGs. Building on these measurements, Evidence-Grounded Failure Review analyzes failed and comparator cases by jointly considering raw waveforms, measurements, and model outputs, enabling the LLM to diagnose classifier limitations and formulate targeted revisions. Candidate revisions are executed and re-evaluated under a fixed problem contract, and only evidence-supported updates are retained. The resulting predictor is frozen after refinement and requires no LLM inference during deployment, while an audit trail links each accepted revision to its supporting evidence. Across PTB-XL, Georgia, and CPSC2018, RecursiveECG consistently outperforms strong baselines, achieving an average relative improvement of 10.0%. Extensive ablation and transfer studies further validate the effectiveness of its evidence-grounded refinement process.

Figures

Figures reproduced from arXiv: 2607.24419 by Jinliang Deng, Qin Luo, Yibo Pan, Yiming Niu, Yongxin Tong, Zhiqi Shao.

Figure 1
Figure 1. Figure 1: Motivation example illustrating how ECG model failures can be analyzed using evidence for refinement. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Comparison of agentic paradigms for ECG analysis. [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Overview of RecursiveECG. It compiles ECG criteria into validated measurement functions, selects influential failure [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: The effectiveness–efficiency tradeoff on PTB-XL. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 6
Figure 6. Figure 6: Model design evolution on PTB-XL. 0 10 20 30 40 50 60 NST_ ISCA PAC STE SB AF +55.1% +46.7% +32.6% +24.1% +14.6% +9.1% 0 5 10 15 20 25 NST_ ISCA PAC STE SB AF +7.6% +5.3% +20.2% +7.6% +1.4% +1.2% PTB-XL CPSC2018 Georgia Absolute F1 gain Absolute AUC gain Absolute improvement over the strongest baseline (percentage points) [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 8
Figure 8. Figure 8: Dataset-adaptive architecture evolution. [PITH_FULL_IMAGE:figures/full_fig_p007_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Evidence-Grounded Failure Review of a WPW pseudo-infarction case. [PITH_FULL_IMAGE:figures/full_fig_p008_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Fine-grained F1 and AUC changes of RecursiveECG relative to the strongest baseline on PTB-XL. [PITH_FULL_IMAGE:figures/full_fig_p017_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: PTB-XL iterative refinement trajectory [PITH_FULL_IMAGE:figures/full_fig_p022_11.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

57 extracted references · 4 canonical work pages

  1. [1]

    Attia, Suraj Kapa, Francisco Lopez-Jimenez, Paul M

    Zachi I. Attia, Suraj Kapa, Francisco Lopez-Jimenez, Paul M. McKie, Dorothy J. Ladewig, Gaurav Satam, Patricia A. Pellikka, Maurice Enriquez-Sarano, Peter A. Noseworthy, Thomas M. Munger, Samuel J. Asirvatham, Christopher G. Scott, Rickey E. Carter, and Paul A. Friedman. 2019. Screening for Cardiac Contractile Dysfunction Using an Artificial Intelligence-...

    doi:10.1038/s41591-018-0240-2 2019

  2. [2]

    Bran, Sam Cox, Oliver Schilter, Carlo Baldassari, Andrew D

    Andres M. Bran, Sam Cox, Oliver Schilter, Carlo Baldassari, Andrew D. White, and Philippe Schwaller. 2024. Augmenting Large Language Models with Chemistry Tools.Nature Machine Intelligence6 (2024), 525–535. https://doi.org/10.1038/ s42256-024-00832-8

    2024

  3. [3]

    DeepSeek-AI. 2026. DeepSeek-V4: Towards Highly Efficient Million-Token Con- text Intelligence

    2026

  4. [4]

    Shanghua Gao, Teddy Koker, Owen Queen, Thomas Hartvigsen, Theodoros Tsiligkaridis, and Marinka Zitnik. 2024. UniTS: A Unified Multi-Task Time Series Model. InAdvances in Neural Information Processing Systems, Vol. 37

    2024

  5. [5]

    Tomer Golany, Gal Lavee, Shai Tejman Yarden, and Kira Radinsky. 2020. Im- proving ECG Classification Using Generative Adversarial Networks. InPro- ceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 13280–13285. https://doi.org/10.1609/aaai.v34i08.7037

    work page doi:10.1609/aaai.v34i08.7037 2020

  6. [6]

    Tomer Golany and Kira Radinsky. 2019. PGANs: Personalized Generative Ad- versarial Networks for ECG Synthesis to Improve Patient-Specific Deep ECG Classification. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 557–564. https://doi.org/10.1609/aaai.v33i01.3301557

    work page doi:10.1609/aaai.v33i01.3301557 2019

  7. [7]

    Juraj Gottweis, Wei-Hung Weng, Alexander Daryin, Tao Tu, Petar Sirkovic, Artiom Myaskovsky, Grzegorz Glowaty, Felix Weissenberger, Alessio Orlandi, Dan Popovici, et al. 2026. Accelerating scientific discovery with Co-Scientist. Nature655, 8122 (2026), 487–496. https://doi.org/10.1038/s41586-026-10644-y

    doi:10.1038/s41586-026-10644-y 2026

  8. [8]

    Nate Gruver, Marc Finzi, Shikai Qiu, and Andrew G Wilson. 2023. Large language models are zero-shot time series forecasters.Advances in neural information processing systems36 (2023), 19622–19635

    2023

  9. [9]

    Yile Gu, Yifan Xiong, Jonathan Mace, Yuting Jiang, Yigong Hu, Baris Kasikci, and Peng Cheng. 2025. Argos: Agentic Time-Series Anomaly Detection with Autonomous Rule Generation via Large Language Models.arXiv preprint arXiv:2501.14170(2025)

    Pith/arXiv arXiv 2025

  10. [10]

    William Hancock, Barbara J

    E. William Hancock, Barbara J. Deal, David M. Mirvis, Peter Okin, Paul Kligfield, Leonard S. Gettes, James J. Bailey, Rory Childers, Anton Gorgels, Mark Josephson, Jan A. Kors, Peter Macfarlane, Jay W. Mason, Olle Pahlm, Pentti M. Rautaharju, Borys Surawicz, Gerard van Herpen, Galen S. Wagner, and Hein Wellens. 2009. AHA/ACCF/HRS Recommendations for the S...

    2009

  11. [11]

    Hannun, Pranav Rajpurkar, Masoumeh Haghpanahi, Geoffrey H

    Awni Y. Hannun, Pranav Rajpurkar, Masoumeh Haghpanahi, Geoffrey H. Tison, Codie Bourn, Mintu P. Turakhia, and Andrew Y. Ng. 2019. Cardiologist-Level Arrhythmia Detection and Classification in Ambulatory Electrocardiograms Using a Deep Neural Network.Nature Medicine25, 1 (2019), 65–69. https: //doi.org/10.1038/s41591-018-0268-3

    doi:10.1038/s41591-018-0268-3 2019

  12. [12]

    Li, Emmanuel Candes, and Jure Leskovec

    Kexin Huang, Ying Jin, Ryan Li, Michael Y. Li, Emmanuel Candes, and Jure Leskovec. 2025. Automated Hypothesis Validation with Agentic Sequential Falsifications. InProceedings of the 42nd International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 267). PMLR, 25372– 25437. https://proceedings.mlr.press/v267/huang25n.html

    2025

  13. [13]

    Qian Huang, Jian Vora, Percy Liang, and Jure Leskovec. 2024. MLAgent- Bench: Evaluating Language Agents on Machine Learning Experimentation. InProceedings of the 41st International Conference on Machine Learning (Pro- ceedings of Machine Learning Research, Vol. 235). PMLR, 20271–20309. https: //proceedings.mlr.press/v235/huang24y.html

    2024

  14. [14]

    Jansen, Marc-Alexandre Côté, Tushar Khot, Erin Bransom, Bhavana Dalvi Mishra, Bodhisattwa Prasad Majumder, Oyvind Tafjord, and Peter Clark

    Peter A. Jansen, Marc-Alexandre Côté, Tushar Khot, Erin Bransom, Bhavana Dalvi Mishra, Bodhisattwa Prasad Majumder, Oyvind Tafjord, and Peter Clark. 2024. DiscoveryWorld: A Virtual Environment for Developing and Evaluating Auto- mated Scientific Discovery Agents. InAdvances in Neural Information Processing Systems, Vol. 37. Curran Associates, Inc., 10088–...

    2024

  15. [15]

    Zhang, Xiaoming Shi, Pin-Yu Chen, Yuxuan Liang, Yuan-Fang Li, Shirui Pan, and Qingsong Wen

    Ming Jin, Shiyu Wang, Lintao Ma, Zhixuan Chu, James Y. Zhang, Xiaoming Shi, Pin-Yu Chen, Yuxuan Liang, Yuan-Fang Li, Shirui Pan, and Qingsong Wen

  16. [16]

    Lei Kang, Xuanshuo Fu, Javier Vazquez-Corral, Ernest Valveny, and Dimosthenis Karatzas. 2025. xLSTM-ECG: Multi-label ECG Classification via Feature Fusion with xLSTM.arXiv preprint arXiv:2504.16101(2025)

    Pith/arXiv arXiv 2025

  17. [17]

    Dani Kiyasseh, Tingting Zhu, and David A. Clifton. 2021. CLOCS: Contrastive Learning of Cardiac Signals Across Space, Time, and Patients. InProceedings of the 38th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 139). PMLR, 5606–5615. https://proceedings.mlr.press/ v139/kiyasseh21a.html

    2021

  18. [18]

    Gettes, James J

    Paul Kligfield, Leonard S. Gettes, James J. Bailey, Rory Childers, Barbara J. Deal, E. William Hancock, Gerard van Herpen, Jan A. Kors, Peter Macfarlane, David M. Mirvis, Olle Pahlm, Pentti Rautaharju, Galen S. Wagner, Mark Josephson, Jay W. Mason, Peter Okin, Borys Surawicz, Hein Wellens, American Heart Association Electrocardiography and Arrhythmias Com...

    2007

  19. [19]

    Che Liu, Zhongwei Wan, Cheng Ouyang, Anand Shah, Wenjia Bai, and Rossella Arcucci. 2024. Zero-Shot ECG Classification with Multimodal Learning and Test- time Clinical Knowledge Enhancement. InInternational Conference on Machine Learning. PMLR, 31949–31963

    2024

  20. [20]

    Tenen- baum, Daniela Rus, Chuang Gan, and Wojciech Matusik

    Pingchuan Ma, Tsun-Hsuan Wang, Minghao Guo, Zhiqing Sun, Joshua B. Tenen- baum, Daniela Rus, Chuang Gan, and Wojciech Matusik. 2024. LLM and Sim- ulation as Bilevel Optimizers: A New Paradigm to Advance Physical Scientific Discovery. InProceedings of the 41st International Conference on Machine Learn- ing (Proceedings of Machine Learning Research, Vol. 23...

    2024

  21. [21]

    Yecheng Jason Ma, William Liang, Guanzhi Wang, De-An Huang, Osbert Bastani, Dinesh Jayaraman, Yuke Zhu, Linxi Fan, and Anima Anandkumar. 2024. Eureka: Human-Level Reward Design via Coding Large Language Models. InThe Twelfth International Conference on Learning Representations. https://openreview.net/ forum?id=IEduRUO55F

    2024

  22. [22]

    Mason, E

    Jay W. Mason, E. William Hancock, Leonard S. Gettes, James J. Bailey, Rory Childers, Barbara J. Deal, Mark Josephson, Paul Kligfield, Jan A. Kors, Peter Mac- farlane, Olle Pahlm, David M. Mirvis, Peter Okin, Pentti M. Rautaharju, Borys Surawicz, Gerard van Herpen, Galen S. Wagner, and Hein Wellens. 2007. Recom- mendations for the Standardization and Inter...

    2007

  23. [23]

    Yeongyeon Na, Minje Park, Yunwon Tae, and Sunghoon Joo. 2024. Guiding Masked Representation Learning to Capture Spatio-Temporal Relationship of Electrocardiogram. InThe Twelfth International Conference on Learning Represen- tations

    2024

  24. [24]

    Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam

    Yuqi Nie, Nam H. Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. 2023. A Time Series is Worth 64 Words: Long-term Forecasting with Transformers. In The Eleventh International Conference on Learning Representations

    2023

  25. [25]

    Alexander Novikov, Ngân V ˜u, Marvin Eisenberger, Emilien Dupont, Po-Sen Huang, Adam Zsolt Wagner, Sergey Shirobokov, Borislav Kozlovskii, Francisco J. R. Ruiz, Abbas Mehrabian, M. Pawan Kumar, Abigail See, Swarat Chaudhuri, George Holland, Alex Davies, Sebastian Nowozin, Pushmeet Kohli, and Matej Ba- log. 2025. AlphaEvolve: A Coding Agent for Scientific ...

    Pith/arXiv arXiv doi:10.48550/arxiv.2506.13131 2025

  26. [26]

    Jungwoo Oh, Gyubok Lee, Seongsu Bae, Joon-myoung Kwon, and Edward Choi

  27. [27]

    Junwoo Park, Kyudan Jung, Dohyun Lee, Hyuck Lee, Daehoon Gwak, ChaeHun Park, Jaegul Choo, and Jaewoong Cho. 2025. Delving into Large Language Models for Effective Time-Series Anomaly Detection. InAdvances in Neural Information Processing Systems, Vol. 38. https://proceedings.neurips.cc/paper_files/paper/ 2025/hash/950a34c7b107111570077078e1b5b8ef-Abstract...

    2025

  28. [28]

    Perez Alday, Annie Gu, Amit J

    Erick A. Perez Alday, Annie Gu, Amit J. Shah, Chad Robichaux, An-Kwok Ian Wong, Chengyu Liu, Feifei Liu, Ali Bahrami Rad, Andoni Elola, Salman Seyedi, Qiao Li, Ashish Sharma, Gari D. Clifford, and Matthew A. Reyna. 2020. Classifi- cation of 12-lead ecgs: the physionet/computing in cardiology challenge 2020. 41, 12 (2020), 124003

    2020

  29. [29]

    Qwen Team. 2026. Qwen3.5: Towards Native Multimodal Agents. https: //qwen.ai/blog?id=qwen3.5

    2026

  30. [30]

    Rautaharju, Borys Surawicz, Leonard S

    Pentti M. Rautaharju, Borys Surawicz, Leonard S. Gettes, James J. Bailey, Rory Childers, Barbara J. Deal, Anton Gorgels, E. William Hancock, Mark Josephson, Paul Kligfield, Jan A. Kors, Peter Macfarlane, Jay W. Mason, David M. Mirvis, Peter Okin, Olle Pahlm, Gerard van Herpen, Galen S. Wagner, and Hein Wellens

  31. [31]

    Ribeiro, Manoel Horta Ribeiro, Gabriela M

    Antônio H. Ribeiro, Manoel Horta Ribeiro, Gabriela M. M. Paixão, Derick M. Oliveira, Paulo R. Gomes, Jéssica A. Canazart, Milton P. S. Ferreira, Carl R. Anders- son, Peter W. Macfarlane, Wagner Meira Jr., Thomas B. Schön, and Antonio Luiz P. Ribeiro. 2020. Automatic Diagnosis of the 12-Lead ECG Using a Deep Neural Network.Nature Communications11 (2020), 1...

    doi:10.1038/s41467- 2020

  32. [32]

    Pawan Kumar, Emilien Dupont, Francisco J

    Bernardino Romera-Paredes, Mohammadamin Barekatain, Alexander Novikov, Matej Balog, M. Pawan Kumar, Emilien Dupont, Francisco J. R. Ruiz, Jordan S. Ellenberg, Pengming Wang, Omar Fawzi, Pushmeet Kohli, and Alhussein Fawzi

  33. [33]

    Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. Toolformer: Language models can teach themselves to use tools.Advances in neural information processing systems36 (2023), 68539–68551

    2023

  34. [34]

    Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: Language Agents with Verbal Reinforcement Learning. InAdvances in Neural Information Processing Systems, Vol. 36

    2023

  35. [35]

    Parshin Shojaee, Kazem Meidani, Shashank Gupta, Amir Barati Farimani, and Chandan K. Reddy. 2025. LLM-SR: Scientific Equation Discovery via Program- ming with Large Language Models. InThe Thirteenth International Conference on Learning Representations. https://proceedings.iclr.cc/paper_files/paper/2025/ hash/28df8e730c054c5331855fd4d5403ba9-Abstract-Confe...

    2025

  36. [36]

    https://doi.org/10.1038/s41586-023-06924-6

    Mathematical Discoveries from Program Search with Large Language Models.Nature625 (2024), 468–475. https://doi.org/10.1038/s41586-023-06924-6

    doi:10.1038/s41586-023-06924-6 2024

  37. [37]

    Nils Strodthoff, Patrick Wagner, Tobias Schaeffter, and Wojciech Samek. 2021. Deep Learning for ECG Analysis: Benchmarks and Insights from PTB-XL.IEEE Journal of Biomedical and Health Informatics25, 5 (2021), 1519–1528. https: //doi.org/10.1109/JBHI.2020.3022989

    arXiv 2021

  38. [38]

    Deal, Leonard S

    Borys Surawicz, Rory Childers, Barbara J. Deal, Leonard S. Gettes, James J. Bailey, Anton Gorgels, E. William Hancock, Mark Josephson, Paul Kligfield, Jan A. Kors, Peter Macfarlane, Jay W. Mason, David M. Mirvis, Peter Okin, Olle Pahlm, Pentti M. Rautaharju, Gerard van Herpen, Galen S. Wagner, and Hein Wellens. 2009. AHA/ACCF/HRS Recommendations for the S...

    2009

  39. [39]

    Xiaoyu Tao, Yuchong Wu, Mingyue Cheng, Ze Guo, and Tian Gao. 2026. Anoma- Mind: Agentic Time Series Anomaly Detection with Tool-Augmented Reasoning. arXiv preprint arXiv:2602.13807(2026). https://doi.org/10.48550/arXiv.2602.13807

    Pith/arXiv arXiv doi:10.48550/arxiv.2602.13807 2026

  40. [40]

    Sprueill, Carl Edwards, Khushbu Agarwal, Mariefel V

    Henry W. Sprueill, Carl Edwards, Khushbu Agarwal, Mariefel V. Olarte, Udishnu Sanyal, Conrad Johnston, Hongbin Liu, Heng Ji, and Sutanay Choudhury. 2024. CHEMREASONER: Heuristic Search over a Large Language Model’s Knowledge Space Using Quantum-Chemical Feedback. InProceedings of the 41st Interna- tional Conference on Machine Learning (Proceedings of Mach...

    2024

  41. [41]

    Patrick Wagner, Nils Strodthoff, Ralf-Dieter Bousseljot, Dieter Kreiseler, Fatima I Lunze, Wojciech Samek, and Tobias Schaeffter. 2020. PTB-XL, a large publicly available electrocardiography dataset.Scientific data7, 1 (2020), 154

    2020

  42. [42]

    Zhongwei Wan, Che Liu, Xin Wang, Chaofan Tao, Hui Shen, Jing Xiong, Rossella Arcucci, Huaxiu Yao, and Mi Zhang. 2025. MEIT: Multimodal electrocar- diogram instruction tuning on large language models for report generation. InFindings of the association for computational linguistics: ACL 2025. Associ- ation for Computational Linguistics, Vienna, Austria, 14...

    work page doi:10.18653/v1/2025.findings-acl.749 2025

  43. [43]

    Chengsen Wang, Qi Qi, Jingyu Wang, Haifeng Sun, Zirui Zhuang, Jinming Wu, Lei Zhang, and Jianxin Liao. 2025. ChatTime: A Unified Multimodal Time Series Foundation Model Bridging Numerical and Textual Data. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 39. 12694–12702. https://doi.org/ 10.1609/aaai.v39i12.33384

    work page doi:10.1609/aaai.v39i12.33384 2025

  44. [44]

    Wagner, Peter Macfarlane, Hein Wellens, Mark Josephson, Anton Gorgels, David M

    Galen S. Wagner, Peter Macfarlane, Hein Wellens, Mark Josephson, Anton Gorgels, David M. Mirvis, Olle Pahlm, Borys Surawicz, Paul Kligfield, Rory Childers, Leonard S. Gettes, James J. Bailey, Barbara J. Deal, E. William Hancock, Jan A. Kors, Jay W. Mason, Peter Okin, Pentti M. Rautaharju, and Gerard van Herpen. 2009. AHA/ACCF/HRS Recommendations for the S...

    2009

  45. [45]

    Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. 2023. TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis. InThe Eleventh International Conference on Learning Representations

    2023

  46. [46]

    Yu, Yue Zhao, and Kai Shu

    Xiongxiao Xu, Haoran Wang, Yueqing Liang, Philip S. Yu, Yue Zhao, and Kai Shu. 2026. Can Multimodal LLMs Perform Time Series Anomaly Detection?. In Proceedings of the ACM Web Conference 2026. 5392–5403. https://doi.org/10.1145/ 3774904.3792376

    arXiv 2026

  47. [47]

    Le, Denny Zhou, and Xinyun Chen

    Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V. Le, Denny Zhou, and Xinyun Chen. 2024. Large Language Models as Optimizers. InThe Twelfth International Conference on Learning Representations

    2024

  48. [48]

    Fuying Wang, Jiacheng Xu, and Lequan Yu. 2025. From Token to Rhythm: A Multi-Scale Approach for ECG-Language Pretraining. InInternational Conference on Machine Learning. PMLR, 65059–65074

    2025

  49. [49]

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. ReAct: Synergizing Reasoning and Acting in Language Models. InThe Eleventh International Conference on Learning Representations. https://openreview.net/forum?id=WE_vluYUL-X

    2023

  50. [50]

    Zhihan Yue, Yujing Wang, Juanyong Duan, Tianmeng Yang, Congrui Huang, Yunhai Tong, and Bixiong Xu. 2022. TS2Vec: Towards Universal Representation of Time Series.Proceedings of the AAAI Conference on Artificial Intelligence36, 8 (2022), 8980–8987. https://doi.org/10.1609/aaai.v36i8.20881

    doi:10.1609/aaai.v36i8.20881 2022

  51. [51]

    Rushuang Zhou, Yuanting Zhang, and Yining Dong. 2025. H-Tuning: Toward Low- Cost and Efficient ECG-based Cardiovascular Disease Detection with Pre-Trained Models. InInternational Conference on Machine Learning. PMLR, 79548–79569

    2025

  52. [52]

    Tiankai Yang, Junjun Liu, Michael Siu, Jiahang Wang, Zhuangzhuang Qian, Chanjuan Song, Cheng Cheng, Xiyang Hu, and Yue Zhao. 2025. AD-AGENT: A Multi-agent Framework for End-to-end Anomaly Detection. InProceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association ...

    2025

  53. [53]

    Hongling Zhu, Cheng Cheng, Hang Yin, Xingyi Li, Ping Zuo, Jia Ding, Fan Lin, Jingyi Wang, Beitong Zhou, Yonge Li, Shouxing Hu, Yulong Xiong, Binran Wang, Guohua Wan, Xiaoyun Yang, and Ye Yuan. 2020. Automatic Multilabel Electrocardiogram Diagnosis of Heart Rhythm or Conduction Abnormalities with Deep Learning: A Cohort Study.The Lancet Digital Health2, 7 ...

    arXiv 2020

  54. [56]

    Tian Zhou, Peisong Niu, Liang Sun, Rong Jin, et al . 2023. One fits all: Power general time series analysis by pretrained lm.Advances in neural information processing systems36 (2023), 43322–43355

    2023

  55. [2009]

    AHA/ACCF/HRS Recommendations for the Standardization and Interpre- tation of the Electrocardiogram: Part IV: The ST Segment, T and U Waves, and the QT Interval: A Scientific Statement From the American Heart Association Conference acronym ’XX, June 03–05, 2018, Woodstock, NY Deng et al. Electrocardiography and Arrhythmias Committee, Council on Clinical Ca...

    2018

  56. [2023]

    InAdvances in Neural Information Processing Systems, Datasets and Benchmarks Track, Vol

    ECG-QA: A Comprehensive Question Answering Dataset Combined With Electrocardiogram. InAdvances in Neural Information Processing Systems, Datasets and Benchmarks Track, Vol. 36. 66277–66288

  57. [2024]

    InThe Twelfth International Conference on Learning Representations

    Time-LLM: Time Series Forecasting by Reprogramming Large Language Models. InThe Twelfth International Conference on Learning Representations. https://openreview.net/forum?id=Unb5CVPtae