REVIEW 3 major objections 5 minor 35 references
Digitizing Paper ECGs at Scale: An Open-Source Algorithm for Clinical Research
T0 review · 3 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash
Pith's one-line read A fully automated, open-source pipeline converts paper ECGs into digital signals, achieving 19.65 dB SNR on hospital scans and beating prior methods on every test category.
desk verdict Open-source ECG digitizer validated zero-shot on the 2024 PhysioNet challenge, outperforming all prior methods; the headline SNR is slightly optimistic but the core contribution is real. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is a five-module pipeline that turns image geometry into signal reconstruction. The key geometric insight is an angle-angle domain derived from a double Hough transform: each set of parallel gridlines collapses to a single point, making perspective correction robust under noise and glare. Signal reconstruction then reduces to graph matching: connected components of the segmented trace are subdivided by a 'snipping' algorithm and re-joined by solving a linear sum assignment problem (a modified Jonker-Volgenant algorithm), stitching fragments into continuous leads. A second, lighter U-Net finds lead labels (III, aVR, etc.) and best-matches them to a template layout,
What would settle it
Run the released pipeline on a fresh set of paper ECGs from another hospital—say 500 digital ECGs printed to thermal paper at 50 mm/s, then scanned and photographed with consumer phones—and compare each reconstructed lead against the original digital trace; if the scanner mean SNR does not approach the reported 19.65 dB or any phone category falls at or below 0 dB (the all-zeros baseline), the central performance and generalization claims would be contradicted.
Extended reading notes
Core claim
The central claim is that ECG digitization needs no manual intervention or domain-specific fine-tuning: a fully automated framework, trained only on synthetic 3×4-layout 12-lead images, can reconstruct clinically usable time-series from flatbed scans, phone photos, and degraded paper. On the Emory Paper Digitization ECG Dataset it outperforms all previously published methods in every category and is the first to exceed the all-zeros baseline (0 dB SNR) on mobile photos and stained, wrinkled, or deteriorated paper; on a real-world hospital dataset it reports a mean SNR of 19.65 dB for scans. The complete code, synthetic training set, and paired clinical dataset are released, so the claims are
Load-bearing premise
Everything depends on the segmentation and layout networks, trained only on synthetic Brazilian 12-lead ECGs in a 3×4 layout, generalizing to real scans and photos from other countries, devices, and paper degradation modes without fine-tuning; the zero-shot benchmark result supports this for that dataset, but the generalization mechanism is unanalyzed and other archives are untested.
Editorial extensions
If this is right
- Retrospective ECG archives can be converted at scale: the pipeline runs unattended in about 5.9 seconds per image on a GPU, so a hospital's backlog of paper-only ECGs can become digital time-series for AI training and research.
- Phone-captured ECGs, including photos of stained or wrinkled paper, yield positive-SNR reconstructions, opening AI-based ECG interpretation to settings where only a camera and paper records are available.
- The complete release—code, synthetic training set, and paired clinical dataset—lets other groups reproduce, benchmark, and extend the method without access to private data.
- Training on synthetically augmented images from a different country and layout transfers zero-shot to real hospital scans, supporting the use of synthetic data when real annotated medical images are scarce.
Reading between the lines
- The per-lead NaN mechanism could be repurposed as a confidence score: downstream AI models could weight each lead by whether it was fully digitized, turning the algorithm's failure mode into a safety feature for clinical deployment.
- Because limb leads obey the Einthoven relation (I + III = II), missing segments—e.g., lead III in the paper's example—could be imputed and self-consistency-checked, a cheap improvement the authors flag but do not implement.
- The perspective-correction and grid-calibration steps are not ECG-specific: any gridded paper record with parallel line families (chart recorders, scrolled waveforms) could likely be digitized with the same angle-angle and autocorrelation machinery, a testable extension beyond electrocardiography.
- The modularity invites an end-to-end learned replacement for the snipping and matching stages; overlapping signals break the rule-based approach, so a differentiable trace-reconstruction network trained on the released synthetic dataset would be a concrete next experiment.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents an open-source, fully automated pipeline for digitizing paper ECGs, consisting of five modules: semantic segmentation, perspective correction, grid-size extraction, layout identification, and segmentation-to-trace conversion. The method is trained exclusively on synthetic images generated from Brazilian CODE15% ECGs and then evaluated prospectively on 1,596 images collected at Akershus University Hospital and zero-shot on the 35,595-image Emory Paper Digitization ECG Database via evaluation by the original challenge organizers. The authors report a mean SNR of 19.65 dB on scanner images at Ahus and positive SNR in all Emory subcategories, outperforming previously published methods.
Significance. If the reported results are robust, this is a substantial contribution: it is a rare fully automated, open-source ECG digitization system validated on a large prospective clinical dataset and on a hidden external test set with minimal circularity. The synthetic-to-real zero-shot generalization is noteworthy, as is the release of the training data, the clinical dataset, and the source code. The positive SNR on mobile photos and deteriorated papers would be a first. The main reservations concern the transparency of the SNR metric and the handling of failed samples, both of which are fixable in revision.
major comments (3)
- [Validation metrics / Ahus Paper Digitization ECG Database] The headline SNR of 19.65 dB is computed after an optimal horizontal shift of up to 100 ms and vertical zero-centering. Since 100 ms is comparable to a full QRS complex, the shift can absorb real temporal misalignment; the reported value is therefore an upper bound on raw reconstruction fidelity. The mean shifts are reported (0.86 ms for scans), but the unshifted SNR is not. Please report the SNR without any horizontal alignment, or at least the shift distribution and the zero-shift SNR, so the reader can assess temporal accuracy. The comparative Emory results are less affected because all methods are evaluated with the same shifted metric, but the absolute claim needs this qualifier.
- [Ahus Paper Digitization ECG Database (NaN handling)] The description of failed samples is contradictory: "an average of 0.21% of samples were set to NaN" is followed by "0.15% of samples were set to zero for the scanned paper." Specify precisely how NaN samples enter the SNR/RMSE/correlation denominators. If NaN leads are excluded from both numerator and denominator, the metrics are computed on a non-random subset and will be optimistically biased; if converted to zero, they are penalized. The 0.21% rate is small, but the ambiguity prevents reproduction of the exact numbers. Also clarify whether the percentages are per sample or per lead.
- [Emory Paper Digitization ECG Database (Table 2)] The caption states "Negative scores are not shown." This is problematic for a comparison table: a zero baseline (all-zero signal) achieves 0 dB, so negative scores of prior methods are informative. If a dash means the method was not applied or reported, say so; if it means a negative value, the table is visually misleading. Please report all numeric scores (or clearly mark "not reported") so the "outperforms all subcategories" claim is verifiable. As written, the table cannot be fully audited.
minor comments (5)
- [Figure 1] The caption contains a typo: "T op" should be "Top."
- [Abstract and text] "W e" appears in the abstract and elsewhere; should be "We."
- [Results and Methods] Stray formatting artifacts: "In this section*" and "The following section*s" should be cleaned up.
- [Figure 11] The caption says "The left pane shows the template" but the template is in the right pane; correct the reference.
- [Algorithm 1] λ is set to 0.5 with no sensitivity analysis or discussion; a brief note on its effect on layout matching would be helpful.
Circularity Check
No significant circularity: training data, hidden test set, and prospective clinical validation keep the core claim independent of its inputs.
full rationale
The paper's central derivation chain is not circular. The segmentation and layout networks are trained exclusively on synthetic images generated from CODE15% ECGs (Training section: 'The initial segmentation model is trained using synthetic ECG images generated using real ECGs from the CODE15% ECG dataset'), and the authors state 'All development was done on synthetic data, and clinical data were only used in the prospective testing.' The Ahus validation set was collected prospectively at Akershus University Hospital, and the Emory dataset was a hidden test set evaluated by the original challenge organizers, with the authors thanking 'James Weigle and Matthew Reyna for kindly evaluating our framework on the Emory Paper Digitization ECG Dataset.' There is no fitting of model parameters to the reported SNR, RMSE, or correlation values. The shifted-SNR evaluation metric (horizontal alignment up to 100 ms, vertical zero-centering) is an evaluation convention also used by previous work, not a fitted input disguised as a prediction; it is disclosed in the Validation metrics section. The ambiguity in NaN handling is a reproducibility concern, not a circularity concern. The only self-citations are references to the authors' prior challenge entry [20], used as a comparison in Table 2 and related work, and to their own data-infrastructure papers [23], [24]; these are not load-bearing for the central claim that the new open-source algorithm outperforms prior methods. No self-definitional, fitted-input-called-prediction, self-citation-chain, uniqueness-imported, or ansatz-smuggling circularity is present.
Assumptions & free parameters
free parameters (5)
- λ (layout matching weight) =
0.5
- Backward connection penalty multiplier =
2
- Maximum alignment shift in SNR evaluation =
100 ms
- Hough transform resolution parameters =
Two-pass low/high resolution, range [−π/4, 3π/4]
- NaN assignment criteria =
Not specified
assumptions (5)
- domain assumption Synthetic training data (generated from CODE15% and ECG-Image-Kit) is sufficiently representative of real-world paper ECGs from different regions, devices, and degradation modes.
- domain assumption ECG paper grid consists of major lines spaced 5d and minor lines spaced d, where d is the unknown pixel spacing.
- domain assumption The hospital's ground-truth time series correctly corresponds to the printed ECG waveform.
- standard math Parallel sets of grid lines map to lines in the Hough angle-radius domain, and the angle-angle transform reliably identifies the perspective.
- domain assumption SNR computed with alignment shifts and zero-centering is a meaningful proxy for clinical utility of digitized ECGs.
Cite this review
Pith. "Pith review of Digitizing Paper ECGs at Scale: An Open-Source Algorithm for Clinical Research." pith.science (2026). https://pith.science/paper/3RMIIJH2
@misc{pith2026251019590,
author = {Pith},
title = {Pith review of: Digitizing Paper ECGs at Scale: An Open-Source Algorithm for Clinical Research},
year = {2026},
howpublished = {\url{https://pith.science/paper/3RMIIJH2}},
note = {Machine review of arXiv:2510.19590}
}
read the original abstract
Millions of clinical ECGs exist only as paper scans, making them unusable for modern automated diagnostics. We introduce a fully automated, modular framework that converts scanned or photographed ECGs into digital signals, suitable for both clinical and research applications. The framework is validated on 37,191 ECG images with 1,596 collected at Akershus University Hospital, where the algorithm obtains a mean signal-to-noise ratio of 19.65 dB on scanned papers with common artifacts. It is further evaluated on the Emory Paper Digitization ECG Dataset, comprising 35,595 images, including images with perspective distortion, wrinkles, and stains. The model improves on the state-of-the-art in all subcategories. The full software is released as open-source, promoting reproducibility and further development. We hope the software will contribute to unlocking retrospective ECG archives and democratize access to AI-driven diagnostics.
Figures
Figures from the paper (10 more)
Reference graph
Works this paper leans on
-
[1]
Retrieved July 2025 from https://www.who.int/news-room/fact- sheets/detail/cardiovascular-diseases-(cvds), 2022
World Health Organization,Cardiovascular diseases (cvds), WHO Fact Sheet, Estimated 17.9 million deaths in 2019 (32% of global deaths). Retrieved July 2025 from https://www.who.int/news-room/fact- sheets/detail/cardiovascular-diseases-(cvds), 2022
2019
-
[2]
Deep learning in ECG diagnosis: A review,
X. Liu, H. Wang, Z. Li, and L. Qin, “Deep learning in ECG diagnosis: A review,”Knowledge-Based Sys- tems, vol. 227, Sep. 2021,doi:10.1016/j.knosys. 2021.107187
arXiv 2021
-
[3]
Deep learning and electrocardio- graphy: Systematic review of current techniques in cardiovascular disease diagnosis and management,
Z. Wu and C. Guo, “Deep learning and electrocardio- graphy: Systematic review of current techniques in cardiovascular disease diagnosis and management,” BioMedical Engineering OnLine, vol. 24, no. 1, 2025
2025
-
[4]
R. Jiang et al., “Deep Learning–Augmented ECG Analysis for Screening and Genotype Prediction of Congenital Long QT Syndrome,”JAMA Cardiology, vol. 9, no. 4, Apr. 2024,doi:10.1001/jamacardio. 2024.0039
arXiv 2024
-
[5]
Automatic multilabel electrocardio- gram diagnosis of heart rhythm or conduction ab- normalities with deep learning: A cohort study,
H. Zhu et al., “Automatic multilabel electrocardio- gram diagnosis of heart rhythm or conduction ab- normalities with deep learning: A cohort study,” En- glish,The Lancet Digital Health, vol. 2, no. 7, Jul. 2020, Publisher: Elsevier,doi:10 . 1016 / S2589 - 7500(20)30107-2
2020
-
[6]
Automatic diagnosis of the 12- lead ecg using a deep neural network,
A. H. Ribeiro et al., “Automatic diagnosis of the 12- lead ecg using a deep neural network,”Nature com- munications, vol. 11, no. 1, 2020
2020
-
[7]
Machine learning for ECG di- agnosis and risk stratification of occlusion myocar- dial infarction,
S. S. Al-Zaiti et al., “Machine learning for ECG di- agnosis and risk stratification of occlusion myocar- dial infarction,” en,Nature Medicine, vol. 29, no. 7, Jul. 2023, Publisher: Nature Publishing Group,doi: 10.1038/s41591-023-02396-3 9
-
[8]
A deep learning algorithm to de- tect anaemia with ECGs: A retrospective, multicen- tre study,
J.-m. Kwon et al., “A deep learning algorithm to de- tect anaemia with ECGs: A retrospective, multicen- tre study,” English,The Lancet Digital Health, vol. 2, no. 7, Jul. 2020, Publisher: Elsevier,doi:10.1016/ S2589-7500(20)30108-4
2020
Show all 35 references
-
[9]
Electrocardiographic deep learn- ingforpredictingpost-proceduralmortality:Amodel development and validation study,
D. Ouyang et al., “Electrocardiographic deep learn- ingforpredictingpost-proceduralmortality:Amodel development and validation study,” English,The Lancet Digital Health, vol. 6, no. 1, Jan. 2024, Pub- lisher: Elsevier,doi:10 . 1016 / S2589 - 7500(23 ) 00220-0
2024
-
[10]
Age and sex estimation using ar- tificial intelligence from standard 12-lead ecgs,
Z. I. Attia et al., “Age and sex estimation using ar- tificial intelligence from standard 12-lead ecgs,”Cir- culation: Arrhythmia and Electrophysiology, vol. 12, no. 9, 2019
2019
-
[11]
ECGScan: A method for conversion of pa- per electrocardiographic printouts to digital electro- cardiographic files,
F. Badilini, T. Erdem, W. Zareba, and A. J. Moss, “ECGScan: A method for conversion of pa- per electrocardiographic printouts to digital electro- cardiographic files,” eng,Journal of Electrocardiol- ogy, vol. 38, no. 4, Oct. 2005,doi:10 . 1016 / j . jelectrocard.2005.04.003
2005
-
[12]
Novel Tool for Complete Digitization of Paper Electrocardiog- raphy Data,
L. Ravichandran, C. Harless, A. J. Shah, C. A. Wick, J. H. Mcclellan, and S. Tridandapani, “Novel Tool for Complete Digitization of Paper Electrocardiog- raphy Data,” eng,IEEE journal of translational en- gineering in health and medicine, vol. 1, 2013,doi: 10.1109/JTEHM.2013.2262024
2013
-
[13]
High Precision Digitization of Paper-Based ECG Records: A Step Toward Machine learning,
“High Precision Digitization of Paper-Based ECG Records: A Step Toward Machine learning,”IEEE Journal of Translational Engineering in Health and Medicine, vol. 7, Nov. 2019,doi:10.1109/JTEHM. 2019.2949784
2019
-
[14]
ECG Paper Record Digitization and Diagnosis Using Deep Learning,
S. Mishra et al., “ECG Paper Record Digitization and Diagnosis Using Deep Learning,” eng,Journal of Medical and Biological Engineering, vol. 41, no. 4, 2021,doi:10.1007/s40846-021-00632-0
2021 doi
-
[15]
J. D. Fortune, N. E. Coppa, K. T. Haq, H. Patel, and L.G.Tereshchenko,Digitizing ECG image: New fully automated method and open-source software code, en, Jul. 2021,doi:10.1101/2021.07.13.21260461
2021 doi
-
[16]
A fully-automated paper ECG digiti- sation algorithm using deep learning,
H. Wu et al., “A fully-automated paper ECG digiti- sation algorithm using deep learning,”Scientific Re- ports, vol. 12, Dec. 2022,doi:10.1038/s41598-022- 25284-1
2022 doi
-
[17]
Digitization and Classification of ECG Images: The George B. Moody PhysioNet Challenge 2024,
M. A. Reyna et al., “Digitization and Classification of ECG Images: The George B. Moody PhysioNet Challenge 2024,” Dec. 2024,doi:10.22489/CinC. 2024.011
2024 doi
-
[18]
Combining Hough Transform and Deep Learning Approaches to Reconstruct ECG Signals From Print- outs,
F. H. Krones, T. J. Lyons, A. Mahdi, and B. Walker, “Combining Hough Transform and Deep Learning Approaches to Reconstruct ECG Signals From Print- outs,” Dec. 2024,doi:10.22489/CinC.2024.097
2024 doi
-
[19]
Segmentation-based Ex- traction of Key Components from ECG Images: A Framework for Precise Classification and Digitiza- tion,
H.-C. Yoon, D.-K. Kim, H.-S. Kim, W.-Y. Seo, C.-H. Heo, and S.-H. Kim, “Segmentation-based Ex- traction of Key Components from ECG Images: A Framework for Precise Classification and Digitiza- tion,” Dec. 2024,doi:10.22489/CinC.2024.227
2024 doi
-
[20]
ECG- DUaL: Pose-invariant Digitization of printed electro- cardiograms via U-net and Local edge detection,
E. Stenhede, B.-J. Singstad, and A. Ranjbar, “ECG- DUaL: Pose-invariant Digitization of printed electro- cardiograms via U-net and Local edge detection,” Dec. 2024,doi:10.22489/CinC.2024.262
2024 doi
-
[21]
High precision ECG digitiza- tion using artificial intelligence,
A. Demolder et al., “High precision ECG digitiza- tion using artificial intelligence,”Journal of Electro- cardiology, vol. 90, May 2025,doi:10 . 1016 / j . jelectrocard.2025.153900
2025
-
[22]
M. A. Reyna et al.,Ecg-image-database: A dataset of ecg images with real-world imaging and scanning artifacts; a foundation for computerized ecg image digitization and analysis, 2024, arXiv:2409 . 16612 [q-bio.QM]
2024
-
[23]
Data quality in health- care for the purpose of artificial intelligence: A case study on ecg digitalization,
A. Ranjbar and J. Ravn, “Data quality in health- care for the purpose of artificial intelligence: A case study on ecg digitalization,” inHealthcare Transfor- mation with Informatics and Artificial Intelligence, IOS Press, 2023
2023
-
[24]
Enabling clinical trials of artificial intelligence: In- frastructure for heart failure predictions.,
A. Ranjbar, J. Ravn, E. Ronningen, and O. Hanseth, “Enabling clinical trials of artificial intelligence: In- frastructure for heart failure predictions.,”Studies in Health Technology and Informatics, vol. 302, 2023
2023
-
[25]
Layout-Invariant U-Net Segmentation for Time-Series Reconstruction from Noisy ECG Im- ages,
S. Jammoul, A. Hassan, E. Zhang, P. Warrick, and J. Afilalo, “Layout-Invariant U-Net Segmentation for Time-Series Reconstruction from Noisy ECG Im- ages,” Dec. 2024,doi:10.22489/CinC.2024.481
2024 doi
-
[26]
Diagnos- tic accuracy of the PMcardio smartphone application for artificial intelligence–based interpretation of elec- trocardiograms in primary care (AMSTELHEART- 1),
J. C. L. Himmelreich and R. E. Harskamp, “Diagnos- tic accuracy of the PMcardio smartphone application for artificial intelligence–based interpretation of elec- trocardiograms in primary care (AMSTELHEART- 1),”Cardiovascular Digital Health Journal, vol. 4, no. 3, Jun. 2023,doi...
2023
- [27]
-
[28]
Medical image segmentation review: The success of u-net,
R. Azad et al., “Medical image segmentation review: The success of u-net,”IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024
2024
-
[29]
Use of the Hough trans- formation to detect lines and curves in pictures,
R. O. Duda and P. E. Hart, “Use of the Hough trans- formation to detect lines and curves in pictures,” Commun. ACM, vol. 15, no. 1, Jan. 1972,doi:10. 1145/361237.361242
1972
-
[30]
On implementing 2D rectangu- lar assignment algorithms,
D. F. Crouse, “On implementing 2D rectangu- lar assignment algorithms,”IEEE Transactions on Aerospace and Electronic Systems, vol. 52, no. 4, Aug. 2016,doi:10.1109/TAES.2016.140952
2016
-
[31]
A. H. Ribeiro et al.,CODE-15%: A large scale anno- tated dataset of 12-lead ECGs, Jun. 2021,doi:10. 5281/zenodo.4916206
2021
-
[32]
ECG-Image-Kit: A synthetic image generation toolbox to facilitate deep learning- based electrocardiogram digitization,
K. K. Shivashankara, Deepanshi, A. Mehri Shervedani, G. D. Clifford, M. A. Reyna, and R. Sameni, “ECG-Image-Kit: A synthetic image generation toolbox to facilitate deep learning- based electrocardiogram digitization,”Physiological Measurement, vol. 45, no. 5, May 2024,doi: 10....
2024 doi
- [33]
- [35]
- [36]
Reviewed August 4, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.