{"id":"b9166e05-8676-4a2b-9e69-8ac8f790c39b","arxiv_id":"2510.19590","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"An open-source ECG digitization pipeline achieves state-of-the-art signal reconstruction, including a mean 19.65 dB SNR on fresh clinical scans and positive SNR on all Emory challenge categories.","lead":"A hospital group built an open-source software pipeline that turns scanned or photographed paper ECGs into digital signals, reporting better reconstruction quality than prior published methods on two large test sets. The tool could let hospitals unlock decades of paper ECG archives for AI-based diagnostics.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported SNR is optimized over horizontal shift up to 100 ms and ambiguous NaN handling; unadjusted SNR not reported.","rationale":"The reader's weakest_assumption focused on synthetic-to-real generalization, which is empirically supported by the zero-shot Emory results and thus less concerning. The reader did flag the SNR alignment/NaN issue as a condition, and I agree that this is the more load-bearing concern for the absolute headline claim. The paper's comparative claim on Emory is likely robust because all methods were evaluated with the same shifted SNR metric, so the state-of-the-art claim is not at risk. However, the 19.65 dB number is presented as a standalone achievement, and without the unshifted value its interpretation is unclear. The concrete test would settle whether the shift optimization materially inflates the metric. If the gap is small, the concern is minor; if large, the paper should reframe the headline. The reader's conditional verdict already requests exactly this clarification, so no verdict change is needed.","tokens_in":12905,"tokens_out":9201,"duration_ms":90164,"concrete_test":"Run the released open-source code on the Ahus Paper Digitization ECG Database and compute SNR under four conditions: (1) as reported (shifted, NaN excluded/set to zero per code), (2) with no horizontal shift but vertical zero-centering, (3) with shifted alignment but NaN samples set to zero instead of excluded, and (4) with no horizontal shift and NaN set to zero. If the unshifted SNR drops by more than 3 dB, or if the NaN-handling variant changes SNR by more than 0.5 dB, the reported 19.65 dB should be qualified as shift-optimized and the manuscript should report all variants. The same check can be done on the Emory hidden dataset only by the external organizers, so the Ahus test is the feasible one.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central quantitative claim—a mean SNR of 19.65 dB on Ahus scanner images—is computed after aligning each lead with a horizontal shift optimized up to 100 ms and vertical zero-centering (Validation metrics section). The shift is chosen to maximize SNR, so the reported value is an upper bound on raw temporal reconstruction accuracy. The paper reports the mean shift (0.86 ms for scans) but not the unshifted SNR, making it impossible to assess how much of the headline performance comes from alignment rather than actual digitization fidelity. A large unshifted-to-shifted gap would materially weaken the absolute claim, even though the comparative claim on Emory (which uses the same shifted metric for all methods) may still hold. Additionally, the description of NaN handling is contradictory: the text says samples are 'set to NaN' but then 'set to zero for the scanned paper.' If failed samples are excluded from the denominator, SNR is inflated; if set to zero, they are penalized. The 0.21% rate is small, but the ambiguity affects reproducibility. This is load-bearing because the headline number is the paper's primary evidence of clinical-grade performance.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents an open-source, fully automated pipeline for digitizing paper ECGs, consisting of five modules: semantic segmentation, perspective correction, grid-size extraction, layout identification, and segmentation-to-trace conversion. The method is trained exclusively on synthetic images generated from Brazilian CODE15% ECGs and then evaluated prospectively on 1,596 images collected at Akershus University Hospital and zero-shot on the 35,595-image Emory Paper Digitization ECG Database via evaluation by the original challenge organizers. The authors report a mean SNR of 19.65 dB on scanner images at Ahus and positive SNR in all Emory subcategories, outperforming previously published methods.","tokens_in":13147,"tokens_out":4679,"duration_ms":43564,"significance":"If the reported results are robust, this is a substantial contribution: it is a rare fully automated, open-source ECG digitization system validated on a large prospective clinical dataset and on a hidden external test set with minimal circularity. The synthetic-to-real zero-shot generalization is noteworthy, as is the release of the training data, the clinical dataset, and the source code. The positive SNR on mobile photos and deteriorated papers would be a first. The main reservations concern the transparency of the SNR metric and the handling of failed samples, both of which are fixable in revision.","major_comments":[{"comment":"The headline SNR of 19.65 dB is computed after an optimal horizontal shift of up to 100 ms and vertical zero-centering. Since 100 ms is comparable to a full QRS complex, the shift can absorb real temporal misalignment; the reported value is therefore an upper bound on raw reconstruction fidelity. The mean shifts are reported (0.86 ms for scans), but the unshifted SNR is not. Please report the SNR without any horizontal alignment, or at least the shift distribution and the zero-shift SNR, so the reader can assess temporal accuracy. The comparative Emory results are less affected because all methods are evaluated with the same shifted metric, but the absolute claim needs this qualifier.","section":"Validation metrics / Ahus Paper Digitization ECG Database"},{"comment":"The description of failed samples is contradictory: \"an average of 0.21% of samples were set to NaN\" is followed by \"0.15% of samples were set to zero for the scanned paper.\" Specify precisely how NaN samples enter the SNR/RMSE/correlation denominators. If NaN leads are excluded from both numerator and denominator, the metrics are computed on a non-random subset and will be optimistically biased; if converted to zero, they are penalized. The 0.21% rate is small, but the ambiguity prevents reproduction of the exact numbers. Also clarify whether the percentages are per sample or per lead.","section":"Ahus Paper Digitization ECG Database (NaN handling)"},{"comment":"The caption states \"Negative scores are not shown.\" This is problematic for a comparison table: a zero baseline (all-zero signal) achieves 0 dB, so negative scores of prior methods are informative. If a dash means the method was not applied or reported, say so; if it means a negative value, the table is visually misleading. Please report all numeric scores (or clearly mark \"not reported\") so the \"outperforms all subcategories\" claim is verifiable. As written, the table cannot be fully audited.","section":"Emory Paper Digitization ECG Database (Table 2)"}],"minor_comments":[{"comment":"The caption contains a typo: \"T op\" should be \"Top.\"","section":"Figure 1"},{"comment":"\"W e\" appears in the abstract and elsewhere; should be \"We.\"","section":"Abstract and text"},{"comment":"Stray formatting artifacts: \"In this section*\" and \"The following section*s\" should be cleaned up.","section":"Results and Methods"},{"comment":"The caption says \"The left pane shows the template\" but the template is in the right pane; correct the reference.","section":"Figure 11"},{"comment":"λ is set to 0.5 with no sensitivity analysis or discussion; a brief note on its effect on layout matching would be helpful.","section":"Algorithm 1"}],"recommendation":"major_revision","confidential_remarks":"The open-source release and the prospective, externally evaluated validation are clear strengths. The major comments are all about reporting transparency and are addressable without new experiments; I would be satisfied with a revised version that reports unshifted SNR, clarifies NaN accounting, and makes Table 2 fully auditable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know this is one of the few digitization papers that actually ships everything: code, synthetic training data, a prospective clinical dataset, and a hidden external evaluation. The Emory results are the strongest part—zero-shot, run by the original organizers, and they beat every published method in every subcategory, including positive SNR on phone photos and deteriorated paper where others went negative. That is a concrete advance.\n\nThe Ahus dataset is also a real asset: 1,596 images from 266 patients, paired with raw time series, collected under a research protocol. The modular pipeline (segmentation, angle-angle perspective correction, layout matching, grid search) is described in enough detail that a competent grad student could reimplement it. The open-source release makes the paper reproducible, which is more than most work in this area.\n\nSoft spots: the main metric is shifted SNR. They optimize a per-lead horizontal shift up to 100 ms, then report the maximum. The mean shift is small (0.86 ms for scans), so the inflation is probably modest, but they never give the unshifted number, which makes the absolute 19.65 dB hard to interpret. The NaN handling is also described ambiguously: text says samples are set to NaN, then a sentence later says set to zero. The rate is only 0.21%, so it's not going to flip the conclusion, but it needs cleaning up for reproducibility.\n\nThe statistics ignore patient-level clustering. With 1,596 images from 266 patients, the per-image SNR numbers are not independent; a mixed model or per-patient averaging would be more honest. That's a minor fix.\n\nThe synthetic-to-real generalization is the load-bearing assumption. The Emory result is strong evidence it works, but they don't analyze failure modes beyond saying misalignment is the main cause. A short error analysis would strengthen the paper.\n\nOverall: solid engineering, honest limitations section, no sign of fitting to the test set. It deserves a serious referee and likely acceptance after the metric and clustering issues are addressed. I'd bring it to reading group and cite it in any ECG digitization work.","headline":"Open-source ECG digitizer validated zero-shot on the 2024 PhysioNet challenge, outperforming all prior methods; the headline SNR is slightly optimistic but the core contribution is real.","tokens_in":13639,"tokens_out":2330,"would_cite":true,"duration_ms":20362,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A fully automated, open-source pipeline converts paper ECGs into digital signals, achieving 19.65 dB SNR on hospital scans and beating prior methods on every test category.","keywords":["ECG digitization","paper ECG","image-to-signal conversion","U-Net segmentation","Hough transform","perspective correction","clinical validation","open-source software"],"falsifier":"Run the released pipeline on a fresh set of paper ECGs from another hospital—say 500 digital ECGs printed to thermal paper at 50 mm/s, then scanned and photographed with consumer phones—and compare each reconstructed lead against the original digital trace; if the scanner mean SNR does not approach the reported 19.65 dB or any phone category falls at or below 0 dB (the all-zeros baseline), the central performance and generalization claims would be contradicted.","tokens_in":12774,"feed_emoji":"📄","tokens_out":8868,"duration_ms":75261,"temperature":0.7,"pith_summary":"This paper is trying to establish that retrospective paper ECG archives—millions of records that exist only as scans or photos—can be turned into standardized digital time-series automatically, accurately, and at scale. The authors build a modular pipeline that segments the ECG trace, corrects perspective distortion, identifies the lead layout, calibrates grid spacing, and reconstructs each lead as a 1D signal. On 1,596 real-world images from a Norwegian hospital, the pipeline reports a mean signal-to-noise ratio of 19.65 dB for flatbed scans and 10–12 dB for phone photos. Evaluated zero-shot on the 35,595-image Emory benchmark, it outperforms all previously published methods in every subcategory and is the first to achieve positive SNR on mobile photos and stained, wrinkled, or deteriorated paper. If these results hold in other archives, the tool would make decades of paper-only ECGs available for AI training, rare-disease studies, and automated diagnosis in low-infrastructure settings.","feed_headline":"Open-source tool digitizes paper ECGs, beating prior methods","feed_subtitle":"Fully automated pipeline scores 19.65 dB SNR on scans and first positive SNR on phone photos of deteriorated paper.","key_machinery":"The load-bearing mechanism is a five-module pipeline that turns image geometry into signal reconstruction. The key geometric insight is an angle-angle domain derived from a double Hough transform: each set of parallel gridlines collapses to a single point, making perspective correction robust under noise and glare. Signal reconstruction then reduces to graph matching: connected components of the segmented trace are subdivided by a 'snipping' algorithm and re-joined by solving a linear sum assignment problem (a modified Jonker-Volgenant algorithm), stitching fragments into continuous leads. A second, lighter U-Net finds lead labels (III, aVR, etc.) and best-matches them to a template layout,","core_discovery":"The central claim is that ECG digitization needs no manual intervention or domain-specific fine-tuning: a fully automated framework, trained only on synthetic 3×4-layout 12-lead images, can reconstruct clinically usable time-series from flatbed scans, phone photos, and degraded paper. On the Emory Paper Digitization ECG Dataset it outperforms all previously published methods in every category and is the first to exceed the all-zeros baseline (0 dB SNR) on mobile photos and stained, wrinkled, or deteriorated paper; on a real-world hospital dataset it reports a mean SNR of 19.65 dB for scans. The complete code, synthetic training set, and paired clinical dataset are released, so the claims are","pith_inferences":["The per-lead NaN mechanism could be repurposed as a confidence score: downstream AI models could weight each lead by whether it was fully digitized, turning the algorithm's failure mode into a safety feature for clinical deployment.","Because limb leads obey the Einthoven relation (I + III = II), missing segments—e.g., lead III in the paper's example—could be imputed and self-consistency-checked, a cheap improvement the authors flag but do not implement.","The perspective-correction and grid-calibration steps are not ECG-specific: any gridded paper record with parallel line families (chart recorders, scrolled waveforms) could likely be digitized with the same angle-angle and autocorrelation machinery, a testable extension beyond electrocardiography.","The modularity invites an end-to-end learned replacement for the snipping and matching stages; overlapping signals break the rule-based approach, so a differentiable trace-reconstruction network trained on the released synthetic dataset would be a concrete next experiment."],"forward_implications":["Retrospective ECG archives can be converted at scale: the pipeline runs unattended in about 5.9 seconds per image on a GPU, so a hospital's backlog of paper-only ECGs can become digital time-series for AI training and research.","Phone-captured ECGs, including photos of stained or wrinkled paper, yield positive-SNR reconstructions, opening AI-based ECG interpretation to settings where only a camera and paper records are available.","The complete release—code, synthetic training set, and paired clinical dataset—lets other groups reproduce, benchmark, and extend the method without access to private data.","Training on synthetically augmented images from a different country and layout transfers zero-shot to real hospital scans, supporting the use of synthetic data when real annotated medical images are scarce."],"fun_headline_variants":["Open-source tool turns paper ECG scans into usable digital data","Digitizing paper ECGs: open-source algorithm beats state-of-the-art","Open-source framework converts paper ECG images to signals","Paper ECGs go digital: new open-source tool tops prior methods","Open-source ECG digitizer handles scans, photos, and damaged paper"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"Everything depends on the segmentation and layout networks, trained only on synthetic Brazilian 12-lead ECGs in a 3×4 layout, generalizing to real scans and photos from other countries, devices, and paper degradation modes without fine-tuning; the zero-shot benchmark result supports this for that dataset, but the generalization mechanism is unanalyzed and other archives are untested.","fun_headline_variants_meta":{"raw":{"variants":["Open-source tool turns paper ECG scans into usable digital data","Digitizing paper ECGs: open-source algorithm beats state-of-the-art","Open-source framework converts paper ECG images to signals","Paper ECGs go digital: new open-source tool tops prior methods","Open-source ECG digitizer handles scans, photos, and damaged paper"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00016,"raw_usage":{"total_tokens":1038,"prompt_tokens":682,"completion_tokens":356,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":426,"completion_tokens_details":{"reasoning_tokens":270}},"tokens_in":426,"tokens_out":356,"duration_ms":3660,"temperature":1.0,"reasoning_tokens":270,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T08:36:32.364683+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the released pipeline on a fresh set of paper ECGs from another hospital—say 500 digital ECGs printed to thermal paper at 50 mm/s, then scanned and photographed with consumer phones—and compare each reconstructed lead against the original digital trace; if the scanner mean SNR does not approach the reported 19.65 dB or any phone category falls at or below 0 dB (the all-zeros baseline), the central performance and generalization claims would be contradicted.","supporting_citations":[],"review_version":1}