{"id":"e3601041-8da6-4185-b6e9-539577cb9e67","arxiv_id":"1908.06865","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"A multi-layer perceptron trained on short-time Fourier transform sparse signatures of ECG heartbeats is reported to reach 95.7% average classification accuracy on MIT-BIH, but the evaluation uses a beat-level split with patient leakage.","lead":"This paper describes a wearable ECG heartbeat classifier that converts each heartbeat into a sparse time-frequency image and feeds it to a neural network. The authors report high accuracy on a standard database, but the evaluation mixes beats from the same patients in training and testing, which may inflate the results.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Beat-level split leaks patient identity; the 95.7% accuracy claim is unverified for new wearable users.","rationale":"The reader identified the beat-level split as the weakest assumption, and I agree that it is the most load-bearing issue. The paper's proposed sparse time-frequency representation is an interesting idea with some plausible motivation, and the authors do compare against alternative representations in Table 2, but the evaluation protocol does not match the wearable deployment claim. A random split by beat within the same patients is known to overestimate accuracy for ECG classification because patient identity leaks into the training set. The paper does not report any held-out-patient evaluation, so the headline accuracy, false-negative reduction, and improvement over prior work are not credible as evidence of generalization. Additional problems, such as missing STFT parameters, sparsity threshold, MLP architecture, and training hyperparameters, further prevent independent replication, but these are secondary to the data-split flaw. The paper also has irrelevant keywords (spiking neural network, PSO, CxQuad, etc.) that appear nowhere in the text, which suggests internal inconsistency, though this does not change the main technical verdict. Because the central claim depends on an evaluation that does not test the actual use case, the REJECT verdict is appropriate and I see no reason to alter it.","tokens_in":8209,"tokens_out":2446,"duration_ms":24768,"concrete_test":"Retrain and test with a patient-disjoint split of MIT-BIH (e.g., train on 24 patients, test on the remaining 24), using the same STFT, sparsity threshold, and MLP configuration as described. Report average per-class accuracy and false-negative rate on the held-out patients. If average accuracy drops materially below 95.7%, or becomes comparable to the baseline [6] under the same split, then the central claim is not supported for the wearable scenario.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim of 95.7% average accuracy is supported only by a beat-level random split of MIT-BIH: Section 5 states that 6500 QRS complexes are 'selected randomly from 120K QRS complexes for 48 patients' and the test set is 'all 120K QRS complexes (except the ones used for the training set)'. Because MIT-BIH contains multiple beats per patient with shared morphology and recording conditions, this split lets the MLP see patient-specific patterns during training and then be tested on other beats from the same patients. For the stated wearable use case, the model must generalize to previously unseen patients, yet no patient-disjoint evaluation is reported. The '22% improvement' and '3.7% false negatives' are therefore not established for new patients. Even the Table 2 'Patient (Avg.)' column reports accuracy 'for 48 patients in the database', which does not address inter-patient generalization. Without a held-out-patient test, the headline result could largely reflect memorization of patient-specific signatures rather than a generalizable heartbeat classifier.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes a two-phase pipeline for heartbeat classification in wearable settings. In the offline phase, QRS complexes from the MIT-BIH arrhythmia database are transformed into sparse 82×82 signatures built from the real and imaginary parts of the short-time Fourier transform, and a multi-layer perceptron (MLP) is trained on these signatures; in the online phase, the Pan-Tompkins algorithm detects QRS peaks in live ECG and the trained MLP classifies each beat. The paper reports an average classification accuracy of 95.7% across 23 heartbeat classes (claimed 22% higher than a time-domain baseline), a false-negative rate of 3.7% for the proposed signatures versus 15.6%–43.5% for three alternative representations, improved robustness to additive white Gaussian noise, and online execution times compatible with real-time use. The central results are presented in Section 5, with accuracy curves in Figure 6, representation comparisons in Table 2, noise analysis in Section 5.3, and timing measurements in Table 3.","tokens_in":8330,"tokens_out":10251,"duration_ms":94141,"significance":"The paper's contribution, if its numbers hold, is a compact sparse time-frequency signature that lets a simple MLP classify ECG beats with high accuracy and low false negatives, plus a plausible real-time implementation story. The signature construction is deterministic — computed from the STFT and a threshold, not fitted to labels — and the controlled comparison across four input representations in Table 2 is a reasonable experimental design. However, the significance is limited by two factors. The reported 95.7% accuracy is produced under a beat-level split in which training and test beats come from the same patients, so it does not support the wearable-use-case claim of generalization to new patients; and the experiment is described with too few details to be reproduced or independently verified. If a patient-disjoint re-evaluation confirms the advantage, the sparse signature representation would be a useful addition to the ECG-classification literature.","major_comments":[{"comment":"The central claim of 95.7% average accuracy rests on a beat-level random split: Section 5 states that 6500 labeled QRS complexes are 'selected randomly from 120K QRS complexes for 48 patients' and that the test set is the remaining beats 'represented from all 48 patients'. Because MIT-BIH contains many beats per patient and beats from the same patient share morphology and recording conditions, this protocol allows the MLP to learn patient-specific patterns during training and then be evaluated on other beats from the very same patients. For the stated wearable deployment scenario, where the classifier must generalize to unseen patients, this does not establish the result; the 'Patient (Avg.)' column in Table 2 averages over the same 48 patients under the same split and does not address inter-patient generalization. The evaluation should be redone with a patient-disjoint split (e.g., the DS1/DS2 split commonly used in the literature and consistent with ANSI/AAMI EC57), reporting accuracy, sensitivity, and positive predictive value on held-out patients, and the corresponding claims in the abstract, Sections 5.1 and 5.2 must then be revised.","section":"§5, Evaluation protocol"},{"comment":"The implementation details needed to reproduce the headline result are missing. The STFT parameters (analysis window, length, stride) are not reported — the paper only mentions 'specgram of python-matplotlib' — and the sparsity threshold that turns the STFT into a sparse signature is not quantified (Section 4.1 says only 'values above a certain threshold (sparsity < 1%)'). The MLP configuration is likewise unspecified: Section 5 says a 'design space exploration' determined the hidden layers and neurons, but the final architecture is never given, and Table 3 lists 2000–10000 neurons without identifying which setting produced the 95.7% accuracy. The paper should report the exact STFT settings, the threshold, the final MLP architecture, and the training hyperparameters (learning rate, epochs, regularization), and should state whether the design-space exploration used the test set, which would introduce additional leakage.","section":"§5 and §4.1, Reproducibility"},{"comment":"The claimed relative improvements — 22% over the time-domain baseline, 89% lower false negatives than the wavelet representation, 91% lower than the STFT-only representation — are only interpretable if the four representations in Table 2 are compared under identical conditions. The patient-average accuracy of 58.4% reported for the wavelet representation is far below typical results for wavelet-based ECG classifiers reported elsewhere, which raises the question of whether the baselines were run with the same pre-processing, the same split, and the same (unspecified) MLP configuration. The authors should specify the common experimental setup, provide confusion matrices for the four representations, and confirm that the false-positive and false-negative definitions in Section 5.2 match those used for the baselines; without this, the abstract's 3.7% false-negative claim is not verifiable.","section":"§5.2, Baseline comparisons"}],"minor_comments":[{"comment":"The keyword list (Spiking neural network, global synapse, particle swarm optimization, CxQuad, spike disorder count, inter-spike distortion) does not correspond to any content in the manuscript; it should be replaced with terms such as ECG, heartbeat classification, short-time Fourier transform, sparse distributed representation, and multi-layer perceptron.","section":"Keywords"},{"comment":"The padded zero component P in the signature definition carries no information; the authors should either justify its inclusion or explicitly describe it as a placeholder, and clarify whether its constant zeros affect the stated sparsity of fewer than 1% nonzero entries.","section":"§4.1, Eq. (2)"},{"comment":"Figure 7 shows the noise-tolerance curve for a single patient (id 106), yet the text claims an 'average 133%' improvement over the time-domain baseline for the SNR range 0–20 dB; the average should be supported by reporting the distribution across patients rather than a statement that a similar trend was observed.","section":"§5.3, Noise tolerance"},{"comment":"The motion-artifact sentence reports 'comparable accuracy' for motion-compensated ECG compared with a wavelet-ICA approach but provides no numbers or details; either quantitative results should be added or the claim removed.","section":"§5.3, Motion artifacts"},{"comment":"The mapping from the 23 MIT-BIH classes to the five classes used in the 'training with 5 classes' column is not described; the authors should state the mapping explicitly so that Table 1 is reproducible.","section":"Table 1"},{"comment":"The abstract states 'an improvement of 22% over state-of-the-art approaches', while Section 5.1 specifies the improvement against a single time-domain baseline [6]; the phrasing should be made precise to avoid overclaiming.","section":"Abstract and §5.1"},{"comment":"Reference [20] misattributes the 1986 backpropagation paper to Williams and Hinton; the correct first author is Rumelhart. In Section 5, 'Hashwell CPU' should read 'Haswell CPU', and the name 'Pan & Tompkin's Algorithm' in Section 3 should be 'Pan & Tompkins'.","section":"References and typos"}],"recommendation":"major_revision","confidential_remarks":"The reader's assessment matches my own on the central issue: the beat-level split invalidates the headline accuracy claim for the stated use case, and a full re-run under a patient-disjoint protocol is required. One additional observation for the editor: the keyword block does not match the body of the paper, and reference [20] is misattributed; these are presentation issues, but they reduce confidence in the care with which the experiments were reported. I would not support acceptance unless the patient-disjoint re-evaluation preserves the qualitative advantage over the baselines."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nYou asked for my take on the heartbeat classification paper. The core claim—95.7% average accuracy for wearable arrhythmia detection—is not supported by the evaluation. The training set is 6500 beats randomly drawn from 120K beats across 48 patients, and the test set is the remaining beats from the same 48 patients. That is a beat-level split. Each patient contributes many beats with similar morphology, so the MLP can memorize patient-specific patterns and still score well on the same patients' held-out beats. For a wearable, the model needs to work on unseen patients; this paper never tests that. So the headline number is essentially unverified.\n\nThat said, there is something real here. The paper converts each QRS complex to a time-frequency representation, keeps both real and imaginary parts, thresholds them into a sparse 82×82 signature, and feeds that to an MLP. That's a simple but plausible way to preserve phase information. The authors also test noise tolerance with AWGN and report real-time latency numbers, which is more than many classification papers do. Table 2 compares their sparse signature against wavelet, STFT, and Numenta SDR with the same MLP classifier; that's a controlled comparison and their representation did best on this data.\n\nThe soft spots beyond the split: implementation details are missing. They don't give the STFT window length, overlap, threshold, or the exact MLP architecture (hidden layers, neurons, learning rate). Without that, no one can reproduce the results. The '22% improvement' over [6] is not a head-to-head comparison, because [6] uses a different classifier. The false negative reduction in Table 2 is cleaner, since it compares representations under the same MLP, but it's still only on the same-patient split. The paper is also sloppy—keywords mention spiking neural networks and particle swarm optimization that don't appear in the body, and there are typos like 'Hashwell' and 'onlne'. That doesn't kill the science, but it doesn't build confidence.\n\nBottom line: the idea is not worthless, but the evaluation does not support the claims. A serious referee would need a patient-disjoint split (e.g., train on some patients, test on others), full parameter disclosure, and fair baselines. I'd send it to peer review because the topic is important and the approach could be made rigorous, but I'd expect a major revision or likely rejection in current form.\n\nBest,","headline":"A beat-level split inflates the headline accuracy; the underlying sparse STFT idea is worth a second look with proper patient-disjoint evaluation.","tokens_in":8921,"tokens_out":3258,"would_cite":false,"duration_ms":29064,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A sparse time-frequency representation of each heartbeat, fed to a multi-layer perceptron, classifies ECG beats with 95.7% average accuracy and cuts false negatives to 3.7%.","keywords":["heartbeat classification","ECG","time-frequency joint distribution","sparse distributed representation","short-time Fourier transform","multi-layer perceptron","arrhythmia detection","wearable health monitoring"],"falsifier":"Retrain the same signature-plus-MLP pipeline on some of the 48 patients in the labeled ECG database and test on the remaining patients, reporting per-patient accuracy; if patient-exclusive accuracy falls materially below 95.7% or false negatives rise above 3.7%, the claimed improvement depends on same-patient beat overlap rather than on the signature.","tokens_in":7950,"feed_emoji":"🫀","tokens_out":7334,"duration_ms":63267,"temperature":0.7,"pith_summary":"This paper tries to establish that a wearable ECG device can classify heartbeats more accurately by representing each QRS complex as a sparse signature built from the joint time-frequency distribution of the signal, rather than from raw time samples or frequency components alone. The authors argue that this representation captures time-varying frequency content that matters for non-stationary signals, and that a multi-layer perceptron trained on these signatures sorts beats into 23 arrhythmia categories with an average accuracy of 95.7%. If the claim holds, it matters because wearable arrhythmia monitoring needs low false negatives, noise tolerance, and real-time classification on small processors. The paper further reports that the signatures reduce false negatives to 3.7%, an 89% reduction relative to the wavelet, STFT, and sparse representations it compares against.","feed_headline":"Sparse ECG signatures classify heartbeats at 95.7% accuracy","feed_subtitle":"Time-frequency signatures plus a neural net cut false negatives to 3.7 percent.","key_machinery":"The load-bearing object is the ECG sparse distributed signature S = [f{R(X)} f{I(X)} P], where X is the short-time Fourier transform of the QRS complex, f is a transformation applied to the real and imaginary parts, and P is a padded zero region. Each signature is an 82x82 matrix with sparsity below 1%, so the positions of high-magnitude entries encode the beat's identity. The signature is fed to a multi-layer perceptron with 6,724 input neurons, and the paper argues that this representation, not any hand-coded feature set, is what drives the accuracy and noise tolerance.","core_discovery":"The central claim is that a sparse distributed signature formed from the real and imaginary parts of the short-time Fourier transform of a QRS complex, combined into an 82x82 matrix with fewer than 1% nonzero entries, carries enough information for a multi-layer perceptron to separate the 23 heartbeat classes. With this representation, the classifier reaches 80.7% to 100% accuracy per class, averaging 95.7%, and reduces false negatives to 3.7%, about 89% lower than wavelet and STFT representations tested with the same classifier. The authors further report that the signature keeps classification accuracy high under additive white Gaussian noise, and that the online pipeline of QRS detection, signature conversion, and classification finishes in milliseconds, supporting real-time wearable use.","pith_inferences":["Because the train and test beats are drawn from the same 48 patients, the reported 95.7% average likely overstates performance on a brand-new person; a held-out-patient evaluation would be the natural next test.","The signature's reliance on the positions of high-value entries suggests the same construction could be applied to other transient biomedical signals, such as EEG or EMG, where non-stationarity is also the norm.","The deliberately padded zero block in the signature leaves room to fuse auxiliary sensor streams, for example accelerometer-based motion data, without changing the classifier, which could address the motion artifacts the paper mentions only briefly."],"forward_implications":["The online pipeline, at roughly 10.4 ms for signature conversion and 1.2 to 4.0 ms for classification at 160 MHz, is fast enough for real-time wearable arrhythmia monitoring.","Training the same classifier on the five clinically grouped heartbeat classes instead of all 23 improves per-class accuracy by 0.4 to 5.1 percentage points, so the method can be specialized to specific clinical questions.","The false-negative rate of 3.7% means fewer missed arrhythmic beats than the wavelet, STFT, and sparse representations compared in the paper, which is the failure mode that matters most for screening.","Under additive white Gaussian noise from 0 to 20 dB, the proposed signatures hold classification accuracy substantially above a time-domain baseline, suggesting resilience to acquisition noise."],"supporting_citations":[{"why":"supplies the labeled heartbeat data and 23 category annotations used for training and evaluation","marker":"[1]"},{"why":"the time-domain method whose average accuracy the claimed 22% improvement is measured against","marker":"[6]"},{"why":"the frequency-domain method compared per heartbeat class in the accuracy results","marker":"[11]"},{"why":"provides the real-time QRS detection algorithm used to segment beats in the online phase","marker":"[13]"},{"why":"defines the short-time Fourier transform formula on which the time-frequency signature is built","marker":"[18]"},{"why":"the sparse ECG representation whose false-negative rate is compared in the preprocessing study","marker":"[21]"},{"why":"the wavelet-transform ECG representation whose accuracy and false negatives are compared","marker":"[12]"}],"fun_headline_variants":["Sparse time-frequency ECG signatures classify heartbeats at 95.7%","Heartbeat classification via sparse ECG time-frequency hits 95.7%","MLP on sparse ECG time-frequency cuts false negatives by 89%","Sparse ECG time-frequency features yield 95.7% heartbeat accuracy","Wearable heartbeat classifier: sparse time-frequency ECG at 95.7%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported accuracy assumes that randomly splitting beats from the same 48 patients into training and test sets predicts how the classifier will perform on a new person's ECG; if beats from the same patient are too similar, the test accuracy is inflated.","fun_headline_variants_meta":{"raw":{"variants":["Sparse time-frequency ECG signatures classify heartbeats at 95.7%","Heartbeat classification via sparse ECG time-frequency hits 95.7%","MLP on sparse ECG time-frequency cuts false negatives by 89%","Sparse ECG time-frequency features yield 95.7% heartbeat accuracy","Wearable heartbeat classifier: sparse time-frequency ECG at 95.7%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00022,"raw_usage":{"total_tokens":1379,"prompt_tokens":808,"completion_tokens":571,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":424,"completion_tokens_details":{"reasoning_tokens":471}},"tokens_in":424,"tokens_out":571,"duration_ms":5523,"temperature":1.0,"reasoning_tokens":471,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T13:29:02.147807+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain the same signature-plus-MLP pipeline on some of the 48 patients in the labeled ECG database and test on the remaining patients, reporting per-patient accuracy; if patient-exclusive accuracy falls materially below 95.7% or false negatives rise above 3.7%, the claimed improvement depends on same-patient beat overlap rather than on the signature.","supporting_citations":[{"cited_title":"The impact of the MIT-BIH arrhythmia database,","cited_arxiv_id":null,"evidence_quote":"supplies the labeled heartbeat data and 23 category annotations used for training and evaluation"},{"cited_title":"Automatic cardiac arrhythmia detection and classification using vectorcardiograms and complex networks,","cited_arxiv_id":null,"evidence_quote":"the time-domain method whose average accuracy the claimed 22% improvement is measured against"},{"cited_title":"Computer aided diagnosis of atrial arrhythmia using dimensionality reduction methods on transform domain representation,","cited_arxiv_id":null,"evidence_quote":"the frequency-domain method compared per heartbeat class in the accuracy results"},{"cited_title":"A real-time QRS detection algorithm,","cited_arxiv_id":null,"evidence_quote":"provides the real-time QRS detection algorithm used to segment beats in the online phase"},{"cited_title":"The fractional fourier transform and time-frequency representa- tions,","cited_arxiv_id":null,"evidence_quote":"defines the short-time Fourier transform formula on which the time-frequency signature is built"},{"cited_title":"Encoding Data for HTM Systems","cited_arxiv_id":"1602.05925","evidence_quote":"the sparse ECG representation whose false-negative rate is compared in the preprocessing study"},{"cited_title":"ECG signals classification based on discrete wavelet transform, time domain and frequency domain features,","cited_arxiv_id":null,"evidence_quote":"the wavelet-transform ECG representation whose accuracy and false negatives are compared"}],"review_version":1}