{"id":"f41297b4-c064-483e-8bf0-73538931d5e0","arxiv_id":"2502.07389","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"A self-supervised ECG forecasting model, trained only on normal signals, detects heartbeat and arrhythmia anomalies by measuring the error between its forecast and the real signal, achieving 83.84% anomaly accuracy and 85.46% normal accuracy on MIT-BIH Arrhythmia.","lead":"FADE is a deep learning system that learns to forecast the next second of a normal ECG signal, then flags anomalies by comparing the forecast with real sensor data. The method works without large labeled anomaly datasets, using self-supervised training plus domain adaptation, and reaches about 85% detection accuracy on the MIT-BIH Arrhythmia benchmark.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Domain adaptation and anomaly test sets are not patient-disjoint: the model fine-tunes on normal windows from the same subjects whose anomalous windows are later tested, likely inflating the claimed 83.84%/85.46% accuracy.","rationale":"The reader's weakest assumption about the four-second normal-prior window is real and acknowledged in the paper's Discussion, but it limits application scope rather than contaminating the reported accuracy. The subject-overlap issue is unacknowledged and directly undermines the central claim of generalizable anomaly detection: because the domain-adaptation training set and the anomaly detection test set share patients, the reported accuracy may reflect patient-specific adaptation rather than a system that can adapt to new patients or contexts. This is a more fundamental threat to the headline numbers. I agree with the reader's overall CONDITIONAL verdict, since a patient-disjoint evaluation could still support the method, but the condition should explicitly require such an evaluation before the accuracy claims are taken at face value. Other reader concerns (threshold supervision, temporal-length selection on the same folds, unverifiable code) remain valid and also support a conditional verdict.","tokens_in":17686,"tokens_out":8745,"duration_ms":79892,"concrete_test":"Re-run the domain adaptation and anomaly detection with a patient-disjoint split. Sort the 35 selected MIT-BIH Arrhythmia subjects, fine-tune the forecasting model only on normal windows from 28 subjects, and test on all anomalous windows from the remaining 7 subjects (plus the normal windows from those held-out subjects for threshold selection and evaluation). Repeat with several leave-7-out subject folds and report average anomaly and normal accuracy. If the patient-wise accuracy drops by more than 5 percentage points relative to the reported 83.84%/85.46%, the current claim overstates generalizability; if it remains similar, the subject-overlap concern is refuted.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim of accurate anomaly detection on MIT-BIH Arrhythmia rests on an evaluation that mixes patients between the domain-adaptation training set and the anomaly-detection test set. Section 4.2.3 states normal samples are 'randomly shuffled and divided according to a ratio of 80% training and 20% test' with no patient-wise separation. Section 4.2.4 then uses 'all abnormal samples' from the same 35-subject pool in the test set, alongside the 20% held-out normal samples. Because the domain-adapted model has been fine-tuned on normal windows from a subject, its ability to flag an anomalous window from that same subject is not evidence of generalization to a new patient, contrary to the claimed domain-adaptation contribution. This is not listed among the acknowledged limitations in the Discussion, so the reader cannot assess how much of the reported accuracy is due to subject-specific adaptation rather than generalizable forecasting. This is a correctness risk: the headline numbers may not transfer to a deployment setting where the model encounters a patient whose normal ECG was never seen during adaptation.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper presents FADE, a self-supervised forecasting model for ECG anomaly detection. The model is trained on normal ECG windows from MIT-BIH NSR using a novel Split-MSE loss, then domain-adapted using normal samples from the MIT-BIH Arrhythmia database, and finally detects anomalies by comparing a forecasted one-second signal with the actual signal through a novel NMAE distance and a threshold. The authors report an average anomaly detection accuracy of 83.84% and normal classification accuracy of 85.46% on MIT-BIH Arrhythmia, and they provide ablation studies, a temporal-length study, and per-anomaly accuracy analysis.","tokens_in":17935,"tokens_out":6319,"duration_ms":54672,"significance":"The paper addresses a relevant problem and proposes an interesting idea: using forecasting of normal ECG as a self-supervised pretext task, avoiding the need for large labeled anomaly datasets. The design of a morphology-aware loss (Split-MSE) and a distance metric (NMAE) is well motivated. The experimental protocol is transparent in many respects (nested train/test splits for threshold selection, 25 combinations, ablation studies) and the authors state that the full code will be released. However, the main accuracy claim currently rests on an evaluation protocol that allows the domain-adaptation training set and the anomaly-detection test set to share subjects, which is a serious threat to the validity of the reported generalization.","major_comments":[{"comment":"The domain-adaptation and anomaly-detection evaluation are not patient-disjoint. Section 4.2.3 states that normal samples are 'randomly shuffled and divided according to a ratio of 80% training and 20% test' with no subject-wise separation, and Section 4.2.4 then uses 'all abnormal samples' from the same 35-subject pool in the anomaly test set. Since the domain-adapted model is fine-tuned on normal windows of a subject, its ability to flag an anomalous window from that same subject may reflect subject-specific adaptation rather than a generalizable forecasting model. This is a correctness risk for the central claims of 83.84% anomaly accuracy and 85.46% normal accuracy, and the Discussion does not list patient overlap among the limitations. The experiments should be re-run with patient-disjoint splits (e.g., leaving out entire subjects from domain adaptation when they appear in the anomaly test set) and per-subject results should be reported.","section":"4.2.3-4.2.4"},{"comment":"The method is repeatedly described as 'fully unsupervised' (last paragraph of Related Work; Section 6) and 'never having seen anomalous samples,' but the anomaly threshold T is selected using labeled normal and abnormal samples (Section 4.2.4). This is standard calibration rather than fully label-free detection. The claims should be revised to state that the pipeline requires a labeled calibration set of anomalous segments to select the threshold, or the threshold-selection procedure should be changed to a fully unsupervised rule (e.g., a fixed percentile of normal distances).","section":"3.4, 4.2.4"},{"comment":"The choice of the input duration WI=4s is made by comparing accuracies on the same test folds that are later used for the reported final performance (Table 2). This is a model-selection-on-test procedure: the reported 84.65% global accuracy is the result of selecting the best of four candidate input lengths on the held-out data. To obtain unbiased estimates, the WI selection should be performed on a separate validation fold, or the paper should present the final accuracy as the result of this selection process with appropriate caveats.","section":"5.4"}],"minor_comments":[{"comment":"A single Leave-One-Subject-Out evaluation (Patient 5 only) is used to assess the baseline forecasting model; this is not a statistically robust estimate of forecasting performance. Please provide error bars or use multiple LOSO folds.","section":"4.2.2"},{"comment":"The NMAE comparison between FADE (0.011) and the reimplemented transformer (0.022) is reported without statistical significance or error bars; the conclusion that FADE 'predicts better future signals' is based on this single comparison.","section":"5.1"},{"comment":"Table 3 reports per-anomaly accuracies but no sample counts. The text mentions classes with fewer than 12 samples but does not identify them; adding counts would help interpret the 0% accuracy for nodal escape beats.","section":"5.3, Table 3"},{"comment":"Typographical issues: 'Arrythmia' in the abstract should be 'Arrhythmia', and the title displays 'F ADE' with an unintended space; also 'commit those values for brevity' in Section 5.1 should be 'omit those values'.","section":"Abstract and title"},{"comment":"The description of the third Split-MSE variation ('separating positive and negative values into two distinct bands') is underspecified; please define the band limits and how the two bands are combined in the loss.","section":"5.6.1"}],"recommendation":"major_revision","confidential_remarks":"I believe the paper is publishable after major revision, provided the patient-disjoint evaluation is carried out and the 'fully unsupervised' claims are calibrated. The journal might also verify that the GitHub code is indeed available and matches the described protocol."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Take a look at this paper if you're working in biosignal anomaly detection. The engineering is solid, but the evaluation has a patient-leakage problem that calls the headline numbers into question.\n\nWhat's new: the FADE pipeline combines SlowFast and U-Net for 1D ECG forecasting, with a Split-MSE loss that treats R-peaks and the segments between them separately, and an NMAE distance metric for anomaly detection. That combination is new to this context, and the empirical results on MIT-BIH Arrhythmia are new numbers. The paper also does useful things: transparent protocol, nested splits, 25 random combinations with standard deviations, ablation studies for the loss and the network paths, per-class anomaly accuracies, and they specify the data splits clearly to help others.\n\nThe load-bearing issue is in Sections 4.2.3-4.2.4. The domain-adaptation training set is a random shuffle of normal samples from the MIT-BIH arrhythmia subjects, split 80/20, with no patient-wise separation. The anomaly test set then uses all abnormal samples from the same 35-subject pool, plus the 20% held-out normal samples. So the adapted model has been fine-tuned on normal windows from the same patients whose anomalous windows are being classified. That makes the 83.84% anomaly / 85.46% normal accuracy partly a within-patient result, not a clean test of generalization to a new patient. The authors acknowledge the requirement of a normal input window, but they don't list this patient overlap as a limitation. If this is meant as a domain-adaptation contribution, the evaluation needs patient-disjoint splits.\n\nTwo smaller points. First, the abstract and intro say the model has never seen anomalous samples; that's overstated, because a labeled subset of anomalies is used to select the threshold in Section 4.2.4. That's standard calibration, but it means the system is not fully unsupervised. Second, the four-second input window was chosen on the same evaluation folds (Section 5.4), and the only comparison is one reimplemented transformer.\n\nWho this is for: people building practical ECG anomaly detectors from normal data plus a small labeled set. The loss and distance metric are worth studying, and the ablation work is honest. But take the headline accuracy as optimistic until patient-disjoint validation is done.\n\nMy recommendation: send it to peer review. The method deserves serious evaluation, and the flaw is fixable — a re-run with patient-disjoint adaptation and test sets, plus a clear statement about the labeled threshold, would make the numbers trustworthy.","headline":"Solid engineering pipeline for ECG anomaly detection, but the evaluation leaks patients between domain adaptation and test, so the headline accuracies are likely optimistic.","tokens_in":18481,"tokens_out":3522,"would_cite":false,"duration_ms":29020,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A self-supervised forecaster trained only on normal ECG signals flags anomalies it never saw by comparing its forecast with the real signal.","keywords":["ECG anomaly detection","self-supervised learning","domain adaptation","ECG forecasting","arrhythmia detection","Split-MSE loss","SlowFast","U-Net"],"falsifier":"Run the detector on sustained arrhythmias, such as continuous atrial fibrillation or ventricular tachycardia with no preceding four seconds of normal rhythm, and measure whether the NMAE threshold still separates them; according to the paper's own discussion, the method should be inapplicable or show a sharp accuracy drop in exactly this setting.","tokens_in":1369,"feed_emoji":"🫀","tokens_out":1909,"duration_ms":46141,"temperature":0.7,"pith_summary":"This paper proposes FADE, a deep learning system that detects cardiac anomalies by forecasting the next second of a normal ECG and measuring how far the forecast falls from the real signal. The model is trained only on normal ECG recordings, so it never needs labeled anomalous waveforms; a domain-adaptation step adjusts it to a new sensor or patient population. On the MIT-BIH Arrhythmia database after adaptation, the system reports 83.84% accuracy for anomalous samples and 85.46% for normal ones. The point, if it holds, is that generic anomaly detection can be built from abundant normal data rather than from scarce expert labels of every possible abnormality.","feed_headline":"An ECG model that only learns normal signals still catches arrhythmias","feed_subtitle":"Forecasting the next beat and measuring mismatch spots abnormalities with no labeled anomalies needed.","key_machinery":"The load-bearing pieces are three. Split-MSE is a loss that computes MSE separately in an inner band near zero (the segments between R-peaks) and outer bands (the R-peaks themselves), forcing the model to learn both morphology and baseline; the authors report it outperforms plain MSE and all tested variants. NMAE is the distance metric used at test time, combining raw mean absolute error with a normalized difference between label and forecast; it outperforms nine alternative distances in their comparison. Domain adaptation fine-tunes the normal-signal forecaster on normal samples from the target dataset, raising anomaly-detection accuracy by roughly 12 percentage points over the non-adapted model. The architecture itself, two frequency paths from SlowFast feeding a U-Net decoder, is what carries the forecasting, and the ablation shows both paths are needed for balanced accuracy.","core_discovery":"The paper's central claim is that anomaly detection in ECG can be reduced to the quality of a forecasting model: a model that accurately predicts the future of normal heart activity will produce a large error exactly where the true signal deviates from normal morphology, and that error separates abnormal beats and rhythms from normal ones. FADE performs this with a SlowFast-style encoder and U-Net-style decoder trained with a morphological loss, then fine-tuned on normal samples from the target domain and paired with a normalized mean-absolute-error distance (NMAE) and a threshold. The result is a binary detector that works without ever having seen an anomalous training sample, detects both beat-level and rhythm-level anomalies, and is evaluated over 25 train/test splits with reported means and standard deviations.","pith_inferences":["A natural extension is to use the forecast error itself as a localization signal: the time indices where the distance is largest may mark where in the beat the anomaly begins, something the current binary threshold does not expose.","Because the forecaster only needs normal data, the same pipeline could transfer to other physiological signals with sparse abnormal events, such as EEG or photoplethysmography, wherever a reliable normal corpus exists.","The four-second normal-window assumption suggests a practical screening design: a wearable could keep a rolling buffer and score only segments whose trailing four seconds pass a normality check, while sustained arrhythmias are handled by a separate trigger.","The accuracy gap between rhythm anomalies (97.48%) and beat anomalies (79.78%) implies that morphology-preserving ectopic beats are the harder case, and a targeted loss weighting or longer context might close that gap."],"forward_implications":["Anomaly detection can be trained without labeled anomalous ECG data, using only normal recordings plus a small labeled set for threshold selection.","The same forecasting mechanism generalizes to both heartbeat-level abnormalities, such as premature ventricular contractions, and rhythm-level abnormalities, such as ventricular tachycardia and bigeminy.","Domain adaptation to a new patient population or sensor setup is a major accuracy lever, improving detection accuracy by about 12 percentage points in the reported experiments.","A four-second input window gives the best balance of anomaly and normal accuracy compared with one-, two-, and three-second windows in the temporal-length study.","A transformer-based comparison baseline retrained under the same setup scores lower in both forecasting quality and anomaly-detection accuracy, indicating that the forecasting mechanism itself matters."],"supporting_citations":[{"why":"Supplies the MIT-BIH Normal Sinus Rhythm database, the normal ECG corpus used to train the baseline forecasting model.","marker":"[4]"},{"why":"Supplies the MIT-BIH Arrhythmia database, used for domain adaptation, threshold selection, and final anomaly-detection evaluation.","marker":"[5]"},{"why":"Provides the transformer-based baseline that the authors re-implement under the same data setup for comparison.","marker":"[21]"},{"why":"Supplies the SlowFast two-pathway slow/fast architecture that the encoder is built on.","marker":"[26]"},{"why":"Supplies the U-Net encoder-decoder design with skip connections used for forecasting.","marker":"[27]"}],"fun_headline_variants":["Forecast normal ECG, catch arrhythmias","Self-supervised ECG forecasting spots anomalies","Anomaly detection without labeled anomalies","Predicting normal beats to find abnormal ones","Learn normal ECG, flag anomalies"],"cache_read_input_tokens":20608,"weakest_assumption_plain":"The method assumes that every test sample has a four-second window of normal heart activity immediately before the one-second interval being checked, because the forecast is only meaningful when the input is normal.","fun_headline_variants_meta":{"raw":{"variants":["Forecast normal ECG, catch arrhythmias","Self-supervised ECG forecasting spots anomalies","Anomaly detection without labeled anomalies","Predicting normal beats to find abnormal ones","Learn normal ECG, flag anomalies"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000188,"raw_usage":{"total_tokens":1353,"prompt_tokens":987,"completion_tokens":366,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":603,"completion_tokens_details":{"reasoning_tokens":304}},"tokens_in":603,"tokens_out":366,"duration_ms":3604,"temperature":1.0,"reasoning_tokens":304,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T12:54:37.051339+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the detector on sustained arrhythmias, such as continuous atrial fibrillation or ventricular tachycardia with no preceding four seconds of normal rhythm, and measure whether the NMAE threshold still separates them; according to the paper's own discussion, the method should be inapplicable or show a sharp accuracy drop in exactly this setting.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the MIT-BIH Normal Sinus Rhythm database, the normal ECG corpus used to train the baseline forecasting model."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the MIT-BIH Arrhythmia database, used for domain adaptation, threshold selection, and final anomaly-detection evaluation."},{"cited_title":"Alamr, A","cited_arxiv_id":null,"evidence_quote":"Provides the transformer-based baseline that the authors re-implement under the same data setup for comparison."},{"cited_title":"Feichtenhofer, H","cited_arxiv_id":null,"evidence_quote":"Supplies the SlowFast two-pathway slow/fast architecture that the encoder is built on."}],"review_version":1}