REVIEW 2 major objections 5 minor 1 cited by
Sleep Staging from Electrocardiography and Respiration with Deep Learning
T0 review · 2 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read Cardiorespiratory signals alone can stage sleep with moderate accuracy in a large clinical population, reaching a five-stage kappa of 0.600.
desk verdict Large-scale cardiorespiratory sleep staging study with a real patient-leakage concern; the headline kappa is likely somewhat optimistic but the qualitative claim that ECG and respiration carry substantial sleep-stage information holds. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The engine is a two-part deep network: a residual convolutional network that learns features from a 270-second window centered on each 30-second epoch to be scored, followed by a bidirectional long short-term memory network that models temporal context across consecutive epochs. The ECG is represented as a binary sequence of R-peak times, and respiratory effort signals are downsampled to 10 Hz; artifactual epochs are removed for CNN training but kept for the LSTM so temporal continuity is preserved. The loss is a class-weighted cross-entropy that counteracts the dominance of N2, and the configuration is selected on a validation set. This architecture is what converts heart-rate and breathing dynamics into sleep-stage probabilities.
What would settle it
Re-run the experiment with a strict patient-level split—all recordings from each patient held out of training—and compare five-stage kappa; a meaningful drop below 0.600 would show the current number depends on overlapping patients between sets. Scoring an external cohort of ECG and respiratory recordings from a different institution or from a wearable device would provide the same sort of check.
Extended reading notes
Core claim
The paper claims that cardiorespiratory signals contain enough information for moderately accurate automatic sleep staging in a large, heterogeneous adult population. Combining R-peak timing from the ECG with the abdominal respiratory effort trace, a deep network reproduces the five clinical sleep stages with a Cohen's kappa of 0.600 and separates wake, NREM, and REM with kappa 0.762. The confusion matrix shows REM epochs are recalled at 92.2%, while most mistakes fall between adjacent or transitional stages, such as wake/N1, N1/N2, and N2/N3. Performance is better in younger participants and in those with a low apnea-hypopnea index, while remaining stable across a wide range of common outpatient medications. The authors read the age and apnea dependence as partly biological: EEG-defined N2 and N3 can share cardiorespiratory signatures, so some apparent errors may reflect real differences between autonomic state and EEG scoring.
Load-bearing premise
The load-bearing assumption is that splitting the 8,682 polysomnograms at the recording level, rather than at the patient level, keeps evaluation independent; if the same patient has several recordings and they appear in both training and testing, the reported agreement could be optimistic for new patients.
Editorial extensions
If this is right
- Sleep staging without EEG becomes practical in intensive care, ambulatory monitoring, and consumer wearables, as long as an ECG lead and a respiratory band are available.
- ECG plus abdominal effort is the signal pairing to prefer for cardiorespiratory staging; it beats ECG alone, abdominal effort alone, and ECG plus chest effort.
- Collapsing sleep into three states (awake, NREM, REM) gives a kappa of 0.762, so coarse sleep tracking is substantially more reliable than five-stage staging and is the more defensible target for wearables.
- Five-stage performance remains moderate even in severe sleep apnea (kappa 0.574), so the approach is not broken by common disordered breathing, though it may need extra signals or calibration for older and higher-AHI patients.
- With REM recall at 92.2%, cardiorespiratory features separate REM especially cleanly, so REM detection is a strong building block for downstream sleep analysis.
Reading between the lines
- A patient-level split—holding out every recording of a patient from training—would be the natural stress test of whether kappa 0.600 reflects generalization to unseen people rather than to unseen recordings.
- The network's medication robustness hints that transfer learning from this dataset could bootstrap sleep staging on ICU or wearable ECG, where full polysomnography is infeasible.
- The systematic disagreement between EEG-based and cardiorespiratory-based staging in older adults and severe apnea could itself become a marker of sleep fragmentation or altered autonomic-cardiorespiratory coupling.
- The same architecture could be adapted to predict continuous sleep-depth measures, such as cardiopulmonary coupling, rather than only discrete stages.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript describes deep neural network models (CNN plus bidirectional LSTM) for automatic sleep staging from ECG and/or respiratory effort signals. Using a clinical dataset of 8,682 polysomnograms from 7,208 patients, the authors train five networks (ECG, chest effort, abdominal effort, ECG+chest, ECG+abdomen) and report held-out test performance on 1,000 PSGs. The best model, ECG+abdomen, achieves a five-stage Cohen's kappa of 0.600 (95% CI 0.599--0.602) and a three-stage (W/NREM/REM) kappa of 0.762 (0.760--0.763). Subgroup analyses examine age, sex, BMI, AHI, PLM, and medication effects.
Significance. If the reported test-set estimates are unbiased, this is a valuable contribution: it demonstrates on a large, heterogeneous clinical population that cardiorespiratory signals carry substantial sleep-stage information and it provides a practical benchmark for non-EEG sleep staging. The paper includes useful comparisons to prior work and appropriate robustness checks (R-peak jitter, group-specific kappas, t-SNE, and signal examples). However, the validity of the headline kappa values depends critically on the independence of the test set from the training set; the current PSG-level split does not guarantee patient-level independence, so the numerical claims must be re-examined with a patient-disjoint split. The work is therefore significant conditionally on this being addressed.
major comments (2)
- [Methods, Training and Evaluating the Network] The random split is performed at the level of PSG ('We randomly split the PSGs into a training set of 6,682 PSGs, a validation set of 1,000 PSGs and a testing set of 1,000 PSGs'), but Table 1 reports 8,682 PSGs from 7,208 patients, so at least 1,474 PSGs are additional recordings from patients who already contribute another PSG. With a PSG-level split, it is highly likely that the same patient appears in both training and test sets, allowing the model to exploit patient-specific ECG/respiratory signatures or technician scoring style and inflate the reported kappa. The manuscript does not report the number of overlapping patients or any patient-stratified analysis. Since the abstract's conclusion is that performance is validated 'in a large population,' the test set must be independent at the level of the patient; please provide a patient-disjoint evaluation (e.g., retrain on a patient-stratified split or evaluate the current model on a subset of test PSGs from patients not seen in training) and report the kappa values and overlap statistics.
- [Methods, Training and Evaluating the Network] The bootstrap confidence intervals are described only as 'sampling with replacement 1,000 times' without specifying whether the resampling unit is an epoch, a PSG, or a patient. Given the hierarchical structure of the data (epochs within PSGs, PSGs within patients), epoch-level bootstrapping would severely underestimate the variance of kappa. Please specify the resampling unit and, if it is not the PSG, recompute the intervals by bootstrapping PSGs (or patients) to reflect the actual sampling variability. This is important because the narrow confidence intervals in Table 2 are used to support claims about the precision of the reported kappa values.
minor comments (5)
- [Results, Table 2] The statement 'Since the testing set has 1,000 PSGs (6.6×10^5 30-second epochs), the confidence interval is narrow. Therefore the differences between kappa values are all significant at 0.05 level' is not justified by the presented CIs; overlapping CIs do not imply significance, and a paired test (e.g., per-PSG difference in kappa with a bootstrap) is needed for this claim.
- [Methods, Dataset] Please specify how many PSGs per patient and the distribution of repeated PSGs; this is needed to understand the potential overlap between the training and test sets.
- [Figure S3] Please define the x-axis in Figure S3; the caption does not state the unit of the jitter standard deviation used in the robustness test.
- [Discussion] The sentence 'Only one prior study used more than 100 participants for training and evaluation' should be clarified with respect to Table 4, since the Radha et al. entry reports 352 ECG participants; please specify whether the count refers to studies with more than 100 participants that also used deep learning.
- [Abstract] The abstract says 'using a dataset including 8,682 polysomnographs' but the Methods section describes exclusion criteria; please clarify that 8,682 is the number of PSGs remaining after exclusions.
Circularity Check
No circularity: the central claim is an empirical benchmark measured on a held-out test set, not a derived or fit-defined result.
full rationale
The paper's central claim is that ECG plus abdominal respiratory effort achieves Cohen's kappa of 0.600 for five-stage sleep staging and 0.762 for wake/NREM/REM discrimination. This is not a derivation from an input; it is an empirical performance measurement on a separate 1,000-PSG test set. The networks are trained with cross-entropy on technician-scored PSG epochs, hyperparameters are selected on a validation set, and the reported kappa is computed on the test set. Nothing in the claimed result is defined in terms of itself, and no fitted parameter is renamed as a prediction. The only self-citations are to a prior EEG-staging paper by the same group and to the ADARRI R-peak artifact method; both are background or preprocessing tools and are not load-bearing for the ECG+ABD performance claim. The reviewer's concern about PSG-level splitting and possible same-patient overlap between training and test is a statistical-independence issue, not circularity: even if leakage inflates the kappa, the reported number remains an empirical measurement rather than a construct that reduces to its own inputs. No circular step can be exhibited, so the appropriate score is 0.
Assumptions & free parameters
free parameters (7)
- LSTM layers and hidden units per input modality =
ECG: 2 layers/20 nodes; CHEST and ECG+CHEST: 3 layers/100 nodes; ABD and ECG+ABD: 2 layers/100 nodes
- Dropout rate =
Not stated
- Context window length =
270 seconds (120 s on each side)
- LSTM training sequence length =
20 epochs (10 minutes)
- Artifact thresholds for ECG =
amplitude > 6 mV, SD < 5 uV, R peaks < 20/min
- Artifact thresholds for respiratory signals =
amplitude > 6 mV, SD < 10 uV
- Class-balance weights =
Inverse of epoch counts per stage
assumptions (5)
- domain assumption Technician-assigned AASM sleep stages are a valid ground truth for sleep staging.
- domain assumption The PSG-level random split produces independent training and test sets.
- domain assumption Pan-Tompkins R-peak detection and ADARRI artifact rejection are accurate on this dataset.
- ad hoc to paper The 270-second context window provides sufficient temporal context for sleep staging.
- standard math Cross-entropy loss with inverse-frequency class weighting is an appropriate objective.
Cite this review
Pith. "Pith review of Sleep Staging from Electrocardiography and Respiration with Deep Learning." pith.science (2026). https://pith.science/paper/GOZNB6EN
@misc{pith2026190811463,
author = {Pith},
title = {Pith review of: Sleep Staging from Electrocardiography and Respiration with Deep Learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/GOZNB6EN}},
note = {Machine review of arXiv:1908.11463}
}
read the original abstract
Study Objective: Sleep is reflected not only in the electroencephalogram but also in heart rhythms and breathing patterns. Therefore, we hypothesize that it is possible to accurately stage sleep based on the electrocardiogram (ECG) and respiratory signals. Methods: Using a dataset including 8,682 polysomnographs, we develop deep neural networks to stage sleep from ECG and respiratory signals. Five deep neural networks consisting of convolutional networks and long short-term memory networks are trained to stage sleep using heart and breathing, including the timing of R peaks from ECG, abdominal and chest respiratory effort, and the combinations of these signals. Results: ECG in combination with the abdominal respiratory effort achieve the best performance for staging all five sleep stages with a Cohen's kappa of 0.600 (95% confidence interval 0.599 -- 0.602); and 0.762 (0.760 -- 0.763) for discriminating awake vs. rapid eye movement vs. non-rapid eye movement sleep. The performance is better for young participants and for those with a low apnea-hypopnea index, while it is robust for commonly used outpatient medications. Conclusions: Our results validate that ECG and respiratory effort provide substantial information about sleep stages in a large population. It opens new possibilities in sleep research and applications where electroencephalography is not readily available or may be infeasible, such as in critically ill patients.
Forward citations
Cited by 1 Pith paper
-
How Far Can Wearable-Compatible Signals Go? A Controlled Decomposition of Non-EEG Sleep Staging
Non-EEG sleep staging is limited by how much sleep-stage information each 30-second epoch's physiology contains, not by the model: κ≈0.49 for lab cardiorespiratory, κ≈0.26 for a consumer watch, κ≈0.80 for EEG/EOG.
Reference graph
Works this paper leans on
-
[1]
The visual scoring of sleep in adults
Silber MH, Ancoli-Israel S, Bonnet MH, et al. The visual scoring of sleep in adults. J Clin Sleep Med. 2007; 3 (2): 121-131
work page 2007
-
[2]
Respiratory cycle-related EEG changes: response to CPAP
Chervin RD, Shelgikar AV, Burns JW. Respiratory cycle-related EEG changes: response to CPAP. Sleep. 2012; 35 (2): 203-209
work page 2012
-
[3]
Niizeki K, Saitoh T. Association Between Phase Coupling of Respiratory Sinus Arrhythmia and Slow Wave Brain Activity During Sleep. Front Physiol. 2018; 9: 1338
work page 2018
-
[4]
Modulations of Heart Rate, ECG, and Cardio- Respiratory Coupling Observed in Polysomnography
Penzel T, Kantelhardt JW, Bartsch RP, et al. Modulations of Heart Rate, ECG, and Cardio- Respiratory Coupling Observed in Polysomnography. Front Physiol. 2016; 7: 460
work page 2016
-
[5]
Thomas RJ, Mietus JE, Peng CK, et al. Relationship between delta power and the electrocardiogram-derived cardiopulmonary spectrogram: possible implications for assessing the effectiveness of sleep. Sleep Med. 2014; 15 (1): 125-131
work page 2014
-
[6]
A Respiration-Coupled Rhythm in the Rat Hippocampus Independent of Theta and Slow Oscillations
Lockmann AL, Laplagne DA, Leao RN, Tort AB. A Respiration-Coupled Rhythm in the Rat Hippocampus Independent of Theta and Slow Oscillations. J Neurosci. 2016; 36 (19): 5338-5352
work page 2016
-
[7]
Iellamo F, Placidi F, Marciani MG, et al. Baroreflex buffering of sympathetic activation during sleep: evidence from autonomic assessment of sleep macroarchitecture and microarchitecture. Hypertension. 2004; 43 (4): 814-819. 14
work page 2004
-
[8]
An electrocardiogram-based technique to assess cardiopulmonary coupling during sleep
Thomas RJ, Mietus JE, Peng CK, Goldberger AL. An electrocardiogram-based technique to assess cardiopulmonary coupling during sleep. Sleep. 2005; 28 (9): 1151-1161
work page 2005
Show all 27 references
-
[9]
Blood pressure surges in REM sleep: A mini review
Sei H. Blood pressure surges in REM sleep: A mini review. Pathophysiology. 2012; 19 (4): 233- 241
2012
-
[10]
Cardiopulmonary coupling spectrogram as an ambulatory clinical biomarker of sleep stability and quality in health, sleep apnea and insomnia
Thomas RJ, Wood C, Bianchi MT. Cardiopulmonary coupling spectrogram as an ambulatory clinical biomarker of sleep stability and quality in health, sleep apnea and insomnia. Sleep. 2017
2017
-
[11]
Sleep devices: wearables and nearables, informational and interventional, consumer and clinical
Bianchi MT. Sleep devices: wearables and nearables, informational and interventional, consumer and clinical. Metabolism. 2018; 84: 99-108
2018
-
[12]
Automatic sleep staging from ventilator signals in non-invasive ventilation
Sady CC, Freitas US, Portmann A, Muir JF, Letellier C, Aguirre LA. Automatic sleep staging from ventilator signals in non-invasive ventilation. Comput Biol Med. 2013; 43 (7): 833-839
2013
-
[13]
Automatic sleep staging based on ballistocardiographic signals recorded through bed sensors
Migliorini M, Bianchi AM, Nistico D, et al. Automatic sleep staging based on ballistocardiographic signals recorded through bed sensors. Conf Proc IEEE Eng Med Biol Soc. 2010; 2010: 3273-3276
2010
-
[14]
Validation of Contact-Free Sleep Monitoring Device with Comparison to Polysomnography
Tal A, Shinar Z, Shaki D, Codish S, Goldbart A. Validation of Contact-Free Sleep Monitoring Device with Comparison to Polysomnography. J Clin Sleep Med. 2017; 13 (3): 517-522
2017
-
[15]
Non-Contact Estimation of Sleep Staging
Zaffaroni A, Doheny EP, Gahan L, et al. Non-Contact Estimation of Sleep Staging. 2018; Singapore
2018
-
[16]
Atypical sleep in ventilated patients: empirical electroencephalography findings and the path toward revised ICU sleep scoring criteria
Watson PL, Pandharipande P, Gehlbach BK, et al. Atypical sleep in ventilated patients: empirical electroencephalography findings and the path toward revised ICU sleep scoring criteria. Crit Care Med. 2013; 41 (8): 1958-1967
2013
-
[17]
Expert-level sleep scoring with deep neural networks
Biswal S, Sun H, Goparaju B, Westover MB, Sun J, Bianchi MT. Expert-level sleep scoring with deep neural networks. Journal of the American Medical Informatics Association. 2018; 25 (12): 1643- 1650
2018
-
[18]
A real-time QRS detection algorithm
Pan J, Tompkins WJ. A real-time QRS detection algorithm. IEEE Trans Biomed Eng. 1985; 32 (3): 230-236
1985
-
[19]
ADARRI: a novel method to detect spurious R-peaks in the electrocardiogram for heart rate variability analysis in the intensive care unit
Rebergen DJ, Nagaraj SB, Rosenthal ES, Bianchi MT, van Putten MJ, Westover MB. ADARRI: a novel method to detect spurious R-peaks in the electrocardiogram for heart rate variability analysis in the intensive care unit. Journal of clinical monitoring and computing. 2018; 32 (1): 53-61
2018
-
[20]
Cardiologist-level arrhythmia detection and classification in ambulatory electrocardiograms using a deep neural network
Hannun AY, Rajpurkar P, Haghpanahi M, et al. Cardiologist-level arrhythmia detection and classification in ambulatory electrocardiograms using a deep neural network. Nature medicine. 2019; 25 (1): 65
2019
-
[21]
Heart Rate Analysis for Human Factors: Development and Validation of an Open Source Toolkit for Noisy Naturalistic Heart Rate Data
van Gent PaF, Haneen and Nes, Nicole and Arem, B. Heart Rate Analysis for Human Factors: Development and Validation of an Open Source Toolkit for Noisy Naturalistic Heart Rate Data. In: proceedings from the the 6th HUMANIST Conference; 2018; the Netherlands
2018
-
[22]
Measuring dissimilarity between respiratory effort signals based on uniform scaling for sleep staging
Long X, Yang J, Weysen T, et al. Measuring dissimilarity between respiratory effort signals based on uniform scaling for sleep staging. Physiological measurement. 2014; 35 (12): 2529
2014
-
[23]
Sleep stage classification with ECG and respiratory effort
Fonseca P, Long X, Radha M, Haakma R, Aarts RM, Rolink J. Sleep stage classification with ECG and respiratory effort. Physiological measurement. 2015; 36 (10): 2027
2015
-
[24]
Learning sleep stages from radio signals: A conditional adversarial architecture
Zhao M, Yue S, Katabi D, Jaakkola TS, Bianchi MT. Learning sleep stages from radio signals: A conditional adversarial architecture. In: proceedings from the Proceedings of the 34th International Conference on Machine Learning-Volume 70; 2017
2017
-
[25]
Sleep stage classification based on multi-level feature learning and recurrent neural networks via wearable device
Zhang X, Kou W, Eric I, et al. Sleep stage classification based on multi-level feature learning and recurrent neural networks via wearable device. Computers in biology and medicine. 2018; 103: 71-81
2018
-
[26]
LSTM knowledge transfer for HRV- based sleep staging
Radha M, Fonseca P, Ross M, Cerny A, Anderer P, Aarts RM. LSTM knowledge transfer for HRV- based sleep staging. arXiv preprint arXiv:180906221. 2018
2018
-
[27]
Odds ratio product of sleep EEG as a continuous measure of sleep state
Younes M, Ostrowski M, Soiferman M, et al. Odds ratio product of sleep EEG as a continuous measure of sleep state. Sleep. 2015; 38 (4): 641-654. 15 16 Supplementary Figures Figure S1. Illustration of signal segmentation. Figure S2. The histogram of C ohen’s kappa of individual...
2015
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.