Pith. sign in

REVIEW 2 major objections 5 minor 1 cited by

Sleep Staging from Electrocardiography and Respiration with Deep Learning

T0 review · 2 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Pith's one-line read Cardiorespiratory signals alone can stage sleep with moderate accuracy in a large clinical population, reaching a five-stage kappa of 0.600.

desk verdict Large-scale cardiorespiratory sleep staging study with a real patient-leakage concern; the headline kappa is likely somewhat optimistic but the qualitative claim that ECG and respiration carry substantial sleep-stage information holds. read the letter →

arxiv 1908.11463 v2 pith:GOZNB6EN submitted 2019-08-29 q-bio.QM eess.SPq-bio.NC

classification q-bio.QMeess.SPq-bio.NC
keywords sleepstagingdeeplearningelectrocardiographyrespirationpolysomnographyconvolutionalneuralnetworklongshort-termmemoryCohen'skappa
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to prove that sleep can be staged automatically from the heart's electrical activity and breathing patterns, without any EEG. Using 8,682 clinical polysomnograms (overnight sleep recordings), the authors train convolutional and long short-term memory networks and report the best five-stage performance with ECG plus abdominal respiratory effort: a chance-corrected agreement score (Cohen's kappa) of 0.600, and 0.762 when the task is collapsed to awake, NREM, and REM. Because the test set contains about 660,000 epochs, the differences between input-signal combinations are statistically clear-cut. If the result holds, sleep staging becomes possible from signals already recorded in hospitals and many wearable devices, which matters where EEG is unavailable or impractical.

What carries the argument

The engine is a two-part deep network: a residual convolutional network that learns features from a 270-second window centered on each 30-second epoch to be scored, followed by a bidirectional long short-term memory network that models temporal context across consecutive epochs. The ECG is represented as a binary sequence of R-peak times, and respiratory effort signals are downsampled to 10 Hz; artifactual epochs are removed for CNN training but kept for the LSTM so temporal continuity is preserved. The loss is a class-weighted cross-entropy that counteracts the dominance of N2, and the configuration is selected on a validation set. This architecture is what converts heart-rate and breathing dynamics into sleep-stage probabilities.

What would settle it

Re-run the experiment with a strict patient-level split—all recordings from each patient held out of training—and compare five-stage kappa; a meaningful drop below 0.600 would show the current number depends on overlapping patients between sets. Scoring an external cohort of ECG and respiratory recordings from a different institution or from a wearable device would provide the same sort of check.

Watch

Extended reading notes

Core claim

The paper claims that cardiorespiratory signals contain enough information for moderately accurate automatic sleep staging in a large, heterogeneous adult population. Combining R-peak timing from the ECG with the abdominal respiratory effort trace, a deep network reproduces the five clinical sleep stages with a Cohen's kappa of 0.600 and separates wake, NREM, and REM with kappa 0.762. The confusion matrix shows REM epochs are recalled at 92.2%, while most mistakes fall between adjacent or transitional stages, such as wake/N1, N1/N2, and N2/N3. Performance is better in younger participants and in those with a low apnea-hypopnea index, while remaining stable across a wide range of common outpatient medications. The authors read the age and apnea dependence as partly biological: EEG-defined N2 and N3 can share cardiorespiratory signatures, so some apparent errors may reflect real differences between autonomic state and EEG scoring.

Load-bearing premise

The load-bearing assumption is that splitting the 8,682 polysomnograms at the recording level, rather than at the patient level, keeps evaluation independent; if the same patient has several recordings and they appear in both training and testing, the reported agreement could be optimistic for new patients.

Editorial extensions

If this is right

  • Sleep staging without EEG becomes practical in intensive care, ambulatory monitoring, and consumer wearables, as long as an ECG lead and a respiratory band are available.
  • ECG plus abdominal effort is the signal pairing to prefer for cardiorespiratory staging; it beats ECG alone, abdominal effort alone, and ECG plus chest effort.
  • Collapsing sleep into three states (awake, NREM, REM) gives a kappa of 0.762, so coarse sleep tracking is substantially more reliable than five-stage staging and is the more defensible target for wearables.
  • Five-stage performance remains moderate even in severe sleep apnea (kappa 0.574), so the approach is not broken by common disordered breathing, though it may need extra signals or calibration for older and higher-AHI patients.
  • With REM recall at 92.2%, cardiorespiratory features separate REM especially cleanly, so REM detection is a strong building block for downstream sleep analysis.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A patient-level split—holding out every recording of a patient from training—would be the natural stress test of whether kappa 0.600 reflects generalization to unseen people rather than to unseen recordings.
  • The network's medication robustness hints that transfer learning from this dataset could bootstrap sleep staging on ICU or wearable ECG, where full polysomnography is infeasible.
  • The systematic disagreement between EEG-based and cardiorespiratory-based staging in older adults and severe apnea could itself become a marker of sleep fragmentation or altered autonomic-cardiorespiratory coupling.
  • The same architecture could be adapted to predict continuous sleep-depth measures, such as cardiopulmonary coupling, rather than only discrete stages.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The manuscript describes deep neural network models (CNN plus bidirectional LSTM) for automatic sleep staging from ECG and/or respiratory effort signals. Using a clinical dataset of 8,682 polysomnograms from 7,208 patients, the authors train five networks (ECG, chest effort, abdominal effort, ECG+chest, ECG+abdomen) and report held-out test performance on 1,000 PSGs. The best model, ECG+abdomen, achieves a five-stage Cohen's kappa of 0.600 (95% CI 0.599--0.602) and a three-stage (W/NREM/REM) kappa of 0.762 (0.760--0.763). Subgroup analyses examine age, sex, BMI, AHI, PLM, and medication effects.

Significance. If the reported test-set estimates are unbiased, this is a valuable contribution: it demonstrates on a large, heterogeneous clinical population that cardiorespiratory signals carry substantial sleep-stage information and it provides a practical benchmark for non-EEG sleep staging. The paper includes useful comparisons to prior work and appropriate robustness checks (R-peak jitter, group-specific kappas, t-SNE, and signal examples). However, the validity of the headline kappa values depends critically on the independence of the test set from the training set; the current PSG-level split does not guarantee patient-level independence, so the numerical claims must be re-examined with a patient-disjoint split. The work is therefore significant conditionally on this being addressed.

major comments (2)
  1. [Methods, Training and Evaluating the Network] The random split is performed at the level of PSG ('We randomly split the PSGs into a training set of 6,682 PSGs, a validation set of 1,000 PSGs and a testing set of 1,000 PSGs'), but Table 1 reports 8,682 PSGs from 7,208 patients, so at least 1,474 PSGs are additional recordings from patients who already contribute another PSG. With a PSG-level split, it is highly likely that the same patient appears in both training and test sets, allowing the model to exploit patient-specific ECG/respiratory signatures or technician scoring style and inflate the reported kappa. The manuscript does not report the number of overlapping patients or any patient-stratified analysis. Since the abstract's conclusion is that performance is validated 'in a large population,' the test set must be independent at the level of the patient; please provide a patient-disjoint evaluation (e.g., retrain on a patient-stratified split or evaluate the current model on a subset of test PSGs from patients not seen in training) and report the kappa values and overlap statistics.
  2. [Methods, Training and Evaluating the Network] The bootstrap confidence intervals are described only as 'sampling with replacement 1,000 times' without specifying whether the resampling unit is an epoch, a PSG, or a patient. Given the hierarchical structure of the data (epochs within PSGs, PSGs within patients), epoch-level bootstrapping would severely underestimate the variance of kappa. Please specify the resampling unit and, if it is not the PSG, recompute the intervals by bootstrapping PSGs (or patients) to reflect the actual sampling variability. This is important because the narrow confidence intervals in Table 2 are used to support claims about the precision of the reported kappa values.
minor comments (5)
  1. [Results, Table 2] The statement 'Since the testing set has 1,000 PSGs (6.6×10^5 30-second epochs), the confidence interval is narrow. Therefore the differences between kappa values are all significant at 0.05 level' is not justified by the presented CIs; overlapping CIs do not imply significance, and a paired test (e.g., per-PSG difference in kappa with a bootstrap) is needed for this claim.
  2. [Methods, Dataset] Please specify how many PSGs per patient and the distribution of repeated PSGs; this is needed to understand the potential overlap between the training and test sets.
  3. [Figure S3] Please define the x-axis in Figure S3; the caption does not state the unit of the jitter standard deviation used in the robustness test.
  4. [Discussion] The sentence 'Only one prior study used more than 100 participants for training and evaluation' should be clarified with respect to Table 4, since the Radha et al. entry reports 352 ECG participants; please specify whether the count refers to studies with more than 100 participants that also used deep learning.
  5. [Abstract] The abstract says 'using a dataset including 8,682 polysomnographs' but the Methods section describes exclusion criteria; please clarify that 8,682 is the number of PSGs remaining after exclusions.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the central claim is an empirical benchmark measured on a held-out test set, not a derived or fit-defined result.

full rationale

The paper's central claim is that ECG plus abdominal respiratory effort achieves Cohen's kappa of 0.600 for five-stage sleep staging and 0.762 for wake/NREM/REM discrimination. This is not a derivation from an input; it is an empirical performance measurement on a separate 1,000-PSG test set. The networks are trained with cross-entropy on technician-scored PSG epochs, hyperparameters are selected on a validation set, and the reported kappa is computed on the test set. Nothing in the claimed result is defined in terms of itself, and no fitted parameter is renamed as a prediction. The only self-citations are to a prior EEG-staging paper by the same group and to the ADARRI R-peak artifact method; both are background or preprocessing tools and are not load-bearing for the ECG+ABD performance claim. The reviewer's concern about PSG-level splitting and possible same-patient overlap between training and test is a statistical-independence issue, not circularity: even if leakage inflates the kappa, the reported number remains an empirical measurement rather than a construct that reduces to its own inputs. No circular step can be exhibited, so the appropriate score is 0.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

The central claim depends on the quality of the clinical dataset, the choice of ground truth, and the held-out evaluation. There are no invented entities. The free parameters are mainly network hyperparameters tuned on the validation set and artifact thresholds. The most important assumption is that the PSG-level split yields independent test data, which is questionable because of repeated patients.

free parameters (7)
  • LSTM layers and hidden units per input modality = ECG: 2 layers/20 nodes; CHEST and ECG+CHEST: 3 layers/100 nodes; ABD and ECG+ABD: 2 layers/100 nodes
    Selected by minimizing validation cross-entropy; these architecture choices affect the reported kappa.
  • Dropout rate = Not stated
    Tuned on the validation set, but the final value is not reported in the paper.
  • Context window length = 270 seconds (120 s on each side)
    Chosen to capture longer-scale cardiorespiratory dynamics; not optimized in the paper.
  • LSTM training sequence length = 20 epochs (10 minutes)
    Used for training the LSTM; fixed by the authors.
  • Artifact thresholds for ECG = amplitude > 6 mV, SD < 5 uV, R peaks < 20/min
    Used to exclude non-physiological ECG epochs; affects which data enter training.
  • Artifact thresholds for respiratory signals = amplitude > 6 mV, SD < 10 uV
    Used to exclude non-physiological respiratory epochs.
  • Class-balance weights = Inverse of epoch counts per stage
    Used in the loss function to counter class imbalance; fixed by design.
assumptions (5)
  • domain assumption Technician-assigned AASM sleep stages are a valid ground truth for sleep staging.
    The entire evaluation treats expert PSG scoring as the reference standard; the paper assumes this is the correct target.
  • domain assumption The PSG-level random split produces independent training and test sets.
    The paper splits by PSG rather than patient, but with 8,682 PSGs from 7,208 patients, some patients may appear in both splits, violating independence.
  • domain assumption Pan-Tompkins R-peak detection and ADARRI artifact rejection are accurate on this dataset.
    These external methods are used as preprocessing without validation on this specific cohort.
  • ad hoc to paper The 270-second context window provides sufficient temporal context for sleep staging.
    This window size is a design choice, not derived from theory or prior evidence.
  • standard math Cross-entropy loss with inverse-frequency class weighting is an appropriate objective.
    Standard classification objective; no special assumption beyond usual practice.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Sleep Staging from Electrocardiography and Respiration with Deep Learning." pith.science (2026). https://pith.science/paper/GOZNB6EN

@misc{pith2026190811463,
  author       = {Pith},
  title        = {Pith review of: Sleep Staging from Electrocardiography and Respiration with Deep Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GOZNB6EN}},
  note         = {Machine review of arXiv:1908.11463}
}
read the original abstract

Study Objective: Sleep is reflected not only in the electroencephalogram but also in heart rhythms and breathing patterns. Therefore, we hypothesize that it is possible to accurately stage sleep based on the electrocardiogram (ECG) and respiratory signals. Methods: Using a dataset including 8,682 polysomnographs, we develop deep neural networks to stage sleep from ECG and respiratory signals. Five deep neural networks consisting of convolutional networks and long short-term memory networks are trained to stage sleep using heart and breathing, including the timing of R peaks from ECG, abdominal and chest respiratory effort, and the combinations of these signals. Results: ECG in combination with the abdominal respiratory effort achieve the best performance for staging all five sleep stages with a Cohen's kappa of 0.600 (95% confidence interval 0.599 -- 0.602); and 0.762 (0.760 -- 0.763) for discriminating awake vs. rapid eye movement vs. non-rapid eye movement sleep. The performance is better for young participants and for those with a low apnea-hypopnea index, while it is robust for commonly used outpatient medications. Conclusions: Our results validate that ECG and respiratory effort provide substantial information about sleep stages in a large population. It opens new possibilities in sleep research and applications where electroencephalography is not readily available or may be infeasible, such as in critically ill patients.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. How Far Can Wearable-Compatible Signals Go? A Controlled Decomposition of Non-EEG Sleep Staging

    cs.HC 2026-07 conditional novelty 6.0 of 10

    Non-EEG sleep staging is limited by how much sleep-stage information each 30-second epoch's physiology contains, not by the model: κ≈0.49 for lab cardiorespiratory, κ≈0.26 for a consumer watch, κ≈0.80 for EEG/EOG.

Reference graph

Works this paper leans on

27 extracted references · 27 canonical work pages · cited by 1 Pith paper

  1. [1]

    The visual scoring of sleep in adults

    Silber MH, Ancoli-Israel S, Bonnet MH, et al. The visual scoring of sleep in adults. J Clin Sleep Med. 2007; 3 (2): 121-131

  2. [2]

    Respiratory cycle-related EEG changes: response to CPAP

    Chervin RD, Shelgikar AV, Burns JW. Respiratory cycle-related EEG changes: response to CPAP. Sleep. 2012; 35 (2): 203-209

  3. [3]

    Association Between Phase Coupling of Respiratory Sinus Arrhythmia and Slow Wave Brain Activity During Sleep

    Niizeki K, Saitoh T. Association Between Phase Coupling of Respiratory Sinus Arrhythmia and Slow Wave Brain Activity During Sleep. Front Physiol. 2018; 9: 1338

  4. [4]

    Modulations of Heart Rate, ECG, and Cardio- Respiratory Coupling Observed in Polysomnography

    Penzel T, Kantelhardt JW, Bartsch RP, et al. Modulations of Heart Rate, ECG, and Cardio- Respiratory Coupling Observed in Polysomnography. Front Physiol. 2016; 7: 460

  5. [5]

    Relationship between delta power and the electrocardiogram-derived cardiopulmonary spectrogram: possible implications for assessing the effectiveness of sleep

    Thomas RJ, Mietus JE, Peng CK, et al. Relationship between delta power and the electrocardiogram-derived cardiopulmonary spectrogram: possible implications for assessing the effectiveness of sleep. Sleep Med. 2014; 15 (1): 125-131

  6. [6]

    A Respiration-Coupled Rhythm in the Rat Hippocampus Independent of Theta and Slow Oscillations

    Lockmann AL, Laplagne DA, Leao RN, Tort AB. A Respiration-Coupled Rhythm in the Rat Hippocampus Independent of Theta and Slow Oscillations. J Neurosci. 2016; 36 (19): 5338-5352

  7. [7]

    Baroreflex buffering of sympathetic activation during sleep: evidence from autonomic assessment of sleep macroarchitecture and microarchitecture

    Iellamo F, Placidi F, Marciani MG, et al. Baroreflex buffering of sympathetic activation during sleep: evidence from autonomic assessment of sleep macroarchitecture and microarchitecture. Hypertension. 2004; 43 (4): 814-819. 14

  8. [8]

    An electrocardiogram-based technique to assess cardiopulmonary coupling during sleep

    Thomas RJ, Mietus JE, Peng CK, Goldberger AL. An electrocardiogram-based technique to assess cardiopulmonary coupling during sleep. Sleep. 2005; 28 (9): 1151-1161

Show all 27 references
  1. [9]

    Blood pressure surges in REM sleep: A mini review

    Sei H. Blood pressure surges in REM sleep: A mini review. Pathophysiology. 2012; 19 (4): 233- 241

  2. [10]

    Cardiopulmonary coupling spectrogram as an ambulatory clinical biomarker of sleep stability and quality in health, sleep apnea and insomnia

    Thomas RJ, Wood C, Bianchi MT. Cardiopulmonary coupling spectrogram as an ambulatory clinical biomarker of sleep stability and quality in health, sleep apnea and insomnia. Sleep. 2017

  3. [11]

    Sleep devices: wearables and nearables, informational and interventional, consumer and clinical

    Bianchi MT. Sleep devices: wearables and nearables, informational and interventional, consumer and clinical. Metabolism. 2018; 84: 99-108

  4. [12]

    Automatic sleep staging from ventilator signals in non-invasive ventilation

    Sady CC, Freitas US, Portmann A, Muir JF, Letellier C, Aguirre LA. Automatic sleep staging from ventilator signals in non-invasive ventilation. Comput Biol Med. 2013; 43 (7): 833-839

  5. [13]

    Automatic sleep staging based on ballistocardiographic signals recorded through bed sensors

    Migliorini M, Bianchi AM, Nistico D, et al. Automatic sleep staging based on ballistocardiographic signals recorded through bed sensors. Conf Proc IEEE Eng Med Biol Soc. 2010; 2010: 3273-3276

  6. [14]

    Validation of Contact-Free Sleep Monitoring Device with Comparison to Polysomnography

    Tal A, Shinar Z, Shaki D, Codish S, Goldbart A. Validation of Contact-Free Sleep Monitoring Device with Comparison to Polysomnography. J Clin Sleep Med. 2017; 13 (3): 517-522

  7. [15]

    Non-Contact Estimation of Sleep Staging

    Zaffaroni A, Doheny EP, Gahan L, et al. Non-Contact Estimation of Sleep Staging. 2018; Singapore

  8. [16]

    Atypical sleep in ventilated patients: empirical electroencephalography findings and the path toward revised ICU sleep scoring criteria

    Watson PL, Pandharipande P, Gehlbach BK, et al. Atypical sleep in ventilated patients: empirical electroencephalography findings and the path toward revised ICU sleep scoring criteria. Crit Care Med. 2013; 41 (8): 1958-1967

  9. [17]

    Expert-level sleep scoring with deep neural networks

    Biswal S, Sun H, Goparaju B, Westover MB, Sun J, Bianchi MT. Expert-level sleep scoring with deep neural networks. Journal of the American Medical Informatics Association. 2018; 25 (12): 1643- 1650

  10. [18]

    A real-time QRS detection algorithm

    Pan J, Tompkins WJ. A real-time QRS detection algorithm. IEEE Trans Biomed Eng. 1985; 32 (3): 230-236

  11. [19]

    ADARRI: a novel method to detect spurious R-peaks in the electrocardiogram for heart rate variability analysis in the intensive care unit

    Rebergen DJ, Nagaraj SB, Rosenthal ES, Bianchi MT, van Putten MJ, Westover MB. ADARRI: a novel method to detect spurious R-peaks in the electrocardiogram for heart rate variability analysis in the intensive care unit. Journal of clinical monitoring and computing. 2018; 32 (1): 53-61

  12. [20]

    Cardiologist-level arrhythmia detection and classification in ambulatory electrocardiograms using a deep neural network

    Hannun AY, Rajpurkar P, Haghpanahi M, et al. Cardiologist-level arrhythmia detection and classification in ambulatory electrocardiograms using a deep neural network. Nature medicine. 2019; 25 (1): 65

  13. [21]

    Heart Rate Analysis for Human Factors: Development and Validation of an Open Source Toolkit for Noisy Naturalistic Heart Rate Data

    van Gent PaF, Haneen and Nes, Nicole and Arem, B. Heart Rate Analysis for Human Factors: Development and Validation of an Open Source Toolkit for Noisy Naturalistic Heart Rate Data. In: proceedings from the the 6th HUMANIST Conference; 2018; the Netherlands

  14. [22]

    Measuring dissimilarity between respiratory effort signals based on uniform scaling for sleep staging

    Long X, Yang J, Weysen T, et al. Measuring dissimilarity between respiratory effort signals based on uniform scaling for sleep staging. Physiological measurement. 2014; 35 (12): 2529

  15. [23]

    Sleep stage classification with ECG and respiratory effort

    Fonseca P, Long X, Radha M, Haakma R, Aarts RM, Rolink J. Sleep stage classification with ECG and respiratory effort. Physiological measurement. 2015; 36 (10): 2027

  16. [24]

    Learning sleep stages from radio signals: A conditional adversarial architecture

    Zhao M, Yue S, Katabi D, Jaakkola TS, Bianchi MT. Learning sleep stages from radio signals: A conditional adversarial architecture. In: proceedings from the Proceedings of the 34th International Conference on Machine Learning-Volume 70; 2017

  17. [25]

    Sleep stage classification based on multi-level feature learning and recurrent neural networks via wearable device

    Zhang X, Kou W, Eric I, et al. Sleep stage classification based on multi-level feature learning and recurrent neural networks via wearable device. Computers in biology and medicine. 2018; 103: 71-81

  18. [26]

    LSTM knowledge transfer for HRV- based sleep staging

    Radha M, Fonseca P, Ross M, Cerny A, Anderer P, Aarts RM. LSTM knowledge transfer for HRV- based sleep staging. arXiv preprint arXiv:180906221. 2018

  19. [27]

    Odds ratio product of sleep EEG as a continuous measure of sleep state

    Younes M, Ostrowski M, Soiferman M, et al. Odds ratio product of sleep EEG as a continuous measure of sleep state. Sleep. 2015; 38 (4): 641-654. 15 16 Supplementary Figures Figure S1. Illustration of signal segmentation. Figure S2. The histogram of C ohen’s kappa of individual...

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.