{"id":"c4ba21ee-9bdf-4b05-a4d6-14feeae695ae","arxiv_id":"2607.19441","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Non-EEG sleep staging is limited by how much sleep-stage information each 30-second epoch's physiology contains, not by the model: κ≈0.49 for lab cardiorespiratory, κ≈0.26 for a consumer watch, κ≈0.80 for EEG/EOG.","lead":"This paper breaks sleep-staging from wearables into four separate choices—sensors, features, timing, and confidence—and measures each one's contribution. It finds that lab-grade heart/lung signals can stage sleep only coarsely, consumer watches even more so, and the large gap to EEG is mostly missing brain information.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The Δκ=+0.304 EEG gap is inferred from a cross-dataset subtraction (SHHS non-EEG vs Sleep-EDF-20 EEG); running the same model on SHHS EEG/EOG would settle whether the gap is modality or cohort.","rationale":"The reader's weakest assumption identifies exactly the load-bearing issue: the EEG gap is computed by comparing non-EEG SHHS results to Sleep-EDF-20 EEG results across datasets, cohorts, and feature pipelines. I agree that this is the least secure part of the paper's headline conclusion. The internal decomposition on SHHS is well controlled: the signal-source ablations, Viterbi marginal gain, negative control, and subject-overlap audit are credible, and they support the weaker claim that within non-EEG signals, per-epoch physiological information matters more than temporal smoothing. But the stronger claim—that Δκ=+0.304 represents missing cortical information rather than dataset or cohort differences—requires a matched EEG baseline. The paper itself acknowledges the cross-dataset reference, but frames it only as making the gap conservative. That is not guaranteed: Sleep-EDF-20's small, healthy cohort and cleaner labels could inflate the EEG result, while SHHS's older, apneic cohort and noisier scoring could deflate the non-EEG result. The concrete test I propose directly removes these confounds by running the same model on SHHS EEG/EOG, using the same labels and folds. If that test were run and confirmed the gap, the central claim would be substantially stronger. If it did not, the conclusion would need to be narrowed to a within-SHHS statement. Since the internal decomposition already supports the main practical message, the appropriate verdict remains CONDITIONAL, matching the reader's assessment.","tokens_in":12002,"tokens_out":5674,"duration_ms":52936,"concrete_test":"Apply the exact same Mamba2 pipeline—including per-subject median/MAD normalization, five-fold subject-disjoint splits, inverse-frequency class weights, Viterbi with the same λ grid, and the same spectral feature extractor—to the EEG and EOG channels in the SHHS records. Compute per-subject κ and bootstrap CIs, and compare with the SHHS combined non-EEG κ=0.492 and the Sleep-EDF-20 κ=0.796. If matched-cohort SHHS EEG/EOG κ is substantially above 0.492 (e.g., >0.7), the modality-gap claim is confirmed within-cohort; if it is near 0.492, the Δκ=+0.304 is largely a cross-dataset artifact and the headline conclusion should be revised to a within-SHHS claim only.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that the residual gap to EEG is 'signal modality rather than model capacity' depends entirely on treating Sleep-EDF-20 κ=0.796 as the EEG ceiling for the same modeling pipeline. But the comparison confounds signal modality with dataset identity: Sleep-EDF-20 is 20 healthy subjects with single-night PSG and simpler spectral features; SHHS is 195 older community subjects with suspected sleep-disordered breathing and a different scoring environment. The paper's Limitations call the reference cross-dataset and 'conservative,' but that assumes the only direction of the confound is understated EEG features. The opposite confound is equally plausible: cleaner subjects/labels in Sleep-EDF-20 could inflate EEG κ relative to what the same architecture would achieve on SHHS EEG. Since SHHS PSG records include EEG/EOG channels under the same protocol and labels used for the non-EEG arms, the Δκ=+0.304 is directly testable within a matched cohort. Without that test, the paper's headline conclusion about irrecoverable cortical information is not yet established; the internal SHHS ablations (Viterbi +0.040, representation +0.078) remain valid, but they do not by themselves quantify the cortical gap.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a four-layer controlled decomposition framework for non-EEG sleep staging, separating signal source, physiological representation, temporal prior, and decision layers. Using the same compact Mamba2 model across a signal-quality ladder (Apple Watch HR/ACC, SHHS ECG/respiratory/SpO2, and Sleep-EDF-20 EEG+EOG), it reports that physiological representation gives the largest within-pipeline gain (Δκ=+0.078/0.089), temporal decoding adds only Δκ=+0.040, and the residual gap to EEG+EOG (Δκ=+0.304) is attributed to signal modality rather than model capacity. Confidence-based abstention is shown to improve κ from 0.452 to 0.512 at 80% coverage, and a label-shuffled control collapses to κ≈0. The paper argues that non-EEG sleep staging is limited by per-epoch physiological information content, not by temporal modeling or model capacity.","tokens_in":12338,"tokens_out":4139,"duration_ms":36228,"significance":"If the central claims hold, the paper contributes a useful diagnostic methodology for wearable sleep staging and provides evidence that non-EEG staging may be inherently bounded by autonomic/respiratory surrogates. The subject-disjoint splits, label-shuffled negative control, and channel ablations are well-designed internal controls that support the within-SHHS decomposition. The abstention analysis is a practical strength with clear translational relevance. However, the headline EEG-gap conclusion depends on a cross-dataset comparison (SHHS non-EEG vs. Sleep-EDF-20 EEG+EOG), which is not a controlled measurement of signal modality. The paper's central claim is therefore not yet established, though the internal ablations are valuable and the gap can be directly tested using SHHS EEG/EOG channels already available under the same protocol.","major_comments":[{"comment":"Table II is internally inconsistent with the text and Table III. Row 1 is labeled 'ECG physiology alone (Viterbi)' with κ=0.403, row 2 '+ Resp/SpO2 physiology' gives κ=0.452 (Δ=+0.049), and row 3 '+ Viterbi temporal prior' gives κ=0.492 (Δ=+0.040). But the text and Table III report ECG argmax=0.373, combined argmax=0.452, and combined Viterbi=0.492. The table appears to mix argmax and Viterbi stages, so the marginal gains are not additive as claimed (0.403→0.452→0.492 is not the same as 0.373→0.452→0.492). The representation gain should be computed from argmax (0.452−0.373=+0.079), not 0.403→0.452. Please correct the table and ensure the four-layer decomposition's marginal Δκ values are consistent with the reported numbers in Tables III and the text.","section":"§IV-A, Table II"},{"comment":"The Viterbi transition weight λ is selected as the 'best Viterbi result per channel configuration' from λ∈{0.1,0.3,0.5,1.0}. No validation procedure is described; if λ is chosen based on test-fold κ, the reported temporal-prior gain (Δκ=+0.040) is optimistically biased. This is load-bearing for the conclusion that temporal modeling contributes only modestly and that the per-epoch representation is the binding constraint. Report results for all λ values or a λ chosen on a held-out validation split. Table III states 'Viterbi at λ=1.0', which suggests a fixed choice, but Section III-C says best per configuration; please disambiguate.","section":"§III-C, Layer 3"},{"comment":"The headline claim that Δκ=+0.304 is attributable to 'signal modality rather than model capacity' rests on comparing SHHS non-EEG results with Sleep-EDF-20 EEG+EOG results. This comparison confounds signal modality with dataset identity, cohort characteristics (20 healthy young subjects vs. 195 older community subjects with suspected sleep-disordered breathing), feature engineering, and scoring environment. The Limitations section acknowledges this but asserts the EEG ceiling is 'conservative'; that is only one direction of the confound. Cleaner subjects and labels in Sleep-EDF-20 could inflate the EEG κ relative to what the same architecture would achieve on SHHS EEG/EOG. Since SHHS PSG includes EEG/EOG channels under the same protocol and labels used for the non-EEG arms, the paper should run the identical Mamba2 pipeline on SHHS EEG/EOG to produce a matched-cohort reference. Without t","section":"§IV-A, Table II row 4; §V; §VI"}],"minor_comments":[{"comment":"'Reflects missing cortical information rather than temporal modeling alone' is too strong given the cross-dataset reference; the abstract should say 'is consistent with missing cortical information' or similar until a matched-cohort EEG comparison is provided.","section":"Abstract and §IV-A"},{"comment":"Row 4 uses the word 'Irrecoverable' for the EEG/EOG ceiling. This is an overstatement even under the authors' interpretation; the gap is 'not recovered by the tested non-EEG features and temporal model,' not proven irrecoverable.","section":"Table II"},{"comment":"The text states the representation gain is Δκ=+0.089 in Viterbi, but Table II reports +0.049 for the same step. Please align these values.","section":"§IV-A"},{"comment":"The SHHS cohort is described as 'rpoint200' but the analysis uses 195 subjects; the dataset description would benefit from explaining why 5 of the 200 subjects were excluded and whether any sensitivity analysis was performed.","section":"§III-B"},{"comment":"The coverage-κ curve would benefit from reporting the number/percentage of epochs abstained at each coverage level per stage, especially for N1, to support the 'physiologically structured uncertainty' claim beyond mean confidence.","section":"§IV-C, Fig. 3"}],"recommendation":"major_revision","confidential_remarks":"The paper has a strong within-SHHS internal decomposition and a clean negative control, but the central EEG-gap attribution is not yet supported because it relies on a cross-dataset comparison. The fix is straightforward and within scope: run the same model on SHHS EEG/EOG channels to create a matched-cohort reference. The Table II inconsistency and λ-selection issue also need correction. I would support publication after these revisions."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth reading: it puts one fixed model (Mamba2, ~120k params) through a four-layer controlled decomposition across consumer HR/ACC, SHHS cardiorespiratory, and Sleep-EDF-20 EEG/EOG, and reports marginal kappa gains for representation, temporal prior, and abstention. The within-SHHS work is clean: representation adds more than Viterbi, the label-shuffled control collapses to zero, subject-overlap is audited. That part supports the broad conclusion that per-epoch physiological information, not temporal modeling, is the binding constraint for non-EEG staging.\n\nThe soft spot is the headline number. Delta kappa = +0.304 is computed by subtracting SHHS non-EEG kappa = 0.492 from Sleep-EDF-20 EEG kappa = 0.796. That is a cross-dataset, cross-cohort, cross-feature subtraction. Sleep-EDF-20 is 20 healthy subjects with single-night PSG and simpler spectral features; SHHS is 195 older community subjects with suspected sleep-disordered breathing. The paper calls the EEG ceiling 'conservative,' but the confound could cut either way: cleaner labels and subjects in Sleep-EDF-20 could inflate EEG kappa just as plausibly as simpler features deflate it. The authors note that SHHS PSG includes EEG/EOG channels under the same protocol and labels; running the same model on SHHS EEG/EOG would settle whether the gap is modality or cohort. Without that, I would treat the +0.304 as an upper-anchor comparison, not a measured modality gap. The internal ablations stand regardless.\n\nMinor issues: Viterbi lambda is selected as 'best per configuration' with no stated validation procedure; given the gain is only +0.040, this is unlikely to change the conclusion but should be nailed down. No code is released. The single-architecture caveat is acknowledged.\n\nWho gets value: anyone working on wearable sleep staging or on evaluation methodology for physiological signal pipelines. The decomposition framework is reusable. It deserves peer review after the EEG/EOG matched-cohort test is either run or explicitly reframed as a cross-dataset anchor. I would send it out rather than desk reject.","headline":"Solid decomposition of non-EEG sleep staging, but the headline EEG-gap number rests on a cross-dataset comparison that should be fixed before it is cited as a measurement of cortical information.","tokens_in":12774,"tokens_out":1908,"would_cite":true,"duration_ms":17229,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Non-EEG sleep staging is limited by missing cortical information, not by model capacity.","keywords":["sleep staging","wearable sensing","non-EEG signals","controlled decomposition","cardiorespiratory physiology","confidence-based abstention","Cohen's kappa","Mamba2"],"falsifier":"Simultaneously record EEG and non-EEG signals in the same subjects, run both through identical feature engineering and the same compact model, and compare the within-subject gap; a gap much smaller than 0.304 would show the headline number was inflated by dataset differences.","tokens_in":1450,"feed_emoji":"😴","tokens_out":3195,"duration_ms":57691,"temperature":0.7,"pith_summary":"This paper asks how much sleep-stage information can be recovered from non-EEG signals and where the performance loss actually originates. Using a signal-quality ladder—consumer wrist HR/ACC, laboratory ECG/respiration/SpO2, and EEG+EOG—the authors hold a single compact model fixed and decompose the pipeline into signal source, physiological representation, temporal prior, and decision layers. They find that adding richer physiology helps most (+0.078 kappa), temporal smoothing helps modestly (+0.040), and the remaining gap to EEG is much larger (+0.304). They conclude that non-EEG staging is information-limited, not model-limited, and that confidence-based abstention is a practical way to make wearable staging trustworthy.","feed_headline":"Non-EEG sleep staging tops out near kappa=0.49; missing cortex is why","feed_subtitle":"A four-layer decomposition shows signal physiology, not model size, binds wearable staging; abstention makes it useful.","key_machinery":"The four-layer controlled decomposition framework, which separates sleep staging into signal source, physiological representation, temporal prior, and decision layers, and quantifies each layer's marginal contribution as the change in Cohen's kappa when that layer is added. The same compact Mamba2 state-space model with multi-scale temporal evidence aggregation is applied across all signal tiers, making signal modality the only variable. The coverage-kappa abstention curve, which ranks epochs by maximum softmax probability and computes agreement at decreasing coverage levels, converts model confidence into an operational tool for deciding when to report a stage and when to abstain.","core_discovery":"Using the same Mamba2 model across all tiers, the paper reports that laboratory cardiorespiratory signals (ECG, respiration, SpO2) reach kappa=0.492 with Viterbi decoding, while EEG+EOG on Sleep-EDF-20 reaches kappa=0.796. The gap of 0.304 is interpreted as the information lost when cortical signals are absent, because the summed contribution of all non-EEG layers is only about +0.089. Consumer HR/ACC reaches only kappa=0.255, so the wearable penalty is split between degraded signal quality and the fundamental absence of EEG. Finally, the model's per-epoch confidence is well-calibrated: dropping the 20% lowest-confidence epochs raises kappa from 0.452 to 0.512, and at 50% coverage kappa=0.62","pith_inferences":["Because the EEG reference comes from a different cohort with different sensors and feature pipelines, a same-subject paired recording might produce a different magnitude of the modality gap; the within-SHHS ablations still support the representation-over-model conclusion.","Extending the decomposition to a transformer or a deeper CNN would test whether the layer contributions are architecture-dependent, as the paper's single-architecture limitation suggests.","If the coverage-kappa curve holds in a clinical cohort, abstention could be used to triage which nights deserve full PSG referral, increasing the utility of at-home monitoring."],"forward_implications":["Investing in larger or deeper sequence models for non-EEG staging is unlikely to close the gap; richer physiological features and confidence calibration are the productive levers.","Wearable sleep reports should separate high-confidence from low-confidence epochs and consider merging N1 into a Light Sleep category, since N1 is intrinsically hard to stage without EEG.","The same controlled-decomposition protocol could be applied to raw PPG waveforms to quantify how much of the consumer penalty is due to derived signals rather than sensing hardware.","The modality ceiling implies that non-EEG staging should be positioned as sleep-structure and trend monitoring, not as a replacement for EEG-based clinical staging."],"fun_headline_variants":["Wearable sleep staging caps at kappa 0.49; missing cortex is the limit","Non-EEG sleep staging hits kappa 0.49 ceiling; EEG gap is cortical","Sleep staging without EEG: kappa 0.49 max, cortex deficit is the barrier","Consumer wearables reach only kappa 0.255 for sleep staging; EEG still king","Drop low-confidence epochs: wearable sleep staging kappa jumps to 0.512"],"cache_read_input_tokens":14080,"weakest_assumption_plain":"The 0.304 gap is treated as pure signal-modality loss, but it is measured across different datasets and feature pipelines, so cross-cohort and cross-feature differences are assumed negligible.","fun_headline_variants_meta":{"raw":{"variants":["Wearable sleep staging caps at kappa 0.49; missing cortex is the limit","Non-EEG sleep staging hits kappa 0.49 ceiling; EEG gap is cortical","Sleep staging without EEG: kappa 0.49 max, cortex deficit is the barrier","Consumer wearables reach only kappa 0.255 for sleep staging; EEG still king","Drop low-confidence epochs: wearable sleep staging kappa jumps to 0.512"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000798,"raw_usage":{"total_tokens":3416,"prompt_tokens":882,"completion_tokens":2534,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":626,"completion_tokens_details":{"reasoning_tokens":2418}},"tokens_in":626,"tokens_out":2534,"duration_ms":15777,"temperature":1.0,"reasoning_tokens":2418,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T14:06:16.516403+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Simultaneously record EEG and non-EEG signals in the same subjects, run both through identical feature engineering and the same compact model, and compare the within-subject gap; a gap much smaller than 0.304 would show the headline number was inflated by dataset differences.","supporting_citations":[],"review_version":1}