{"id":"650a470f-abf0-4c69-b526-431f3076a6aa","arxiv_id":"2509.02982","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"The paper proposes StableSleep, a streaming test-time adaptation method for sleep staging, but does not actually report the frozen-baseline comparison that its central claim requires.","lead":"This short paper proposes a test-time adaptation recipe for sleep staging that refreshes batch-norm statistics, minimizes prediction entropy, and adds gates and resets to avoid drift. A careful read shows the key claim, that adaptation beats a frozen model, is made without reporting any frozen-baseline numbers.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline claim 'consistent gains over a frozen baseline' is never quantified: the manuscript reports only the adapted model's absolute metrics, with no frozen-baseline comparison, no per-subject deltas, and no sensitivity analysis.","rationale":"The reader identified the strongest claim correctly: the paper's central assertion is consistent gains over a frozen baseline. The reader's rationale also correctly notes that 'the central claim ... is not supported by any comparative data in the manuscript' and that Table 1 reports only adapted-model metrics. However, the reader's formal 'weakest_assumption' field points to hyperparameter transfer from validation, which is a secondary concern: sensitivity analysis would be valuable, but the absence of any frozen-baseline comparison is the first-order, load-bearing issue. My independent reading of the full text confirms that no comparative measurement appears anywhere: not in Table 1, Section 4.5, the appendices, or the placeholder sections 4.3 and 4.4. The claim is therefore unverified in the submitted manuscript. This is not a disagreement with the field's consensus or a stylistic complaint; it is a missing quantity necessary to establish the central result. A paper whose headline effect is never quantified cannot support its stated conclusion. The concrete test—reporting frozen-baseline metrics and paired deltas on the same splits—would settle the matter, and until then the appropriate disposition is rejection of the current submission. I concur with the reader's REJECT verdict, and note that adding the missing comparison in a revision could plausibly change the outcome.","tokens_in":6003,"tokens_out":2175,"duration_ms":27145,"concrete_test":"Re-run the exact StableSleep pipeline on the same Sleep-EDF Expanded subject-disjoint splits and report a single table with at least four columns: (a) frozen baseline (no adaptation), (b) BN-only statistic refresh, (c) Tent with entropy gate and EMA reset (StableSleep), and (d) per-subject mean difference (c) minus (a) with a paired significance test (e.g., Wilcoxon signed-rank). If the per-subject deltas are not consistently positive across accuracy, macro-F1, and κ—or if BN-only matches or exceeds StableSleep—the 'consistent gains' claim fails. This check settles the concern because it directly measures the claimed effect on the same data and splits.","verdict_should_be":"REJECT","load_bearing_attack":"The central claim of the paper, stated in the abstract and Section 5, is that StableSleep shows 'consistent gains over a frozen baseline' at seconds-level latency and minimal memory. This claim is load-bearing: if it is true, the method is a practical contribution; if it is false or unmeasured, the paper's contribution collapses. The full text, however, never reports the frozen baseline's metrics. Table 1 gives adapted-model accuracy, macro-F1, κ, weighted F1, balanced accuracy, MCC, and ECE on validation and test, but there is no column or delta for the frozen baseline, no BN-only numeric comparison, no Tent-without-gates comparison, and no per-subject paired differences. Section 4.5, titled 'qualitative summary,' asserts that 'BN-only improves over frozen inference' and that 'Tent adds further gains,' but gives no numbers. The appendices show subject-wise distributions and confusion matrices only for the adapted model. Sections 4.3 and 4.4 are placeholders ('added back') rather than quantitative analyses. Consequently, the manuscript does not supply the evidence needed to evaluate the headline claim. The hyperparameter-transfer concern raised by the reader is real but secondary: even if the entropy-gate thresholds, EMA reset criterion, BN momentum, and median filter width transfer perfectly, the paper still does not demonstrate any gain, because the baseline comparison is absent. Without a measured baseline, the absolute accuracy of 67.0% and κ of 0.394 cannot be attributed to adaptation; they could reflect the source model alone, or even a net degradation masked by the chosen split.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes StableSleep, a streaming, source-free test-time adaptation (TTA) recipe for sleep staging. It combines BatchNorm (BN) statistic refresh with entropy minimization (Tent) and adds two safety rails: an entropy gate that pauses adaptation on uncertain windows and an EMA-based reset that reverts to a snapshot when drift is detected, followed by a causal median filter. The method is evaluated on Sleep-EDF Expanded using single-lead EEG (Fpz-Cz), with subject-disjoint splits and no test-label access. The authors claim consistent gains over a frozen baseline at seconds-level latency and minimal memory. The central weakness is that these claims are not supported by the reported evidence: Table 1 reports only adapted-model aggregate metrics, Section 4.5 is a qualitative summary, Sections 4.3 and 4.4 are explicit placeholders, and no frozen-baseline numbers or hyperparameter values are given.","tokens_in":6403,"tokens_out":4546,"duration_ms":44435,"significance":"If the headline claim were substantiated, StableSleep would be a practical and useful contribution: it is simple, privacy-preserving (no source data or patient calibration), and plausibly suitable for on-device use. The evaluation protocol has appropriate elements: subject-disjoint splits, a streaming constraint, and no test-label access. However, as presented, the empirical support is incomplete. The absolute metrics (test accuracy 67.0%, macro-F1 35.1%, Cohen's kappa 0.394) cannot be interpreted without a frozen-baseline comparison or numeric ablations. The paper provides no machine-checked proofs and no code release; reproducibility is promised only upon acceptance.","major_comments":[{"comment":"The abstract and Section 5 state 'consistent gains over a frozen baseline,' but no frozen-baseline metrics are reported anywhere. Table 1 shows only adapted-model aggregate metrics for validation and test; there is no baseline column, no per-subject deltas, no BN-only numeric comparison, and no Tent-without-gates comparison. Section 4.5 asserts that 'BN-only improves over frozen inference' and 'Tent adds further gains' without presenting any numbers. Because the central contribution is the improvement over the frozen model, these comparisons must be reported, including per-subject variability.","section":"Section 3.4, Table 1, Section 5"},{"comment":"The ablation study is described as a 'qualitative summary' with no numerical results, standard deviations, or paired significance tests. Sections 4.3 and 4.4 are explicitly placeholders ('added back'), so the calibration and stage-distribution analyses are not actually provided. This is more than a presentation issue: the safety rails (entropy gate, EMA reset) are core contributions, and the paper needs quantitative ablations (gate on/off, reset on/off, BN-only vs. Tent) plus subject-level spread to support the claimed benefits.","section":"Section 4.5, Sections 4.3/4.4"},{"comment":"The adaptation hyperparameters — entropy gate thresholds hmin/hmax, EMA reset criterion and momentum, BN running-statistics momentum, and causal median filter width — are defined but no values are given. Section 4.5 says these are fixed from validation, but no sensitivity analysis is reported. If the thresholds are validation-specific, the claimed consistency on test may not transfer; the authors should report the chosen values and a sensitivity sweep over them.","section":"Section 3.4"},{"comment":"The abstract and conclusion promise 'seconds-level latency and minimal memory,' but no inference latency or memory measurements are reported. Appendix B.1 gives training runtime only. Since deployment efficiency is part of the claimed contribution, the paper should include profiling results on the target hardware or at least characterize per-epoch computational cost.","section":"Section 5"}],"minor_comments":[{"comment":"The headings 'Calibration (added back)' and 'Stage distribution (added back)' are placeholders; they should either be completed with actual analyses and figures or removed.","section":"Sections 4.3/4.4"},{"comment":"Several references are duplicated or incomplete: [1]/[2], [3]/[4], [15]/[16], and [17]/[18] appear to be near-duplicates; [5] lacks a full author list; [32] is listed but not cited in the text. Please consolidate and check all citations.","section":"References"},{"comment":"The phrase 'model-agnostic' is overstated if adaptation relies on BN layers; specify the class of architectures for which the recipe applies.","section":"Section 3.3"},{"comment":"Figure 8 is referenced for subject-wise distributions, but numeric standard deviations or interquartile ranges are not reported. Adding these would strengthen the claims about consistency.","section":"Appendix A"},{"comment":"Please clarify whether per-record streaming standardization is updated before or after TTA updates affect the BN statistics; this affects the online/streaming claim.","section":"Section 3.2"}],"recommendation":"major_revision","confidential_remarks":"The manuscript appears incomplete in its current form (placeholder sections, missing baseline comparison). The central claim is likely fixable by adding the frozen-baseline table, numeric ablations, and hyperparameter details, so I do not recommend rejection on grounds of circularity or fundamental flaw. However, the paper should not be accepted until these load-bearing empirical gaps are closed. I also note that the abstract's 'consistent gains' phrasing is premature given the reported evidence."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nThe thing to know about this paper is that the abstract promises consistent gains over a frozen baseline, but the manuscript never reports a single number for that baseline. Table 1 gives the adapted model's accuracy, F1, kappa, etc., with no delta, no baseline column, no per-subject paired comparison. Section 4.5 claims BN-only improves and Tent adds further gains without any quantitative support. The central claim is simply not backed by data.\n\nWhat's the paper good for? The method itself is a clean, plausible combination of existing TTA ingredients: Tent-style entropy minimization, BatchNorm statistic refresh, an entropy gate, and an EMA reset. Applying this to single-lead sleep staging is reasonable, and the authors are explicit about the streaming constraint and the causal median filter. The related work is appropriate, and the limitations section is honest about the single-dataset evaluation. If the baseline had been measured, this could be a useful applied contribution.\n\nThe soft spots are large. The missing baseline is load-bearing. Without it, the absolute accuracy of 67.0% and kappa of 0.394 could just as easily be the source model with no adaptation. Sections 4.3 and 4.4 are literally placeholders ('added back') rather than analyses. There are no standard deviations, no sensitivity analysis for hyperparameters (entropy gate thresholds, EMA reset criterion, BN momentum, median filter width), and code is withheld until acceptance. The appendix shows subject-wise distributions for the adapted model only, so it doesn't fill the gap.\n\nDoes this deserve a serious referee? In current form, no. The main result is unmeasured and the paper reads as incomplete. A desk rejection would be appropriate, with the authors asked to add the frozen baseline, quantify the ablations, and replace the placeholder sections. If they do that, the recipe could be a reasonable field note. As is, it doesn't support its own abstract.\n\nMy take: reject with clear revision guidance rather than a harsh verdict—the idea is fine, the execution is incomplete.\n\nBest,\n[You]","headline":"A sensible TTA recipe for sleep staging whose central claim—gains over a frozen baseline—is never actually measured.","tokens_in":6822,"tokens_out":2736,"would_cite":false,"duration_ms":29584,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A lightweight recipe adapts sleep-staging models to new patients in real time, using only unlabeled EEG and no source data.","keywords":["test-time adaptation","sleep staging","entropy minimization","batch normalization","single-lead EEG","streaming inference","source-free domain adaptation","Sleep-EDF Expanded"],"falsifier":"Deploy the exact recipe, with validation-fixed hyperparameters untouched, on a different public sleep dataset (different montage or population) and compare the adapted model against the same frozen baseline; if the gains flip sign on the first external cohort, the claim of consistent deployment-ready gains is refuted. A cheaper check inside the corpus: re-split subjects and re-tune the gate bounds on the new validation set — if the optimal [hmin, hmax] moves with the split, the settings are not transferable.","tokens_in":5959,"feed_emoji":"😴","tokens_out":16095,"duration_ms":149891,"temperature":0.7,"pith_summary":"The paper claims that a sleep-staging model can be adapted live, patient by patient, using only the unlabeled EEG stream, with no access to the original training data or to a new patient's expert labels. The recipe pairs two established ideas — refreshing batch-normalization statistics and minimizing prediction entropy in the normalization layers only — with two safety rails: an entropy gate that pauses updates on uncertain or artefactual 30-second windows, and an exponential-moving-average (EMA) reset that restores the model when drift builds up. On the Sleep-EDF Expanded corpus with single-lead EEG, the authors report consistent gains over a frozen baseline at seconds-level latency and minimal memory, with test accuracy 67.0%, weighted F1 70.1%, and κ = 0.394. If the claim holds, it matters because sleep-stage models are known to degrade on unseen montages, amplifiers, and populations, while re-collecting or centralizing data for retraining is often blocked by privacy; a source-free, label-free, streaming recipe is the missing practical piece for bedside or wearable deployment.","feed_headline":"Adapt sleep staging to new patients in seconds, no labels needed","feed_subtitle":"A stream of unlabeled EEG refreshes the model as it runs — no retraining, no patient calibration, no privacy transfer.","key_machinery":"The mechanism is the two-rail adaptation loop wrapped around Tent. On each streaming micro-batch the model minimizes the entropy of its softmax output while touching only batch-norm affine parameters and running statistics; an entropy gate keeps an exponential-moving-average (EMA) of batch entropy and suppresses updates outside [hmin, hmax], so near-uniform or artefactual windows cannot corrupt the weights; an EMA snapshot of the adapted parameters acts as the reset anchor, restoring the model when a drift criterion fires. A causal median filter of width 5 smooths prediction flicker. The gate and reset are the load-bearing additions: they turn a memoryless entropy minimizer into a process th","core_discovery":"This is an engineering claim about what makes test-time adaptation safe in a clinical stream: entropy minimization alone is not enough, and the contribution is the wrapper that stabilizes it. A source-trained model with explicit batch-norm layers adapts on a new patient's stream by minimizing prediction entropy per micro-batch, touching only batch-norm affine parameters and running statistics, skipping updates when the EMA of batch entropy leaves [hmin, hmax], and resetting to an EMA snapshot when drift is detected. On Sleep-EDF Expanded (Fpz–Cz, 100 Hz, 30 s epochs; subject-disjoint; settings fixed from validation; test labels unused), the model reaches 67.0% accuracy and κ = 0.394 — stated","pith_inferences":["The two rails use nothing sleep-specific, so the same wrapper should transfer to other streaming physiological classifiers (seizure detection, cardiac rhythm monitoring), where artefact windows and slow drift are the same failure modes — a connection the paper does not draw.","The absolute gate bounds [hmin, hmax] are a tuning liability; a gate defined relative to the running entropy distribution could plausibly remove the validation-tuning step, a testable variant the paper does not explore.","Because calibration stays at ECE ≈ 0.08, entropy minimization does not automatically buy trustworthy confidence; adding confidence-based deferral or post-hoc recalibration could convert the accuracy gains into clinically usable uncertainty.","The untested edge of the claim is external-cohort transfer: the recipe would be most valuable exactly where it is hardest, moving from Sleep-EDF to a different montage and population."],"forward_implications":["Frozen staging models can be fielded on a new patient with no retraining, no source data, and no labels — the model adapts itself from the incoming EEG stream.","Adaptation runs at seconds-level latency with minimal memory, so it fits on laptop-class or bedside hardware rather than research clusters.","Because only batch-norm layers change, the wrapper is model-agnostic: any staging backbone with explicit batch-norm can carry the same rails.","Privacy is satisfied by construction: no patient labels and no source data ever enter the adaptation loop.","The gain profile is uneven by stage — N1 remains hardest with dominant N2/REM confusions — so deployments should still expect transitional epochs to be the weak spot."],"supporting_citations":[{"why":"Tent: supplies the entropy-minimization update the recipe wraps; the method being adapted at test time.","marker":"[41]"},{"why":"AdaBN: supplies the batch-norm statistic refresh used both as the BN-only baseline and as the statistics component of the recipe.","marker":"[27, 28]"},{"why":"CoTTA: supplies the continual-stabilization idea (EMA-based recovery) that the paper's reset rail borrows.","marker":"[42]"},{"why":"Sleep-EDF Expanded: the evaluation corpus supplying single-lead EEG and expert hypnograms.","marker":"[21, 22, 11]"},{"why":"Defines the chance-corrected agreement coefficient (kappa) used as the headline evaluation metric.","marker":"[7]"},{"why":"Scoring standards that fix the five-stage clinical label space (W/N1/N2/N3/REM) used for training and evaluation.","marker":"[38, 19, 3, 1]"},{"why":"Inter-database validation study documenting cross-cohort sleep-staging degradation, the problem the recipe targets.","marker":"[43]"}],"fun_headline_variants":["Sleep staging adapts on the fly, no labels, no calibration","New-patient sleep staging in seconds, with safety rails","Unlabeled EEG stream refreshes sleep models safely","Entropy-plus-rails TTA for sleep staging on new patients"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The whole result rests on the assumption that the adaptation settings fixed on the validation subjects — the entropy-gate bounds, the reset trigger, the batch-norm momentum, the smoothing width — carry over unchanged to any new patient; the paper gives no sensitivity analysis, so if those thresholds are patient-specific, the reported consistency could be an artifact of the chosen validation split.","fun_headline_variants_meta":{"raw":{"variants":["Sleep staging adapts on the fly, no labels, no calibration","New-patient sleep staging in seconds, with safety rails","Unlabeled EEG stream refreshes sleep models safely","Entropy-plus-rails TTA for sleep staging on new patients"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000918,"raw_usage":{"total_tokens":3744,"prompt_tokens":680,"completion_tokens":3064,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":424,"completion_tokens_details":{"reasoning_tokens":3008}},"tokens_in":424,"tokens_out":3064,"duration_ms":24386,"temperature":1.0,"reasoning_tokens":3008,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T11:12:30.136409+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Deploy the exact recipe, with validation-fixed hyperparameters untouched, on a different public sleep dataset (different montage or population) and compare the adapted model against the same frozen baseline; if the gains flip sign on the first external cohort, the claim of consistent deployment-ready gains is refuted. A cheaper check inside the corpus: re-split subjects and re-tune the gate bounds on the new validation set — if the optimal [hmin, hmax] moves with the split, the settings are not transferable.","supporting_citations":[{"cited_title":"Tent: Fully test-time adaptation by entropy minimization","cited_arxiv_id":null,"evidence_quote":"Tent: supplies the entropy-minimization update the recipe wraps; the method being adapted at test time."},{"cited_title":"Continual test-time domain adaptation","cited_arxiv_id":null,"evidence_quote":"CoTTA: supplies the continual-stabilization idea (EMA-based recovery) that the paper's reset rail borrows."},{"cited_title":"Inter-database validation of a deep learning approach for automatic sleep staging","cited_arxiv_id":null,"evidence_quote":"Inter-database validation study documenting cross-cohort sleep-staging degradation, the problem the recipe targets."}],"review_version":1}