Pith. sign in

REVIEW 4 major objections 5 minor 20 references

LG-Sleep: Local and Global Temporal Dependencies for Mice Sleep Scoring

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Pith's one-line read LG-Sleep claims an autoencoder-regularized CNN-LSTM scores 3-class mouse sleep from EEG at 0.85 accuracy and 0.75 macro F1, beating CNN, LSTM, FC, and three prior methods on subject-independent folds.

desk verdict Plausible incremental architecture for mice sleep scoring, but the reported margin over baselines is not statistically established. read the letter →

arxiv 2412.15412 v1 pith:7TETZATK submitted 2024-12-19 cs.LG

classification cs.LG
keywords sleepscoringmouseEEGtemporaldependenciesCNNLSTMautoencoder-decodersubject-independentclassificationlimitedlabeleddata
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a deep network combining time-distributed convolutions and an LSTM, trained jointly with a reconstruction loss in an autoencoder-decoder setup, can score three mouse sleep stages (wake, NREM, REM) from EEG in a subject-independent way. On 16 mice with four-fold subject-out cross-validation, the network reports wide-band accuracy of 0.85 and macro F1 of 0.75, compared with lower accuracy and much lower F1 for CNN, LSTM, fully connected, and three re-implemented prior sleep scorers. It also reports that performance degrades gracefully when only 25-50% of labeled training samples are used. A sympathetic reader would take the central message to be that explicitly modeling both short local transitions (500 ms kernel slices) and long global sequence structure pays off, and that the reconstruction loss helps the learned features transfer across mice and tolerate scarce labels.

What carries the argument

The carrying mechanism is the paired encoder-decoder with a shared latent vector. The encoder uses time-distributed 1D CNNs (64 filters, kernel width 500 ms), max pooling, and an LSTM with hidden state size 32 to produce a flat latent representation, which is fed to a fully connected layer and softmax classifier $\mathrm{softmax}(W A + b)$. The decoder repeats that latent vector 18 times, passes it through an LSTM of size 256, upsamples by a factor of 8, and applies two 1D CNN layers to reconstruct the input; the total loss is $\mathcal{L} = \mathcal{L}_c + \mathcal{L}_{\mathrm{mse}}$. The reconstruction constraint is what the paper argues forces the latent representation to retain enough information to rebuild the raw EEG while also being discriminative for sleep stages, producing cross-subject generalization and graceful degradation with limited labeled data.

What would settle it

Take the 32-dimensional latent vectors of held-out mice and color them by mouse identity: if a human or a simple classifier can separate subjects more cleanly than wake/NREM/REM stages, then the autoencoder is capturing subject-specific signal, and the claimed subject-independent generalization would not hold.

Watch

Extended reading notes

Core claim

On its own terms, LG-Sleep establishes that a subject-independent mouse sleep scorer built from time-distributed 1D CNN filters over 500 ms EEG segments, an LSTM over the sequence, and an autoencoder-decoder reconstruction branch achieves mean accuracy 0.85 +/- 0.09 and macro F1 0.75 +/- 0.10 on wide-band EEG, outperforming CNN (0.73/0.57), LSTM (0.75/0.61), FC (0.66/0.48), and re-implementations of three published sleep scorers. The two-stage training first minimizes classification plus MSE reconstruction loss, then classification-only with class weights 1.5, 1, and 7 for wake, REM, and NREM, respectively; this is what the paper credits with robustness when training labels are limited. The paper also reports that delta, theta, alpha, and beta subbands perform similarly, while gamma drops to 0.62 accuracy and 0.29 F1.

Load-bearing premise

The load-bearing premise is that the MSE reconstruction loss, together with the latent vector repeated 18 times before the decoder LSTM, forces the latent representation to encode sleep-stage information that transfers across individual mice; the paper does not directly test whether latent vectors separate by stage rather than by mouse identity.

Editorial extensions

If this is right

  • A lab with EEG from a few mice could score wake/NREM/REM automatically at 0.85 accuracy without collecting per-subject labels, if the reported subject-out folds reflect realistic use.
  • The macro F1 of 0.75 shows that REM sleep is the limiting class, so scoring pipelines should focus data augmentation or cost weighting on REM rather than on overall accuracy.
  • Joint reconstruction-plus-classification training is a concrete regularization recipe that other EEG classifiers could adopt, with the second classification-only stage acting as fine-tuning.
  • Under annotation scarcity the drop is mild: 0.76 accuracy at 50% labeled data and 0.67 at 25%, making the architecture viable for low-label sleep studies.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the same local/global encoder-decoder design could transfer to human sleep staging if the window length and kernel width are rescaled to human EEG rhythms, since the split into local and global temporal structure is not mouse-specific.
  • Editorial inference: the reconstruction branch is never probed directly, so a latent-space analysis (for example nearest-neighbor retrieval by stage) would reveal whether the autoencoder is the actual cause of cross-subject transfer or merely a regularizer.
  • Editorial inference: the 500 ms overlap between adjacent one-second slices means adjacent training trials share EEG content, so the reported fold standard deviations likely understate uncertainty; re-running with non-overlapping slices would give a more conservative reliability estimate.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces LG-Sleep, a deep neural network for three-class mouse sleep scoring (wake, REM, NREM) from single-channel EEG. The architecture uses a time-distributed 1D CNN to extract local temporal transitions, an LSTM to model global transitions, and an autoencoder-decoder layout with a reconstruction loss added to the classification loss. Experiments on an internal dataset of 16 mice with subject-independent four-fold cross-validation report a wide-band accuracy of 0.85 and macro F1-score of 0.75, and the paper claims superiority over CNN, LSTM, FC, and three re-implemented prior methods, as well as robustness when only 25--50% of labeled training data are used.

Significance. If the reported results are valid, LG-Sleep would be a useful baseline for automated mouse sleep scoring, particularly because it addresses class imbalance and limited labeled data, which are practical concerns in preclinical sleep research. The paper has several strengths: the architecture is clearly described, frequency-subband and ablation analyses are included, limited-data experiments are reported, and the limitations paragraph candidly acknowledges the REM detection difficulty and training cost. However, the central quantitative claims rest on an internal dataset with no shared code or data, re-implemented baselines, and an evaluation protocol that does not account for within-mouse temporal dependence; these issues currently weaken the evidence for the headline accuracy and superiority claims.

major comments (4)
  1. [Section III (Experiments and Discussion)] The evaluation protocol does not justify the reported uncertainties. With 16 mice, a 4-fold subject-out split yields only 4 test mice per fold, and 'repeating 4 times' does not create new independent subjects; the effective between-subject sample size remains 16. In addition, Section II-B describes 1-second windows with 500-ms overlap derived from 10-second trials, so adjacent windows are strongly non-independent, and even non-overlapping trials from the same mouse are autocorrelated because sleep stages persist across many trials. The reported standard deviations and the t-test mentioned in Section III-3 therefore appear to treat correlated windows as independent samples, which will underestimate variance and can make small performance differences appear significant. The authors should aggregate predictions per mouse (or per independent recording epoch) and use cluster-level statistics, and should report the number of independent units used in the t-test.
  2. [Table 5 and Section III-3] The comparison with existing methods is not auditable because all baselines were 're-implemented in Python using TensorFlow based on the provided descriptions' rather than taken from official releases or code. Since the paper does not provide the re-implementation code, a reader cannot verify that the baselines were given fair hyperparameter treatment, and the claimed margins (0.85 vs 0.68--0.71 accuracy) could be affected by implementation details. To support the superiority claim, the authors should release and run official implementations where available, or provide detailed per-method hyperparameter search and code for the re-implementations.
  3. [Fig. 3 and Section III-1] The REM recall for LG-Sleep is 0.57, meaning 43% of REM windows are missed. Because REM comprises only 8% of the dataset, the macro F1 of 0.75 masks this weakness. The abstract and conclusion state that LG-Sleep 'performs well across different sleep stages' and 'achieves good performance across different sleep stages,' which is overstated relative to the confusion matrix. Per-class precision and recall should be reported and discussed, especially since the limitations paragraph itself acknowledges 'difficulties in detecting REM due to the limited number of samples.'
  4. [Section II-C and Section III] The paper asserts that the autoencoder reconstruction loss, with the latent vector repeated n times before the decoder LSTM, makes the learned representation stage-relevant and subject-invariant, but this mechanism is never directly tested. Specifically, there is no analysis showing that the latent space separates sleep stages independently of mouse identity, or that reconstruction primarily encodes stage-related signal rather than subject-specific noise. Without such evidence, the claimed generalization advantage over a plain CNN+LSTM remains plausible but unsupported. The authors could add latent-space visualization or a quantitative analysis of reconstruction errors across subjects and stages.
minor comments (5)
  1. [General formatting] There are several typographical issues, including 'JUL Y 2017' in the header, 'LG-SLeep' in the Table 5 caption, and inconsistent spacing throughout the text.
  2. [Table 1 and Section II-C] Table 1 lists the decoder LSTM as 'LSTM (512; 0.2)' while Section II-C states the decoder LSTM has a hidden state size of 256. Please clarify which value is correct and ensure the table and text are consistent.
  3. [Sections II-B and II-D] The slicing details are inconsistent: Section II-D says each sample size is 19×512×1, but the implementation section says the encoder output vector is repeated 18 times before the decoder LSTM. With a 10-second trial, 1-second windows, and 500-ms overlap, one would expect 19 windows; the relationship between the 19 slices and the 18 repeats should be explained or corrected.
  4. [Fig. 2 caption] The caption states the background colors purple, red, and green correspond to wake, NREM, and REM, but the text in Section III describes a sample of true versus predicted labels without naming which colors correspond to which stages in subfigure (a) and (b). Please make the mapping explicit and consistent between text and figure.
  5. [Section III-3] The t-test is mentioned but no details are given about whether it is paired, how many observations are used, or whether it accounts for the repeated folds. Please specify the test, the unit of analysis, and the number of independent samples.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: LG-Sleep's gains are empirical findings, not quantities forced by definition or by a self-citation chain.

full rationale

LG-Sleep is an empirical machine-learning paper. It proposes an architecture (time-distributed CNN plus LSTM in an autoencoder-decoder with joint reconstruction and classification losses) and reports held-out, subject-independent accuracy and F1 scores. There is no derivation chain in which an output quantity is defined by an input quantity. The reconstruction loss does not define the sleep-stage prediction: labels enter only through the cross-entropy term L_c, while L_mse is computed on EEG inputs, so the reported accuracy is not forced by construction. Hyperparameters such as class weights (1.5, 1, 7), kernel widths, and loss weights are selected on validation data, but the paper does not present them as predictions, so no fitted-input-called-prediction issue arises. The only same-author citations are [13], a general CNN/attention method for EEG emotion recognition, and [18], the data-collection protocol and vigilance-state definitions for the mice dataset; neither is used to forbid alternatives or to justify a uniqueness claim. Baseline comparisons are re-implementations of external methods; any weaknesses there are evaluation-validity or reproducibility concerns, not circular reasoning. Statistical concerns about overlapping windows and repeated subject-out folds may weaken the strength of the empirical claim, but they are not a form of circularity. Therefore no circular steps were identified.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim rests on standard deep learning components plus several hand-chosen hyperparameters and a domain assumption about EEG subbands. No new physical or biological entities are introduced.

free parameters (5)
  • Class weights (wake, REM, NREM) = 1.5, 1, 7
    Set by hand in Section II-D to address class imbalance; no sensitivity analysis is provided.
  • CNN filter count and kernel width = 64 filters, 500 ms kernel
    Described as empirically set in Section II-C; no systematic tuning reported.
  • LSTM hidden sizes and repeat count = Encoder 32, decoder 256, repeated 18 times
    Chosen without reported search; the 18 repeats are not justified.
  • Learning rates and epochs = 0.0001 (20 epochs), 0.001 (100 epochs)
    Adam settings in Section II-D; no ablation on these choices.
  • Loss combination weight = implicitly 1 for L_c and L_mse
    Final loss is the unweighted sum L = L_c + L_mse; no ablation on the relative weighting.
assumptions (4)
  • domain assumption Manual expert scoring of EEG vigilance states is a reliable ground truth (Section II-A3).
    Used to define labels; no human-scorer agreement metrics are reported.
  • domain assumption The 1-45 Hz wide-band filter and the five subband filters preserve stage-discriminative information (Sections II-B, II-D).
    Subband selection is stated as reflecting known sleep physiology, not validated against alternative filtering choices.
  • ad hoc to paper The autoencoder reconstruction loss, summed with cross-entropy, guides the shared latent to be stage-relevant and subject-invariant (Sections II-C, III).
    This is the core mechanism behind the claimed generalization benefit, but no direct latent-space analysis is given.
  • standard math Softmax classification, LSTM gates, and backpropagation are assumed correct.
    Standard deep learning components invoked without proof.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LG-Sleep: Local and Global Temporal Dependencies for Mice Sleep Scoring." pith.science (2026). https://pith.science/paper/7TETZATK

@misc{pith2026241215412,
  author       = {Pith},
  title        = {Pith review of: LG-Sleep: Local and Global Temporal Dependencies for Mice Sleep Scoring},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7TETZATK}},
  note         = {Machine review of arXiv:2412.15412}
}
read the original abstract

Efficiently identifying sleep stages is crucial for unraveling the intricacies of sleep in both preclinical and clinical research. The labor-intensive nature of manual sleep scoring, demanding substantial expertise, has prompted a surge of interest in automated alternatives. Sleep studies in mice play a significant role in understanding sleep patterns and disorders and underscore the need for robust scoring methodologies. In response, this study introduces LG-Sleep, a novel subject-independent deep neural network architecture designed for mice sleep scoring through electroencephalogram (EEG) signals. LG-Sleep extracts local and global temporal transitions within EEG signals to categorize sleep data into three stages: wake, rapid eye movement (REM) sleep, and non-rapid eye movement (NREM) sleep. The model leverages local and global temporal information by employing time-distributed convolutional neural networks to discern local temporal transitions in EEG data. Subsequently, features derived from the convolutional filters traverse long short-term memory blocks, capturing global transitions over extended periods. Crucially, the model is optimized in an autoencoder-decoder fashion, facilitating generalization across distinct subjects and adapting to limited training samples. Experimental findings demonstrate superior performance of LG-Sleep compared to conventional deep neural networks. Moreover, the model exhibits good performance across different sleep stages even when tasked with scoring based on limited training samples.

Figures

Figures reproduced from arXiv: 2412.15412 by the authors.

Figure 1
Figure 1. Overview of the proposed LG-Sleep deep neural network. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Sample EEG trace and the corresponding sleep stages: a) true [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Confusion matrices of the proposed method and model back [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

20 extracted references · 20 canonical work pages

  1. [1]

    The research of sleep staging based on single-lead electrocardiogram and deep neural network,

    R. Wei, X. Zhang, J. Wang, and X. Dang, “The research of sleep staging based on single-lead electrocardiogram and deep neural network,” Biomed. Eng. Lett. , vol. 8, pp. 87–93, 2018

  2. [2]

    Rapid assessment of sleep-wake behavior in mice,

    S. P. Fisher, S. I. H. Godinho, C. A. Pothecary, M. W. Hankins, R. G. Foster, and S. N. Peirson, “Rapid assessment of sleep-wake behavior in mice,” J. Biol. Rhythms, vol. 27, no. 1, pp. 48–58, 2012

  3. [3]

    Automated analysis of sleep–wake state in rats,

    R. Stephenson, A. M. Caron, D. B. Cassel, and J. C. Kostela, “Automated analysis of sleep–wake state in rats,” J. Neurosci. Methods , vol. 184, no. 2, pp. 263–274, 2009

  4. [4]

    Robust, automated sleep scoring by a compact neural network with distributional shift correction,

    Z. Barger, C. G. Frye, D. Liu, Y . Dan, and K. E. Bouchard, “Robust, automated sleep scoring by a compact neural network with distributional shift correction,” PLoS One , vol. 14, no. 12, pp. e0224642, 2019

  5. [5]

    Automatic analysis of sleep electroencephalograms by hybrid computation,

    J. R. Smith, M. Negin, and A. H. Nevis, “Automatic analysis of sleep electroencephalograms by hybrid computation,” IEEE Trans. Syst. Sci. Cybern. , vol. 5, no. 4, pp. 278–284, 1969

  6. [6]

    Pattern recognition applied to sleep state classification,

    D. B. Drane, W. B. Martin, and S. S. Viglione, “Pattern recognition applied to sleep state classification,” Electroencephalogr . Clin. Neurophysiol., vol. 26, no. 2, pp. 238, 1969

  7. [7]

    Sleep and its disorders in translational medicine,

    L. M. Paterson, D. J. Nutt, and S. J. Wilson, “Sleep and its disorders in translational medicine,” J. Psychopharmacol., vol. 25, no. 9, pp. 1226–1234, 2011

  8. [8]

    EEG gamma frequency and sleep–wake scoring in mice: comparing two types of supervised classifiers,

    J. Branka ˇck, V . I. Kukushka, A. L. Vyssotski, and A. Draguhn, “EEG gamma frequency and sleep–wake scoring in mice: comparing two types of supervised classifiers,” Brain Res. , vol. 1322, pp. 59–71, 2010

Show all 20 references
  1. [9]

    Automated scoring of pre-REM sleep in mice with deep learning,

    N. Grieger, J. T. C. Schwabedal, S. Wendel, Y . Ritze, and S. Bialonski, “Automated scoring of pre-REM sleep in mice with deep learning,” Sci. Rep. , vol. 11, no. 1, pp. 1–14, 2021

  2. [10]

    MC-SleepNet: large-scale sleep stage scoring in mice by deep neural networks,

    M. Yamabe, K. Horie, H. Shiokawa, H. Funato, M. Yanagisawa, and H. Kitagawa, “MC-SleepNet: large-scale sleep stage scoring in mice by deep neural networks,” Sci. Rep. , vol. 9, no. 1, pp. 15793, 2019

  3. [11]

    Employing a long-short-term memory neural network to improve automatic sleep stage classification of pharmaco-EEG profiles,

    H. Alsolai, S. Qureshi, S. M. Zeeshan Iqbal, A. Ameer, D. Cheaha, L. E. Henesey, and S. Karrila, “Employing a long-short-term memory neural network to improve automatic sleep stage classification of pharmaco-EEG profiles,” Appl. Sci. , vol. 12, no. 10, pp. 5248, 2022

  4. [12]

    Automated sleep scoring: A review of the latest approaches,

    L. Fiorillo, A. Puiatti, M. Papandrea, P.-L. Ratti, P. Favaro, C. Roth, P. Bargiotas, C. L. Bassetti, and F. D. Faraci, “Automated sleep scoring: A review of the latest approaches,” Sleep Med. Rev. , vol. 48, pp. 101204, 2019

  5. [13]

    A hybrid end-to-end spatio-temporal attention neural network with graph-smooth signals for EEG emotion recognition,

    S. Sartipi, M. Torkamani-Azar, and M. Cetin, “A hybrid end-to-end spatio-temporal attention neural network with graph-smooth signals for EEG emotion recognition,” IEEE Trans. Cogn. Dev. Syst. , 2023

  6. [14]

    Real-time patient-specific ECG classification by 1-d convolutional neural networks,

    S. Kiranyaz, T. Ince, and M. Gabbouj, “Real-time patient-specific ECG classification by 1-d convolutional neural networks,” IEEE Trans. Biomed. Eng. , vol. 63, no. 3, pp. 664–675, 2015

  7. [15]

    Framewise phoneme classification with bidirectional lstm and other neural network architectures,

    A. Graves and J. Schmidhuber, “Framewise phoneme classification with bidirectional lstm and other neural network architectures,” Neural Netw. , vol. 18, no. 5-6, pp. 602–610, 2005

  8. [16]

    Medical image analysis using convolutional neural networks: a review,

    S. M. Anwar and et al., “Medical image analysis using convolutional neural networks: a review,” J. Med. Syst. , vol. 42, pp. 1–13, 2018

  9. [17]

    Sleep–wake cycle in young and older mice,

    S. Soltani, S. Chauvette, O. Bukhtiyarova, J.-M. Lina, J. Dubé, J. Seigneur, J. Carrier, and I. Timofeev, “Sleep–wake cycle in young and older mice,” Front. Syst. Neurosci., vol. 13, pp. 51, 2019

  10. [18]

    Memory-enhancing properties of sleep depend on the oscillatory amplitude of norepinephrine,

    C. Kjaerby and et al., “Memory-enhancing properties of sleep depend on the oscillatory amplitude of norepinephrine,” Nat. Neurosci. , vol. 25, no. 8, pp. 1059–1070, 2022

  11. [19]

    Scoring transitions to REM sleep in rats based on the EEG phenomena of pre-REM sleep: an improved analysis of sleep structure,

    J. H. Benington, S. K. Kodali, and H. C. Heller, “Scoring transitions to REM sleep in rats based on the EEG phenomena of pre-REM sleep: an improved analysis of sleep structure,” Sleep, vol. 17, no. 1, pp. 28–36, 1994

  12. [20]

    EEG-based sleep staging analysis with functional connectivity,

    H. Huang, J. Zhang, L. Zhu, J. Tang, G. Lin, W. Kong, X. Lei, and L. Zhu, “EEG-based sleep staging analysis with functional connectivity,” Sensors, vol. 21, no. 6, pp. 1988, 2021

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.