REVIEW 4 major objections 5 minor 20 references
LG-Sleep: Local and Global Temporal Dependencies for Mice Sleep Scoring
T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read LG-Sleep claims an autoencoder-regularized CNN-LSTM scores 3-class mouse sleep from EEG at 0.85 accuracy and 0.75 macro F1, beating CNN, LSTM, FC, and three prior methods on subject-independent folds.
desk verdict Plausible incremental architecture for mice sleep scoring, but the reported margin over baselines is not statistically established. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The carrying mechanism is the paired encoder-decoder with a shared latent vector. The encoder uses time-distributed 1D CNNs (64 filters, kernel width 500 ms), max pooling, and an LSTM with hidden state size 32 to produce a flat latent representation, which is fed to a fully connected layer and softmax classifier $\mathrm{softmax}(W A + b)$. The decoder repeats that latent vector 18 times, passes it through an LSTM of size 256, upsamples by a factor of 8, and applies two 1D CNN layers to reconstruct the input; the total loss is $\mathcal{L} = \mathcal{L}_c + \mathcal{L}_{\mathrm{mse}}$. The reconstruction constraint is what the paper argues forces the latent representation to retain enough information to rebuild the raw EEG while also being discriminative for sleep stages, producing cross-subject generalization and graceful degradation with limited labeled data.
What would settle it
Take the 32-dimensional latent vectors of held-out mice and color them by mouse identity: if a human or a simple classifier can separate subjects more cleanly than wake/NREM/REM stages, then the autoencoder is capturing subject-specific signal, and the claimed subject-independent generalization would not hold.
Extended reading notes
Core claim
On its own terms, LG-Sleep establishes that a subject-independent mouse sleep scorer built from time-distributed 1D CNN filters over 500 ms EEG segments, an LSTM over the sequence, and an autoencoder-decoder reconstruction branch achieves mean accuracy 0.85 +/- 0.09 and macro F1 0.75 +/- 0.10 on wide-band EEG, outperforming CNN (0.73/0.57), LSTM (0.75/0.61), FC (0.66/0.48), and re-implementations of three published sleep scorers. The two-stage training first minimizes classification plus MSE reconstruction loss, then classification-only with class weights 1.5, 1, and 7 for wake, REM, and NREM, respectively; this is what the paper credits with robustness when training labels are limited. The paper also reports that delta, theta, alpha, and beta subbands perform similarly, while gamma drops to 0.62 accuracy and 0.29 F1.
Load-bearing premise
The load-bearing premise is that the MSE reconstruction loss, together with the latent vector repeated 18 times before the decoder LSTM, forces the latent representation to encode sleep-stage information that transfers across individual mice; the paper does not directly test whether latent vectors separate by stage rather than by mouse identity.
Editorial extensions
If this is right
- A lab with EEG from a few mice could score wake/NREM/REM automatically at 0.85 accuracy without collecting per-subject labels, if the reported subject-out folds reflect realistic use.
- The macro F1 of 0.75 shows that REM sleep is the limiting class, so scoring pipelines should focus data augmentation or cost weighting on REM rather than on overall accuracy.
- Joint reconstruction-plus-classification training is a concrete regularization recipe that other EEG classifiers could adopt, with the second classification-only stage acting as fine-tuning.
- Under annotation scarcity the drop is mild: 0.76 accuracy at 50% labeled data and 0.67 at 25%, making the architecture viable for low-label sleep studies.
Reading between the lines
- Editorial inference: the same local/global encoder-decoder design could transfer to human sleep staging if the window length and kernel width are rescaled to human EEG rhythms, since the split into local and global temporal structure is not mouse-specific.
- Editorial inference: the reconstruction branch is never probed directly, so a latent-space analysis (for example nearest-neighbor retrieval by stage) would reveal whether the autoencoder is the actual cause of cross-subject transfer or merely a regularizer.
- Editorial inference: the 500 ms overlap between adjacent one-second slices means adjacent training trials share EEG content, so the reported fold standard deviations likely understate uncertainty; re-running with non-overlapping slices would give a more conservative reliability estimate.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces LG-Sleep, a deep neural network for three-class mouse sleep scoring (wake, REM, NREM) from single-channel EEG. The architecture uses a time-distributed 1D CNN to extract local temporal transitions, an LSTM to model global transitions, and an autoencoder-decoder layout with a reconstruction loss added to the classification loss. Experiments on an internal dataset of 16 mice with subject-independent four-fold cross-validation report a wide-band accuracy of 0.85 and macro F1-score of 0.75, and the paper claims superiority over CNN, LSTM, FC, and three re-implemented prior methods, as well as robustness when only 25--50% of labeled training data are used.
Significance. If the reported results are valid, LG-Sleep would be a useful baseline for automated mouse sleep scoring, particularly because it addresses class imbalance and limited labeled data, which are practical concerns in preclinical sleep research. The paper has several strengths: the architecture is clearly described, frequency-subband and ablation analyses are included, limited-data experiments are reported, and the limitations paragraph candidly acknowledges the REM detection difficulty and training cost. However, the central quantitative claims rest on an internal dataset with no shared code or data, re-implemented baselines, and an evaluation protocol that does not account for within-mouse temporal dependence; these issues currently weaken the evidence for the headline accuracy and superiority claims.
major comments (4)
- [Section III (Experiments and Discussion)] The evaluation protocol does not justify the reported uncertainties. With 16 mice, a 4-fold subject-out split yields only 4 test mice per fold, and 'repeating 4 times' does not create new independent subjects; the effective between-subject sample size remains 16. In addition, Section II-B describes 1-second windows with 500-ms overlap derived from 10-second trials, so adjacent windows are strongly non-independent, and even non-overlapping trials from the same mouse are autocorrelated because sleep stages persist across many trials. The reported standard deviations and the t-test mentioned in Section III-3 therefore appear to treat correlated windows as independent samples, which will underestimate variance and can make small performance differences appear significant. The authors should aggregate predictions per mouse (or per independent recording epoch) and use cluster-level statistics, and should report the number of independent units used in the t-test.
- [Table 5 and Section III-3] The comparison with existing methods is not auditable because all baselines were 're-implemented in Python using TensorFlow based on the provided descriptions' rather than taken from official releases or code. Since the paper does not provide the re-implementation code, a reader cannot verify that the baselines were given fair hyperparameter treatment, and the claimed margins (0.85 vs 0.68--0.71 accuracy) could be affected by implementation details. To support the superiority claim, the authors should release and run official implementations where available, or provide detailed per-method hyperparameter search and code for the re-implementations.
- [Fig. 3 and Section III-1] The REM recall for LG-Sleep is 0.57, meaning 43% of REM windows are missed. Because REM comprises only 8% of the dataset, the macro F1 of 0.75 masks this weakness. The abstract and conclusion state that LG-Sleep 'performs well across different sleep stages' and 'achieves good performance across different sleep stages,' which is overstated relative to the confusion matrix. Per-class precision and recall should be reported and discussed, especially since the limitations paragraph itself acknowledges 'difficulties in detecting REM due to the limited number of samples.'
- [Section II-C and Section III] The paper asserts that the autoencoder reconstruction loss, with the latent vector repeated n times before the decoder LSTM, makes the learned representation stage-relevant and subject-invariant, but this mechanism is never directly tested. Specifically, there is no analysis showing that the latent space separates sleep stages independently of mouse identity, or that reconstruction primarily encodes stage-related signal rather than subject-specific noise. Without such evidence, the claimed generalization advantage over a plain CNN+LSTM remains plausible but unsupported. The authors could add latent-space visualization or a quantitative analysis of reconstruction errors across subjects and stages.
minor comments (5)
- [General formatting] There are several typographical issues, including 'JUL Y 2017' in the header, 'LG-SLeep' in the Table 5 caption, and inconsistent spacing throughout the text.
- [Table 1 and Section II-C] Table 1 lists the decoder LSTM as 'LSTM (512; 0.2)' while Section II-C states the decoder LSTM has a hidden state size of 256. Please clarify which value is correct and ensure the table and text are consistent.
- [Sections II-B and II-D] The slicing details are inconsistent: Section II-D says each sample size is 19×512×1, but the implementation section says the encoder output vector is repeated 18 times before the decoder LSTM. With a 10-second trial, 1-second windows, and 500-ms overlap, one would expect 19 windows; the relationship between the 19 slices and the 18 repeats should be explained or corrected.
- [Fig. 2 caption] The caption states the background colors purple, red, and green correspond to wake, NREM, and REM, but the text in Section III describes a sample of true versus predicted labels without naming which colors correspond to which stages in subfigure (a) and (b). Please make the mapping explicit and consistent between text and figure.
- [Section III-3] The t-test is mentioned but no details are given about whether it is paired, how many observations are used, or whether it accounts for the repeated folds. Please specify the test, the unit of analysis, and the number of independent samples.
Circularity Check
No significant circularity: LG-Sleep's gains are empirical findings, not quantities forced by definition or by a self-citation chain.
full rationale
LG-Sleep is an empirical machine-learning paper. It proposes an architecture (time-distributed CNN plus LSTM in an autoencoder-decoder with joint reconstruction and classification losses) and reports held-out, subject-independent accuracy and F1 scores. There is no derivation chain in which an output quantity is defined by an input quantity. The reconstruction loss does not define the sleep-stage prediction: labels enter only through the cross-entropy term L_c, while L_mse is computed on EEG inputs, so the reported accuracy is not forced by construction. Hyperparameters such as class weights (1.5, 1, 7), kernel widths, and loss weights are selected on validation data, but the paper does not present them as predictions, so no fitted-input-called-prediction issue arises. The only same-author citations are [13], a general CNN/attention method for EEG emotion recognition, and [18], the data-collection protocol and vigilance-state definitions for the mice dataset; neither is used to forbid alternatives or to justify a uniqueness claim. Baseline comparisons are re-implementations of external methods; any weaknesses there are evaluation-validity or reproducibility concerns, not circular reasoning. Statistical concerns about overlapping windows and repeated subject-out folds may weaken the strength of the empirical claim, but they are not a form of circularity. Therefore no circular steps were identified.
Assumptions & free parameters
free parameters (5)
- Class weights (wake, REM, NREM) =
1.5, 1, 7
- CNN filter count and kernel width =
64 filters, 500 ms kernel
- LSTM hidden sizes and repeat count =
Encoder 32, decoder 256, repeated 18 times
- Learning rates and epochs =
0.0001 (20 epochs), 0.001 (100 epochs)
- Loss combination weight =
implicitly 1 for L_c and L_mse
assumptions (4)
- domain assumption Manual expert scoring of EEG vigilance states is a reliable ground truth (Section II-A3).
- domain assumption The 1-45 Hz wide-band filter and the five subband filters preserve stage-discriminative information (Sections II-B, II-D).
- ad hoc to paper The autoencoder reconstruction loss, summed with cross-entropy, guides the shared latent to be stage-relevant and subject-invariant (Sections II-C, III).
- standard math Softmax classification, LSTM gates, and backpropagation are assumed correct.
Cite this review
Pith. "Pith review of LG-Sleep: Local and Global Temporal Dependencies for Mice Sleep Scoring." pith.science (2026). https://pith.science/paper/7TETZATK
@misc{pith2026241215412,
author = {Pith},
title = {Pith review of: LG-Sleep: Local and Global Temporal Dependencies for Mice Sleep Scoring},
year = {2026},
howpublished = {\url{https://pith.science/paper/7TETZATK}},
note = {Machine review of arXiv:2412.15412}
}
read the original abstract
Efficiently identifying sleep stages is crucial for unraveling the intricacies of sleep in both preclinical and clinical research. The labor-intensive nature of manual sleep scoring, demanding substantial expertise, has prompted a surge of interest in automated alternatives. Sleep studies in mice play a significant role in understanding sleep patterns and disorders and underscore the need for robust scoring methodologies. In response, this study introduces LG-Sleep, a novel subject-independent deep neural network architecture designed for mice sleep scoring through electroencephalogram (EEG) signals. LG-Sleep extracts local and global temporal transitions within EEG signals to categorize sleep data into three stages: wake, rapid eye movement (REM) sleep, and non-rapid eye movement (NREM) sleep. The model leverages local and global temporal information by employing time-distributed convolutional neural networks to discern local temporal transitions in EEG data. Subsequently, features derived from the convolutional filters traverse long short-term memory blocks, capturing global transitions over extended periods. Crucially, the model is optimized in an autoencoder-decoder fashion, facilitating generalization across distinct subjects and adapting to limited training samples. Experimental findings demonstrate superior performance of LG-Sleep compared to conventional deep neural networks. Moreover, the model exhibits good performance across different sleep stages even when tasked with scoring based on limited training samples.
Figures
Reference graph
Works this paper leans on
-
[1]
The research of sleep staging based on single-lead electrocardiogram and deep neural network,
R. Wei, X. Zhang, J. Wang, and X. Dang, “The research of sleep staging based on single-lead electrocardiogram and deep neural network,” Biomed. Eng. Lett. , vol. 8, pp. 87–93, 2018
work page 2018
-
[2]
Rapid assessment of sleep-wake behavior in mice,
S. P. Fisher, S. I. H. Godinho, C. A. Pothecary, M. W. Hankins, R. G. Foster, and S. N. Peirson, “Rapid assessment of sleep-wake behavior in mice,” J. Biol. Rhythms, vol. 27, no. 1, pp. 48–58, 2012
work page 2012
-
[3]
Automated analysis of sleep–wake state in rats,
R. Stephenson, A. M. Caron, D. B. Cassel, and J. C. Kostela, “Automated analysis of sleep–wake state in rats,” J. Neurosci. Methods , vol. 184, no. 2, pp. 263–274, 2009
work page 2009
-
[4]
Robust, automated sleep scoring by a compact neural network with distributional shift correction,
Z. Barger, C. G. Frye, D. Liu, Y . Dan, and K. E. Bouchard, “Robust, automated sleep scoring by a compact neural network with distributional shift correction,” PLoS One , vol. 14, no. 12, pp. e0224642, 2019
work page 2019
-
[5]
Automatic analysis of sleep electroencephalograms by hybrid computation,
J. R. Smith, M. Negin, and A. H. Nevis, “Automatic analysis of sleep electroencephalograms by hybrid computation,” IEEE Trans. Syst. Sci. Cybern. , vol. 5, no. 4, pp. 278–284, 1969
work page 1969
-
[6]
Pattern recognition applied to sleep state classification,
D. B. Drane, W. B. Martin, and S. S. Viglione, “Pattern recognition applied to sleep state classification,” Electroencephalogr . Clin. Neurophysiol., vol. 26, no. 2, pp. 238, 1969
work page 1969
-
[7]
Sleep and its disorders in translational medicine,
L. M. Paterson, D. J. Nutt, and S. J. Wilson, “Sleep and its disorders in translational medicine,” J. Psychopharmacol., vol. 25, no. 9, pp. 1226–1234, 2011
work page 2011
-
[8]
EEG gamma frequency and sleep–wake scoring in mice: comparing two types of supervised classifiers,
J. Branka ˇck, V . I. Kukushka, A. L. Vyssotski, and A. Draguhn, “EEG gamma frequency and sleep–wake scoring in mice: comparing two types of supervised classifiers,” Brain Res. , vol. 1322, pp. 59–71, 2010
work page 2010
Show all 20 references
-
[9]
Automated scoring of pre-REM sleep in mice with deep learning,
N. Grieger, J. T. C. Schwabedal, S. Wendel, Y . Ritze, and S. Bialonski, “Automated scoring of pre-REM sleep in mice with deep learning,” Sci. Rep. , vol. 11, no. 1, pp. 1–14, 2021
2021
-
[10]
MC-SleepNet: large-scale sleep stage scoring in mice by deep neural networks,
M. Yamabe, K. Horie, H. Shiokawa, H. Funato, M. Yanagisawa, and H. Kitagawa, “MC-SleepNet: large-scale sleep stage scoring in mice by deep neural networks,” Sci. Rep. , vol. 9, no. 1, pp. 15793, 2019
2019
-
[11]
Employing a long-short-term memory neural network to improve automatic sleep stage classification of pharmaco-EEG profiles,
H. Alsolai, S. Qureshi, S. M. Zeeshan Iqbal, A. Ameer, D. Cheaha, L. E. Henesey, and S. Karrila, “Employing a long-short-term memory neural network to improve automatic sleep stage classification of pharmaco-EEG profiles,” Appl. Sci. , vol. 12, no. 10, pp. 5248, 2022
2022
-
[12]
Automated sleep scoring: A review of the latest approaches,
L. Fiorillo, A. Puiatti, M. Papandrea, P.-L. Ratti, P. Favaro, C. Roth, P. Bargiotas, C. L. Bassetti, and F. D. Faraci, “Automated sleep scoring: A review of the latest approaches,” Sleep Med. Rev. , vol. 48, pp. 101204, 2019
2019
-
[13]
A hybrid end-to-end spatio-temporal attention neural network with graph-smooth signals for EEG emotion recognition,
S. Sartipi, M. Torkamani-Azar, and M. Cetin, “A hybrid end-to-end spatio-temporal attention neural network with graph-smooth signals for EEG emotion recognition,” IEEE Trans. Cogn. Dev. Syst. , 2023
2023
-
[14]
Real-time patient-specific ECG classification by 1-d convolutional neural networks,
S. Kiranyaz, T. Ince, and M. Gabbouj, “Real-time patient-specific ECG classification by 1-d convolutional neural networks,” IEEE Trans. Biomed. Eng. , vol. 63, no. 3, pp. 664–675, 2015
2015
-
[15]
Framewise phoneme classification with bidirectional lstm and other neural network architectures,
A. Graves and J. Schmidhuber, “Framewise phoneme classification with bidirectional lstm and other neural network architectures,” Neural Netw. , vol. 18, no. 5-6, pp. 602–610, 2005
2005
-
[16]
Medical image analysis using convolutional neural networks: a review,
S. M. Anwar and et al., “Medical image analysis using convolutional neural networks: a review,” J. Med. Syst. , vol. 42, pp. 1–13, 2018
2018
-
[17]
Sleep–wake cycle in young and older mice,
S. Soltani, S. Chauvette, O. Bukhtiyarova, J.-M. Lina, J. Dubé, J. Seigneur, J. Carrier, and I. Timofeev, “Sleep–wake cycle in young and older mice,” Front. Syst. Neurosci., vol. 13, pp. 51, 2019
2019
-
[18]
Memory-enhancing properties of sleep depend on the oscillatory amplitude of norepinephrine,
C. Kjaerby and et al., “Memory-enhancing properties of sleep depend on the oscillatory amplitude of norepinephrine,” Nat. Neurosci. , vol. 25, no. 8, pp. 1059–1070, 2022
2022
-
[19]
Scoring transitions to REM sleep in rats based on the EEG phenomena of pre-REM sleep: an improved analysis of sleep structure,
J. H. Benington, S. K. Kodali, and H. C. Heller, “Scoring transitions to REM sleep in rats based on the EEG phenomena of pre-REM sleep: an improved analysis of sleep structure,” Sleep, vol. 17, no. 1, pp. 28–36, 1994
1994
-
[20]
EEG-based sleep staging analysis with functional connectivity,
H. Huang, J. Zhang, L. Zhu, J. Tang, G. Lin, W. Kong, X. Lei, and L. Zhu, “EEG-based sleep staging analysis with functional connectivity,” Sensors, vol. 21, no. 6, pp. 1988, 2021
1988
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.