REVIEW 5 major objections 4 minor 39 references
MC2SleepNet: Multi-modal Cross-masking with Contrastive Learning for Sleep Stage Classification
T0 review · 5 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read MC2SleepNet claims top sleep staging accuracy by cross-masking EEG and spectrogram views.
desk verdict A credible new multi-modal sleep staging method with a small SOTA gain, but the SHHS split is under-specified and the headline accuracy may partly reflect test-set tuning. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism is 'Cross-Masking': during sequence-level pre-training, 50% of the feature tokens in the raw-signal sequence and in the spectrogram sequence are randomly masked, and two Transformer blocks recover the masked tokens by cross-attending to the other modality's unmasked features. The load-bearing idea is that random masking plus cross-modal reconstruction forces the CNN and Transformer backbones to exchange information neither view carries alone. Supporting this are the InfoNCE loss, which aligns epoch-level embeddings from the two backbones, and a two-stage training scheme that freezes the backbones before fine-tuning the sequence head.
What would settle it
Re-run the SHHS experiment with a subject-disjoint split: ensure that all epochs from a given sleeper are assigned wholly to training, validation, or test, and compare accuracy to the reported 88.6%; if the gap is large, the generalization claim would be undermined by data leakage.
Extended reading notes
Core claim
The paper's central claim is that multi-modal sleep stage classification improves when the two input views are forced to reconstruct each other, not merely fused. MC2SleepNet passes raw 30-second EEG epochs through a CNN backbone and their log-magnitude spectrograms through a Transformer backbone; an InfoNCE loss aligns the two embeddings at epoch level, and a 'cross-masking' scheme at sequence level masks 50% of the feature tokens in each modality's sequence and uses cross-attention to recover them from the other modality's unmasked tokens. After pre-training, the backbones are frozen and only the sequence-level prediction head is fine-tuned. The paper reports 84.6% accuracy on SleepEDF-78 and 88.6% on SHHS, with per-class F1 gains in NREM1, NREM3, and REM over the XSleepNet baseline, and frames these numbers as evidence that the cross-modal reconstruction objective transfers across dataset sizes.
Load-bearing premise
The SHHS result rests on a train/test partition described only as '70:30 after excluding 100 validation patients'; if the split is not subject-wise, epochs from the same sleeper could appear on both sides and inflate accuracy.
Editorial extensions
If this is right
- The paper reports top accuracy on both datasets, with per-class F1 gains in NREM1, NREM3, and REM compared with XSleepNet on SleepEDF-78.
- Combining contrastive learning and cross-masking during pre-training gives larger gains than either alone, and the best results appear only after fine-tuning the sequence-level head with frozen backbones.
- The masking ratio acts as a class-priority dial: 15% masking favors the rare stages NREM1 and NREM3, while 50% masking favors NREM2 and REM.
- The model reaches these results with a sequence length of 21 epochs, much shorter than the L=200 used by L-SeqSleepNet, and with training time estimated to be faster than XSleepNet.
Reading between the lines
- If cross-masking is the active ingredient, the same pre-training scheme should transfer to other paired physiological modalities, such as ECG or EMG paired with their spectrograms, without changing the architecture.
- The masking-ratio results suggest a testable curriculum: starting pre-training at a high masking ratio and annealing down to 50% during fine-tuning may capture both the robustness benefit and the fine-tuning gain.
- A direct ablation that removes InfoNCE while keeping cross-masking, or vice versa, would isolate whether the two self-supervised losses are complementary or redundant; the paper only compares both together against neither.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes MC2SleepNet, a multi-modal sleep stage classifier that combines a CNN backbone on raw single-channel EEG with a Transformer backbone on spectrograms. Training uses an InfoNCE contrastive loss at epoch level and a cross-masking, cross-attention sequence-level pretraining objective, followed by fine-tuning. The authors report state-of-the-art accuracy of 84.6% on SleepEDF-78 and 88.6% on SHHS, and they claim effective generalization across small and large datasets. The main technical contribution is the integration of contrastive learning and cross-masking to align and fuse the two modalities.
Significance. If the results are reproducible and properly validated, the architecture is a meaningful contribution to sleep stage classification, showing a viable way to combine raw signals and spectrograms with modern Transformer-based sequence modeling and self-supervised pretraining. The work also provides a useful comparison point against earlier multi-modal systems such as XSleepNet. The study is empirical rather than theoretical; its value depends on the reliability of the reported accuracies and on the soundness of the evaluation protocol. The paper would benefit from releasing code or detailed pseudocode, as the described training pipeline is intricate.
major comments (5)
- [Section 4.1.1] The SHHS evaluation split is not described as subject-wise. The text says that 100 patients are excluded for validation and the remaining data are divided into training and testing at a 70:30 ratio, but it does not state whether the partition is made at the subject level. In sleep staging, consecutive epochs from the same subject are highly correlated; if epochs from the same subject appear in both training and test sets, the reported 88.6% accuracy could reflect within-subject leakage rather than generalization. Please specify explicitly that the split is subject-wise and describe the subject-level assignment procedure.
- [Table 4] The masking ratio is selected using the same SHHS test set on which the headline 88.593% accuracy is reported. Table 4 lists accuracies for masking ratios between 0% and 85%, and the final reported 88.6% is the maximum after fine-tuning at 50%. This is a test-set-selected maximum and can inflate the reported performance. The masking ratio and other hyperparameters should be selected on the validation set (the excluded 100 patients), and the test set should be used only once for the final evaluation.
- [Tables 3-5] No error bars, standard deviations, or repeated-run statistics are reported for any of the headline numbers. The reported improvements over prior work are small and could easily be within run-to-run variation: on SHHS, the margin over L-SeqSleepNet is 0.24%, and on SleepEDF-78 the accuracy is tied with SleePyco at 84.6%. Without multiple seeds or statistical significance testing, the claim of state-of-the-art performance is not supported. Please report mean and standard deviation over at least three runs for the proposed method and, where feasible, for the main baselines.
- [Section 3.2, Eq. (17)] The cross-attention equation is inconsistent with the surrounding text. The text states that O_Sg is used as a query and O_Sp as key and value, but Eq. (17) computes softmax over (W_K O_Sg) · (W_Q O_Sp)^T and multiplies by W_V O_Sg, which effectively uses O_Sg as the key and value and O_Sp as the query. This discrepancy prevents reproduction of the cross-masking mechanism. Please correct the equation or the text and ensure that the roles of query, key, and value are consistent throughout.
- [Section 4.2] The training-time comparison is not a controlled comparison. The proposed MC2SleepNet and SleepTransformer were run on an RTX 3090, while XSleepNet was run on a Tesla V100, and the V100 runtime for MC2SleepNet is estimated using a speedup ratio derived from running SleepTransformer on two different GPUs. This extrapolation does not account for differences in GPU architecture, memory, or the models' scaling behavior, so the claimed 1.32x faster training than XSleepNet is not established. Report wall-clock times for all models on the same GPU model, or at least provide raw measurements without heuristic extrapolation.
minor comments (4)
- [Table 3] The 'TF+CNN+PT+FT' row has a missing value for SleepEDF-78; please clarify whether this configuration was not run, did not converge, or was omitted for another reason.
- [Section 3.2] The sentence 'Our masked tokens are shared and learnable within an expanded batch' is unclear. Please define what 'expanded batch' means in relation to the batch size B, and describe how the mask token is initialized and updated.
- [Eqs. (14)-(15)] The InfoNCE loss denominators sum over k=1..B, which appears to include the positive sample (k=i). The text says 'negative samples are collected from a batch.' Please clarify whether the positive pair is included in the denominator as is standard for InfoNCE, or whether the sum is over negatives only.
- [Table 5] For the SleepEDF-78 rows marked with '+', the model is pre-trained on SHHS 5463 and fine-tuned on SleepEDF-78. Please clarify whether the SHHS pre-training uses the full 5463-patient set including the SHHS test portion, and whether the SleepEDF-78 10-fold cross-validation subject split is preserved when transferring.
Circularity Check
Headline SHHS accuracy is a test-set-selected maximum; no derivation-level circularity, but the masking-ratio search makes the reported 88.6% partly selected rather than independently predicted.
-
fitted input called prediction
[Section 4.2, Table 4 (masking ratio search) and Table 3 (reported highest accuracy).]
"Controlling the masking ratio is crucial for model accuracy. ... Overall, the optimal performance (88.593) was observed at the masking ratio of 50% after fine-tuning. ... Finally, with the addition of further fine-tuning, the highest accuracy (88.6%) is observed in 'TF+CNN+PT(CL+M)+FT'."
The 50% masking ratio is selected because it gives the best accuracy (88.593%) in Table 4, and that table is evaluated on the same SHHS 5463 test partition that is later used for the headline 88.6% accuracy. The reported state-of-the-art number is therefore the maximum of a hyperparameter grid evaluated on the test set itself, not an independent prediction on unseen data. The selected masking ratio is effectively fitted to the test set, so the final accuracy is statistically forced by the selection procedure rather than being a free prediction.
full rationale
There is no derivation-level circularity: MC2SleepNet is an empirical architecture paper, and the accuracy numbers come from training and testing on held-out data, so the central claim is not a tautology. The paper does not rely on load-bearing self-citations, uniqueness theorems, or ansatz-by-citation. The main circularity concern is the masking-ratio selection: Table 4 searches masking ratios on the SHHS 5463 evaluation partition, and Tables 3 and 5 then report the best result (88.6%) as the state of the art, so the headline SHHS number is a test-set-selected maximum rather than an unbiased evaluation. Separately, the SHHS 70:30 split in Section 4.1.1 is not explicitly documented as subject-wise, so within-subject epoch leakage could inflate the result; this is a missing-support/data-leakage concern, not circularity. The 84.6% SleepEDF-78 row is obtained after SHHS pre-training, which weakens the small-dataset generalization claim, but that is a comparison fairness issue rather than a circular derivation.
Assumptions & free parameters
free parameters (5)
- masking ratio =
0.5 (final)
- sequence length L =
21
- loss weights w1, w2, w3 =
1, 0.1, 0.1 (pretraining); 1, 1, 1 (fine-tuning)
- InfoNCE temperature tau =
not specified
- learning rate, weight decay, batch size =
5e-4, 1e-5, 32
assumptions (5)
- domain assumption AASM five-stage labeling (Wake, N1, N2, N3, REM) is reliable ground truth.
- domain assumption Single-channel EEG contains sufficient information for sleep staging.
- domain assumption The spectrogram computed with window 200, overlap 100, and normalization to 0-128 preserves relevant information.
- ad hoc to paper Raw signal and spectrogram views provide complementary information that can be aligned via InfoNCE.
- domain assumption Performance on SleepEDF-78 and SHHS is a valid proxy for clinical utility.
Cite this review
Pith. "Pith review of MC2SleepNet: Multi-modal Cross-masking with Contrastive Learning for Sleep Stage Classification." pith.science (2026). https://pith.science/paper/II2UNGWP
@misc{pith2026250217470,
author = {Pith},
title = {Pith review of: MC2SleepNet: Multi-modal Cross-masking with Contrastive Learning for Sleep Stage Classification},
year = {2026},
howpublished = {\url{https://pith.science/paper/II2UNGWP}},
note = {Machine review of arXiv:2502.17470}
}
read the original abstract
Sleep profoundly affects our health, and sleep deficiency or disorders can cause physical and mental problems. Despite significant findings from previous studies, challenges persist in optimizing deep learning models, especially in multi-modal learning for high-accuracy sleep stage classification. Our research introduces MC2SleepNet (Multi-modal Cross-masking with Contrastive learning for Sleep stage classification Network). It aims to facilitate the effective collaboration between Convolutional Neural Networks (CNNs) and Transformer architectures for multi-modal training with the help of contrastive learning and cross-masking. Raw single channel EEG signals and corresponding spectrogram data provide differently characterized modalities for multi-modal learning. Our MC2SleepNet has achieved state-of-the-art performance with an accuracy of both 84.6% on the SleepEDF-78 and 88.6% accuracy on the Sleep Heart Health Study (SHHS). These results demonstrate the effective generalization of our proposed network across both small and large datasets.
Figures
Reference graph
Works this paper leans on
-
[1]
Yoshua Bengio, Li Yao, Guillaume Alain, and Pascal Vincent. 2013. Generalized De- noising Auto-Encoders as Generative Models. In Advances in Neural Information Processing Systems, C.J. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K.Q. Weinberger (Eds.), Vol. 26. Curran Associates, Inc. https://proceedings.neurips. cc/paper_files/paper/2013/file/559cb...
work page 2013
-
[2]
Richard B Berry, Rita Brooks, Charlene E Gamaldo, Susan M Harding, C Marcus, Bradley V Vaughn, et al. 2012. The AASM manual for the scoring of sleep and associated events. Rules, Terminology and Technical Specifications, Darien, Illinois, American Academy of Sleep Medicine 176 (2012), 2012
work page 2012
-
[3]
Siddharth Biswal, Joshua Kulas, Haoqi Sun, Balaji Goparaju, M Brandon Westover, Matt T Bianchi, and Jimeng Sun. 2017. SLEEPNET: automated sleep staging system via deep learning. arXiv preprint arXiv:1707.08262 (2017)
work page Pith review arXiv 2017
-
[4]
Florian Chapotot and Guillaume Becq. 2010. Automated sleep–wake staging combining robust feature extraction, artificial neural network classification, and flexible decision rules. International Journal of Adaptive Control and Signal Processing 24, 5 (2010), 409–423
work page 2010
-
[5]
Hsiang-Yun Sherry Chien, Hanlin Goh, Christopher M. Sandino, and Joseph Y. Cheng. 2022. MAEEG: Masked Auto-encoder for EEG Representation Learning. arXiv:2211.02625 [eess.SP]
arXiv 2022
-
[6]
Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. 2014. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555 (2014)
arXiv 2014
-
[7]
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv:1810.04805 [cs.CL]
arXiv 2019
-
[8]
Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory.Neural computation 9, 8 (1997), 1735–1780
1997
Show all 39 references
-
[9]
Xue Jiang, Jianhui Zhao, Bo Du, and Zhiyong Yuan. 2021. Self-supervised Con- trastive Learning for EEG-based Sleep Staging. In 2021 International Joint Confer- ence on Neural Networks (IJCNN) . 1–8. https://doi.org/10.1109/IJCNN52387.2021. 9533305
2021
-
[10]
Kingma and Jimmy Ba
Diederik P. Kingma and Jimmy Ba. 2017. Adam: A Method for Stochastic Opti- mization. arXiv:1412.6980 [cs.LG]
2017 arXiv
-
[11]
Konstantinos Kontras, Christos Chatzichristos, Huy Phan, Johan Suykens, and Maarten De Vos. 2023. CoRe-Sleep: A Multimodal Fusion Framework for Time Series Robust to Imperfect Modalities. arXiv:2304.06485 [eess.SP]
2023 arXiv
-
[12]
Demetres Kostas, Stephane Aroca-Ouellette, and Frank Rudzicz. 2021. BENDR: using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG data. arXiv:2101.12037 [cs.LG]
2021 arXiv
-
[13]
Seongju Lee, Yeonguk Yu, Seunghyeok Back, Hogeon Seo, and Kyoobin Lee. 2024. Sleepyco: Automatic sleep scoring with feature pyramid and contrastive learning. Expert Systems with Applications 240 (2024), 122551
2024
-
[14]
Yujie Li, Jingrui Chen, Wenjun Ma, Gansen Zhao, and Xiaomao Fan. 2022. MVF- sleepnet: Multi-view fusion network for sleep stage classification. IEEE Journal of Biomedical and Health Informatics (2022)
2022
-
[15]
Kuna, Ruth Benca, Clete A
Atul Malhotra, Magdy Younes, Samuel T. Kuna, Ruth Benca, Clete A. Kushida, James Walsh, Alexandra Hanlon, Bethany Staley, Allan I. Pack, and Grace W. Pien. 2013. Performance of an Automated Polysomnography Scoring System Versus Computer-Assisted Manual Scoring. Sleep 36, 4 (04...
2013 doi
-
[16]
Nicola Michielli, U Rajendra Acharya, and Filippo Molinari. 2019. Cascaded LSTM recurrent neural network for automated sleep stage classification using single-channel EEG signals. Computers in biology and medicine 106 (2019), 71–81
2019
-
[17]
Sajad Mousavi, Fatemeh Afghah, and U Rajendra Acharya. 2019. SleepEEGNet: Automated sleep stage scoring with sequence to sequence deep learning approach. PloS one 14, 5 (2019), e0216456
2019
-
[18]
Rajendra Acharya
Sajad Mousavi, Fatemeh Afghah, and U. Rajendra Acharya. 2019. SleepEEGNet: Automated sleep stage scoring with sequence to sequence deep learning approach. PLOS ONE 14, 5 (05 2019), 1–15. https://doi.org/10.1371/journal.pone.0216456
2019 doi
-
[19]
Edgar Oropesa, Hans L Cycon, and Marc Jobert. 1999. Sleep stage classification using wavelet transform and neural network. International computer science institute 2 (1999)
1999
-
[20]
Mathias Perslev, Michael Hejselbak Jensen, Sune Darkner, Poul Jørgen Jennum, and Christian Igel. 2019. U-Time: A Fully Convolutional Network for Time Series Segmentation Applied to Sleep Staging. arXiv:1910.11162 [cs.LG]
2019 arXiv
-
[21]
Huy Phan, Fernando Andreotti, Navin Cooray, Oliver Y Chén, and Maarten De Vos. 2018. Joint classification and prediction CNN framework for automatic sleep stage classification. IEEE Transactions on Biomedical Engineering 66, 5 (2018), 1285–1296
2018
-
[22]
Huy Phan, Fernando Andreotti, Navin Cooray, Oliver Y Chén, and Maarten De Vos. 2019. SeqSleepNet: end-to-end hierarchical recurrent neural network for sequence-to-sequence automatic sleep staging. IEEE Transactions on Neural Systems and Rehabilitation Engineering 27, 3 (2019), 400–410
2019
-
[23]
Chèn, and Maarten De Vos
Huy Phan, Fernando Andreotti, Navin Cooray, Oliver Y. Chèn, and Maarten De Vos. 2018. DNN Filter Bank Improves 1-Max Pooling CNN for Single-Channel EEG Automatic Sleep Stage Classification. In 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Bi...
2018
-
[24]
Huy Phan, Oliver Y Chén, Minh C Tran, Philipp Koch, Alfred Mertins, and Maarten De Vos. 2021. XSleepNet: Multi-view sequential model for automatic sleep staging. IEEE Transactions on Pattern Analysis and Machine Intelligence 44, 9 (2021), 5903–5915
2021
-
[25]
Huy Phan, Kristian P Lorenzen, Elisabeth Heremans, Oliver Y Chén, Minh C Tran, Philipp Koch, Alfred Mertins, Mathias Baumert, Kaare B Mikkelsen, and Maarten De Vos. 2023. L-SeqSleepNet: Whole-cycle long sequence modelling for automatic sleep staging. IEEE Journal of Biomedical...
2023
-
[26]
Huy Phan, Kaare Mikkelsen, Oliver Y Chén, Philipp Koch, Alfred Mertins, and Maarten De Vos. 2022. Sleeptransformer: Automatic sleep staging with inter- pretability and uncertainty quantification. IEEE Transactions on Biomedical Engineering 69, 8 (2022), 2456–2467
2022
-
[27]
Allan Rechtschaffen. 1968. A manual of standardized terminology, techniques and scoring system for sleep stages of human subjects. 睡眠波アトラス標準用語 手技 判定法 (1968), 1–55
1968
-
[28]
Hogeon Seo, Seunghyeok Back, Seongju Lee, Deokhwan Park, Tae Kim, and Kyoobin Lee. 2020. Intra- and inter-epoch temporal context network (IITNet) using sub-epoch features for automatic sleep scoring on raw single-channel EEG. Biomedical Signal Processing and Control 61 (2020),...
2020
-
[29]
Kihyuk Sohn, Wenling Shang, and Honglak Lee. 2014. Improved Multimodal Deep Learning with Variation of Information. In Advances in Neural Information Processing Systems, Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K.Q. Weinberger (Eds.), Vol. 27. Curran Associates, ...
2014
-
[30]
Arnaud Sors, Stéphane Bonnet, Sébastien Mirek, Laurent Vercueil, and Jean- François Payen. 2018. A convolutional neural network for sleep stage scoring from raw single-channel EEG. Biomedical Signal Processing and Control 42 (2018), 107–114
2018
-
[31]
Arnaud Sors, Stéphane Bonnet, Sébastien Mirek, Laurent Vercueil, and Jean- François Payen. 2018. A convolutional neural network for sleep stage scoring from raw single-channel EEG. Biomedical Signal Processing and Control 42 (2018), 107–114. https://doi.org/10.1016/j.bspc.2017.12.001
2018 doi
-
[32]
Jingpeng Sun, Rongxiao Wang, Gangming Zhao, Chen Chen, Yixiao Qu, Jinpeng Li, Xiyuan Hu, and Yizhou Yu. 2023. START: Automatic Sleep Staging with Attention-based Cross-modal Learning Transformer. In 2023 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) . ...
2023
-
[33]
Akara Supratak, Hao Dong, Chao Wu, and Yike Guo. 2017. DeepSleepNet: A model for automatic sleep stage scoring based on raw single-channel EEG. IEEE Transactions on Neural Systems and Rehabilitation Engineering 25, 11 (2017), 1998– 2008
2017
-
[35]
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2019. Representation Learning with Contrastive Predictive Coding. arXiv:1807.03748 [cs.LG]
2019 arXiv
-
[36]
Weiyao Wang, Du Tran, and Matt Feiszli. 2019. What Makes Training Multi- Modal Networks Hard? CoRR abs/1905.12681 (2019). arXiv:1905.12681 http: //arxiv.org/abs/1905.12681
2019 arXiv
-
[37]
Katharina Wulff, Silvia Gatti, Joseph G Wettstein, and Russell G Foster. 2010. Sleep and circadian rhythm disruption in psychiatric and neurodegenerative disease. Nature Reviews Neuroscience 11, 8 (2010), 589–599
2010
-
[38]
Jianan Ye, Qinfeng Xiao, Jing Wang, Hongjun Zhang, Jiaoxue Deng, and Youfang Lin. 2022. CoSleep: A Multi-View Representation Learning Framework for Self- Supervised Learning of Sleep Stage Classification. IEEE Signal Processing Letters 29 (2022), 189–193. https://doi.org/10.11...
2022
-
[39]
Hangyu Zhu, Wei Zhou, Cong Fu, Yonglin Wu, Ning Shen, Feng Shu, Huan Yu, Wei Chen, and Chen Chen. 2023. MaskSleepNet: A Cross-Modality Adaptation Neural Network for Heterogeneous Signals Processing in Sleep Staging. IEEE Journal of Biomedical and Health Informatics 27, 5 (2023...
2023
-
[456]
https://doi.org/10.1109/EMBC.2018.8512286
2018
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.