Pith. sign in

REVIEW 4 major objections 6 minor 51 references

sDREAMER: Self-distilled Mixture-of-Modality-Experts Transformer for Automatic Sleep Staging

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read One transformer beats sleep-staging baselines on EEG, EMG, or both

desk verdict A sensible adaptation of MoME plus self-distillation to mouse sleep staging, with informative ablations, but the comparative claims rest on single runs with no error bars and there's a typo in Eq 14. read the letter →

arxiv 2501.16329 v1 pith:ECGOWJEX submitted 2025-01-27 cs.LG cs.AI

Jingyuan Chen , Yuan Yao , Mie Anderson , Natalie Hauglund , Celia Kjaerby , Verena Untiet , Maiken Nedergaard , Jiebo Luo This is my paper · ORCID
classification cs.LGcs.AI
keywords sleepstagingmixture-of-modalityexpertsself-distillationtransformerEEG-EMGmultimodalmousedataset
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that automatic sleep staging can be done with a single transformer that is trained on multi-channel EEG and EMG signals yet remains accurate when given only one channel at test time. The proposed sDREAMER model gives each modality its own expert pathway while sharing attention weights, and adds a mixture pathway whose predictions distill knowledge back into the EEG and EMG pathways. On a mouse sleep dataset with a subject-wise split, the model reports higher accuracy and F1 scores than the transformer baselines it is compared against, in both the epoch-level and sequence-level settings.

What carries the argument

The central mechanism is the mixture-of-modality-experts (MoME) module: a transformer layer with a shared multi-head self-attention and three modality-specific feed-forward networks, called experts, for EEG, EMG, and their concatenation. A mapping function $\psi(m, \ell)$ chooses which expert processes a token at a given layer, so the same model routes EEG, EMG, or mixed tokens through their own pathways. The self-distillation loss, a temperature-scaled KL divergence from the mix pathway's logits to the EEG and EMG logits, transfers knowledge from the multi-modal pathway to the mono-modal ones. This design is what lets a single model handle both single-channel and multi-channel inference.

What would settle it

Have two or more independent experts score the same recordings, train and evaluate sDREAMER against a consensus or majority label, and compare its accuracy with the transformer baselines under the same labels; if the gap over baselines disappears or reverses, the claimed advantage would be specific to a single annotator.

Watch

Extended reading notes

Core claim

The central claim is that a mixture-of-modality-experts transformer with partially shared weights and self-distillation outperforms the existing transformer-based sleep staging methods on a mouse EEG-EMG dataset. The model is trained with both channels but offers three inference paths: EEG-only, EMG-only, or combined, and the authors report that each path is competitive, with the multi-channel path reaching 91.72% accuracy and 87.64% F1. The authors interpret this as evidence that cross-modal interaction during training, rather than late fusion, improves sleep staging, and that self-distillation lets mono-modal pathways benefit from multi-modal supervision.

Load-bearing premise

The evaluation treats the sleep-stage labels produced by a single expert as ground truth; if those labels are noisy or idiosyncratic, the reported accuracy and baseline comparisons could reflect fitting one person's annotation style rather than objective sleep stages.

Editorial extensions

If this is right

  • A single trained model could serve sleep labs regardless of whether they record EEG, EMG, or both channels.
  • Single-channel inference from a multi-channel-trained model could make at-home sleep monitoring with a wearable EEG electrode more practical.
  • The self-distillation scheme could be applied to other multi-modal physiological signals where a channel may be missing at deployment.
  • The architecture could be retrained for other species or for human sleep staging, since it does not depend on mouse-specific features.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The self-distillation from the mix pathway to the mono-modal pathways may act as a regularizer that prevents overfitting to the annotation style of a single expert.
  • A testable extension is to apply the same training scheme to human polysomnography datasets and check whether the cross-modal supervision transfers when only a single EEG channel is available.
  • The shared-attention design could be probed with interpretability tools to see whether the mix pathway learns distinct sleep-stage signatures rather than simply averaging the modalities.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes sDREAMER, a transformer architecture for automatic sleep staging from EEG and EMG signals. The model has three pathways (EEG, EMG, and mixed) with partially shared weights in a mixture-of-modality-experts (MoME) module, and it is trained with a self-distillation scheme where the mixed pathway distills knowledge to the mono-modal pathways. This allows the model to be trained on multi-channel signals and then used for either multi-channel or single-channel inference. The authors evaluate epoch-level and sequence-level variants on a mouse sleep dataset with expert labels, reporting accuracy and F1 for both multi-modal and mono-modal settings, and claim outperformance over existing transformer-based baselines.

Significance. If the empirical claims hold, the proposed architecture is a useful step toward unified sleep staging models that can accommodate both multi-channel and single-channel inputs. The idea of using a multi-modal pathway as a teacher for mono-modal pathways is interesting, and the paper includes extensive ablations and comparisons to multiple baselines. However, the strength of the central claim is currently limited by single-run evaluations without statistical significance testing, self-implemented and unreleased baselines, and an internal inconsistency in the formal definition of the self-distillation loss.

major comments (4)
  1. [IV-F, Eq. (14)] The total loss is written as L_mome = (1-alpha)Lce + (alpha/2)(Lsd-emg + Lsd-emg), which omits Lsd-eeg and duplicates Lsd-emg. The surrounding text and Table III clearly indicate that both EEG and EMG self-distillation are used. This makes the formal definition inconsistent with the described method and the ablation. Please correct the equation and clarify which loss was actually implemented.
  2. [V-D, Tables I and II] The central comparative claim rests on single point estimates with no standard deviations, confidence intervals, or significance tests. For example, the margin over the strongest sequence baseline is 0.79 accuracy points (91.72 vs 90.93), which could easily be within seed-to-seed variation for transformer training. Please report results over multiple runs (e.g., 5 seeds) with mean and standard deviation, and ideally perform paired significance tests for the main comparisons.
  3. [V-C, Tables I and II] The baseline models are self-implemented, but no code, hyperparameter tuning budgets, or verification that the re-implemented Cross-Modal Transformer [32] matches its original configuration are provided. This weakens the claim that sDREAMER 'outperforms the existing transformer-based sleep scoring methods,' since the comparison is only against the authors' reimplementations. Please describe the tuning process for baselines and, if possible, release the baseline implementations.
  4. [V-E, Table III] The no-self-distillation ablation shows EEG-Acc of 62.71% and EMG-Acc of 7.85%, which suggests that without distillation the mono-modal pathways receive essentially no training signal, because the only cross-entropy loss defined (Eq. 13) is on the mix pathway. This ablation therefore conflates 'no self-distillation' with 'mono-modal pathways untrained,' so the large improvements from adding self-distillation do not cleanly demonstrate the benefit of the distillation mechanism. Please clarify the training losses for each pathway and provide an ablation where the mono-modal pathways are trained with an equivalent supervised loss without distillation.
minor comments (6)
  1. [V-A] The text says 'resulting in a total of 10,000 epoch data samples,' but the class distributions in Fig. 5 sum to far more than 10,000 (the training set alone exceeds 90,000). Please correct the dataset size description and provide the actual number of subjects, epochs, and class counts.
  2. [Table I] The Cross-Modal Transformer [32] rows report accuracy but no F1-score, despite the text stating that both metrics are reported. Please provide F1 values or explain the omission.
  3. [IV-B, Eq. (5)] The residual term T^{m'}_ell is not defined in the paper; it likely should be T^m_{ell-1} or a similar expression. Please clarify the notation.
  4. [IV-D and IV-F] The text in Section IV-D says all three pathways are enabled during training, but Eq. (13) defines the cross-entropy loss only for the mix pathway. Please clarify whether the EEG and EMG classifiers also receive a direct cross-entropy loss during training or are trained solely through the self-distillation loss.
  5. [I] There is a duplicated word in 'EEG time series signals signals' in the Introduction; please fix this typo.
  6. [References] Reference [24] is cited for the 'Channel-Independent Transformer' baseline, but the cited paper is a time-series forecasting work, not a sleep staging method. Please clarify how this baseline was adapted or use a more specific reference.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: sDREAMER's performance claims rest on held-out test evaluation; self-distillation and single-channel inference are training and evaluation procedures, not quantities defined in terms of fitted parameters.

full rationale

The paper's central claims are empirical accuracy and F1 comparisons on a subject-wise held-out test set. Section V-A states that 'the records of 12 mice subjects were used for training, while the records of 4 mice subjects were used for testing,' and Tables I and II report measurements against expert labels on those test subjects. No equation defines a predicted quantity in terms of the fitted parameters or in terms of the claimed output. The single-channel results in Table II are obtained by feeding only EEG or EMG to the trained model and evaluating against expert labels; they are not the distillation pseudo-targets. The self-distillation losses in Equations 11, 12, and 14 are training regularizers, and their contribution is assessed empirically through the ablation in Table III, not assumed by construction. The dataset citation [18] involves overlapping authors, but it is used as a data source rather than as justification for the model's superiority; the labels are manually produced and the paper explicitly concedes in Section VI that 'the performance of the model is currently compared to one expert, who is well-trained but not perfect.' That limitation affects external validity, not circularity of the derivation. One internal inconsistency is present in Equation 14, which writes the total loss as (1-alpha)Lce + (alpha/2)(Lsd-emg + Lsd-emg), duplicating the EMG term and omitting Lsd-eeg defined in Equation 11 and credited in Table III; this is a correctness and reproducibility concern, but it does not make any reported result equivalent to its input. No uniqueness theorem is imported from the authors' prior work, and the MoME and self-distillation ideas are attributed to external prior works [5] and [44]. The comparative claim is single-run and lacks confidence intervals or significance tests, but that is a statistical-robustness issue, not a circularity.

Assumptions & free parameters 10 free parameters · 5 assumptions · 0 invented entities

Free parameters are training and architecture hyperparameters chosen by hand rather than fitted to data. The axioms are mostly domain assumptions about labels and preprocessing plus two architectural premises validated only by internal ablations. The heaviest external burden is the unverified single-expert ground truth.

free parameters (10)
  • distillation weight α = 0.33
    Set in Section V-B to balance cross-entropy and KL distillation loss in Eq 14; affects mono-modal versus multi-modal performance.
  • EEG distillation temperature τ_eeg = 1.0
    Softening factor in Eq 10 for the EEG pathway, chosen in Section V-B.
  • EMG distillation temperature τ_emg = 3.0
    Softening factor in Eq 10 for the EMG pathway, chosen in Section V-B.
  • sequence length K = 16
    Number of consecutive epochs in the sequence setting, used in Table I and Section IV-E.
  • MoME layer counts = 4-layer epoch; sequence: 2 epoch-level + 3 sequence-level
    Architecture depth set in Section V-B; only the last layer of Epoch sDREAMER and the final sequence layer contain the mix expert.
  • expert FFN dimension = 512
    Feed-forward network dimension for each modality expert, stated in Section V-B.
  • learning rate = 1e-3
    AdamW optimizer setting, stated in Section V-B.
  • weight decay = 1e-4
    AdamW optimizer setting, stated in Section V-B.
  • batch size = 256 epoch / 16 sequence
    Section V-B reports different batch sizes for the epoch and sequence models.
  • patch length W, token dimension D, heads, dropout, training epochs
    Referenced in Section IV-B and V-B but not reported numerically; these hand-chosen hyperparameters are central to replication.
assumptions (5)
  • domain assumption The single expert's Wake/SWS/REM annotations are reliable ground truth.
    Section III-A describes expert labeling; Section VI concedes only one expert was used.
  • domain assumption Unlabeled epochs can be safely ignored during loss computation without biasing learning.
    Section III-B 'Irregularity Removal' drops missing labels from the loss, with no analysis of possible bias.
  • ad hoc to paper Shared multi-head self-attention across EEG, EMG, and mix pathways promotes beneficial cross-modal alignment.
    Section IV-C states this design; it is an architectural premise validated only by the paper's own ablations.
  • ad hoc to paper Distilling from the mix pathway's logits to the EEG/EMG pathways transfers information useful for mono-modal scoring.
    Section IV-F defines the self-distillation objective; its benefit is demonstrated only in Table III on this dataset.
  • domain assumption Subject-wise normalization and temporal slicing preserve stationarity and avoid data leakage.
    Section III-B describes these preprocessing steps, but no validation or leakage analysis is reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of sDREAMER: Self-distilled Mixture-of-Modality-Experts Transformer for Automatic Sleep Staging." pith.science (2026). https://pith.science/paper/ECGOWJEX

@misc{pith2026250116329,
  author       = {Pith},
  title        = {Pith review of: sDREAMER: Self-distilled Mixture-of-Modality-Experts Transformer for Automatic Sleep Staging},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ECGOWJEX}},
  note         = {Machine review of arXiv:2501.16329}
}
read the original abstract

Automatic sleep staging based on electroencephalography (EEG) and electromyography (EMG) signals is an important aspect of sleep-related research. Current sleep staging methods suffer from two major drawbacks. First, there are limited information interactions between modalities in the existing methods. Second, current methods do not develop unified models that can handle different sources of input. To address these issues, we propose a novel sleep stage scoring model sDREAMER, which emphasizes cross-modality interaction and per-channel performance. Specifically, we develop a mixture-of-modality-expert (MoME) model with three pathways for EEG, EMG, and mixed signals with partially shared weights. We further propose a self-distillation training scheme for further information interaction across modalities. Our model is trained with multi-channel inputs and can make classifications on either single-channel or multi-channel inputs. Experiments demonstrate that our model outperforms the existing transformer-based sleep scoring methods for multi-channel inference. For single-channel inference, our model also outperforms the transformer-based models trained with single-channel signals.

Figures

Figures reproduced from arXiv: 2501.16329 by the authors.

Figure 1
Figure 1. The raw EMG and EEG time-series signals. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The overall structure of the mixture-of-modality-experts module and the epoch sDREAMER model. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Overview of the sequence sDREAMER model. The structure of MoME module has already been illustrated in Fig. 2. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (4 more)
Figure 5
Figure 5. Figure 5: Bar plot of the class distribution, including (a) training [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 4
Figure 4. Figure 4: An example of the collected raw data. The two [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 6
Figure 6. Figure 6: Visualization of learned embeddings using t-SNE. [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Visualization of the Predictions given by the sDREAMER. [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

51 extracted references · 36 canonical work pages

  1. [32]

    Toward Interpretable Sleep Stage Classification Using Cross-Modal Transformers

    Jathurshan Pradeepkumar, Mithunjha Anandakumar, Vinith Kugathasan, Dhinesh Suntharalingham, Simon L Kappel, Anjula C De Silva, and Chamira US Edussooriya. Towards interpretable sleep stage classifica- tion using cross-modal transformers. arXiv preprint arXiv:2208.06991 , 2022

    work page Pith review arXiv 2022

  2. [1]

    A deep learning algorithm for sleep stage scoring in mice based on a multimodal network with fine-tuning technique

    Keishi Akada, Takuya Yagi, Yuji Miura, Carsten T Beuckmann, Noriyuki Koyama, and Ken Aoshima. A deep learning algorithm for sleep stage scoring in mice based on a multimodal network with fine-tuning technique. Neuroscience Research, 173:99–105, 2021

    work page 2021

  3. [2]

    Towards understanding ensemble, knowledge distillation and self-distillation in deep learning

    Zeyuan Allen-Zhu and Yuanzhi Li. Towards understanding ensemble, knowledge distillation and self-distillation in deep learning. arXiv preprint arXiv:2012.09816, 2020

    arXiv 2012

  4. [3]

    Robust cross- modal representation learning with progressive self-distillation

    Alex Andonian, Shixing Chen, and Raffay Hamid. Robust cross- modal representation learning with progressive self-distillation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16430–16441, 2022

    work page 2022

  5. [4]

    Multichannel sleep stage classification and transfer learning using convolutional neural networks

    Fernando Andreotti, Huy Phan, Navin Cooray, Christine Lo, Michele TM Hu, and Maarten De V os. Multichannel sleep stage classification and transfer learning using convolutional neural networks. In 2018 40th annual international conference of the IEEE engineering in medicine and biology society (EMBC) , pages 171–174. IEEE, 2018

    work page 2018

  6. [5]

    Vlmo: Unified vision-language pre-training with mixture-of-modality- experts

    Hangbo Bao, Wenhui Wang, Li Dong, Qiang Liu, Owais Khan Mo- hammed, Kriti Aggarwal, Subhojit Som, Songhao Piao, and Furu Wei. Vlmo: Unified vision-language pre-training with mixture-of-modality- experts. Advances in Neural Information Processing Systems, 35:32897– 32912, 2022

    2022

  7. [6]

    Sleepnet: au- tomated sleep staging system via deep learning

    Siddharth Biswal, Joshua Kulas, Haoqi Sun, Balaji Goparaju, M Bran- don Westover, Matt T Bianchi, and Jimeng Sun. Sleepnet: au- tomated sleep staging system via deep learning. arXiv preprint arXiv:1707.08262, 2017

    arXiv 2017

  8. [7]

    Xgboost: A scalable tree boosting system

    Tianqi Chen and Carlos Guestrin. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining , pages 785–794, 2016

    2016

Show all 51 references
  1. [8]

    Maskclip: Masked self-distillation advances contrastive language- image pretraining

    Xiaoyi Dong, Yinglin Zheng, Jianmin Bao, Ting Zhang, Dongdong Chen, Hao Yang, Ming Zeng, Weiming Zhang, Lu Yuan, Dong Chen, et al. Maskclip: Masked self-distillation advances contrastive language- image pretraining. arXiv preprint arXiv:2208.12262 , 2022

    2022 arXiv

  2. [9]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weis- senborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arX...

    2010 arXiv

  3. [10]

    An attention-based deep learning approach for sleep stage classification with single-channel eeg

    Emadeldeen Eldele, Zhenghua Chen, Chengyu Liu, Min Wu, Chee- Keong Kwoh, Xiaoli Li, and Cuntai Guan. An attention-based deep learning approach for sleep stage classification with single-channel eeg. IEEE Transactions on Neural Systems and Rehabilitation Engineering , 29:809–818, 2021

    2021

  4. [11]

    Learning lightweight lane detection cnns by self attention distillation

    Yuenan Hou, Zheng Ma, Chunxiao Liu, and Chen Change Loy. Learning lightweight lane detection cnns by self attention distillation. In Pro- ceedings of the IEEE/CVF international conference on computer vision , pages 1013–1021, 2019

    2019

  5. [12]

    An improved neural network based on senet for sleep stage classification

    Jing Huang, Lifeng Ren, Xiaokang Zhou, and Ke Yan. An improved neural network based on senet for sleep stage classification. IEEE Journal of Biomedical and Health Informatics, 26(10):4948–4956, 2022

    2022

  6. [13]

    Scaling up visual and vision-language representation learning with noisy text supervision

    Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representation learning with noisy text supervision. In International Conference on Machine Learning , pages 4904–4916...

    2021

  7. [14]

    Multi-modal physiological signals based squeeze-and-excitation network with domain adversarial learning for sleep staging

    Ziyu Jia, Xiyang Cai, and Zehui Jiao. Multi-modal physiological signals based squeeze-and-excitation network with domain adversarial learning for sleep staging. IEEE Sensors Journal , 22(4):3464–3471, 2022

    2022

  8. [15]

    Salientsleepnet: Multimodal salient wave detection network for sleep staging

    Ziyu Jia, Youfang Lin, Jing Wang, Xuehui Wang, Peiyi Xie, and Yingbin Zhang. Salientsleepnet: Multimodal salient wave detection network for sleep staging. arXiv preprint arXiv:2105.13864 , 2021

    2021 arXiv

  9. [16]

    Neural controlled differential equations for irregular time series

    Patrick Kidger, James Morrill, James Foster, and Terry Lyons. Neural controlled differential equations for irregular time series. Advances in Neural Information Processing Systems , 33:6696–6707, 2020

    2020

  10. [17]

    Automatic sleep stage classification using eeg and emg signal

    Hyungjik Kim and Sunwoong Choi. Automatic sleep stage classification using eeg and emg signal. In 2018 Tenth International Conference on Ubiquitous and Future Networks (ICUFN), pages 207–212. IEEE, 2018

    2018

  11. [18]

    Memory-enhancing properties of sleep depend on the os- cillatory amplitude of norepinephrine

    Celia Kjaerby, Mie Andersen, Natalie Hauglund, Verena Untiet, Camilla Dall, Björn Sigurdsson, Fengfei Ding, Jiesi Feng, Yulong Li, Pia Weikop, et al. Memory-enhancing properties of sleep depend on the os- cillatory amplitude of norepinephrine. Nature neuroscience, 25(8):1059– ...

    2022

  12. [19]

    Align before fuse: Vision and language representation learning with momentum distillation

    Junnan Li, Ramprasaath Selvaraju, Akhilesh Gotmare, Shafiq Joty, Caiming Xiong, and Steven Chu Hong Hoi. Align before fuse: Vision and language representation learning with momentum distillation. Ad- vances in neural information processing systems , 34:9694–9705, 2021

    2021

  13. [20]

    Decoupled weight decay regulariza- tion

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regulariza- tion. arXiv preprint arXiv:1711.05101 , 2017

    2017 arXiv

  14. [21]

    Personalizing deep learning models for automatic sleep staging

    Kaare Mikkelsen and Maarten De V os. Personalizing deep learning models for automatic sleep staging. arXiv preprint arXiv:1801.02645 , 2018

    2018 arXiv

  15. [22]

    Spin- dle: End-to-end learning from eeg/emg to extrapolate animal sleep scor- ing across experimental settings, labs and species

    Ðor ¯de Miladinovi ´c, Christine Muheim, Stefan Bauer, Andrea Spinnler, Daniela Noain, Mojtaba Bandarabadi, Benjamin Gallusser, Gabriel Krummenacher, Christian Baumann, Antoine Adamantidis, et al. Spin- dle: End-to-end learning from eeg/emg to extrapolate animal sleep scor- in...

    2019

  16. [23]

    Self-distillation amplifies regularization in hilbert space

    Hossein Mobahi, Mehrdad Farajtabar, and Peter Bartlett. Self-distillation amplifies regularization in hilbert space. Advances in Neural Information Processing Systems, 33:3351–3361, 2020

    2020

  17. [24]

    A time series is worth 64 words: Long-term forecasting with transformers

    Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. arXiv preprint arXiv:2211.14730 , 2022

    2022 arXiv

  18. [25]

    Stqs: Interpretable multi-modal spatial-temporal- sequential model for automatic sleep scoring

    Shreyasi Pathak, Changqing Lu, Sunil Belur Nagaraj, Michel van Putten, and Christin Seifert. Stqs: Interpretable multi-modal spatial-temporal- sequential model for automatic sleep scoring. Artificial Intelligence in Medicine, 114:102038, 2021

    2021

  19. [26]

    Automatic sleep stage classification using single- channel eeg: Learning sequential features with attention-based recurrent neural networks

    Huy Phan, Fernando Andreotti, Navin Cooray, Oliver Y Chén, and Maarten De V os. Automatic sleep stage classification using single- channel eeg: Learning sequential features with attention-based recurrent neural networks. In 2018 40th annual international conference of the IEEE...

    2018

  20. [27]

    Dnn filter bank improves 1-max pooling cnn for single-channel eeg automatic sleep stage classification

    Huy Phan, Fernando Andreotti, Navin Cooray, Oliver Y Chén, and Maarten De V os. Dnn filter bank improves 1-max pooling cnn for single-channel eeg automatic sleep stage classification. In 2018 40th annual international conference of the IEEE engineering in medicine and biology ...

    2018

  21. [28]

    Joint classification and prediction cnn framework for automatic sleep stage classification

    Huy Phan, Fernando Andreotti, Navin Cooray, Oliver Y Chén, and Maarten De V os. Joint classification and prediction cnn framework for automatic sleep stage classification. IEEE Transactions on Biomedical Engineering, 66(5):1285–1296, 2018

    2018

  22. [29]

    Seqsleepnet: end-to-end hierarchical recurrent neural network for sequence-to-sequence automatic sleep staging

    Huy Phan, Fernando Andreotti, Navin Cooray, Oliver Y Chén, and Maarten De V os. Seqsleepnet: end-to-end hierarchical recurrent neural network for sequence-to-sequence automatic sleep staging. IEEE Trans- actions on Neural Systems and Rehabilitation Engineering , 27(3):400– 410, 2019

    2019

  23. [30]

    Xsleepnet: Multi-view sequential model for automatic sleep staging

    Huy Phan, Oliver Y Chén, Minh C Tran, Philipp Koch, Alfred Mertins, and Maarten De V os. Xsleepnet: Multi-view sequential model for automatic sleep staging. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021

    2021

  24. [31]

    Sleeptransformer: Automatic sleep staging with interpretability and uncertainty quantification

    Huy Phan, Kaare Mikkelsen, Oliver Y Chén, Philipp Koch, Alfred Mertins, and Maarten De V os. Sleeptransformer: Automatic sleep staging with interpretability and uncertainty quantification. IEEE Trans- actions on Biomedical Engineering , 69(8):2456–2467, 2022

    2022

  25. [33]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning , p...

    2021

  26. [34]

    Automated classification of sleep stages and eeg artifacts in mice with deep learning

    Justus TC Schwabedal, Daniel Sippel, Moritz D Brandt, and Stephan Bialonski. Automated classification of sleep stages and eeg artifacts in mice with deep learning. arXiv preprint arXiv:1809.08443 , 2018

    2018 arXiv

  27. [35]

    Self-distillation from the last mini-batch for consistency regularization

    Yiqing Shen, Liwu Xu, Yuzhe Yang, Yaqian Li, and Yandong Guo. Self-distillation from the last mini-batch for consistency regularization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11943–11952, 2022

    2022

  28. [36]

    A convolutional neural network for sleep stage scoring from raw single-channel eeg

    Arnaud Sors, Stéphane Bonnet, Sébastien Mirek, Laurent Vercueil, and Jean-François Payen. A convolutional neural network for sleep stage scoring from raw single-channel eeg. Biomedical Signal Processing and Control, 42:107–114, 2018

    2018

  29. [37]

    Neural network analysis of sleep stages enables efficient diagnosis of narcolepsy

    Jens B Stephansen, Alexander N Olesen, Mads Olsen, Aditya Ambati, Eileen B Leary, Hyatt E Moore, Oscar Carrillo, Ling Lin, Fang Han, Han Yan, et al. Neural network analysis of sleep stages enables efficient diagnosis of narcolepsy. Nature communications, 9(1):5229, 2018

    2018

  30. [38]

    Deepsleepnet: A model for automatic sleep stage scoring based on raw single-channel eeg

    Akara Supratak, Hao Dong, Chao Wu, and Yike Guo. Deepsleepnet: A model for automatic sleep stage scoring based on raw single-channel eeg. IEEE Transactions on Neural Systems and Rehabilitation Engineering , 25(11):1998–2008, 2017

    1998

  31. [39]

    Lxmert: Learning cross-modality encoder representations from transformers

    Hao Tan and Mohit Bansal. Lxmert: Learning cross-modality encoder representations from transformers. arXiv preprint arXiv:1908.07490 , 2019

    1908 arXiv

  32. [40]

    Visualizing data using t-sne

    Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research , 9(11), 2008

    2008

  33. [41]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems , 30, 2017

    2017

  34. [42]

    Deep convolutional neural networks for interpretable analysis of eeg sleep stage scoring

    Albert Vilamala, Kristoffer H Madsen, and Lars K Hansen. Deep convolutional neural networks for interpretable analysis of eeg sleep stage scoring. In 2017 IEEE 27th international workshop on machine learning for signal processing (MLSP) , pages 1–6. IEEE, 2017

    2017

  35. [43]

    Image as a foreign language: Beit pretraining for all vision and vision-language tasks

    Wenhui Wang, Hangbo Bao, Li Dong, Johan Bjorck, Zhiliang Peng, Qiang Liu, Kriti Aggarwal, Owais Khan Mohammed, Saksham Singhal, Subhojit Som, et al. Image as a foreign language: Beit pretraining for all vision and vision-language tasks. arXiv preprint arXiv:2208.10442 , 2022

    2022 arXiv

  36. [44]

    Distilled dual-encoder model for vision-language understand- ing

    Zekun Wang, Wenhui Wang, Haichao Zhu, Ming Liu, Bing Qin, and Furu Wei. Distilled dual-encoder model for vision-language understand- ing. arXiv preprint arXiv:2112.08723 , 2021

    2021 arXiv

  37. [45]

    Snapshot distilla- tion: Teacher-student optimization in one generation

    Chenglin Yang, Lingxi Xie, Chi Su, and Alan L Yuille. Snapshot distilla- tion: Teacher-student optimization in one generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2859–2868, 2019

    2019

  38. [46]

    Cross-modality deep feature learning for brain tumor segmentation

    Dingwen Zhang, Guohai Huang, Qiang Zhang, Jungong Han, Junwei Han, and Yizhou Yu. Cross-modality deep feature learning for brain tumor segmentation. Pattern Recognition, 110:107562, 2021

    2021

  39. [47]

    Be your own teacher: Improve the performance of convolutional neural networks via self distillation

    Linfeng Zhang, Jiebo Song, Anni Gao, Jingwei Chen, Chenglong Bao, and Kaisheng Ma. Be your own teacher: Improve the performance of convolutional neural networks via self distillation. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 3713– 3722, 2019

    2019

  40. [48]

    Vinvl: Revisiting visual representations in vision-language models

    Pengchuan Zhang, Xiujun Li, Xiaowei Hu, Jianwei Yang, Lei Zhang, Lijuan Wang, Yejin Choi, and Jianfeng Gao. Vinvl: Revisiting visual representations in vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 5579–5588, 2021

    2021

  41. [49]

    Perturbed self-distillation: Weakly supervised large-scale point cloud semantic segmentation

    Yachao Zhang, Yanyun Qu, Yuan Xie, Zonghao Li, Shanshan Zheng, and Cuihua Li. Perturbed self-distillation: Weakly supervised large-scale point cloud semantic segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 15520–15528, 2021

    2021

  42. [50]

    Self-distillation as instance-specific label smoothing

    Zhilu Zhang and Mert Sabuncu. Self-distillation as instance-specific label smoothing. Advances in Neural Information Processing Systems , 33:2184–2195, 2020

    2020

  43. [51]

    Knowl- edge distillation via instance-level sequence learning

    Haoran Zhao, Xin Sun, Junyu Dong, Zihe Dong, and Qiong Li. Knowl- edge distillation via instance-level sequence learning. Knowledge-Based Systems, 233:107519, 2021

    2021

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.