{"id":"10757162-93fd-43d5-95d4-244135be9356","arxiv_id":"2501.16329","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":10,"one_line_summary":"A mixture-of-modality-experts transformer with self-distillation reports improved mouse sleep staging and enables single-channel inference after multi-channel training.","lead":"Sleep staging in mice is labor-intensive, so automating it matters. This paper proposes sDREAMER, a transformer that fuses EEG and EMG, then can score sleep from either both signals or just one, reporting higher accuracy than several transformer baselines on a mouse dataset.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The comparative claim over transformer baselines rests on single-run point estimates with no confidence intervals or significance tests; the margin over the strongest sequence baseline (91.72 vs 90.93 accuracy) is small enough to be seed noise.","rationale":"The paper is a coherent empirical study of a plausible architecture: the MoME module with partially shared attention and modality-specific FFNs is well motivated, the epoch/sequence hierarchy is standard, and the ablation studies internally support the value of self-distillation. The authors also honestly acknowledge the single-expert label limitation in Section VI, which is a point in their favor. However, the most load-bearing weakness for the stated central claim is not the label noise itself but the lack of any uncertainty quantification around the comparative results. Since the claim is 'outperforms existing transformer-based methods,' the evaluation must show that the gap is larger than training noise and that the baselines are faithful, reasonably tuned representatives of those methods. The current paper provides neither confidence intervals nor baseline tuning details, and the margin over the best sequence baseline is small. The reader's weakest assumption focused on single-expert labels; I agree that this limits generalizability and absolute accuracy claims, but for the relative superiority claim it is less decisive because all methods share the same labels. My concern complements the reader's: it targets the statistical reliability of the comparison itself. The verdict remains CONDITIONAL as the reader proposed, pending multi-seed evaluation, released artifacts, and correction of Equation 14.","tokens_in":13667,"tokens_out":4860,"duration_ms":47837,"concrete_test":"Retrain Sequence sDREAMER, SeqCross-Attention Transformer, Cross-Attention Transformer, and Cross-Modal Transformer on the same subject split with 10 random seeds each; report mean and 95% confidence intervals for accuracy and F1, and run a paired significance test (e.g., Wilcoxon signed-rank) between sDREAMER and each baseline. If the advantage over SeqCross-Attention does not survive with non-overlapping intervals or p<0.05, the abstract's superiority claim is unsupported. As a secondary check, instrument the training loss to confirm that both Lsd-eeg and Lsd-emg are included; if only the EMG distillation term is present, the method as implemented differs from the paper's description and the ablation results in Table III need re-examination.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is explicitly differential: sDREAMER 'outperforms the existing transformer-based sleep scoring methods' in both multi-channel and single-channel settings. In Table I, the strongest sequence transformer baseline (SeqCross-Attention Transformer) reaches 90.93% accuracy and 86.48% F1, while Sequence sDREAMER reaches 91.72% accuracy and 87.64% F1; for the epoch setting, Epoch sDREAMER is 88.25%/81.30% versus 87.07%/79.13% for the closest epoch transformer baseline. Every number in Tables I and II is a single run on one fixed subject split, with no error bars, no repeated-seed variance, and no statistical test. Transformer training is seed-sensitive, so a 0.79-point accuracy gap could easily be within run-to-run noise. The baselines are also self-implemented, with no reported tuning budget or verification that the re-implemented Cross-Modal Transformer [32] matches its original configuration; thus the phrase 'existing transformer-based methods' is broader than what was actually evaluated. The single-expert label limitation acknowledged in Section VI is real, but because all models are trained and tested on the same labels, it mostly threatens external validity, not the internal relative comparison. A separate internal inconsistency is that Equation 14 writes the distillation loss as (alpha/2)(Lsd-emg + Lsd-emg), omitting Lsd-eeg entirely; if the implementation follows this equation, the EEG self-distillation described in Section IV-F and credited in Table III is not actually trained.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes sDREAMER, a transformer architecture for automatic sleep staging from EEG and EMG signals. The model has three pathways (EEG, EMG, and mixed) with partially shared weights in a mixture-of-modality-experts (MoME) module, and it is trained with a self-distillation scheme where the mixed pathway distills knowledge to the mono-modal pathways. This allows the model to be trained on multi-channel signals and then used for either multi-channel or single-channel inference. The authors evaluate epoch-level and sequence-level variants on a mouse sleep dataset with expert labels, reporting accuracy and F1 for both multi-modal and mono-modal settings, and claim outperformance over existing transformer-based baselines.","tokens_in":14027,"tokens_out":5987,"duration_ms":55624,"significance":"If the empirical claims hold, the proposed architecture is a useful step toward unified sleep staging models that can accommodate both multi-channel and single-channel inputs. The idea of using a multi-modal pathway as a teacher for mono-modal pathways is interesting, and the paper includes extensive ablations and comparisons to multiple baselines. However, the strength of the central claim is currently limited by single-run evaluations without statistical significance testing, self-implemented and unreleased baselines, and an internal inconsistency in the formal definition of the self-distillation loss.","major_comments":[{"comment":"The total loss is written as L_mome = (1-alpha)Lce + (alpha/2)(Lsd-emg + Lsd-emg), which omits Lsd-eeg and duplicates Lsd-emg. The surrounding text and Table III clearly indicate that both EEG and EMG self-distillation are used. This makes the formal definition inconsistent with the described method and the ablation. Please correct the equation and clarify which loss was actually implemented.","section":"IV-F, Eq. (14)"},{"comment":"The central comparative claim rests on single point estimates with no standard deviations, confidence intervals, or significance tests. For example, the margin over the strongest sequence baseline is 0.79 accuracy points (91.72 vs 90.93), which could easily be within seed-to-seed variation for transformer training. Please report results over multiple runs (e.g., 5 seeds) with mean and standard deviation, and ideally perform paired significance tests for the main comparisons.","section":"V-D, Tables I and II"},{"comment":"The baseline models are self-implemented, but no code, hyperparameter tuning budgets, or verification that the re-implemented Cross-Modal Transformer [32] matches its original configuration are provided. This weakens the claim that sDREAMER 'outperforms the existing transformer-based sleep scoring methods,' since the comparison is only against the authors' reimplementations. Please describe the tuning process for baselines and, if possible, release the baseline implementations.","section":"V-C, Tables I and II"},{"comment":"The no-self-distillation ablation shows EEG-Acc of 62.71% and EMG-Acc of 7.85%, which suggests that without distillation the mono-modal pathways receive essentially no training signal, because the only cross-entropy loss defined (Eq. 13) is on the mix pathway. This ablation therefore conflates 'no self-distillation' with 'mono-modal pathways untrained,' so the large improvements from adding self-distillation do not cleanly demonstrate the benefit of the distillation mechanism. Please clarify the training losses for each pathway and provide an ablation where the mono-modal pathways are trained with an equivalent supervised loss without distillation.","section":"V-E, Table III"}],"minor_comments":[{"comment":"The text says 'resulting in a total of 10,000 epoch data samples,' but the class distributions in Fig. 5 sum to far more than 10,000 (the training set alone exceeds 90,000). Please correct the dataset size description and provide the actual number of subjects, epochs, and class counts.","section":"V-A"},{"comment":"The Cross-Modal Transformer [32] rows report accuracy but no F1-score, despite the text stating that both metrics are reported. Please provide F1 values or explain the omission.","section":"Table I"},{"comment":"The residual term T^{m'}_ell is not defined in the paper; it likely should be T^m_{ell-1} or a similar expression. Please clarify the notation.","section":"IV-B, Eq. (5)"},{"comment":"The text in Section IV-D says all three pathways are enabled during training, but Eq. (13) defines the cross-entropy loss only for the mix pathway. Please clarify whether the EEG and EMG classifiers also receive a direct cross-entropy loss during training or are trained solely through the self-distillation loss.","section":"IV-D and IV-F"},{"comment":"There is a duplicated word in 'EEG time series signals signals' in the Introduction; please fix this typo.","section":"I"},{"comment":"Reference [24] is cited for the 'Channel-Independent Transformer' baseline, but the cited paper is a time-series forecasting work, not a sleep staging method. Please clarify how this baseline was adapted or use a more specific reference.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The main barrier is statistical: the comparative claims are supported by single runs on a small, single-expert-labeled dataset. The authors should add repeated-seed experiments and significance tests. The Eq. (14) typo and the ablation confound in Table III are also substantive and should be fixed. The paper is within the journal's scope and the architectural idea is promising, but the evidence is not yet convincing."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: this is a competent, narrowly scoped empirical paper that adapts the VLMo mixture-of-modality-experts recipe to mouse sleep staging and adds a self-distillation scheme that clearly helps mono-channel inference. The architecture is not conceptually new—shared attention, modality experts, and distillation all exist in the vision-language literature—but the specific arrangement for EEG/EMG staging is not in the cited prior work, and the paper evaluates it on real mouse data. The ablations are the strongest part: removing self-distillation drops EMG-only accuracy from 83.78% to 7.85%, which shows the mix pathway is truly teaching the mono pathways something, not just regularizing.\n\nThe soft spots are all around evidence quality. Every number in Tables I and II is a single run on a fixed subject split: no error bars, no seeds, no significance tests. The sequence model's margin over the best sequence transformer baseline (91.72 vs 90.93 accuracy) is 0.79 points—well within seed noise for transformers. The baselines are self-implemented and unreleased, so the abstract's 'outperforms existing transformer-based methods' overreaches. There's also a clear typo in Eq 14: the loss is written as (1-alpha)L_ce + alpha/2 (L_sd-emg + L_sd-emg), which omits L_sd-eeg entirely. That contradicts Section IV-F and the ablation in Table III. It's almost certainly a typo in the write-up, not the implementation, but it needs fixing. Finally, the ground truth is one expert's labels, which the authors concede; this mainly threatens external validity, not the internal comparison, since all models share the same labels.\n\nOverall, this is a solid subfield contribution that deserves a serious referee. The architecture is sensible, the ablations are informative, and the single-expert limitation is honestly acknowledged. But as written, the comparative claims are not supported by the statistics. I'd send it to review with a request for repeated-seed runs with confidence intervals, code release, and a corrected Eq 14.","headline":"A sensible adaptation of MoME plus self-distillation to mouse sleep staging, with informative ablations, but the comparative claims rest on single runs with no error bars and there's a typo in Eq 14.","tokens_in":14554,"tokens_out":2581,"would_cite":false,"duration_ms":24389,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"One transformer beats sleep-staging baselines on EEG, EMG, or both","keywords":["sleep staging","mixture-of-modality experts","self-distillation","transformer","EEG-EMG multimodal","mouse sleep dataset"],"falsifier":"Have two or more independent experts score the same recordings, train and evaluate sDREAMER against a consensus or majority label, and compare its accuracy with the transformer baselines under the same labels; if the gap over baselines disappears or reverses, the claimed advantage would be specific to a single annotator.","tokens_in":13440,"feed_emoji":"💤","tokens_out":5882,"duration_ms":49342,"temperature":0.7,"pith_summary":"The paper argues that automatic sleep staging can be done with a single transformer that is trained on multi-channel EEG and EMG signals yet remains accurate when given only one channel at test time. The proposed sDREAMER model gives each modality its own expert pathway while sharing attention weights, and adds a mixture pathway whose predictions distill knowledge back into the EEG and EMG pathways. On a mouse sleep dataset with a subject-wise split, the model reports higher accuracy and F1 scores than the transformer baselines it is compared against, in both the epoch-level and sequence-level settings.","feed_headline":"One transformer beats sleep-staging baselines on EEG, EMG, or both","feed_subtitle":"Multi-channel training plus self-distillation gives accurate single-channel sleep staging in mice.","key_machinery":"The central mechanism is the mixture-of-modality-experts (MoME) module: a transformer layer with a shared multi-head self-attention and three modality-specific feed-forward networks, called experts, for EEG, EMG, and their concatenation. A mapping function $\\psi(m, \\ell)$ chooses which expert processes a token at a given layer, so the same model routes EEG, EMG, or mixed tokens through their own pathways. The self-distillation loss, a temperature-scaled KL divergence from the mix pathway's logits to the EEG and EMG logits, transfers knowledge from the multi-modal pathway to the mono-modal ones. This design is what lets a single model handle both single-channel and multi-channel inference.","core_discovery":"The central claim is that a mixture-of-modality-experts transformer with partially shared weights and self-distillation outperforms the existing transformer-based sleep staging methods on a mouse EEG-EMG dataset. The model is trained with both channels but offers three inference paths: EEG-only, EMG-only, or combined, and the authors report that each path is competitive, with the multi-channel path reaching 91.72% accuracy and 87.64% F1. The authors interpret this as evidence that cross-modal interaction during training, rather than late fusion, improves sleep staging, and that self-distillation lets mono-modal pathways benefit from multi-modal supervision.","pith_inferences":["The self-distillation from the mix pathway to the mono-modal pathways may act as a regularizer that prevents overfitting to the annotation style of a single expert.","A testable extension is to apply the same training scheme to human polysomnography datasets and check whether the cross-modal supervision transfers when only a single EEG channel is available.","The shared-attention design could be probed with interpretability tools to see whether the mix pathway learns distinct sleep-stage signatures rather than simply averaging the modalities."],"forward_implications":["A single trained model could serve sleep labs regardless of whether they record EEG, EMG, or both channels.","Single-channel inference from a multi-channel-trained model could make at-home sleep monitoring with a wearable EEG electrode more practical.","The self-distillation scheme could be applied to other multi-modal physiological signals where a channel may be missing at deployment.","The architecture could be retrained for other species or for human sleep staging, since it does not depend on mouse-specific features."],"supporting_citations":[{"why":"Supplies the mixture-of-modality-experts idea for cross-modal learning that the paper adapts to sleep staging.","marker":"[5]"},{"why":"Provides the public mouse sleep dataset that the experiments are run on.","marker":"[18]"},{"why":"The state-of-the-art cross-modal transformer baseline that the paper compares against and aims to outperform.","marker":"[32]"},{"why":"The channel-independent transformer baseline, whose shared-attention design the paper extends into the MoME module.","marker":"[24]"},{"why":"The Vision Transformer patch embedding and [CLS] token design used as the input representation.","marker":"[9]"},{"why":"The transformer backbone and multi-head self-attention that all compared models are built on.","marker":"[41]"},{"why":"Previous work distilling across vision and language domains that inspires the proposed cross-modality self-distillation.","marker":"[44]"}],"fun_headline_variants":["Self-distilled transformer tops sleep staging on EEG, EMG, or both","Cross-modality self-distillation boosts sleep staging models","Mixture-of-modality experts with self-distillation beat sleep baselines","One transformer, three pathways: better sleep staging","Self-distillation makes single-channel sleep staging more accurate"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The evaluation treats the sleep-stage labels produced by a single expert as ground truth; if those labels are noisy or idiosyncratic, the reported accuracy and baseline comparisons could reflect fitting one person's annotation style rather than objective sleep stages.","fun_headline_variants_meta":{"raw":{"variants":["Self-distilled transformer tops sleep staging on EEG, EMG, or both","Cross-modality self-distillation boosts sleep staging models","Mixture-of-modality experts with self-distillation beat sleep baselines","One transformer, three pathways: better sleep staging","Self-distillation makes single-channel sleep staging more accurate"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000562,"raw_usage":{"total_tokens":2623,"prompt_tokens":858,"completion_tokens":1765,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":474,"completion_tokens_details":{"reasoning_tokens":1681}},"tokens_in":474,"tokens_out":1765,"duration_ms":13265,"temperature":1.0,"reasoning_tokens":1681,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T13:30:13.676046+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Have two or more independent experts score the same recordings, train and evaluate sDREAMER against a consensus or majority label, and compare its accuracy with the transformer baselines under the same labels; if the gap over baselines disappears or reverses, the claimed advantage would be specific to a single annotator.","supporting_citations":[{"cited_title":"Memory-enhancing properties of sleep depend on the os- cillatory amplitude of norepinephrine","cited_arxiv_id":null,"evidence_quote":"Provides the public mouse sleep dataset that the experiments are run on."},{"cited_title":"Toward Interpretable Sleep Stage Classification Using Cross-Modal Transformers","cited_arxiv_id":"2208.06991","evidence_quote":"The state-of-the-art cross-modal transformer baseline that the paper compares against and aims to outperform."}],"review_version":1}