REVIEW 1 major objections 4 minor 40 references
Interpretable EEG biomarkers with bag-of-waves: Spatial and temporal waveform dictionaries for low-data regimes
T0 review · 1 major / 4 minor · reviewed 2026-08-01 · deepseek-v4-flash
Pith's one-line read Bag-of-waves shows that a small, label-free dictionary of EEG waveform shapes, plus the order in which they occur, can match large pretrained models on classification while remaining fully inspectable.
desk verdict A well-motivated, clearly written extension of bag-of-waves whose headline numbers are weakened by per-task hyperparameter selection without nested validation—worth refereeing, but the evaluation needs tightening. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the atom: a short, learned EEG waveform template of fixed duration, found by shift-invariant k-means over all windows of the training recordings. Matching is done by cosine similarity after trying all temporal shifts, which makes the match insensitive to amplitude and to where a waveform starts inside the window. The continuous signal is thereby turned into a token stream, and the features are unigram counts plus bigram and trigram transition counts, weighted by term frequency-inverse document frequency and selected by variance. For multichannel data, atoms can be regional (per channel or region) or spatial (a single cross-channel pattern after common-average-reference
What would settle it
Test the method on a task where amplitude is diagnostic — for instance grading burst suppression in coma EEG. If adding an amplitude feature to the same token counts improves accuracy over the gain-stripped version, then the deliberate removal of amplitude is discarding clinically relevant information; if it does not, the shape-only representation is sufficient as claimed. Separately, find recordings where two distinct waveforms overlap within a single window and check whether the one-atom-per-window assignment can still recover the clinician's labels.
Extended reading notes
Core claim
The paper proposes that EEG can be faithfully summarized by a small, unsupervisedly learned dictionary of waveform 'atoms,' with the order of atoms captured by n-gram transitions. It extends the single-channel bag-of-waves idea to regional and cross-channel spatial atoms, so an atom can encode a topography across electrodes. Across three datasets — single-channel mouse genotype (16 animals), resting-state dementia (88 subjects), and a clinical event benchmark (roughly 84,000 training windows) — bag-of-waves reaches accuracy in the same range as deep and pretrained models while using orders of magnitude fewer parameters, and the label-free atoms recover known waveform morphologies: genotype-s
Load-bearing premise
The load-bearing premise is that each EEG window can be reduced to a single shape match: amplitude, phase within the window, and any second event overlapping in the same window are all thrown away, and if a diagnosis depends on those, the representation loses the signal.
Editorial extensions
If this is right
- A recording is summarized by waveform rates and transitions, so whole-recording questions (genotype, diagnosis) and event-level questions (clinical event type) are handled by the same token stream.
- Because the dictionary is learned without labels, it can be built before annotations exist and reused as labels arrive, which suits clinical workflows where recordings precede annotation.
- Short atoms with bigram transitions outperform a single longer atom over the same span, so the method's effective unit of description is the ordered pair of sub-second states, not the stretched waveform.
- The small dictionary is not just an interpretability trade-off; it is required for the transition features to work. At K equal to or larger than 128 the bigram space explodes and accuracy collapses, so the method's success rests on keeping the vocabulary small.
- Rare events and recordings shorter than about a minute produce unstable tokens, so the method is suited to recurring, relatively frequent waveform patterns rather than to one-off events.
Reading between the lines
- The paper names continuous atom embedding and cross-dataset transfer as future work; a natural next step would be to learn a continuous space over atoms from their co-occurrence statistics, which could allow soft assignment and transfer without losing the discrete, inspectable vocabulary.
- The dementia result — spatial atoms beat regional atoms, bigrams beat unigrams — suggests that the diagnostic signal in resting-state EEG lives in fast transitions between short topographies; a testable prediction is that connectivity-based deep models on the same data owe their performance to the same cross-channel transition information.
- Because cosine matching deliberately strips amplitude, any EEG biomarker defined by amplitude (for example burst-suppression ratio or voltage attenuation) would be invisible to the current features; adding a per-atom amplitude channel could extend the method to those tasks without sacrificing shape interpretability.
- The method's competitive kappa on the event benchmark despite using no within-window superposition suggests that most clinical event structure in 5-second windows is sequential rather than simultaneous; if true, the single-atom assignment is less restrictive than it looks.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents bag-of-waves, an unsupervised EEG representation learning method based on shift-invariant k-means dictionary learning. A small dictionary of atom waveforms is learned without labels, the continuous EEG is tokenized into a sequence of atoms, and unigram/n-gram counts are used as features for downstream classification or clustering. The method is extended from prior single-channel work to regional and spatial (multivariate) atoms and to atom-to-atom transition n-grams. It is evaluated on three datasets: single-channel mouse genotype clustering, resting-state dementia classification, and the TUEV six-class event benchmark. The authors report performance competitive with state-of-the-art deep and foundation models, while using far fewer parameters and providing interpretable atoms that recover known clinical morphologies (alpha slowing, PLED, GPED). They emphasize the method's low-data and interpretability advantages.
Significance. If the results hold, the paper offers a useful contribution: a simple, computationally cheap, and interpretable EEG feature extractor that works in low-data regimes. The empirical design has real strengths: subject-disjoint cross-validation on dementia, the official patient-disjoint split on TUEV, a permutation null on the mouse clustering, and direct comparisons with HMM and traditional-feature baselines. The qualitative finding that label-free atoms recover clinically meaningful waveforms is compelling and suggests practical value for biomarker discovery. However, the central quantitative claims depend critically on the unresolved model-selection and data-leakage issues detailed below; these need to be addressed before the reported numbers can be taken at face value.
major comments (1)
- [Section II-A, II-C, II-E] Table II(b) and Section IV-C report TUEV results as point estimates without error bars or any variance measure, while all published baselines are reported as mean ± SD. Since the central claim is 'competitive' performance against these baselines, the lack of variance makes it impossible to assess whether the observed differences (e.g., kappa 0.44 vs. BIOT's 0.45) are meaningful. Please report standard deviations across seeds, bootstrap resamples, or at least confidence intervals.
minor comments (4)
- [Section IV-A] The text reports a silhouette of 0.78 for the best configuration, but later states 'the permutation null over 200 shuffles places the observed silhouette of 0.54 far from the null mean'. Clarify which configuration the permutation test applies to and why the silhouettes differ.
- [Section IV-D, Table III(b)] The description of the 'unsupervised HMM whose Viterbi-decoded transition matrix and state-occupancy vector feed a random forest' is ambiguous. Specify how the transition matrix is converted into features and whether the HMM is trained per class or globally.
- [Section II-A] The convergence criterion 'mean squared atom change falls below 10^{-4} of the training-window variance' is unclear: is 10^{-4} a fraction or an absolute value? The threshold should be written as a dimensionless relative quantity.
- [Section II-C] When n-gram features are kept only if they occur in the training set, clarify how unseen n-grams in the test set are handled (mapped to zero? ignored?). This matters for the variance-based feature selection and for reproducibility.
Circularity Check
TUEV 'label-free' morphology recovery is partly by construction via the label-specific balanced dictionary; core classification results remain empirical.
-
fitted input called prediction
[Section IV-C (TUEV: event classification) and Section IV, sweep-axes paragraph]
"The second is the dictionary source: on TUEV , a label-specific dictionary (balanced), a fully unsupervised dictionary tiled across whole recordings (whole), their union, and per-region dictionaries ... Because the labels are expert event types, it also lets us check against ground truth that the label-free atoms recover clinical morphologies (the periodic discharges PLED and GPED)."
The 'label-free' recovery claim is evaluated with a dictionary variant whose construction uses the event labels ('label-specific dictionary (balanced)'). Atoms in that variant are class-conditioned templates, so matching them back to the same GPED/PLED annotations is a by-construction property of fitting per-class prototypes, not an independent unsupervised discovery. The reported best TUEV configuration is the union of the balanced and whole dictionaries, and the balanced dictionary is the one strong on GPED (F1 around 0.62 vs 0.46 for the unsupervised whole dictionary). Thus the specific 'label-free atoms recover GPED' claim reduces to its supervised input for that class.
full rationale
The paper's numerical pipeline is not circular: Eqs. (1)-(2) define an unsupervised dictionary; tokenization, n-gram counting, and downstream classification are deterministic empirical steps; and the mouse/dementia results are reported as out-of-sample or permutation-tested metrics rather than constants renamed as predictions. The central classification claims therefore rest on empirical fits. The one exhibited reduction is in the TUEV interpretability claim: the sweep includes a 'label-specific dictionary (balanced)', and the best/interpreted atoms come from its union with the fully unsupervised whole dictionary. Recovering GPED from class-conditioned templates is a by-construction match to the labels used to build that dictionary, so the 'label-free atoms recover known morphologies' contribution is partially circular. The paper's own limitation that 'we choose hyperparameters per task' is a selection-bias risk (no nested validation is described), but that is a correctness concern, not circularity, and does not by itself raise the score beyond 4.
Assumptions & free parameters
free parameters (5)
- Dictionary size K =
12 (mouse), 8 (dementia), 32 (TUEV)
- Atom duration D =
1.0 s (mouse), 0.5 s (dementia), multiscale 0.25–1.0 s (TUEV)
- n-gram order =
1 (mouse), 2 (dementia), 1–3 (TUEV)
- Feature selection count =
200 (mouse/dementia), 300 (TUEV)
- Downstream model hyperparameters =
RF 100 trees depth 5 (dementia), 300–400 trees (TUEV); k-means with 2 clusters (mouse)
assumptions (4)
- domain assumption EEG discriminative information is preserved when amplitude is removed (cosine similarity / gain-stripping)
- domain assumption A single shifted, scaled atom can represent each EEG window; overlapping events within a window are not modeled
- domain assumption Counts and n-gram transitions of learned tokens preserve enough signal for classification/clustering
- ad hoc to paper The chosen hyperparameters (K, D, n-gram order) generalize across subjects
Cite this review
Pith. "Pith review of Interpretable EEG biomarkers with bag-of-waves: Spatial and temporal waveform dictionaries for low-data regimes." pith.science (2026). https://pith.science/paper/CCH3B6DR
@misc{pith2026260722508,
author = {Pith},
title = {Pith review of: Interpretable EEG biomarkers with bag-of-waves: Spatial and temporal waveform dictionaries for low-data regimes},
year = {2026},
howpublished = {\url{https://pith.science/paper/CCH3B6DR}},
note = {Machine review of arXiv:2607.22508}
}
read the original abstract
Electroencephalography (EEG) is widely used to diagnose neurological conditions, but its analysis usually relies on either predefined spectral features or deep neural networks. Predefined features carry a strong bias, since they fix in advance what counts as informative, while deep neural networks and foundation models are hard to interpret and need large amounts of data and compute. We present bag-of-waves, an interpretable framework that learns a small dictionary of recurring EEG waveform templates, called atoms, using shift-invariant k-means without labels. The continuous EEG is then turned into a sequence of atom tokens, whose counts feed a simple downstream classifier or clustering step. We extend this representation in two ways: we add atom-to-atom transitions, which we call n- grams, to capture temporal structure, and we move from single-channel atoms to regional and cross-channel spatial atoms for the multichannel case. We test the method on three complementary datasets, each probing a different aspect: single-channel mouse genotype clustering with only sixteen animals (the low-data and temporal case), resting-state dementia classification (the spatial case), and the TUEV benchmark, a six-way classification of clinical EEG events (a high-data comparison against strong deep and foundation baselines). Across all three datasets, bag-of-waves achieves performance competitive with state-of-the-art deep and foundation models. Yet, it operates with a fraction of the parameter count and provides full interpretability: because every atom corresponds to an inspectable waveform, the method explicitly recovers known clinical morphologies that a neurophysiologist can directly validate. Its main advantage is that it works in the low-data regime where heavier models are a poor fit.
Figures
Reference graph
Works this paper leans on
-
[1]
The Temple University Hospital EEG data corpus,
I. Obeid and J. Picone, “The Temple University Hospital EEG data corpus,” Frontiers in Neuroscience, vol. 10, p. 196, 2016
2016
-
[2]
BIOT: cross-data biosignal learning in the wild,
C. Yang, M. Westover, and J. Sun, “BIOT: cross-data biosignal learning in the wild,” in Proc. Advances in Neural Information Processing Systems (NeurIPS), 2023
2023
-
[3]
Large brain model for learning generic representations with tremendous EEG data in BCI,
W. Jiang, L. Zhao, and B. Lu, “Large brain model for learning generic representations with tremendous EEG data in BCI,” in Proc. Interna- tional Conference on Learning Representations (ICLR), 2024
2024
-
[4]
Development of expert-level classification of seizures and rhythmic and periodic patterns during EEG interpretation,
J. Jing et al., “Development of expert-level classification of seizures and rhythmic and periodic patterns during EEG interpretation,” Neurology, vol. 100, no. 17, pp. e1750–e1762, 2023
2023
-
[5]
EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces,
V . Lawhern et al., “EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces,” Journal of Neural Engineering, vol. 15, no. 5, 2018
2018
-
[6]
EEG Conformer: convolutional transformer for EEG decoding and visualization,
Y . Song et al., “EEG Conformer: convolutional transformer for EEG decoding and visualization,” IEEE Trans. Neural Systems and Rehabil- itation Engineering, vol. 31, pp. 710–719, 2023
2023
-
[7]
A dataset of scalp EEG recordings of Alzheimer’s disease, frontotemporal dementia and healthy subjects from routine EEG,
A. Miltiadous, K. D. Tzimourta, T. Afrantou, P. Ioannidis, N. Grigo- riadis, D. G. Tsalikakis, P. Angelidis, M. G. Tsipouras, E. Glavas, N. Giannakeas, and A. T. Tzallas, “A dataset of scalp EEG recordings of Alzheimer’s disease, frontotemporal dementia and healthy subjects from routine EEG,” Data, vol. 8, no. 6, p. 95, 2023
2023
-
[8]
Heterozygous mutations of the voltage-gated sodium channel SCN8A are associated with spike-wave discharges and absence epilepsy in mice,
L. A. Papale, B. Beyer, J. M. Jones, L. M. Sharkey, S. Tufik, M. Epstein, V . A. Letts, M. H. Meisler, W. N. Frankel, and A. Escayg, “Heterozygous mutations of the voltage-gated sodium channel SCN8A are associated with spike-wave discharges and absence epilepsy in mice,” Human Molecular Genetics, vol. 18, no. 9, pp. 1633–1641, 2009
2009
Show all 40 references
-
[9]
Interpretable EEG biomarkers for neurological disease models in mice using bag-of-waves classifiers,
M. I. Cano Achuri, M. K. Lara, K. Abed Rabbo, B. T. Wilson, A. Meek, J. M. Mahoney, A. E. Hernan, and A. J. Brockmeier, “Interpretable EEG biomarkers for neurological disease models in mice using bag-of-waves classifiers,” Journal of Neural Engineering, vol. 23, no. 3, p. 036016, 2026
2026
-
[10]
A statistical interpretation of term specificity and its application in retrieval,
K. Sparck Jones, “A statistical interpretation of term specificity and its application in retrieval,” Journal of Documentation, vol. 28, no. 1, pp. 11–21, 1972
1972
-
[11]
Comparing partitions,
L. Hubert and P. Arabie, “Comparing partitions,” Journal of Classifica- tion, vol. 2, no. 1, pp. 193–218, 1985
1985
-
[12]
A coefficient of agreement for nominal scales,
J. Cohen, “A coefficient of agreement for nominal scales,” Educational and Psychological Measurement, vol. 20, no. 1, pp. 37–46, 1960
1960
-
[13]
Periodic lateralized epileptiform discharges and afterdischarges: common dynamic mechanisms,
G. P. Kalamangalam and J. D. Slater, “Periodic lateralized epileptiform discharges and afterdischarges: common dynamic mechanisms,” Journal of Clinical Neurophysiology, vol. 32, no. 4, pp. 331–340, 2015
2015
-
[14]
EEG microstates as a tool for studying the temporal dynamics of whole-brain neuronal networks: a review,
C. Michel and T. Koenig, “EEG microstates as a tool for studying the temporal dynamics of whole-brain neuronal networks: a review,” NeuroImage, vol. 180, pp. 577–593, 2018
2018
-
[15]
Spectrally resolved fast transient brain states in electrophysiological data,
D. Vidaurre et al., “Spectrally resolved fast transient brain states in electrophysiological data,” NeuroImage, vol. 126, pp. 81–95, 2016
2016
-
[16]
EEG dynamics in patients with Alzheimer’s disease,
J. Jeong, “EEG dynamics in patients with Alzheimer’s disease,” Clinical Neurophysiology, vol. 115, no. 7, pp. 1490–1505, 2004
2004
-
[17]
Diagnosis of Alzheimer’s disease from EEG signals: where are we standing?
J. Dauwels, F. Vialatte, and A. Cichocki, “Diagnosis of Alzheimer’s disease from EEG signals: where are we standing?” Current Alzheimer Research, vol. 7, no. 6, pp. 487–505, 2010
2010
-
[18]
Self-supervised elec- troencephalogram representation learning for automatic sleep staging,
C. Yang, C. Xiao, M. B. Westover, and J. Sun, “Self-supervised elec- troencephalogram representation learning for automatic sleep staging,” JMIR AI, vol. 2, p. e46769, 2023
2023
-
[19]
Transformer convolutional neural networks for automated artifact detection in scalp EEG,
W. Y . Peh, Y . Yao, and J. Dauwels, “Transformer convolutional neural networks for automated artifact detection in scalp EEG,” in Proc. IEEE Eng. Med. Biol. Soc. (EMBC), 2022, pp. 3599–3602
2022
-
[20]
Motor imagery EEG clas- sification algorithm based on CNN-LSTM feature fusion network,
H. Li, M. Ding, R. Zhang, and C. Xiu, “Motor imagery EEG clas- sification algorithm based on CNN-LSTM feature fusion network,” Biomedical Signal Processing and Control, vol. 72, p. 103342, 2022
2022
-
[21]
Transformer-based spatial- temporal feature learning for EEG decoding,
Y . Song, X. Jia, L. Yang, and L. Xie, “Transformer-based spatial- temporal feature learning for EEG decoding,” arXiv:2106.11170, 2021
2021 arXiv
-
[22]
Tokenizing single- channel EEG with time-frequency motif learning,
J. Pradeepkumar, X. Piao, Z. Chen, and J. Sun, “Tokenizing single- channel EEG with time-frequency motif learning,” in Proc. International Conference on Learning Representations (ICLR), 2026
2026
-
[23]
SEBSFormer: a spectral-enhanced bi- stream transformer for robust EEG decoding,
L. Zhang, S. Tu, and L. Xu, “SEBSFormer: a spectral-enhanced bi- stream transformer for robust EEG decoding,” in Proc. AAAI Confer- ence on Artificial Intelligence (AAAI), 2026
2026
-
[24]
DICE-Net: a novel convolution-transformer architecture for Alzheimer detection in EEG signals,
A. Miltiadous, E. Gionanidis, K. D. Tzimourta, N. Giannakeas, and A. T. Tzallas, “DICE-Net: a novel convolution-transformer architecture for Alzheimer detection in EEG signals,” IEEE Access, vol. 11, pp. 71840– 71858, 2023
2023
-
[25]
The role of data partitioning on the performance of EEG-based deep learning models in supervised cross-subject analysis: a preliminary study,
F. Del Pup, A. Zanola, L. F. Tshimanga, A. Bertoldo, L. Finos, and M. Atzori, “The role of data partitioning on the performance of EEG-based deep learning models in supervised cross-subject analysis: a preliminary study,” Computers in Biology and Medicine, vol. 196, p. 110608, 2025
2025
-
[26]
EEG-based classification of Alzheimer’s disease and frontotemporal dementia using functional connectivity,
T. Mlinari ˇc, A. Van Den Kerchove, Z. I. Barinaga, and M. M. Van Hulle, “EEG-based classification of Alzheimer’s disease and frontotemporal dementia using functional connectivity,” Scientific Reports, vol. 16, p. 4903, 2026
2026
-
[27]
Learning recurrent waveforms within EEGs,
A. J. Brockmeier and J. C. Principe, “Learning recurrent waveforms within EEGs,” IEEE Trans. Biomedical Engineering, vol. 63, no. 1, pp. 43–54, 2016
2016
-
[28]
Shift-invariant wave- form learning on epileptic ECoG,
C. H. Mendoza-Cardenas and A. J. Brockmeier, “Shift-invariant wave- form learning on epileptic ECoG,” in Proc. IEEE Eng. Med. Biol. Soc. (EMBC), 2021, pp. 1136–1139
2021
-
[29]
Coding time-varying signals using sparse, shift-invariant representations,
M. S. Lewicki and T. J. Sejnowski, “Coding time-varying signals using sparse, shift-invariant representations,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 11. MIT Press, 1998
1998
-
[30]
Multivariate temporal dictionary learning for EEG,
Q. Barthelemy, C. Gouy-Pailler, Y . Isaac, A. Souloumiac, A. Larue, and J. I. Mars, “Multivariate temporal dictionary learning for EEG,” Journal of Neuroscience Methods, vol. 215, no. 1, pp. 19–28, 2013
2013
-
[31]
Multivariate convolutional sparse coding for electromagnetic brain signals,
T. Dupre la Tour, T. Moreau, M. Jas, and A. Gramfort, “Multivariate convolutional sparse coding for electromagnetic brain signals,” in Proc. Advances in Neural Information Processing Systems (NeurIPS), 2018
2018
-
[32]
Systematic review on resting-state EEG for Alzheimer’s disease diagnosis and progression assessment,
R. Cassani, M. Estarellas, R. San-Martin, F. J. Fraga, and T. H. Falk, “Systematic review on resting-state EEG for Alzheimer’s disease diagnosis and progression assessment,” Disease Markers, vol. 2018, p. 5174815, 2018
2018
-
[33]
Random forests,
L. Breiman, “Random forests,” Machine Learning, vol. 45, no. 1, pp. 5–32, 2001
2001
-
[34]
The Viterbi algorithm,
G. D. Forney, Jr., “The Viterbi algorithm,” Proceedings of the IEEE, vol. 61, no. 3, pp. 268–278, 1973
1973
-
[35]
Statistical inference for probabilistic func- tions of finite state Markov chains,
L. E. Baum and T. Petrie, “Statistical inference for probabilistic func- tions of finite state Markov chains,” The Annals of Mathematical Statistics, vol. 37, no. 6, pp. 1554–1563, 1966
1966
-
[36]
Hastie, R
T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd ed. New York, NY , USA: Springer, 2009
2009
-
[37]
Silhouettes: a graphical aid to the interpretation and validation of cluster analysis,
P. J. Rousseeuw, “Silhouettes: a graphical aid to the interpretation and validation of cluster analysis,” Journal of Computational and Applied Mathematics, vol. 20, pp. 53–65, 1987
1987
-
[38]
The origin of extracellular fields and currents — EEG, ECoG, LFP and spikes,
G. Buzs ´aki, C. Anastassiou, and C. Koch, “The origin of extracellular fields and currents — EEG, ECoG, LFP and spikes,” Nature Reviews Neuroscience, vol. 13, no. 6, pp. 407–420, 2012
2012
-
[39]
The use of electroencephalography in neurodegenerative disease and its utility in dementia,
J. Guo and C. P. Lisgaras, “The use of electroencephalography in neurodegenerative disease and its utility in dementia,” npj Dementia, vol. 2, p. 36, 2026
2026
-
[40]
A revised glos- sary of terms most commonly used by clinical electroencephalographers and updated proposal for the report format of the EEG findings. Revision 2017,
N. Kane, J. Acharya, S. Beniczky, L. Caboclo, S. Finnigan, P. W. Kaplan, H. Shibasaki, R. Pressler, and M. J. A. M. van Putten, “A revised glos- sary of terms most commonly used by clinical electroencephalographers and updated proposal for the report format of the EEG findings...
2017
Reviewed August 1, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.