Pith. sign in

REVIEW 2 major objections 2 minor 16 references

Band-wise VQ-VAE tokenizers on separate EEG frequency bands produce discrete codes that transfer better to new cognitive and motor tasks than broadband alternatives.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

2026-06-30 12:00 UTC pith:EB4VKRRJ

load-bearing objection BandVQ adds per-band VQ-VAEs to EEG pretraining at decent scale, but the performance edge isn't shown to come from the band split. the 2 major comments →

arxiv 2605.24921 v1 pith:EB4VKRRJ submitted 2026-05-24 cs.LG

BandVQ: Band-Wise Vector-Quantized EEG Foundation Model

Jamiyan Sukhbaatar , Satoshi Imamura , Toshihisa Tanaka This is my paper
classification cs.LG
keywords EEG foundation modelvector quantizationfrequency bandstransfer learningmasked modelingVQ-VAEsubject-independent evaluation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper sets out to show that decomposing EEG signals into the five standard frequency bands and training a dedicated VQ-VAE tokenizer on each band yields discrete token sequences better suited for large-scale pretraining than either continuous patches or a single broadband tokenizer. These per-band code indices, together with power tokens, spatial-temporal embeddings, and metadata prefixes, are fed to a shared Transformer that is pretrained with region-based masking on a corpus of more than 9,200 subjects. When the resulting encoder is transferred to six held-out subject-independent classification datasets, it records the highest numbers reported so far on three cognitive tasks while remaining competitive on three motor-imagery tasks.

Core claim

Decomposing EEG into delta, theta, alpha, beta, and gamma bands, learning an independent VQ-VAE tokenizer for each band, and pretraining a shared Transformer encoder on the resulting discrete code indices produces representations that transfer strongly to new cognitive and motor-imagery classification problems under subject-independent evaluation.

What carries the argument

A set of five independent VQ-VAE tokenizers, one per standard frequency band, that map band-limited EEG segments to discrete code indices used for masked pretraining of a single Transformer encoder.

Load-bearing premise

Training separate VQ-VAE tokenizers on each frequency band will produce more transferable discrete representations than training one broadband tokenizer or using continuous patches.

What would settle it

A controlled experiment that replaces the five band-specific VQ-VAEs with a single broadband VQ-VAE, retrains the Transformer under identical conditions, and measures whether transfer accuracy on the same six datasets falls below the band-wise results.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Frequency-specific discrete tokens allow the encoder to preserve activity that would otherwise be averaged away in broadband representations.
  • Region-based masking forces the model to rely on non-adjacent electrodes rather than trivial spatial interpolation.
  • Metadata prefix tokens enable the same encoder to condition on recording reference, band identity, task family, and experimental phase.
  • Pretraining on 357,000 single-channel hours across 71 corpora produces representations that generalize across montages and task types without task-specific fine-tuning during pretraining.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same band-wise discretization strategy could be tested on other oscillatory biosignals whose diagnostic value lies in narrow frequency ranges.
  • Because the tokens are discrete, the pretrained encoder may support efficient retrieval or generation pipelines that continuous patch models cannot match.
  • If the per-band codes prove to be neurophysiologically interpretable, they could serve as an intermediate layer for hybrid symbolic-neural EEG analysis.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 2 minor

Summary. The paper proposes BandVQ, a band-wise vector-quantized EEG foundation model. It decomposes EEG signals into five standard frequency bands (delta, theta, alpha, beta, gamma), trains an independent VQ-VAE tokenizer per band, and pretrains a shared Transformer encoder on the resulting discrete code indices using masked code tokens, quantized log-power tokens, channel/temporal embeddings, metadata prefixes (reference, band, task family, phase), and region-based masking. The model is pretrained on 71 public corpora (>9200 subjects, 357k single-channel hours) and evaluated on six subject-independent classification datasets, reporting the highest results on three cognitive tasks and competitive results on three motor-imagery tasks.

Significance. If the band-wise design is shown to drive the reported gains, the approach could advance EEG foundation modeling by producing more frequency-specific discrete tokens than broadband or continuous-patch alternatives, with the large-scale pretraining and metadata conditioning as additional strengths. The evaluation on diverse subject-independent tasks provides a reasonable testbed for transfer claims.

major comments (2)
  1. [Abstract / Evaluation] Abstract and Evaluation section: the headline claim that band-wise VQ yields superior transferable representations (highest results on three cognitive tasks) is not supported by any ablation against a single broadband VQ-VAE or continuous-patch baseline trained on the identical 71 corpora and 357k hours; without this comparison the performance advantage cannot be attributed to the per-band decomposition rather than scale, metadata prefixes, or region masking.
  2. [Methods] Methods section (tokenizer training): no details are provided on whether the five independent VQ-VAEs share codebook sizes, training hyperparameters, or reconstruction objectives, nor on how the discrete indices are aligned across bands before being fed to the shared encoder; these choices are load-bearing for the claim that band-specific quantization improves transfer.
minor comments (2)
  1. [Abstract] The abstract states 'quantized absolute log-power tokens' without defining the quantization scheme or how these tokens interact with the masked code tokens in the encoder input.
  2. [Evaluation] Dataset-split details (subject-independent partitioning, any overlap between pretraining corpora and the six evaluation sets) are not described, which is needed to assess leakage risk.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the detailed and constructive feedback. We address each major comment below, indicating planned revisions to the manuscript where appropriate.

read point-by-point responses

  1. Referee: [Abstract / Evaluation] Abstract and Evaluation section: the headline claim that band-wise VQ yields superior transferable representations (highest results on three cognitive tasks) is not supported by any ablation against a single broadband VQ-VAE or continuous-patch baseline trained on the identical 71 corpora and 357k hours; without this comparison the performance advantage cannot be attributed to the per-band decomposition rather than scale, metadata prefixes, or region masking.

    Authors: We agree that without a controlled ablation on the identical pretraining corpus it is not possible to isolate the contribution of band-wise decomposition from other factors such as scale, metadata conditioning, or region-based masking. Our reported results are the highest published under the subject-independent transfer setting, but we do not claim direct causal attribution to the band-wise design in the absence of such a baseline. We will revise the abstract and evaluation sections to remove any implication of superiority due specifically to band-wise VQ and will add an explicit limitations paragraph discussing this point. A full-scale ablation is not feasible within the current revision timeline due to compute constraints, so the revision will be partial. revision: partial

  2. Referee: [Methods] Methods section (tokenizer training): no details are provided on whether the five independent VQ-VAEs share codebook sizes, training hyperparameters, or reconstruction objectives, nor on how the discrete indices are aligned across bands before being fed to the shared encoder; these choices are load-bearing for the claim that band-specific quantization improves transfer.

    Authors: We will expand the tokenizer subsection of the Methods to specify that each of the five VQ-VAEs uses an independent codebook of size 1024, identical reconstruction objectives (MSE on band-filtered waveforms plus commitment loss), and the same training hyperparameters (learning rate, batch size, epochs). Discrete indices are aligned by treating each band as a separate token sequence; band-specific embeddings are added before concatenation along the sequence dimension for input to the shared Transformer. These details will be added in the next revision. revision: yes

Circularity Check

0 steps flagged

No significant circularity; performance claims rest on external held-out evaluation

full rationale

The paper describes a standard pretraining + transfer pipeline: band decomposition, per-band VQ-VAE tokenizers, Transformer pretraining on 71 corpora, then evaluation on six separate subject-independent classification datasets. No equation, procedure, or self-citation is shown that would make the headline transfer numbers reduce by construction to quantities fitted inside the same run. The evaluation setting uses held-out data and subject-independent splits, supplying external benchmarks. The absence of an ablation against broadband VQ-VAE is a missing-control issue, not a circularity issue.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The design rests on the domain assumption that the five classical EEG bands are the right granularity for separate tokenization and that standard VQ-VAE training will yield useful discrete codes; no free parameters or invented entities are named in the abstract.

axioms (1)
  • domain assumption EEG signals can be usefully decomposed into the five fixed frequency bands delta, theta, alpha, beta, gamma for independent tokenization
    Invoked by the choice to train one VQ-VAE per band rather than a single broadband model.

pith-pipeline@v0.9.1-grok · 5739 in / 1282 out tokens · 44567 ms · 2026-06-30T12:00:55.137694+00:00 · methodology

0 comments
read the original abstract

A central challenge in electroencephalography (EEG) foundation modeling is learning transferable representations across recordings with diverse tasks, montages, references, and spectral characteristics. Existing masked modeling approaches often rely on broadband continuous patches or a single discrete representation, which may underrepresent frequency-specific activity. This paper proposes BandVQ, a band-wise vector-quantized EEG foundation model that decomposes EEG into delta, theta, alpha, beta, and gamma bands, trains an independent VQ-VAE tokenizer for each band, and pretrains a shared Transformer encoder on the resulting discrete VQ code indices. The encoder uses masked code tokens, quantized absolute log-power tokens, channel and temporal embeddings, and metadata prefix tokens representing reference, band, task family, and phase. Region-based masking is also introduced to reduce the trivial reconstruction of spatially adjacent electrodes. The model is pretrained on 71 public EEG corpora comprising over 9,200 subjects and 357,000 single-channel hours and evaluated on six subject-independent classification datasets. Under the current evaluation setting, the proposed model achieves strong transfer performance, with the highest reported results on three cognitive tasks and competitive performance on three motor imagery tasks.

Figures

Figures reproduced from arXiv: 2605.24921 by Jamiyan Sukhbaatar, Satoshi Imamura, Toshihisa Tanaka.

Figure 1
Figure 1. Figure 1: Overview of BandVQ, the proposed band-wise vector-quantized EEG foundation model. Band-specific VQ-VAE tokenizers are using quantized absolute log-power tokens, metadata prefix tokens, and region-based masking. trained offline and then frozen to produce discrete code indices. A shared Transformer encoder is pretrained with masked code prediction [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

16 extracted references · 16 canonical work pages · 1 internal anchor

  1. [1]

    International Journal of Psychophysiology97(3), 210–220 (2015)

    Burle, B., Spieser, L., Roger, C., Casini, L., Hasbroucq, T., Vidal, F.: Spatial and temporal resolutions of EEG: Is it really black and white? a scalp current density view. International Journal of Psychophysiology97(3), 210–220 (2015)

    work page 2015

  2. [2]

    Dreyer, P., Roc, A., Pillette, L., Rimbert, S., Lotte, F.: A large EEG database with users’ profile information for motor imagery brain-computer interface research. Sci. Data10(1), 580 (2023)

    work page 2023

  3. [3]

    In: International Conference on Learning Rep- resentations (2024), https://openreview.net/forum?id=QzTpTRVtrP

    Jiang, W., Zhao, L., Lu, B.: Large brain model for learning generic representations with tremendous EEG data in BCI. In: International Conference on Learning Rep- resentations (2024), https://openreview.net/forum?id=QzTpTRVtrP

    work page 2024

  4. [4]

    Frontiers in Human Neuroscience15, 653659 (2021)

    Kostas, D., Aroca-Ouellette, S., Rudzicz, F.: BENDR: Using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG data. Frontiers in Human Neuroscience15, 653659 (2021)

    work page 2021

  5. [5]

    Journal of Neural Engineering15(3), 031005 (2018)

    Lotte, F., Bougrain, L., Cichocki, A., Clerc, M., Congedo, M., Rakotomamonjy, A., Yger, F.: A review of classification algorithms for EEG-based brain–computer interfaces: A 10 year update. Journal of Neural Engineering15(3), 031005 (2018)

    work page 2018

  6. [6]

    Melnik, A., Legkov, P., Izdebski, K., Kärcher, S.M., Hairston, W.D., Ferris, D.P., König, P.: Systems, subjects, sessions: To what extent do these factors influence EEG data? Frontiers in Human Neuroscience11, 150 (2017)

    work page 2017

  7. [7]

    Scientific Data6, 103 (2019)

    Pernet, C.R., Appelhoff, S., Gorgolewski, K.J., Flandin, G., Phillips, C., Delorme, A., Oostenveld, R.: EEG-BIDS, an extension to the brain imaging data structure for electroencephalography. Scientific Data6, 103 (2019)

    work page 2019

  8. [8]

    Frontiers in Computational Neuro- science13, 87 (2020)

    Saha, S., Baumert, M.: Intra- and inter-subject variability in EEG-based senso- rimotor brain computer interface: A review. Frontiers in Computational Neuro- science13, 87 (2020)

    work page 2020

  9. [9]

    Shin, J., Von Lühmann, A., Kim, D.W., Mehnert, J., Hwang, H.J., Müller, K.R.: Simultaneous acquisition of EEG and NIRS during cognitive tasks for an open access dataset. Sci. Data5(1), 1–16 (2018)

    work page 2018

  10. [10]

    Sukhbaatar, J., Imamura, S., Inoue, I., Murakami, S., Hassan, K.M., Han, S., Chanpornpakdi, I., Tanaka, T.: SingLEM: Single-channel large EEG model (2025), https://arxiv.org/abs/2509.17920

    work page internal anchor Pith review arXiv 2025

  11. [11]

    Advances in neural information processing systems30(2017)

    Van Den Oord, A., Vinyals, O., et al.: Neural discrete representation learning. Advances in neural information processing systems30(2017)

    work page 2017

  12. [12]

    Journal of Neuroscience35(38), 13257–13265 (2015)

    Voytek, B., Kramer, M.A., Case, J., Lepage, K.Q., Tempesta, Z.R., Knight, R.T., Gazzaley, A.: Age-related changes in 1/f neural electrophysiological noise. Journal of Neuroscience35(38), 13257–13265 (2015)

    work page 2015

  13. [13]

    In: International Conference on Learning Representations (2025), https://openreview.net/forum?id=NPNUHgHF2w

    Wang, J., Zhao, S., Luo, Z., Zhou, Y., Jiang, H., Li, S., Li, T., Pan, G.: CBraMod: A criss-cross brain foundation model for EEG de- coding. In: International Conference on Learning Representations (2025), https://openreview.net/forum?id=NPNUHgHF2w

    work page 2025

  14. [14]

    Frontiers in Human Neuroscience14, 103 (2020)

    Xu, L., Xu, M., Ke, Y., An, X., Liu, S., Ming, D.: Cross-dataset variability problem in EEG decoding with deep learning. Frontiers in Human Neuroscience14, 103 (2020)

    work page 2020

  15. [15]

    Yang, B., Rong, F., Xie, Y., Li, D., Zhang, J., Li, F., Shi, G., Gao, X.: A multi-day and high-quality EEG dataset for motor imagery brain-computer interface. Sci. Data12(1), 488 (2025)

    work page 2025

  16. [16]

    In: Advances in Neural Information Processing Systems

    Yang, C., Westover, M.B., Sun, J.: BIOT: Biosignal transformer for cross-data learning in the wild. In: Advances in Neural Information Processing Systems. vol. 36, pp. 78240–78260 (2023)

    work page 2023