REVIEW 4 major objections 4 minor 38 references
Lower variability in EEG responses across time, space, and frequency is associated with higher brain-computer interface performance.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-01 08:22 UTC pith:OTOGEQBZ
load-bearing objection A coherent and useful variability-measurement toolkit for BCI research, but the 'independent modality' claim is stronger than the math supports and should be toned down or backed with a synthetic validation. the 4 major comments →
Quantifying Event-Related (De)Synchronization Variability for Brain-Computer Interface: A Unified and Interpretable Framework
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The paper's central claim is that event-related (de)synchronization (ERD/S) variability can be quantified as the mean distance between extracted slices of an ERD/S tensor and a reference centroid, and that this single dispersion formula, v = (1/n) Σ distance(x_i, M), reproduces and refines prior variability measures. When applied to 133 users across two datasets, the metrics show that within-trial and between-trial variability in time, space, and frequency carry negative rank-order correlations with BCI performance in most conditions, with the test user's own variability predicting cross-user accuracy while the training group's average does not. The paper further observes that a tangent-spac
What carries the argument
The load-bearing object is a marginal-slice dispersion metric. For each trial, the ERD/S tensor is reduced to one modality by averaging over the other two dimensions—time series via averaging over channels and frequencies, spatial maps via averaging over frequencies and time, spectral patterns via averaging over channels and time. Variability is then v = (1/n) Σ distance(x_i, M), with per-modality distances: squared Euclidean for within-trial temporal, angular distance for spatial and spectral patterns, and dynamic time warping for between-trial and between-group time series. The centroid M is chosen as the ordinary average for Euclidean distances, or as the point on the unit sphere that min
Load-bearing premise
The metrics are presented as independent measures of time, space, and frequency, but each one is computed on a slice obtained by averaging over the other two modalities; the load-bearing, untested premise is that fluctuations in one modality do not bleed into another, so spatial jitter could masquerade as temporal variability.
What would settle it
Simulate ERD/S tensors with known, separately controllable jitter in time, space, and frequency; if the temporal variability score rises when only spatial jitter is injected, the independence claim at the heart of the framework is falsified.
If this is right
- If the observed negative correlations are real, then real-time variability scores can serve as feedback during user training, giving users a target to stabilize rather than just an accuracy number.
- The test-user-only effect implies that adaptive BCIs should compute variability from the current user's live signals, not from the training population's averages.
- The stronger variability sensitivity of the Riemannian classifier relative to the deep network suggests that datasets with known spatial pattern drift are better handled by convolutional architectures, or that covariance-based pipelines need spatial alignment before classification.
- The positive test-referenced group-wise correlations, if causal, suggest that enriching training sets with users whose spatial and spectral patterns are unlike the target user may improve cross-user generalization.
Where Pith is reading between the lines
- The same slice-and-dispersion recipe should transfer to other EEG tensors—ERP latencies, connectivity matrices, or time-frequency images from other paradigms—so the framework is a generic variability meter, not just an ERD/S one.
- The independence of the three modalities is untested: because each slice marginalizes away the other dimensions, cross-contamination is plausible. A synthetic experiment that injects jitter in one modality and checks whether the other metrics move would settle this cheaply.
- If the negative correlation is causal, then adding a variability penalty on the temporal slice to a deep decoder's loss function is a direct, testable way to convert this paper's correlation into an intervention.
- The training-group null result suggests a practical ceiling for static cross-user calibration—collecting more diverse training data will not fix an unstable current user, so online adaptation using the user's own variability score is the obvious next step.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a unified framework for quantifying temporal, spatial, and frequency variability of event-related (de)synchronization (ERD/S) in EEG-based BCIs. Variability is defined as the average distance between extracted ERD/S slices and a reference centroid, with three operators (Eqs. 2–4) that average over all dimensions except the target modality, yielding nine metrics at within-trial, between-trial, and between-trial-group levels. Using two motor imagery datasets (Dreyer2023, Lee2019; N=133 after exclusions), the authors compute Spearman correlations between these metrics and BCI classification performance for within-user and cross-user settings, with Benjamini–Hochberg correction and bootstrap standard errors. They report modest negative correlations (−0.2 to −0.4) across most conditions, suggesting lower variability is associated with better performance, and they compare the proposed metrics against existing variability metrics (STDERD, class stability, MITT). The framework is released as an open-source Python package.
Significance. If the central claim holds, the framework provides a useful toolbox for characterizing EEG variability in MI-BCI, with a flexible and interpretable formulation that could support dataset comparison, classifier robustness analysis, and variability-aware user training. The use of two public datasets, explicit statistical corrections, and an open-source release are notable strengths. However, the headline claim that the metrics 'independently quantify' temporal, spatial, and frequency variability is not established by the marginal-projection definitions, and the correlational validation would benefit from a non-variability control and a more careful treatment of test-user-referenced metrics. The contribution is potentially significant for the BCI community, but the current evidence is weaker than the abstract suggests.
major comments (4)
- [Section II, Eqs. (2)–(4) and Section II-4]
- [Section IV and Section V-A (correlational validation)]
- [Section IV, Fig. 3 and Section V-B (BtwTrialGrp-TR)]
- [Section II-B1a (WiTrialTemp)]
minor comments (4)
- [Throughout]
- [Section III-B]
- [Section II-3]
- [Section IV, Fig. 1]
Circularity Check
No significant circularity: the variability metrics are explicit definitions, and the reported correlations are empirical outcomes rather than fitted predictions.
full rationale
The paper's central construction is Eq. (5), v = (1/n) Σ distance(x_i, M), which is a definition of dispersion around a centroid, not a derived result. The operators in Eqs. (2)–(4) are explicitly stated marginal averages; their possible cross-modal leakage is a validity/interpretability concern, not a circularity, because no equation is assumed equal to the output it is used to explain. The proposed metrics are not fitted to classification performance: no free parameter is adjusted to maximize the reported Spearman correlations. The existing metrics (STDERD, class stability, MITT) are used as baselines computed independently, not as inputs to the definition of the new metrics. The self-citations by the authors (e.g., refs. [6], [14], [15]) provide background motivation or serve as comparison metrics; none is load-bearing for the new derivation, and there is no imported uniqueness theorem or ansatz that forces the framework's form. The correlations between variability and BCI performance are empirical findings on two datasets, not predictions derived from the metric definitions. Therefore no circular step can be identified.
Axiom & Free-Parameter Ledger
free parameters (5)
- sub-epoch length/stride (WiTrial metrics) =
0.1 s / 0.1 s
- DPSS multitaper window =
time-bandwidth=4, tapers=3, window=1 s
- baseline interval =
−2 to 0 s
- frequency band and channels =
8–30 Hz; C3/C4 (temporal/freq), all channels (spatial)
- decimation/downsampling =
128 Hz resample; decimation by 2
axioms (4)
- domain assumption ERD/S is a valid normalization of task-related power changes relative to baseline.
- standard math The Fréchet mean on the unit hypersphere (Eq. 7) is a stable reference for angular distances.
- ad hoc to paper Sub-epochs are independent observations.
- domain assumption Channels C3/C4 capture the relevant motor-imagery responses for the temporal and frequency metrics.
read the original abstract
Objective: Brain-Computer Interfaces (BCIs) enable the control of external devices by decoding user intentions from electroencephalography (EEG). However, substantial EEG variability within and between users remains a major challenge. To better understand this variability, we propose interpretable metrics that independently quantify temporal, spatial, and frequency variability in BCI related brain activity within and between users. Methods: We propose a framework to quantify variability by extracting EEG features and defining variability as their dispersion around their centroid using appropriate distance functions. Using two motor imagery BCI datasets (N = 133 users), we investigated the relationship between BCI performance and the variability metrics through within-user and cross-user classification experiments. Results: Negative correlations of -0.2 to -0.4 were observed across most conditions, suggesting that lower variability is associated with higher BCI performance. Moreover, the metrics revealed differences in robustness to variability between the deep learning and Riemannian-based classifiers, with the former showing weaker correlations. Conclusion: The results demonstrate the effectiveness of the proposed variability metrics and suggest that reducing variability may improve BCI performance while revealing differences in the sensitivity of classification models to different types of variability. Significance: The framework quantifies temporal, spatial, and frequency variability at multiple hierarchical levels (within-trial, between-trial, and between-trial-group), providing interpretable measures to better understand EEG variability and support more robust BCIs. It could also be used to characterize dataset variability, evaluate classifier sensitivity, incorporate variability into objective functions, and provide variability-based user feedback.
Figures
Reference graph
Works this paper leans on
-
[1]
Chevallier, Sylvain and Carrara, Igor and Aristimunha, Bruno and Guetschel, Pierre and Sedlar, Sara and Lopes, Bruna and Velut, Sebastien and Khazem, Salim and Moreau, Thomas , year = 2024, month = apr, number =. The Largest. doi:10.48550/arXiv.2404.15319 , urldate =. arXiv , keywords =:2404.15319 , primaryclass =
-
[2]
Progress in brain research , volume=
Advances in user-training for mental-imagery-based BCI control: Psychological and cognitive factors and their neural correlates , author=. Progress in brain research , volume=. 2016 , publisher=
2016
-
[3]
and Rimbert, Sébastien and Tabie, Marc and Rekrut, Maurice , booktitle =
Lotte, Fabien and Kojima, Simon and Kueper, Niklas and Dreyer, Pauline and Math, Rafael and Sharma, Mansi and Roy, Raphaëlle N. and Rimbert, Sébastien and Tabie, Marc and Rekrut, Maurice , booktitle =. Tools, Challenges and Opportunities to Address Intra- and Inter-User Variability in
-
[4]
Dreyer, Pauline and Roc, Aline and Pillette, L. A Large. Sci. Data , volume =. doi:10.1038/s41597-023-02445-z , urldate =
-
[5]
Ivanov, Nicolas and Wong, Madeline and Chau, Tom , year = 2026, month = feb, journal =. A. doi:10.1142/S0129065725500753 , urldate =
-
[6]
doi:10.1093/gigascience/giz002 , urldate =
Lee, Min-Ho and Kwon, O-Yeon and Kim, Yong-Jeong and Kim, Hong-Kyung and Lee, Young-Eun and Williamson, John and Fazli, Siamac and Lee, Seong-Whan , year = 2019, month = may, journal =. doi:10.1093/gigascience/giz002 , urldate =
-
[7]
Defining and Quantifying Users' Mental Imagery-Based
Lotte, Fabien and Jeunet, Camille , year = 2018, month = jun, journal =. Defining and Quantifying Users' Mental Imagery-Based. doi:10.1088/1741-2552/aac577 , urldate =
-
[8]
Pfurtscheller, G. and. Event-Related. Clin. Neurophysiol. , volume =. doi:10.1016/S1388-2457(99)00141-8 , urldate =
-
[9]
Is. Proc. IEEE MetroXRAINE , author =. doi:10.1109/MetroXRAINE54828.2022.9967551 , urldate =
arXiv 2022
-
[10]
Dynamic Programming Algorithm Optimization for Spoken Word Recognition , author =. IEEE Trans. Acoust. Speech Signal Process. , volume =. doi:10.1109/TASSP.1978.1163055 , urldate =
arXiv 1978
-
[11]
, year = 1978, month = may, journal =
Slepian, D. , year = 1978, month = may, journal =. Prolate Spheroidal Wave Functions, Fourier Analysis, and Uncertainty ---. doi:10.1002/j.1538-7305.1978.tb02104.x , urldate =
arXiv 1978
-
[12]
Rodrigues, Pedro Luiz Coelho and Jutten, Christian and Congedo, Marco , year = 2019, month = aug, journal =. Riemannian. doi:10.1109/TBME.2018.2889705 , urldate =
arXiv 2019
-
[13]
Saha, Simanto and Baumert, Mathias , year = 2020, month = jan, journal =. Intra- and. doi:10.3389/fncom.2019.00087 , urldate =
arXiv 2020
-
[14]
Interaction between Steady-State Visually Evoked Potentials at Nearby Flicker Frequencies , author =. Sci. Rep. , volume =. doi:10.1038/s41598-020-62180-y , urldate =
-
[15]
Wolpaw, J.R. and Birbaumer, N. and Heetderks, W.J. and McFarland, D.J. and Peckham, P.H. and Schalk, G. and Donchin, E. and Quatrano, L.A. and Robinson, C.J. and Vaughan, T.M. , year = 2000, month = jun, journal =. Brain-Computer Interface Technology: A Review of the First International Meeting , shorttitle =. doi:10.1109/TRE.2000.847807 , urldate =
arXiv 2000
-
[16]
Brusini, Lorenza and Stival, Francesca and Setti, Francesco and Menegatti, Emanuele and Menegaz, Gloria and Storti, Silvia Francesca , year = 2021, month = dec, journal =. A. doi:10.1109/THMS.2021.3115094 , urldate =
arXiv 2021
-
[17]
doi:10.3390/s19061423 , urldate =
Padfield, Natasha and Zabalza, Jaime and Zhao, Huimin and Masero, Valentin and Ren, Jinchang , year = 2019, month = mar, journal =. doi:10.3390/s19061423 , urldate =
-
[18]
Vidaurre, Carmen and Blankertz, Benjamin , year = 2010, month = jun, journal =. Towards a. doi:10.1007/s10548-009-0121-6 , urldate =
-
[19]
Hammer, Eva Maria and Halder, Sebastian and Blankertz, Benjamin and Sannelli, Claudia and Dickhaus, Thorsten and Kleih, Sonja and M. Psychological Predictors of. Biol. Psychol. , volume =. doi:10.1016/j.biopsycho.2011.09.006 , urldate =
-
[20]
Discrepancy between Inter- and Intra-Subject Variability in
Huang, Gan and Zhao, Zhiheng and Zhang, Shaorong and Hu, Zhenxing and Fan, Jiaming and Fu, Meisong and Chen, Jiale and Xiao, Yaqiong and Wang, Jun and Dan, Guo , year = 2023, month = feb, journal =. Discrepancy between Inter- and Intra-Subject Variability in. doi:10.3389/fnins.2023.1122661 , urldate =
arXiv 2023
-
[21]
Saha, Simanto and Ahmed, Khawza Iftekhar Uddin and Mostafa, Raqibul and Hadjileontiadis, Leontios and Khandoker, Ahsan , year = 2018, month = feb, journal =. Evidence of. doi:10.1109/TNSRE.2017.2778178 , urldate =
arXiv 2018
-
[22]
Large Scale Investigation of the Effect of Gender on Mu Rhythm Suppression in Motor Imagery Brain-Computer Interfaces , author =. Brain-Comput. Interfaces , volume =. doi:10.1080/2326263X.2024.2345449 , urldate =
arXiv 2024
-
[23]
Subject Inefficiency Phenomenon of Motor Imagery Brain-Computer Interface:
Zhang, Rui and Li, Fali and Zhang, Tao and Yao, Dezhong and Xu, Peng , year = 2020, month = sep, journal =. Subject Inefficiency Phenomenon of Motor Imagery Brain-Computer Interface:. doi:10.26599/BSA.2020.9050021 , urldate =
arXiv 2020
-
[24]
A Review of Classification Algorithms for
Lotte, F and Congedo, M and L. A Review of Classification Algorithms for. J. Neur. Eng. , volume =. doi:10.1088/1741-2560/4/2/R01 , urldate =
-
[25]
A Review of Classification Algorithms for
Lotte, F and Bougrain, L and Cichocki, A and Clerc, M and Congedo, M and Rakotomamonjy, A and Yger, F , year = 2018, month = apr, journal =. A Review of Classification Algorithms for. doi:10.1088/1741-2552/aab2f2 , urldate =
- [26]
-
[27]
Zhang, Kai and Xu, Guanghua and Zheng, Xiaowei and Li, Huanzhong and Zhang, Sicong and Yu, Yunhui and Liang, Renghao , year = 2020, month = nov, journal =. Application of. doi:10.3390/s20216321 , urldate =
-
[28]
Li, Mingai and Xu, Dongqin , year = 2024, month = feb, journal =. Transfer. doi:10.1007/s12204-022-2488-4 , urldate =
- [29]
-
[30]
Deep Learning with Convolutional Neural Networks for
Schirrmeister, Robin Tibor and Springenberg, Jost Tobias and Fiederer, Lukas Dominique Josef and Glasstetter, Martin and Eggensperger, Katharina and Tangermann, Michael and Hutter, Frank and Burgard, Wolfram and Ball, Tonio , year = 2017, journal =. Deep Learning with Convolutional Neural Networks for. doi:10.1002/hbm.23730 , urldate =
-
[31]
Yger, Florian and Berar, Maxime and Lotte, Fabien , year = 2017, month = oct, journal =. Riemannian. doi:10.1109/TNSRE.2016.2627016 , urldate =
arXiv 2017
-
[32]
A Well-Conditioned Estimator for Large-Dimensional Covariance Matrices , author =. J. Multivar. Anal. , volume =. doi:10.1016/S0047-259X(03)00096-4 , urldate =
-
[33]
Barachant, Alexandre and Barthélemy, Quentin and King, Jean-Rémi and Gramfort, Alexandre and Chevallier, Sylvain and Rodrigues, Pedro L. C. and Olivetti, Emanuele and Goncharenko, Vladislav and Wagner vom Berg, Gabriel and Reguig, Ghiles and Lebeurrier, Arthur and Bjäreholt, Erik and Yamamoto, Maria Sayu and Clisson, Pierre and Corsi, Marie-Constance and ...
-
[34]
Aristimunha, Bruno and Guetshel, Pierre and Wimpff, Martin and Gemein, Lukas and Rommel, Cedric and Banville, Hubert and Sliwowski, Maciej and Wilson, Daniel and Brandt, Simon and Gnassounou, Th. Braindecode: Toolbox for Decoding Raw Electrophysiological Brain Data with Deep Learning Models , shorttitle =. doi:10.5281/zenodo.17699192 , urldate =
-
[35]
Benjamini, Yoav and Hochberg, Yosef , year = 1995, journal =. Controlling the. doi:10.1111/j.2517-6161.1995.tb02031.x , urldate =
arXiv 1995
-
[36]
A Procedure to Minimize. Biomed. Signal Process. Control. , volume =. doi:10.1016/j.bspc.2023.105745 , urldate =
arXiv 2023
-
[37]
Wriessnegger, Selina C. and. Inter- and. Front. Hum. Neurosci. , volume =. doi:10.3389/fnhum.2020.576241 , urldate =
arXiv 2020
-
[38]
Pymanopt: A Python Toolbox for Optimization on Manifolds using Automatic Differentiation , url =
James Townsend and Niklas Koep and Sebastian Weichwald , journal =. Pymanopt: A Python Toolbox for Optimization on Manifolds using Automatic Differentiation , url =
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.