REVIEW 8 cited by
The largest EEG-based BCI reproducibility study for open science: the MOABB benchmark
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
The largest EEG-based BCI reproducibility study for open science: the MOABB benchmark
read the original abstract
Objective. This study conduct an extensive Brain-computer interfaces (BCI) reproducibility analysis on open electroencephalography datasets, aiming to assess existing solutions and establish open and reproducible benchmarks for effective comparison within the field. The need for such benchmark lies in the rapid industrial progress that has given rise to undisclosed proprietary solutions. Furthermore, the scientific literature is dense, often featuring challenging-to-reproduce evaluations, making comparisons between existing approaches arduous. Approach. Within an open framework, 30 machine learning pipelines (separated into raw signal: 11, Riemannian: 13, deep learning: 6) are meticulously re-implemented and evaluated across 36 publicly available datasets, including motor imagery (14), P300 (15), and SSVEP (7). The analysis incorporates statistical meta-analysis techniques for results assessment, encompassing execution time and environmental impact considerations. Main results. The study yields principled and robust results applicable to various BCI paradigms, emphasizing motor imagery, P300, and SSVEP. Notably, Riemannian approaches utilizing spatial covariance matrices exhibit superior performance, underscoring the necessity for significant data volumes to achieve competitive outcomes with deep learning techniques. The comprehensive results are openly accessible, paving the way for future research to further enhance reproducibility in the BCI domain. Significance. The significance of this study lies in its contribution to establishing a rigorous and transparent benchmark for BCI research, offering insights into optimal methodologies and highlighting the importance of reproducibility in driving advancements within the field.
Forward citations
Cited by 8 Pith papers
-
Subject-Level Heterogeneity in EEG Motor Imagery Decoding: A Large-Scale Benchmark and Portfolio-Based Reduction of the Search Space
Across 165 subjects and 216,714 pipeline evaluations, no single EEG decoding pipeline wins for everyone; compact portfolios of 12 pipelines recover 90-96% of the per-subject oracle.
-
Quantifying Event-Related (De)Synchronization Variability for Brain-Computer Interface: A Unified and Interpretable Framework
A dispersion-based framework quantifies temporal, spatial, and frequency ERD/S variability at within-trial, between-trial, and between-group levels, with correlations of −0.2 to −0.4 to BCI classification performance.
-
Riemannian Geometry for Pre-trained Language Model Embeddings
Aggregating per-token pullback metrics via the Fréchet mean on the SPD manifold outperforms Euclidean mean pooling for sentence classification, with most of the gain attributable to geometric aggregation rather than l...
-
Average Rankings Mask Per-Subject Optimality: A Friedman-Nemenyi Benchmark of EEG Motor-Imagery BCI Decoders
Large-scale benchmark of EEG BCI decoders shows average rankings mask per-subject differences and quantifies gains from participant-aware pipeline selection.
-
DANCE: Detect and Classify Events in EEG
DANCE frames EEG event identification as a set-prediction problem to jointly detect and classify events directly from raw, unaligned signals, outperforming existing methods on seizure monitoring and matching onset-inf...
-
FedSPDnet: Geometry-Aware Federated Deep Learning with SPDnet
FedSPDnet uses manifold projections and retractions to average Stiefel-constrained parameters in federated SPDnet, outperforming standard federated EEGnet on EEG motor imagery benchmarks in F1 score and robustness.
-
Stacked LoRA for Subject-Adaptive EEG Foundation Models in Motor Imagery Decoding
Stacked LoRA, combining global and subject-specific low-rank adapters on frozen EEG foundation models, yields the best motor-imagery accuracy in most backbone–dataset pairs and shows when each path helps.
-
Rethinking Generalized BCIs: Benchmarking 340,000+ Unique Algorithmic Configurations for EEG Mental Command Decoding
No universal EEG decoding pipeline exists: covariance tangent-space projection and CSP rank best on average across three datasets, but per-subject winners vary — and the claimed 340,000+ unique configurations is reall...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.