{"id":"db61c6ea-950b-4708-92f6-b9219eb7ceaf","arxiv_id":"2506.16168","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A perspective paper argues that EEG-BCI generalization failures stem from causally induced distribution shifts and that current EEG datasets are orders of magnitude too small to support large brain models.","lead":"This paper maps EEG-based brain-computer interface research through a causal lens, arguing that distribution shifts and scarce training data are the core obstacles to real-world AI-BCI systems. It compares the scale of EEG datasets to LLM training data and proposes future directions including interventional data collection, spiking networks, and causality-aware large brain models.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 1 contradicts Section 4.1's 'at most millions of tokens' claim, undermining the quantitative data-scarcity argument.","rationale":"The reader's verdict is CONDITIONAL, and the reader identified the token-definition comparability as the weakest assumption while also noting the 'millions vs. hundreds of millions' inconsistency in the rationale. My stress-test agrees that the internal inconsistency is the most concrete and load-bearing issue, because it is a factual contradiction within the paper's own quantitative argument. The central qualitative claim—that EEG-BCI generalization is hampered by causal distribution shifts and that available EEG data is far smaller than NLP-scale corpora—does not depend on the exact token count and remains plausible. However, the paper's quantitative section is presented as a formalization ('we formalize to allow for a quantitative comparison against LLMs'), so an order-of-magnitude error in the text undermines the authority of that comparison. Since the reader already conditioned the verdict on correcting this and on adding dataset inclusion criteria, my concern does not move the verdict; it reinforces the need for the same corrections. A revised version that fixes the 'millions' sentence and clarifies tokenization assumptions would satisfy the condition. No ad hominem is intended; the issue is a straightforward numerical inconsistency that is easily verifiable from the paper's own table.","tokens_in":14744,"tokens_out":1695,"duration_ms":20433,"concrete_test":"Recompute the token counts in Table 1 from the referenced metadata repository using N_tokens = (T_i * C_i) / 1s, and compare the results to the text in Section 4.1. Independently verify at least one row (e.g., Seizure Prediction: 2969 h * 18 channels * 3600 s/h = 192.4M tokens) to confirm the table matches the formula. If the recomputed values remain in the 10^8 range, the 'at most millions of tokens' sentence must be corrected or explicitly labeled as an order-of-magnitude illustration—this is a factual error, not a stylistic issue. Additionally, test whether using an alternative tokenization (e.g., multi-channel patches or 0.1 s windows) changes the order of magnitude of the gap to LLM data; if not, the qualitative scarcity argument survives but should be stated with corrected numbers.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central quantitative claim is that EEG data for LBM training is scarce relative to LLM-scale corpora. Section 4.1 formalizes token count as N_tokens = (T_i * C_i) / L with L = 1 s, and Table 1 reports values ranging from 81M (Brain-to-Image) to 406M (Emotion Recognition) tokens. However, the text immediately after Table 1 states: 'These results highlight that LBMs are trained on datasets containing at most millions of tokens (10^6) if we consider all data across tasks.' This is internally inconsistent by two orders of magnitude: each individual task already yields tens to hundreds of millions of tokens. The data-scarcity argument depends on this quantitative comparison to LLaMA 3.1's 15 trillion tokens. Although even corrected 10^8-scale numbers still show a large gap (four orders of magnitude), the specific 'millions' statement is factually wrong as written. A secondary concern is that the token definition (one second of one channel) may not align with how EEG foundation models actually tokenize signals—e.g., LaBraM uses multi-channel time windows and continuous representations—so the comparability to NLP word tokens is not well established. Nevertheless, the clearest load-bearing flaw is the internal contradiction, because it directly affects the credibility of the paper's headline quantitative comparison.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This review/position paper argues that EEG-based BCI systems fail to generalize in real-world settings because the data-generating process is causal: stimulus, subject engagement, device, environment, and neuroplasticity jointly induce distribution shifts that violate the i.i.d. assumption underlying standard deep learning. The authors organize BCI paradigms along two axes (exogenous/endogenous and voluntary/involuntary), present a causal graph, and identify two grand challenges: distribution shifts and data scarcity. They formalize data scarcity by estimating the number of EEG tokens per task, compare these estimates with LLaMA 3.1's 15-trillion-token training corpus, and argue that datasets must include interventional data to support robust benchmarking. The paper closes with technological, methodological, and ethical recommendations, including spiking neural networks, continuous-time reasoning, and causality-aware large brain models.","tokens_in":14967,"tokens_out":4324,"duration_ms":49032,"significance":"The causal framing is a useful and relatively novel organizing principle for a scattered literature, and the proposed two-axis taxonomy gives practitioners a clear way to anticipate which distribution shifts matter for a given BCI paradigm. The explicit token-count comparison, although currently flawed in presentation, is a valuable attempt to make the data-scarcity argument concrete and falsifiable. The call for leave-one-intervention-out benchmarking is actionable and aligns with recent trends in causally motivated evaluation. The authors also share dataset metadata publicly via a GitHub repository, which supports reproducibility. If the quantitative inconsistency is fixed and the token-equivalence assumption is justified, the paper could serve as a useful roadmap for the field.","major_comments":[{"comment":"The table reports 81M, 108M, 406M, 179M, 161M, and 193M tokens for the six tasks, i.e., values in the 10^7 to 10^8 range, yet the text immediately after the table states that LBMs are trained on datasets containing 'at most millions of tokens (10^6)' if all data across tasks are considered. This is internally inconsistent by two orders of magnitude. Because the comparison with LLaMA 3.1's 15-trillion-token corpus is the headline evidence for the second grand challenge, the statement must be corrected or the table recomputed before publication. If the intended claim is 'hundreds of millions,' the gap to 15 trillion is still large, but the current wording is factually wrong.","section":"Section 4.1, Table 1 and following paragraph"},{"comment":"The token definition N_tokens = (T_i * C_i) / L with L = 1 s assumes that one second of one EEG channel is directly comparable to an NLP subword token. This assumption is not justified in the text; current EEG foundation models such as LaBraM use multi-channel time windows and continuous or patch-based representations rather than per-channel-per-second tokenization. Depending on the tokenizer, the estimated token counts could change by more than an order of magnitude. The authors should either justify the comparability with specific references to how LBMs tokenize EEG, or reframe the comparison as an order-of-magnitude illustration rather than a precise accounting. The qualitative distribution-shift argument does not depend on this assumption, but the quantitative data-scarcity claim does.","section":"Section 4.1, Eq. for N_tokens"}],"minor_comments":[{"comment":"The acronym ERP is used both for Event-Related Potentials (Section 2.1.3) and Error-Related Potentials (Section 2.1.4), which is confusing; please disambiguate, for example by writing 'ErrP' for the error-related potential.","section":"Section 2.1.3 and Section 2.1.4"},{"comment":"The exponents in '10^6' and '1012' are missing superscript formatting; '15 trillion (1012) tokens' should read '10^12'.","section":"Section 4.1"},{"comment":"The phrase 'V oluntarily engaged' contains an erroneous space and should read 'Voluntarily engaged'.","section":"Section 2"},{"comment":"The GitHub links are rendered as '/githubhttps://github.com/...' and '1/github...', which appears to be a typesetting artifact; these should be cleaned up in the final version.","section":"Section 4.1 and GitHub link"},{"comment":"The caption describes inner and outer circles for recording hours per task and per dataset, but the figure as rendered is hard to interpret; please add a clearer legend or report the underlying totals in the text or table.","section":"Figure 2a"},{"comment":"The sentence introducing Adaptive Resonance Theory as a biologically inspired learning approach lacks a citation specific to its use in EEG or BCI contexts; the Grossberg reference alone does not establish that connection.","section":"Section 5.2.1"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is within the scope of the journal as a review/position paper. The self-citations (e.g., Craik et al. 2019, Pacheco-Ramírez et al. 2024) are topically appropriate and do not carry the central claims. The main concern is the internal inconsistency in the quantitative data-scarcity argument, which is fixable within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This paper is a worthwhile perspective on why EEG-based BCIs fail to generalize and what the field should do about it. The causal framing of the data generating process—stimulus, subject engagement, device, environment—is not new (they credit Barmpas et al. 2024 and Schölkopf et al. 2021), but they apply it clearly to the BCI landscape and derive a useful four-quadrant taxonomy (exogenous/endogenous × voluntary/involuntary). The call for interventional data and leave-one-intervention-out evaluation is sensible and actionable. The compiled dataset metadata is a modest but real contribution, and the idea of quantifying the data gap against LLM corpora is the right instinct.\n\nThe quantitative core of Section 4.1 has a two-order-of-magnitude internal inconsistency. The text after Table 1 says LBMs are trained on \"at most millions of tokens (10^6)\", but the table reports 81M to 406M tokens per task—that's 10^7 to 10^8. Even the smallest task is two orders above the stated claim. The stress-test note is correct; this is a load-bearing flaw because it undermines the credibility of the headline comparison. It's fixable: the qualitative point still holds (a 10^8 gap vs. LLaMA's 10^12 is still dramatic), but the numbers as written are wrong.\n\nSecondary issues are the token definition (one second of one channel) and the lack of stated inclusion criteria for the dataset metadata. The token definition is arbitrary—LaBraM and other LBMs use different tokenization—so the precise comparison to NLP tokens is not well grounded. And without knowing how datasets were selected, the metadata table is hard to reproduce. These are fixable with clearer methodological statements.\n\nOverall, the paper is a competent position piece. The causal and data-scarcity arguments are sound qualitatively, and the roadmap (interventional data, better evaluation, causal reasoning in LBMs) is worth airing. The internal inconsistency is significant but it does not sink the paper; the logic does not collapse without it.\n\nI'd send this to peer review. The authors should correct the token-count statement, label the token calculation as an order-of-magnitude illustration, and document dataset selection criteria. After that, it could be a useful reference for the EEG-BCI community.\n\nWorth a serious referee, not a desk reject.","headline":"A useful causal roadmap for EEG-BCI generalization, but the data-scarcity numbers are internally inconsistent as written.","tokens_in":15574,"tokens_out":3005,"would_cite":false,"duration_ms":31094,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper argues that EEG-based brain-computer interfaces fail outside the lab because the data-generating process is causal—each new subject, device, stimulus, or environment shifts the distribution—and because EEG data is orders of…","keywords":["EEG","brain-computer interface","distribution shift","causality","data scarcity","foundation models","large brain models","generalization"],"falsifier":"A concrete test: take a state-of-the-art EEG foundation model pre-trained only on currently available public datasets and evaluate it on a held-out intervention—a new EEG device, a new subject population, and a new physical environment—with no fine-tuning. If it generalizes to within a small margin of same-distribution performance, the claim that data scarcity and causal distribution shifts are the binding constraints on real-world BCIs is falsified. A weaker falsifier would be to recompute the token counts using alternative EEG tokenizations, such as multi-channel patches, waveform tokens, or windows longer than one second, and show the gap to LLM corpora shrinks to within an order of magnitude.","tokens_in":14499,"feed_emoji":"🧠","tokens_out":9820,"duration_ms":91551,"temperature":0.7,"pith_summary":"The authors set out to explain why AI models for EEG-based brain-computer interfaces (BCIs) rarely survive contact with the real world. Their central thesis is that EEG data is generated by a causal process, so every change in stimulus, subject engagement, recording device, or environment shifts the data distribution and breaks the i.i.d. assumption most models are trained under. They give the field a map of BCI paradigms organized by whether the task is exogenous or endogenous and voluntary or involuntary, and they attach a specific distribution shift to each family. They also quantify a second challenge: counted as tokens, all public EEG datasets for six BCI tasks sum to between roughly 81 million and 406 million tokens per task, against 15 trillion tokens used to train a modern large language model. If this diagnosis is right, the route to deployable BCIs runs through collecting interventional data—varying one causal factor at a time—and through models that treat causality as part of the learning problem.","feed_headline":"Brain-computer AI is data-starved: millions of tokens, not trillions","feed_subtitle":"EEG signals shift across subjects, devices, and settings, and public datasets hold far fewer tokens than LLM corpora.","key_machinery":"The load-bearing object is the causal graph in Figure 1, which connects the unobserved brain activity $Z$, the measured EEG $X$, the stimulus $S$, the subject $P$, the device $D$, the engagement $I$, the environment $E$, and the decoded condition $Y$, with arrows colored by paradigm. It makes the distribution-shift argument legible by showing that a shift is not noise but an intervention: change the device, and the path from $Z$ to $X$ changes; change the stimulus, and the path from $S$ to $Z$ changes. The second piece of machinery is the token-count estimate $N_{\\text{tokens}}(i) = (T_i \\times C_i)/L$, with $L = 1$ second, which converts total recorded hours and average channel count into a number comparable to the token counts used in language modeling and exposes the data-scarcity claim.","core_discovery":"The paper's core claim is that the generalization problem in EEG-based BCIs is a data-generation problem, not just a modeling problem. It represents the generation of an EEG trace $X$ as the output of a causal graph whose nodes include the unobserved brain activity $Z$, the stimulus $S$, the subject $P$, the device $D$, the level of engagement $I$, and the environment $E$; the target variable $Y$ is the condition or message being decoded. Each of the four paradigm families—exogenous and voluntary, exogenous and involuntary, endogenous and voluntary, endogenous and involuntary—realizes a different causal chain, and any intervention on a node changes the distribution of the observed signal. The authors support this with a quantitative data census: using a token defined as one second of one channel, the public datasets for Brain-to-Image, Brain-to-Text, Emotion Recognition, Motor Imagery, Neurodegenerative Disease, and Seizure Prediction contain on the order of $10^8$ tokens, while LLaMA 3.1 405B used $10^{12}$ tokens. The conclusion argued is that current self-supervised foundation models for EEG are trained on too little, too narrow data, and that the field should shift toward interventional data collection and causality-aware architectures.","pith_inferences":["A direct extension is that the token-scarcity numbers likely understate the usable information: EEG channels are highly correlated, so raw sample tokens over-count redundant data, and the effective unique information per second is even smaller than the already-small token totals.","The causal-graph view predicts that domain-invariant representation learning, meaning features that remain unchanged across devices and subjects, is a more direct target than brute-force scaling, and it could be tested with synthetic data before collecting expensive interventional EEG.","A testable extension for the community is to build a standardized intervention benchmark, for example a common set of subjects recorded with three electrode types in both lab and home settings, and to report leave-one-setting-out accuracy as a routine metric.","If the authors are right that reliability is condition-specific, then the ethical discussion should include a duty to state the operating envelope of a BCI; a system validated in a lab is not equally reliable when used in an unvalidated setting."],"forward_implications":["If the causal diagnosis is right, strong results on a single EEG dataset should not be read as evidence of real-world readiness; they mostly reflect fitting one fixed distribution.","The field's benchmarks should shift toward leave-one-intervention-out testing, where a model is evaluated after training on all conditions except one, such as a new device, subject group, or environment.","Data augmentation and self-supervised pre-training can enrich the training distribution but cannot replace the diversity of real interventions, so they are complements to, not substitutes for, interventional data collection.","Because EEG token budgets are around $10^6$ to $10^8$ rather than $10^{12}$, scaling current EEG foundation models alone will not follow the language-model trajectory; progress depends on data-efficient and causality-aware learning.","Real-world BCI applications, such as assistive communication, wheelchair control, and drowsiness detection, will require systems that either are invariant to acquisition and subject shifts or can adapt with minimal calibration data."],"supporting_citations":[{"why":"Supplies the causal representation learning framework connecting interventions to distribution shifts and violations of the i.i.d. assumption.","marker":"Schölkopf et al., 2021"},{"why":"The earlier causal perspective on brainwave modeling that the paper extends; it establishes that causal connections between variables vary with the experimental setting.","marker":"Barmpas, Panagakis, Zoumpourlis, et al., 2024"},{"why":"The LaBraM large brain model that motivates the one-second, one-channel token definition used in the paper's token-count comparison.","marker":"Jiang, Zhao, & Lu, 2024"},{"why":"Grounds the call for interventional data and distributional robustness in cognitive neuroscience benchmarking.","marker":"Weichwald & Peters, 2021"},{"why":"Reference review of deep learning for EEG classification that frames the paper's contrast between early DL-based EEG analyses and current foundation models.","marker":"Craik, He, & Contreras-Vidal, 2019"}],"fun_headline_variants":["EEG AI is data-hungry: public sets offer a fraction of LLM scale","The real bottleneck for EEG BCI: not models, but data generation","Millions of EEG tokens won't cut it when you need trillions","EEG BCI generalization is a data problem, not just a model problem","To feed EEG AI, we need causal interventional data, not just more tokens"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The quantitative data-scarcity headline rests on the assumption that one second of one EEG channel is a token comparable to the word and subword tokens used to train large language models; if EEG models instead tokenize multi-channel patches or longer windows, the computed gap could shrink or grow, although the qualitative distribution-shift argument stands on its own.","fun_headline_variants_meta":{"raw":{"variants":["EEG AI is data-hungry: public sets offer a fraction of LLM scale","The real bottleneck for EEG BCI: not models, but data generation","Millions of EEG tokens won't cut it when you need trillions","EEG BCI generalization is a data problem, not just a model problem","To feed EEG AI, we need causal interventional data, not just more tokens"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001692,"raw_usage":{"total_tokens":6767,"prompt_tokens":1071,"completion_tokens":5696,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":687,"completion_tokens_details":{"reasoning_tokens":5592}},"tokens_in":687,"tokens_out":5696,"duration_ms":37952,"temperature":1.0,"reasoning_tokens":5592,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T23:43:49.100466+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete test: take a state-of-the-art EEG foundation model pre-trained only on currently available public datasets and evaluate it on a held-out intervention—a new EEG device, a new subject population, and a new physical environment—with no fine-tuning. If it generalizes to within a small margin of same-distribution performance, the claim that data scarcity and causal distribution shifts are the binding constraints on real-world BCIs is falsified. A weaker falsifier would be to recompute the token counts using alternative EEG tokenizations, such as multi-channel patches, waveform tokens, or windows longer than one second, and show the gap to LLM corpora shrinks to within an order of magnitude.","supporting_citations":[{"cited_title":", Zhao, L","cited_arxiv_id":null,"evidence_quote":"The LaBraM large brain model that motivates the one-second, one-channel token definition used in the paper's token-count comparison."},{"cited_title":"\\ Peters, J","cited_arxiv_id":null,"evidence_quote":"Grounds the call for interventional data and distributional robustness in cognitive neuroscience benchmarking."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Reference review of deep learning for EEG classification that frames the paper's contrast between early DL-based EEG analyses and current foundation models."}],"review_version":1}