The Identity Trap in EEG Foundation Models: A Diagnostic Audit
Pith reviewed 2026-06-28 02:58 UTC · model grok-4.3
The pith
EEG foundation models capture subject identity far more than clinical labels, and erasing that linear axis boosts decoding where labels vary within subjects.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The Identity Trap is universal: frozen subject-variance is 13-89x a random null in all 12 pairs and rises further under fine-tuning. This dominance forms a removable linear axis; erasing it improves label decoding where the label varies within subject (+6 to +12 pp in primary cells; +4 to +27 pp across external cohorts). Aperiodic 1/f acts as one subject carrier in two of the models, while fine-tuning amplifies label variance only when a literature-established cross-subject marker is present.
What carries the argument
FMScope, a frozen-representation protocol packaging variance decomposition, subject-axis erasure, aperiodic 1/f ablation, layer-wise label probing, and within-subject direction consistency, that separates subject-identity variance from clinical label variance in a 2x2 layout of subject-label relation and marker presence.
If this is right
- Erasing the subject-identity axis raises within-subject label decoding by 6-12 percentage points in the primary 2x2 cells.
- The same erasure yields 4-27 percentage point gains on external cohorts.
- Fine-tuning increases subject-variance dominance in every tested model-dataset pair.
- Removing aperiodic 1/f content drops subject-probe accuracy by 9-19 points on LaBraM and CBraMod but not on REVE.
Where Pith is reading between the lines
- The same diagnostic pattern may appear in other biosignal foundation models whenever subject metadata correlates with the target label.
- Explicit regularization against the subject axis during pre-training could reduce reliance on identity shortcuts.
- Current benchmarks that rely solely on subject-disjoint splits may systematically overestimate clinical utility of EEG models.
Load-bearing premise
The five FMScope diagnostics can isolate subject-identity variance from clinical label variance without residual confounding from dataset-specific correlations or unmeasured physiological factors.
What would settle it
No gain in label decoding accuracy after subject-axis erasure on datasets where the clinical label varies within subject would falsify the claim that the identity axis functions as a removable shortcut.
Figures
read the original abstract
Objective. EEG foundation models (FMs) report strong accuracy on clinical resting-state EEG. However, high accuracy under subject-disjoint cross-validation remains ambiguous: it can reflect a genuine clinical biomarker, or subject-identity features that correlate with the label. We name this the Identity Trap and ask whether it can be diagnosed at the representation level before fine-tuning. Approach. We propose FMScope, a frozen-representation protocol packaging five diagnostics: variance decomposition, subject-axis erasure, aperiodic 1/f ablation, layer-wise label probing, and within-subject direction consistency. We apply it to three pretrained FMs (LaBraM, CBraMod, REVE) across four datasets in a 2x2 layout: subject relation of label x presence of a consensus cross-subject EEG marker. Main results. (i) The Identity Trap is universal: frozen subject-variance is 13-89x a random null in 12/12 pairs, rising in all 12 under fine-tuning (+10 to +63 pp). This dominance is a removable linear axis: erasing it improves label decoding where the label varies within subject (+6 to +12 pp in primary cells; +4 to +27 pp across external cohorts). (ii) Aperiodic 1/f is one subject carrier: removing it drops the subject probe by 9-19 pp on LaBraM and CBraMod. REVE saturates subject identity without measurable aperiodic dependence. (iii) Fine-tuning amplifies label-variance only in cells with a literature-established cross-subject marker. Significance. The Identity Trap is a physically-grounded instance of shortcut learning: the preferred cue has a measurable physiological component, and subject-disjoint splitting alone cannot rule it out. FMScope separates gains reflecting a biological marker from those reflecting subject identity.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper claims that EEG foundation models exhibit an 'Identity Trap' in which subject-identity variance dominates clinical label variance in frozen representations. It introduces FMScope, a protocol of five diagnostics (variance decomposition, subject-axis erasure, aperiodic 1/f ablation, layer-wise probing, within-subject consistency) applied to LaBraM, CBraMod, and REVE across four datasets in a 2x2 layout (subject relation of label × presence of consensus cross-subject marker). Key findings: subject variance is 13-89× a random null in all 12 model-dataset pairs and increases under fine-tuning; erasing the subject axis improves within-subject label decoding by 6-12 pp (primary) and 4-27 pp (external cohorts); aperiodic 1/f carries subject identity in two models; fine-tuning amplifies label variance only when a literature marker exists. The work positions the trap as a physiologically grounded shortcut and FMScope as a pre-fine-tuning diagnostic.
Significance. If the diagnostics cleanly separate identity from label variance, the audit would establish a concrete, measurable instance of shortcut learning in EEG FMs and supply a practical mitigation (linear erasure) that improves generalization where labels vary within subject. The paper's strengths include direct empirical measurements on pretrained models with no circularity (all quantities are observed, not derived from fitted parameters) and consistent patterns across 12 pairs plus external cohorts. The result would matter for any downstream clinical use of these models.
major comments (2)
- [Abstract] Abstract (and the 2×2 layout + variance decomposition steps): the central attribution of dominance (13-89× subject variance) and the erasure benefit (+6-12 pp) to a removable identity axis assumes the decomposition isolates subject-identity without residual confounding from dataset-specific correlations, demographics, site effects, or label-correlated physiological states. No explicit orthogonality check against label-predictive covariates is described, so the reported improvements could partly reflect removal of those factors rather than identity alone.
- [Abstract] Abstract (main results (i)): the universality claim rests on 12/12 pairs showing subject-variance dominance and consistent improvement after erasure. Without the full methods, data exclusion criteria, or statistical controls referenced in the reader's soundness assessment, it is not yet possible to verify that the 2×2 design and five diagnostics rule out the confounding scenario raised in the stress-test note.
Simulated Author's Rebuttal
We thank the referee for the constructive comments on potential confounding in the variance decomposition and universality claims. We respond point by point below, agreeing where additional checks are warranted and clarifying the controls already present in the full manuscript.
read point-by-point responses
-
Referee: [Abstract] Abstract (and the 2×2 layout + variance decomposition steps): the central attribution of dominance (13-89× subject variance) and the erasure benefit (+6-12 pp) to a removable identity axis assumes the decomposition isolates subject-identity without residual confounding from dataset-specific correlations, demographics, site effects, or label-correlated physiological states. No explicit orthogonality check against label-predictive covariates is described, so the reported improvements could partly reflect removal of those factors rather than identity alone.
Authors: We agree this is a valid concern and that an explicit orthogonality check would strengthen the isolation claim. The 2×2 layout was chosen precisely to vary label-subject relations and the presence of consensus cross-subject markers, providing indirect control over label-correlated states. However, to directly address residual confounding from demographics and site effects, we will add a post-hoc correlation analysis between the extracted subject axis and available covariates (age, sex, recording site) in the revised manuscript, reporting these as an extension to the variance decomposition diagnostic. revision: yes
-
Referee: [Abstract] Abstract (main results (i)): the universality claim rests on 12/12 pairs showing subject-variance dominance and consistent improvement after erasure. Without the full methods, data exclusion criteria, or statistical controls referenced in the reader's soundness assessment, it is not yet possible to verify that the 2×2 design and five diagnostics rule out the confounding scenario raised in the stress-test note.
Authors: The full manuscript (Section 3 and Appendix) specifies the methods, including subject-disjoint splits, data exclusion (minimum 20 trials per subject-label combination, removal of sessions with >30% artifact), and statistical controls (bootstrap CIs on variance ratios, permutation tests on erasure gains, and layer-wise probing). The 2×2 explicitly tests the identity trap under conditions where within-subject label variation is present or absent and where literature markers exist or do not. These elements, together with the five diagnostics applied uniformly to all 12 pairs, are designed to surface confounding if present; the consistent 13-89× dominance and selective erasure benefits align with the physiological grounding rather than dataset artifacts. We are happy to expand any specific control referenced in the soundness assessment if clarified. revision: no
Circularity Check
No significant circularity: claims rest on direct empirical measurements
full rationale
The paper's core results (subject-variance ratios of 13-89x, fine-tuning increases, erasure improvements of +6 to +12 pp) are presented as direct outputs of applying the five FMScope diagnostics to pretrained models across datasets. The abstract and main results describe variance decomposition and axis erasure as measurement protocols in a 2x2 layout, with no equations shown that define the target quantities in terms of the same fitted parameters or reduce improvements to inputs by construction. No self-citation chains, uniqueness theorems, or ansatzes are invoked as load-bearing steps. The derivation chain consists of empirical application rather than self-referential fitting or renaming.
Axiom & Free-Parameter Ledger
axioms (1)
- standard math Variance decomposition and linear probing classifiers rest on standard statistical assumptions of linearity and independence of components.
Reference graph
Works this paper leans on
-
[1]
Bruno Aristimunha, Dung Truong, Pierre Guetschel, Seyed Yahya Shirazi, Isabelle Guyon, Alexandre R. Franco, Michael P. Milham, Aviv Dotan, Scott Makeig, Alexandre Gramfort, Jean-Remi King, Marie- Constance Corsi, Pedro A. Valdés-Sosa, Amit Majumdar, Alan Evans, Terrence J. Sejnowski, Oren Shriki, Sylvain Chevallier, and Arnaud Delorme. EEG foundation chal...
-
[2]
Hubert Banville, Omar Chehab, Aapo Hyvärinen, Denis-Alexander Engemann, and Alexandre Gramfort
doi: 10.1002/alz.12311. Hubert Banville, Omar Chehab, Aapo Hyvärinen, Denis-Alexander Engemann, and Alexandre Gramfort. Uncovering the structure of clinical EEG signals with self-supervised learning.Journal of Neural Engineering, 18(4):046020,
-
[3]
doi: 10.1088/1741-2552/abca18. Nora Belrose, David Schneider-Joseph, Shauli Ravfogel, Ryan Cotterell, Edward Raff, and Stella Biderman. LEACE: Perfect linear concept erasure in closed form. InAdvances in Neural Information Processing Systems (NeurIPS),
-
[4]
Patrizio Campisi and Daria La Rocca
doi: 10.3389/fnins.2024.1373515. Patrizio Campisi and Daria La Rocca. Brain waves for automatic biometric-based user recognition.IEEE Trans. Information Forensics and Security, 9(5):782–800,
-
[5]
Rita Ceponiene, Marissa Westerfield, Mara Torki, and Jeanne Townsend
doi: 10.1109/TIFS.2014.2308640. Rita Ceponiene, Marissa Westerfield, Mara Torki, and Jeanne Townsend. Modality-specificity of sensory aging in vision and audition: Evidence from event-related potentials.Brain Research, 1215:53–68,
-
[6]
Matteo Demuru and Matteo Fraschini
doi: 10.1016/j.brainres.2008.02.010. Matteo Demuru and Matteo Fraschini. EEG fingerprinting: Subject-specific signature based on the aperiodic component of power spectrum.Computers in Biology and Medicine, 120:103748,
-
[7]
doi: 10.1016/j. compbiomed.2020.103748. Thomas Donoghue, Matar Haller, Erik J. Peterson, Paroma Varma, Priyadarshini Sebastian, Richard Gao, Torben Noto, Antonio H. Lara, Joni D. Wallis, Robert T. Knight, Avgusta Shestyuk, and Bradley Voytek. Parameterizing neural power spectra into periodic and aperiodic components.Nature Neuroscience, 23: 1655–1665,
work page doi:10.1016/j 2020
-
[8]
Preprint 26 Yassine El Ouahidi, Jonathan Lys, Philipp Thölke, Nicolas Farrugia, Bastien Pasdeloup, Vincent Gripon, Karim Jerbi, and Giulia Lioi. REVE: A foundation model for EEG – adapting to any setup with large-scale pretraining on 25,000 subjects.arXiv preprint arXiv:2510.21585,
-
[9]
doi: 10.1038/s42256-020-00257-z. Alan Gevins, Michael E. Smith, Linda McEvoy, and Daphne Yu. High-resolution EEG mapping of cortical activation related to working memory: effects of task difficulty, type of processing, and practice.Cerebral Cortex, 7(4):374–385,
-
[10]
doi: 10.1093/cercor/7.4.374. Rajdeep Ghosh, Nabamita Deb, Kakuli Sengupta, Achyut Phukan, Nimisha Choudhury, Sayan Kashyap, Souvik Phadikar, Rachita Saha, Pranesh Das, Nidul Sinha, and Paramita Dutta. SAM 40: Dataset of 40 subject EEG recordings to monitor the induced-stress while performing stroop color-word test, arithmetic task, and mirror image recogn...
-
[11]
Wei-Bang Jiang, Liming Zhao, and Bao-Liang Lu
doi: 10.1016/j.dib.2021.107772. Wei-Bang Jiang, Liming Zhao, and Bao-Liang Lu. LaBraM: Large brain model for learning generic repre- sentations with tremendous EEG data in BCI. InInternational Conference on Learning Representations,
-
[12]
Timon Klein, Piotr Minakowski, Sebastian Sager, and Steffen Schotthöfer
arXiv:2512.08959. Timon Klein, Piotr Minakowski, Sebastian Sager, and Steffen Schotthöfer. Mitigating subject dependency in EEG decoding with subject-specific low-rank adapters.arXiv preprint arXiv:2510.08059,
-
[13]
Oleksii Komarov, Li-Wei Ko, and Tzyy-Ping Jung
doi: 10.7554/eLife.10989. Oleksii Komarov, Li-Wei Ko, and Tzyy-Ping Jung. Associations among emotional state, sleep quality, and resting-state EEG spectra: A longitudinal study in graduate students.IEEE Transactions on Neural Systems and Rehabilitation Engineering, 28(4):795–804,
-
[14]
Preprint 27 Demetres Kostas, Stéphane Aroca-Ouellette, and Frank Rudzicz
doi: 10.1016/j.nbd.2023.106380. Preprint 27 Demetres Kostas, Stéphane Aroca-Ouellette, and Frank Rudzicz. BENDR: Using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG data.Frontiers in Human Neuroscience, 15:653659,
-
[15]
doi: 10.3389/fnhum.2021.653659
doi: 10.3389/fnhum.2021.653659. Gayal Kuruppu, Neeraj Wagh, Vaclav Kremen, Sandipan Pati, Gregory Worrell, and Yogatheesan Varathara- jah. EEG foundation models: A critical review of current progress and future directions.arXiv preprint arXiv:2507.11783,
-
[16]
A review of classification algorithms for EEG-based brain–computer interfaces: A 10 year update
doi: 10.1088/1741-2552/aab2f2. Sydney H. Lovibond and Peter F. Lovibond. The structure of negative emotional states: Comparison of the Depression Anxiety Stress Scales (DASS) with the Beck depression and anxiety inventories.Behaviour Research and Therapy, 33(3):335–343,
-
[17]
Jingying Ma, Feng Wu, Yucheng Xing, Qika Lin, Tianyu Liu, Chenyu Liu, Ziyu Jia, and Mengling Feng. SCOPE: Structured prototype-guided adaptation for EEG foundation models with limited labels.arXiv preprint arXiv:2602.17251,
-
[18]
Sébastien Marcel and José del R
doi: 10.1038/s42003-022-03185-3. Sébastien Marcel and José del R. Millán. Person authentication using brainwaves (EEG) and maximum a posteriori model adaptation.IEEE Trans. Pattern Anal. Mach. Intell.,
-
[19]
doi: 10.1111/psyp.12965. Andrew M. Saxe, James L. McClelland, and Surya Ganguli. A mathematical theory of semantic development in deep neural networks.Proceedings of the National Academy of Sciences, 116(23):11537–11546,
-
[20]
doi: 10.1073/pnas.1820226116. Gerwin Schalk, Dennis J. McFarland, Thilo Hinterberger, Niels Birbaumer, and Jonathan R. Wolpaw. BCI2000: A general-purpose brain-computer interface (BCI) system.IEEE Transactions on Biomedical Engineering, 51(6):1034–1043,
-
[21]
doi: 10.1109/TBME.2004.827072. Robin Tibor Schirrmeister, Jost Tobias Springenberg, Lukas Dominique Josef Fiederer, Martin Glasstetter, Katharina Eggensperger, Michael Tangermann, Frank Hutter, Wolfram Burgard, and Tonio Ball. Deep learning with convolutional neural networks for EEG decoding and visualization.Human Brain Mapping, 38(11):5391–5420,
-
[22]
Preprint 28 Fanqi Shen, Enhong Yang, Jiahe Li, Junru Hong, Xiaoran Pan, Zhizhang Yuan, Meng Li, and Yang Yang. Brain4FMs: A benchmark of foundation models for electrical brain signal.arXiv preprint arXiv:2602.11558,
-
[23]
What do EEG foundation models capture from human brain signals?arXiv preprint arXiv:2605.11410,
Ling Tang, Qian Chen, Jilin Mei, Houshi Xu, Quanshi Zhang, Jing Shao, Na Zou, Xia Hu, and Dongrui Liu. What do EEG foundation models capture from human brain signals?arXiv preprint arXiv:2605.11410,
-
[24]
doi: 10.1016/j.nicl.2017.07.006. Hanneke van Dijk, Guido van Wingen, Damiaan Denys, Sebastian Olbrich, Rosalinde van Ruth, and Martijn Arns. The two decades brainclinics research archive for insights in neurophysiology (TDBRAIN) database. Scientific Data, 9(1):333,
-
[25]
doi: 10.1038/s41597-022-01409-z. Jiquan Wang et al. CBraMod: A criss-cross brain foundation model for EEG decoding.ICLR, 2025a. Siwen Wang, Shitou Zhang, Wan-Lin Chen, Dung Truong, and Tzyy-Ping Jung. From theory to application: Fine-tuning large EEG model with real-world stress data.arXiv preprint arXiv:2505.23042, 2025b. Yulin Wang, Wei Duan, Debo Dong,...
-
[26]
doi: 10.1038/ s41597-022-01607-9. Jiamin Wu, Zichen Ren, Junyu Wang, Pengyu Zhu, Yonghao Song, Mianxin Liu, Qihao Zheng, Lei Bai, Wanli Ouyang, and Chunfeng Song. AdaBrain-Bench: Benchmarking brain foundation models for brain-computer interface applications.arXiv preprint arXiv:2507.09882,
-
[27]
Wei Xiong, Jiangtong Li, Jie Li, Kun Zhu, and Changjun Jiang
doi: 10.1038/s41597-024-03268-2. Wei Xiong, Jiangtong Li, Jie Li, Kun Zhu, and Changjun Jiang. EEG-FM-Bench: A comprehensive benchmark for the systematic evaluation of EEG foundation models.arXiv preprint arXiv:2508.17742,
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.