pith. sign in

arxiv: 2606.06647 · v1 · pith:JI42FZSMnew · submitted 2026-06-04 · 💻 cs.LG · q-bio.NC

The Identity Trap in EEG Foundation Models: A Diagnostic Audit

Pith reviewed 2026-06-28 02:58 UTC · model grok-4.3

classification 💻 cs.LG q-bio.NC
keywords EEG foundation modelsIdentity Trapshortcut learningsubject identityFMScopevariance decompositionclinical biomarkersresting-state EEG
0
0 comments X

The pith

EEG foundation models capture subject identity far more than clinical labels, and erasing that linear axis boosts decoding where labels vary within subjects.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper establishes that EEG foundation models suffer from an Identity Trap in which subject-specific features dominate representations even under subject-disjoint evaluation. FMScope, a set of five frozen-representation diagnostics, reveals this pattern across three models and four datasets by decomposing variance and testing axis erasure. A sympathetic reader cares because high reported accuracies may reflect physiological subject cues rather than genuine biomarkers, and subject-disjoint splits alone do not rule this out. The work shows the identity component is removable and that its removal improves label decoding precisely in the cells where labels vary within subject.

Core claim

The Identity Trap is universal: frozen subject-variance is 13-89x a random null in all 12 pairs and rises further under fine-tuning. This dominance forms a removable linear axis; erasing it improves label decoding where the label varies within subject (+6 to +12 pp in primary cells; +4 to +27 pp across external cohorts). Aperiodic 1/f acts as one subject carrier in two of the models, while fine-tuning amplifies label variance only when a literature-established cross-subject marker is present.

What carries the argument

FMScope, a frozen-representation protocol packaging variance decomposition, subject-axis erasure, aperiodic 1/f ablation, layer-wise label probing, and within-subject direction consistency, that separates subject-identity variance from clinical label variance in a 2x2 layout of subject-label relation and marker presence.

If this is right

  • Erasing the subject-identity axis raises within-subject label decoding by 6-12 percentage points in the primary 2x2 cells.
  • The same erasure yields 4-27 percentage point gains on external cohorts.
  • Fine-tuning increases subject-variance dominance in every tested model-dataset pair.
  • Removing aperiodic 1/f content drops subject-probe accuracy by 9-19 points on LaBraM and CBraMod but not on REVE.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same diagnostic pattern may appear in other biosignal foundation models whenever subject metadata correlates with the target label.
  • Explicit regularization against the subject axis during pre-training could reduce reliance on identity shortcuts.
  • Current benchmarks that rely solely on subject-disjoint splits may systematically overestimate clinical utility of EEG models.

Load-bearing premise

The five FMScope diagnostics can isolate subject-identity variance from clinical label variance without residual confounding from dataset-specific correlations or unmeasured physiological factors.

What would settle it

No gain in label decoding accuracy after subject-axis erasure on datasets where the clinical label varies within subject would falsify the claim that the identity axis functions as a removable shortcut.

Figures

Figures reproduced from arXiv: 2606.06647 by Jun-You Lin, Tzyy-Ping Jung, Ying Choon Wu.

Figure 1
Figure 1. Figure 1: FMScope overview. Five frozen-representation diagnostics applied to embeddings from a pretrained transformer EEG-FM. Two of the five establish the Identity Trap: variance decomposition and subject-axis erasure (LEACE). The other three characterize its origin and structure: aperiodic input ablation, layer-wise subject/label probe, and within-subject direction consistency. Center: subject identity forms the … view at source ↗
Figure 2
Figure 2. Figure 2: FMScope diagnostic pipeline. The framework evaluates frozen representations from EEG foundation models across three sequential phases. Phase I establishes the existence of the Identity Trap by quantifying subject-variance dominance and testing its linear removability via least-squares concept erasure (LEACE). Phase II characterizes the underlying mechanisms using three tools: localizing the depth of subjec… view at source ↗
Figure 3
Figure 3. Figure 3: Variance decomposition across the four cells. Window-level subject and label fractions for frozen and fine-tuned features. Stacked bars: subject (lower) + label (upper); gap to 100% is residual. Dashed red line marks the matched random-Gaussian null fsubj (mean over 20 seeds; per-cell numeric callout in each panel). Frozen subject fraction exceeds the null by 13–89× across 12 (cell, FM) pairs; under fine-t… view at source ↗
Figure 4
Figure 4. Figure 4: Layer-wise subject and label probes. Rows show the subject relation of the label (within-subject paired on top, trait on the bottom); columns show the consensus axis (consensus on the left, no-consensus on the right). Each panel shows two probes: the temporal-block subject-ID probe (red) and the canonical recording-level label probe (blue). Lines are the mean across the three FMs (LaBraM, CBraMod, REVE); s… view at source ↗
Figure 5
Figure 5. Figure 5: Aperiodic and periodic ablation of the input, frozen and intervention-FT. (a) Repre￾sentative log-log power spectrum per cell with FOOOF decomposition: black solid, measured PSD; orange dashed, 1/f aperiodic fit; blue shading, periodic peaks (PSD minus aperiodic). The aperiodic fit defines the input ablation (Sec. 3.7.3). (b) ∆ probe BA under FOOOF ablation; circles mark −aperiodic, squares mark −periodic,… view at source ↗
Figure 6
Figure 6. Figure 6: Within-subject direction and SNR (frozen vs. FT). For each subject the contrast vector vi = µi,1 − µi,0 is formed in the FM’s full feature space; the group consensus is vc = vi/∥vi∥. Filled gray = EEGMAT, outlined black = SleepDep. (a) Polar half-circle rose (frozen), one panel per FM: θi = arccos⟨vi , vc⟩; 0 ◦ aligns, 90◦ is the high-dimensional isotropic null. (b) Group-level c¯ (Eq. 6) across 3 FMs × 2 … view at source ↗
read the original abstract

Objective. EEG foundation models (FMs) report strong accuracy on clinical resting-state EEG. However, high accuracy under subject-disjoint cross-validation remains ambiguous: it can reflect a genuine clinical biomarker, or subject-identity features that correlate with the label. We name this the Identity Trap and ask whether it can be diagnosed at the representation level before fine-tuning. Approach. We propose FMScope, a frozen-representation protocol packaging five diagnostics: variance decomposition, subject-axis erasure, aperiodic 1/f ablation, layer-wise label probing, and within-subject direction consistency. We apply it to three pretrained FMs (LaBraM, CBraMod, REVE) across four datasets in a 2x2 layout: subject relation of label x presence of a consensus cross-subject EEG marker. Main results. (i) The Identity Trap is universal: frozen subject-variance is 13-89x a random null in 12/12 pairs, rising in all 12 under fine-tuning (+10 to +63 pp). This dominance is a removable linear axis: erasing it improves label decoding where the label varies within subject (+6 to +12 pp in primary cells; +4 to +27 pp across external cohorts). (ii) Aperiodic 1/f is one subject carrier: removing it drops the subject probe by 9-19 pp on LaBraM and CBraMod. REVE saturates subject identity without measurable aperiodic dependence. (iii) Fine-tuning amplifies label-variance only in cells with a literature-established cross-subject marker. Significance. The Identity Trap is a physically-grounded instance of shortcut learning: the preferred cue has a measurable physiological component, and subject-disjoint splitting alone cannot rule it out. FMScope separates gains reflecting a biological marker from those reflecting subject identity.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 0 minor

Summary. The paper claims that EEG foundation models exhibit an 'Identity Trap' in which subject-identity variance dominates clinical label variance in frozen representations. It introduces FMScope, a protocol of five diagnostics (variance decomposition, subject-axis erasure, aperiodic 1/f ablation, layer-wise probing, within-subject consistency) applied to LaBraM, CBraMod, and REVE across four datasets in a 2x2 layout (subject relation of label × presence of consensus cross-subject marker). Key findings: subject variance is 13-89× a random null in all 12 model-dataset pairs and increases under fine-tuning; erasing the subject axis improves within-subject label decoding by 6-12 pp (primary) and 4-27 pp (external cohorts); aperiodic 1/f carries subject identity in two models; fine-tuning amplifies label variance only when a literature marker exists. The work positions the trap as a physiologically grounded shortcut and FMScope as a pre-fine-tuning diagnostic.

Significance. If the diagnostics cleanly separate identity from label variance, the audit would establish a concrete, measurable instance of shortcut learning in EEG FMs and supply a practical mitigation (linear erasure) that improves generalization where labels vary within subject. The paper's strengths include direct empirical measurements on pretrained models with no circularity (all quantities are observed, not derived from fitted parameters) and consistent patterns across 12 pairs plus external cohorts. The result would matter for any downstream clinical use of these models.

major comments (2)
  1. [Abstract] Abstract (and the 2×2 layout + variance decomposition steps): the central attribution of dominance (13-89× subject variance) and the erasure benefit (+6-12 pp) to a removable identity axis assumes the decomposition isolates subject-identity without residual confounding from dataset-specific correlations, demographics, site effects, or label-correlated physiological states. No explicit orthogonality check against label-predictive covariates is described, so the reported improvements could partly reflect removal of those factors rather than identity alone.
  2. [Abstract] Abstract (main results (i)): the universality claim rests on 12/12 pairs showing subject-variance dominance and consistent improvement after erasure. Without the full methods, data exclusion criteria, or statistical controls referenced in the reader's soundness assessment, it is not yet possible to verify that the 2×2 design and five diagnostics rule out the confounding scenario raised in the stress-test note.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments on potential confounding in the variance decomposition and universality claims. We respond point by point below, agreeing where additional checks are warranted and clarifying the controls already present in the full manuscript.

read point-by-point responses
  1. Referee: [Abstract] Abstract (and the 2×2 layout + variance decomposition steps): the central attribution of dominance (13-89× subject variance) and the erasure benefit (+6-12 pp) to a removable identity axis assumes the decomposition isolates subject-identity without residual confounding from dataset-specific correlations, demographics, site effects, or label-correlated physiological states. No explicit orthogonality check against label-predictive covariates is described, so the reported improvements could partly reflect removal of those factors rather than identity alone.

    Authors: We agree this is a valid concern and that an explicit orthogonality check would strengthen the isolation claim. The 2×2 layout was chosen precisely to vary label-subject relations and the presence of consensus cross-subject markers, providing indirect control over label-correlated states. However, to directly address residual confounding from demographics and site effects, we will add a post-hoc correlation analysis between the extracted subject axis and available covariates (age, sex, recording site) in the revised manuscript, reporting these as an extension to the variance decomposition diagnostic. revision: yes

  2. Referee: [Abstract] Abstract (main results (i)): the universality claim rests on 12/12 pairs showing subject-variance dominance and consistent improvement after erasure. Without the full methods, data exclusion criteria, or statistical controls referenced in the reader's soundness assessment, it is not yet possible to verify that the 2×2 design and five diagnostics rule out the confounding scenario raised in the stress-test note.

    Authors: The full manuscript (Section 3 and Appendix) specifies the methods, including subject-disjoint splits, data exclusion (minimum 20 trials per subject-label combination, removal of sessions with >30% artifact), and statistical controls (bootstrap CIs on variance ratios, permutation tests on erasure gains, and layer-wise probing). The 2×2 explicitly tests the identity trap under conditions where within-subject label variation is present or absent and where literature markers exist or do not. These elements, together with the five diagnostics applied uniformly to all 12 pairs, are designed to surface confounding if present; the consistent 13-89× dominance and selective erasure benefits align with the physiological grounding rather than dataset artifacts. We are happy to expand any specific control referenced in the soundness assessment if clarified. revision: no

Circularity Check

0 steps flagged

No significant circularity: claims rest on direct empirical measurements

full rationale

The paper's core results (subject-variance ratios of 13-89x, fine-tuning increases, erasure improvements of +6 to +12 pp) are presented as direct outputs of applying the five FMScope diagnostics to pretrained models across datasets. The abstract and main results describe variance decomposition and axis erasure as measurement protocols in a 2x2 layout, with no equations shown that define the target quantities in terms of the same fitted parameters or reduce improvements to inputs by construction. No self-citation chains, uniqueness theorems, or ansatzes are invoked as load-bearing steps. The derivation chain consists of empirical application rather than self-referential fitting or renaming.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The work is an empirical diagnostic audit that introduces no new mathematical derivations, fitted parameters, or postulated entities beyond standard statistical assumptions used in variance decomposition and linear probing.

axioms (1)
  • standard math Variance decomposition and linear probing classifiers rest on standard statistical assumptions of linearity and independence of components.
    Invoked when attributing variance to subject identity versus label.

pith-pipeline@v0.9.1-grok · 5875 in / 1321 out tokens · 58215 ms · 2026-06-28T02:58:14.323866+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

27 extracted references · 20 canonical work pages

  1. [1]

    Franco, Michael P

    Bruno Aristimunha, Dung Truong, Pierre Guetschel, Seyed Yahya Shirazi, Isabelle Guyon, Alexandre R. Franco, Michael P. Milham, Aviv Dotan, Scott Makeig, Alexandre Gramfort, Jean-Remi King, Marie- Constance Corsi, Pedro A. Valdés-Sosa, Amit Majumdar, Alan Evans, Terrence J. Sejnowski, Oren Shriki, Sylvain Chevallier, and Arnaud Delorme. EEG foundation chal...

  2. [2]

    Hubert Banville, Omar Chehab, Aapo Hyvärinen, Denis-Alexander Engemann, and Alexandre Gramfort

    doi: 10.1002/alz.12311. Hubert Banville, Omar Chehab, Aapo Hyvärinen, Denis-Alexander Engemann, and Alexandre Gramfort. Uncovering the structure of clinical EEG signals with self-supervised learning.Journal of Neural Engineering, 18(4):046020,

  3. [3]

    Nora Belrose, David Schneider-Joseph, Shauli Ravfogel, Ryan Cotterell, Edward Raff, and Stella Biderman

    doi: 10.1088/1741-2552/abca18. Nora Belrose, David Schneider-Joseph, Shauli Ravfogel, Ryan Cotterell, Edward Raff, and Stella Biderman. LEACE: Perfect linear concept erasure in closed form. InAdvances in Neural Information Processing Systems (NeurIPS),

  4. [4]

    Patrizio Campisi and Daria La Rocca

    doi: 10.3389/fnins.2024.1373515. Patrizio Campisi and Daria La Rocca. Brain waves for automatic biometric-based user recognition.IEEE Trans. Information Forensics and Security, 9(5):782–800,

  5. [5]

    Rita Ceponiene, Marissa Westerfield, Mara Torki, and Jeanne Townsend

    doi: 10.1109/TIFS.2014.2308640. Rita Ceponiene, Marissa Westerfield, Mara Torki, and Jeanne Townsend. Modality-specificity of sensory aging in vision and audition: Evidence from event-related potentials.Brain Research, 1215:53–68,

  6. [6]

    Matteo Demuru and Matteo Fraschini

    doi: 10.1016/j.brainres.2008.02.010. Matteo Demuru and Matteo Fraschini. EEG fingerprinting: Subject-specific signature based on the aperiodic component of power spectrum.Computers in Biology and Medicine, 120:103748,

  7. [7]

    Frisch, Ian P

    doi: 10.1016/j. compbiomed.2020.103748. Thomas Donoghue, Matar Haller, Erik J. Peterson, Paroma Varma, Priyadarshini Sebastian, Richard Gao, Torben Noto, Antonio H. Lara, Joni D. Wallis, Robert T. Knight, Avgusta Shestyuk, and Bradley Voytek. Parameterizing neural power spectra into periodic and aperiodic components.Nature Neuroscience, 23: 1655–1665,

  8. [8]

    REVE: A foundation model for EEG – adapting to any setup with large-scale pretraining on 25,000 subjects.arXiv preprint arXiv:2510.21585,

    Preprint 26 Yassine El Ouahidi, Jonathan Lys, Philipp Thölke, Nicolas Farrugia, Bastien Pasdeloup, Vincent Gripon, Karim Jerbi, and Giulia Lioi. REVE: A foundation model for EEG – adapting to any setup with large-scale pretraining on 25,000 subjects.arXiv preprint arXiv:2510.21585,

  9. [9]

    Wichmann

    doi: 10.1038/s42256-020-00257-z. Alan Gevins, Michael E. Smith, Linda McEvoy, and Daphne Yu. High-resolution EEG mapping of cortical activation related to working memory: effects of task difficulty, type of processing, and practice.Cerebral Cortex, 7(4):374–385,

  10. [10]

    Rajdeep Ghosh, Nabamita Deb, Kakuli Sengupta, Achyut Phukan, Nimisha Choudhury, Sayan Kashyap, Souvik Phadikar, Rachita Saha, Pranesh Das, Nidul Sinha, and Paramita Dutta

    doi: 10.1093/cercor/7.4.374. Rajdeep Ghosh, Nabamita Deb, Kakuli Sengupta, Achyut Phukan, Nimisha Choudhury, Sayan Kashyap, Souvik Phadikar, Rachita Saha, Pranesh Das, Nidul Sinha, and Paramita Dutta. SAM 40: Dataset of 40 subject EEG recordings to monitor the induced-stress while performing stroop color-word test, arithmetic task, and mirror image recogn...

  11. [11]

    Wei-Bang Jiang, Liming Zhao, and Bao-Liang Lu

    doi: 10.1016/j.dib.2021.107772. Wei-Bang Jiang, Liming Zhao, and Bao-Liang Lu. LaBraM: Large brain model for learning generic repre- sentations with tremendous EEG data in BCI. InInternational Conference on Learning Representations,

  12. [12]

    Timon Klein, Piotr Minakowski, Sebastian Sager, and Steffen Schotthöfer

    arXiv:2512.08959. Timon Klein, Piotr Minakowski, Sebastian Sager, and Steffen Schotthöfer. Mitigating subject dependency in EEG decoding with subject-specific low-rank adapters.arXiv preprint arXiv:2510.08059,

  13. [13]

    Oleksii Komarov, Li-Wei Ko, and Tzyy-Ping Jung

    doi: 10.7554/eLife.10989. Oleksii Komarov, Li-Wei Ko, and Tzyy-Ping Jung. Associations among emotional state, sleep quality, and resting-state EEG spectra: A longitudinal study in graduate students.IEEE Transactions on Neural Systems and Rehabilitation Engineering, 28(4):795–804,

  14. [14]

    Preprint 27 Demetres Kostas, Stéphane Aroca-Ouellette, and Frank Rudzicz

    doi: 10.1016/j.nbd.2023.106380. Preprint 27 Demetres Kostas, Stéphane Aroca-Ouellette, and Frank Rudzicz. BENDR: Using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG data.Frontiers in Human Neuroscience, 15:653659,

  15. [15]

    doi: 10.3389/fnhum.2021.653659

    doi: 10.3389/fnhum.2021.653659. Gayal Kuruppu, Neeraj Wagh, Vaclav Kremen, Sandipan Pati, Gregory Worrell, and Yogatheesan Varathara- jah. EEG foundation models: A critical review of current progress and future directions.arXiv preprint arXiv:2507.11783,

  16. [16]

    A review of classification algorithms for EEG-based brain–computer interfaces: A 10 year update

    doi: 10.1088/1741-2552/aab2f2. Sydney H. Lovibond and Peter F. Lovibond. The structure of negative emotional states: Comparison of the Depression Anxiety Stress Scales (DASS) with the Beck depression and anxiety inventories.Behaviour Research and Therapy, 33(3):335–343,

  17. [17]

    SCOPE: Structured prototype-guided adaptation for EEG foundation models with limited labels.arXiv preprint arXiv:2602.17251,

    Jingying Ma, Feng Wu, Yucheng Xing, Qika Lin, Tianyu Liu, Chenyu Liu, Ziyu Jia, and Mengling Feng. SCOPE: Structured prototype-guided adaptation for EEG foundation models with limited labels.arXiv preprint arXiv:2602.17251,

  18. [18]

    Sébastien Marcel and José del R

    doi: 10.1038/s42003-022-03185-3. Sébastien Marcel and José del R. Millán. Person authentication using brainwaves (EEG) and maximum a posteriori model adaptation.IEEE Trans. Pattern Anal. Mach. Intell.,

  19. [19]

    Andrew M

    doi: 10.1111/psyp.12965. Andrew M. Saxe, James L. McClelland, and Surya Ganguli. A mathematical theory of semantic development in deep neural networks.Proceedings of the National Academy of Sciences, 116(23):11537–11546,

  20. [20]

    Gerwin Schalk, Dennis J

    doi: 10.1073/pnas.1820226116. Gerwin Schalk, Dennis J. McFarland, Thilo Hinterberger, Niels Birbaumer, and Jonathan R. Wolpaw. BCI2000: A general-purpose brain-computer interface (BCI) system.IEEE Transactions on Biomedical Engineering, 51(6):1034–1043,

  21. [21]

    doi: 10.1109/TBME.2004.827072. Robin Tibor Schirrmeister, Jost Tobias Springenberg, Lukas Dominique Josef Fiederer, Martin Glasstetter, Katharina Eggensperger, Michael Tangermann, Frank Hutter, Wolfram Burgard, and Tonio Ball. Deep learning with convolutional neural networks for EEG decoding and visualization.Human Brain Mapping, 38(11):5391–5420,

  22. [22]

    Brain4FMs: A benchmark of foundation models for electrical brain signal.arXiv preprint arXiv:2602.11558,

    Preprint 28 Fanqi Shen, Enhong Yang, Jiahe Li, Junru Hong, Xiaoran Pan, Zhizhang Yuan, Meng Li, and Yang Yang. Brain4FMs: A benchmark of foundation models for electrical brain signal.arXiv preprint arXiv:2602.11558,

  23. [23]

    What do EEG foundation models capture from human brain signals?arXiv preprint arXiv:2605.11410,

    Ling Tang, Qian Chen, Jilin Mei, Houshi Xu, Quanshi Zhang, Jing Shao, Na Zou, Xia Hu, and Dongrui Liu. What do EEG foundation models capture from human brain signals?arXiv preprint arXiv:2605.11410,

  24. [24]

    Hanneke van Dijk, Guido van Wingen, Damiaan Denys, Sebastian Olbrich, Rosalinde van Ruth, and Martijn Arns

    doi: 10.1016/j.nicl.2017.07.006. Hanneke van Dijk, Guido van Wingen, Damiaan Denys, Sebastian Olbrich, Rosalinde van Ruth, and Martijn Arns. The two decades brainclinics research archive for insights in neurophysiology (TDBRAIN) database. Scientific Data, 9(1):333,

  25. [25]

    Jiquan Wang et al

    doi: 10.1038/s41597-022-01409-z. Jiquan Wang et al. CBraMod: A criss-cross brain foundation model for EEG decoding.ICLR, 2025a. Siwen Wang, Shitou Zhang, Wan-Lin Chen, Dung Truong, and Tzyy-Ping Jung. From theory to application: Fine-tuning large EEG model with real-world stress data.arXiv preprint arXiv:2505.23042, 2025b. Yulin Wang, Wei Duan, Debo Dong,...

  26. [26]

    Jiamin Wu, Zichen Ren, Junyu Wang, Pengyu Zhu, Yonghao Song, Mianxin Liu, Qihao Zheng, Lei Bai, Wanli Ouyang, and Chunfeng Song

    doi: 10.1038/ s41597-022-01607-9. Jiamin Wu, Zichen Ren, Junyu Wang, Pengyu Zhu, Yonghao Song, Mianxin Liu, Qihao Zheng, Lei Bai, Wanli Ouyang, and Chunfeng Song. AdaBrain-Bench: Benchmarking brain foundation models for brain-computer interface applications.arXiv preprint arXiv:2507.09882,

  27. [27]

    Wei Xiong, Jiangtong Li, Jie Li, Kun Zhu, and Changjun Jiang

    doi: 10.1038/s41597-024-03268-2. Wei Xiong, Jiangtong Li, Jie Li, Kun Zhu, and Changjun Jiang. EEG-FM-Bench: A comprehensive benchmark for the systematic evaluation of EEG foundation models.arXiv preprint arXiv:2508.17742,