CORTEG: Foundation Models Enable Cross-Modality Representation Transfer from Scalp to Intracranial Brain Recordings
Pith reviewed 2026-05-12 04:43 UTC · model grok-4.3
The pith
Scalp EEG foundation models can be adapted to decode intracranial ECoG signals competitively with minimal new data.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
CORTEG demonstrates that representations learned from scalp-EEG foundation models contain transferable information to ECoG, allowing a pretrained backbone combined with a spatial adapter and dual-stream tokenizer to match or exceed task-specific baselines on finger trajectory and audio envelope regression, with notable advantages in low-data calibration scenarios for new patients.
What carries the argument
The CORTEG framework, which integrates a pretrained EEG foundation model backbone, an electrode-aware KNNSoftFourier spatial adapter, a dual-stream tokenizer for low-frequency and high-gamma activity, and leave-one-subject-out fine-tuning.
If this is right
- CORTEG enables competitive decoding performance on ECoG without relying solely on subject-specific training.
- It supports cross-patient learning by adapting from large scalp datasets to individual intracranial recordings.
- Per-patient calibration can be completed in 10-30 minutes on a single GPU for practical use.
- Feature analyses from the model align with established neurophysiological patterns.
- Latent representations capture low-dimensional structures in movements like finger trajectories.
Where Pith is reading between the lines
- This suggests that non-invasive scalp data can serve as a foundation for improving invasive neural interfaces in data-limited settings.
- Similar transfer strategies might apply to other brain signal modalities or decoding tasks beyond regression.
- Future work could test the framework on additional ECoG datasets to confirm generalizability across different electrode configurations.
Load-bearing premise
The representations learned from scalp EEG contain information that remains useful for ECoG despite variations in signal noise, electrode density, and the specific brain areas covered.
What would settle it
Demonstrating no performance gain or a performance drop when applying the transferred model to a new ECoG dataset compared to training a model from scratch on that same limited data.
Figures
read the original abstract
Intracranial electrocorticography (ECoG) offers high-signal-to-noise access to cortical activity for brain-computer interfaces, yet limited per-patient data has led most prior work to rely on small, subject-specific decoders that neglect information shared across patients. We investigate whether large pretrained scalp-EEG foundation models (EEG FMs) can be adapted to ECoG, enabling cross-patient learning and competitive decoding performance while calibrating to a held-out patient in 10-30 minutes on a single GPU. We introduce CORTEG, a cross-modality transfer framework that combines a pretrained EEG FM backbone, an electrode-aware KNNSoftFourier spatial adapter, a dual-stream tokenizer for low-frequency and high-gamma activity, and a leave-one-subject-out fine-tuning strategy. We evaluate CORTEG on two challenging regression tasks: public finger trajectory regression (n=9) and private audio envelope regression (n=16). CORTEG matches or exceeds the strongest task-specific baselines on both tasks: it reaches the highest mean correlation among compared methods on the public finger benchmark (gain not statistically significant on n=9 subjects), with larger and statistically significant gains on the audio task and in low-data per-patient calibration. Feature analyses align with neurophysiology, and latent manifolds capture low-dimensional finger-movement structure. CORTEG provides systematic evidence that scalp-EEG pretraining can be repurposed for ECoG decoding, enabling data-efficient intracranial BCIs that can adapt to new patients.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript introduces CORTEG, a framework for adapting pretrained scalp-EEG foundation models to intracranial ECoG recordings. It combines a pretrained EEG FM backbone with an electrode-aware KNNSoftFourier spatial adapter, a dual-stream tokenizer separating low-frequency and high-gamma activity, and leave-one-subject-out fine-tuning. The approach is evaluated on finger trajectory regression using a public dataset (n=9 subjects) and audio envelope regression using a private dataset (n=16 subjects), with claims that it matches or exceeds task-specific baselines, achieving the highest mean correlation on the finger task (not statistically significant) and larger, statistically significant gains on the audio task and in low-data per-patient calibration scenarios. Feature analyses are reported to align with neurophysiology and latent spaces capture movement structure.
Significance. If the central claims hold after addressing the noted gaps, the work would offer systematic evidence that large-scale scalp-EEG pretraining can be repurposed for ECoG decoding. This could advance data-efficient intracranial BCIs by reducing per-patient calibration time to 10-30 minutes and enabling cross-patient learning, with potential implications for clinical BCI applications where data scarcity is a limiting factor.
major comments (2)
- [Methods and Results] The load-bearing claim that pretrained scalp-EEG foundation model representations drive cross-modality transfer to ECoG is not isolated from the effects of the adapters and training procedure. No ablation is presented that freezes or randomizes the backbone weights while retaining the electrode-aware KNNSoftFourier adapter, dual-stream tokenizer, and leave-one-subject-out fine-tuning; without this control, it remains unclear whether the reported gains (particularly the statistically significant improvements on audio regression and low-data calibration) arise from transferable representations learned on scalp data or from the adapter architecture and fine-tuning recipe alone.
- [Abstract and Results] On the public finger trajectory benchmark (n=9 subjects), the highest mean correlation is reported as not statistically significant relative to the strongest baselines. Given the small sample size, this weakens support for the broader claim of matching or exceeding task-specific methods across both tasks, especially since the audio task (n=16, private data) carries the weight of the statistically significant results.
minor comments (3)
- Exact implementation details for the compared baselines, full numerical metrics with error bars or confidence intervals, and subject/trial exclusion criteria are not fully specified, which hinders precise reproduction and assessment of the performance claims.
- The private audio dataset (n=16) precludes independent verification of the key statistically significant gains; public release of the data or additional public benchmarks would strengthen the work.
- Notation for the dual-stream tokenizer and KNNSoftFourier adapter could be clarified with explicit equations or pseudocode to improve reproducibility.
Simulated Author's Rebuttal
We thank the referee for their thorough and constructive comments on our manuscript. We address each major comment below and outline the revisions we will make to strengthen the paper.
read point-by-point responses
-
Referee: [Methods and Results] The load-bearing claim that pretrained scalp-EEG foundation model representations drive cross-modality transfer to ECoG is not isolated from the effects of the adapters and training procedure. No ablation is presented that freezes or randomizes the backbone weights while retaining the electrode-aware KNNSoftFourier adapter, dual-stream tokenizer, and leave-one-subject-out fine-tuning; without this control, it remains unclear whether the reported gains (particularly the statistically significant improvements on audio regression and low-data calibration) arise from transferable representations learned on scalp data or from the adapter architecture and fine-tuning recipe alone.
Authors: We agree that isolating the contribution of the pretrained backbone is crucial for validating our central claim. In the revised manuscript, we will include an ablation study where the backbone weights are frozen during fine-tuning, as well as a control with randomized backbone weights, while retaining the adapter, tokenizer, and fine-tuning procedure. This will help demonstrate that the performance improvements are driven by the transferable representations from the scalp-EEG foundation model rather than the adapter architecture alone. revision: yes
-
Referee: [Abstract and Results] On the public finger trajectory benchmark (n=9 subjects), the highest mean correlation is reported as not statistically significant relative to the strongest baselines. Given the small sample size, this weakens support for the broader claim of matching or exceeding task-specific methods across both tasks, especially since the audio task (n=16, private data) carries the weight of the statistically significant results.
Authors: We acknowledge the limitation posed by the small sample size (n=9) on the finger trajectory task, where the improvement is not statistically significant. This is already noted in the manuscript. To address this, we will revise the abstract and results section to more carefully qualify the claims: CORTEG matches the strongest baselines on the finger task and achieves statistically significant improvements on the audio task and in low-data per-patient scenarios. We will also emphasize the consistency across tasks and the practical benefits in data-efficient settings. revision: partial
Circularity Check
No circularity: purely empirical framework evaluation
full rationale
The paper presents CORTEG as an empirical transfer framework evaluated via direct comparisons to task-specific baselines on two regression tasks. No mathematical derivations, equations, predictions, or uniqueness theorems are invoked. Performance metrics arise from standard training and testing procedures rather than any self-referential fitting or renaming. Self-citations (if present for the pretrained backbone) are not load-bearing for any claimed derivation, as the central results are falsifiable experimental outcomes. This is self-contained empirical work with no reduction of outputs to inputs by construction.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption Representations learned by scalp-EEG foundation models contain information transferable to intracranial ECoG signals
invented entities (2)
-
electrode-aware KNNSoftFourier spatial adapter
no independent evidence
-
dual-stream tokenizer for low-frequency and high-gamma activity
no independent evidence
Reference graph
Works this paper leans on
-
[1]
Alim Louis Benabid, Thomas Costecalde, Andrey Eliseyev, Guillaume Charvet, Alexandre Verney, Serpil Karakas, Michael Foerster, Aurélien Lambert, Boris Morinière, Neil Abroug, et al. An exoskeleton controlled by an epidural wireless brain–machine interface in a tetraplegic patient: a proof-of-concept demonstration.The Lancet Neurology, 18(12):1112–1122, 2019
work page 2019
-
[2]
Henri Lorach, Andrea Galvez, Valeria Spagnolo, Felix Martel, Serpil Karakas, Nadine Intering, Molywan Vat, Olivier Faivre, Cathal Harte, Salif Komi, et al. Walking naturally after spinal cord injury using a brain–spine interface.Nature, 618(7963):126–133, 2023
work page 2023
-
[3]
Sean L Metzger, Jessie R Liu, David A Moses, Maximilian E Dougherty, Margaret P Seaton, Kaylo T Littlejohn, Josh Chartier, Gopala K Anumanchipalli, Adelyn Tu-Chan, Karunesh Ganguly, et al. Generalizable spelling using a speech neuroprosthesis in an individual with severe limb and vocal paralysis.Nature communications, 13(1):6510, 2022
work page 2022
-
[4]
Kaylo T Littlejohn, Cheol Jun Cho, Jessie R Liu, Alexander B Silva, Bohan Yu, Vanessa R Anderson, Cady M Kurtz-Miott, Samantha Brosler, Anshul P Kashyap, Irina P Hallinan, et al. A streaming brain-to-voice neuroprosthesis to restore naturalistic communication.Nature neuroscience, 28(4):902–912, 2025
work page 2025
-
[5]
ECoG-based movement classification and limbs 3D translation prediction: a deep learning study
Quentin Ferdinand, Rémi Souriau, Lucas Struber, Henri Lorach, Philippe Ciuciu, Marina Reyboz, and Tetiana Aksenova. ECoG-based movement classification and limbs 3D translation prediction: a deep learning study. In2025 International Joint Conference on Neural Networks (IJCNN), pages 1–10. IEEE, 2025
work page 2025
-
[6]
Hsiang-Yun Sherry Chien, Hanlin Goh, Christopher M. Sandino, and Joseph Y . Cheng. Maeeg: Masked auto-encoder for eeg representation learning, 2022. URL https://arxiv.org/abs/ 2211.02625
-
[7]
arXiv preprint arXiv:2305.10351 , year=
Chaoqi Yang, M. Brandon Westover, and Jimeng Sun. BIOT: Cross-data biosignal learning in the wild, 2023. URLhttps://arxiv.org/abs/2305.10351
-
[8]
arXiv preprint arXiv:2405.18765 , year=
Wei-Bang Jiang, Li-Ming Zhao, and Bao-Liang Lu. Large brain model for learning generic representations with tremendous EEG data in BCI, 2024. URL https://arxiv.org/abs/ 2405.18765
-
[9]
Jiquan Wang, Sha Zhao, Zhiling Luo, Yangxuan Zhou, Haiteng Jiang, Shijian Li, Tao Li, and Gang Pan. CBraMod: A criss-cross brain foundation model for EEG decoding, 2025. URL https://arxiv.org/abs/2412.07236
-
[10]
Liuyin Yang, Qiang Sun, Ang Li, and Marc M. Van Hulle. Are EEG foundation models worth it? comparative evaluation with traditional decoders in diverse BCI tasks. InThe Fourteenth International Conference on Learning Representations, 2026. URL https://openreview. net/forum?id=5Xwm8e6vbh
work page 2026
-
[11]
Nathan E Crone, Dana Boatman, Barry Gordon, and Lei Hao. Induced electrocorticographic gamma activity during auditory perception.Clinical neurophysiology, 112(4):565–582, 2001
work page 2001
-
[12]
Qiang Sun, Liuyin Yang, Eva Calvo Merino, and Marc M Van Hulle. Large EEG foundation model learns informative low-frequency representations from intracranial brain signals. In1st ICLR Workshop on Time Series in the Age of Large Models, 2026
work page 2026
-
[13]
JOJWGSJ Kubanek, Kai J Miller, Jeffrey G Ojemann, Jonathan R Wolpaw, and Gerwin Schalk. Decoding flexion of individual fingers using electrocorticographic signals in humans.Journal of neural engineering, 6(6):066001, 2009. 10
work page 2009
-
[14]
Ziqian Xie, Odelia Schwartz, and Abhishek Prasad. Decoding of finger trajectory from ECoG using deep learning.Journal of neural engineering, 15(3):036009, 2018
work page 2018
-
[15]
Lin Yao, Bingzhao Zhu, and Mahsa Shoaran. Fast and accurate decoding of finger movements from ECoG through riemannian features and modern machine learning techniques.Journal of Neural Engineering, 19(1):016037, 2022
work page 2022
-
[16]
Axel Faes, Flavio Camarrone, and Marc M Van Hulle. Single finger trajectory prediction from intracranial brain activity using block-term tensor regression with fast and automatic component extraction.IEEE Transactions on Neural Networks and Learning Systems, 35(7):8897–8908, 2022
work page 2022
-
[17]
Zhanhui Lin, Xinyu Jiang, Chenyun Dai, and Fumin Jia. Towards real time efficient and robust ECoG decoding for mobile brain–computer interface.Journal of Neural Engineering, 22(4): 046012, 2025
work page 2025
-
[18]
Jonas Vanthornhout, Lien Decruy, Jan Wouters, Jonathan Z. Simon, and Tom Francart. Speech intelligibility predicted from neural entrainment of the speech envelope.Journal of the Associa- tion for Research in Otolaryngology, 19(2):181–191, 2018. doi: 10.1007/s10162-018-0654-z
-
[19]
Bernd Accou, Jonas Vanthornhout, Hugo Van hamme, and Tom Francart. Decoding of the speech envelope from EEG using the VLAAI deep neural network.Scientific Reports, 13(1): 812, 2023. doi: 10.1038/s41598-022-27332-2
-
[20]
Alexandre Défossez, Charlotte Caucheteux, Jérémy Rapin, Ori Kabeli, and Jean-Rémi King. Decoding speech perception from non-invasive brain recordings.Nature Machine Intelligence, 5(10):1097–1107, 2023
work page 2023
-
[21]
Ludovic Bellier, Anaïs Llorens, Déborah Marciano, Aysegul Gunduz, Gerwin Schalk, Peter Brunner, and Robert T. Knight. Music can be reconstructed from human auditory cortex activity using nonlinear decoding models.PLOS Biology, 21(8):e3002176, 2023. doi: 10.1371/journal. pbio.3002176
-
[22]
Emergence of language in the developing brain, 2025
Linnea Evanson, Christine Bulteau, Mathilde Chipaux, Georg Dorfmüller, Sarah Ferrand- Sorbets, Emmanuel Raffo, Sarah Rosenberg, Pierre Bourdillon, and Jean-Rémi King. Emergence of language in the developing brain, 2025. Preprint
work page 2025
-
[23]
He He and Dongrui Wu. Transfer learning for brain–computer interfaces: A euclidean space data alignment approach.IEEE Transactions on Biomedical Engineering, 67(2):399–410, 2019
work page 2019
-
[24]
Generalized neural decoders for transfer learning across participants and recording modalities
Steven M Peterson, Zoe Steine-Hanson, Nathan Davis, Rajesh PN Rao, and Bingni W Brunton. Generalized neural decoders for transfer learning across participants and recording modalities. Journal of Neural Engineering, 18(2):026014, 2021
work page 2021
-
[25]
René J Huster, Stefan Debener, Tom Eichele, and Christoph S Herrmann. Methods for si- multaneous eeg-fmri: an introductory review.Journal of Neuroscience, 32(18):6053–6060, 2012
work page 2012
-
[26]
Richard N Henson, Daniel G Wakeman, Vladimir Litvak, and Karl J Friston. A parametric empirical bayesian framework for the eeg/meg inverse problem: generative models for multi- subject and multi-modal integration.Frontiers in human neuroscience, 5:76, 2011
work page 2011
-
[27]
LoRA: Low-rank adaptation of large language models
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations (ICLR), 2022
work page 2022
-
[28]
Parameter-efficient transfer learning for nlp
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for nlp. InInternational conference on machine learning, pages 2790–2799. PMLR, 2019
work page 2019
-
[29]
Nerf: Representing scenes as neural radiance fields for view synthesis
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoor- thi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65(1):99–106, 2021. 11
work page 2021
-
[30]
Matthew Tancik, Pratul Srinivasan, Ben Mildenhall, Sara Fridovich-Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan Barron, and Ren Ng. Fourier features let networks learn high frequency functions in low dimensional domains.Advances in neural information processing systems, 33:7537–7547, 2020
work page 2020
-
[31]
On estimating regression.Theory of Probability & Its Applications, 9(1): 141–142, 1964
Elizbar A Nadaraya. On estimating regression.Theory of Probability & Its Applications, 9(1): 141–142, 1964
work page 1964
-
[32]
Geoffrey S Watson. Smooth regression analysis.Sankhy ¯a: The Indian Journal of Statistics, Series A, pages 359–372, 1964
work page 1964
-
[33]
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020
work page internal anchor Pith review Pith/arXiv arXiv 2010
-
[34]
June 23, 2025.DOI:10.48550/arXiv.2506.19141
Bruno Aristimunha, Dung Truong, Pierre Guetschel, Seyed Yahya Shirazi, Isabelle Guyon, Alexandre R. Franco, Michael P. Milham, Aviv Dotan, Scott Makeig, Alexandre Gramfort, Jean-Remi King, Marie-Constance Corsi, Pedro A. Valdés-Sosa, Amit Majumdar, Alan Evans, Terrence J Sejnowski, Oren Shriki, Sylvain Chevallier, and Arnaud Delorme. Eeg foundation challe...
-
[35]
Kai J Miller. A library of human electrocorticographic data and analyses.Nature human behaviour, 3(11):1225–1235, 2019
work page 2019
-
[36]
Qiang Sun, Eva Calvo Merino, Bob Van Dyck, Yuan Yang, Jiayuan He, and Marc M Van Hulle. Spectro-temporal fusion of high-gamma and low-frequency ecog signals for intracranial finger movement decoding.TechRxiv, 2025
work page 2025
-
[37]
Wei Tao, Jianghao Hou, Yi Yang, Xun Chen, Tzyy-Ping Jung, and Feng Wan. Deepfingernet predicts finger trajectory from ecog measurements.IEEE Transactions on Instrumentation and Measurement, 74:1–12, 2025
work page 2025
-
[38]
Vasilii Feofanov, Songkang Wen, Jianfeng Zhang, Lujia Pan, and Ievgen Redko. Mantisv2: Closing the zero-shot gap in time series classification with synthetic data and test-time strategies,
- [39]
-
[40]
The elements of statistical learning: data mining, inference, and prediction, 2009
Trevor Hastie. The elements of statistical learning: data mining, inference, and prediction, 2009
work page 2009
-
[41]
Demixed principal component analysis of neural population data.elife, 5:e10989, 2016
Dmitry Kobak, Wieland Brendel, Christos Constantinidis, Claudia E Feierstein, Adam Kepecs, Zachary F Mainen, Xue-Lian Qi, Ranulfo Romo, Naoshige Uchida, and Christian K Machens. Demixed principal component analysis of neural population data.elife, 5:e10989, 2016
work page 2016
-
[42]
Kai J Miller, Dora Hermes, Christopher J Honey, Adam O Hebb, Nick F Ramsey, Robert T Knight, Jeffrey G Ojemann, and Eberhard E Fetz. Human motor cortical activity is selectively phase-entrained on underlying rhythms.PLoS Computational Biology, 8(9), 2012
work page 2012
-
[43]
Human Motor Cortical Activity Is Selectively Phase- Entrained on Underlying Rhythms
Peter L. Søndergaard and Piotr Majdak. The auditory modeling toolbox. In Jens Blauert, editor, The Technology of Binaural Listening, Modern Acoustics and Signal Processing, pages 33–56. Springer, Berlin, Heidelberg, 2013. 12 A Appendix A.1 LOO-FT Strategy Sweep and Failed Alternatives Loss and metric.CORTEG minimizes per-subject z-scored MSE L= 1 N NX i=1...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.