REVIEW 9 cited by
BrainBERT: Self-supervised representation learning for intracranial recordings
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
We create a reusable Transformer, BrainBERT, for intracranial recordings bringing modern representation learning approaches to neuroscience. Much like in NLP and speech recognition, this Transformer enables classifying complex concepts, i.e., decoding neural data, with higher accuracy and with much less data by being pretrained in an unsupervised manner on a large corpus of unannotated neural recordings. Our approach generalizes to new subjects with electrodes in new positions and to unrelated tasks showing that the representations robustly disentangle the neural signal. Just like in NLP where one can study language by investigating what a language model learns, this approach opens the door to investigating the brain by what a model of the brain learns. As a first step along this path, we demonstrate a new analysis of the intrinsic dimensionality of the computations in different areas of the brain. To construct these representations, we combine a technique for producing super-resolution spectrograms of neural data with an approach designed for generating contextual representations of audio by masking. In the future, far more concepts will be decodable from neural recordings by using representation learning, potentially unlocking the brain like language models unlocked language.
Forward citations
Cited by 9 Pith papers
-
Understanding and Correcting Low-Frequency Bias in EEG Foundation Model
FAME, a frequency-balanced masked autoencoder that standardizes and equally weights per-band reconstruction targets, reduces the low-frequency bias of EEG representations and beats prior models on 24 of 41 downstream tasks.
-
CSBrain: A Cross-scale Spatiotemporal Brain Foundation Model for EEG Decoding
CSBrain, a cross-scale spatiotemporal EEG foundation model, reports state-of-the-art results across 11 tasks and 16 datasets, but its claim of consistent superiority over all baselines is not supported by its own tables.
-
DiffEEG: A Self-Supervised Denoising Diffusion Model for Learning EEG Generic Representations
A diffusion U-Net pretrained on unlabeled TUHSZ EEG plus an F1-maximizing RL decision layer yields clinically usable patient-wise seizure detection and subtyping under severe class imbalance.
-
STST-JEPA: Shallow-Target Spatio-Temporal Joint Embedding Prediction Architecture For EEG Self-Supervised Learning
A JEPA-style EEG foundation model with shallow EMA targets plus light reconstruction reaches strong multi-task transfer and 3.06-year validation age MAE on a large multi-site corpus.
-
Transformer-based EEG Decoding: A Survey
A survey that classifies Transformer-based EEG decoding models into backbone, hybrid, and customized categories and reviews their applications and limitations.
-
Cross-Modal Epileptic Signal Harmonization: Frequency Domain Mapping Quantization for Pre-training a Unified Neurophysiological Transformer
EpiNT, a transformer pretrained on over 2,700 hours of EEG and iEEG from 1,199 patients with masked autoencoding and a frequency-domain quantizer, matches or beats other pretrained models on six epilepsy classificatio...
-
CRIA: A Cross-View Interaction and Instance-Adapted Pre-training Framework for Generalizable EEG Representations
Fusing temporal, spectral, and spatial EEG views with cross-attention and view-wise masking improves downstream classification and cross-dataset generalization over prior EEG pretraining models.
-
Neuro2Semantic: A Transfer Learning Framework for Semantic Reconstruction of Continuous Language from Human Intracranial EEG
A two-stage model aligns intracranial EEG signals with text embeddings and reconstructs the semantic content of perceived speech from as little as 30 minutes of neural data.
-
BrainStratify: Coarse-to-Fine Disentanglement of Intracranial Neural Dynamics
BrainStratify's coarse-to-fine disentanglement, electrode clustering plus decoupled product quantization, modestly improves speech decoding over prior methods on sEEG and epidural ECoG datasets.
Discussion (0). Continue with ORCID to comment.