REVIEW 15 cited by
NeuroLM: A Universal Multi-task Foundation Model for Bridging the Gap between Language and EEG Signals
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
NeuroLM: A Universal Multi-task Foundation Model for Bridging the Gap between Language and EEG Signals
read the original abstract
Recent advancements for large-scale pre-training with neural signals such as electroencephalogram (EEG) have shown promising results, significantly boosting the development of brain-computer interfaces (BCIs) and healthcare. However, these pre-trained models often require full fine-tuning on each downstream task to achieve substantial improvements, limiting their versatility and usability, and leading to considerable resource wastage. To tackle these challenges, we propose NeuroLM, the first multi-task foundation model that leverages the capabilities of Large Language Models (LLMs) by regarding EEG signals as a foreign language, endowing the model with multi-task learning and inference capabilities. Our approach begins with learning a text-aligned neural tokenizer through vector-quantized temporal-frequency prediction, which encodes EEG signals into discrete neural tokens. These EEG tokens, generated by the frozen vector-quantized (VQ) encoder, are then fed into an LLM that learns causal EEG information via multi-channel autoregression. Consequently, NeuroLM can understand both EEG and language modalities. Finally, multi-task instruction tuning adapts NeuroLM to various downstream tasks. We are the first to demonstrate that, by specific incorporation with LLMs, NeuroLM unifies diverse EEG tasks within a single model through instruction tuning. The largest variant NeuroLM-XL has record-breaking 1.7B parameters for EEG signal processing, and is pre-trained on a large-scale corpus comprising approximately 25,000-hour EEG data. When evaluated on six diverse downstream datasets, NeuroLM showcases the huge potential of this multi-task learning paradigm.
Forward citations
Cited by 15 Pith papers
-
EvoBrain: Continual Learning of EEG Foundation Models Across Heterogeneous BCI Tasks
EvoBrain introduces a continual learning method with Neuro-Spectral Task Normalization and Response-Affinity Distillation to enable unified EEG decoding across heterogeneous BCI tasks.
-
OmniEEG-Bench: A Standardized Evaluation Benchmark for EEG Foundation Models
OmniEEG-Bench unifies 54 EEG datasets into six task families and benchmarks 10 foundation models, finding that pretraining diversity and model size correlate with better average performance ranks.
-
Let EEG Models Learn EEG
JET is a conditional flow matching framework that generates EEG as continuous raw sequences with added constraints for spectral and temporal properties, achieving over 40% lower TS-FID than prior discrete denoising me...
-
NeuroAtlas: Benchmarking Foundation Models for Clinical EEG and Brain-Computer Interfaces
NeuroAtlas benchmarks foundation models on 42 EEG datasets and reports that EEG-specific models do not consistently outperform generic time-series models, standard metrics miss clinical utility, and rankings vary by domain.
-
Visualizing the Invisible: Generative Visual Grounding Empowers Universal EEG Understanding in MLLMs
Generative Visual Grounding creates instance-specific visual proxy images from EEG signals to enhance MLLM understanding of brain activity beyond text-only alignment.
-
Visualizing the Invisible: Generative Visual Grounding Empowers Universal EEG Understanding in MLLMs
Generative Visual Grounding creates visual proxy images from EEG to enhance MLLM understanding of brain signals beyond text-only alignment.
-
CLEF: EEG Foundation Model for Learning Clinical Semantics
CLEF, a long-context EEG foundation model using 3D multitaper spectrograms and contrastive alignment with reports and EHR, beats prior models on 229 of 234 clinical tasks and raises mean AUROC from 0.65 to 0.74.
-
EduGage: Methods and Dataset for Sensor-Based Momentary Assessment of Engagement in Self-Guided Video Learning
EduGage releases a multimodal sensor dataset and models for estimating learner engagement in self-guided video learning, reporting MAE of 0.81 and outperforming baselines with 16 participants.
-
LLM as Clinical Graph Structure Refiner: Enhancing Representation Learning in EEG Seizure Diagnosis
LLM-based refinement of edges in transformer-constructed EEG graphs improves seizure detection accuracy and produces cleaner, more interpretable structures on the TUSZ dataset.
-
fMRI-LM: Towards a Universal Foundation Model for Language-Aligned fMRI Understanding
fMRI-LM builds a foundation model that aligns fMRI signals with language through tokenization, LLM adaptation, and instruction tuning to enable semantic understanding of brain activity.
-
Tokenizing Single-Channel EEG with Time-Frequency Motif Learning
TFM-Tokenizer learns a vocabulary of time-frequency motifs from single-channel EEG via a dual-path masked architecture and encodes signals into discrete tokens, reporting up to 11% Cohen's Kappa gains on benchmarks an...
-
MindAU: EEG-Conditioned Facial Action Unit Editing via Dual-Stream Manifold Alignment
MindAU is a dual-stream manifold alignment system that conditions a multimodal diffusion editor on EEG signals to perform fine-grained, identity-preserving facial action unit edits.
-
Wearable AI in the Era of Large Sensor Models
Large Sensor Models trained on large-scale multimodal wearable data can provide a scalable, general framework for wearable AI by learning transferable representations across modalities and tasks.
-
Benchmarking ERP Analysis: Manual Features, Deep Learning, and Foundation Models
A unified benchmark across 12 ERP datasets finds that foundation models and deep learning generally outperform traditional manual features for stimulus classification and disease detection, with specific embedding str...
-
NeuroWeaver: An Autonomous Evolutionary Agent for Exploring the Programmatic Space of EEG Analysis Pipelines
NeuroWeaver reformulates EEG pipeline design as constrained evolutionary optimization with domain-informed initialization, yielding lightweight pipelines that outperform task-specific methods and match foundation mode...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.