REVIEW 4 cited by
Mind's Eye: Image Recognition by EEG via Multimodal Similarity-Keeping Contrastive Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Mind's Eye: Image Recognition by EEG via Multimodal Similarity-Keeping Contrastive Learning
read the original abstract
Decoding images from non-invasive electroencephalographic (EEG) signals has been a grand challenge in understanding how the human brain process visual information in real-world scenarios. To cope with the issues of signal-to-noise ratio and nonstationarity, this paper introduces a MUltimodal Similarity-keeping contrastivE learning (MUSE) framework for zero-shot EEG-based image classification. We develop a series of multivariate time-series encoders tailored for EEG signals and assess the efficacy of regularized contrastive EEG-Image pretraining using an extensive visual EEG dataset. Our method achieves state-of-the-art performance, with a top-1 accuracy of 19.3% and a top-5 accuracy of 48.8% in 200-way zero-shot image classification. Furthermore, we visualize neural patterns via model interpretation, shedding light on the visual processing dynamics in the human brain. The code repository for this work is available at: https://github.com/ChiShengChen/MUSE_EEG.
Forward citations
Cited by 4 Pith papers
-
SATTC: Structure-Aware Label-Free Test-Time Calibration for Cross-Subject EEG-to-Image Retrieval
SATTC improves top-k accuracy in cross-subject EEG-to-image retrieval by fusing geometric whitening and structural nearest-neighbor experts on the similarity matrix without labels.
-
Quantum Adaptive Self-Attention for Quantum Transformer Models
QASA with one 36-parameter quantum layer in a Transformer achieves best MSE on 4 of 9 synthetic tasks and 6% MAE reduction on ETTh1, outperforming larger quantum models on chaotic/noisy signals while classical models ...
-
Decoding Visual Neural Representations by Multimodal with Dynamic Balancing
A multimodal EEG-image-text contrastive framework with dynamic gradient balancing and stochastic noise improves zero-shot object recognition from EEG on ThingsEEG, raising top-1 accuracy from 13.8% to 15.8%.
-
Quantum-Enhanced Natural Language Generation: A Multi-Model Framework with Hybrid Quantum-Classical Architectures
A benchmark of QASA, QRWKV, and QKSAN against Transformer and MLP on five tiny datasets, with results that contradict the paper's own tables.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.