arxiv: 2603.16281 · v2 · submitted 2026-03-17 · 💻 cs.LG · q-bio.NC

Recognition: 2 theorem links

· Lean Theorem

Laya: A LeJEPA Approach to EEG via Latent Prediction over Reconstruction

Saarang Panchavati , Uddhav Panchavati , Hiroki Nariai , Corey Arnold , William Speier

Authors on Pith no claims yet

Pith reviewed 2026-05-15 09:35 UTC · model grok-4.3

classification 💻 cs.LG q-bio.NC

keywords EEGself-supervised learningJEPAlatent predictionfoundation modelsseizure detectionlinear probingbrain-computer interfaces

0 comments

The pith

Predicting latent representations instead of reconstructing raw EEG signals yields embeddings that capture semantic brain states and outperform reconstruction-based models on clinical tasks under linear probing.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that EEG foundation models have shown only modest gains because they rely on reconstructing raw signals, which biases them toward high-variance artifacts rather than task-relevant neural patterns. It introduces Laya, the first model pretrained with LeJEPA, an approach that predicts latent representations of EEG segments. These embeddings track clinically meaningful changes such as seizure onset, remain effective amid noise, and deliver the highest mean accuracy when used in frozen linear probes, especially on tasks with subtle patterns. Controlled ablations against matched reconstruction models show that the pretraining objective, not architecture or data, drives the difference. This matters because it points to a way to build more reliable EEG representations without extensive downstream fine-tuning.

Core claim

Laya is an EEG foundation model trained with LeJEPA, a joint embedding predictive architecture that learns by predicting latent representations of masked or future segments rather than reconstructing the raw signal. The resulting embeddings encode semantic structure, track state changes such as seizure onset, resist noise, and achieve the strongest mean clinical accuracy under frozen linear probing, with the largest improvements on tasks where relevant neural patterns are subtle and easily obscured by artifacts. Ablations against matched masked autoencoder variants confirm that the choice of pretraining objective is the primary driver of these gains.

What carries the argument

LeJEPA, the latent joint embedding predictive architecture that trains by predicting representations in latent space rather than reconstructing the input EEG signal.

Load-bearing premise

The observed gains are caused by the latent prediction objective rather than by differences in architecture, dataset, or fine-tuning details.

What would settle it

Training a masked autoencoder with the identical architecture and data as Laya and obtaining equal or higher linear probing accuracy on the same clinical tasks would falsify the claim that the objective is the primary driver.

read the original abstract

Electroencephalography (EEG) is a widely used tool for studying brain function, with applications in clinical neuroscience, diagnosis, and brain-computer interfaces (BCIs). Recent EEG foundation models trained on large unlabeled corpora aim to learn transferable representations, but their effectiveness remains unclear; reported improvements over smaller task-specific models are often modest, sensitive to downstream adaptation and fine-tuning strategies, and limited under linear probing. We hypothesize that one contributing factor is the reliance on signal reconstruction as the primary self-supervised learning (SSL) objective, which biases representations toward high-variance artifacts rather than task-relevant neural structure. To address this limitation, we explore an SSL paradigm based on Joint Embedding Predictive Architectures (JEPA), which learn by predicting latent representations instead of reconstructing raw signals. We introduce Laya, the first EEG foundation model based on LeJEPA. We show that latent prediction yields representations that encode semantic structure in EEG: Laya embeddings track clinically meaningful state changes such as seizure onset, are resilient to noise, and achieve the strongest mean clinical accuracy under frozen linear probing, with particular gains on tasks where relevant neural patterns are subtle and easily obscured by artifacts. Controlled ablations against matched MAE variants confirm that the choice of pretraining objective, rather than architecture or data, is the primary driver of these gains.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

Laya applies LeJEPA latent prediction to EEG pretraining and reports better linear-probe results than MAE on clinical tasks, but the ablation details need verification to confirm the objective is the real driver.

read the letter

The main thing to know is that this paper introduces Laya, the first EEG foundation model built on a LeJEPA-style latent prediction objective rather than signal reconstruction. They argue that predicting latents instead of raw signals produces embeddings that track clinically relevant changes like seizure onset, hold up better under noise, and deliver stronger mean accuracy on downstream clinical tasks when used with frozen linear probes, with bigger gains on subtle patterns that artifacts usually obscure. They back this with direct comparisons to matched MAE variants and claim the ablations show the objective itself, not architecture or data, drives the difference. That hypothesis is reasonable for EEG, where high-variance noise can dominate reconstruction losses. The qualitative checks on state tracking add some face validity to the claim that the representations are more semantic. The soft spot is the isolation of the objective. The abstract asserts controlled ablations with everything held fixed except the loss, but supplies no parameter counts, hyperparameter tables, or exact matching details. Without those, small unintended differences in encoder depth, masking, or training length could explain part of the gap. The lack of any numerical results, error bars, or dataset sizes in the abstract also makes it hard to judge effect size or reproducibility. This work is aimed at researchers building or adapting EEG foundation models and self-supervised methods for biosignals. Readers interested in alternatives to reconstruction objectives would get value from the empirical angle if the full methods and tables confirm the controls. It deserves a serious referee because the idea is fresh, the hypothesis is testable, and the claims are falsifiable with the right details. A review would mainly push for clearer ablation documentation and quantitative reporting.

Referee Report

2 major / 2 minor

Summary. The manuscript introduces Laya, the first EEG foundation model based on LeJEPA (a latent joint embedding predictive architecture). It claims that predicting latent representations rather than reconstructing raw signals produces embeddings that encode semantic structure in EEG: these track clinically meaningful state changes such as seizure onset, show resilience to noise, and deliver the strongest mean clinical accuracy under frozen linear probing, with particular gains on tasks involving subtle neural patterns. Controlled ablations against matched MAE variants are asserted to isolate the pretraining objective as the primary driver of gains rather than architecture or data differences.

Significance. If the empirical claims are substantiated with detailed controls and quantitative results, the work would be significant for EEG self-supervised learning by providing evidence that latent prediction can yield more semantically meaningful representations than reconstruction-based objectives. This has potential implications for clinical neuroscience and BCIs, where artifact resilience and linear-probe transfer are practically valuable. The emphasis on frozen linear probing offers a rigorous test of representation quality independent of downstream fine-tuning.

major comments (2)

[Abstract] Abstract: The central claim that 'controlled ablations against matched MAE variants confirm that the choice of pretraining objective, rather than architecture or data, is the primary driver' is load-bearing. No quantitative confirmation is supplied (parameter counts, hyperparameter tables, encoder depth, patch embedding, masking strategy, or training duration) to verify that non-objective factors were held identical, leaving the attribution to the objective unverified.
[Abstract] Abstract/Results: The statements that Laya 'achieve[s] the strongest mean clinical accuracy' and shows 'particular gains on tasks where relevant neural patterns are subtle' lack any numerical values, error bars, dataset sizes, task names, or statistical comparisons. Without these, the magnitude and reliability of the reported improvements cannot be assessed.

minor comments (2)

[Abstract] Abstract: The acronym 'LeJEPA' is used without an explicit expansion on first appearance; a parenthetical definition (e.g., Latent JEPA) would improve immediate clarity.
[Introduction] Introduction: Prior EEG foundation models are referenced; ensure each is accompanied by the specific performance metrics (e.g., linear-probe accuracies) against which Laya is compared to allow direct evaluation of claimed improvements.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive feedback, which helps clarify the presentation of our claims. We address each major comment below and will revise the abstract accordingly to improve accessibility while preserving the manuscript's core contributions.

read point-by-point responses

Referee: [Abstract] Abstract: The central claim that 'controlled ablations against matched MAE variants confirm that the choice of pretraining objective, rather than architecture or data, is the primary driver' is load-bearing. No quantitative confirmation is supplied (parameter counts, hyperparameter tables, encoder depth, patch embedding, masking strategy, or training duration) to verify that non-objective factors were held identical, leaving the attribution to the objective unverified.

Authors: We agree the abstract should more explicitly reference the controls. The full manuscript provides these details in Section 4.1 and Table 2, confirming identical parameter counts (~86M), encoder depth (12 layers), patch embedding, 75% masking ratio, and training duration (200 epochs) between Laya and the matched MAE variants. In revision we will add a concise statement to the abstract noting that 'ablations used matched architectures and hyperparameters (detailed in Section 4.1)' and ensure the comparison table is highlighted in the main text. revision: yes
Referee: [Abstract] Abstract/Results: The statements that Laya 'achieve[s] the strongest mean clinical accuracy' and shows 'particular gains on tasks where relevant neural patterns are subtle' lack any numerical values, error bars, dataset sizes, task names, or statistical comparisons. Without these, the magnitude and reliability of the reported improvements cannot be assessed.

Authors: We acknowledge that the abstract would be strengthened by including representative quantitative results. The manuscript reports these in Section 5, Tables 3-4, and Figure 4, with mean accuracy across clinical tasks, standard deviations, dataset sizes, specific task names, and statistical comparisons. In the revised abstract we will incorporate key summary statistics (e.g., mean accuracy and gains on subtle-pattern tasks) while maintaining conciseness, and we will ensure error bars and significance tests remain clearly visible in the results section. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical claims rest on external experimental comparisons

full rationale

The paper advances an empirical hypothesis that latent prediction (LeJEPA) produces better EEG representations than reconstruction (MAE) for clinical tasks. This is tested via frozen linear probing accuracies and stated controlled ablations that hold architecture and data fixed while varying only the loss. No equations, derivations, or first-principles results appear that reduce to fitted parameters or self-definitions by construction. No load-bearing self-citations or uniqueness theorems are invoked to force the result. The central claim therefore remains falsifiable against external benchmarks and does not collapse into its own inputs.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on the domain assumption that reconstruction objectives bias toward artifacts; no free parameters or invented entities are introduced in the abstract.

axioms (1)

domain assumption Reconstruction-based SSL biases representations toward high-variance artifacts rather than task-relevant neural structure.
Stated as the motivating hypothesis for switching to latent prediction.

pith-pipeline@v0.9.0 · 5549 in / 1101 out tokens · 36875 ms · 2026-05-15T09:35:59.003830+00:00 · methodology

discussion (0)

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

IndisputableMonolith/Cost/FunctionalEquation.lean washburn_uniqueness_aczel echoes

?

echoes
ECHOES: this paper passage has the same mathematical shape or conceptual pattern as the Recognition theorem, but is not a direct formal dependency.

We hypothesize that one contributing factor is the reliance on signal reconstruction as the primary self-supervised learning (SSL) objective, which biases representations toward high-variance artifacts rather than task-relevant neural structure... latent prediction objectives are better for learning EEG representations, naturally filtering out irrelevant noise.
IndisputableMonolith/Foundation/BranchSelection.lean branch_selection echoes

?

echoes
ECHOES: this paper passage has the same mathematical shape or conceptual pattern as the Recognition theorem, but is not a direct formal dependency.

LeJEPA addresses these limitations by explicitly regularizing the geometry of the latent space... encouraging embeddings within a batch to be diverse and approximately isotropic

What do these tags mean?

matches: The paper's claim is directly supported by a theorem in the formal canon.
supports: The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends: The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses: The paper appears to rely on the theorem as machinery.
contradicts: The paper's claim conflicts with a theorem or certificate in the canon.
unclear: Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

Sonata: A Hybrid World Model for Inertial Kinematics under Clinical Data Scarcity
cs.LG 2026-04 unverdicted novelty 6.0

Sonata is a small hybrid world model pre-trained to predict future IMU states that outperforms autoregressive baselines on clinical discrimination, fall-risk prediction, and cross-cohort transfer while fitting on-devi...