NeuroWorld: A Latent Brain World Model for Stimulus-Conditioned Human Brain Dynamics

Jan Paolo Macapinlac Balagtas; Jianxiong Zhou; Juan Helen Zhou; Kwun Kei Ng; Zhizhou Li; Zijian Dong; Zijiao Chen

REVIEW 4 major objections 4 minor 32 references

NeuroWorld: A Latent Brain World Model for Stimulus-Conditioned Human Brain Dynamics

T0 review · 4 major / 4 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read NeuroWorld casts naturalistic brain activity prediction as stimulus-conditioned evolution in a learned latent state space, and reports state-of-the-art multi-step rollout on three movie-fMRI benchmarks under strictly causal stimulus access.

desk verdict A well-engineered two-stage latent world model for fMRI forecasting with a genuinely new dataset, but the 'strictly causal' claim is undercut by bidirectional stimulus encoders and an unaddressed leakage risk in the SG-MIND split. read the letter →

arxiv 2608.01773 v1 pith:DZCA2CGK submitted 2026-08-03 q-bio.NC

Zijian Dong , Jianxiong Zhou , Kwun Kei Ng , Jan Paolo Macapinlac Balagtas , Zhizhou Li , Zijiao Chen , Juan Helen Zhou This is my paper

classification q-bio.NC

keywords brainworldmodelfMRIencodingnaturalisticmoviestimulicausalrolloutlatentdynamicsstimulus-conditionedpredictionSG-MINDdatasetwhole-braindecoding

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that forecasting human brain activity during naturalistic movie viewing should be treated as a causal world-modeling problem, not as stimulus-to-response regression. It introduces NeuroWorld, which learns a latent brain state from fMRI and a stimulus-conditioned transition that predicts the next latent state without ever seeing future stimuli, then decodes rolled-out latent trajectories back to whole-brain responses. On three movie-fMRI benchmarks, including a newly collected 20-participant SG-MIND dataset with 8,519 paired clips and 140.7 person-hours of viewing, NeuroWorld reports state-of-the-art multi-step rollout accuracy under strictly causal stimulus access and smaller drift as the horizon grows. If right, the work establishes that high retrospective encoding accuracy is not enough for simulation: reliable forecasting requires a representation optimized for transition stability, and it opens a path to studying how alternative stimulus histories would shape brain trajectories.

What carries the argument

The central object is the latent brain state z_t = E_phi(x_t), a low-dimensional vector per fMRI TR learned jointly with a causal predictor P_theta. Multimodal movie features are encoded into action tokens a_t and injected through adaptive layer normalization; P_theta is trained by next-latent prediction with SIGReg regularization, with no fMRI reconstruction objective. A second stage freezes this dynamics, rolls the latent forward from an observed prefix, and trains a shared decoder trunk with subject-specific linear heads to map the trajectory to ROI-level BOLD. This separation of state learning from observation decoding is what carries the causal-rollout claim.

What would settle it

Inspect the SG-MIND train/validation split for overlapping movie clips across participants: if any validation clip also appeared in training for another subject, retrain after excluding it and see whether NeuroWorld's rollout advantage over the TRIBE/MIRAGE baselines persists. A complementary test is to add an fMRI reconstruction loss to the latent dynamics stage; if that variant matches NeuroWorld's long-horizon rollout, the no-reconstruction design is not what carries the result.

Watch

Extended reading notes

Core claim

The paper's central claim is that naturalistic brain dynamics can be modeled as stimulus-conditioned evolution in a learned latent state space, and that this formulation forecasts better than regressing stimuli onto fMRI. NeuroWorld learns the latent state by next-state prediction without reconstructing the observed fMRI signal, rolls the latent forward under past-only stimulus access, and decodes it into subject-specific whole-brain responses. Across Algonauts 2025, CineBrain, and the new SG-MIND benchmark, the paper reports state-of-the-art causal rollout performance and greater robustness to long-horizon autoregressive drift, supporting the premise that encoding accuracy alone does not ma

Load-bearing premise

The reported generalization depends on validation clips being truly absent from training for every participant, but the paper does not explain how the shared pool of clips is kept out of training for all subjects when a clip appears in validation for one.

Editorial extensions

If this is right

Whole-brain responses can be simulated for 20 to 100 TRs from a single observed fMRI prefix using only past stimuli, so future movie content cannot influence forecasts.
Retrospective encoding accuracy is decoupled from forecasting skill: a simulator needs a representation and transition trained explicitly for next-state prediction.
The latent world model is shared across subjects while only a linear readout is subject-specific, localizing individual differences at the decoding stage.
Generalization to a held-out movie season and gradual drift at long horizons imply the dynamics capture general stimulus–response structure rather than memorized clips.
Modality ablations show visual and audio streams dominate the forecast, with text adding complementary predictive signal in higher-order association regions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Because the dynamics are frozen and shared, the same latent world model might transfer to unseen participants by fitting only a new linear head; the paper trains only on seen subjects, but the architecture makes this a cheap test.
If the transition is genuinely stimulus-conditioned, the framework can answer counterfactual questions about altered sensory histories, a direction the paper names but does not demonstrate.
The SG-MIND shared pool of 291 clips permits cross-subject comparisons on identical stimuli; analyzing trajectories from different participants watching the same clip would separate stimulus-locked from subject-specific dynamics, which the paper does not report.
A direct ablation varying the reconstruction objective under identical compute would test the paper's central design claim that 'no fMRI reconstruction' is what stabilizes rollout; the paper contrasts with BrainVista but does not run this within NeuroWorld.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

I read NeuroWorld with the SG-MIND dataset. The core idea is genuinely new: learn a latent state via next-latent prediction without fMRI reconstruction, then freeze it and train a decoder to roll out trajectories. That two-stage design is a real departure from BrainVista and BrainWorld, and the causal evaluation protocol (multi-step rollout, masked baselines, season-6 split) is more demanding than what's typical in this literature. The new dataset, SG-MIND, is a solid contribution if it's actually released—20 participants, 140+ hours of viewing, carefully acquired.

The math and writing are clean. I have no issue with the formulation itself.

But the central claim—'strictly causal stimulus access'—doesn't hold up under inspection. The stimulus-action encoder A_α is never constrained to be causal. The features come from Video-JEPA 2, Wav2Vec-Bert 2.0, and Llama-3.2, and the first two are bidirectional. The paper doesn't describe any causal masking or future-free chunking of these encoders. Once you mean-pool and shift by the hemodynamic lag, an action token at time t can encode stimulus content from after time t if the encoder window exceeds the lag. That directly undermines the headline property, and it's the 'strict' in 'strictly causal' that matters. It's true the baselines likely share the same leaky features, so the comparison might be fair, but then the paper shouldn't claim strict causality as its distinguishing feature.

Second, the SG-MIND split: the paper asserts validation stimuli were held out, but with a common pool of 291 clips across subjects, an 80/20 run split can put the same clip in training for one subject and validation for another. The shared action encoder and transition model could memorize clip-specific response patterns. No description of how this was avoided. This is a real concern for the SG-MIND numbers.

Third, the empirical section has no error bars, one fixed split, and no code or data. The gap on CineBrain and SG-MIND is large, but without variance estimates or repeated splits, 'state-of-the-art' is not established.

The interpretability analyses are a plus, especially the Mismatch/Oracle control, which shows the stimulus conditioning has real effect.

Bottom line: the paper is worth reading and worth a serious referee. But the authors need to fix or at least transparently address the stimulus-encoder causality and the SG-MIND splitting, and they should release code and data. If the causality issue can't be resolved, the claims should be softened to 'causal in the transition model' rather than 'strictly causal stimulus access.' I'd engage with it and push for revision, not rejection.

Referee Report

4 major / 4 minor

Summary. The paper introduces NeuroWorld, a two-stage latent world model for forecasting whole-brain fMRI responses during naturalistic movie viewing. Stage 1 (LDL) learns an fMRI encoder and a causal, action-conditioned latent transition via next-latent prediction without an fMRI reconstruction loss, regularized by SIGReg. Stage 2 (LRD) freezes these components, rolls latent states forward autoregressively from an observed fMRI prefix, and decodes them to subject-specific ROI-level fMRI with a shared decoder and per-subject linear heads. The model is evaluated on Algonauts 2025, CineBrain, and a newly collected SG-MIND dataset (20 participants, 8,519 clips), reporting state-of-the-art multi-step rollout performance under a claimed strictly causal stimulus-access protocol, together with ablations, horizon scaling, and interpretability analyses.

Significance. If the empirical claims are sustained, NeuroWorld is a meaningful conceptual advance: it replaces stimulus-to-response regression with an explicit, transition-sufficient latent state optimized for recursive forecasting, and it provides a larger cross-subject benchmark (SG-MIND) that the community can reuse. The paper also ships a useful set of ablations and interpretability analyses. However, the central claim of 'strictly causal stimulus access' is not yet supported, and the statistical evidence for state-of-the-art performance is incomplete. These issues are fixable but currently block acceptance.

major comments (4)

[§IV.A.3 and Eq. (2)-(5)] The central claim of 'strictly causal stimulus access' is not established for the stimulus-action encoder A_α. The paper states that action tokens are derived from frozen pretrained Wav2Vec-Bert 2.0 and Video-JEPA 2 features, but these encoders are bidirectional/full-clip and can encode future stimulus content. Even after the hemodynamic shift, an action token used at TR t may contain information from stimulus times > t. The paper only states that P_θ is causal (Eq. 5); no causal masking or future-free extraction is described for the stimulus features themselves. This concern applies to all three benchmarks and undermines the core methodological claim. Please specify the exact temporal window and causality enforcement used in feature extraction, or rerun the evaluation with provably causal features.
[Tables II and III] The headline results are reported as single point estimates with no error bars, confidence intervals, or significance tests. The evaluation uses one fixed 80/20 split per dataset, so it is impossible to assess whether NeuroWorld's advantages over baselines are statistically reliable, especially given the small cohorts (Algonauts n=4, CineBrain n=6). I request per-subject or bootstrapped variances, multiple splits, and paired significance tests for at least the main comparison in Table II.
[§IV.A.2 and SG-MIND split] For SG-MIND the paper states that 'validation stimuli were held out from training to prevent data leakage,' but SG-MIND uses a common pool of 291 clips shared across participants. The paper does not describe how a clip that appears in validation for one subject is excluded from training for all other subjects. If this cross-subject clip exclusion is not implemented, the model can memorize stimulus-specific response patterns from other participants, inflating the reported SG-MIND results. Please provide the exact split protocol, e.g., clip-level partitioning across all subjects, or a clarification if the current split is at the run level only.
[§IV.B and Table II] The causal masking of baseline models is described only as 'we modified the transformer blocks in encoders with causal masks.' This is insufficiently specified to ensure a fair comparison. If, for example, the temporal context length of TRIBE/MIRAGE is truncated or their cross-attention is masked in a way that cripples their capacity, the comparison could be biased. Please provide the exact masking procedure, the modified architecture, and ideally a sensitivity analysis showing the baselines under their original bidirectional setting (even if that setting is not causal) to quantify the cost of the causal constraint.

minor comments (4)

[Fig. 6] The 'Mismatch' condition is not defined in the text. Please specify how stimulus–brain alignment is broken (e.g., time-shuffled actions, random permutation, or across-clip mismatch) and whether the same initial state is used.
[Table I / §IV.A.3] For CineBrain, Δ=0 is stated with no justification, despite a TR of 0.8 s. If the dataset is already aligned to the hemodynamic response, please state this explicitly; otherwise the zero lag may be questionable.
[Table II footnote] The BrainVista results are quoted from the original paper and not reproduced under the protocol. This is acknowledged, but the numbers should not appear in the main comparison table without a clear visual separation, as they are not directly comparable.
[General] The paper says code will be made public upon acceptance. Since reproducibility is a strength of this line of work, consider releasing the split definitions and feature extraction scripts along with the final version.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: NeuroWorld's core result is an empirical evaluation of a learned latent dynamics model; no prediction reduces to a fitted input or to a self-citation chain.

full rationale

The paper's central claim is empirical: LDL trains the fMRI encoder E_phi and predictor P_theta with a next-latent prediction loss (Eq. 7), then LRD freezes those components and trains a decoder on autoregressively rolled-out latents to predict fMRI (Eq. 13). The evaluation on held-out validation runs (Section IV.B) measures prediction of actual fMRI responses, not a quantity that was used as a training target of the same fitted parameters. The latent state z_t = E_phi(x_t) is learned from observed fMRI, and the transition target is z_{t+1} = E_phi(x_{t+1}); this is self-supervised representation learning, not circular, because the predictor never receives the future latent as an input at test time. The LRD decoder is supervised with actual fMRI, but that is standard supervised decoding rather than a fit renamed as a prediction. Self-citations to Brain-JEPA [9] and Brain Harmonix [13] appear only in related work and design motivation; the state-of-the-art claim is supported by direct comparisons on three benchmarks, not by those citations. No uniqueness theorem or ansatz is imported from the authors' prior work, and no equation in the paper is equivalent to its inputs by construction. The external concerns about strict causality of frozen stimulus encoders and potential validation-stimulus leakage across subjects are correctness/robustness issues, not circularity: they do not make the reported performance equal to a training input by construction. Therefore, no circular step is identified, and the circularity score is 0.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central contribution rests on the latent-state formulation and the empirical evaluation. The ledger lists hand-tuned constants, unreported architecture choices, and domain assumptions about fMRI latency, stimulus features, and split leakage. The paper introduces no new physical entities.

free parameters (3)

SIGReg strength λ
Eq. 9 regularization weight; no value reported; tuned to balance next-latent prediction and latent collapse prevention.
Hemodynamic lag Δ per dataset = 3 TR (Algonauts), 6 TR (SG-MIND), 0 (CineBrain)
Chosen from physiological delay (~4.4 s), not fitted to the target result, but alignment of stimulus to response is central to the causal setting.
Latent dimension d_z and action dimension d_a
Not reported; architecture choices that determine model capacity and are selected by the authors without stated justification.

assumptions (5)

domain assumption BOLD fMRI at ROI level is a sufficient observable: a latent state z_t = E_phi(x_t) containing all information needed to predict future brain responses given stimulus history.
Section III.C; the transition model P_theta assumes a transition-sufficient latent state. This is the core premise of the world model and is not independently validated.
domain assumption Stimulus features from frozen pretrained models (Video-JEPA 2, Wav2Vec-Bert-2.0, Llama-3.2-3B) adequately represent the exogenous drive to the brain.
Section IV.A.3; the model conditions on these features and does not learn them; failures in these representations would limit any forecasting model.
domain assumption Fixed hemodynamic delay and TR-grid alignment are correct for each dataset.
Section IV.A.3; misalignment would break the causal mapping from stimulus to response and could artificially inflate or deflate performance.
domain assumption The validation split prevents stimulus leakage across subjects.
Section IV.A.2; the authors claim held-out stimuli but do not specify how shared clips are removed from all subjects' training sets.
standard math Standard calculus and stochastic optimization; the losses are standard MSE and a sketched normality statistic.
The method uses standard neural network training and does not rely on nonstandard mathematical facts.

how reviews work

0 comments

Cite this review

Pith. "Pith review of NeuroWorld: A Latent Brain World Model for Stimulus-Conditioned Human Brain Dynamics." pith.science (2026). https://pith.science/paper/DZCA2CGK

@misc{pith2026260801773,
  author       = {Pith},
  title        = {Pith review of: NeuroWorld: A Latent Brain World Model for Stimulus-Conditioned Human Brain Dynamics},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DZCA2CGK}},
  note         = {Machine review of arXiv:2608.01773}
}

read the original abstract

Forecasting human brain activity during naturalistic experience requires modeling how endogenous neural states evolve causally under continuous sensory drive. Existing brain encoding models instead frame this as stimulus-to-response regression without strict temporal constraints, allowing future stimuli to leak into current predictions. We introduce NeuroWorld, to our knowledge the first brain world model, which casts naturalistic brain functional dynamics prediction as stimulus-conditioned evolution in a learned latent brain-state space, separating endogenous states (measured via fMRI) from exogenous multimodal stimuli across two stages. Latent Dynamics Learning (LDL) jointly learns a transition-sufficient representation and causal dynamics through next-latent prediction, without reconstructing the observed fMRI signal. Latent Rollout Decoding (LRD) freezes LDL, autoregressively rolls latent states forward from an observed fMRI prefix, and decodes them into subject-specific whole-brain responses. Across three naturalistic movie-fMRI benchmarks spanning 30 participants, including our newly collected Singapore Multimodal Imaging & Naturalistic Dataset (SG-MIND; 20 participants, 8,519 paired stimulus-response clips, 140.7 person-hours of viewing), NeuroWorld achieves state-of-the-art multi-step rollout performance under strictly causal stimulus access, with greater robustness to long-horizon autoregressive drift, supporting reliable simulation of extended brain-state trajectories. Extensive interpretability analyses characterize the functional organization of the learned dynamics, establishing latent-space world modeling as a principled framework for causal forecasting of human brain activity.

Figures

Figures reproduced from arXiv: 2608.01773 by the authors.

**Figure 1.** Conceptual comparison between conventional bidirectional paradigms and NeuroWorld. Left: Bidirectional attention permits future-stimulus leakage and learns brain representations with a reconstruction-oriented objective. Right: NeuroWorld enforces causal stimulus access and learns transition-sufficient latent brain states through next-latent prediction for autoregressive rollout. at - stimulus token; Zt - latent br… view at source ↗

**Figure 2.** Overview of the SG-MIND Left: SG-MIND expands the cohort size to 20 participants, compared with four in Algonauts 2025 and six in CineBrain. Right: Sampled temporally aligned video, audio, and cortical fMRI signals from SG-MIND. is shaped jointly by the sensory evidence received so far (exogenous drive) and the state propagated from earlier activity (endogenous dynamics), not by stimuli yet to arrive [4]–[7]. Recent… view at source ↗

**Figure 3.** Overview of NeuroWorld’s two-stage framework. Left: Latent Dynamics Learning (LDL) jointly learns the fMRI encoder Eϕ and predictor Pθ through next-latent prediction with SIGReg, without an fMRI reconstruction objective. Right: Latent Rollout Decoding (LRD) freezes the learned world model, autoregressively rolls latent states forward under causal stimulus conditioning, and trains a shared decoder Dω with subject-spe… view at source ↗

Figures from the paper (5 more)

**Figure 4.** Figure 4: Cortical distribution of NeuroWorld’s prediction performance on Algonauts 2025, with higher correlations concentrated in bilateral occipital and posterior temporal regions. at most 0.7619 on the season-6 split. The contrast is far more pronounced on the two harder ben…

**Figure 6.** Figure 6: Dissecting NeuroWorld’s predictive power. Aligned stimuli and accurate latent transitions jointly support autoregressive rollout. Pearson correlations for Mismatch, the main model (20-step autoregressive rollout), and the teacher-forced Oracle. Points denote ROIs. a c…

**Figure 7.** Figure 7: Cognitive decoding of stimulus-brain alignment gains. A: ROI-wise differences in Pearson’s r between aligned and Mismatch conditions, with the top 20 cognitive terms for representative high-gain parcels. B: Cognitive terms showing significant whole-map correspondence w…

**Figure 8.** Figure 8: Cognitive decoding of multimodal gains. A: ROI-wise differences in Pearson’s r between the multimodal and best unimodal models, with the top 20 cognitive terms for representative high-gain parcels. B: Cognitive terms showing significant whole-map correspondence with th…

**Figure 9.** Figure 9: Cortical mosaic of relative unimodal encoding profiles. Colors encode relative ROI-wise predictive performance based on validation-set Pearson’s (r), for the text-only (red), audio-only (green), and video-only (blue) models. Blended colors indicate overlapping modality…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

32 extracted references · 20 canonical work pages

[1]

Tribe: Trimodal brain encoder for whole-brain fmri response prediction,

S. d’Ascoli, J. Rapin, Y . Benchetrit, H. Banville, and J.-R. King, “Tribe: Trimodal brain encoder for whole-brain fmri response prediction,”arXiv preprint arXiv:2507.22229, 2025

arXiv 2025
[2]

A foundation model of vision, audition, and language for in-silico neuroscience,

S. d’Ascoli, J. Rapin, Y . Benchetrit, T. Brooks, K. Begany, J. Raugel, H. Banville, and J.-R. King, “A foundation model of vision, audition, and language for in-silico neuroscience,”arXiv preprint arXiv:2605.04326, 2026

arXiv 2026
[3]

MIRAGE: Adaptive Multimodal Gating for Whole-Brain fMRI Encoding

A. Gokce, B. AlKhamissi, and M. Schrimpf, “Mirage: Adaptive multimodal gating for whole-brain fmri encoding,”arXiv preprint arXiv:2605.29850, 2026

work page Pith review arXiv 2026
[4]

A theory of cortical responses,

K. Friston, “A theory of cortical responses,”Philosophical transactions of the Royal Society B: Biological sciences, vol. 360, no. 1456, p. 815, 2005

work page 2005
[5]

A hierarchy of temporal receptive windows in human cortex,

U. Hasson, E. Yang, I. Vallines, D. J. Heeger, and N. Rubin, “A hierarchy of temporal receptive windows in human cortex,”Journal of neuroscience, vol. 28, no. 10, pp. 2539–2550, 2008

work page 2008
[6]

Slow cortical dynamics and the accumulation of information over long timescales,

C. J. Honey, T. Thesen, T. H. Donner, L. J. Silbert, C. E. Carlson, O. Devinsky, W. K. Doyle, N. Rubin, D. J. Heeger, and U. Hasson, “Slow cortical dynamics and the accumulation of information over long timescales,”Neuron, vol. 76, no. 2, pp. 423–434, 2012

work page 2012
[7]

A hierarchy of intrinsic timescales across primate cortex,

J. D. Murray, A. Bernacchia, D. J. Freedman, R. Romo, J. D. Wallis, X. Cai, C. Padoa-Schioppa, T. Pasternak, H. Seo, D. Leeet al., “A hierarchy of intrinsic timescales across primate cortex,”Nature neuroscience, vol. 17, no. 12, pp. 1661–1663, 2014

work page 2014
[8]

Brainvista: Modeling naturalistic brain dynamics as multimodal next-token prediction,

X. Yin, R. Zhao, L. Yao, and W. Cai, “Brainvista: Modeling naturalistic brain dynamics as multimodal next-token prediction,”arXiv preprint arXiv:2602.04512, 2026

work page arXiv 2026

Show all 32 references

[9]

Brain-jepa: Brain dynamics foundation model with gradient positioning and spatiotemporal masking,

Z. Dong, R. Li, Y . Wu, T. T. Nguyen, J. S. Chong, F. Ji, N. R. Tong, C. L. Chen, and J. H. Zhou, “Brain-jepa: Brain dynamics foundation model with gradient positioning and spatiotemporal masking,”Advances in Neural Information Processing Systems, vol. 37, pp. 86 048–86 073, 2024

2024
[10]

The algonauts project 2025 challenge: How the human brain makes sense of multimodal movies,

A. T. Gifford, D. Bersch, M. St-Laurent, B. Pinsard, J. Boyle, L. Bellec, A. Oliva, G. Roig, and R. M. Cichy, “The algonauts project 2025 challenge: How the human brain makes sense of multimodal movies,” arXiv preprint arXiv:2501.00504, 2024

2025 arXiv
[11]

Cinebrain: A large- scale multi-modal brain dataset during naturalistic audiovisual narrative processing,

J. Gao, Y . Liu, B. Yang, J. Feng, and Y . Fu, “Cinebrain: A large- scale multi-modal brain dataset during naturalistic audiovisual narrative processing,”arXiv preprint arXiv:2503.06940, 2025

2025
[12]

Brainlm: A foundation model for brain activity recordings,

J. Ortega Caro, A. H. de Oliveira Fonseca, S. Rizvi, M. Rosati, C. Averill, J. Cross, P. Mittal, E. Zappala, R. Dhodapkar, C. Abdallah et al., “Brainlm: A foundation model for brain activity recordings,” in International Conference on Learning Representations, vol. 2024, 2024,...

2024
[13]

Brain harmony: a multimodal foundation model unifying morphology and function into 1d tokens,

Z. Dong, R. Li, J. Chong, N. Dehestani, Y . Teng, Y . Lin, Z. Li, Y . Zhang, Y . Xie, L. Ooiet al., “Brain harmony: a multimodal foundation model unifying morphology and function into 1d tokens,”Advances in Neural Information Processing Systems, vol. 38, pp. 122 100–122 127, 2026

2026
[14]

Towards a general-purpose foundation model for fmri analysis,

C. Wang, Y . Jiang, Z. Peng, C. Li, C. Bang, L. Zhao, W. Fu, J. Lv, J. Sepulcre, C. Yanget al., “Towards a general-purpose foundation model for fmri analysis,”arXiv preprint arXiv:2506.11167, 2025

2025
[15]

Brainworld: A structural-prior-conditioned generative model for whole-brain 4d fmri dynamics,

J. Xia, W. Ye, J. Zhang, X. Pan, M. Wang, and Q. Liu, “Brainworld: A structural-prior-conditioned generative model for whole-brain 4d fmri dynamics,”arXiv preprint arXiv:2606.17742, 2026

2026 arXiv
[16]

Scalable diffusion models with transformers,

W. Peebles and S. Xie, “Scalable diffusion models with transformers,” inProceedings of the IEEE/CVF international conference on computer vision, 2023, pp. 4195–4205

2023
[17]

Lejepa: Provable and scalable self-supervised learning without the heuristics,

R. Balestriero and Y . LeCun, “Lejepa: Provable and scalable self-supervised learning without the heuristics,”arXiv preprint arXiv:2511.08544, 2025

2025 arXiv
[18]

Leworld- model: Stable end-to-end joint-embedding predictive architecture from pixels,

L. Maes, Q. L. Lidec, D. Scieur, Y . LeCun, and R. Balestriero, “Leworld- model: Stable end-to-end joint-embedding predictive architecture from pixels,”arXiv preprint arXiv:2603.19312, 2026

2026 arXiv
[19]

Methods to detect, characterize, and remove motion artifact in resting state fmri,

J. D. Power, A. Mitra, T. O. Laumann, A. Z. Snyder, B. L. Schlaggar, and S. E. Petersen, “Methods to detect, characterize, and remove motion artifact in resting state fmri,”neuroimage, vol. 84, pp. 320–341, 2014

2014
[20]

V-jepa 2: Self- supervised video models enable understanding, prediction and planning,

M. Assran, A. Bardes, D. Fan, Q. Garrido, R. Howes, M. Muckley, A. Rizvi, C. Roberts, K. Sinha, A. Zholuset al., “V-jepa 2: Self- supervised video models enable understanding, prediction and planning,” arXiv preprint arXiv:2506.09985, 2025

2025 arXiv
[21]

W2v-bert: Combining contrastive learning and masked language mod- eling for self-supervised speech pre-training,

Y .-A. Chung, Y . Zhang, W. Han, C.-C. Chiu, J. Qin, R. Pang, and Y . Wu, “W2v-bert: Combining contrastive learning and masked language mod- eling for self-supervised speech pre-training,” in2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU). IEEE, 2021, ...

2021
[22]

The llama 3 herd of models,

A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Vaughanet al., “The llama 3 herd of models,”arXiv preprint arXiv:2407.21783, 2024

2024 arXiv
[23]

Local-global parcellation of the human cerebral cortex from intrinsic functional connectivity mri,

A. Schaefer, R. Kong, E. M. Gordon, T. O. Laumann, X.-N. Zuo, A. J. Holmes, S. B. Eickhoff, and B. T. Yeo, “Local-global parcellation of the human cerebral cortex from intrinsic functional connectivity mri,” Cerebral cortex, vol. 28, no. 9, pp. 3095–3114, 2018

2018
[24]

The organization of the human cerebral cortex estimated by intrinsic functional connectivity,

B. T. Yeo, F. M. Krienen, J. Sepulcre, M. R. Sabuncu, D. Lashkari, M. Hollinshead, J. L. Roffman, J. W. Smoller, L. Z ¨ollei, J. R. Polimeni et al., “The organization of the human cerebral cortex estimated by intrinsic functional connectivity,”Journal of neurophysiology, 2011

2011
[25]

Integration of auditory and visual information about objects in superior temporal sulcus,

M. S. Beauchamp, K. E. Lee, B. D. Argall, and A. Martin, “Integration of auditory and visual information about objects in superior temporal sulcus,”Neuron, vol. 41, no. 5, pp. 809–823, 2004

2004
[26]

Intersubject synchronization of cortical activity during natural vision,

U. Hasson, Y . Nir, I. Levy, G. Fuhrmann, and R. Malach, “Intersubject synchronization of cortical activity during natural vision,”science, vol. 303, no. 5664, pp. 1634–1640, 2004

2004
[27]

The morphospace of the brain- cognition organisation,

V . Pacella, V . Nozais, L. Talozzi, M. Abdallah, D. Wassermann, S. J. Forkel, and M. Thiebaut de Schotten, “The morphospace of the brain- cognition organisation,”Nature Communications, vol. 15, no. 1, p. 8452, 2024

2024
[28]

Neuromaps: structural and functional interpretation of brain maps,

R. D. Markello, J. Y . Hansen, Z.-Q. Liu, V . Bazinet, G. Shafiei, L. E. Su´arez, N. Blostein, J. Seidlitz, S. Baillet, T. D. Satterthwaiteet al., “Neuromaps: structural and functional interpretation of brain maps,” Nature methods, vol. 19, no. 11, pp. 1472–1479, 2022

2022
[29]

Nimare: neuroimaging meta-analysis research environment,

T. Salo, T. Yarkoni, T. E. Nichols, J.-B. Poline, M. Bilgel, K. L. Bottenhorn, D. Jarecka, J. D. Kent, A. Kimbler, D. M. Nielsonet al., “Nimare: neuroimaging meta-analysis research environment,”Aperture Neuro, vol. 3, pp. 1–32, 2023

2023
[30]

Parallel and distributed encoding of speech across human auditory cortex,

L. S. Hamilton, Y . Oganian, J. Hall, and E. F. Chang, “Parallel and distributed encoding of speech across human auditory cortex,”Cell, vol. 184, no. 18, pp. 4626–4639, 2021

2021
[31]

How does the “default mode

L. Fernandino and J. R. Binder, “How does the “default mode” network contribute to semantic cognition?”Brain and language, vol. 252, p. 105405, 2024

2024
[32]

Diverse frontoparietal connectivity supports semantic prediction and in- tegration in sentence comprehension,

Y . He, X. Shao, C. Liu, C. Fan, E. Jefferies, M. Zhang, and X. Li, “Diverse frontoparietal connectivity supports semantic prediction and in- tegration in sentence comprehension,”Journal of neuroscience, vol. 45, no. 5, 2025

2025

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.

[1] [1]

Tribe: Trimodal brain encoder for whole-brain fmri response prediction,

S. d’Ascoli, J. Rapin, Y . Benchetrit, H. Banville, and J.-R. King, “Tribe: Trimodal brain encoder for whole-brain fmri response prediction,”arXiv preprint arXiv:2507.22229, 2025

arXiv 2025

[2] [2]

A foundation model of vision, audition, and language for in-silico neuroscience,

S. d’Ascoli, J. Rapin, Y . Benchetrit, T. Brooks, K. Begany, J. Raugel, H. Banville, and J.-R. King, “A foundation model of vision, audition, and language for in-silico neuroscience,”arXiv preprint arXiv:2605.04326, 2026

arXiv 2026

[3] [3]

MIRAGE: Adaptive Multimodal Gating for Whole-Brain fMRI Encoding

A. Gokce, B. AlKhamissi, and M. Schrimpf, “Mirage: Adaptive multimodal gating for whole-brain fmri encoding,”arXiv preprint arXiv:2605.29850, 2026

work page Pith review arXiv 2026

[4] [4]

A theory of cortical responses,

K. Friston, “A theory of cortical responses,”Philosophical transactions of the Royal Society B: Biological sciences, vol. 360, no. 1456, p. 815, 2005

work page 2005

[5] [5]

A hierarchy of temporal receptive windows in human cortex,

U. Hasson, E. Yang, I. Vallines, D. J. Heeger, and N. Rubin, “A hierarchy of temporal receptive windows in human cortex,”Journal of neuroscience, vol. 28, no. 10, pp. 2539–2550, 2008

work page 2008

[6] [6]

Slow cortical dynamics and the accumulation of information over long timescales,

C. J. Honey, T. Thesen, T. H. Donner, L. J. Silbert, C. E. Carlson, O. Devinsky, W. K. Doyle, N. Rubin, D. J. Heeger, and U. Hasson, “Slow cortical dynamics and the accumulation of information over long timescales,”Neuron, vol. 76, no. 2, pp. 423–434, 2012

work page 2012

[7] [7]

A hierarchy of intrinsic timescales across primate cortex,

J. D. Murray, A. Bernacchia, D. J. Freedman, R. Romo, J. D. Wallis, X. Cai, C. Padoa-Schioppa, T. Pasternak, H. Seo, D. Leeet al., “A hierarchy of intrinsic timescales across primate cortex,”Nature neuroscience, vol. 17, no. 12, pp. 1661–1663, 2014

work page 2014

[8] [8]

Brainvista: Modeling naturalistic brain dynamics as multimodal next-token prediction,

X. Yin, R. Zhao, L. Yao, and W. Cai, “Brainvista: Modeling naturalistic brain dynamics as multimodal next-token prediction,”arXiv preprint arXiv:2602.04512, 2026

work page arXiv 2026