Nonlocal operator learning for fMRI encoding and decoding tasks
Pith reviewed 2026-05-21 08:05 UTC · model grok-4.3
The pith
Neural integral operators model fMRI dynamics by capturing nonlocal spatiotemporal context through latent fixed-point iterations.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The authors establish that a latent neural integral operator framework, which performs fixed-point iterations in an auxiliary space before classification or prediction, provides a workable approach for fMRI encoding and decoding tasks, and that expanding the spatiotemporal context through longer temporal windows and whole-brain recordings produces measurable gains in accuracy together with more structured latent geometry compared with shorter or more local inputs.
What carries the argument
Latent neural integral operator that performs fixed-point iterations in an auxiliary space to capture nonlocal spatiotemporal dependencies.
If this is right
- Larger temporal windows produce consistent performance gains in both decoding and encoding across the tested datasets.
- Whole-brain recordings support better results than visual-cortex-only inputs for these operator-based models.
- The learned latent space frequently shows sharper separation between stimulus classes than the raw fMRI data in decoding experiments.
- Longer contexts lead to more structured geometry in the auxiliary latent representations.
Where Pith is reading between the lines
- The same nonlocal operator style could be tested on other distributed neural recordings such as EEG or intracranial signals to check whether the context benefit generalizes.
- If the latent iterations remain stable at scale, the approach might support real-time decoding systems that integrate information across longer histories than current local models allow.
- Architectures that explicitly iterate in auxiliary space may offer a route to reduce the sample complexity of learning distributed brain dynamics compared with purely local convolutional or recurrent networks.
Load-bearing premise
Fixed-point iterations inside the auxiliary latent space can reliably locate and use the nonlocal spatiotemporal dependencies present in fMRI recordings.
What would settle it
Running the same decoding and encoding tasks on the same datasets but with a non-integral baseline model or with fixed short windows and finding equal or higher accuracy plus equally structured representations would indicate the integral operator and broader context add no advantage.
Figures
read the original abstract
Functional MRI data exhibit high-dimensional spatiotemporal structure, making both prediction and decoding challenging. In this work, we investigate neural integral-operator-based models for encoding and decoding tasks in fMRI, with particular emphasis on the role of nonlocal spatiotemporal context. We implement a latent neural integral operator framework that performs fixed point iterations in an auxiliary space from which classification and stimuli prediction is performed via a decoder. We evaluate our model on two open-source fMRI datasets. Our experiments examine both decoding of stimuli from fMRI recordings and encoding of fMRI dynamics from stimulus representations. A main focus is the effect of spatiotemporal context: we systematically compare short and long temporal windows, as well as the use of visual cortex vs whole brain recordings, and analyze their influence on performance and latent-space geometry. Across tasks and datasets, larger temporal windows generally improve results and produce more structured learned representations. In decoding experiments, the learned latent space often provides clearer class separation than the raw data. In encoding experiments, although absolute performance remains moderate due to the difficulty of the task, longer temporal windows still yield consistent gains. These findings suggest that neural integral operators provide a promising framework for modeling fMRI dynamics and that broader spatiotemporal context can be beneficial for both prediction and representation learning. More broadly, the results indicate that exploiting distributed nonlocal structure in brain dynamics requires model architectures specifically designed to capture such dependencies.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes a latent neural integral operator framework for fMRI encoding and decoding tasks. Fixed-point iterations are performed in an auxiliary latent space to capture nonlocal spatiotemporal dependencies, after which a decoder handles classification or stimulus prediction. Experiments on two open fMRI datasets systematically vary temporal window length and input region (visual cortex versus whole brain), reporting that longer windows improve performance and yield more structured latent representations with clearer class separation in decoding tasks.
Significance. If the performance gains can be attributed specifically to the nonlocal operator mechanism rather than raw increases in context length or model capacity, the work would offer a useful architecture for high-dimensional spatiotemporal brain data. The focus on open datasets and controlled variation of spatiotemporal context provides a concrete empirical test bed for operator-based models in neuroimaging.
major comments (3)
- Abstract: the claim that 'larger temporal windows generally improve results and produce more structured learned representations' supplies no quantitative metrics, error bars, statistical tests, or description of data splits and evaluation procedures, rendering the central empirical claims unverifiable from the provided text.
- Abstract / Model Description: the central modeling choice that fixed-point iterations in the auxiliary latent space extract and exploit nonlocal spatiotemporal dependencies is not supported by ablation studies, convergence analysis of the iterations, or comparisons to capacity-matched baselines (e.g., standard RNN or attention layers on identical windows). Without such controls, observed gains could be explained by increased input context alone.
- Experiments section: no direct comparison is reported between the proposed operator model and simpler architectures with equivalent parameter counts or context lengths, which is required to isolate the contribution of the neural integral operator framework to the reported improvements in both decoding and encoding tasks.
minor comments (2)
- Abstract: the two open-source fMRI datasets are not named or referenced, which hinders immediate reproducibility.
- The description of latent-space geometry and class separation would benefit from explicit quantification (e.g., silhouette scores or visualization details) rather than qualitative statements.
Simulated Author's Rebuttal
We thank the referee for the constructive and detailed feedback. The comments identify key areas where additional clarity and controls will strengthen the manuscript. We address each major comment below and commit to the indicated revisions.
read point-by-point responses
-
Referee: Abstract: the claim that 'larger temporal windows generally improve results and produce more structured learned representations' supplies no quantitative metrics, error bars, statistical tests, or description of data splits and evaluation procedures, rendering the central empirical claims unverifiable from the provided text.
Authors: We agree that the abstract, due to length constraints, presents the findings at a high level without the supporting quantitative details. In the revised manuscript we will augment the abstract with representative performance metrics (including means and standard deviations across runs), and we will ensure the methods and results sections explicitly describe the data splits, cross-validation procedures, and statistical tests used to support the claims regarding temporal window length and latent-space structure. revision: yes
-
Referee: Abstract / Model Description: the central modeling choice that fixed-point iterations in the auxiliary latent space extract and exploit nonlocal spatiotemporal dependencies is not supported by ablation studies, convergence analysis of the iterations, or comparisons to capacity-matched baselines (e.g., standard RNN or attention layers on identical windows). Without such controls, observed gains could be explained by increased input context alone.
Authors: We acknowledge that the current manuscript does not contain explicit ablation studies that isolate the contribution of the fixed-point iterations or convergence diagnostics for those iterations. While the architecture is derived from the neural integral operator framework precisely to enable nonlocal interactions via the auxiliary latent space, we will add (i) an ablation that replaces the fixed-point iteration block with a single forward pass of equivalent depth, (ii) convergence plots for the iterations across representative samples, and (iii) comparisons against capacity-matched RNN and attention baselines that receive identical temporal windows. These additions will appear in a new subsection of the experiments. revision: yes
-
Referee: Experiments section: no direct comparison is reported between the proposed operator model and simpler architectures with equivalent parameter counts or context lengths, which is required to isolate the contribution of the neural integral operator framework to the reported improvements in both decoding and encoding tasks.
Authors: We agree that matched-parameter and matched-context comparisons are necessary to attribute gains specifically to the operator-based mechanism rather than to increased context or capacity. In the revised version we will report results for the proposed model alongside RNN, LSTM, and transformer baselines that are configured to have approximately the same number of parameters and that operate on the same temporal windows and spatial regions. These comparisons will be presented for both the decoding and encoding tasks on the two datasets. revision: yes
Circularity Check
No significant circularity; empirical evaluation on open datasets
full rationale
The paper presents an empirical study implementing a latent neural integral operator framework for fMRI tasks and evaluating it on open-source datasets via comparisons of temporal windows and brain regions. No mathematical derivation chain exists that reduces predictions or results to inputs by construction, self-definition, or self-citation load-bearing. Performance gains and latent-space observations are reported from direct experiments rather than fitted quantities renamed as predictions. The fixed-point iteration modeling choice is an architectural decision tested empirically, with no uniqueness theorems or ansatzes imported circularly from prior self-work.
Axiom & Free-Parameter Ledger
Lean theorems connected to this paper
-
IndisputableMonolith/Cost/FunctionalEquation.leanwashburn_uniqueness_aczel unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
We implement a latent neural integral operator framework that performs fixed point iterations in an auxiliary space... T(u)(x, t) = ∫∫ K_θ(u(z, s), x, t, z, s) dz ds
-
IndisputableMonolith/Foundation/RealityFromDistinction.leanreality_from_one_distinction unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
nonlocal spatiotemporal context... long-range anatomical and functional pathways
What do these tags mean?
- matches
- The paper's claim is directly supported by a theorem in the formal canon.
- supports
- The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
- extends
- The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
- uses
- The paper appears to rely on the theorem as machinery.
- contradicts
- The paper's claim conflicts with a theorem or certificate in the canon.
- unclear
- Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.
Reference graph
Works this paper leans on
-
[1]
Decoding hand kinematics from local field potentials using long short-term memory (lstm) network
Nur Ahmadi, Timothy G Constandinou, and Christos-Savvas Bouganis. Decoding hand kinematics from local field potentials using long short-term memory (lstm) network. In 2019 9th international IEEE/EMBS conference on neural engineering (NER) , pages 415--419. IEEE, 2019
work page 2019
-
[2]
Homogeneous nets of neuron-like elements
Shun-Ichi Amari. Homogeneous nets of neuron-like elements. Biological cybernetics , 17(4):211--220, 1975
work page 1975
-
[3]
Dynamics of pattern formation in lateral-inhibition type neural fields
Shun-ichi Amari. Dynamics of pattern formation in lateral-inhibition type neural fields. Biological cybernetics , 27(2):77--87, 1977
work page 1977
-
[4]
A transformer model for learning spatiotemporal contextual representation in fmri data
Nima Asadi, Ingrid R Olson, and Zoran Obradovic. A transformer model for learning spatiotemporal contextual representation in fmri data. Network Neuroscience , 7(1):22--47, 2023
work page 2023
-
[5]
Bolt: Fused window transformers for fmri time series analysis
Hasan A Bedel, Irmak Sivgin, Onat Dalmaz, Salman UH Dar, and Tolga C ukur. Bolt: Fused window transformers for fmri time series analysis. Medical image analysis , 88:102841, 2023
work page 2023
-
[6]
Wilson--cowan equations for neocortical dynamics
Jack D Cowan, Jeremy Neuman, and Wim van Drongelen. Wilson--cowan equations for neocortical dynamics. The Journal of Mathematical Neuroscience , 6(1):1, 2016
work page 2016
-
[7]
Hyoungshin Choi, Yeongjun Park, Jong-eun Lee, Sunghun Kim, Bo-yong Park, and Hyunjin Park. Spatiotemporal characterization of the functional mri latency structure with respect to neural signaling and brain hierarchy. Advanced Science , 12(43):e04956, 2025
work page 2025
-
[8]
Spatio-temporal correlation tensors reveal functional structure in human brain
Zhaohua Ding, Allen T Newton, Ran Xu, Adam W Anderson, Victoria L Morgan, and John C Gore. Spatio-temporal correlation tensors reveal functional structure in human brain. PloS one , 8(12):e82107, 2013
work page 2013
-
[9]
The human brain is intrinsically organized into dynamic, anticorrelated functional networks
Michael D Fox, Abraham Z Snyder, Justin L Vincent, Maurizio Corbetta, David C Van Essen, and Marcus E Raichle. The human brain is intrinsically organized into dynamic, anticorrelated functional networks. Proceedings of the National Academy of Sciences , 102(27):9673--9678, 2005
work page 2005
-
[10]
Machine learning for neural decoding
Joshua I Glaser, Ari S Benjamin, Raeed H Chowdhury, Matthew G Perich, Lee E Miller, and Konrad P Kording. Machine learning for neural decoding. eneuro , 7(4), 2020
work page 2020
-
[11]
3d masked autoencoder with spatiotemporal transformer for modeling of 4d fmri data
Jie Gao, Bao Ge, Ning Qiang, and Shijie Zhao. 3d masked autoencoder with spatiotemporal transformer for modeling of 4d fmri data. Medical Image Analysis , page 103861, 2025
work page 2025
-
[12]
Chlo \'e Gomez, Lynn Uhrig, Vincent Frouin, Edouard Duchesnay, B \'e chir Jarraya, and Antoine Grigis. Deep learning models reveal the link between dynamic brain connectivity patterns and states of consciousness. Scientific Reports , 14(1):31606, 2024
work page 2024
-
[13]
Scale-free properties of the functional magnetic resonance imaging signal during rest and task
Biyu J He. Scale-free properties of the functional magnetic resonance imaging signal during rest and task. Journal of Neuroscience , 31(39):13786--13795, 2011
work page 2011
-
[14]
Distributed and overlapping representations of faces and objects in ventral temporal cortex
James V Haxby, M Ida Gobbini, Maura L Furey, Alumit Ishai, Jennifer L Schouten, and Pietro Pietrini. Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science , 293(5539):2425--2430, 2001
work page 2001
-
[15]
Rethinking functional brain connectome analysis: Do graph deep learning models help?, 2025
Keqi Han, Yao Su, Lifang He, Liang Zhan, Sergey Plis, Vince Calhoun, and Carl Yang. Rethinking functional brain connectome analysis: Do graph deep learning models help?, 2025. URL https://arxiv. org/abs/2501.17207
-
[16]
It’s about time: Linking dynamical systems with human neuroimaging to understand the brain
Yohan J John, Kayle S Sawyer, Karthik Srinivasan, Eli J M \"u ller, Brandon R Munn, and James M Shine. It’s about time: Linking dynamical systems with human neuroimaging to understand the brain. Network Neuroscience , 6(4):960--979, 2022
work page 2022
-
[17]
Swift: Swin 4d fmri transformer
Peter Kim, Junbeom Kwon, Sunghwan Joo, Sangyoon Bae, Donggyu Lee, Yoonho Jung, Shinjae Yoo, Jiook Cha, and Taesup Moon. Swift: Swin 4d fmri transformer. Advances in Neural Information Processing Systems , 36:42015--42037, 2023
work page 2023
-
[18]
Naoko Koide-Majima, Shinji Nishimoto, and Kei Majima. Mental image reconstruction from human brain activity: Neural decoding of mental imagery via deep neural network-based bayesian estimation. Neural Networks , 170:349--363, 2024
work page 2024
-
[19]
Visual image reconstruction from brain activity via latent representation
Yukiyasu Kamitani, Misato Tanaka, and Ken Shirakawa. Visual image reconstruction from brain activity via latent representation. Annual Review of Vision Science , 11(2025):611--634, 2025
work page 2025
-
[20]
Consciousness-specific dynamic interactions of brain integration and functional diversity
Andrea I Luppi, Michael M Craig, Ioannis Pappas, Paola Finoia, Guy B Williams, Judith Allanson, John D Pickard, Adrian M Owen, Lorina Naci, David K Menon, et al. Consciousness-specific dynamic interactions of brain integration and functional diversity. Nature communications , 10(1):4616, 2019
work page 2019
-
[21]
Generative adversarial networks in brain imaging: A narrative review
Maria Elena Laino, Pierandrea Cancian, Letterio Salvatore Politi, Matteo Giovanni Della Porta, Luca Saba, and Victor Savevski. Generative adversarial networks in brain imaging: A narrative review. Journal of imaging , 8(4):83, 2022
work page 2022
-
[22]
Hongming Li and Yong Fan. Interpretable, highly accurate brain decoding of subtly distinct brain states from functional mri using intrinsic functional networks and long short-term memory recurrent neural networks. NeuroImage , 202:116059, 2019
work page 2019
-
[23]
Guoshi Li and Pew-Thian Yap. From descriptive connectome to mechanistic connectome: Generative modeling in functional magnetic resonance imaging analysis. Frontiers in Human Neuroscience , 16:940842, 2022
work page 2022
-
[24]
Neural decoding and feature selection methods for closed-loop control of avoidance behavior
Jinhan Liu, Rebecca Younk, Lauren M Drahos, Sumedh S Nagrale, Shreya Yadav, Alik S Widge, and Mahsa Shoaran. Neural decoding and feature selection methods for closed-loop control of avoidance behavior. Journal of neural engineering , 21(5):056041, 2024
work page 2024
-
[25]
Deep learning applications in fmri--a review work
Jiangxue Li and Peize Zhao. Deep learning applications in fmri--a review work. In Proceedings of the 2023 13th International Conference on Bioscience, Biochemistry and Bioinformatics , pages 75--80, 2023
work page 2023
-
[26]
Graph neural networks in brain connectivity studies: Methods, challenges, and future directions
Hamed Mohammadi and Waldemar Karwowski. Graph neural networks in brain connectivity studies: Methods, challenges, and future directions. Brain Sciences , 15(1):17, 2024
work page 2024
-
[27]
Yoichi Miyawaki, Hajime Uchida, Okito Yamashita, Masa-aki Sato, Yusuke Morito, Hiroki C Tanabe, Norihiro Sadato, and Yukiyasu Kamitani. Visual image reconstruction from human brain activity using a combination of multiscale local image decoders. Neuron , 60(5):915--929, 2008
work page 2008
-
[28]
Nilearn contributors . nilearn. https://github.com/nilearn/nilearn, 2023. Software. DOI: 10.5281/zenodo.8397156
-
[29]
Inferring single-trial neural population dynamics using sequential auto-encoders
Chethan Pandarinath, Daniel J O’Shea, Jasmine Collins, Rafal Jozefowicz, Sergey D Stavisky, Jonathan C Kao, Eric M Trautmann, Matthew T Kaufman, Stephen I Ryu, Leigh R Hochberg, et al. Inferring single-trial neural population dynamics using sequential auto-encoders. Nature methods , 15(10):805--815, 2018
work page 2018
-
[30]
Roland Potthast. Amari model. In Encyclopedia of Computational Neuroscience , pages 171--175. Springer, 2022
work page 2022
-
[31]
Interpreting models interpreting brain dynamics
Md Mahfuzur Rahman, Usman Mahmood, Noah Lewis, Harshvardhan Gazula, Alex Fedorov, Zening Fu, Vince D Calhoun, and Sergey M Plis. Interpreting models interpreting brain dynamics. Scientific reports , 12(1):12023, 2022
work page 2022
-
[32]
A neural network that finds a naturalistic solution for the production of muscle activity
David Sussillo, Mark M Churchland, Matthew T Kaufman, and Krishna V Shenoy. A neural network that finds a naturalistic solution for the production of muscle activity. Nature neuroscience , 18(7):1025--1033, 2015
work page 2015
-
[33]
Breakdown of long-range temporal dependence in default mode and attention networks during deep sleep
Enzo Tagliazucchi, Frederic von Wegner, Astrid Morzelewski, Verena Brodbeck, Kolja Jahnke, and Helmut Laufs. Breakdown of long-range temporal dependence in default mode and attention networks during deep sleep. Proceedings of the National Academy of Sciences , 110(38):15419--15424, 2013
work page 2013
-
[34]
Sujitha Venkatapathy, Mikhail Votinov, Lisa Wagels, Sangyun Kim, Munseob Lee, Ute Habel, In-Ho Ra, and Han-Gue Jo. Ensemble graph neural network model for classification of major depressive disorder using whole-brain functional connectivity. Frontiers in psychiatry , 14:1125339, 2023
work page 2023
-
[35]
A mathematical theory of the functional dynamics of cortical and thalamic nervous tissue
Hugh R Wilson and Jack D Cowan. A mathematical theory of the functional dynamics of cortical and thalamic nervous tissue. Kybernetik , 13(2):55--80, 1973
work page 1973
-
[36]
Decoding and mapping task states of the human brain via deep learning
Xiaoxiao Wang, Xiao Liang, Zhoufan Jiang, Benedictor A Nguchu, Yawen Zhou, Yanming Wang, Huijuan Wang, Yu Li, Yuying Zhu, Feng Wu, et al. Decoding and mapping task states of the human brain via deep learning. Human brain mapping , 41(6):1505--1519, 2020
work page 2020
-
[37]
The organization of the human cerebral cortex estimated by intrinsic functional connectivity
BT Thomas Yeo, Fenna M Krienen, Jorge Sepulcre, Mert R Sabuncu, Danial Lashkari, Marisa Hollinshead, Joshua L Roffman, Jordan W Smoller, Lilla Z \"o llei, Jonathan R Polimeni, et al. The organization of the human cerebral cortex estimated by intrinsic functional connectivity. Journal of neurophysiology , 2011
work page 2011
-
[38]
Time-resolved resting-state brain networks
Andrew Zalesky, Alex Fornito, Luca Cocchi, Leonardo L Gollo, and Michael Breakspear. Time-resolved resting-state brain networks. Proceedings of the National Academy of Sciences , 111(28):10341--10346, 2014
work page 2014
-
[39]
Learning integral operators via neural integral equations
Emanuele Zappala, Antonio Henrique de Oliveira Fonseca, Josue Ortega Caro, Andrew Henry Moberly, Michael James Higley, Jessica Cardin, and David van Dijk. Learning integral operators via neural integral equations. Nature Machine Intelligence , 6(9):1046--1062, 2024
work page 2024
-
[40]
Neural integro-differential equations
Emanuele Zappala, Antonio H de O Fonseca, Andrew H Moberly, Michael J Higley, Chadi Abdallah, Jessica A Cardin, and David van Dijk. Neural integro-differential equations. In Proceedings of the aaai conference on artificial intelligence , volume 37, pages 11104--11112, 2023
work page 2023
-
[41]
Liang Zou, Jiannan Zheng, Chunyan Miao, Martin J Mckeown, and Z Jane Wang. 3d cnn based automatic diagnosis of attention deficit hyperactivity disorder using functional and structural mri. Ieee Access , 5:23626--23636, 2017
work page 2017
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.