Pith. sign in

REVIEW 4 major objections 4 minor 54 references

From Flat to Round: Redefining Brain Decoding with Surface-Based fMRI and Cortex Structure

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Treating fMRI as 2D data on the brain's curved cortical surface, rather than flattened 1D signals, improves visual reconstruction from brain activity and generalizes across individuals.

desk verdict A useful surface-based tokenizer for fMRI decoding, but the abstract oversells it as SOTA and the 'generalizability across individuals' claim is untested. read the letter →

arxiv 2507.16389 v1 pith:D6SLFQVJ submitted 2025-07-22 cs.CV cs.AI

Sijin Yu , Zijiao Chen , Wenxuan Wu , Shengxian Chen , Zhongliang Liu , Jingxin Nie , Xiaofen Xing , Xiangmin Xu
show 1 more author
Xin Zhang
This is my paper · ORCID
classification cs.CVcs.AI
keywords fMRIimagereconstructionspheretokenizersphericalconvolutioncorticalsurfacestructure-functioncouplingcross-subjectbraindecodingcontrastivelearningNaturalScenesDataset
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that fMRI-based visual reconstruction has been throwing away information by flattening cortical activity into one-dimensional vectors, and that respecting the cortex's actual geometry recovers it. It proposes a sphere tokenizer that treats fMRI as 2D signals on the cortical surface, injects each subject's cortical structure (thickness, surface area, sulcal depth, curvature) along with sphere-location embeddings, and trains with a positive-sample mixup that mirrors the averaging used at inference. On the Natural Scenes Dataset, at an equal 9K-voxel resolution, the resulting model beats the flat MindBridge baseline on seven of eight reconstruction metrics, and its Grad-CAM attention maps look nearly identical across subjects — which the paper takes as evidence for its central claim that spherical, structure-aware decoding improves accuracy, interpretability, and cross-subject generalization.

What carries the argument

The load-bearing object is the sphere tokenizer: a SphericalUNet-based autoencoder that downsamples each hemisphere's fsaverage6 fMRI (40,962 voxels) through conditional ResNet blocks to 642-voxel fsaverage3 tokens, then upsamples back to the original resolution to compute reconstruction losses. Each ResNet block runs sphere convolutions with a one-hop-neighbor receptive field, and between convolutions it injects two conditions: cortex structure (thickness, surface area, sulcal depth, curvature, resampled per subject) and a sphere positional embedding built from voxel 3D coordinates with random-rotation augmentation. A supporting mechanism, positive-sample mixup, blends the three fMRI scans of one image with Dirichlet-sampled weights during contrastive training so that training sees the same averaged-input distribution used at inference; the model then aligns selected visual tokens with CLIP image and text embeddings and reconstructs the image with a diffusion decoder.

What would settle it

A direct test would be running the full pipeline on a denser spherical atlas — or on native fsaverage7 data — so the vision-voxel count matches the 14K used by the flat baselines. If the sphere model's margin over the equal-number 1D baseline does not grow with resolution, or if the structure-conditioning ablation stops mattering once voxels are dense, the claim that spherical geometry and cortical structure carry the gains collapses and the shortfall would be purely a resolution problem rather than a geometry insight.

Watch

Extended reading notes

Core claim

The central discovery is that the spatial layout of fMRI on the cortical surface is usable signal, not noise to be flattened away. The paper shows that a tokenizer built on sphere convolutions — where each voxel updates from its one-hop neighbors on a standard fsaverage6 sphere — combined with structural and positional conditioning, reconstructs visual images more accurately than an equal-resolution 1D pipeline, and that ablation experiments attribute the gain to the spherical geometry and the structure and position conditions rather than to the autoencoder objective. Operationally, the sphere-based model at 9,548 vision voxels improves on MindBridge at the same resolution on seven of eight metrics (e.g., PixCorr 0.165 vs 0.143, AlexNet(2) 78.2% vs 75.0%, CLIP 88.3% vs 87.0%) and also tops several 14K-voxel methods on high-level semantic metrics such as CLIP and Inception.

Load-bearing premise

The load-bearing premise is that resampling each subject's native fsaverage7 fMRI onto the shared fsaverage6 sphere and keeping only the 9,548 visual voxels preserves the fine-grained cortical topography the tokenizer exploits; the paper itself concedes in Section 4.2 and in its limitations that this 9K resolution, an artifact of SphericalUNet's standard meshes, is the main reason it trails 14K-resolution methods.

Editorial extensions

If this is right

  • At equal 9K resolution the sphere-based model beats MindBridge 9K-1D on seven of eight metrics (PixCorr 0.165 vs 0.143, AlexNet(2) 78.2% vs 75.0%, CLIP 88.3% vs 87.0%), so the reported gain comes from spherical geometry and structure conditioning rather than from more data.
  • Adding cortex structure lifts high-level metrics (CLIP 86.0% → 88.3% over the no-structure ablation) and produces near-identical Grad-CAM maps across the four subjects, supporting the claim that the model learns subject-invariant visual representations.
  • Positive-sample mixup with K = 3 and λ = 0.5 removes the single-scan-versus-averaged-scan distribution gap, and both too little and too much mixing degrade performance, consistent with the paper's account of the gap.
  • Because 9K-sphere uses fewer voxels than 14K-1D, the approach is compatible with lower-resolution 3T fMRI and still beats some 14K methods on high-level metrics (e.g., CLIP 88.3% vs MindEye's 85.0%).
  • Switching the tokenizer from Conv1×1 or ROI-level embeddings to voxel-wise sphere tokens raises Inception from 76.7% to 85.1% and CLIP from 79.4% to 88.3%, evidence that voxel-level spatial modeling is what the downstream encoder needs.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 9K ceiling is an artifact of the standard sphere mesh, not of the spherical idea; porting the tokenizer to a denser surface atlas should close the gap to 14K flat methods, which is a testable next step.
  • The structure-conditioning mechanism should transfer to any cortical phenotype map (for example, myelination or parcellation priors) and any dataset with repeated trials per stimulus, including MEG and EEG recordings with averaged trials.
  • If the near-identical cross-subject attention maps are caused by the structure and position conditions, then ablating those conditions should measurably increase inter-subject differences in the heatmaps — a check the paper does not run.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes a new fMRI-to-image reconstruction pipeline that treats fMRI data as spherical surface signals. A sphere tokenizer downsamples cortical data on the fsaverage6 sphere, injects structural MRI (thickness, area, sulcal depth, curvature) and positional embeddings, and is pretrained with an autoencoder. The resulting tokens are fed to a transformer encoder aligned with CLIP, and images are generated with Versatile Diffusion. A 'positive sample mixup' is introduced to better match the averaging of multiple fMRI scans at inference. Experiments are conducted on NSD with four subjects; comparisons with previous methods are in Table 1, ablations in Table 2, and qualitative examples are provided.

Significance. If the claims are supported, the sphere tokenizer offers a principled way to incorporate cortical geometry and individual anatomy into brain decoding, and the mixup strategy is a simple, useful idea for datasets with multiple scans per stimulus. The paper includes detailed hyperparameters, ablations, and an honest limitations appendix, which are strengths. However, the current evidence falls short of the abstract's 'superior SOTA' and 'generalizability across individuals' claims, and the absence of error bars makes the small reported improvements difficult to assess. The contribution would be strengthened by matched-resolution comparisons with variance, and by a leave-one-subject-out evaluation.

major comments (4)
  1. [Abstract; Table 1; Section 4.2] The abstract claims 'Experiments demonstrate superior reconstruction performance compared to SOTA methods,' but Table 1 shows that Ours (9K-sphere) attains PixCorr 0.165 and SSIM 0.305, whereas Neuro-Vision (14K-1D) attains 0.265 and 0.357 and Psychometry attains 0.297 and 0.340. Section 4.2 itself states that 'compared to the current state-of-the-art, our model performs relatively worse due to the use of a lower fMRI resolution.' This contradiction between the headline claim and the reported numbers should be resolved by qualifying the SOTA claim to matched-resolution comparisons or surface-based methods.
  2. [Section 3.1; Section 3.2; Table 2; Figure 10] The abstract claims 'generalizability across individuals,' but Section 3.1 shows that subjects 01, 02, 05, and 07 contribute 8,500 training images and the shared 1,000-image set is used for testing, so no held-out subject is ever evaluated. The cortical structure maps (thickness, area, sulcal depth, curvature) are static per-subject files injected at every conditional ResNet block (Section 3.2); the improvement from #3 to #7 in Table 2 could therefore be driven by the model using these maps as subject identifiers rather than by a transferable structure-function mapping. A leave-one-subject-out experiment is required to support the generalization claim, and the Grad-CAM similarity in Figure 10 cannot address this because all four subjects were in training.
  3. [Tables 1, 2, D-1; Figure 9] All quantitative results appear to come from a single run, with no standard deviations, confidence intervals, or significance tests. The claimed gains over MindBridge 9K-1D are small (PixCorr +0.022, EffNet-B -0.036, SSIM -0.002), and the ablation differences are similarly small; without repeated seeds or statistical testing, the reliability of these differences is unknown. Reporting variance or significance would make the claims about the tokenizer and mixup strategy actionable.
  4. [Section 4.4] The 'biological interpretability' claim is based on Grad-CAM heatmaps of the model itself. The observation that the model attends to visual areas and that the four subjects have similar heatmaps is a property of the model trained on those subjects and aligned to CLIP, not independent evidence about biological interpretability. The paper should either soften this claim or validate it against an external functional atlas or a subject group not used in training.
minor comments (4)
  1. [Equation (4)] Equation (4) is typeset with missing summation symbols ('bP' appears in place of the batch sum); the notation should be corrected.
  2. [Section 4, first paragraph] Section 4, first paragraph: 'the averages computed across four subject' should be 'subjects'.
  3. [Section 3.3] Section 3.3 contains a 'Note' stating that mixup is applied at the tokenizer's input level, while the text and Eq. (2) describe mixing vision tokens; clarify which representation is actually mixed in Algorithm B-2.
  4. [Figure 2] Figure 2 is very dense; consider separating the tokenizer schematic from the network block details for readability.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reconstruction claim is externally benchmarked and the method stages do not reduce to their inputs.

full rationale

The central derivation chain is self-contained against external benchmarks. Stage 1 trains a sphere tokenizer as an autoencoder on fMRI inputs, but that is auxiliary; Stage 2 aligns tokenizer outputs with CLIP embeddings and Stage 3 reconstructs images, with final metrics computed on held-out NSD test images and compared to published baselines, including MindBridge at the same 9K resolution in Table 1. No fitted parameter is renamed as a prediction, and no equation equates the target reconstruction to a training input. The cortex-structure and position conditions are inputs to the tokenizer, not outputs of the evaluation. Positive-sample mixup is a training augmentation whose inference form is the average of the three scans, explicitly stated, so it does not smuggle the test result into training. Self-citations such as Chen et al. [4,5] by co-author Zijiao Chen appear only in background and related-work comparisons and are not load-bearing for the proposed method, which relies on external components like SphericalUNet [51], CLIP [30], and Versatile Diffusion [47]. The manuscript's 'generalizability across individuals' wording is broader than the same-subject train/test split supports, and the Grad-CAM interpretability discussion is self-referential rather than independent evidence; however, these are evidentiary and validity concerns, not circular reductions of the claimed result to its inputs.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

The central claim rests on standard ML assumptions about the NSD dataset, CLIP space, and spherical convolution, plus several hyperparameters tuned by hand or by ablations. No new physical or biological entities are introduced; the sphere tokens, cortical structure features, and mixup samples are computational constructions within existing data.

free parameters (6)
  • mixup_ratio_lambda = 0.5
    Selected from Figure 9 as the peak of the ablation sweep for positive sample mixup; it changes the training distribution and the final metrics.
  • mixup_sample_count_K = 3
    Selected from Figure 9 together with lambda=0.5; it controls augmentation strength and the number of negatives within a batch.
  • tokenizer_mixup_beta = 0.3
    Data augmentation hyperparameter for sphere tokenizer pretraining, set by hand in Appendix C.
  • rotation_max_angle = 5 degrees
    Maximum rotation angle for sphere positional embedding augmentation, set by hand in Appendix C.
  • infonce_temperature = 0.1
    Contrastive temperature for bidirectional InfoNCE, set by hand in Appendix C.
  • clip_embedding_clamp = [-1.5, 1.5]
    Clipping range for CLIP embeddings to reduce outlier influence, chosen by hand in Appendix C.
assumptions (6)
  • domain assumption NSD single-trial beta estimates reflect stimulus-driven neural activity that can be decoded into images
    The entire task assumes fMRI responses to images contain reconstructable visual information; this is the standard premise of the field.
  • domain assumption Resampling native fsaverage7 fMRI to fsaverage6 (40,962 voxels per hemisphere) preserves the cortical neighborhoods and visual information needed for reconstruction
    Appendix A performs this resampling; without this, the 9K-sphere input loses the information required to compete with 14K methods.
  • domain assumption Cortex structure features (thickness, area, sulcal depth, curvature) provide subject-specific anatomical conditioning that improves cross-subject alignment
    The structure condition is introduced in Section 3.2 and ablated in Table 2; the paper assumes these features are relevant to functional decoding.
  • domain assumption CLIP image and text embeddings form a shared semantic space that fMRI tokens can be contrastively aligned to for image generation
    The Stage 2 and Stage 3 pipeline depends on CLIP space as the latent target; this is inherited from prior work.
  • domain assumption Convex combinations of the three fMRI scans approximate the averaged scan used at inference
    Positive sample mixup in Section 3.3 assumes random Dirichlet mixes of the three scans train the encoder to match mean(x1,x2,x3); the paper presents this as a heuristic.
  • domain assumption Grad-CAM heatmaps averaged over the test set indicate which brain regions the model uses and support cross-subject interpretability claims
    Section 4.4 interprets heatmap similarity as subject-independent decoding; this is a model-interpretation assumption, not an independent biological measurement.

how reviews work

0 comments
Cite this review

Pith. "Pith review of From Flat to Round: Redefining Brain Decoding with Surface-Based fMRI and Cortex Structure." pith.science (2026). https://pith.science/paper/D6SLFQVJ

@misc{pith2026250716389,
  author       = {Pith},
  title        = {Pith review of: From Flat to Round: Redefining Brain Decoding with Surface-Based fMRI and Cortex Structure},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/D6SLFQVJ}},
  note         = {Machine review of arXiv:2507.16389}
}
read the original abstract

Reconstructing visual stimuli from human brain activity (e.g., fMRI) bridges neuroscience and computer vision by decoding neural representations. However, existing methods often overlook critical brain structure-function relationships, flattening spatial information and neglecting individual anatomical variations. To address these issues, we propose (1) a novel sphere tokenizer that explicitly models fMRI signals as spatially coherent 2D spherical data on the cortical surface; (2) integration of structural MRI (sMRI) data, enabling personalized encoding of individual anatomical variations; and (3) a positive-sample mixup strategy for efficiently leveraging multiple fMRI scans associated with the same visual stimulus. Collectively, these innovations enhance reconstruction accuracy, biological interpretability, and generalizability across individuals. Experiments demonstrate superior reconstruction performance compared to SOTA methods, highlighting the effectiveness and interpretability of our biologically informed approach.

Figures

Figures reproduced from arXiv: 2507.16389 by the authors.

Figure 1
Figure 1. The drawbacks of previous work and the contributions of this paper. cerebral cortex and sphere positional embedding into the Sphere Tokenizer. The cortical structure allows the model to perceive differences in brain structures across subjects, while the sphere positional embedding enables the model to account for functional pattern differences across different regions of the cortex. To the best of our knowledge, we … view at source ↗
Figure 2
Figure 2. Illustration of sphere tokenizer (§3.2). (a) [blue] The tokenizer downsamples the fMRI data three times and maps it to fMRI tokens. (b) [brown, bottom] Each downsampling layer consists of L conditional ResNet blocks. (c) [purple] The ResNet blocks are based on the sphere convolution, a 2D convolution on the sphere with a receptive field of one-hop neighbors. (d) [brown, top] The conditional ResNet block has two type… view at source ↗
Figure 3
Figure 3. Illustration of the fMRI encoder (§3.3) and the image reconstruction pipeline (§3.4). (top) We select vision-relevant tokens from the sphere tokenizer output and feed them into the fMRI encoder, which predicts both image and text embeddings aligned with CLIP. (bottom left) Our positive sample mixup strategy: when selected, multiple fMRI scans of the same image are mixed with random weights to better approximate the … view at source ↗
Figures from the paper (4 more)
Figure 5
Figure 5. Figure 5: fMRI vision voxels reconstruction results using the [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Qualitative results on NSD [1] test subj01. In comparison to previous single-subject models [24, 32, 40, 45] and recent cross-subject models [37, 43], our model is trained on lower fMRI resolution without subject-specific parameters. The visual results of reconstructio…
Figure 7
Figure 7. Figure 7: Failed cases in the NSD [1] test subj01. In these cases, although the reconstruction results are not entirely satisfactory, our model has managed to reconstruct partial or analogous semantics. See related analysis in §4.2. level metrics PixCorr and SSIM, and 0.892/0.50…
Figure 10
Figure 10. Figure 10: Average neural activity representation differential [PITH_FULL_IMAGE:figures/full_fig_p008_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

54 extracted references · 44 canonical work pages

  1. [1]

    A massive 7t fmri dataset to bridge cognitive neuroscience and artificial intelligence

    Emily J Allen, Ghislain St-Yves, Yihan Wu, Jesse L Breedlove, Jacob S Prince, Logan T Dowdle, Matthias Nau, Brad Caron, Franco Pestilli, Ian Charest, et al. A massive 7t fmri dataset to bridge cognitive neuroscience and artificial intelligence. Nature neuroscience, 25(1):116–126, 2022. 2, 4, 5, 6, 7, 8, 12, 14, 15, 16, 17, 18

    work page 2022

  2. [2]

    From voxels to pixels and back: Self-supervision in natural-image reconstruction from fmri

    Roman Beliy, Guy Gaziv, Assaf Hoogi, Francesca Strappini, Tal Golan, and Michal Irani. From voxels to pixels and back: Self-supervision in natural-image reconstruction from fmri. Advances in Neural Information Processing Systems , 32, 2019. 1

    work page 2019

  3. [3]

    Unsupervised learning of visual features by contrasting cluster assignments

    Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Pi- otr Bojanowski, and Armand Joulin. Unsupervised learning of visual features by contrasting cluster assignments. Ad- vances in neural information processing systems , 33:9912– 9924, 2020. 5

    work page 2020

  4. [4]

    Seeing beyond the brain: Conditional dif- fusion model with sparse masked modeling for vision decod- ing

    Zijiao Chen, Jiaxin Qing, Tiange Xiang, Wan Lin Yue, and Juan Helen Zhou. Seeing beyond the brain: Conditional dif- fusion model with sparse masked modeling for vision decod- ing. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 22710–22720,

    work page

  5. [5]

    Cinematic mindscapes: High-quality video reconstruction from brain activity

    Zijiao Chen, Jiaxin Qing, and Juan Helen Zhou. Cinematic mindscapes: High-quality video reconstruction from brain activity. NeurIPS, 2023. 1

    work page 2023

  6. [6]

    Sss: Signature-sequence-statistical model for exploring the impact of prenatal depression on newborns’ brain using fnirs

    Zhaojin Chen, Sijin Yu, Xuejiao Li, Huirong Lei, Jiyu Qian, Yingxue Yao, Zicong Zheng, Guodong Liang, Xiaofen Xing, Xin Zhang, et al. Sss: Signature-sequence-statistical model for exploring the impact of prenatal depression on newborns’ brain using fnirs. In 2025 IEEE 22nd International Sympo- sium on Biomedical Imaging (ISBI), pages 1–4. IEEE, 2025. 14

    work page 2025

  7. [7]

    brain reading

    David D Cox and Robert L Savoy. Functional magnetic res- onance imaging (fmri)“brain reading”: detecting and clas- sifying distributed patterns of fmri activity in human visual cortex. Neuroimage, 19(2):261–270, 2003. 1

    work page 2003

  8. [8]

    Diffusion models beat gans on image synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural informa- tion processing systems, 34:8780–8794, 2021. 1, 2

    2021

Show all 54 references
  1. [9]

    Decoding visual neural representations by multimodal learn- ing of brain-visual-linguistic features

    Changde Du, Kaicheng Fu, Jinpeng Li, and Huiguang He. Decoding visual neural representations by multimodal learn- ing of brain-visual-linguistic features. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(9):10760– 10777, 2023. 1, 2

    2023

  2. [10]

    Freesurfer

    Bruce Fischl. Freesurfer. Neuroimage, 62(2):774–781, 2012. 1, 2

    2012

  3. [11]

    Cortical surface-based analysis: Ii: inflation, flattening, and a surface- based coordinate system

    Bruce Fischl, Martin I Sereno, and Anders M Dale. Cortical surface-based analysis: Ii: inflation, flattening, and a surface- based coordinate system. Neuroimage, 9(2):195–207, 1999. 1

    1999

  4. [12]

    Structure–function coupling in macroscale human brain networks

    Panagiotis Fotiadis, Linden Parkes, Kathryn A Davis, Theodore D Satterthwaite, Russell T Shinohara, and Dani S Bassett. Structure–function coupling in macroscale human brain networks. Nature Reviews Neuroscience, 25(10):688– 704, 2024. 1, 3

    2024

  5. [13]

    Decoding natural image stimuli from fmri data with a surface-based convolutional network

    Zijin Gu, Keith Jamison, Amy Kuceyeski, and Mert R Sabuncu. Decoding natural image stimuli from fmri data with a surface-based convolutional network. In Medical Imaging with Deep Learning, pages 107–118. PMLR, 2023. 2, 6

    2023

  6. [14]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. 1, 2

    2020

  7. [15]

    Generic de- coding of seen and imagined objects using hierarchical vi- sual features

    Tomoyasu Horikawa and Yukiyasu Kamitani. Generic de- coding of seen and imagined objects using hierarchical vi- sual features. Nature communications, 8(1):15037, 2017. 2

    2017

  8. [16]

    Neuropictor: Refining fmri-to-image reconstruction via multi-individual pretrain- ing and multi-level modulation

    Jingyang Huo, Yikai Wang, Yun Wang, Xuelin Qian, Chong Li, Yanwei Fu, and Jianfeng Feng. Neuropictor: Refining fmri-to-image reconstruction via multi-individual pretrain- ing and multi-level modulation. In European Conference on Computer Vision, pages 56–73. Springer, 2025. 1,...

    2025

  9. [17]

    Spherical cnns on unstructured grids

    Chiyu Jiang, Jingwei Huang, Karthik Kashinath, Philip Mar- cus, Matthias Niessner, et al. Spherical cnns on unstructured grids. arXiv preprint arXiv:1901.02039, 2019. 1

    1901 arXiv

  10. [18]

    Decoding the visual and subjective contents of the human brain.Nature neuroscience, 8(5):679–685, 2005

    Yukiyasu Kamitani and Frank Tong. Decoding the visual and subjective contents of the human brain.Nature neuroscience, 8(5):679–685, 2005. 1

    2005

  11. [19]

    Cortical response to nat- uralistic stimuli is largely predictable with deep neural net- works

    Meenakshi Khosla, Gia H Ngo, Keith Jamison, Amy Kuceyeski, and Mert R Sabuncu. Cortical response to nat- uralistic stimuli is largely predictable with deep neural net- works. Science Advances, 7(22):eabe7547, 2021. 1

    2021

  12. [20]

    Imagenet classification with deep convolutional neural net- works

    Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural net- works. Advances in neural information processing systems , 25, 2012. 1, 5

    2012

  13. [21]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceeding...

    2014

  14. [22]

    Eeg2video: Towards decoding dynamic visual perception from eeg signals

    Xuan-Hao Liu, Yan-Kai Liu, Yansen Wang, Kan Ren, Han- wen Shi, Zilong Wang, Dongsheng Li, Bao-Liang Lu, and Wei-Long Zheng. Eeg2video: Towards decoding dynamic visual perception from eeg signals. Advances in Neural In- formation Processing Systems, 37:72245–72273, 2024. 14

    2024

  15. [23]

    Repre- sentation learning with contrastive predictive coding

    Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Repre- sentation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748, 2018. 5, 13

    2018 arXiv

  16. [24]

    Natural scene recon- struction from fmri signals using generative latent diffusion

    Furkan Ozcelik and Rufin VanRullen. Natural scene recon- struction from fmri signals using generative latent diffusion. Scientific Reports, 13(1):15666, 2023. 1, 6, 7

    2023

  17. [25]

    Reconstruction of perceived images from fmri patterns and semantic brain exploration us- ing instance-conditioned gans

    Furkan Ozcelik, Bhavin Choksi, Milad Mozafari, Leila Reddy, and Rufin VanRullen. Reconstruction of perceived images from fmri patterns and semantic brain exploration us- ing instance-conditioned gans. In 2022 International Joint Conference on Neural Networks (IJCNN), pages 1–8. IEEE,

    2022

  18. [26]

    Geometric constraints on human brain func- tion

    James C Pang, Kevin M Aquino, Marianne Oldehinkel, Pe- ter A Robinson, Ben D Fulcher, Michael Breakspear, and Alex Fornito. Geometric constraints on human brain func- tion. Nature, 618(7965):566–574, 2023. 1, 3 9

    2023

  19. [27]

    Joint fmri decoding and encoding with latent embedding alignment

    Xuelin Qian, Yikai Wang, Yanwei Fu, Xinwei Sun, Xi- angyang Xue, and Jianfeng Feng. Joint fmri decoding and encoding with latent embedding alignment. arXiv preprint arXiv:2303.14730, 2023. 2, 7

    2023 arXiv

  20. [28]

    A closer look at gan priors: Exploit- ing intermediate features for enhanced model inversion at- tacks

    Yixiang Qiu, Hao Fang, Hongyao Yu, Bin Chen, MeiKang Qiu, and Shu-Tao Xia. A closer look at gan priors: Exploit- ing intermediate features for enhanced model inversion at- tacks. In European Conference on Computer Vision, pages 109–126. Springer, 2024. 14

    2024

  21. [29]

    Psychometry: An omnifit model for image re- construction from human brain activity

    Ruijie Quan, Wenguan Wang, Zhibo Tian, Fan Ma, and Yi Yang. Psychometry: An omnifit model for image re- construction from human brain activity. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 233–243, 2024. 1, 2, 5, 6, 7, 14

    2024

  22. [30]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  23. [31]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 1, 2, 5

    2022

  24. [32]

    Re- constructing the mind’s eye: fmri-to-image with contrastive learning and diffusion priors

    Paul Scotti, Atmadeep Banerjee, Jimmie Goode, Stepan Sha- balin, Alex Nguyen, Aidan Dempster, Nathalie Verlinde, Elad Yundler, David Weisberg, Kenneth Norman, et al. Re- constructing the mind’s eye: fmri-to-image with contrastive learning and diffusion priors. Advances in Neur...

    2023

  25. [33]

    Generative adver- sarial networks for reconstructing natural images from brain activity

    Katja Seeliger, Umut G ¨uc ¸l¨u, Luca Ambrogioni, Yagmur G¨uc ¸l¨ut¨urk, and Marcel AJ Van Gerven. Generative adver- sarial networks for reconstructing natural images from brain activity. NeuroImage, 181:775–785, 2018. 2

    2018

  26. [34]

    Grad-cam: Visual explanations from deep networks via gradient-based localization

    Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE in- ternational conference on computer vision , pages 618–626,

  27. [35]

    Deep image reconstruction from human brain activity

    Guohua Shen, Tomoyasu Horikawa, Kei Majima, and Yukiyasu Kamitani. Deep image reconstruction from human brain activity. PLoS computational biology, 15(1):e1006633,

  28. [36]

    Neuro-vision to language: Enhancing brain recording-based visual reconstruction and language interaction

    Guobin Shen, Dongcheng Zhao, Xiang He, Linghao Feng, Yiting Dong, Jihang Wang, Qian Zhang, and Yi Zeng. Neuro-vision to language: Enhancing brain recording-based visual reconstruction and language interaction. In The Thirty-eighth Annual Conference on Neural Information Proces...

    2024

  29. [37]

    Neuro-vision to language: Enhancing brain recording-based visual reconstruction and language interaction

    Guobin Shen, Dongcheng Zhao, Xiang He, Linghao Feng, Yiting Dong, Jihang Wang, Qian Zhang, and Yi Zeng. Neuro-vision to language: Enhancing brain recording-based visual reconstruction and language interaction. Advances in Neural Information Processing Systems , 37:98083–98110,

  30. [38]

    Contrast, attend and diffuse to decode high-resolution images from brain ac- tivities

    Jingyuan Sun, Mingxiao Li, Zijiao Chen, Yunhao Zhang, Shaonan Wang, and Marie-Francine Moens. Contrast, attend and diffuse to decode high-resolution images from brain ac- tivities. Advances in Neural Information Processing Systems, 36, 2024. 1, 2

    2024

  31. [39]

    Rethinking the inception archi- tecture for computer vision

    Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception archi- tecture for computer vision. In Proceedings of the IEEE con- ference on computer vision and pattern recognition , pages 2818–2826, 2016. 5

    2016

  32. [40]

    High-resolution image re- construction with latent diffusion models from human brain activity

    Yu Takagi and Shinji Nishimoto. High-resolution image re- construction with latent diffusion models from human brain activity. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 14453– 14463, 2023. 1, 2, 6, 7

    2023

  33. [41]

    Efficientnet: Rethinking model scaling for convolutional neural networks

    Mingxing Tan and Quoc Le. Efficientnet: Rethinking model scaling for convolutional neural networks. In International conference on machine learning, pages 6105–6114. PMLR,

  34. [42]

    Attention is all you need

    A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017. 4

    2017

  35. [43]

    Mindbridge: A cross-subject brain decoding frame- work

    Shizun Wang, Songhua Liu, Zhenxiong Tan, and Xinchao Wang. Mindbridge: A cross-subject brain decoding frame- work. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 11333–11342,

  36. [44]

    Image quality assessment: from error visibility to structural similarity

    Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Si- moncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4):600–612, 2004. 5

    2004

  37. [45]

    Dream: Visual decoding from reversing human vi- sual system

    Weihao Xia, Raoul de Charette, Cengiz Oztireli, and Jing- Hao Xue. Dream: Visual decoding from reversing human vi- sual system. In Proceedings of the IEEE/CVF Winter Confer- ence on Applications of Computer Vision, pages 8226–8235,

  38. [46]

    Umbrae: Unified multimodal brain decoding

    Weihao Xia, Raoul de Charette, Cengiz Oztireli, and Jing- Hao Xue. Umbrae: Unified multimodal brain decoding. In European Conference on Computer Vision, pages 242–259. Springer, 2025. 1, 2, 6, 14

    2025

  39. [47]

    Versatile diffusion: Text, images and variations all in one diffusion model

    Xingqian Xu, Zhangyang Wang, Gong Zhang, Kai Wang, and Humphrey Shi. Versatile diffusion: Text, images and variations all in one diffusion model. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 7754–7765, 2023. 5

    2023

  40. [48]

    Calor: Towards comprehensive model inversion defense

    Hongyao Yu, Yixiang Qiu, Hao Fang, Bin Chen, Sijin Yu, Bin Wang, Shu-Tao Xia, and Ke Xu. Calor: Towards comprehensive model inversion defense. arXiv preprint arXiv:2410.05814, 2024. 14

    2024

  41. [49]

    Icas: Detecting training data from autoregressive image gen- erative models

    Hongyao Yu, Yixiang Qiu, Yiheng Yang, Hao Fang, Tianqu Zhuang, Jiaxin Hong, Bin Chen, Hao Wu, and Shu-Tao Xia. Icas: Detecting training data from autoregressive image gen- erative models. arXiv preprint arXiv:2507.05068, 2025. 14

    2025

  42. [50]

    Attention-based-features-fusion emotion-guided fnirs classification network for prenatal de- pression recognition

    Sijin Yu, Xuejiao Li, Huirong Lei, Yingxue Yao, Zhaojin Chen, Zicong Zheng, Guodong Liang, Xiaofen Xing, Xin Zhang, and Chengfang Xu. Attention-based-features-fusion emotion-guided fnirs classification network for prenatal de- pression recognition. In International Workshop on...

    2024

  43. [51]

    Spherical u-net on cortical surfaces: meth- ods and applications

    Fenqiang Zhao, Shunren Xia, Zhengwang Wu, Dingna Duan, Li Wang, Weili Lin, John H Gilmore, Dinggang Shen, and Gang Li. Spherical u-net on cortical surfaces: meth- ods and applications. In Information Processing in Medical Imaging: 26th International Conference, IPMI 2019, Hong...

    2019

  44. [52]

    Stealthy shield defense: A con- ditional mutual information-based approach against black- box model inversion attacks

    Tianqu Zhuang, Hongyao Yu, Yixiang Qiu, Hao Fang, Bin Chen, and Shu-Tao Xia. Stealthy shield defense: A con- ditional mutual information-based approach against black- box model inversion attacks. In The Thirteenth International Conference on Learning Representations, 2025. 14 ...

    2025

  45. [54]

    Then, we independently apply zero-score normalization for each voxel within the train data of a subject

    package. Then, we independently apply zero-score normalization for each voxel within the train data of a subject. The val and test data are zero-centered using the mean and variance from the train data. The data split follows the standard setup used in previous works. Cortex S...

  46. [866]

    1, 3, 6, 12

    Springer, 2019. 1, 3, 6, 12

    2019

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.