Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

Structuring fMRI conditioning along the visual cortex hierarchy improves semantic image reconstruction from brain activity.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Hi-DREAM conditions a latent diffusion U-Net on early/mid/late visual-ROI streams via a multi-scale cortical pyramid and depth-matched ControlNet, reporting state-of-the-art semantic metrics on NSD fMRI-to-image reconstruction.

T0 review reviewed 2026-08-03 challenge →

load-bearing objection A genuinely new conditioning architecture for fMRI-to-image, but the headline SOTA numbers rest on an unverified metric swap and a missing MHLA spec. the 4 major comments →

arxiv 2511.11437 v2 pith:SFYFFBU2 submitted 2025-11-14 cs.CV cs.HC

Hi-DREAM: Brain-Inspired Hierarchical Diffusion for fMRI-to-Image Reconstruction via ROI Encoder and VisuAl Mapping

Guowei Zhang , Yun Zhao , Kai Sun , Moein Khajehnejad , Adeel Razi , Dinh Phung , Levin Kuhlmann This is my paper
classification cs.CV cs.HC
keywords fMRIimage reconstructiondiffusion modelvisual cortexregion of interestControlNetlatent diffusionbrain decoding
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that replacing a single global fMRI embedding with a hierarchy of region-of-interest (ROI) signals—early, middle, and late visual areas—improves how a latent diffusion model reconstructs natural images from brain activity. The method builds a multi-scale cortical pyramid from these ROI streams and injects each scale into the U-Net depth where it is most informative: shallow layers get early-area layout and edge hints, deeper layers get late-area semantic hints. On the four full-coverage subjects of a large-scale natural-scene fMRI dataset, Hi-DREAM reaches state-of-the-art semantic metrics (Inception and CLIP identification) while keeping low-level pixel and structural fidelity competitive. The work also claims interpretability: each ROI group has a distinct, inspectable contribution to the reconstruction, offering a lens on visual cortex function.

Core claim

The central claim is that the visual cortex's hierarchical organization can be mapped onto the depth structure of a diffusion U-Net, and that injecting fMRI-derived spatial priors at matched depths yields better semantic alignment than a single global embedding. A ROI adapter converts fMRI beta amplitudes and subject-specific ROI masks into a four-level pyramid of early/mid/late streams, and a depth-matched ControlNet injects these hints via residual and FiLM fusion with a per-scale strength schedule. On subjects who completed the full scan schedule, Hi-DREAM achieves the best Inception and CLIP two-way identification accuracy and the lowest feature distances among compared decoders, while P

What carries the argument

The key machinery is the ROI adapter and the depth-matched ROI-conditioned ControlNet. The adapter creates a multi-scale cortical pyramid by blurring and downsampling ROI masks, applying learnable per-scale gates and group-specific mixers, and aggregating with nonnegative coefficients; the ControlNet translates each scale into a hint and injects it into the corresponding U-Net block using residual plus FiLM modulation, with a cosine strength schedule over diffusion time and a soft budget that limits shallow-scale dominance. This realizes the selective-depth hypothesis: early ROIs dominate shallow scales, late ROIs dominate deep scales, and middle ROIs bridge them.

Load-bearing premise

The claim rests on the assumption that early visual areas are most useful at shallow U-Net depths and late areas at deep depths; if the observed gains actually come from the extra parameters or the attention module, the brain-inspired interpretation collapses.

What would settle it

Run the full model but swap the depth assignments so early-ROI hints go to deep layers and late-ROI hints to shallow layers; if Inception/CLIP accuracy does not drop, the selective-depth mapping is not the driver. A second control would use the same architecture and parameter count but with randomly grouped ROIs instead of the early/mid/late split.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If the hierarchy-depth alignment is right, brain decoding can be improved by respecting cortical organization rather than flattening fMRI into one embedding.
  • The per-ROI attribution suggests early, middle, and late visual areas play complementary, inspectable roles in generation, matching neuroscientific divisions of labor.
  • The compact 2D pyramid keeps extra compute and parameters negligible relative to the diffusion backbone, enabling single-GPU training and fast iteration.
  • The depth-specific guidance strengths and ROI weights act as interpretable knobs that can trade texture/layout against semantics and visualize which regions drive each stage.
  • The approach is complementary to cross-subject and perceptual-objective methods, so it can likely be combined with them for further gains.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A direct test of the depth-alignment premise would be to permute which ROI group is injected at which U-Net depth; if semantic metrics do not drop, the brain-inspired mapping is not the cause of the gains.
  • A parameter-matched control that replaces the early/mid/late grouping with random partitions of the same ROIs would isolate whether the hierarchy or just extra capacity matters.
  • Because the late group is limited to a face-selective area, extending to scene-selective regions could broaden semantic coverage; whether that dilutes identity cues is an open question.
  • The depth-wise modulation maps could be repurposed as predictions of which cortical areas are engaged by particular image features, turning the decoder into a hypothesis generator for neuroscience.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes Hi-DREAM, a brain-inspired diffusion decoder for fMRI-to-image reconstruction that explicitly structures fMRI conditioning by early, middle, and late visual ROI streams. A ROI adapter builds a multi-scale cortical pyramid, and a depth-matched ROI-Conditioned ControlNet injects these priors at corresponding U-Net depths during denoising. Evaluated on NSD subjects 1, 2, 5, and 7, the paper reports state-of-the-art high-level semantic alignment (Inception 98.13%, CLIP 97.45%, Eff/SwAV distances 0.537/0.339) while acknowledging lower low-level fidelity than MindEye2. Ablations of early/middle/late ROI groups and of the adapter/MHLA modules are used to support a division-of-labor interpretation consistent with visual cortex hierarchy. The authors also explicitly list limitations, including partial anatomical coverage and low-level fidelity.

Significance. If the empirical claims hold, Hi-DREAM is a useful contribution: it offers a compact, interpretable alternative to monolithic fMRI embeddings, makes the cortical hierarchy explicit in conditioning, and provides a concrete framework for probing which visual areas drive which reconstruction aspects. The core idea is well motivated and the ROI-group ablations are informative. The paper also ships several positive features: a clear two-stage training schedule, explicit regularization, and a self-critical Limitations section. However, the central SOTA claim currently rests on an unverified comparison of metric definitions, and the MHLA module credited with large gains is never specified. With those repaired, the work could be a solid contribution to the fMRI-to-image reconstruction literature.

major comments (4)
  1. [§4.2, Table 1] The headline SOTA claim on Eff/SwAV is unverifiable. The protocol states these are “mean correlation distances (↓),” while the cited baseline papers report EfficientNet/SwAV as two-way identification accuracy (↑). If the baseline numbers are taken from the published papers, the table mixes incompatible scales; if the authors recomputed them, no code or rerun details are given. The phrase “harmonized” is not enough. Please report all methods under the same metric (preferably the standard accuracy) and include per-subject error bars or significance tests, as the reported margins are small.
  2. [§3 (Method) vs. Table 3 and Fig. 2] MHLA (Multi-Head Latent Attention) is central to the ablation story: Table 3 credits it with the large jumps in Inception (92.89%→98.13%) and CLIP (91.42%→97.45%). But MHLA is never defined in the Method section. It appears only in the caption of Fig. 2 and in the ablation text. The reader cannot determine its inputs, gating mechanism, depth locations, parameter count, or whether the gain comes from extra parameters rather than “cross-group cooperation.” This blocks reproducibility and weakens the brain-inspired interpretation. Add a full specification or remove MHLA from the model and ablations.
  3. [§4.2, Table 1] The “state-of-the-art high-level semantic” claim is selective. Hi-DREAM’s AlexNet(2)=87.68% and AlexNet(5)=92.27% are below MindEye2 (96.1%, 98.6%) and Psychometry (96.4%, 98.6%), and low-level PixCorr/SSIM are well below MindEye2. If AlexNet features are considered high-level, the statement is not supportable; if not, the metric taxonomy should be explicitly defined. The abstract and contributions should be qualified accordingly.
  4. [§3.2, Eqs. (1)-(2), and §4.3(C)] The central premise—early/mid/late ROI streams should map to shallow/middle/deep U-Net depths—is never tested. The ablations remove entire ROI groups but do not permute or shuffle the depth assignment, so the observed gains could come from any structured multi-scale conditioning rather than from the specific cortical-depth alignment. The face-only analysis is also somewhat circular: FFA is deliberately placed in the late group, so the finding that the late group helps faces does not independently validate FFA’s functional role. A depth-shuffle control and a control with a different late-stream ROI would substantially strengthen the neuroscience claim.
minor comments (4)
  1. [Title and Abstract] “VisuAl Mapping” in the title appears to be an accidental capitalization; “an Region of Interest” should be “a Region of Interest.”
  2. [Table 2] The three rows with two checkmarks are ambiguous—the reader must infer which ROI group is missing from the text. Label the rows explicitly (e.g., “w/o Late,” “w/o Early,” “w/o Middle”).
  3. [§4.2, Protocol] The metric definitions should be stated before the results are presented. In particular, clarify what “mean correlation distance” means, how distances are normalized, and how a “two-way” protocol applies when top-k is mentioned.
  4. [§4.1] “NSD includes 8healthy adults” has a spacing typo; also specify how many total test trials are averaged per subject beyond the statement of unique stimuli.

Circularity Check

1 steps flagged

Reconstruction benchmark is externally grounded, but the FFA/face attribution and the depth-alignment narrative partly restate the architecture's own ROI definitions and priors.

specific steps
  1. self definitional [§4.3(C) Face-only subset analysis; group definitions in Figure 1/§3.2]
    "Our Late group contains only FFA (a face-selective region), making faces an informative probe of high-level guidance. ... This matches FFA's role: Late-ROI evidence steers deeper U-Net blocks toward identity-consistent features."

    The Late stream is by construction the FFA mask (Figure 1 labels Late as FFA). Evaluating a face-only subset and then attributing the face-identity gain to 'FFA's role' is unpacking the input definition: the model was given FFA activity as the late/semantic channel, so observing that this channel helps faces is the designed pairing, not an independent derivation of FFA function. It supports the architecture's interpretability claim but does not make the image-reconstruction benchmark itself circular.

full rationale

The central reconstruction claim (Table 1) is evaluated against held-out NSD ground-truth stimuli, so the performance result is externally grounded and not equivalent to its inputs. I found no load-bearing self-citation, no imported uniqueness theorem, and no ansatz smuggled in via citation: the references are to independent prior systems, and the authors do not rely on their own prior work to force the design. The only notable circularity is interpretive: the late semantic stream is defined as FFA, and the face-only subset is then used to conclude that Late-ROI/FFA guidance drives face identity, which restates the construction. A related but milder built-in element is the depth-alignment prior: Eq. (1) and the w_{r,s} gate prior bias early→shallow and late→deep, so the later 'division of labor' narrative partly confirms a prior encoded in the architecture. The Eff/SwAV metric-convention issue raised in the skeptic headline is serious but is a comparability/verifiability problem (reported units and baseline protocols are not fully established), not a reduction of a prediction to its own inputs; per the rules I do not score it as circularity. Overall: one self-definitional interpretive step, while the main benchmark remains independent, hence score 3.

Axiom & Free-Parameter Ledger

8 free parameters · 5 axioms · 0 invented entities

No new physical entities or natural-kind postulates are introduced; the multi-scale cortical pyramid is a learned representation rather than a distinct entity with independent evidence. The main free parameters are the learned weights and hand-chosen hyperparameters that determine how ROI streams are mapped to U-Net depths.

free parameters (8)
  • α_{g,s} per-group/per-scale aggregation weights = not reported
    Learned aggregation coefficients over ROI groups at each scale (Eq. 1); trained from NSD data.
  • w_{r,s} ROI gates = not reported
    Learnable per-ROI, per-scale gates in [0,1] biasing early ROIs to shallow scales; trained.
  • λ_s guidance strengths = (0.8, 0.6, 0.5) for shallow/mid/deep
    Chosen on validation to balance structure and semantics (§3.5, §4.1).
  • λ_max^s and ρ_s cosine schedule parameters = not specified
    Define the time-dependent injection schedule λ_s(t); values not reported.
  • soft energy budget η = not specified
    Bounds injected ControlNet energy across depths (§3.3); exact value not reported.
  • pyramid scales S = {64, 128, 256, 512}
    Hand-chosen to match U-Net blocks (§3.2).
  • guidance channels d_s = {8, 16, 32}
    Channel counts per scale for ROI mixers (§3.2/§4.1).
  • regularization weights (L2 on α, KL on w, TV) = not reported
    Loss coefficients for stability regularizers (§3.5); exact values absent.
axioms (5)
  • domain assumption Human visual cortex is hierarchically organized: V1/V2 encode local structure, V3/V4 parts/color, FFA categories/faces.
    Adopted from neuroscience and used to define ROI groups in §3.1-3.2.
  • domain assumption NSD-derived ROI masks and normalized beta amplitudes carry enough spatial and semantic information to guide image reconstruction.
    Used to build evidence maps a_r, m_r in §3.4; not independently verified in this paper.
  • domain assumption Latent diffusion U-Net depths correspond to a low-to-high feature hierarchy (shallow = edges/layout, deep = semantics).
    Basis for matching pyramid scales to U-Net depths (§3.2, Eq. 2).
  • ad hoc to paper The underspecified MHLA module performs gated cross-attention between ROI latents and U-Net features as described only in Fig. 2/Table 3.
    No equation or implementation specification is provided; the ablation attributes gains to MHLA.
  • standard math Standard diffusion/ControlNet machinery (DDIM, PLMS, zero-conv, cross-attention) behaves as in prior literature.
    Relies on [24, 29, 36, 37, 38]; assumed correct.

reviewed 2026-08-03 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Hi-DREAM: Brain-Inspired Hierarchical Diffusion for fMRI-to-Image Reconstruction via ROI Encoder and VisuAl Mapping." pith.science (2026). https://pith.science/paper/SFYFFBU2

@misc{pith2026251111437,
  author       = {Pith},
  title        = {Pith review of: Hi-DREAM: Brain-Inspired Hierarchical Diffusion for fMRI-to-Image Reconstruction via ROI Encoder and VisuAl Mapping},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SFYFFBU2}},
  note         = {Machine review of arXiv:2511.11437}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Reconstructing natural images from fMRI requires bridging neural activity with both the structural and semantic representations used by modern generative models. Existing diffusion-based decoders often condition on a single global fMRI embedding, which limits their ability to exploit the hierarchical organization of the visual cortex and makes the contribution of different visual areas difficult to inspect. We propose Hi-DREAM, a brain-inspired hierarchical diffusion framework that structures fMRI conditioning according to early, middle, and late visual Regions of Interest (ROI) streams. A ROI adapter converts these streams into a multi-scale cortical pyramid, and a lightweight ROI-conditioned ControlNet injects the resulting anatomy-aware priors into matched U-Net depths during denoising. Experiments on the Natural Scenes Dataset (NSD) show that Hi-DREAM achieves state-of-the-art high-level semantic reconstruction while retaining strong low-level structure. Further ablation and attribution analyses show that the proposed hierarchy-aware conditioning is effective, and that different ROI streams provide complementary, inspectable contributions to reconstruction.

Figures

Figures reproduced from arXiv: 2511.11437 by Adeel Razi, Dinh Phung, Guowei Zhang, Kai Sun, Levin Kuhlmann, Moein Khajehnejad, Yun Zhao.

Figure 1
Figure 1. Figure 1: Brain-inspired decoding pipeline. HI-DREAMHi-DREAM mirrors the visual hierarchy: fMRI signals are grouped into early/mid/late ROIs (e.g., V1/V2 for edges, V3/V4 for color and parts, LOC/FFA for semantics) and transformed by a ROI adapter into a multi-scale cortical pyramid. Right: visual stimuli and few-sample reconstructions from HI-DREAM, illustrating faithful structure and semantics while retaining inte… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of HI-DREAM. fMRI signals are summarized into hierarchical ROI streams (early/mid/late), each with group-specific processing. A Multi-Head Latent Attention (MHLA) module performs gated cross-attention [36, 37] between ROI-derived latents and U-Net features at multiple depths to capture cooperative interactions among areas. In parallel, a lightweight ROI-conditioned ControlNet consumes compact cond… view at source ↗
Figure 3
Figure 3. Figure 3: Qualitative comparison on the NSD test set. Each group shows the stimulus (left) and the corresponding [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Ablation on hierarchy modules (accuracy). Bars show, from left to right, [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Face-only subset. HI-DREAM better preserves identity- and semantics-related cues (eyes, mouth, hairline), consistent with Late-ROI (FFA) guidance, while Early-ROI guidance stabilizes contours and local geometry. addition, our anatomy is partial: the model conditions on a limited subset of visual ROIs (e.g., a late-stage emphasis on FFA), whereas cortical organization is far richer. This subset choice const… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. HyNeuralMap: Hyperbolic Mapping of Visual Semantics to Neural Hierarchies

    cs.CV 2026-05 unverdicted novelty 5.0

    HyNeuralMap applies the hyperbolic Lorentz model to embed visual semantics and neural responses into a shared hierarchical space, outperforming Euclidean baselines on semantic prediction and cross-modal retrieval.

Reference graph

Works this paper leans on

63 extracted references · 9 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Joseph P. Hornak. The basics of mri.https://www.cis.rit.edu/htbooks/mri/, 1997. Online resource

    1997

  2. [2]

    Norman Cheng, E

    Robert W Brown, Y-C. Norman Cheng, E. Mark Haacke, Michael R Thompson, and Ramesh Venkatesan. Magnetic Resonance Imaging: Physical Principles and Sequence Design. Wiley, 2nd edition, 2014

    2014

  3. [3]

    Cambridge University Press, 2006

    Donald W McRobbie, Elizabeth A Moore, Martin J Graves, and Martin R Prince.MRI from Picture to Proton. Cambridge University Press, 2006

    2006

  4. [4]

    Nayak, and Paul Glynn

    Seiji Ogawa, Tso-Ming Lee, Anil S. Nayak, and Paul Glynn. Oxygenation-sensitive contrast in magnetic resonance image of rodent brain at high magnetic fields.Proceedings of the National Academy of Sciences, 87(24):9868–9872, 1990

    1990

  5. [5]

    Logothetis, Josef Pauls, Mark Augath, Tandava Trinath, and Axel Oeltermann

    Nikos K. Logothetis, Josef Pauls, Mark Augath, Tandava Trinath, and Axel Oeltermann. Neurophysiological investigation of the basis of the fmri signal.Nature, 412(6843):150–157, 2001

    2001

  6. [6]

    Gary H. Glover. Deconvolution of impulse response in event-related bold fmri.NeuroImage, 9(4):416–429, 1999

    1999

  7. [7]

    Functional magnetic resonance imaging: an introduction to methods.Sinauer Associates, 2004

    Scott A Huettel, Allen W Song, and Gregory McCarthy. Functional magnetic resonance imaging: an introduction to methods.Sinauer Associates, 2004. Chapter 1: Introduction to fMRI

    2004

  8. [8]

    V oxel-based morphometry–the methods.Neuroimage, 11(6 Pt 1):805–821, 2000

    John Ashburner and Karl J Friston. V oxel-based morphometry–the methods.Neuroimage, 11(6 Pt 1):805–821, 2000

    2000

  9. [9]

    Freesurfer.Neuroimage, 62(2):774–781, 2012

    Bruce Fischl. Freesurfer.Neuroimage, 62(2):774–781, 2012

    2012

  10. [10]

    Brain-streams: fmri-to-image reconstruction with multi-modal guidance.arXiv preprint arXiv:2409.12099, 2024

    Jaehoon Joo, Taejin Jeong, and Seongjae Hwang. Brain-streams: fmri-to-image reconstruction with multi-modal guidance.arXiv preprint arXiv:2409.12099, 2024

    Pith/arXiv arXiv 2024

  11. [11]

    Attention module improves both performance and inter- pretability of 4d fmri decoding neural network.arXiv preprint arXiv:2110.00920, 2021

    Zhoufan Jiang, Yanming Wang, ChenWei Shi, et al. Attention module improves both performance and inter- pretability of 4d fmri decoding neural network.arXiv preprint arXiv:2110.00920, 2021

    Pith/arXiv arXiv 2021

  12. [12]

    High-resolution image reconstruction with latent diffusion models from human brain activity

    Yu Takagi and Shinji Nishimoto. High-resolution image reconstruction with latent diffusion models from human brain activity. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14453–14463, 2023

    2023

  13. [13]

    Mindeye2: Shared-subject models enable fmri-to-image with 1 hour of data.arXiv preprint arXiv:2403.11207, 2024

    Paul S Scotti, Mihir Tripathy, Cesar Kadir Torrico Villanueva, Reese Kneeland, Tong Chen, Ashutosh Narang, Charan Santhirasegaran, Jonathan Xu, Thomas Naselaris, Kenneth A Norman, et al. Mindeye2: Shared-subject models enable fmri-to-image with 1 hour of data.arXiv preprint arXiv:2403.11207, 2024

    Pith/arXiv arXiv 2024

  14. [14]

    Mindbridge: A cross-subject brain decoding framework

    Shizun Wang, Songhua Liu, Zhenxiong Tan, and Xinchao Wang. Mindbridge: A cross-subject brain decoding framework. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11333–11342, 2024

    2024

  15. [15]

    Psychometry: An omnifit model for image reconstruction from human brain activity

    Ruijie Quan, Wenguan Wang, Zhibo Tian, Fan Ma, and Yi Yang. Psychometry: An omnifit model for image reconstruction from human brain activity. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 233–243, 2024

    2024

  16. [16]

    Cinematic mindscapes: High-quality video reconstruction from brain activity.Advances in Neural Information Processing Systems, 36, 2024

    Zijiao Chen, Jiaxin Qing, and Juan Helen Zhou. Cinematic mindscapes: High-quality video reconstruction from brain activity.Advances in Neural Information Processing Systems, 36, 2024

    2024

  17. [17]

    Seeing beyond the brain: Masked modeling conditioned diffusion model for human vision decoding

    Zijiao Chen, Jiaxin Qing, Tiange Xiang, Wan Lin Yue, and Juan Helen Zhou. Seeing beyond the brain: Masked modeling conditioned diffusion model for human vision decoding. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023

    2023

  18. [18]

    Mindtuner: Cross-subject visual decoding with visual fingerprint and lora fine-tuning

    Ji Wang et al. Mindtuner: Cross-subject visual decoding with visual fingerprint and lora fine-tuning. 2024

    2024

  19. [19]

    Mind reader: Reconstructing complex images from brain activities.Advances in Neural Information Processing Systems, 35:29624–29636, 2022

    Sikun Lin, Thomas Sprague, and Ambuj K Singh. Mind reader: Reconstructing complex images from brain activities.Advances in Neural Information Processing Systems, 35:29624–29636, 2022

    2022

  20. [20]

    Distributed and overlapping representation of faces and objects in ventral temporal cortex.Science, 293:20–27, 2001

    J Haxby. Distributed and overlapping representation of faces and objects in ventral temporal cortex.Science, 293:20–27, 2001

    2001

  21. [21]

    Reconstructing visual experiences from brain activity evoked by natural movies.Current biology, 21(19):1641–1646, 2011

    Shinji Nishimoto, An T Vu, Thomas Naselaris, Yuval Benjamini, Bin Yu, and Jack L Gallant. Reconstructing visual experiences from brain activity evoked by natural movies.Current biology, 21(19):1641–1646, 2011

    2011

  22. [22]

    Bayesian reconstruction of natural images from human brain activity.Neuron, 63(6):902–915, 2009

    Thomas Naselaris, Ryan J Prenger, Kendrick N Kay, Michael Oliver, and Jack L Gallant. Bayesian reconstruction of natural images from human brain activity.Neuron, 63(6):902–915, 2009

    2009

  23. [23]

    Deep image reconstruction from human brain activity.PLoS Computational Biology, 15(1):e1006633, 2019

    Guohua Shen, Tomoyasu Horikawa, Kei Majima, and Yukiyasu Kamitani. Deep image reconstruction from human brain activity.PLoS Computational Biology, 15(1):e1006633, 2019. 10

    2019

  24. [24]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. InProceedings of the IEEE/CVF international conference on computer vision, pages 3836–3847, 2023

    2023

  25. [25]

    Through their eyes: Multi -subject brain decoding with simple alignment.Imaginative Minds Journal, 2023

    Awangga et al. Through their eyes: Multi -subject brain decoding with simple alignment.Imaginative Minds Journal, 2023

    2023

  26. [26]

    Neuropictor: Refining fmri-to-image reconstruction via multi-individual pretraining and single-subject refining.arXiv preprint arXiv:2403.18211, 2024

    Jaehoon Joo et al. Neuropictor: Refining fmri-to-image reconstruction via multi-individual pretraining and single-subject refining.arXiv preprint arXiv:2403.18211, 2024

    Pith/arXiv arXiv 2024

  27. [27]

    Uncovering cognitive taskonomy through transfer learning in masked autoencoder- based fmri reconstruction.arXiv preprint arXiv:2407.00033, 2024

    Youzhi Qu, Junfeng Xia, et al. Uncovering cognitive taskonomy through transfer learning in masked autoencoder- based fmri reconstruction.arXiv preprint arXiv:2407.00033, 2024

    Pith/arXiv arXiv 2024

  28. [28]

    A study of deep perceptual metrics for image quality assessment.ArXiv, page 2202.08692, 2022

    Rémi Kazmierczak, Gianni Franchi, Nacim Belkhir, Antoine Manzanera, and David Filliat. A study of deep perceptual metrics for image quality assessment.ArXiv, page 2202.08692, 2022

    Pith/arXiv arXiv 2022

  29. [29]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022

    2022

  30. [30]

    Denoising diffusion probabilistic models.arXiv preprint arXiv:2006.11239, 2020

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models.arXiv preprint arXiv:2006.11239, 2020

    Pith/arXiv arXiv 2006

  31. [31]

    Multimodal fusion of brain imaging data: Methods and applications

    Na Luo, Weiyang Shi, Zhengyi Yang, et al. Multimodal fusion of brain imaging data: Methods and applications. Machine Intelligence Research, 21:136–152, 2024

    2024

  32. [32]

    An adaptively weighted averaging method for regional time series extraction of fmri-based brain decoding.arXiv preprint arXiv:2407.08174, 2024

    Jianfei Zhu, Baichun Wei, Jiaru Tian, Feng Jiang, and Chunzhi Yi. An adaptively weighted averaging method for regional time series extraction of fmri-based brain decoding.arXiv preprint arXiv:2407.08174, 2024

    Pith/arXiv arXiv 2024

  33. [33]

    A massive 7t fmri dataset to bridge cognitive neuroscience and artificial intelligence.Nature neuroscience, 25(1):116–126, 2022

    Emily J Allen, Ghislain St-Yves, Yihan Wu, Jesse L Breedlove, Jacob S Prince, Logan T Dowdle, Matthias Nau, Brad Caron, Franco Pestilli, Ian Charest, et al. A massive 7t fmri dataset to bridge cognitive neuroscience and artificial intelligence.Nature neuroscience, 25(1):116–126, 2022

    2022

  34. [34]

    Across-subject ensemble-learning alleviates the need for large samples for fmri decoding

    Himanshu Aggarwal, Liza Al-Shikhley, and Bertrand Thirion. Across-subject ensemble-learning alleviates the need for large samples for fmri decoding. InMICCAI 2024, 2024

    2024

  35. [35]

    Enhancing cross-subject fmri-to-video decoding with global-local functional alignment

    Chong Li, Xuelin Qian, Yun Wang, Jingyang Huo, Xiangyang Xue, Yanwei Fu, and Jianfeng Feng. Enhancing cross-subject fmri-to-video decoding with global-local functional alignment. InEuropean Conference on Computer Vision, pages 353–369. Springer, 2024

    2024

  36. [36]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. InAdvances in Neural Information Processing Systems (NeurIPS), volume 30, 2017

    2017

  37. [37]

    Align before fuse: Vision and language representation learning with momentum distillation

    Xiaowei Li, Nan Duan, Yeyun Fang, Ming Gong, Daxin Jiang, and Ming Zhou. Align before fuse: Vision and language representation learning with momentum distillation. InNeurIPS, 2021

    2021

  38. [38]

    Classifier-free diffusion guidance

    Jonathan Ho, Tim Salimans, William Chan, John Schulman, Jascha Sohl-Dickstein, and Eric Chen. Classifier-free diffusion guidance. InAdvances in Neural Information Processing Systems (NeurIPS), 2022

    2022

  39. [39]

    What’s new in neuroimaging methods?Annals of the New York Academy of Sciences, 1156(1):260–293, 2009

    Peter A Bandettini. What’s new in neuroimaging methods?Annals of the New York Academy of Sciences, 1156(1):260–293, 2009

    2009

  40. [40]

    Magnetoencephalography for brain electrophysiology and imaging.Nature Neuroscience, 20(3):327–339, 2017

    Sylvain Baillet. Magnetoencephalography for brain electrophysiology and imaging.Nature Neuroscience, 20(3):327–339, 2017

    2017

  41. [41]

    Where does eeg come from and what does it mean?Trends in Neurosciences, 40(4):208–218, 2017

    Michael X Cohen. Where does eeg come from and what does it mean?Trends in Neurosciences, 40(4):208–218, 2017

    2017

  42. [42]

    Felleman and David C

    Daniel J. Felleman and David C. Van Essen. Distributed hierarchical processing in the primate cerebral cortex. Cerebral Cortex, 1(1):1–47, 1991

    1991

  43. [43]

    The human visual cortex.Annual Review of Neuroscience, 27:649–677, 2004

    Kalanit Grill-Spector and Rafael Malach. The human visual cortex.Annual Review of Neuroscience, 27:649–677, 2004

    2004

  44. [44]

    DiCarlo, Davide Zoccolan, and Nicole C

    James J. DiCarlo, Davide Zoccolan, and Nicole C. Rust. How does the brain solve visual object recognition? Neuron, 73(3):415–434, 2012

    2012

  45. [45]

    Kravitz, Laura D

    Dwight J. Kravitz, Laura D. Vinson, and Chris I. Baker. From ’what’ to ’where’ and back again: Cortical organization of the object processing pathway.Frontiers in Psychology, 4:176, 2013

    2013

  46. [46]

    Wandell and Jonathan Winawer

    Brian A. Wandell and Jonathan Winawer. Computational neuroimaging and population receptive fields.Current Opinion in Neurobiology, 25:94–102, 2015. 11

    2015

  47. [47]

    Compressive spatial summation in human visual cortex.Journal of neurophysiology, 110(2):481–494, 2013

    Kendrick N Kay, Jonathan Winawer, Aviv Mezer, and Brian A Wandell. Compressive spatial summation in human visual cortex.Journal of neurophysiology, 110(2):481–494, 2013

    2013

  48. [48]

    Functional connectome fingerprinting: identifying individuals using patterns of brain connectivity.Nature neuroscience, 18(11):1664–1671, 2015

    Emily S Finn, Xilin Shen, Dustin Scheinost, Monica D Rosenberg, Jessica Huang, Marvin M Chun, Xenophon Papademetris, and R Todd Constable. Functional connectome fingerprinting: identifying individuals using patterns of brain connectivity.Nature neuroscience, 18(11):1664–1671, 2015

    2015

  49. [49]

    Natural speech reveals the semantic maps that tile human cerebral cortex.Nature, 532(7600):453–458, 2016

    Alexander G Huth, Wendy A De Heer, Thomas L Griffiths, Frédéric E Theunissen, and Jack L Gallant. Natural speech reveals the semantic maps that tile human cerebral cortex.Nature, 532(7600):453–458, 2016

    2016

  50. [50]

    The functional architecture of the ventral temporal cortex and its role in categorization.Nature Reviews Neuroscience, 15(8):536–548, 2014

    Kalanit Grill-Spector and Kevin S Weiner. The functional architecture of the ventral temporal cortex and its role in categorization.Nature Reviews Neuroscience, 15(8):536–548, 2014

    2014

  51. [51]

    The human connectome project 7 tesla retinotopy dataset: Description and population receptive field analysis.Journal of vision, 18(13):23–23, 2018

    Noah C Benson, Keith W Jamison, Michael J Arcaro, An T Vu, Matthew F Glasser, Timothy S Coalson, David C Van Essen, Essa Yacoub, Kamil Ugurbil, Jonathan Winawer, et al. The human connectome project 7 tesla retinotopy dataset: Description and population receptive field analysis.Journal of vision, 18(13):23–23, 2018

    2018

  52. [52]

    Distributed hierarchical processing in the primate cerebral cortex

    Daniel J Felleman and David C Van Essen. Distributed hierarchical processing in the primate cerebral cortex. Cerebral Cortex, 1(1):1–47, 1991

    1991

  53. [53]

    Visual field maps in human cortex.Neuron, 56(2):366–383, 2007

    Brian A Wandell, Serge O Dumoulin, and Alyssa A Brewer. Visual field maps in human cortex.Neuron, 56(2):366–383, 2007

    2007

  54. [54]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. InInternational Conference on Learning Representations (ICLR), 2015. arXiv:1412.6980

    Pith/arXiv arXiv 2015

  55. [55]

    Incorporating nesterov momentum into adam

    Timothy Dozat. Incorporating nesterov momentum into adam. Stanford CS229 Report, 2016

    2016

  56. [56]

    Dream: Visual decoding from reversing human visual system

    Weihao Xia, Raoul De Charette, Cengiz Oztireli, and Jing-Hao Xue. Dream: Visual decoding from reversing human visual system. InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 8226–8235, 2024

    2024

  57. [57]

    Reconstructing perceived and imagined images from human brain activity using a latent diffusion model.bioRxiv, 2023

    Jiaming Ren, Xinyu Li, Haohan Zhu, et al. Reconstructing perceived and imagined images from human brain activity using a latent diffusion model.bioRxiv, 2023

    2023

  58. [58]

    Image quality assessment: From error visibility to structural similarity.IEEE Transactions on Image Processing, 13(4):600–612, 2004

    Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: From error visibility to structural similarity.IEEE Transactions on Image Processing, 13(4):600–612, 2004

    2004

  59. [59]

    Imagenet classification with deep convolutional neural networks.Advances in neural information processing systems, 25, 2012

    Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks.Advances in neural information processing systems, 25, 2012

    2012

  60. [60]

    Rethinking the inception architecture for computer vision

    Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 2818–2826, 2016

    2016

  61. [61]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Christopher Hallacy, Aditya Ramesh, et al. Learning transferable visual models from natural language supervision. InICML, 2021

    2021

  62. [62]

    Efficientnet: Rethinking model scaling for convolutional neural networks

    Mingxing Tan and Quoc V Le. Efficientnet: Rethinking model scaling for convolutional neural networks. In International Conference on Machine Learning (ICML), pages 6105–6114, 2019

    2019

  63. [63]

    Unsupervised learning of visual features by contrasting cluster assignments.Advances in neural information processing systems, 33:9912–9924, 2020

    Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsupervised learning of visual features by contrasting cluster assignments.Advances in neural information processing systems, 33:9912–9924, 2020. 12

    2020

This paper was first reviewed by deepseek-v4-flash on August 3, 2026.