Pith. sign in

REVIEW 4 major objections 5 minor 64 references

Cross-Subject Mind Decoding from Inaccurate Representations

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A cross-subject fMRI decoder with bidirectional representation learning and error-tolerant diffusion conditioning reconstructs held-out natural scenes more faithfully than prior methods.

desk verdict A credible cross-subject fMRI decoder with a real-but-unvalidated synthetic-corruption training setup; deserves review, but the SOTA claim needs error bars and a proxy check. read the letter →

arxiv 2507.19071 v1 pith:CZLNBGP2 submitted 2025-07-25 cs.CV

Yangyang Xu , Bangzhen Liu , Wenqi Shao , Yong Du , Shengfeng He , Tingting Zhu This is my paper · ORCID
classification cs.CV
keywords fMRIdecodingcross-subjectmindbidirectionalautoencodersubjectbiasmodulationsemanticrefinementvisualcoherencediffusionmodelreconstructionNaturalScenesDataset
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that cross-subject fMRI-to-image decoding can be made accurate by fixing both error sources: the initial representation mapping and the diffusion decoding of imperfect representations. It proposes a Bidirectional Autoencoder Intertwining framework that maps fMRI voxels to and from semantic, edge, and color representations, with per-subject bias modulation, then adds semantic refinement and visual coherence modules that keep the frozen diffusion decoder faithful even when the predicted representations are rough. If true, a single decoder could serve many subjects and be personalized to a new subject with only hundreds of training samples, substantially lowering the data cost of brain-decoding systems.

What carries the argument

The load-bearing object is the Bidirectional Autoencoder Intertwining (BAI), defined by two autoencoders—one encoding fMRI voxels, one encoding semantic/edge/color representations—whose decoders are swapped to translate between domains. It is the first stage that predicts representations and is trained with cycle-consistency losses that enforce round-trip agreement; the SBMM (per-subject MLPs modulating normalized feature mean and variance) removes subject-specific bias from the shared latent space; the SRM (a transformer with self- and cross-attention) pulls noisy semantic embeddings toward clean CLIP text embeddings; and the VCM (a convolutional network predicting two spatial weight maps $\alpha_e$ and $\alpha_c$) reweights the edge and color conditioning features inside the frozen ControlNet and Stable Diffusion UNet. Together, these modules make the pipeline tolerant to the very inaccuracies the first stage produces.

What would settle it

Take a held-out subject or a held-out set of fMRI runs, collect the real BAI-predicted representations for those runs, and train SRM and VCM on those real prediction-target pairs instead of on the pseudo-imprecise pairs from 10,000 Stable Diffusion images; if reconstruction metrics drop when the modules trained on synthetic errors are applied to real errors, the central transfer assumption is false.

Watch

Extended reading notes

Core claim

The paper claims that cross-subject mind decoding fails for two sequential reasons: unidirectional fMRI-to-representation mappings produce inaccurate representations, and the diffusion decoder treats those inaccuracies as ground truth. To fix both, it builds the Bidirectional Autoencoder Intertwining (BAI): two autoencoders, one for fMRI voxels and one for image representations (semantic embedding, edge map, color palette), that share a subject-invariant latent space and support translation in both directions by swapping decoders, trained with reconstruction, translation, and cycle-consistency losses. A Subject Bias Modulation Module (SBMM) inside the fMRI encoder and decoder applies per-subject mean and variance modulation. Then, for the decoding stage, a Semantic Refinement Module (SRM) refines the predicted semantic embedding, and a Visual Coherence Module (VCM) learns per-pixel fusion weights that down-weight inaccurate edge and color conditions before they enter ControlNet and Stable Diffusion. With this pipeline, the paper reports the highest or tied-best values on most of the eight NSD benchmark metrics and shows that adapting a new subject by training only its SBMM with 500 or 1,500 samples outperforms training the full model from scratch.

Load-bearing premise

The refinement and coherence modules are trained on pseudo-imprecise representation-image pairs created by running 10,000 Stable Diffusion images through BAI's reconstruction, and the framework assumes these synthetic corruption patterns approximate the errors BAI actually makes on real held-out fMRI.

Editorial extensions

If this is right

  • A single cross-subject model replaces per-subject retraining of the whole decoder, since the shared latent space plus per-subject modulation handles individual variability.
  • For a new subject, only the small bias-modulation module needs training; with 500 or 1,500 samples this adapts better than training the full model from scratch.
  • Because the second stage tolerates inaccurate representations, the framework can use rough predicted edges, colors, and semantics and still produce faithful images, reducing sensitivity to representation quality.
  • The bidirectional mapping also synthesizes subject-specific fMRI from an image, so the same model can generate neural data for unobserved stimuli, useful for simulation and augmentation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: If the pseudo-imprecise training transfer holds, error-tolerant conditioning could be applied to other domains where a predictor feeds noisy intermediate representations into a generative model, such as text-to-speech or video generation.
  • Editorial inference: The large gain from SBMM-only adaptation suggests subject differences are largely low-dimensional statistical shifts; measuring how much of the cross-subject gap remains after bias-only adaptation would tell whether deeper personalization is needed.
  • Editorial inference: The reported voxel-gradient alignment with visual hierarchy implies the learned representation space could be used as a hypothesis probe in neuroscience, for example by comparing model-internal representations against known cortical selectivity maps.
  • Editorial inference: A direct test of the synthetic-error assumption is to compare SRM and VCM trained on BAI's actual errors on held-out fMRI versus on the synthetic Stable Diffusion reconstructions; if the synthetic-trained versions match, the data-augmentation trick should transfer broadly.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The manuscript proposes a cross-subject fMRI-to-image decoding framework that combines a Bidirectional Autoencoder Intertwining (BAI) module with a Subject Bias Modulation Module (SBMM) to learn subject-invariant representations, and then uses a Semantic Refinement Module (SRM) and a Visual Coherence Module (VCM) to make a frozen Stable Diffusion/ControlNet decoder tolerant to imperfect predicted semantic, edge, and color representations. The authors evaluate on the Natural Scenes Dataset (NSD), compare with prior cross-subject and subject-specific methods, and report ablations of the unidirectional mapping, the removal of SBMM, and the removal of SRM/VCM. They also present adaptation experiments with 500 and 1,500 samples for a new subject, together with appendix results on NOD, GOD, and BOLD5000.

Significance. The proposed system is well motivated: it explicitly attacks two error sources, namely representation prediction and downstream diffusion conditioning, and the ablation design in Table 3 supports the contribution of each main component. The appendix contains useful analyses, including t-SNE visualizations, VCM weight visualizations, voxel-level gradient analysis, and pseudo-code for the BAI architecture. If the empirical claims hold, the work is a meaningful step for practical cross-subject decoding because adaptation to a new subject requires only retraining the SBMM. However, the central quantitative claims rest on single averages without error bars or significance tests, the pseudo-imprecise training distribution is not validated against the true inference-time error distribution, and one training equation is internally inconsistent. These issues must be resolved before the state-of-the-art claim is credible.

major comments (4)
  1. [Section 4.2, Eq. (4) vs. Eq. (7)] The pseudo-imprecise representations used to train SRM and VCM are produced by passing SD-generated images' (S,E,C) through BAI's reconstruction pipeline, namely the representation autoencoder of Eq. (4). At inference, however, SRM and VCM receive the outputs of the full fMRI-to-representation translation pipeline of Eq. (7), which includes the fMRI encoder EV, the MLP MLPV⇒R, and the representation decoder DR. The corruption statistics of these two paths are not shown to match in magnitude or structure, so the modules may be calibrated on autoencoder noise that does not correspond to real prediction errors. The authors should provide a validation experiment, for example by training or evaluating SRM/VCM on the actual Eq. (7) predictions for held-out subjects and comparing refinement gains, or by directly comparing the error distributions of the two paths.
  2. [Section 3.2, Eq. (15)] In Eq. (15), the VCM loss is written as ||ε - εθ(z_t, t, SRM(S̃), C̃, Ẽ)||², with no dependence on the learned weights αe and αc from Eqs. (13)-(14). As printed, backpropagating this loss provides no gradient to the VCM, so the VCM cannot be trained by the stated objective. The authors should either correct the loss to include the modulated control features, such as αe⊙Ê and αc⊙Ĉ, or describe the actual training procedure used for VCM.
  3. [Section 3.1, Eq. (10)] Equation (10), the total BAI training loss, contains an unreadable corruption: after LRec_V the expression is filled with scrambled bracket and symbol tokens, and the same corruption repeats for the translation and cycle-consistency terms. Because this equation defines the central training objective, the corruption prevents verification of how the reconstruction, translation, and cycle losses are weighted and combined. The equation must be restored to a readable form.
  4. [Section 4.3, Table 1] The abstract and Section 4.3 claim that the method outperforms state-of-the-art approaches on SSIM, but Table 1 reports SSIM=0.356 for Ours, which is equal to BrainDiffuser and lower than NeuroPictor (0.375). Moreover, all comparisons are reported as single averages over four subjects with no standard deviations, repeated-seed variance, or significance tests, while several margins are small or tied (for example, EffNet-B is 0.639 for both Ours and NeuroPictor). The state-of-the-art claim should be qualified with per-subject breakdowns or statistical evaluation, and the SSIM statement should be corrected.
minor comments (5)
  1. [Section 4.4, Table 2] Table 2 lists 'MindBridge 500' and 'Ours 500' twice each and similarly for 1,500 samples, but the 'Adaptation?' column does not clearly distinguish the from-scratch condition from the adapted condition in the rendered table; please add explicit labels such as 'from scratch' and 'adapted' for each row.
  2. [Section 4.5] In the description of the UM variant, the text says 'we remove the decoder of fMRI Ev, the encoder of representations Dv'; these should be the fMRI decoder DV and the representation encoder ER.
  3. [Introduction] The phrase 'both sources of inaccurate must be addressed' should be reworded to 'both sources of inaccuracy must be addressed' or similar.
  4. [Section 3.2, Eq. (12)] Equation (12) writes the SRM input as S̃, although at test time SRM receives the predicted representation Ŝ produced by Eq. (7); unifying this notation would prevent confusion with the autoencoder reconstruction S̃.
  5. [References] Reference [4] has a garbled author list and should be corrected.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the NSD results are held-out evaluations, and each module is trained against external targets rather than the final metric.

full rationale

The derivation chain is self-contained. BAI+SBMM predicts representations via Eq. 7, SRM is trained to map perturbed semantic embeddings to ground-truth CLIP embeddings (Eq. 12), and VCM is trained with a denoising loss over conditions (Eq. 15); none of these objectives is the NSD test metric, and the final reconstructions are scored on the 982 held-out images common to the four subjects (Section 4.3). The only self-referential element is Section 4.2, where SRM/VCM are trained on 'pseudo-imprecise' pairs made by reconstructing SD-generated images through BAI's reconstruction pipelines (Eqs. 1-6) rather than through the fMRI translation pipeline (Eq. 7). This is a genuine distribution-matching risk for the claimed gains, but it is not a circular reduction: the training targets are ground-truth S/E/C and diffusion noise, not the paper's own outputs, and the evaluation set is external. The appendix limitation (SBMM must be retrained per subject) and the inherited SD failure modes are acknowledged by the authors. No load-bearing argument depends on a self-citation; the cited prior works are baselines and external modules. Eq. 15 is terse but can be read as the denoising loss on the VCM-modified UNet, so it does not by itself make the VCM claim unfalsifiable. Overall, no step of the derivation assumes the conclusion.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central claim rests on standard supervised learning assumptions plus one ad hoc training-distribution assumption (synthetic pseudo-imprecise data). No new physical or architectural entities are introduced beyond learned modules.

free parameters (2)
  • loss weights lambda_1, lambda_2, lambda_3 = 1.0, 1.0, 0.5
    Set empirically in Eq. 10; the paper gives no sensitivity analysis, and the cyclic losses depend on them.
  • synthetic pseudo-imprecise dataset size = 10,000 images
    Hand-chosen number of SD-generated images used to train SRM and VCM; no ablation on this size is reported.
assumptions (3)
  • domain assumption Affine modulation of per-subject mean and variance in a shared latent space is sufficient to remove subject-specific bias.
    SBMM Eq. 2 encodes the entire cross-subject alignment strategy; the paper does not validate this against alternative alignment methods.
  • domain assumption Semantic embedding, edge map, and color palette are sufficient and necessary representations for faithful reconstruction.
    Inherited from DREAM; the quality of decoding depends on these targets being the right targets.
  • ad hoc to paper Pseudo-imprecise representations reconstructed from SD-generated images approximate the prediction-error distribution of BAI on real fMRI.
    Section 4.2 uses 10,000 SD images to create training pairs for SRM and VCM; there is no evidence this matches real fMRI-driven errors.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Cross-Subject Mind Decoding from Inaccurate Representations." pith.science (2026). https://pith.science/paper/CZLNBGP2

@misc{pith2026250719071,
  author       = {Pith},
  title        = {Pith review of: Cross-Subject Mind Decoding from Inaccurate Representations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CZLNBGP2}},
  note         = {Machine review of arXiv:2507.19071}
}
read the original abstract

Decoding stimulus images from fMRI signals has advanced with pre-trained generative models. However, existing methods struggle with cross-subject mappings due to cognitive variability and subject-specific differences. This challenge arises from sequential errors, where unidirectional mappings generate partially inaccurate representations that, when fed into diffusion models, accumulate errors and degrade reconstruction fidelity. To address this, we propose the Bidirectional Autoencoder Intertwining framework for accurate decoded representation prediction. Our approach unifies multiple subjects through a Subject Bias Modulation Module while leveraging bidirectional mapping to better capture data distributions for precise representation prediction. To further enhance fidelity when decoding representations into stimulus images, we introduce a Semantic Refinement Module to improve semantic representations and a Visual Coherence Module to mitigate the effects of inaccurate visual representations. Integrated with ControlNet and Stable Diffusion, our method outperforms state-of-the-art approaches on benchmark datasets in both qualitative and quantitative evaluations. Moreover, our framework exhibits strong adaptability to new subjects with minimal training samples.

Figures

Figures reproduced from arXiv: 2507.19071 by the authors.

Figure 1
Figure 1. We propose a framework for faithful cross-subject mind decoding. Unlike prior approaches that struggle with cross-subject [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of our framework. It consists of two autoencoders: one for fMRI voxels [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Structure of SBMM. To obtain subject-invariant features, we reduce subjec￾tive bias in V_{x} by introduc￾ing a Subject Bias Modulation Module (SBMM) within the fMRI encoder for each sub￾ject. As shown in [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (10 more)
Figure 5
Figure 5. Figure 5: Detailed structure of our SRM and VCM. while this module use the similarly losses in training of BAI, this module tolerates imprecise semantic representations, thereby improving the fidelity during decoding stage. We introduce VCM that harmonizes the imprecise visual r…
Figure 6
Figure 6. Figure 6: Qualitative comparison with competitors on mind decoding. Our reconstructed images are consistency with the stimulus images on [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: t-SNE visualizations of representations learned by subject [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 9
Figure 9. Figure 9: Voxel-level gradient analysis of visual features across [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]
Figure 10
Figure 10. Figure 10: More qualitative comparison with competitors on mind decoding. [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]
Figure 11
Figure 11. Figure 11: Qualitative comparison with various variants. [PITH_FULL_IMAGE:figures/full_fig_p015_11.png]
Figure 12
Figure 12. Figure 12: Visualization of VCM’s output weights αe and αc, they control the fusion weights to relax the influence of predicted edge and color conditions to output. Sketch +#500 Sketch +#1,500 Adaptation +#500 Adaptation +#1,500 Stimulus [PITH_FULL_IMAGE:figures/full_fig_p016_12.png]
Figure 13
Figure 13. Figure 13: Qualitative comparison under different data limitation scenarios. [PITH_FULL_IMAGE:figures/full_fig_p016_13.png]
Figure 14
Figure 14. Figure 14: Our framework also supports the synthesis of fMRI for specific subject based on an unseen image, and the synthesized fMRI [PITH_FULL_IMAGE:figures/full_fig_p017_14.png]
Figure 15
Figure 15. Figure 15: Comparison on Cross-subject Mind decoding. [PITH_FULL_IMAGE:figures/full_fig_p018_15.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

64 extracted references · 57 canonical work pages

  1. [1]

    A massive 7t fmri dataset to bridge cognitive neuroscience and artificial intelligence

    Emily J Allen, Ghislain St-Yves, Yihan Wu, Jesse L Breedlove, Jacob S Prince, Logan T Dowdle, Matthias Nau, Brad Caron, Franco Pestilli, Ian Charest, et al. A massive 7t fmri dataset to bridge cognitive neuroscience and artificial intelligence. Nature neuroscience, 25(1):116–126, 2022. 2, 5

    work page 2022

  2. [2]

    Loosec- ontrol: Lifting controlnet for generalized depth conditioning

    Shariq Farooq Bhat, Niloy Mitra, and Peter Wonka. Loosec- ontrol: Lifting controlnet for generalized depth conditioning. In SIGGRAPH, pages 1–11, 2024. 3

    work page 2024

  3. [3]

    Large scale gan training for high fidelity natural image synthesis

    Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale gan training for high fidelity natural image synthesis. In ICLR, 2019. 1, 2

    work page 2019

  4. [4]

    Brain netflix: Scaling data to reconstruct videos from brain signals

    Bowen Pan Alex Andonian Emilie Josephs Alex Lascelles Camilo Fosco, Benjamin Lahner and Aude Oliva. Brain netflix: Scaling data to reconstruct videos from brain signals. In ECCV, 2024. 2

    work page 2024

  5. [5]

    Unsupervised learning of visual features by contrasting cluster assignments.NeurIPS, 33:9912–9924, 2020

    Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsupervised learning of visual features by contrasting cluster assignments.NeurIPS, 33:9912–9924, 2020. 12

    work page 2020

  6. [6]

    Bold5000, a public fmri dataset while viewing 5000 visual images

    Nadine Chang, John A Pyles, Austin Marcus, Abhinav Gupta, Michael J Tarr, and Elissa M Aminoff. Bold5000, a public fmri dataset while viewing 5000 visual images. Scientific data, 6(1):49, 2019. 13

    work page 2019

  7. [7]

    Seeing beyond the brain: Conditional diffu- sion model with sparse masked modeling for vision decoding

    Zijiao Chen, Jiaxin Qing, Tiange Xiang, Wan Lin Yue, and Juan Helen Zhou. Seeing beyond the brain: Conditional diffu- sion model with sparse masked modeling for vision decoding. In CVPR, pages 22710–22720, 2023. 1, 2, 6, 13

    work page 2023

  8. [8]

    brain reading

    David D Cox and Robert L Savoy. Functional magnetic resonance imaging (fmri)“brain reading”: detecting and clas- sifying distributed patterns of fmri activity in human visual cortex. Neuroimage, 19(2):261–270, 2003. 2

    work page 2003

Show all 64 references
  1. [9]

    fmri brain decoding and its applications in brain–computer interface: A survey

    Bing Du, Xiaomu Cheng, Yiping Duan, and Huansheng Ning. fmri brain decoding and its applications in brain–computer interface: A survey. Brain Sciences, 12(2):228, 2022. 1

    2022

  2. [10]

    A large-scale fmri dataset for the visual processing of naturalistic scenes

    Zhengxin Gong, Ming Zhou, Yuxuan Dai, Yushan Wen, Youyi Liu, and Zonglei Zhen. A large-scale fmri dataset for the visual processing of naturalistic scenes. Scientific Data, 10 (1):559, 2023. 13

    2023

  3. [11]

    Decoding natural image stimuli from fmri data with a surface- based convolutional network

    Zijin Gu, Keith Jamison, Amy Kuceyeski, and Mert Sabuncu. Decoding natural image stimuli from fmri data with a surface- based convolutional network. In PMLR, 2024. 5, 6

    2024

  4. [12]

    Generic decod- ing of seen and imagined objects using hierarchical visual features

    Tomoyasu Horikawa and Yukiyasu Kamitani. Generic decod- ing of seen and imagined objects using hierarchical visual features. Nature communications, 8(1):15037, 2017. 1, 2, 13

    2017

  5. [13]

    Neuropictor: Refining fmri-to-image reconstruction via multi-individual pretraining and multi-level modulation

    Jingyang Huo, Yikai Wang, Yun Wang, Xuelin Qian, Chong Li, Yanwei Fu, and Jianfeng Feng. Neuropictor: Refining fmri-to-image reconstruction via multi-individual pretraining and multi-level modulation. In ECCV, pages 56–73, 2024. 6, 7, 14

    2024

  6. [14]

    Selectivity for food in human ventral visual cortex

    Nidhi Jain, Aria Wang, Margaret M Henderson, Ruogu Lin, Jacob S Prince, Michael J Tarr, and Leila Wehbe. Selectivity for food in human ventral visual cortex. Communications Biology, 6(1):175, 2023. 1

    2023

  7. [15]

    Decoding the visual and subjective contents of the human brain

    Yukiyasu Kamitani and Frank Tong. Decoding the visual and subjective contents of the human brain. Nature neuroscience, 8(5):679–685, 2005. 2

    2005

  8. [16]

    The fusiform face area: A module in human extrastriate cortex specialized for face perception

    Nancy Kanwisher, Josh McDermott, and Marvin M Chun. The fusiform face area: A module in human extrastriate cortex specialized for face perception. Journal of Neuroscience, 17 (11):4302–4311, 2002. 1

    2002

  9. [17]

    Analyzing and improving the image quality of stylegan

    Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and improving the image quality of stylegan. In CVPR, pages 8110–8119,

  10. [18]

    Im- agenet classification with deep convolutional neural networks

    Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Im- agenet classification with deep convolutional neural networks. NeurIPS, 25, 2012. 12

    2012

  11. [19]

    Modeling short visual events through the bold moments video fmri dataset and metadata

    Benjamin Lahner, Kshitij Dwivedi, Polina Iamshchinina, Monika Graumann, Alex Lascelles, Gemma Roig, Alessan- dro Thomas Gifford, Bowen Pan, SouYoung Jin, N Apurva Ratan Murty, Oliva Aude Kay Kendrick, and Cichy Radoslaw. Modeling short visual events through the bold moments vi...

    2024

  12. [20]

    Blip- 2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip- 2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In ICML, pages 19730–19742, 2023. 12

    2023

  13. [21]

    Decoupled weight decay regularization

    Loshchilov LIlya and Hutter Frank. Decoupled weight decay regularization. In ICLR, 2017. 6

    2017

  14. [22]

    Mind reader: Reconstructing complex images from brain activities

    Sikun Lin, Thomas Sprague, and Ambuj K Singh. Mind reader: Reconstructing complex images from brain activities. NeurIPS, 35:29624–29636, 2022. 2, 6

    2022

  15. [23]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In ECCV, pages 740–755, 2014. 5

    2014

  16. [24]

    Drag your noise: Interactive point-based editing via diffusion semantic propagation

    Haofeng Liu, Chenshu Xu, Yifei Yang, Lihua Zeng, and Shengfeng He. Drag your noise: Interactive point-based editing via diffusion semantic propagation. In CVPR, pages 6743–6752, 2024. 3

    2024

  17. [25]

    Draw like an artist: Complex scene gen- eration with diffusion model via composition, painting, and retouching

    Minghao Liu, Le Zhang, Yingjie Tian, Xiaochao Qu, Luoqi Liu, and Ting Liu. Draw like an artist: Complex scene gen- eration with diffusion model via composition, painting, and retouching. arXiv preprint arXiv:2408.13858, 2024. 12

    2024 arXiv

  18. [26]

    Smartcontrol: Enhancing controlnet for handling rough visual conditions

    Xiaoyu Liu, Yuxiang Wei, Ming Liu, Xianhui Lin, Peiran Ren, Xuansong Xie, and Wangmeng Zuo. Smartcontrol: Enhancing controlnet for handling rough visual conditions. In ECCV, 2024. 2, 3, 5

    2024

  19. [27]

    Minddiffuser: Controlled image reconstruc- tion from human brain activity with semantic and structural diffusion

    Yizhuo Lu, Changde Du, Qiongyi Zhou, Dianpeng Wang, and Huiguang He. Minddiffuser: Controlled image reconstruc- tion from human brain activity with semantic and structural diffusion. In ACM Multimedia, pages 5899–5908, 2023. 1, 2

    2023

  20. [28]

    Controlnet++: Improv- ing conditional controls with efficient consistency feedback

    Huafeng Kuang Jie Wu Zhaoning Wang Xuefeng Xiao Chen Chen Ming Li, Taojiannan Yang. Controlnet++: Improv- ing conditional controls with efficient consistency feedback. In ECCV, 2024. 3

    2024

  21. [29]

    T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models

    Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, and Ying Shan. T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. In AAAI, pages 4296–4304, 2024. 2, 3, 5, 6

    2024

  22. [30]

    Recon- structing natural scenes from fmri patterns using bigbigan

    Milad Mozafari, Leila Reddy, and Rufin VanRullen. Recon- structing natural scenes from fmri patterns using bigbigan. In IJCNN, pages 1–8, 2020. 2

    2020

  23. [31]

    Natural scene recon- struction from fmri signals using generative latent diffusion

    Furkan Ozcelik and Rufin VanRullen. Natural scene recon- struction from fmri signals using generative latent diffusion. Scientific Reports, 13(1):15666, 2023. 1, 3, 6, 7, 14

    2023

  24. [32]

    Reconstruction of perceived images from fmri patterns and semantic brain exploration using instance-conditioned gans

    Furkan Ozcelik, Bhavin Choksi, Milad Mozafari, Leila Reddy, and Rufin VanRullen. Reconstruction of perceived images from fmri patterns and semantic brain exploration using instance-conditioned gans. In IJCNN, pages 1–8, 2022. 2, 13

    2022

  25. [33]

    Unicontrol: A unified diffu- sion model for controllable visual generation in the wild

    Can Qin, Shu Zhang, Ning Yu, Yihao Feng, Xinyi Yang, Yingbo Zhou, Huan Wang, Juan Carlos Niebles, Caiming Xiong, Silvio Savarese, et al. Unicontrol: A unified diffu- sion model for controllable visual generation in the wild. In NeurIPS, 2023. 3

    2023

  26. [34]

    Psychometry: An omnifit model for image reconstruc- tion from human brain activity

    Ruijie Quan, Wenguan Wang, Zhibo Tian, Fan Ma, and Yi Yang. Psychometry: An omnifit model for image reconstruc- tion from human brain activity. In CVPR, pages 233–243,

  27. [35]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In ICLR, pages 8748–8763, 2021. 2, 5, 12

    2021

  28. [36]

    Hierarchical text-conditional image genera- tion with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image genera- tion with clip latents. arXiv preprint arXiv:2204.06125, 1(2): 3, 2022. 2

    2022 arXiv

  29. [37]

    Ul- trapixel: Advancing ultra high-resolution image synthesis to new peaks

    Jingjing Ren, Wenbo Li, Haoyu Chen, Renjing Pei, Bin Shao, Yong Guo, Long Peng, Fenglong Song, and Lei Zhu. Ul- trapixel: Advancing ultra high-resolution image synthesis to new peaks. In NeurIPS, pages 111131–111171, 2024

    2024

  30. [38]

    Turbo2k: Towards ultra-efficient and high-quality 2k video synthesis

    Jingjing Ren, Wenbo Li, Zhongdao Wang, Haoze Sun, Bangzhen Liu, Haoyu Chen, Jiaqi Xu, Aoxue Li, Shifeng Zhang, Bin Shao, et al. Turbo2k: Towards ultra-efficient and high-quality 2k video synthesis. arXiv preprint arXiv:2504.14470, 2025

    2025 arXiv

  31. [39]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. InCVPR, pages 10684– 10695, 2022. 1, 2

    2022

  32. [40]

    Photorealistic text-to-image diffusion models with deep lan- guage understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep lan- guage understanding. In NeurIPS, pages 36479–3649...

    2022

  33. [41]

    Recon- structing the mind’s eye: fmri-to-image with contrastive learn- ing and diffusion priors

    Paul Scotti, Atmadeep Banerjee, Jimmie Goode, Stepan Sha- balin, Alex Nguyen, Aidan Dempster, Nathalie Verlinde, Elad Yundler, David Weisberg, Kenneth Norman, et al. Recon- structing the mind’s eye: fmri-to-image with contrastive learn- ing and diffusion priors. In NeurIPS, 20...

    2023

  34. [42]

    Mindeye2: Shared-subject models enable fmri-to-image with 1 hour of data

    Paul S Scotti, Mihir Tripathy, Cesar Kadir Torrico Villanueva, Reese Kneeland, Tong Chen, Ashutosh Narang, Charan San- thirasegaran, Jonathan Xu, Thomas Naselaris, Kenneth A Norman, et al. Mindeye2: Shared-subject models enable fmri-to-image with 1 hour of data. In ICML, 2024....

    2024

  35. [43]

    Functional neuroanatomy of face and object processing: a positron emission tomography study

    Justine Sergent, Shinsuke Ohta, and Brennan Macdonald. Functional neuroanatomy of face and object processing: a positron emission tomography study. Brain, 115(1):15–36,

  36. [44]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In ICLR, 2021. 5

    2021

  37. [45]

    Lightweight pixel dif- ference networks for efficient visual representation learning

    Zhuo Su, Jiehua Zhang, Longguang Wang, Hua Zhang, Zhen Liu, Matti Pietik ¨ainen, and Li Liu. Lightweight pixel dif- ference networks for efficient visual representation learning. IEEE TPAMI, 45(12):14956–14974, 2023. 5

    2023

  38. [46]

    Rethinking the inception ar- chitecture for computer vision

    Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception ar- chitecture for computer vision. In CVPR, pages 2818–2826,

  39. [47]

    High-resolution image reconstruction with latent diffusion models from human brain activity

    Yu Takagi and Shinji Nishimoto. High-resolution image reconstruction with latent diffusion models from human brain activity. In CVPR, pages 14453–14463, 2023. 1, 2, 5, 6, 7, 14

    2023

  40. [48]

    Efficientnet: Rethinking model scaling for convolutional neural networks

    Mingxing Tan. Efficientnet: Rethinking model scaling for convolutional neural networks. In ICML, pages 6105–6114,

  41. [49]

    Inverse retinotopy: inferring the visual content of images from brain activation patterns

    Bertrand Thirion, Edouard Duchesnay, Edward Hubbard, Jes- sica Dubois, Jean-Baptiste Poline, Denis Lebihan, and Stanis- las Dehaene. Inverse retinotopy: inferring the visual content of images from brain activation patterns. Neuroimage, 33(4): 1104–1116, 2006. 2

    2006

  42. [50]

    Mindbridge: A cross-subject brain decoding frame- work

    Shizun Wang, Songhua Liu, Zhenxiong Tan, and Xinchao Wang. Mindbridge: A cross-subject brain decoding frame- work. In CVPR, pages 11333–11342, 2024. 1, 2, 3, 4, 5, 6, 7, 12, 14

    2024

  43. [51]

    Image quality assessment: from error visibility to structural similarity

    Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE TIP, 13(4):600–612, 2004. 12

    2004

  44. [52]

    Dream: Visual decoding from reversing human visual system

    Weihao Xia, Raoul de Charette, Cengiz Oztireli, and Jing- Hao Xue. Dream: Visual decoding from reversing human visual system. In WACV, pages 8226–8235, 2024. 2, 3, 5, 6

    2024

  45. [53]

    Umbrae: Unified multimodal brain decoding

    Weihao Xia, Raoul de Charette, Cengiz Oztireli, and Jing- Hao Xue. Umbrae: Unified multimodal brain decoding. In ECCV, pages 242–259, 2024. 2, 6, 12

    2024

  46. [54]

    Dreamanime: Learning style-identity textual disentanglement for anime and beyond

    Chenshu Xu, Yangyang Xu, Huaidong Zhang, Xuemiao Xu, and Shengfeng He. Dreamanime: Learning style-identity textual disentanglement for anime and beyond. 2024. 3

    2024

  47. [55]

    Versatile diffusion: Text, images and variations all in one diffusion model

    Xingqian Xu, Zhangyang Wang, Gong Zhang, Kai Wang, and Humphrey Shi. Versatile diffusion: Text, images and variations all in one diffusion model. In ICCV, pages 7754– 7765, 2023. 1, 2, 6

    2023

  48. [56]

    Law-diffusion: Complex scene generation by diffusion with layouts

    Binbin Yang, Yi Luo, Ziliang Chen, Guangrun Wang, Xiao- dan Liang, and Liang Lin. Law-diffusion: Complex scene generation by diffusion with layouts. In ICCV, pages 22669– 22679, 2023. 12

    2023

  49. [57]

    Beyond textual con- straints: Learning novel diffusion conditions with fewer ex- amples

    Yuyang Yu, Bangzhen Liu, Chenxi Zheng, Xuemiao Xu, Huaidong Zhang, and Shengfeng He. Beyond textual con- straints: Learning novel diffusion conditions with fewer ex- amples. In CVPR, pages 7109–7118, 2024. 3

    2024

  50. [58]

    Controllable mind visual diffusion model

    Bohan Zeng, Shanglin Li, Xuhui Liu, Sicheng Gao, Xiaolong Jiang, Xu Tang, Yao Hu, Jianzhuang Liu, and Baochang Zhang. Controllable mind visual diffusion model. In AAAI, pages 6935–6943, 2024. 13

    2024

  51. [59]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In ICCV, pages 3836–3847, 2023. 2, 3, 5

    2023

  52. [60]

    Uni- controlnet: All-in-one control to text-to-image diffusion mod- els

    Shihao Zhao, Dongdong Chen, Yen-Chun Chen, Jianmin Bao, Shaozhe Hao, Lu Yuan, and Kwan-Yee K Wong. Uni- controlnet: All-in-one control to text-to-image diffusion mod- els. In NeurIPS, 2023. 3

    2023

  53. [61]

    Towards language-free training for text-to-image gener- ation

    Yufan Zhou, Ruiyi Zhang, Changyou Chen, Chunyuan Li, Chris Tensmeyer, Tong Yu, Jiuxiang Gu, Jinhui Xu, and Tong Sun. Towards language-free training for text-to-image gener- ation. In CVPR, pages 17907–17917, 2022. 1, 2

    2022

  54. [62]

    Stable score distillation

    Haiming Zhu, Yangyang Xu, Chenshu Xu, Tingrui Shen, Wenxi Liu, Yong Du, Jun Yu, and Shengfeng He. Stable score distillation. In ICCV, 2025. 3

    2025

  55. [63]

    Unpaired image-to-image translation using cycle-consistent adversarial networks

    Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. In ICCV, pages 2223–2232, 2017. 3 4 2 0 2 4 6 8 2 0 2 4 6 8 Subj1 Subj2 Subj5 Subj7 0.4 0.3 0.2 0.1 0.0 0.1 0.2 0.3 0.4 0.3 0.2 0.1 0....

    2017

  56. [64]

    Broccoli

    Appendix Analysis of Shared Latent Space Our framework learns a shared latent space that aligns fMRI and visual features across subjects, enabling general- ization by capturing subject invariant patterns. We present the t-SNE visualization of subject-specific and cross-subject...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.