Pith. sign in

REVIEW 5 major objections 6 minor 1 cited by

Making Your Dreams A Reality: Decoding the Dreams into a Coherent Video Story from fMRI Signals

T0 review · 5 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims dreams can be reconstructed as viewable video frames directly from sleep fMRI, without dream-report labels.

desk verdict A genuine first attempt at decoding dream imagery into video narratives, but the central claim rests on an unvalidated zero-shot transfer and a subject-confounded evaluation; worth a serious referee, not acceptance as-is. read the letter →

arxiv 2501.09350 v1 pith:REE7XPNZ submitted 2025-01-16 cs.CV

Yanwei Fu , Jianxiong Gao , Baofeng Yang , Jianfeng Feng This is my paper · ORCID
classification cs.CV
keywords fMRIdecodingdreamvisualimageryzero-shottransferdiffusionmodelslargelanguagesleepvideonarrativegeneration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the visual content of a dream can be recovered from sleep fMRI signals as viewable images, without relying on category-level dream labels. It builds on the premise that dreaming and waking perception share brain activity patterns in visual cortex, so a reconstruction model trained on awake visual-stimulus data can be transferred directly to sleep data. The authors propose a three-stage pipeline: reconstruct real visual stimuli from a large awake fMRI dataset, decode each sleep fMRI time point into a dream snapshot with the same model, then use language models to stitch the snapshots into a coherent narrated video. If the central claim holds, dreams become a new input modality for multimedia generation and a scientific window into subjective visual experience.

What carries the argument

The load-bearing machinery is the cross-state transfer of a multi-subject fMRI-to-image reconstruction model: a transformer encoder $E$ maps fMRI surface images to a unified brain representation, and a diffusion-based generative network $G$, composed of a high-level guiding network and a low-level manipulation network, turns that representation into an image. Trained on thousands of awake visual-stimulus fMRI-image pairs, the same $G(E(\cdot))$ is applied unchanged to averaged sleep fMRI windows in Eq. (3). The shared-activity-pattern assumption is what licenses this zero-shot transfer, and the language-model integration stage then converts the discrete decoded frames into a story.

What would settle it

Decode sleep fMRI after shuffling the temporal order of the volumes within each dream segment, or decode fMRI recorded during quiet wakefulness with the same model: if the resulting images stay equally aligned with the dream descriptions, the pipeline is responding to generic model priors rather than dream-specific neural signal.

Watch

Extended reading notes

Core claim

The paper's central claim is that Eq. (3), $I_i^{\text{dream}} = G(E(F_i))$, yields a high-fidelity reconstruction of what a person was visually dreaming at time $i$: apply the pre-trained reconstruction function $G$ to the sleep fMRI volume $F_i$ encoded by $E$, with no training on dream data. The authors argue that because the same visual cortex processes both real visual stimuli and dream imagery, the shared brain activity patterns allow the awake-trained model to generalize zero-shot to sleep-state fMRI. This would surpass prior dream decoding, which only classified dream content into semantic categories such as objects, by producing direct visualizations of the dream experience. They further claim that assembling these decoded snapshots into a narrative video through large language models captures the continuous story-like nature of dreams rather than isolated frames.

Load-bearing premise

The awake-trained model is assumed to work on sleep fMRI of different people recorded in a different scanner and brain state, without any adaptation.

Editorial extensions

If this is right

  • Dream content can be visualized directly from sleep fMRI, bypassing the need for dream reports as training labels.
  • The same awake-trained reconstruction model can be carried across individuals and brain states, so dream decoding does not require per-person retraining on dream data.
  • Decoded dream frames, once captioned and narrated by language models, can be assembled into a video that reflects the reported dream sequence.
  • The reported quantitative results indicate that decoded frames from positive dream segments match the corresponding dream description better than frames from other participants, with significant differences for two of the three tested dream labels.
  • Dream experiences are treated as continuous narratives rather than isolated snapshots, aligning the decoding target with how dreams are actually recalled.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Until the zero-shot cross-scanner transfer is validated, a cautious reader should treat the decoded images as a blend of learned visual prior and sleep signal, because a strong generative prior can produce plausible scenes even from weak or noisy input.
  • A direct control experiment that decodes sleep fMRI with the temporal order of volumes scrambled, or decodes quiet-wakefulness fMRI with the same model, would separate dream-specific neural signal from generic prior-driven generation.
  • The narrative integration step may inflate apparent success, since a fluent language model can weave unrelated captions into a coherent story; story coherence should be assessed separately from frame-level decoding accuracy.
  • Because the quantitative evaluation uses text-image similarity against the participant's own dream caption, a forced-choice test among several candidate dream scenes would be a stronger check of whether the decoded frames truly carry the reported dream content.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes a three-stage pipeline for turning sleep fMRI into a coherent dream video: (1) train a NeuroPictor-style fMRI-to-image reconstruction model on the 7T Natural Scenes Dataset, (2) apply this model zero-shot to 3T sleep fMRI windows via Eq. (3), I_dream_i = G(E(F_i)), to decode dream snapshots, and (3) use LENS image captioning and ChatGPT to assemble the snapshots into a narrative video. The authors collected 3.3 hours of sleep fMRI from three participants, with five clearly recalled dream reports, and analyze three of those segments. Quantitative evaluation in Section 6.5 compares CLIP similarity of decoded images to dream labels between positive and negative samples, followed by Mann-Whitney U tests.

Significance. If the main claim were established, this would be a noteworthy first step toward high-fidelity visual decoding of dream content and a novel multimedia application. The paper builds on the public NSD benchmark and an existing reconstruction model (NeuroPictor), and the authors report a new sleep-fMRI dataset, which are useful resources. However, the current evidence does not support the central claim: the evaluation is confounded by subject identity, the analysis is selective, no baselines are reported, and the narrative integration step is circular as evidence of decoding fidelity. The scientific claim of directly decoding dream images from fMRI therefore remains unsubstantiated.

major comments (5)
  1. [Section 6.5, Table 2] The positive and negative sample comparison is confounded by participant identity. For a given dream label, positive samples come from the participant who reported that dream while negative samples come exclusively from the other participants. For example, for 'cat', positives are Sub-3's sleep volumes and negatives are Sub-1's and Sub-2's volumes. Higher CLIP similarity of the positive set could therefore reflect subject-specific output biases of the zero-shot decoder (e.g., a tendency to generate certain image statistics for a given subject) rather than decoded dream content. The paper does not report the necessary within-subject control, even though it is available in the authors' own data: Sub-1 has two distinct dream reports and Sub-3 has two distinct dream reports. Without such a control, the reported p-values in Table 2 do not test whether dream content was decoded.
  2. [Section 6.1.2] Two of the five clearly recalled dream segments were excluded post hoc with the stated reason of 'unavoidable domain gaps and complex semantics.' This is a selection on the dependent variable and it removes exactly the data needed for the within-subject controls described above. The quantitative evaluation is therefore based on only three segments, and the exclusion criteria are not principled or pre-registered, which further weakens the support for the general claim that the method decodes dream imagery.
  3. [Section 6.3 and Section 6.5] No baseline or chance level is reported for the CLIP similarity evaluation. The paper only compares positive and negative samples to each other. To establish decoding, the authors would need to compare against baselines such as decoded images from temporally permuted fMRI volumes, images from a model with scrambled or shuffled fMRI inputs, or a chance-level similarity distribution. Additionally, the p-values in Table 2 are not corrected for multiple comparisons, and no sample sizes, effect sizes, or per-subject breakdowns are reported, so the statistical claims in Section 6.5 are incomplete.
  4. [Section 5.2, Eq. (3)] The central decoding pipeline assumes that the NSD-trained encoder E and generator G, trained on 7T awake visual stimulus data, transfer zero-shot to 3T sleep fMRI from unseen participants without any adaptation. The paper offers no validation of this cross-scanner, cross-state transfer. Section 6.1.2's admission that two of five segments failed due to domain gaps indicates that the transfer is not robust. At minimum, the authors would need a control experiment showing that the identical pipeline behaves appropriately on matched 3T awake data, or an explicit adaptation step. Without this, Eq. (3) rests on an untested assumption.
  5. [Section 5.3 and Section 6.4] The narrative integration stage is circular as evidence for decoding fidelity. LENS first generates captions from the decoded dream images, and ChatGPT then composes a coherent story from those captions. The vividness and coherence of the final narrative reflect the language model's generative capabilities, not the accuracy of the neural decoding. Consequently, the qualitative claims in Section 6.4 that the resulting videos 'align with participants' descriptions' cannot be taken as support for the central claim that dream images were directly decoded from fMRI.
minor comments (6)
  1. [Equation (1)] There is a typo in 'fMRI encider'; it should be 'fMRI encoder.'
  2. [Section 6.3] The set of augmented COCO class labels is not specified. The authors should report the exact label list, including how 'skis' was derived from the dream caption 'skiing with a snowboard' and how other non-COCO labels were formed.
  3. [Figure 8] The figure caption lacks axis labels, error bars, and a per-subject or per-segment breakdown, making it difficult to assess the magnitude and variability of the reported similarities.
  4. [Table 2] The table reports only p-values. It should also report the number of positive and negative samples, the mean and standard deviation of the similarity scores, and the effect size for each comparison.
  5. [Section 6.2] The statement that inference used a single GTX 3090Ti GPU is not an implementation detail relevant to the method; please provide instead the fMRI window size used for temporal averaging, the number of decoded frames per dream segment, and the exact inference settings.
  6. [Abstract and Section 4 footnote] The phrase 'brave new idea' is informal and repeated throughout; the footnote distinguishing the colloquial use of 'dreams' from neurology's precise sleep-stage definitions should be integrated into the main text for scientific clarity.

Circularity Check

2 steps flagged · score 3.0 of 10

Central fMRI-to-image decoding is not circular because it uses an external NSD-trained model, but the quantitative validation is confounded by participant identity by construction and the narrative-coherence evaluation is self-confirmatory.

  1. other [Section 6.5, Quantitative Results (Figure 8 and Table 2)]
    "Since the dreams of the three participants are distinct, for a given dream description, we can designate the corresponding participant’s sleep data as positive samples and the sleep data from the other participants as negative samples."

    By construction, the positive set is exactly one participant's sleep fMRI and the negative set is the other participants' sleep fMRI. The reported CLIP-similarity contrast therefore tests whether the zero-shot decoder produces participant-specific output distributions, not whether it recovers dream content. Any subject- or session-specific bias in G(E(F_i)) — which is expected under unvalidated 3T sleep transfer — would push positive above negative even if no dream content were decoded. The conclusion 'indicating a closer match between our decoded dream imagery and the actual dream descriptions' does not follow, because the dream label and participant identity are perfectly collinear in this split.

  2. self definitional [Section 5.3 (Dream Story Composition) and Section 6.4 (Qualitative Results)]
    "we utilize ChatGPT to generate a logically coherent narrative with a dream-like storytelling style ... our constructed dream narratives are vivid, coherent, seamlessly weaving together disparate dream scenes."

    The coherence of the dream narrative is an explicit instruction in the ChatGPT prompt ('transitions between different materials should be natural'), and the qualitative result then reports that the resulting narratives are coherent. This is a self-confirmatory observation: the story is a paraphrase of LENS captions of the decoded images, and its coherence is inherited from the LLM's instruction-following ability, not from any demonstrated property of the fMRI decoding. This is peripheral to the central decoding claim, but it is presented as evidence of successful 'dream narrative integration.'

full rationale

The central derivation chain is not circular: Eq. (3) applies the NSD-trained encoder E and generator G zero-shot to sleep fMRI, and the dream data are never used to fit the reconstruction model. The use of NeuroPictor (same research group) as the backbone is a normal modular dependency, since that model was trained on the external Natural Scenes Dataset; similarly, the Intelligent Director framework is reused only for story assembly. The load-bearing validity concern is the zero-shot transfer from 7T awake NSD data to 3T sleep fMRI from different participants, which is an untested assumption rather than a circularity. The quantitative evaluation, however, is structurally confounded: positive and negative samples are defined by participant identity, so the reported p-values and similarity differences can be driven by subject-specific output biases of the zero-shot model, making the test a between-subject comparison by construction. Additionally, Section 6.1.2 excludes two of the five dream instances because their performance is 'relatively lower due to unavoidable domain gaps,' so the reported results come from a selected subset. The narrative integration stage is self-confirmatory in that coherence is requested from ChatGPT and then reported as achieved, but this does not make the fMRI-to-image decoding itself circular. Overall, the central generation is independent of the dream labels, so the score is modest; the confounded evaluation and peripheral self-confirmation keep it from being a clean zero.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claims rest on several unvalidated assumptions about cross-domain, cross-scanner transfer of a model trained on awake visual perception, plus a post-hoc selection of evaluation samples. No new physical or conceptual entities are introduced.

free parameters (4)
  • Sleep fMRI window size = not stated (assumed 3-4 seconds)
    Section 5.2 says sleep fMRI volumes are averaged with a window size to align with the 3-4 second stimulus presentation in NSD, but no explicit value is given. This choice affects which fMRI volumes map to which decoded images.
  • Analyzed dream segments = 3 of 5 clearly recalled dreams
    Section 6.1.2 states two of five dreams were excluded due to 'domain gaps and complex semantics'. This post-hoc selection affects all quantitative results.
  • Evaluation class labels = 80 COCO classes plus labels from dream descriptions
    Section 6.3 augments the COCO class list with labels extracted from the ground truth dream descriptions, changing the evaluation metric and potentially inflating similarity for known dream labels.
  • Visual cortex ROI list = 27 named regions
    Section 4 lists a hand-picked set of early and higher visual cortex regions. This selection influences which fMRI features are fed to the reconstruction model.
assumptions (5)
  • domain assumption Shared brain activity patterns between real visual perception and dream imagery
    Section 5.1 cites Horikawa et al. [20] to justify zero-shot transfer of a model trained on real visual stimuli to dream fMRI. If this sharing is incomplete or scanner-dependent, the transfer fails.
  • ad hoc to paper Cross-scanner transferability (7T NSD to 3T sleep data)
    The NSD model was trained on 7T data; the dream data is 3T. The paper provides no evidence that the trained encoder generalizes across field strengths, participants, and sleep state.
  • domain assumption Temporal alignment of window-averaged sleep fMRI to dream snapshots
    Section 5.2 assumes that averaging sleep fMRI over a window yields a representation comparable to one stimulus presentation in NSD, and that each window corresponds to a distinct dream visual at that moment.
  • domain assumption Dream reports are accurate ground truth
    Section 6.1.2 relies on participants' descriptions of their dreams as ground truth for evaluation, a standard but imperfect assumption for subjective dream content.
  • domain assumption CLIP similarity is a valid proxy for semantic correspondence
    Section 6.3 uses CLIP zero-shot similarity to compare decoded images with text labels. This assumes CLIP scores reflect true semantic alignment, which is not established for dream imagery.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Making Your Dreams A Reality: Decoding the Dreams into a Coherent Video Story from fMRI Signals." pith.science (2026). https://pith.science/paper/REE7XPNZ

@misc{pith2026250109350,
  author       = {Pith},
  title        = {Pith review of: Making Your Dreams A Reality: Decoding the Dreams into a Coherent Video Story from fMRI Signals},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/REE7XPNZ}},
  note         = {Machine review of arXiv:2501.09350}
}
read the original abstract

This paper studies the brave new idea for Multimedia community, and proposes a novel framework to convert dreams into coherent video narratives using fMRI data. Essentially, dreams have intrigued humanity for centuries, offering glimpses into our subconscious minds. Recent advancements in brain imaging, particularly functional magnetic resonance imaging (fMRI), have provided new ways to explore the neural basis of dreaming. By combining subjective dream experiences with objective neurophysiological data, we aim to understand the visual aspects of dreams and create complete video narratives. Our process involves three main steps: reconstructing visual perception, decoding dream imagery, and integrating dream stories. Using innovative techniques in fMRI analysis and language modeling, we seek to push the boundaries of dream research and gain deeper insights into visual experiences during sleep. This technical report introduces a novel approach to visually decoding dreams using fMRI signals and weaving dream visuals into narratives using language models. We gather a dataset of dreams along with descriptions to assess the effectiveness of our framework.

Figures

Figures reproduced from arXiv: 2501.09350 by the authors.

Figure 1
Figure 1. Our Dream decoding process blends fMRI decoding with tasks related to real and imagined visuals, seamlessly turning [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. We use a 3 Tesla (3T) MRI scanner for data collection. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. The fMRI dream decoding process is divided into three stages: i) Visual Stimulus Perception Reconstruction: Decoding [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Pipeline of Dream Narrative Integration. This process includes three steps: Single-Shot Dream Description, Dream [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Visualization of Dream Narrative Video "some cat". [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Visualization of Dream Narrative Video "skiing with a snowboard". [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Visualization of Dream Narrative Video "people running". [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Comparision of the average similarity between the [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Coherence-Oriented Dream Scene Visualisation

    cs.AI 2026-08 conditional novelty 6.0 of 10

    A dream-to-image pipeline with LLM decomposition, img2img chaining, and a CLIP feedback loop yields stylistically coherent panels while scoring lower on CLIP than a baseline, likely due to SDXL's 77-token prompt truncation.

Reference graph

Works this paper leans on

42 extracted references · 28 canonical work pages · cited by 1 Pith paper

  1. [1]

    Emily J Allen, Ghislain St-Yves, Yihan Wu, Jesse L Breedlove, Jacob S Prince, Logan T Dowdle, Matthias Nau, Brad Caron, Franco Pestilli, Ian Charest, et al

  2. [2]

    William Berrios, Gautam Mittal, Tristan Thrush, Douwe Kiela, and Amanpreet Singh. 2023. Towards Language Models That Can See: Computer Vision Through the LENS of Natural Language. arXiv:2306.16410 [cs.CL]

    arXiv 2023

  3. [3]

    Allen R Braun, TJ Balkin, NJ Wesenten, Richard Ellis Carson, M Varga, Pl Baldwin, S Selbie, Gregory Belenky, and Peter Herscovitch. 1997. Regional cerebral blood flow throughout the sleep-wake cycle. An H2 (15) O PET study. Brain: a journal of neurology 120, 7 (1997), 1173–1197

    work page 1997

  4. [4]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877–1901

    2020

  5. [5]

    Zijiao Chen, Jiaxin Qing, Tiange Xiang, Wan Lin Yue, and Juan Helen Zhou

    work page

  6. [6]

    brain reading

    David D Cox and Robert L Savoy. 2003. Functional magnetic resonance imaging (fMRI)“brain reading”: detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage 19, 2 (2003), 261–270

    work page 2003

  7. [7]

    Markiewicz, Shoshana L

    Oscar Esteban, Ross Blair, Christopher J. Markiewicz, Shoshana L. Berleant, Craig Moodie, Feilong Ma, Ayse Ilkay Isik, Asier Erramuzpe, Mathias Kent, James D. andGoncalves, Elizabeth DuPre, Kevin R. Sitek, Daniel E. P. Gomez, Daniel J. Lurie, Zhifang Ye, Russell A. Poldrack, and Krzysztof J. Gorgolewski. Making Your Dreams A Reality: Decoding the Dreams i...

    work page 2024

  8. [8]

    Oscar Esteban, Christopher Markiewicz, Ross W Blair, Craig Moodie, Ayse Ilkay Isik, Asier Erramuzpe Aliaga, James Kent, Mathias Goncalves, Elizabeth DuPre, Madeleine Snyder, Hiroyuki Oya, Satrajit Ghosh, Jessey Wright, Joke Durnez, Russell Poldrack, and Krzysztof Jacek Gorgolewski. 2019. fMRIPrep: a robust preprocessing pipeline for functional MRI. Nature...

    doi:10.1038/s41592-018-0235-4 2019

Show all 42 references
  1. [9]

    Tao Fang, Qian Zheng, and Gang Pan. 2024. Alleviating the Semantic Gap for Generalized fMRI-to-Image Reconstruction. Advances in Neural Information Processing Systems 36 (2024)

    2024

  2. [10]

    Matteo Ferrante, Furkan Ozcelik, Tommaso Boccato, Rufin VanRullen, and Nicola Toschi. 2023. Brain Captioning: Decoding human brain activity into images and text. arXiv preprint arXiv:2305.11560 (2023)

    2023 arXiv

  3. [11]

    William David Foulkes. 1962. Dream reports from different stages of sleep. The Journal of Abnormal and Social Psychology 65, 1 (1962), 14

    1962

  4. [12]

    Jianxiong Gao, Yuqian Fu, Yun Wang, Xuelin Qian, Jianfeng Feng, and Yanwei Fu. 2023. MinD-3D: Reconstruct High-quality 3D objects in Human Brain. arXiv preprint arXiv:2312.07485 (2023)

    2023 arXiv

  5. [13]

    Jianxiong Gao, Yuqian Fu, Yun Wang, Xuelin Qian, Jianfeng Feng, and Yanwei Fu. 2024. fMRI-3D: A Comprehensive Dataset for Enhancing fMRI-based 3D Reconstruction. arXiv preprint arXiv:2409.11315 (2024)

    2024 arXiv

  6. [14]

    Zijin Gu, Keith Wakefield Jamison, Meenakshi Khosla, Emily J Allen, Yihan Wu, Ghislain St-Yves, Thomas Naselaris, Kendrick Kay, Mert R Sabuncu, and Amy Kuceyeski. 2022. Neurogen: activation optimized image synthesis for discovery neuroscience. NeuroImage 247 (2022), 118812

    2022

  7. [15]

    James V Haxby, M Ida Gobbini, Maura L Furey, Alumit Ishai, Jennifer L Schouten, and Pietro Pietrini. 2001. Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science 293, 5539 (2001), 2425–2430

    2001

  8. [16]

    J Allan Hobson, Edward F Pace-Schott, and Robert Stickgold. 2000. Dreaming and the brain: toward a cognitive neuroscience of conscious states. Behavioral and brain sciences 23, 6 (2000), 793–842

    2000

  9. [17]

    Charles Chong-Hwa Hong, James C Harris, Godfrey D Pearlson, Jin-Suh Kim, Vince D Calhoun, James H Fallon, Xavier Golay, Joseph S Gillen, Daniel J Sim- monds, Peter CM Van Zijl, et al. 2009. fMRI evidence for multisensory recruitment associated with rapid eye movements during s...

    2009

  10. [18]

    Tomoyasu Horikawa and Yukiyasu Kamitani. 2017. Generic decoding of seen and imagined objects using hierarchical visual features. Nature communications 8, 1 (2017), 15037

    2017

  11. [19]

    Tomoyasu Horikawa and Yukiyasu Kamitani. 2017. Hierarchical neural represen- tation of dreamed objects revealed by brain decoding with deep neural network features. Frontiers in computational neuroscience 11 (2017), 4

    2017

  12. [20]

    Tomoyasu Horikawa, Masako Tamaki, Yoichi Miyawaki, and Yukiyasu Kamitani

  13. [21]

    Jingyang Huo, Yikai Wang, Yun Wang, Xuelin Qian, Chong Li, Yanwei Fu, and Jianfeng Feng. 2025. Neuropictor: Refining fmri-to-image reconstruction via multi-individual pretraining and multi-level modulation. In European Conference on Computer Vision. Springer, 56–73

    2025

  14. [22]

    Kendrick N Kay, Thomas Naselaris, Ryan J Prenger, and Jack L Gallant. 2008. Identifying natural images from human brain activity. Nature 452, 7185 (2008), 352–355

    2008

  15. [23]

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. 2014. Microsoft coco: Common objects in context. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proce...

    2014

  16. [24]

    Pierre Maquet, Jean-Marie Péters, Joël Aerts, Guy Delfiore, Christian Degueldre, André Luxen, and Georges Franck. 1996. Functional neuroanatomy of human rapid-eye-movement sleep and dreaming. Nature 383, 6596 (1996), 163–166

    1996

  17. [25]

    Karla L Miller, Fidel Alfaro-Almagro, Neal K Bangerter, David L Thomas, Essa Yacoub, Junqian Xu, Andreas J Bartsch, Saad Jbabdi, Stamatios N Sotiropoulos, Jesper LR Andersson, et al. 2016. Multimodal population brain imaging in the UK Biobank prospective epidemiological study....

    2016

  18. [26]

    Milad Mozafari, Leila Reddy, and Rufin VanRullen. 2020. Reconstructing Natural Scenes from fMRI Patterns using BigBiGAN. In2020 International Joint Conference on Neural Networks, IJCNN 2020, Glasgow, United Kingdom, July 19-24, 2020 . IEEE, 1–8

    2020

  19. [27]

    Georg Northoff, Andrea Scalabrini, and Stuart Fogel. 2023. Topographic-dynamic reorganisation model of dreams (TRoD)–A spatiotemporal approach.Neuroscience & Biobehavioral Reviews 148 (2023), 105117

    2023

  20. [28]

    Furkan Ozcelik and Rufin VanRullen. 2023. Natural scene reconstruction from fMRI signals using generative latent diffusion. Scientific Reports 13, 1 (2023), 15666

    2023

  21. [29]

    Xuelin Qian, Yun Wang, Jingyang Huo, Jianfeng Feng, and Yanwei Fu. 2023. fMRI-PTE: A Large-scale fMRI Pretrained Transformer Encoder for Multi-Subject Brain Activity Decoding. arXiv preprint arXiv:2311.00342 (2023)

    2023 arXiv

  22. [30]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learni...

    2021

  23. [31]

    Ziqi Ren, Jie Li, Xuetong Xue, Xin Li, Fan Yang, Zhicheng Jiao, and Xinbo Gao

  24. [32]

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 10684–10695

    2022

  25. [33]

    Paul S Scotti, Atmadeep Banerjee, Jimmie Goode, Stepan Shabalin, Alex Nguyen, Ethan Cohen, Aidan J Dempster, Nathalie Verlinde, Elad Yundler, David Weisberg, et al. 2023. Reconstructing the Mind’s Eye: fMRI-to-Image with Contrastive Learning and Diffusion Priors. arXiv preprin...

    2023 arXiv

  26. [34]

    Guohua Shen, Tomoyasu Horikawa, Kei Majima, and Yukiyasu Kamitani. 2019. Deep image reconstruction from human brain activity. PLoS computational biology 15, 1 (2019), e1006633

    2019

  27. [35]

    Bertrand Thirion, Edouard Duchesnay, Edward Hubbard, Jessica Dubois, Jean- Baptiste Poline, Denis Lebihan, and Stanislas Dehaene. 2006. Inverse retinotopy: inferring the visual content of images from brain activation patterns.Neuroimage 33, 4 (2006), 1104–1116

    2006

  28. [36]

    Bohan Zeng, Shanglin Li, Xuhui Liu, Sicheng Gao, Xiaolong Jiang, Xu Tang, Yao Hu, Jianzhuang Liu, and Baochang Zhang. 2023. Controllable Mind Visual Diffusion Model. arXiv preprint arXiv:2305.10135 (2023)

    2023 arXiv

  29. [37]

    Sixiao Zheng, Jingyang Huo, Yu Wang, and Yanwei Fu. 2024. Intelligent Director: An Automatic Framework for Dynamic Visual Composition using ChatGPT. arXiv preprint arXiv:2402.15746 (2024)

    2024 arXiv

  30. [2013]

    Science 340, 6132 (2013), 639–642

    Neural decoding of visual imagery during sleep. Science 340, 6132 (2013), 639–642

    2013

  31. [2018]

    Software (2018)

    fMRIPrep. Software (2018). https://doi.org/10.5281/zenodo.852659

    2018 doi

  32. [2021]

    NeuroImage 228 (2021), 117602

    Reconstructing seen image from brain activity by visually-guided cognitive representation and adversarial learning. NeuroImage 228 (2021), 117602

    2021

  33. [2022]

    Nature neuroscience 25, 1 (2022), 116–126

    A massive 7T fMRI dataset to bridge cognitive neuroscience and artificial intelligence. Nature neuroscience 25, 1 (2022), 116–126

    2022

  34. [2023]

    In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Seeing beyond the brain: Conditional diffusion model with sparse masked modeling for vision decoding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 22710–22720

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.