Dynadiff: Single-stage Decoding of Images from Continuously Evolving fMRI

Jean-R\'emi King; Marl\`ene Careil; Yohann Benchetrit

REVIEW 4 major objections 5 minor 1 cited by

Dynadiff: Single-stage Decoding of Images from Continuously Evolving fMRI

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that a single-stage diffusion decoder trained directly on time-varying fMRI can outperform multi-stage brain-to-image pipelines on semantic metrics.

desk verdict A solid single-stage time-resolved fMRI decoder whose main advantage over prior SOTA is plausible but rests on a weak baseline adaptation; worth reviewing seriously. read the letter →

arxiv 2505.14556 v1 pith:QI4GWZ7T submitted 2025-05-20 cs.CV

Marlène Careil , Yohann Benchetrit , Jean-Rémi King This is my paper

classification cs.CV

keywords fMRI-to-imagedecodingdiffusionmodeltime-resolvedMRINaturalScenesDatasetsingle-stagetrainingBOLDtimeseriesbrainlatent

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that time-resolved fMRI decoding does not need the multi-stage, time-collapsing pipelines that currently dominate brain-to-image reconstruction. It introduces Dynadiff, which trains a single brain module and LoRA adapters on a pretrained latent diffusion model to map a window of BOLD volumes directly to an image. On the Natural Scenes Dataset, it reports higher semantic and low-level scores than MindEye2 on single-trial time series, and competitive results on the standard averaged-beta benchmark. A sympathetic reader would care because a simpler, time-resolved decoder both lowers the engineering barrier for brain decoding and opens a window onto how image representations evolve in the brain over seconds.

What carries the argument

The load-bearing object is the brain module, a small MLP that turns a $C \times T$ fMRI window into the conditioning embedding of a pretrained latent diffusion model: a subject-specific linear layer projects each volume to 1,552 channels, a timestep-specific linear layer applies distinct weights to each time sample, a linear temporal-aggregation layer collapses time, and a final linear layer produces the $257 \times 768$ conditional embedding. The diffusion model's cross-attention layers are adapted with LoRA (rank 4) while the rest of the generation model stays frozen, and the whole system is trained end-to-end with the standard diffusion loss, bicubic sampling, offset noise, and 10% dropped conditioning for classifier-free guidance. The timestep-specific layer is what carries the time-resolved claim: ablating it to a shared linear layer costs 2.95 CLIP-12 points, and moving temporal aggregation earlier in the module also hurts.

What would settle it

Run the same single-trial Natural Scenes Dataset evaluation with a time-aware variant of MindEye2, for example per-time linear projections or a small temporal transformer instead of flattening, and compare AlexNet(2), CLIP-12, and DreamSim; if that variant matches or exceeds Dynadiff's 95.82, 93.53, and 52.52, the paper's central superiority claim fails.

Watch

Extended reading notes

Core claim

Dynadiff's central claim is that a seen image can be reconstructed directly from a window $X \in \mathbb{R}^{C \times T}$ of BOLD fMRI volumes by fine-tuning a pretrained latent diffusion model in one stage, without time-collapsing $\beta$ values and without separate alignment, captioning, refinement, or candidate-selection modules. On single-trial evaluation over the four Natural Scenes Dataset subjects, the paper reports that this single-stage model outperforms MindEye2 on AlexNet(2) (95.82 vs 94.15), AlexNet(5) (98.20 vs 97.34), CLIP-12 (93.53 vs 90.38), and DreamSim (52.52 vs 56.28, lower is better), and keeps competitive SSIM and PixCorr. The same architecture, with its timestep-specific and temporal-aggregation layers removed, remains competitive on the averaged $\beta$ values used by previous work, which the paper reads as evidence that the temporal machinery is what buys the time-resolved advantage.

Load-bearing premise

The claim that Dynadiff is the best time-resolved decoder depends on the flattened time-window input being a fair adaptation of MindEye1 and MindEye2, which were originally built for time-averaged brain responses; a stronger time-aware baseline could erase the reported gap.

Editorial extensions

If this is right

Time-resolved fMRI decoding can be done in one training stage: no pretrained fMRI encoder, contrastive alignment, image captioning, candidate selection, or separate low- and high-level decoders are needed to beat multi-stage pipelines on semantic metrics.
A decoder trained on a fixed 8-second window generalizes to nearby time windows but performs best when trained for the specific onset, so image representations in fMRI are not stationary across time.
A specialized decoder can still read the current image from windows shifted up to about 10 seconds after onset, while a general decoder shifted by one stimulus period reconstructs the previous or next image, implying successive images are simultaneously represented in the signal.
On standard time-collapsed beta values, the same single-stage model is competitive with or better than DREAM, UMBRAE, MindBridge, and MindEye1, and close to MindEye2 and NeuroPictor, so the simplification does not sacrifice the usual benchmark.
Ablations show the timestep-specific layers and their placement carry most of the temporal benefit, pointing to the temporal aggregation design as the reason Dynadiff works on time series.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A direct next step the paper leaves implicit is applying the same single-stage time-window decoder to movie-viewing fMRI without averaging repetitions; if the mechanism transfers, video decoding could inherit the same simplicity.
The specialized-versus-general decoding pattern suggests a testable prediction about neural codes: the representation of an image at a shifted delay is not a time-shifted copy of its representation at the training delay, otherwise the general model would track the specialized one.
Because the paper's beta-value results are competitive even without temporal layers, the temporal machinery is most valuable where timing is preserved; gains on time-collapsed benchmarks may come more from generative conditioning than from fMRI preprocessing.
The cross-subject pretraining results suggest that a single shared brain module fine-tuned per subject could reduce per-subject data needs; quantifying this on truly unseen subjects with one hour of data would extend the paper's own finetuning curves.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

First, what's worth knowing. Dynadiff shows that a single diffusion model, with a small brain module and LoRA adapters on cross-attention, can decode images from raw fMRI timeseries on NSD, landing at roughly the same level as much heavier multi-stage pipelines. The single-stage training is a real simplification, and the temporal generalization analysis (specialized vs general models across shifted windows) is genuinely informative. Those are the two actual contributions.

The main weakness is the baseline comparison behind the headline claim. MindEye1 and MindEye2 were built for averaged beta values; adapting them to time series by flattening the six TRs into one vector and inflating the first linear layer is a weak interface. It throws away temporal structure and probably understates what a time-aware version of those models could do. So the exact AlexNet2/CLIP/DreamSim margins over MindEye2 on time-resolved data should be read with a grain of salt. The paper does show the full metric table, and MindEye2 still wins SSIM and PixCorr, so the story is 'competitive with an edge on high-level similarity' more than a clean sweep. The beta-value comparison in Table 4 uses numbers from the original papers, which is standard but less controlled; the authors admit this.

The rest is in decent shape. The ablations are sensible and cover the intended components. The time-shift experiment uses the authors' own models, so the observation that specialized models outperform a general model outside the training window is a fair empirical finding. The dynamic coding interpretation is speculative, but they say so in the Limits section, and the data support the observation even if the neural interpretation is open. One gap: the test metrics come from a single random selection of one of three repetitions per image, with SEM across subjects; a few seeds and error bars over that selection would have been better. That is a minor omission, not a fatal one.

If I were handling this, I would send it to review. The method is simple, reproducible, and the core result is likely to hold in broad strokes even if the exact margins shrink with fairer baselines. I would ask referees to push for a time-aware baseline comparison and repetition-seed sensitivity checks, but the paper has real content and a clear presentation. Yes, worth a serious referee.

Referee Report

4 major / 5 minor

Summary. The manuscript introduces Dynadiff, a single-stage latent diffusion decoder that reconstructs natural images from time-resolved BOLD fMRI signals. The brain module maps each fMRI volume to a shared embedding space, applies timestep-specific linear layers, temporally aggregates the volumes, and feeds the resulting conditioning embedding into a pretrained latent diffusion model whose cross-attention layers are fine-tuned with LoRA. Training uses the standard diffusion objective, with classifier-free guidance at inference. On the Natural Scenes Dataset (subjects 1, 2, 5, 7), the authors evaluate single-trial reconstructions against WAVE, Brain-Diffuser, MindEye1, and MindEye2, reporting advantages on several high-level semantic metrics (AlexNet(2/5), CLIP-12, Inception, mIoU, DreamSim) while trailing MindEye2 on SSIM and PixCorr. The paper also presents a time-resolved analysis using 'general' and 'specialized' decoders across shifted time windows, together with ablations of window duration, brain-module design, and diffusion-finetuning strategy.

Significance. If the empirical comparisons hold, the contribution is significant: a simpler single-stage decoder that works directly on BOLD time series would be a useful benchmark and a step toward time-resolved brain decoding. The paper has several concrete strengths: public code is provided, evaluation is performed on single trials without averaging same-image repetitions, and the core result is an empirical decoder measured on held-out images rather than a circular or parameter-free claim. The time-shift experiment also addresses an interesting question about the temporal evolution of fMRI representations. However, the headline advantage over MindEye2 currently rests on a baseline adaptation that may be unfair, and the reported numeric inconsistencies in the ablations need to be corrected before the central claims can be fully assessed.

major comments (4)

[Section 3.1, Table 1; Appendix A] The headline claim that Dynadiff 'outperforms state-of-the-art models on time-resolved fMRI signals' is not yet supported because MindEye1 and MindEye2 are adapted to time series by flattening each C x T window into a single vector and increasing the first linear layer from C to C x T. This adaptation destroys temporal ordering, enlarges the first-layer parameter budget roughly sixfold, and is not accompanied by retuning of the rest of the pipeline. A fairer comparison would include at least one time-aware adaptation of these beta-oriented models, such as per-volume shared linear layers followed by temporal aggregation, or a lightweight temporal fusion layer applied to the original pretrained features. Without such a baseline, the observed lead on AlexNet(2), CLIP-12, and DreamSim could reflect an underpowered baseline rather than a genuine advantage of single-stage temporal decoding.
[Section 3.1, Table 1] The reported SEM is computed across the four subjects, but the evaluation protocol randomly selects one of three repetitions per test image. The paper provides no test of whether the central metric differences (e.g., AlexNet(2) 95.82 vs 94.15, CLIP-12 93.53 vs 90.38) are stable across repetition selections. Moreover, Dynadiff is below MindEye2 on SSIM (0.34 vs 0.36) and PixCorr (0.21 vs 0.24), so even the point estimates only partially support an 'outperforms' statement. Please provide bootstrap or repeated-selection confidence intervals for the Dynadiff-versus-MindEye2 differences and state precisely which metrics support the claimed superiority.
[Section 2.3, Figures 4 and 5] The run-based time-resolved split is not fully specified. Because each image was presented three times, a test run that contains an image whose other presentations fall in training runs would create image-level leakage; conversely, selecting test runs that contain only the original held-out images is nontrivial given NSD's interleaved presentation order. Please report the image overlap between the run-based training and test sets and evaluate the time-resolved analysis on the original 1,000-image test split as a check. This matters because the 'specialized vs general' finding is one of the paper's three stated contributions.
[Tables 2 and 3] The reported numbers for the main configuration are internally inconsistent. Table 2's full row ('tick OUT') reports AlexNet(2)=94.67 and CLIP-12=97.45, while Table 1 reports 95.82 and 93.53 for the same model; Table 3 repeats the 94.67/97.45 values under 'LoRA on Cross-Attn'. The ablation text's stated drops (2.95 CLIP-12 and 1.33 AlexNet(2)) do not correspond to any pairwise difference in Table 2. These inconsistencies must be resolved because the conclusions about timestep-specific layers and LoRA adaptation depend on these numbers.

minor comments (5)

[Section 2.3] The text says the new time-resolved split is used for 'Figures 3 and 5', but Figure 3 does not show successive-image reconstructions; the intended references are likely Figures 4 and 5.
[Section 4] The dynamic-coding discussion says 'This result is clearest in Figure 3'; the intended reference is probably Figure 4, which shows previous/current/next decoding.
[Table 8] The Subject 5 row is missing the mIoU entry; please complete the table.
[Table 4 and Appendix C] Comparisons on beta values are imported from the respective original papers rather than re-run under identical conditions; this should be stated prominently in the main text, and the claim of being 'competitive' on preprocessed fMRI data should be interpreted accordingly.
[Figure 5] The x-axis of Figure 5 is defined as the end of the time window, while Figure 4 uses delta in units of TR; aligning the notation or adding a schematic of the window positions would improve readability.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation; the claims are empirical benchmark results with external baselines and ablations.

full rationale

Dynadiff makes no parameter-free derivation: its headline results are obtained by training a diffusion model with a standard diffusion loss and evaluating on held-out test images against externally published baselines (MindEye1/MindEye2, Brain-Diffuser, WAVE). The time-resolved advantage is a measured outcome of models trained on different time windows, and the 'specialized beats general' pattern in Figure 5 is an empirical observation, not an identity imposed by construction. Self-citations to Benchetrit et al. 2024, King and Dehaene 2014, and Défossez et al. 2022 appear as related-work context, an architectural channel-count choice, and an interpretive analogy; none is load-bearing in the sense of replacing measurement or forbidding alternatives. The flattening adaptation of MindEye baselines in Appendix A could raise a comparison-fairness concern, but it is not circular: the baseline numbers are not derived from Dynadiff's own outputs or fitted parameters. Overall, the derivation chain is self-contained as an empirical study, and no step reduces to its own inputs by definition.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central claim rests on standard deep learning assumptions (pretrained generative model, CLIP conditioning) and on dataset and temporal assumptions. No new physical entities or ad hoc mathematical axioms are introduced. The main free parameters are architectural and inference hyperparameters chosen by hand; the per-voxel detrending and z-score statistics are fitted during preprocessing.

free parameters (6)

Temporal window start t = 3 s after stimulus onset
Main training window uses t = 3 s; chosen to align with the hemodynamic response, consistent with prior NSD decoders.
Temporal window duration d = 8 s (6 TR)
Hand-set to match prior work; Section 3.2 ablation shows d = 3 TR (3.9 s) is near-optimal and 6 TR gives a small gain.
LoRA rank and alpha = rank = 4, alpha = 4
Appendix B; small capacity chosen to avoid overfitting the 27k-trial training set; no sweep reported.
Classifier-free guidance scale = 3
Inference hyperparameter used with 20 DDIM steps; not tuned per metric.
Brain module dropout = p = 0.5
Applied after GELU in the temporal MLP; chosen by hand.
Per-voxel detrending and z-score statistics = per-voxel cosine-drift coefficients, mean, std
Cosine-drift linear model fit to each voxel and subtracted; each voxel time series z-scored. Standard preprocessing, but these statistics are fit on the training fMRI data.

assumptions (4)

domain assumption Versatile Diffusion and CLIP encoders remain valid when conditioned with fMRI-derived embeddings
Dynadiff replaces CLIP image embeddings with brain module outputs; the pretrained U-Net's cross-attention must still produce coherent latents for these inputs (Section 2.2).
domain assumption Single-trial BOLD at TR = 1.3 s retains decodable stimulus information after detrending and z-scoring
The entire task assumes time-resolved fMRI contains enough signal; supported empirically by Table 1 but not proven.
domain assumption The standard NSD train/test split and the new run-based split prevent train-test contamination across temporally adjacent trials
NSD interleaves train and test image presentations; the time-resolved split uses contiguous runs to avoid leakage (Section 2.3).
domain assumption The diffusion loss and LoRA fine-tuning do not rely on memorized training images to reconstruct the 1,000 test images
Test images are distinct from training images, but all come from MS-COCO; the model could exploit dataset-level statistics (Discussion, Limits).

how reviews work

0 comments

Cite this review

Pith. "Pith review of Dynadiff: Single-stage Decoding of Images from Continuously Evolving fMRI." pith.science (2026). https://pith.science/paper/QI4GWZ7T

@misc{pith2026250514556,
  author       = {Pith},
  title        = {Pith review of: Dynadiff: Single-stage Decoding of Images from Continuously Evolving fMRI},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QI4GWZ7T}},
  note         = {Machine review of arXiv:2505.14556}
}

read the original abstract

Brain-to-image decoding has been recently propelled by the progress in generative AI models and the availability of large ultra-high field functional Magnetic Resonance Imaging (fMRI). However, current approaches depend on complicated multi-stage pipelines and preprocessing steps that typically collapse the temporal dimension of brain recordings, thereby limiting time-resolved brain decoders. Here, we introduce Dynadiff (Dynamic Neural Activity Diffusion for Image Reconstruction), a new single-stage diffusion model designed for reconstructing images from dynamically evolving fMRI recordings. Our approach offers three main contributions. First, Dynadiff simplifies training as compared to existing approaches. Second, our model outperforms state-of-the-art models on time-resolved fMRI signals, especially on high-level semantic image reconstruction metrics, while remaining competitive on preprocessed fMRI data that collapse time. Third, this approach allows a precise characterization of the evolution of image representations in brain activity. Overall, this work lays the foundation for time-resolved brain-to-image decoding.

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Real-time Reconstruction of Human Visual Perception from fMRI
cs.CV 2026-07 conditional novelty 6.0 of 10

First demonstration that single-trial visual images can be decoded from fMRI in near-real-time (about 10-15 seconds) with roughly one hour of fine-tuning data.

Reference graph

Works this paper leans on

17 extracted references · 7 canonical work pages · cited by 1 Pith paper

[3]

Brain captioning: Decoding human brain activity into images and text, 2023.https://arxiv.org/abs/2305.11560

Matteo Ferrante, Furkan Ozcelik, Tommaso Boccato, Rufin VanRullen, and Nicola Toschi. Brain captioning: Decoding human brain activity into images and text, 2023.https://arxiv.org/abs/2305.11560. Bruce Fischl, Martin I. Sereno, Roger B.H. Tootell, and Anders M. Dale. High-resolution intersubject averaging and a coordinate system for the cortical surface.Hu...

arXiv 2023
[4]

Dreamsim: Learning new dimensions of human visual similarity using synthetic data, 2023.https://arxiv.org/abs/2306.09344

Stephanie Fu, Netanel Tamir, Shobhita Sundaram, Lucy Chai, Richard Zhang, Tali Dekel, and Phillip Isola. Dreamsim: Learning new dimensions of human visual similarity using synthetic data, 2023.https://arxiv.org/abs/2306.09344. Zijin Gu, Keith Jamison, Amy Kuceyeski, and Mert Sabuncu. Decoding natural image stimuli from fmri data with a surface-based convo...

arXiv 2023
[8]

Yan Liu, Zehao Zhao, Minpeng Xu, Haiqing Yu, Yanming Zhu, Jie Zhang, Linghao Bu, Xiaoluo Zhang, Junfeng Lu, Yuanning Li, et al

http://arxiv.org/abs/1405.0312. Yan Liu, Zehao Zhao, Minpeng Xu, Haiqing Yu, Yanming Zhu, Jie Zhang, Linghao Bu, Xiaoluo Zhang, Junfeng Lu, Yuanning Li, et al. Decoding and synthesizing tonal language speech from brain activity.Science Advances, 9(23): eadh0478,

arXiv
[11]

Reconstructing the mind’s eye: fMRI-to-image with contrastive learning and diffusion priors.arXiv preprint arXiv:2305.18274,

Paul S Scotti, Atmadeep Banerjee, Jimmie Goode, Stepan Shabalin, Alex Nguyen, Ethan Cohen, Aidan J Dempster, Nathalie Verlinde, Elad Yundler, David Weisberg, et al. Reconstructing the mind’s eye: fMRI-to-image with contrastive learning and diffusion priors.arXiv preprint arXiv:2305.18274,

arXiv
[12]

MindEye2: Shared-subject models enable fMRI-to-image with 1 hour of data.arXiv preprint arXiv:2403.11207,

Paul S Scotti, Mihir Tripathy, Cesar Kadir Torrico Villanueva, Reese Kneeland, Tong Chen, Ashutosh Narang, Charan Santhirasegaran, Jonathan Xu, Thomas Naselaris, Kenneth A Norman, et al. MindEye2: Shared-subject models enable fMRI-to-image with 1 hour of data.arXiv preprint arXiv:2403.11207,

arXiv
[14]

11.18.517004

doi: 10.1101/2022.11.18.517004.https://www.biorxiv.org/content/early/2023/03/11/2022. 11.18.517004. Jerry Tang, Amanda LeBel, Shailee Jain, and Alexander G Huth. Semantic reconstruction of continuous language from non-invasive brain recordings.Nature Neuroscience, 26(5):858–866,

work page doi:10.1101/2022.11.18.517004.https://www.biorxiv.org/content/early/2023/03/11/2022 2022
[15]

Mindbridge: A cross-subject brain decoding framework

Shizun Wang, Songhua Liu, Zhenxiong Tan, and Xinchao Wang. Mindbridge: A cross-subject brain decoding framework. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11333–11342, 2024a. Yanchen Wang, Adam Turnbull, Tiange Xiang, Yunlong Xu, Sa Zhou, Adnan Masoud, Shekoofeh Azizi, Feng Vankee Lin, and Ehsan Adeli. Dec...

work page arXiv
[16]

(2024) approaches reconstruct the stimuli from NSD using volumes of time-collapsed fMRI averaged beta-values restricted to the Region Of Interest ‘nsdgeneral‘

and MindEye 2 Scotti et al. (2024) approaches reconstruct the stimuli from NSD using volumes of time-collapsed fMRI averaged beta-values restricted to the Region Of Interest ‘nsdgeneral‘. We keep the exact same stages, hyperparameters and architectures as in Scotti et al. (2023, 2024), with one exception. Since we train and evaluate on BOLD timeseries of ...

work page 2024

Show all 17 references

[17]

0.30 0.21 94.77 97.34 93.54 91.85 0.69 0.36 8.39 52.80 To complete our experimental study, we trained Dynadiff on the beta values used in all previous image-decoding works on NSD

0.43 0.32 96.10 98.61 92.97 95.41 0.62 0.34 Dynadiff 0.37 0.21 95.72 98.11 94.09 95.03 0.61 0.34 Table 5 Preprocessing ablation Preprocessing type Low-level Semantic and High-level ↑SSIM ↑PixCorr ↑AlexNet(2) ↑AlexNet(5) ↑CLIP-12 ↑Incep ↓Eff ↓SwAV ↑mIoU ↑DreamSim Dynadiff w/ fM...

2023
[18]

D Additional ablation Preprocessing types

and MindEye2 (Scotti et al., 2024), which all require more than one stage of training (unlike Dynadiff). D Additional ablation Preprocessing types. To measure the impact of data preprocessing on our model’s performance, we provide an additional fMRI preprocessing ablation in T...

2024
[2014]

Reconstructing seen images from human brain activity via guided stochastic search, 2023.https://arxiv.org/abs/2305.00556

Reese Kneeland, Jordyn Ojeda, Ghislain St-Yves, and Thomas Naselaris. Reconstructing seen images from human brain activity via guided stochastic search, 2023.https://arxiv.org/abs/2305.00556. Lynn Le, Thirza Dado, Katja Seeliger, Paolo Papale, Antonio Lozano, Pieter Roelfsema,...

2023 arXiv
[2017]

Lora: Low-rank adaptation of large language models.arXiv preprint arXiv:2106.09685,

Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models.arXiv preprint arXiv:2106.09685,

arXiv
[2019]

Aoki, Kei Majima, Yusuke Muraki, and Yukiyasu Kamitani

Ken Shirakawa, Yoshihiro Nagano, Misato Tanaka, Shuntaro C. Aoki, Kei Majima, Yusuke Muraki, and Yukiyasu Kamitani. Spurious reconstruction from brain activity, 2024.https://arxiv.org/abs/2405.10078. Yu Takagi and Shinji Nishimoto. High-resolution image reconstruction with lat...

2024 arXiv
[2021]

High-resolution image synthesis with latent diffusion models, 2022.https://arxiv.org/abs/2112.10752

Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models, 2022.https://arxiv.org/abs/2112.10752. 12 Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biome...

2022 arXiv
[2022]

Brain decoding: toward real-time reconstruction of visual perception

Yohann Benchetrit, Hubert Banville, and Jean-Rémi King. Brain decoding: toward real-time reconstruction of visual perception. In ICLR 2024,

2024
[2023]

Reconstruction of perceived images from fmri patterns and semantic brain exploration using instance-conditioned gans, 2022.https://arxiv.org/ abs/2202.12692

Furkan Ozcelik, Bhavin Choksi, Milad Mozafari, Leila Reddy, and Rufin VanRullen. Reconstruction of perceived images from fmri patterns and semantic brain exploration using instance-conditioned gans, 2022.https://arxiv.org/ abs/2202.12692. Dustin Podell, Zion English, Kyle Lace...

2022 arXiv
[2024]

Alexandre Défossez, Charlotte Caucheteux, Jérémy Rapin, Ori Kabeli, and Jean-Rémi King

https://arxiv.org/abs/2412.17829. Alexandre Défossez, Charlotte Caucheteux, Jérémy Rapin, Ori Kabeli, and Jean-Rémi King. Decoding speech from non-invasive brain recordings.arXiv preprint arXiv:2208.12266,

arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [3]

Brain captioning: Decoding human brain activity into images and text, 2023.https://arxiv.org/abs/2305.11560

Matteo Ferrante, Furkan Ozcelik, Tommaso Boccato, Rufin VanRullen, and Nicola Toschi. Brain captioning: Decoding human brain activity into images and text, 2023.https://arxiv.org/abs/2305.11560. Bruce Fischl, Martin I. Sereno, Roger B.H. Tootell, and Anders M. Dale. High-resolution intersubject averaging and a coordinate system for the cortical surface.Hu...

arXiv 2023

[2] [4]

Dreamsim: Learning new dimensions of human visual similarity using synthetic data, 2023.https://arxiv.org/abs/2306.09344

Stephanie Fu, Netanel Tamir, Shobhita Sundaram, Lucy Chai, Richard Zhang, Tali Dekel, and Phillip Isola. Dreamsim: Learning new dimensions of human visual similarity using synthetic data, 2023.https://arxiv.org/abs/2306.09344. Zijin Gu, Keith Jamison, Amy Kuceyeski, and Mert Sabuncu. Decoding natural image stimuli from fmri data with a surface-based convo...

arXiv 2023

[3] [8]

Yan Liu, Zehao Zhao, Minpeng Xu, Haiqing Yu, Yanming Zhu, Jie Zhang, Linghao Bu, Xiaoluo Zhang, Junfeng Lu, Yuanning Li, et al

http://arxiv.org/abs/1405.0312. Yan Liu, Zehao Zhao, Minpeng Xu, Haiqing Yu, Yanming Zhu, Jie Zhang, Linghao Bu, Xiaoluo Zhang, Junfeng Lu, Yuanning Li, et al. Decoding and synthesizing tonal language speech from brain activity.Science Advances, 9(23): eadh0478,

arXiv

[4] [11]

Reconstructing the mind’s eye: fMRI-to-image with contrastive learning and diffusion priors.arXiv preprint arXiv:2305.18274,

Paul S Scotti, Atmadeep Banerjee, Jimmie Goode, Stepan Shabalin, Alex Nguyen, Ethan Cohen, Aidan J Dempster, Nathalie Verlinde, Elad Yundler, David Weisberg, et al. Reconstructing the mind’s eye: fMRI-to-image with contrastive learning and diffusion priors.arXiv preprint arXiv:2305.18274,

arXiv

[5] [12]

MindEye2: Shared-subject models enable fMRI-to-image with 1 hour of data.arXiv preprint arXiv:2403.11207,

Paul S Scotti, Mihir Tripathy, Cesar Kadir Torrico Villanueva, Reese Kneeland, Tong Chen, Ashutosh Narang, Charan Santhirasegaran, Jonathan Xu, Thomas Naselaris, Kenneth A Norman, et al. MindEye2: Shared-subject models enable fMRI-to-image with 1 hour of data.arXiv preprint arXiv:2403.11207,

arXiv

[6] [14]

11.18.517004

doi: 10.1101/2022.11.18.517004.https://www.biorxiv.org/content/early/2023/03/11/2022. 11.18.517004. Jerry Tang, Amanda LeBel, Shailee Jain, and Alexander G Huth. Semantic reconstruction of continuous language from non-invasive brain recordings.Nature Neuroscience, 26(5):858–866,

work page doi:10.1101/2022.11.18.517004.https://www.biorxiv.org/content/early/2023/03/11/2022 2022

[7] [15]

Mindbridge: A cross-subject brain decoding framework

Shizun Wang, Songhua Liu, Zhenxiong Tan, and Xinchao Wang. Mindbridge: A cross-subject brain decoding framework. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11333–11342, 2024a. Yanchen Wang, Adam Turnbull, Tiange Xiang, Yunlong Xu, Sa Zhou, Adnan Masoud, Shekoofeh Azizi, Feng Vankee Lin, and Ehsan Adeli. Dec...

work page arXiv

[8] [16]

(2024) approaches reconstruct the stimuli from NSD using volumes of time-collapsed fMRI averaged beta-values restricted to the Region Of Interest ‘nsdgeneral‘

and MindEye 2 Scotti et al. (2024) approaches reconstruct the stimuli from NSD using volumes of time-collapsed fMRI averaged beta-values restricted to the Region Of Interest ‘nsdgeneral‘. We keep the exact same stages, hyperparameters and architectures as in Scotti et al. (2023, 2024), with one exception. Since we train and evaluate on BOLD timeseries of ...

work page 2024