Latent graph encoding of multimodal neuroimaging features with generative AI architectures

Ishaan Batta; Meenu Ajith; Vince Calhoun

REVIEW 3 major objections 6 minor 26 references

Latent graph encoding of multimodal neuroimaging features with generative AI architectures

T0 review · 3 major / 6 minor · reviewed 2026-07-09 · glm-5.2

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Graph-encoded brain connectivity beats vectorized generative models

desk verdict Solid systematic comparison of graph vs vectorized encoders for multimodal neuroimaging generation, but the abstract overclaims superiority — generation is mixed, reconstruction is partly circular, and gMMVAE is actually worst on latent discriminability with dual conditioning. read the letter →

arxiv 2607.07027 v1 pith:WOWHFOLM submitted 2026-07-08 cs.LG cs.AIcs.CV

Ishaan Batta , Meenu Ajith , Vince Calhoun This is my paper · ORCID

classification cs.LGcs.AIcs.CV

keywords graphneuralnetworksvariationalautoencodermultimodalfusionneuroimagingfunctionalconnectivitygenerativemodelslatentdiffusionbrainnetworktopology

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that when building generative models for multimodal brain imaging data, the way each modality is encoded into a latent representation matters more than the choice of generative architecture itself. Functional brain connectivity, naturally structured as a graph of relationships between brain regions, should be encoded with a graph neural network rather than flattened into a vector. Structural brain features, which lack relational structure, can remain as vectors. The authors fuse these two modality-specific encodings into a shared latent space using a mixture-of-experts variational autoencoder framework, and show that this combination, called gMMVAE, matches or surpasses diffusion models, transformers, and GANs across reconstruction quality, generation fidelity, computational efficiency, and latent-space discriminability. A key subsidiary finding is that conditioning both the encoder and decoder on subject-level variables like sex substantially improves the discriminative quality of the learned latent space, especially for diffusion-based models.

What carries the argument

GATv2 graph attention network encoder for functional connectivity matrices, MLP encoder for structural volume vectors, mixture-of-experts variational multimodal fusion into a shared 40-dimensional latent space, FiLM conditioning at the decoder, and dual encoder-and-decoder conditioning with subject covariates

What would settle it

If statistical tests showed that the near-identical reconstruction metrics across gMMVAE, gLDM, and gDiT are indistinguishable, and if the generation-quality advantages of gMMVAE in Wasserstein distance and KL divergence were offset by its losses in MMD, the central claim of superiority would weaken to a claim of parity with greater efficiency.

Watch

Extended reading notes

Core claim

Graph-based encoding of functional connectivity into a shared latent space yields generative performance comparable to or better than computationally heavier diffusion and transformer approaches, while requiring far fewer parameters and enabling single-pass generation. The advantage is attributable to the graph encoder preserving topological structure of brain networks, not to the specific generative architecture, since graph-based variants of VAE, latent diffusion, and diffusion transformer all perform nearly identically on reconstruction.

Load-bearing premise

The claim that gMMVAE surpasses alternatives across multiple metrics rests on comparisons where several reconstruction metrics are nearly identical across the top three graph-based models and no statistical significance testing is reported, making it unclear whether the observed differences reflect real advantages rather than noise.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The headline: this is a useful systematic comparison of graph-based vs vectorized encoders across VAE, GAN, diffusion, and transformer generative models for multimodal neuroimaging, but the abstract's claim that gMMVAE 'surpasses across multiple metrics' is overstated. The honest claim is 'best efficiency-quality tradeoff among graph-based models.' The stress-test note is mostly right but miscounts one table — more on that below. What's genuinely new and well done: the systematic evaluation across 8 models with consistent metrics is valuable for practitioners. The graph-encoding advantage for fMRI reconstruction is clearly demonstrated (Table 2: SSIM 0.67 vs 0.51 for vectorized variants, PSNR 20.7 vs 18.6 dB). The efficiency analysis (Table 3) is unambiguous — gMMVAE at 31ms latency vs 146ms for gLDM and 544ms for gDiT. The ablation on encoder-only vs dual conditioning (Table 5) is informative, showing diffusion models benefit more from decoder conditioning than VAEs do. The dataset is large (10,000 UK Biobank subjects). Now the soft spots, in proportion. First, the generation results (Table 4) are genuinely mixed, not a clean win. gMMVAE wins 4 of 6 metrics (fMRI WD, fMRI KL, sMRI MMD, sMRI KL), while gDiT wins fMRI MMD and sMRI WD. The stress-test says '3-of-6' — that's a miscount; it's 4-of-6 for gMMVAE. Still, losing on fMRI MMD by 34% undercuts 'surpasses.' Second, the reconstruction comparison between gMMVAE and gLDM/gDiT is partly circular: §3.2 states gLDM and gDiT use two-stage training with MMVAE pretraining followed by diffusion refinement. So the near-identical reconstruction numbers (Table 2: fMRI correlation 0.87±0.04 for all three) reflect architectural nesting, not an independent comparison. The paper acknowledges this in passing ('advantage is due to VAE-based latent representations rather than specific diffusion architectures') but the abstract doesn't flag it. Third, on latent discriminability with dual conditioning (Table 5), gMMVAE is actually the worst of the three (89.17% vs 89.87% for gLDM and 90.47% for gDiT). The differences are within standard deviations, which brings up the broader issue: no statistical significance testing anywhere. Several reconstruction metrics overlap substantially across models. Finally, no code is released, which limits reproducibility. The individual components (GATv2, MoE fusion, FiLM conditioning) are all from prior work — the contribution is the specific combination and the empirical comparison. That's a legitimate contribution for a methods paper, but it needs the claims tempered and ideally some significance testing. Who benefits: neuroimaging methods researchers choosing encoder architectures for multimodal generative models. The systematic comparison is the real value, not the gMMVAE architecture itself. Recommend conditional acceptance pending claim revision and statistical testing.

Referee Report

3 major / 6 minor

Summary. This paper proposes a multimodal generative framework for structural (GMV) and functional (sFNC) MRI features. The central architectural choice is a modality-aware graph encoder (GATv2) for functional connectivity paired with an MLP encoder for structural features, fused via a Mixture-of-Experts (MoE) variational autoencoder. The authors compare VAEs, latent diffusion models, diffusion transformers, GANs, and data-space DDPMs across reconstruction quality, generation fidelity, computational efficiency, and latent-space discriminability on 10,000 UK Biobank subjects. The headline claim is that graph-based architectures outperform vectorized or data-space baselines, and that the proposed gMMVAE surpasses alternative generative variants across multiple metrics.

Significance. The systematic comparison of generative architectures (VAE, LDM, DiT, DDPM, GAN) within a unified multimodal neuroimaging pipeline is a useful contribution to the field. The use of a graph attention encoder for sFNC matrices is well-motivated by the network structure of functional connectivity data. The efficiency analysis (Table 3) is a practical strength, showing that gMMVAE achieves a 4.7x latency reduction over gLDM and 17.5x over gDiT while maintaining comparable reconstruction quality. The ablation study on dual conditioning (Table 5) provides actionable insight that diffusion-based models benefit more from decoder-level conditioning than VAEs. The framework is evaluated on a large dataset with a comprehensive metric suite.

major comments (3)

The abstract states that gMMVAE 'surpasses alternative generative variants across multiple metrics for generation fidelity, reconstruction quality, efficiency, and latent space discriminability.' This claim is only clearly supported on the efficiency axis (Table 3). On generation fidelity (Table 4), results are mixed: gDiT achieves better fMRI MMD (0.0215 vs. 0.0326) and better sMRI WD (2.7466 vs. 3.1965), while gMMVAE wins on the remaining three metric-modality pairs. On latent space discriminability (Table 5, dual conditioning), gMMVAE achieves the lowest accuracy (89.17%) compared to gLDM (89.87%) and gDiT (90.47%). The abstract claim should be revised to accurately reflect that gMMVAE offers the best efficiency-quality tradeoff, not universal superiority across all named axes.
§3.2 states that 'gLDM and gDiT used two stage training consisting of MMVAE pretraining followed by diffusion refinement.' This means gLDM and gDiT contain gMMVAE as a pretrained subcomponent. Consequently, the near-identical reconstruction metrics in Table 2 (e.g., fMRI correlation 0.87±0.04 and SSIM 0.67±0.08 for gMMVAE, gLDM, and gDiT) are expected by construction rather than being an independent comparison of architectures. The paper should explicitly acknowledge this architectural nesting in §4.1 and clarify whether the reconstruction comparison between gMMVAE and its diffusion refinements (gLDM, gDiT) is a fair or meaningful comparison, or whether the relevant comparison is between graph-based and vectorized encoders (gMMVAE vs. vMMVAE), where the advantage is clear.
No statistical significance testing is reported for any comparison across Tables 2, 4, and 5. Several standard deviations overlap substantially (e.g., Table 2 fMRI correlation for gDiT, gLDM, and gMMVAE are all 0.87±0.04; Table 5 accuracy values overlap within one standard deviation). Without paired statistical tests or confidence intervals on the differences, it is unclear whether the observed metric differences (e.g., gMMVAE fMRI PSNR 20.70 vs. gLDM 20.67) reflect meaningful improvements or sampling noise. The authors should add appropriate statistical tests (e.g., paired permutation tests or bootstrap CIs) for the key comparisons, or at minimum qualify the claims accordingly.

minor comments (6)

Table 1: The 'Beta schedule' row is marked '-' for vMMVAE and WGAN-GP, but §3.2 mentions a quadratic beta schedule for gLDM and gDiT. It is unclear whether vMMVAE uses a different schedule (e.g., linear or none) or if this is an omission. Please clarify.
§2.2, Eq. (3): The notation uses f_μ and f_σ for modality encoder functions, but Eq. (2) uses p_θf and p_θs for decoders. Meanwhile Eq. (4) uses β(·) as a FiLM projection, which conflicts with β in Eq. (2) controlling the KL trade-off. Consider disambiguating these symbols.
§4.3: The statement 'gMMVAE outperformed diffusion variants in WD and KL divergence' for fMRI is correct per Table 4, but the subsequent claim that this means it 'accurately models the overall distribution structure' would be strengthened by noting that MMD, WD, and KL capture different aspects (local vs. global). The text partially addresses this but could be more precise.
Figure 2: The axis labels and legend are small. Consider enlarging the font sizes and labeling the rows/columns (mean vs. std, sFNC vs. GMV) explicitly to improve readability.
§3.1: The random forest classifier hyperparameters (number of trees, depth, etc.) are not specified. Please include these details for reproducibility.
The paper mentions Optuna hyperparameter optimization in §3.2 but does not report the search space or ranges. Consider including these in a supplementary table.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found — standard architectures, externally cited formulations, empirical evaluation

full rationale

The paper's derivation chain is entirely self-contained and non-circular. All load-bearing formulations are standard and externally cited: the MMVAE/MoE fusion mechanism (Eq. 1) is from Shi et al. 2019 [16], the ELBO objective (Eq. 2) is the standard VAE loss, the GATv2 encoder is from Brody et al. 2021 [15], and the FiLM conditioning (Eq. 4) is from Perez et al. 2018 [17]. No equation reduces to its own inputs by construction. The authors do not cite their own prior work as load-bearing for any mathematical claim. The central contribution is empirical — an architecture evaluated on UK Biobank data against independently motivated baselines (DDPM, WGAN-GP, DiT, LDM, MMVAE) using standard external metrics (MSE, MMD, WD, KL, SSIM, PSNR, classification accuracy). The skeptic's concern that gLDM/gDiT use gMMVAE as a pretrained subcomponent (§3.2) is a valid methodological concern about fair comparison and claim strength, but it is not circularity: the paper does not claim to 'predict' or 'derive' a result that was fitted by construction. The near-identical reconstruction numbers between gMMVAE and gLDM/gDiT reflect architectural nesting, not a definitional reduction. This is a correctness/overclaiming risk, not a circularity in the derivation chain.

Assumptions & free parameters 7 free parameters · 4 assumptions · 0 invented entities

No new entities, particles, forces, or dimensions are introduced. The contribution is architectural, not ontological.

free parameters (7)

Latent dimension = 40
Chosen dimensionality of the shared latent space; not derived from theory.
Beta (KL weight) = Not specified numerically
Controls reconstruction-regularization trade-off in ELBO (Eq. 2); value not reported despite being a key hyperparameter.
MoE mixture coefficients α_m = Learned
Mixture weights for modality fusion (Eq. 1); learned during training, not independently constrained.
Number of GATv2 layers = 5
Encoder depth; chosen empirically.
Hidden dimension = 256
Network width; chosen empirically.
Diffusion timesteps = 100
For gLDM and gDiT; chosen empirically.
Beta schedule range = 1e-4 to 1.5e-3
Quadratic schedule endpoints; chosen empirically.

assumptions (4)

domain assumption sFNC matrices can be meaningfully represented as weighted graphs where nodes are brain components and edges are Pearson correlations.
§2.1: 'The sFNC matrix is modeled as a weighted graph, where nodes correspond to brain regions and edge weights reflect connectivity strengths.' This is a standard assumption in network neuroscience but determines the entire architectural advantage claimed.
domain assumption The 53-component NeuroMark parcellation provides sufficient granularity for graph-based encoding to outperform vectorized approaches.
§3.1: The graph structure is defined over 53 nodes. If the parcellation is too coarse, graph topology may not differ meaningfully from vectorized representations.
domain assumption Sex is a biologically relevant conditioning variable for brain structure and function.
§2.2: 'sex is used as a representative covariate due to its biological relevance.' This is well-supported in the literature but is a modeling choice that affects latent space structure.
standard math The ELBO objective with MoE fusion produces a meaningful shared latent space for multimodal data.
§2.1, Eq. 2: Standard VAE framework from Shi et al. 2019.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Latent graph encoding of multimodal neuroimaging features with generative AI architectures." pith.science (2026). https://pith.science/paper/WOWHFOLM

@misc{pith2026260707027,
  author       = {Pith},
  title        = {Pith review of: Latent graph encoding of multimodal neuroimaging features with generative AI architectures},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WOWHFOLM}},
  note         = {Machine review of arXiv:2607.07027}
}

read the original abstract

While generative models enable encoding of complex neuroimaging data for feature generation and reconstruction, developing optimal architectural frameworks with appropriate encoding and latent space processes is crucial for studying structural and functional properties of the brain. We design a multimodal generative framework for structural and functional magnetic resonance imaging (MRI) features through systematic evaluation of encoding strategies, latent multimodal fusion, and generative model selection. Using structural gray matter volume (GMV) and static functional network connectivity (sFNC) features from a large neuroimaging dataset, we analyze generative frameworks involving variational autoencoders (VAEs), transformers, generative adversarial networks (GANs), and diffusion models. Architectures that employ modality-aware graph encoding of functional connectivity into a lower-dimensional latent space outperform vectorized encoders or direct data space approaches. The proposed multimodal graph VAE (gMMVAE) surpasses alternative generative variants across multiple metrics for generation fidelity, reconstruction quality, efficiency, and latent space discriminability, highlighting its potential for robust multimodal neuroimaging analysis.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

26 extracted references · 26 canonical work pages

[1]

INTRODUCTION Understanding how brain structure and function relate to externally observed clinical and demographic variables has driven the development of neuroimaging studies leveraging magnetic resonance imaging (MRI) and modern machine learning methods [1, 2]. While one major line of studies focus on learning the associations for accurately predicting ...

work page Pith review arXiv 2026
[2]

The fMRI modality is represented using sFNC matri- ces, while the sMRI modality is represented as a vector of GMV features

METHODS We present a framework for multimodal generative model- ing and fusion of neuroimaging data, where each subject is characterized by both structural and functional MRI informa- tion. The fMRI modality is represented using sFNC matri- ces, while the sMRI modality is represented as a vector of GMV features. Let(X f ,X s)denote paired samples where Xf...

work page
[3]

v” for MLP-based (vMMV AE, vLDM, vDiT), “t

is employed to effectively capture higher-order depen- dencies and preserve network topology. The GATv2 encoder adaptively aggregates information from neighboring nodes through attention-based message passing, resulting in a la- tent representation that captures the relational structure of the functional brain networks. In contrast, the sMRI GMV features ...

work page
[4]

Datasets and Evaluation Metrics This study used the fMRI and sMRI data from 10, 000 UK Biobank subjects (5376 males and 4624 females) [18]

EXPERIMENTS 3.1. Datasets and Evaluation Metrics This study used the fMRI and sMRI data from 10, 000 UK Biobank subjects (5376 males and 4624 females) [18]. Pre- Fig. 1. Overview of the latent generative models architectures based on the choice of encoders and latent space processes. Table 1. Hyperparameter configuration for baseline models Hyperparameter...

work page
[5]

The sFNC features were computed as Pearson cor- relations between TCs of extracted brain components

to obtain subject-specific spatial maps and time courses (TCs). The sFNC features were computed as Pearson cor- relations between TCs of extracted brain components. We used the Neuromark fMRI 1.0 template, consisting of 53 data-driven brain components divided into 7 functional sub- domains: Subcortical (SC), Auditory (AUD), Sensorimotor (SM), Visual (VIS)...

work page
[6]

Reconstruction quality assessment A comprehensive evaluation of reconstruction quality was conducted across 8 generative approaches in Table

RESULTS 4.1. Reconstruction quality assessment A comprehensive evaluation of reconstruction quality was conducted across 8 generative approaches in Table. 2, show- ing that the graph-based models substantially outperform their MLP-based and baseline counterparts. For fMRI, gM- MV AE, gLDM, and gDiT achieve the lowest MSE, superior Frobenius norms, highest...

work page 1965
[7]

Comprehensive evaluation on a large dataset demonstrates that graph-based generative models substantially outperform vectorized approaches

CONCLUSION This paper presents graph-based generative frameworks for multimodal neuroimaging fusion. Comprehensive evaluation on a large dataset demonstrates that graph-based generative models substantially outperform vectorized approaches. The proposed gMMV AE efficiently reconstructs and generates both modalities with high fidelity, surpassing baseline ...

work page
[8]

ACKNOWLEDGMENTS This work was supported by NIH grants 1R01AG090597, R01AG073949 awarded to Vince Calhoun

work page

Show all 26 references

[9]

Machine learn- ing in neuroimaging: from research to clinical practice,

Karl-Heinz Nenning and Georg Langs, “Machine learn- ing in neuroimaging: from research to clinical practice,” Die Radiologie, vol. 62, no. Suppl 1, pp. 1–10, 2022

2022
[10]

Machine learning in neuroimag- ing: Progress and challenges,

Christos Davatzikos, “Machine learning in neuroimag- ing: Progress and challenges,” 2019

2019
[11]

Neuromark: An auto- mated and adaptive ica based pipeline to identify repro- ducible fmri markers of brain disorders,

Yuhui Du, Zening Fu, Jing Sui, Shuang Gao, Ying Xing, Dongdong Lin, Mustafa Salman, Anees Abrol, Md Ab- dur Rahaman, Jiayu Chen, et al., “Neuromark: An auto- mated and adaptive ica based pipeline to identify repro- ducible fmri markers of brain disorders,”NeuroImage: Clinical,...

2020
[12]

A review of feature reduction techniques in neuroimag- ing,

Benson Mwangi, Tian Siva Tian, and Jair C Soares, “A review of feature reduction techniques in neuroimag- ing,”Neuroinformatics, vol. 12, pp. 229–244, 2014

2014
[13]

Deep learn- ing in neuroimaging data analysis: Applications, chal- lenges, and solutions,

Lev Kiar Avber ˇsek and Grega Repov ˇs, “Deep learn- ing in neuroimaging data analysis: Applications, chal- lenges, and solutions,”Frontiers in neuroimaging, vol. 1, pp. 981642, 2022

2022
[14]

Generative ai for brain image computing and brain network computing: a review,

Changwei Gong, Changhong Jing, Xuhang Chen, Chi Man Pun, Guoli Huang, Ashirbani Saha, Martin Nieuwoudt, Han-Xiong Li, Yong Hu, and Shuqiang Wang, “Generative ai for brain image computing and brain network computing: a review,”Frontiers in Neu- roscience, vol. 17, pp. 1203104, 2023

2023
[15]

Comparing the latent space of generative models,

Andrea Asperti and Valerio Tonelli, “Comparing the latent space of generative models,”Neural Computing and Applications, vol. 35, no. 4, pp. 3155–3172, 2023

2023
[16]

The connectomics of brain disorders,

Alex Fornito, Andrew Zalesky, and Michael Breakspear, “The connectomics of brain disorders,”Nature Reviews Neuroscience, vol. 16, no. 3, pp. 159–172, 2015

2015
[17]

Multimodal fusion of brain imaging data: a key to finding the missing link (s) in complex mental illness,

Vince D Calhoun and Jing Sui, “Multimodal fusion of brain imaging data: a key to finding the missing link (s) in complex mental illness,”Biological psychiatry: cognitive neuroscience and neuroimaging, vol. 1, no. 3, pp. 230–244, 2016

2016
[18]

Graph neural networks in network neuro- science,

Alaa Bessadok, Mohamed Ali Mahjoub, and Islem Rekik, “Graph neural networks in network neuro- science,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 5, pp. 5833–5848, 2022

2022
[19]

Representation learning of resting state fmri with variational autoen- coder,

Jung-Hoon Kim, Yizhen Zhang, Kuan Han, Zheyu Wen, Minkyu Choi, and Zhongming Liu, “Representation learning of resting state fmri with variational autoen- coder,”NeuroImage, vol. 241, pp. 118423, 2021

2021
[20]

Applica- tions of generative adversarial networks in neuroimag- ing and clinical neuroscience,

Rongguang Wang, Vishnu Bashyam, Zhijian Yang, Fanyang Yu, Vasiliki Tassopoulou, Sai Spandana Chintapalli, Ioanna Skampardoni, Lasya P Sreepada, Dushyant Sahoo, Konstantina Nikita, et al., “Applica- tions of generative adversarial networks in neuroimag- ing and clinical neurosc...

2023
[21]

Transformer-based approaches for neu- roimaging: an in-depth review of their role in classi- fication and regression tasks,

Xinyu Zhu, Shen Sun, Lan Lin, Yutong Wu, and Xi- angge Ma, “Transformer-based approaches for neu- roimaging: an in-depth review of their role in classi- fication and regression tasks,”Reviews in the Neuro- sciences, vol. 36, no. 2, pp. 209–228, 2025

2025
[22]

Brain imaging generation with latent diffusion mod- els,

Walter HL Pinaya, Petru-Daniel Tudosiu, Jessica Daf- flon, Pedro F Da Costa, Virginia Fernandez, Parashkev Nachev, Sebastien Ourselin, and M Jorge Cardoso, “Brain imaging generation with latent diffusion mod- els,” inMICCAI workshop on deep generative models. Springer, 2022, p...

2022
[23]

How at- tentive are graph attention networks?,

Shaked Brody, Uri Alon, and Eran Yahav, “How at- tentive are graph attention networks?,”arXiv preprint arXiv:2105.14491, 2021

2021 arXiv
[24]

Variational mixture-of-experts autoencoders for multi-modal deep generative models,

Yuge Shi, Brooks Paige, Philip Torr, et al., “Variational mixture-of-experts autoencoders for multi-modal deep generative models,”Advances in neural information processing systems, vol. 32, 2019

2019
[25]

Film: Visual reasoning with a general conditioning layer,

Ethan Perez, Florian Strub, Harm De Vries, Vincent Du- moulin, and Aaron Courville, “Film: Visual reasoning with a general conditioning layer,” inProceedings of the AAAI conference on artificial intelligence, 2018, vol. 32

2018
[26]

Multimodal popula- tion brain imaging in the uk biobank prospective epi- demiological study,

Karla L Miller, Fidel Alfaro-Almagro, Neal K Bangerter, David L Thomas, Essa Yacoub, Junqian Xu, Andreas J Bartsch, Saad Jbabdi, Stamatios N Sotiropou- los, Jesper LR Andersson, et al., “Multimodal popula- tion brain imaging in the uk biobank prospective epi- demiological stud...

2016

Pith tools

Reviewed July 9, 2026 · model on record in the stance chip above.

[1] [1]

INTRODUCTION Understanding how brain structure and function relate to externally observed clinical and demographic variables has driven the development of neuroimaging studies leveraging magnetic resonance imaging (MRI) and modern machine learning methods [1, 2]. While one major line of studies focus on learning the associations for accurately predicting ...

work page Pith review arXiv 2026

[2] [2]

The fMRI modality is represented using sFNC matri- ces, while the sMRI modality is represented as a vector of GMV features

METHODS We present a framework for multimodal generative model- ing and fusion of neuroimaging data, where each subject is characterized by both structural and functional MRI informa- tion. The fMRI modality is represented using sFNC matri- ces, while the sMRI modality is represented as a vector of GMV features. Let(X f ,X s)denote paired samples where Xf...

work page

[3] [3]

v” for MLP-based (vMMV AE, vLDM, vDiT), “t

is employed to effectively capture higher-order depen- dencies and preserve network topology. The GATv2 encoder adaptively aggregates information from neighboring nodes through attention-based message passing, resulting in a la- tent representation that captures the relational structure of the functional brain networks. In contrast, the sMRI GMV features ...

work page

[4] [4]

Datasets and Evaluation Metrics This study used the fMRI and sMRI data from 10, 000 UK Biobank subjects (5376 males and 4624 females) [18]

EXPERIMENTS 3.1. Datasets and Evaluation Metrics This study used the fMRI and sMRI data from 10, 000 UK Biobank subjects (5376 males and 4624 females) [18]. Pre- Fig. 1. Overview of the latent generative models architectures based on the choice of encoders and latent space processes. Table 1. Hyperparameter configuration for baseline models Hyperparameter...

work page

[5] [5]

The sFNC features were computed as Pearson cor- relations between TCs of extracted brain components

to obtain subject-specific spatial maps and time courses (TCs). The sFNC features were computed as Pearson cor- relations between TCs of extracted brain components. We used the Neuromark fMRI 1.0 template, consisting of 53 data-driven brain components divided into 7 functional sub- domains: Subcortical (SC), Auditory (AUD), Sensorimotor (SM), Visual (VIS)...

work page

[6] [6]

Reconstruction quality assessment A comprehensive evaluation of reconstruction quality was conducted across 8 generative approaches in Table

RESULTS 4.1. Reconstruction quality assessment A comprehensive evaluation of reconstruction quality was conducted across 8 generative approaches in Table. 2, show- ing that the graph-based models substantially outperform their MLP-based and baseline counterparts. For fMRI, gM- MV AE, gLDM, and gDiT achieve the lowest MSE, superior Frobenius norms, highest...

work page 1965

[7] [7]

Comprehensive evaluation on a large dataset demonstrates that graph-based generative models substantially outperform vectorized approaches

CONCLUSION This paper presents graph-based generative frameworks for multimodal neuroimaging fusion. Comprehensive evaluation on a large dataset demonstrates that graph-based generative models substantially outperform vectorized approaches. The proposed gMMV AE efficiently reconstructs and generates both modalities with high fidelity, surpassing baseline ...

work page

[8] [8]

ACKNOWLEDGMENTS This work was supported by NIH grants 1R01AG090597, R01AG073949 awarded to Vince Calhoun

work page