Generation of 3D Brain MRI Using Auto-Encoding Generative Adversarial Networks

Chihye Han; Dae-shik Kim; Gihyun Kwon

REVIEW 4 major objections 4 minor 18 references

Generation of 3D Brain MRI Using Auto-Encoding Generative Adversarial Networks

T0 review · 4 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A 3D auto-encoding GAN can generate realistic, diverse whole-brain MRI volumes directly from random vectors, this paper argues.

desk verdict A plausible 3D alpha-GAN + WGAN-GP for brain MRI with good qualitative output, but the headline MMD numbers reduce to a mean-matching statistic and shouldn't be used as evidence. read the letter →

arxiv 1908.02498 v1 pith:VQJXND6S submitted 2019-08-07 eess.IV cs.CV

Gihyun Kwon , Chihye Han , Dae-shik Kim This is my paper · ORCID

classification eess.IVcs.CV

keywords 3DbrainMRIgenerationgenerativeadversarialnetworksalpha-GANWGAN-GPmodecollapsemedicalimagesynthesisdataaugmentationauto-encodingGAN

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a 3D auto-encoding GAN can generate realistic, diverse whole-brain MRI volumes directly from random vectors, using only a modest set of training images. It adapts the α-GAN architecture—an encoder plus a code discriminator on top of the usual generator and discriminator—and trains it with a Wasserstein GAN with gradient penalty loss. If the claim is right, medical imaging gains a data-augmentation route that does not require paired image-to-image translation or large multi-domain datasets. The paper reports that its model achieves the lowest distribution-distance score and diversity scores close to real data among the compared 3D baselines, and that the same setup generates tumor and stroke volumes as well.

What carries the argument

The central mechanism is the α-GAN architecture extended to three dimensions: a generator, an image discriminator, an encoder, and a code discriminator that plays an adversarial matching game between the encoder's latent codes and random prior vectors, trained with a Wasserstein GAN with gradient penalty loss plus an L1 reconstruction term. The code discriminator replaces explicit variational inference, so the latent prior is matched implicitly; the 1000-dimensional latent vector provides capacity for whole-brain variety. Each component targets one failure mode: the code discriminator prevents mode collapse, WGAN-GP stabilizes training, and the reconstruction loss counters blurriness.

What would settle it

Recompute the distribution comparison with a kernel two-sample test on the full 64×64×64 volumes, reporting confidence intervals for the difference between this model and the closest baseline; if the intervals include zero or favor the baseline, the claimed outperformance is not supported.

Watch

Extended reading notes

Core claim

The central claim is that random-vector generation of 3D brain MRI is feasible, and that the proposed architecture—α-GAN with a WGAN-GP objective and a 1000-dimensional latent vector—produces volumes whose distribution is closer to real data than 3D-WGAN-GP, 3D-VAE-GAN, or 3D-α-GAN with the standard GAN loss. In the paper's quantitative comparison, this model has the lowest MMD (0.072, against 0.075 for the closest baseline) and an MS-SSIM value (0.829) near the real-data value (0.846), while visual and PCA evidence indicate it avoids both mode collapse and blurriness. The paper also trains the same model on brain-tumor and stroke-lesion volumes, arguing that the mechanism transfers across healthy and diseased anatomy and across modalities.

Load-bearing premise

The claim rests on the assumption that the similarity scores used in Table 1—computed without confidence intervals—reliably capture how close the generated volumes are to real brain MRI; the winning margin over the nearest competitor is 0.003, which could be measurement noise.

Editorial extensions

If this is right

A model trained on a few hundred whole-brain volumes can produce new 3D volumes of normal, tumor, and stroke anatomy from random latent vectors.
Because generation does not rely on image-to-image translation, the method needs only one dataset of the target type, not paired or multi-domain data.
The design separates failure modes: the code discriminator targets mode collapse, the reconstruction loss targets blurriness, and WGAN-GP targets training instability.
Generated diseased volumes with lesions at varied positions suggest that synthetic examples could supplement scarce medical datasets for downstream analysis.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The reported MMD margin over the nearest baseline is 0.003 with no confidence interval, so a stricter kernel two-sample test could easily reverse the ordering; the outperformance claim should be read as provisional.
The same latent-space matching could support controlled interpolation between healthy and diseased volumes, but the paper does not test whether latent arithmetic yields meaningful anatomical axes.
Downstream validation—training a segmentation or classification network on real-plus-generated data and comparing with real-only training—would test the data-augmentation promise more directly than the similarity metrics used here.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this is the first 3D whole-brain MRI generation from random vectors that I know of, and the qualitative results are genuinely encouraging. But the quantitative claim of superiority rests on a mislabeled metric that doesn't measure what it claims.

What's new: the architecture is a direct 3D adaptation of alpha-GAN with WGAN-GP loss. The components are published, but the 3D implementation, the empirical latent-size finding, and the application to tumor and stroke volumes are new. Code is linked, datasets are external, and the generated slices look plausible, with visible diversity. The authors deserve credit for getting 3D generation to work at all.

The soft spot is in Table 1. The quantity labeled MMD is not Gretton et al.'s MMD. With the formula given — 1/B^2 times summed pairwise linear dot products — you get exactly the squared Euclidean distance between the batch means. That's a first-moment statistic. It says nothing about mode collapse, variance, or anatomical feature matching. A generator that always outputs the average brain would score 0. So the 0.072 vs 0.075 win over 3D-VAE-GAN is a difference in mean-image matching, not distributional fidelity, and there are no error bars or significance tests anyway. The MS-SSIM numbers are more informative for diversity, but they too lack error bars and the 3D extension is underspecified. The PCA and slice visualizations are the most convincing evidence, and they do support the authors' model over the baselines they compared.

Other issues are minor: the disease experiments are purely qualitative, lambda1/lambda2 are fixed without sensitivity analysis, and 'small training data' means a few hundred subjects, not dozens. None of this undermines the core method; it undermines the evidence as presented.

This paper is for people working on medical image synthesis and augmentation. Worth reading for the architecture and qualitative results; worth citing only after the evaluation is fixed. I'd send it to peer review — the work is reproducible, the method is sensible, and referees can push for a corrected metric, proper error bars, and ideally a downstream segmentation or classification task. As it stands, the hidden gem is a good lesson in why you should check what a metric actually computes.

Referee Report

4 major / 4 minor

Summary. The paper proposes a 3D auto-encoding generative adversarial network, built on alpha-GAN with WGAN-GP losses, that generates 64x64x64 brain MRI volumes from random latent vectors. The model uses an encoder, generator, discriminator, and code discriminator, with a reconstruction loss and gradient penalties. The authors train on 991 ADNI normal T1 volumes and on BRATS 2018 tumor and ATLAS stroke datasets, comparing against 3D-WGAN-GP, 3D-VAE-GAN, and 3D-alpha-GAN baselines. Quantitative comparisons are made with a batch-wise MMD score and MS-SSIM, and qualitative comparisons use slice visualizations and PCA. The central claim is that the proposed model outperforms the baselines in both quantitative and qualitative measurements and can generate realistic, diverse 3D brain MRI of normal and diseased brains.

Significance. If the quantitative claims held, this would be a useful practical contribution to 3D medical image generation from random vectors, an area where most prior work is 2D or relies on image-to-image translation. The paper is clearly written, provides architectural details, and releases code, and the qualitative full-volume slices are a strength. However, the core quantitative evidence is currently too weak to support the claimed superiority: the MMD metric as implemented is not a distributional distance, no uncertainty or significance testing is reported, and the disease experiments are only qualitative. The contribution is plausible and potentially publishable, but the load-bearing quantitative claims need to be reworked.

major comments (4)

[Section 4, Quantitative Results (MMD formula)] The batch-wise MMD formula, written as 1/B^2 times the sum of g.g^T + r.r^T - 2g.r^T, is algebraically equivalent to the squared Euclidean distance between the batch means, ||mean(g) - mean(r)||^2, when g and r are flattened batches and no nonlinear kernel is applied. This is a first-moment statistic, not a maximum mean discrepancy over a characteristic kernel, so it cannot detect differences in variance, mode collapse, or higher-order anatomical feature mismatches. A degenerate generator that always outputs an average brain could score well. The reported margin of 0.072 versus 0.075 (x10^-4) between Ours-z1000 and 3D-VAE-GAN is therefore a difference in mean-image matching, not a demonstrated distributional improvement. The authors should replace this with a proper kernel MMD (e.g., Gaussian RBF) or another distributional distance, and report the kernel and its parameters.
[Table 1 and Section 4, Quantitative Results] No variances, confidence intervals, or significance tests are reported for any metric. The MMD values are averaged over 100 batch tests and MS-SSIM over 1000 sample pairs, but without error bars the 0.003 (x10^-4) MMD gap over 3D-VAE-GAN and the MS-SSIM gap of 0.829 versus real data 0.846 cannot be interpreted as meaningful. The authors should report standard deviations or confidence intervals and, ideally, a paired statistical test across multiple training runs or bootstrap resamples.
[Section 4, Quantitative Results (MS-SSIM)] MS-SSIM is the only diversity-sensitive quantitative metric in Table 1, but the paper does not specify how this 2D perceptual metric was extended to 64^3 volumes, nor how the 1000 sample pairs were selected from the generated and real sets. Without this information, the comparison of 0.829 (Ours-z1000) with 0.846 (Real) is not reproducible or interpretable, and the diversity claim rests on an underspecified procedure.
[Section 4, Diseased MRI generation] The brain tumor and stroke experiments are evaluated only qualitatively, with no quantitative assessment of lesion plausibility, lesion location diversity, or distributional similarity to the diseased datasets. The abstract and conclusion claim wide applicability and accurate reflection of disease attributes, but these claims are not supported by any measured outcome. A quantitative evaluation, such as lesion overlap with segmentations, disease-specific FID/MMD, or a downstream classification task, is needed to substantiate the versatility claim.

minor comments (4)

[Throughout] The model is referred to variously as 'our model', '3D-alpha-WGAN-GP' in Figure 3, and 'Ours' in Table 1; please use one consistent name throughout the manuscript.
[References] Several references are incomplete: 'P Kingma, D., Welling, M.' should be 'Kingma, D.P., Welling, M.', and 'Gulrajani, et al.' should list the full author set. Please update the reference list.
[Figure 2 and Figure 4] The center-cut slices are informative, but the figure captions do not state which anatomical plane is shown; specifying the plane (axial, coronal, or sagittal) in the main-text figures would make the qualitative comparison easier to follow.
[Section 2, Loss Function] The text says 'For the parameters lambda1 and lambda2, we use the fixed value of 10 in both cases', but it may be worth stating explicitly that lambda1 multiplies both gradient penalty terms L_GP-D and L_GP-C, since the equation numbering appears only for the main losses.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper is an empirical GAN construction evaluated on external data, with no load-bearing self-citation or fitted-input-as-prediction.

full rationale

The paper makes no claimed theoretical derivation; it adapts the external alpha-GAN framework and WGAN-GP loss to 3D brain MRI generation and evaluates the resulting model on held-out external datasets (ADNI, BRATS, ATLAS). No parameter is fitted to the reported evaluation metric and then presented as a prediction. The only self-citation is Han et al. [8], and it is used as background for prior 2D slice generation, not as load-bearing support for the proposed model's validity or uniqueness. The quantitative claim relies on MMD and MS-SSIM values computed from real and generated samples; the concern that the batch-wise linear-kernel MMD reduces to a mean-difference statistic is a legitimate measurement-validity issue for the strength of the outperformance claim, but it is not a circularity issue, because the metric is not an input to the model's construction or loss function. Thus the derivation chain is self-contained relative to the paper's empirical claims, and no circular step can be exhibited.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The model is an empirical combination of published components (alpha-GAN, WGAN-GP). The main hand-chosen numbers are the latent vector size and the two loss weights. The load-bearing assumptions are about the validity of the evaluation metrics and the unstated latent prior.

free parameters (3)

latent vector size = 1000
Chosen after empirical verification (Section 4, Experiment Details); the paper states 100 was too small and 2048 produced more out-of-distribution samples. This is a hand-tuned architecture choice central to the diversity claim.
lambda1 = 10
Fixed weight for gradient penalty terms in LD and LC (Equation 1 and 3); taken from WGAN-GP convention, not fitted to this dataset.
lambda2 = 10
Fixed weight for reconstruction L1 loss in LG (Equation 2); chosen by hand, not optimized.

assumptions (5)

domain assumption alpha-GAN's code discriminator implicitly matches the encoder posterior to the latent prior
Section 2 Model Architecture relies on this to avoid explicit variational inference; if false, the model may not learn a meaningful latent space.
domain assumption WGAN-GP gradient penalty enforces the 1-Lipschitz condition and stabilizes 3D training
Section 2 Loss Function; the effectiveness of the loss depends on this theoretical property holding in practice.
domain assumption The latent prior P(z) is a standard Gaussian
The paper never specifies the prior distribution for zr, but alpha-GAN assumes a standard normal; sampling uses random vectors in that convention.
domain assumption MMD with a linear kernel and MS-SSIM are valid proxies for realism and diversity
Section 4 Quantitative Results uses these metrics without validating them for 3D brain anatomy; the MMD kernel is not specified.
domain assumption The ADNI, BRATS and ATLAS datasets are representative of their respective populations
Section 3 uses these datasets as ground truth; if labels or subjects are not representative, the generated 'normal' and 'diseased' images inherit the bias.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Generation of 3D Brain MRI Using Auto-Encoding Generative Adversarial Networks." pith.science (2026). https://pith.science/paper/VQJXND6S

@misc{pith2026190802498,
  author       = {Pith},
  title        = {Pith review of: Generation of 3D Brain MRI Using Auto-Encoding Generative Adversarial Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VQJXND6S}},
  note         = {Machine review of arXiv:1908.02498}
}

read the original abstract

As deep learning is showing unprecedented success in medical image analysis tasks, the lack of sufficient medical data is emerging as a critical problem. While recent attempts to solve the limited data problem using Generative Adversarial Networks (GAN) have been successful in generating realistic images with diversity, most of them are based on image-to-image translation and thus require extensive datasets from different domains. Here, we propose a novel model that can successfully generate 3D brain MRI data from random vectors by learning the data distribution. Our 3D GAN model solves both image blurriness and mode collapse problems by leveraging alpha-GAN that combines the advantages of Variational Auto-Encoder (VAE) and GAN with an additional code discriminator network. We also use the Wasserstein GAN with Gradient Penalty (WGAN-GP) loss to lower the training instability. To demonstrate the effectiveness of our model, we generate new images of normal brain MRI and show that our model outperforms baseline models in both quantitative and qualitative measurements. We also train the model to synthesize brain disorder MRI data to demonstrate the wide applicability of our model. Our results suggest that the proposed model can successfully generate various types and modalities of 3D whole brain volumes from a small set of training data.

Figures

Figures reproduced from arXiv: 1908.02498 by the authors.

**Figure 1.** Detailed architecture of the proposed model. n, k, s, and p refer to the number of the channels, the kernel size, the stride size, and the padding size, respectively. xrand is the generator output from random vectors zr and xrec is the output from encoded vectors ze. fake and zr as real, the encoder and the code discriminator play an adversarial game. When C is not able to discriminate both, the posterior and the pr… view at source ↗

**Figure 2.** Real and generated samples of normal brain MRI [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

**Figure 3.** PCA results of generated samples. (a) Samples from real data, 3D-WGANGP, 3D-VAE-GAN, Ours (3D-α-WGAN-GP). (b) Samples from real data, 3D-α-GAN, Ours (3D-α-WGAN-GP). (c) Samples with changing latent vector sizes [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Figures from the paper (7 more)

**Figure 4.** Figure 4: Real and generated samples of diseased brain MRI. distribution range, but it produces a limited variety of images. With the baseline 3D-α-GAN model, the mode collapse problem is solved, but the distribution is still not closely matched to the real data. Compared to the…

**Figure 1.** Figure 1: Slices of normal brain 3D samples from (a) real data, (b) Ours, (c) 3D [PITH_FULL_IMAGE:figures/full_fig_p011_1.png]

**Figure 2.** Figure 2: Generated center-cut views of diseased brain samples from baseline mod [PITH_FULL_IMAGE:figures/full_fig_p012_2.png]

**Figure 3.** Figure 3: PCA results of generated tumor brain samples. (a) Tumor-FLAIR Sam [PITH_FULL_IMAGE:figures/full_fig_p013_3.png]

**Figure 4.** Figure 4: Slices of tumor-FLAIR brain 3D samples from real data and our model [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]

**Figure 5.** Figure 5: Slices of tumor-T2 brain 3D samples from real data and our model along [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]

**Figure 6.** Figure 6: Slices of stroke-T1w brain 3D samples from real data and our model along [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

18 extracted references · 15 canonical work pages

[1]

In: International Conference on Machine Learning

Arjovsky, M., Chintala, S., Bottou, L.: Wasserstein generative adversarial networks. In: International Conference on Machine Learning. pp. 214–223 (2017)

2017
[2]

Scientiﬁc Data 4, 170117 (2017)

Bakas, S., et al.: Advancing the cancer genome atlas glioma mri collections with expert segmentation labels and radiomic features. Scientiﬁc Data 4, 170117 (2017)

work page 2017
[3]

Pro- ceedings of SPIE–the International Society for Optical Engineering 10574 (2018)

Bermudez, C., et al.: Learning implicit brain mri manifolds with deep learning. Pro- ceedings of SPIE–the International Society for Optical Engineering 10574 (2018)

work page 2018
[4]

IEEE Transactions on Medical Imaging pp

Dar, S.U., et al.: Image synthesis in multi-contrast MRI with conditional generative adversarial networks. IEEE Transactions on Medical Imaging pp. 1–1 (2019)

work page 2019
[5]

In: Advanced in Neural Infor- mation Processing Systems

Goodfellow, I.J., et al.: Generative adversarial nets. In: Advanced in Neural Infor- mation Processing Systems. pp. 2672–2680 (2014)

work page 2014
[6]

Journal of Machine Learning Research 13, 723–773 (2012)

Gretton, A., et al.: A kernel two-sample test. Journal of Machine Learning Research 13, 723–773 (2012)

work page 2012
[7]

In: Advances in Neural Information Processing Systems, pp

Gulrajani, et al.: Improved training of wasserstein gans. In: Advances in Neural Information Processing Systems, pp. 5767–5777 (2017)

work page 2017
[8]

In: 2018 IEEE 15th International Symposium on Biomedical Imaging

Han, C., et al.: Gan-based synthetic brain mr image generation. In: 2018 IEEE 15th International Symposium on Biomedical Imaging. pp. 734–738 (2018) Generation of 3D Brain MRI Using Auto-Encoding GAN 9

work page 2018

Show all 18 references

[9]

In: International Conference on Machine Learning

Larsen, A.B.L., et al.: Autoencoding beyond pixels using a learned similarity met- ric. In: International Conference on Machine Learning. pp. 1558–1566 (2016)

2016
[10]

Scientiﬁc Data 5, 180011 (2018)

Liew, S.L., et al.: A large, open source dataset of stroke anatomical brain images and manual lesion segmentations. Scientiﬁc Data 5, 180011 (2018)

2018
[11]

IEEE Transactions on Medical Imaging 34(10), 1993–2024 (2015)

Menze, B.H., et al.: The multimodal brain tumor image segmentation benchmark (brats). IEEE Transactions on Medical Imaging 34(10), 1993–2024 (2015)

2015
[12]

Distill (2016)

Odena, A., Dumoulin, V., Olah, C.: Deconvolution and checkerboard artifacts. Distill (2016). https://doi.org/10.23915/distill.00003

2016 doi
[13]

In: International Conference on Machine Learning

Odena, A., Olah, C., Shlens, J.: Conditional image synthesis with auxiliary classiﬁer GANs. In: International Conference on Machine Learning. pp. 2642–2651 (2017)

2017
[14]

In: International Conference on Learning Representations (2014)

P Kingma, D., Welling, M.: Auto-encoding variational bayes. In: International Conference on Learning Representations (2014)

2014
[15]

IEEE Journal of Biomedical and Health Informatics 21(1), 4–21 (2017)

Ravi, D., et al.: Deep learning for health informatics. IEEE Journal of Biomedical and Health Informatics 21(1), 4–21 (2017)

2017
[16]

arXiv preprint arXiv:1706.04987 (2017)

Rosca, M., et al.: Variational Approaches for Auto-Encoding Generative Adversar- ial Networks. arXiv preprint arXiv:1706.04987 (2017)

2017 arXiv
[17]

In: Simulation and Synthesis in Medical Imaging, pp

Shin, H.C., et al.: Medical image synthesis for data augmentation and anonymiza- tion using generative adversarial networks. In: Simulation and Synthesis in Medical Imaging, pp. 1–11 (2018)

2018
[18]

In: 2018 IEEE 15th International Symposium on Biomedical Imaging

Yu, B., et al.: 3d cgan based cross-modality mr image synthesis for brain tumor seg- mentation. In: 2018 IEEE 15th International Symposium on Biomedical Imaging. pp. 626–630 (2018) 1 Supplementary Material (a) Real (b) Ours (c) 3D- α-GAN 2 (d) 3D-V AE-GAN (e) 3D-WGAN-GP Fig. 1...

2018

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

In: International Conference on Machine Learning

Arjovsky, M., Chintala, S., Bottou, L.: Wasserstein generative adversarial networks. In: International Conference on Machine Learning. pp. 214–223 (2017)

2017

[2] [2]

Scientiﬁc Data 4, 170117 (2017)

Bakas, S., et al.: Advancing the cancer genome atlas glioma mri collections with expert segmentation labels and radiomic features. Scientiﬁc Data 4, 170117 (2017)

work page 2017

[3] [3]

Pro- ceedings of SPIE–the International Society for Optical Engineering 10574 (2018)

Bermudez, C., et al.: Learning implicit brain mri manifolds with deep learning. Pro- ceedings of SPIE–the International Society for Optical Engineering 10574 (2018)

work page 2018

[4] [4]

IEEE Transactions on Medical Imaging pp

Dar, S.U., et al.: Image synthesis in multi-contrast MRI with conditional generative adversarial networks. IEEE Transactions on Medical Imaging pp. 1–1 (2019)

work page 2019

[5] [5]

In: Advanced in Neural Infor- mation Processing Systems

Goodfellow, I.J., et al.: Generative adversarial nets. In: Advanced in Neural Infor- mation Processing Systems. pp. 2672–2680 (2014)

work page 2014

[6] [6]

Journal of Machine Learning Research 13, 723–773 (2012)

Gretton, A., et al.: A kernel two-sample test. Journal of Machine Learning Research 13, 723–773 (2012)

work page 2012

[7] [7]

In: Advances in Neural Information Processing Systems, pp

Gulrajani, et al.: Improved training of wasserstein gans. In: Advances in Neural Information Processing Systems, pp. 5767–5777 (2017)

work page 2017

[8] [8]

In: 2018 IEEE 15th International Symposium on Biomedical Imaging

Han, C., et al.: Gan-based synthetic brain mr image generation. In: 2018 IEEE 15th International Symposium on Biomedical Imaging. pp. 734–738 (2018) Generation of 3D Brain MRI Using Auto-Encoding GAN 9

work page 2018