Enhanced generative adversarial network for 3D brain MRI super-resolution

James Gee; Jianbo Shi; Jiancong Wang; Yifan Wu; Yuhua Chen

arxiv: 1907.04835 · v2 · pith:MM3VXYMWnew · submitted 2019-07-10 · 📡 eess.IV · cs.CV

Enhanced generative adversarial network for 3D brain MRI super-resolution

Jiancong Wang , Yuhua Chen , Yifan Wu , Jianbo Shi , James Gee This is my paper

Pith reviewed 2026-05-24 23:13 UTC · model grok-4.3

classification 📡 eess.IV cs.CV

keywords GANsuper-resolutionMRIbrain imaging3D SISRresidual dense blockanatomical fidelitypatch discriminator

0 comments

The pith

A memory-efficient residual-in-residual dense block generator enhances GAN performance for 3D brain MRI super-resolution.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper develops enhancements to generative adversarial networks for single-image super-resolution of 3D brain MRI volumes. It introduces a residual-in-residual dense block generator that improves standard image quality metrics while remaining memory efficient. A patch-based discriminator is added to improve convergence and recover fine brain textures. Results are evaluated for anatomical fidelity by passing them through a pre-trained brain parcellation network. A simple balancing step then trades off pixel accuracy against texture detail in the final images.

Core claim

The authors claim that their residual-in-residual dense block (RRDG) generator, paired with a patch GAN discriminator and evaluated through a pre-trained parcellation network, delivers state-of-the-art results on PSNR, SSIM, and NRMSE for 3D single-image super-resolution of brain MRI while remaining memory efficient.

What carries the argument

residual-in-residual dense block (RRDG) generator, which stacks dense blocks inside residual connections to reuse features efficiently during 3D volume reconstruction

Load-bearing premise

The pre-trained brain parcellation network provides an unbiased and accurate measure of anatomical fidelity that correlates with true clinical or structural quality of the super-resolved images.

What would settle it

If super-resolved volumes that score well on the parcellation network produce worse results than the original low-resolution inputs on a downstream clinical task such as automated segmentation accuracy or lesion detection, the anatomical fidelity claim would be falsified.

Figures

Figures reproduced from arXiv: 1907.04835 by James Gee, Jianbo Shi, Jiancong Wang, Yifan Wu, Yuhua Chen.

**Figure 1.** Figure 1: Model training and blending pipeline. 2.1 Memory efficient residual-in-residual dense block generator (RRDG) In SISR task with GAN framework, the network architecture of the generator is of paramount importance of generated image quality. Ledig et al. [9] introduced [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

**Figure 2.** Figure 2: Architecture of the proposed RRDG network and RRDB. Like SRResNet [8], RRDG consists of a global residual connection and consecutive basic blocks, except basic resblocks are replaced by RRDB. Within each RRDB, three consecutive dense blocks are chained by a weighted residual connection and a block level residual connection. 2.2 Patch GAN discriminator Our discriminator follows the work in [2, 9], as shown… view at source ↗

**Figure 3.** Figure 3: Architecture of the discriminator, which is a VGG-style feed forward network [15], consisting of 2 strided convolution blocks for down-sampling followed by plain convolution-layer norm-leakyReLU layers. The outputs are globally spatial pooled to produce a single number. two strided convolutions, we reduced the receptive field of our discriminator and facilitate discrimination of local texture that in turn… view at source ↗

**Figure 4.** Figure 4: Left to right: Super-resolution output from FSRCNN, SRResNet, mDCSRN, RRDB, RRDB with patch GAN training, and ground truth. Bottom row is magnified portion of the same image region across the different SISR outputs. Our proposed RRDG and its patch GAN augmented variant were evaluated against state-of-the-art FSRCNN, SRResNet, and mDCSRN models for SISR reconstruction. The FSRCNN and SRResNet are adapted to… view at source ↗

**Figure 5.** Figure 5: Sample image appearance as a function of blending between GAN oriented model (α = 1) and PSNR oriented model (α = 0), compared with ground truth. 4 Discussion In this work, we investigated enhancements to CNN-based solutions to 3D brain MRI super-resolution. The RRDG was shown to exhibit superior performance against the state-of-the-art, and amenable to memory optimization to make possible efficient train… view at source ↗

read the original abstract

Single image super-resolution (SISR) reconstruction for magnetic resonance imaging (MRI) has generated significant interest because of its potential to not only speed up imaging but to improve quantitative processing and analysis of available image data. Generative Adversarial Networks (GAN) have proven to perform well in recovering image texture detail, and many variants have therefore been proposed for SISR. In this work, we develop an enhancement to tackle GAN-based 3D SISR by introducing a new residual-in-residual dense block (RRDG) generator that is both memory efficient and achieves state-of-the-art performance in terms of PSNR (Peak Signal to Noise Ratio), SSIM (Structural Similarity) and NRMSE (Normalized Root Mean Squared Error) metrics. We also introduce a patch GAN discriminator with improved convergence behavior to better model brain image texture. We proposed a novel the anatomical fidelity evaluation of the results using a pre-trained brain parcellation network. Finally, these developments are combined through a simple and efficient method to balance etween image and texture quality in the final output.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

The paper adds a residual-in-residual dense generator block and a parcellation-based anatomical score to 3D MRI GAN super-resolution, but supplies no evidence the new score tracks real structural quality.

read the letter

The core move is introducing an RRDG generator block that they say is memory-efficient for 3D volumes, plus a patch discriminator and a pre-trained parcellation network to score anatomical fidelity beyond standard PSNR/SSIM/NRMSE. They also mention a simple balance factor between image and texture terms. That combination is the actual new material relative to prior GAN SISR work on MRI. The memory-efficiency angle and the texture-focused discriminator are reasonable engineering choices for brain volumes where detail matters for downstream analysis. The parcellation idea is a fresh way to try to capture anatomical consistency instead of just pixel-level metrics. On the positive side, the abstract frames the problem clearly around faster MRI acquisition and the need for texture that still respects brain structure. The balance method sounds straightforward to implement. The soft spots sit mostly with the evaluation. The abstract claims state-of-the-art numbers but shows none of the baselines, ablations, or dataset details that would let a reader judge whether the RRDG block actually drives the gains. More critically, nothing indicates that the parcellation network's scores correlate with expert labels, clinical utility, or even simple checks like edge preservation in known structures. Without that link, the novel fidelity metric remains an untested assumption rather than a demonstrated improvement. The free balance parameter also needs sensitivity results to show it is not just another hyperparameter that requires heavy tuning. This work is aimed at groups already running GAN experiments on medical volumes who might want to try the RRDG block or the parcellation scoring idea. A reader looking for a fully validated new method or strong empirical comparisons will come away wanting more data. It is worth sending to peer review so the authors can add the missing ablations, baseline tables, and validation of the anatomical metric against independent measures.

Referee Report

2 major / 2 minor

Summary. The paper claims to enhance GAN-based 3D single-image super-resolution for brain MRI via a new residual-in-residual dense block (RRDG) generator that is memory-efficient and achieves state-of-the-art PSNR, SSIM and NRMSE; a patch-GAN discriminator with improved convergence for texture modeling; a novel anatomical-fidelity metric obtained from a pre-trained brain parcellation network; and a simple balancing procedure between image and texture quality.

Significance. If the empirical claims and the new evaluation protocol are substantiated, the work would supply a practical, memory-efficient generator architecture together with an evaluation approach that directly targets anatomical plausibility rather than relying solely on pixel-wise or perceptual metrics. Such a combination could be useful for downstream quantitative MRI analysis where structural fidelity matters.

major comments (2)

[Abstract and §4] Abstract and §4 (evaluation): the central claim that the pre-trained parcellation network supplies an unbiased anatomical-fidelity score rests on an unverified assumption; no correlation analysis against expert labels, no sensitivity study, and no cross-check against alternative segmenters are reported, rendering the novel metric load-bearing yet unsupported.
[Abstract] Abstract: the assertion of state-of-the-art performance on PSNR/SSIM/NRMSE is presented without any tabulated comparison to prior 3D GAN or CNN baselines, without ablation of the RRDG block, and without dataset size or train/test split details, so the performance claim cannot be assessed from the supplied information.

minor comments (2)

[Abstract] Abstract contains a repeated definite article and a typo: 'We proposed a novel the anatomical fidelity' and 'balance etween image and texture quality'.
Notation for the balance factor between image and texture quality is introduced only descriptively; an explicit equation or hyper-parameter table would improve reproducibility.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments. We respond point-by-point below and indicate planned revisions to address the concerns raised.

read point-by-point responses

Referee: [Abstract and §4] Abstract and §4 (evaluation): the central claim that the pre-trained parcellation network supplies an unbiased anatomical-fidelity score rests on an unverified assumption; no correlation analysis against expert labels, no sensitivity study, and no cross-check against alternative segmenters are reported, rendering the novel metric load-bearing yet unsupported.

Authors: We agree that further validation would strengthen the anatomical fidelity metric. The pre-trained parcellation network serves as a standard proxy for structural accuracy in neuroimaging. In the revision we will add to §4 a correlation analysis against expert labels on a held-out subset together with a sensitivity study using an alternative segmenter. This directly addresses the load-bearing nature of the metric. revision: yes
Referee: [Abstract] Abstract: the assertion of state-of-the-art performance on PSNR/SSIM/NRMSE is presented without any tabulated comparison to prior 3D GAN or CNN baselines, without ablation of the RRDG block, and without dataset size or train/test split details, so the performance claim cannot be assessed from the supplied information.

Authors: The full manuscript already contains tabulated comparisons to prior 3D GAN and CNN baselines, ablation results for the RRDG block, and explicit dataset size plus train/test split information in Sections 3 and 4. The abstract is a high-level summary. We will revise the abstract to reference these comparative results and include the key dataset details for improved self-containment. revision: partial

Circularity Check

0 steps flagged

No significant circularity; claims rest on empirical metrics

full rationale

The paper introduces architectural modifications (RRDG generator, patch GAN discriminator) and a new evaluation method (pre-trained parcellation network) for 3D MRI SISR, then reports performance via standard metrics (PSNR, SSIM, NRMSE). No derivation chain, equations, or predictions are presented that reduce by construction to fitted inputs, self-definitions, or self-citation load-bearing steps. The evaluation proposal is empirical and does not invoke uniqueness theorems or ansatzes from prior self-work. This is a standard empirical ML paper whose central claims are falsifiable via external benchmarks.

Axiom & Free-Parameter Ledger

1 free parameters · 2 axioms · 0 invented entities

The work relies on standard deep-learning training assumptions and the reliability of an external pre-trained model; one tunable balance parameter is implied.

free parameters (1)

balance factor between image and texture quality
The abstract describes a simple method to balance image and texture quality in the final output, implying a tunable hyperparameter.

axioms (2)

domain assumption GAN training converges to a useful equilibrium for texture modeling in medical images.
Implicit foundation of all GAN-based super-resolution methods.
domain assumption A pre-trained brain parcellation network yields reliable anatomical labels for fidelity assessment.
Central premise of the novel evaluation component.

pith-pipeline@v0.9.0 · 5723 in / 1236 out tokens · 31034 ms · 2026-05-24T23:13:01.652744+00:00 · methodology

discussion (0)

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

IndisputableMonolith/Foundation/RealityFromDistinction.lean reality_from_one_distinction unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

new residual-in-residual dense block (RRDG) generator... patch GAN discriminator... anatomical fidelity evaluation... pre-trained brain parcellation network
IndisputableMonolith/Cost/FunctionalEquation.lean washburn_uniqueness_aczel unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

HighRes3DNet... dice scores on 160 tissue types

What do these tags mean?

matches: The paper's claim is directly supported by a theorem in the formal canon.
supports: The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends: The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses: The paper appears to rely on the theorem as machinery.
contradicts: The paper's claim conflicts with a theorem or certificate in the canon.
unclear: Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

20 extracted references · 20 canonical work pages · 3 internal anchors

[1]

Computer Vision and Image Understanding 179, 41–65 (2019)

Borji, A.: Pros and cons of gan evaluation measures. Computer Vision and Image Understanding 179, 41–65 (2019)

work page 2019
[2]

In: MICCAI (2018)

Chen, Y., Shi, F., Christodoulou, A.G., Xie, Y., Zhou, Z., Li, D.: Eﬃcient and ac- curate MRI super-resolution using a generative adversarial network and 3D multi- Level densely connected network. In: MICCAI (2018)

work page 2018
[3]

In: ECCV (2016)

Dong, C., Loy, C.C., Tang, X.: Accelerating the super-resolution convolutional neural network. In: ECCV (2016)

work page 2016
[4]

In: NeurIPS (2017)

Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., Courville, A.C.: Improved training of wasserstein gans. In: NeurIPS (2017)

work page 2017
[5]

In: CVPR (2016)

He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR (2016)

work page 2016
[6]

In: CVPR (2017)

Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: CVPR (2017)

work page 2017
[7]

In: European Conference on Computer Vision

Ignatov, A., Timofte, R., Van Vu, T., Luu, T.M., Pham, T.X., Van Nguyen, C., Kim, Y., Choi, J.S., Kim, M., Huang, J., et al.: Pirm challenge on perceptual im- age enhancement on smartphones: report. In: European Conference on Computer Vision. pp. 315–333. Springer (2018)

work page 2018
[8]

In: CVPR (2017)

Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with condi- tional adversarial networks. In: CVPR (2017)

work page 2017
[9]

In: CVPR (2017)

Ledig, C., Theis, L., Husz´ ar, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z., et al.: Photo-realistic single image super- resolution using a generative adversarial network. In: CVPR (2017)

work page 2017
[10]

In: IPMI (2017)

Li, W., Wang, G., Fidon, L., Ourselin, S., Cardoso, M.J., Vercauteren, T.: On the compactness, eﬃciency, and representation of 3D convolutional networks: brain parcellation as a pretext task. In: IPMI (2017)

work page 2017
[11]

In: ISBI (2017)

Pham, C.H., Ducournau, A., Fablet, R., Rousseau, F.: Brain MRI super-resolution using deep 3D convolutional networks. In: ISBI (2017)

work page 2017
[12]

Memory-Efficient Implementation of DenseNets

Pleiss, G., Chen, D., Huang, G., Li, T., van der Maaten, L., Weinberger, K.Q.: Memory-eﬃcient implementation of densenets. arXiv:1707.06990 (2017)

work page internal anchor Pith review Pith/arXiv arXiv 2017
[13]

Plenge, E., Poot, D.H., Bernsen, M., Kotek, G., Houston, G., Wielopolski, P., van der Weerd, L., Niessen, W.J., Meijering, E.: Super-resolution methods in MRI: Can they improve the trade-oﬀ between resolution, signal-to-noise ratio, and ac- quisition time? Magnetic resonance in medicine 68(6), 1983–1993 (2012)

work page 1983
[14]

Brain MRI super-resolution using 3D generative adversarial networks

S´ anchez, I., Vilaplana, V.: Brain MRI super-resolution using 3D generative adver- sarial networks. arXiv preprint arXiv:1812.11440 (2018)

work page internal anchor Pith review Pith/arXiv arXiv 2018
[15]

Very Deep Convolutional Networks for Large-Scale Image Recognition

Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556 (2014) EGN for brain MRI SR 9

work page internal anchor Pith review Pith/arXiv arXiv 2014
[16]

IEEE transactions on medical imag- ing 29(6), 1310 (2010)

Tustison, N.J., Avants, B.B., Cook, P.A., Zheng, Y., Egan, A., Yushkevich, P.A., Gee, J.C.: N4itk: improved n3 bias correction. IEEE transactions on medical imag- ing 29(6), 1310 (2010)

work page 2010
[17]

In: CVPR (2018)

Wang, X., Yu, K., Dong, C., Change Loy, C.: Recovering realistic texture in image super-resolution by deep spatial feature transform. In: CVPR (2018)

work page 2018
[18]

In: ECCV (2018)

Wang, X., Yu, K., Wu, S., Gu, J., Liu, Y., Dong, C., Qiao, Y., Loy, C.C.: Esrgan: Enhanced super-resolution generative adversarial networks. In: ECCV (2018)

work page 2018
[19]

In: CVPR (2018)

Zhang, Y., Tian, Y., Kong, Y., Zhong, B., Fu, Y.: Residual dense network for image super-resolution. In: CVPR (2018)

work page 2018
[20]

In: ISBI (2018)

Zhao, C., Carass, A., Dewey, B.E., Prince, J.L.: Self super-resolution for magnetic resonance images using deep networks. In: ISBI (2018)

work page 2018

[1] [1]

Computer Vision and Image Understanding 179, 41–65 (2019)

Borji, A.: Pros and cons of gan evaluation measures. Computer Vision and Image Understanding 179, 41–65 (2019)

work page 2019

[2] [2]

In: MICCAI (2018)

Chen, Y., Shi, F., Christodoulou, A.G., Xie, Y., Zhou, Z., Li, D.: Eﬃcient and ac- curate MRI super-resolution using a generative adversarial network and 3D multi- Level densely connected network. In: MICCAI (2018)

work page 2018

[3] [3]

In: ECCV (2016)

Dong, C., Loy, C.C., Tang, X.: Accelerating the super-resolution convolutional neural network. In: ECCV (2016)

work page 2016

[4] [4]

In: NeurIPS (2017)

Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., Courville, A.C.: Improved training of wasserstein gans. In: NeurIPS (2017)

work page 2017

[5] [5]

In: CVPR (2016)

He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR (2016)

work page 2016

[6] [6]

In: CVPR (2017)

Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: CVPR (2017)

work page 2017

[7] [7]

In: European Conference on Computer Vision

Ignatov, A., Timofte, R., Van Vu, T., Luu, T.M., Pham, T.X., Van Nguyen, C., Kim, Y., Choi, J.S., Kim, M., Huang, J., et al.: Pirm challenge on perceptual im- age enhancement on smartphones: report. In: European Conference on Computer Vision. pp. 315–333. Springer (2018)

work page 2018

[8] [8]

In: CVPR (2017)

Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with condi- tional adversarial networks. In: CVPR (2017)

work page 2017

[9] [9]

In: CVPR (2017)

Ledig, C., Theis, L., Husz´ ar, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z., et al.: Photo-realistic single image super- resolution using a generative adversarial network. In: CVPR (2017)

work page 2017

[10] [10]

In: IPMI (2017)

Li, W., Wang, G., Fidon, L., Ourselin, S., Cardoso, M.J., Vercauteren, T.: On the compactness, eﬃciency, and representation of 3D convolutional networks: brain parcellation as a pretext task. In: IPMI (2017)

work page 2017

[11] [11]

In: ISBI (2017)

Pham, C.H., Ducournau, A., Fablet, R., Rousseau, F.: Brain MRI super-resolution using deep 3D convolutional networks. In: ISBI (2017)

work page 2017

[12] [12]

Memory-Efficient Implementation of DenseNets

Pleiss, G., Chen, D., Huang, G., Li, T., van der Maaten, L., Weinberger, K.Q.: Memory-eﬃcient implementation of densenets. arXiv:1707.06990 (2017)

work page internal anchor Pith review Pith/arXiv arXiv 2017

[13] [13]

Plenge, E., Poot, D.H., Bernsen, M., Kotek, G., Houston, G., Wielopolski, P., van der Weerd, L., Niessen, W.J., Meijering, E.: Super-resolution methods in MRI: Can they improve the trade-oﬀ between resolution, signal-to-noise ratio, and ac- quisition time? Magnetic resonance in medicine 68(6), 1983–1993 (2012)

work page 1983

[14] [14]

Brain MRI super-resolution using 3D generative adversarial networks

S´ anchez, I., Vilaplana, V.: Brain MRI super-resolution using 3D generative adver- sarial networks. arXiv preprint arXiv:1812.11440 (2018)

work page internal anchor Pith review Pith/arXiv arXiv 2018

[15] [15]

Very Deep Convolutional Networks for Large-Scale Image Recognition

Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556 (2014) EGN for brain MRI SR 9

work page internal anchor Pith review Pith/arXiv arXiv 2014

[16] [16]

IEEE transactions on medical imag- ing 29(6), 1310 (2010)

Tustison, N.J., Avants, B.B., Cook, P.A., Zheng, Y., Egan, A., Yushkevich, P.A., Gee, J.C.: N4itk: improved n3 bias correction. IEEE transactions on medical imag- ing 29(6), 1310 (2010)

work page 2010

[17] [17]

In: CVPR (2018)

Wang, X., Yu, K., Dong, C., Change Loy, C.: Recovering realistic texture in image super-resolution by deep spatial feature transform. In: CVPR (2018)

work page 2018

[18] [18]

In: ECCV (2018)

Wang, X., Yu, K., Wu, S., Gu, J., Liu, Y., Dong, C., Qiao, Y., Loy, C.C.: Esrgan: Enhanced super-resolution generative adversarial networks. In: ECCV (2018)

work page 2018

[19] [19]

In: CVPR (2018)

Zhang, Y., Tian, Y., Kong, Y., Zhong, B., Fu, Y.: Residual dense network for image super-resolution. In: CVPR (2018)

work page 2018

[20] [20]

In: ISBI (2018)

Zhao, C., Carass, A., Dewey, B.E., Prince, J.L.: Self super-resolution for magnetic resonance images using deep networks. In: ISBI (2018)

work page 2018