pith. sign in

arxiv: 1907.11711 · v1 · pith:NMAYKROPnew · submitted 2019-07-26 · 📡 eess.IV · cs.CV· cs.LG· eess.SP· physics.med-ph· stat.ML

Deep MRI Reconstruction: Unrolled Optimization Algorithms Meet Neural Networks

Dong Liang , Jing Cheng , Ziwen Ke , Leslie Ying This is my paper

Pith reviewed 2026-05-24 15:27 UTC · model grok-4.3

classification 📡 eess.IV cs.CVcs.LGeess.SPphysics.med-phstat.ML
keywords deep learningMRI reconstructionfast MRIundersampled k-spacedata-driven networksmodel-driven networksunrolled optimizationimage reconstruction
0
0 comments X

The pith

Deep learning approaches for MRI reconstruction from undersampled data are grouped into data-driven, model-driven, and integrated categories to highlight shared structures and signal processing issues.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper surveys how neural networks reconstruct MR images when k-space measurements are reduced to shorten scan times. It sorts existing methods into three groups according to the degree to which they rely on learned data versus explicit MRI physics. Each group’s typical network layout is described, after which the review isolates elements that appear across groups and the points at which the groups diverge. Several open signal-processing questions are then raised whose resolution, the authors argue, would help produce more effective networks. Readers interested in medical imaging may care because these questions directly affect whether deep methods can deliver reliable speed-ups in clinical practice.

Core claim

The central claim is that deep-learning MRI reconstruction methods fall into three types—data-driven networks that learn mappings from training pairs, model-driven networks obtained by unrolling iterative optimization algorithms, and integrated networks that embed both—and that mapping their architectures, common components, and differences supplies the basis for discussing signal-processing issues that must be resolved to realize the full potential of deep reconstruction for fast MRI.

What carries the argument

The tripartite taxonomy (data-driven, model-driven, integrated) together with the unrolling of optimization steps into successive network layers that enforce data consistency and regularization.

If this is right

  • Networks can be assembled from reusable blocks once shared components across the three categories are identified.
  • Differences between categories indicate when a data-only approach suffices versus when explicit physics must be retained.
  • Resolving the listed signal-processing questions improves both reconstruction accuracy and generalization to new acquisition settings.
  • The review supplies a common language that allows performance comparisons across otherwise dissimilar architectures.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same taxonomy and issues may apply to deep reconstruction in other modalities such as CT or PET where undersampling or dose reduction is also desired.
  • Explicit incorporation of the discussed signal-processing constraints could be tested by retraining existing networks with added consistency layers.
  • A follow-on theoretical analysis could derive performance guarantees once the open signal questions receive concrete answers.

Load-bearing premise

That existing deep-learning MRI methods fit neatly into the three stated categories and that examining their common parts, differences, and signal-processing issues will guide the design of better networks.

What would settle it

A high-performing deep MRI reconstruction network that cannot be placed in any of the three categories or that achieves superior results while ignoring the signal-processing issues raised in the review.

Figures

Figures reproduced from arXiv: 1907.11711 by Dong Liang, Jing Cheng, Leslie Ying, Ziwen Ke.

Figure 1
Figure 1. Figure 1: (a) Architecture of MLP with three layers. (b) Architecture of RNN. (c) Architecture of GAN [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 5
Figure 5. Figure 5: Architecture of KIKI integrated approaches. -net [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
read the original abstract

Image reconstruction from undersampled k-space data has been playing an important role for fast MRI. Recently, deep learning has demonstrated tremendous success in various fields and also shown potential to significantly speed up MR reconstruction with reduced measurements. This article gives an overview of deep learning-based image reconstruction methods for MRI. Three types of deep learning-based approaches are reviewed, the data-driven, model-driven and integrated approaches. The main structure of each network in three approaches is explained and the analysis of common parts of reviewed networks and differences in-between are highlighted. Based on the review, a number of signal processing issues are discussed for maximizing the potential of deep reconstruction for fast MRI. the discussion may facilitate further development of "optimal" network and performance analysis from a theoretical point of view.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

0 major / 1 minor

Summary. The paper is a survey reviewing deep learning methods for MRI reconstruction from undersampled k-space data. It organizes the literature into three categories (data-driven, model-driven, and integrated approaches), describes the main network structures in each, analyzes common parts and differences across reviewed networks, and discusses signal-processing issues relevant to maximizing performance for fast MRI.

Significance. If the taxonomy and analysis hold, the survey provides a useful organizational framework for a fast-growing area, potentially aiding researchers by synthesizing common elements and highlighting signal-processing considerations that could inform future network design and theoretical analysis.

minor comments (1)
  1. [Abstract] Abstract, final sentence: 'the discussion may facilitate' begins with a lowercase letter and should be capitalized.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for the positive summary, significance assessment, and recommendation of minor revision. No major comments were provided in the report.

Circularity Check

0 steps flagged

No circularity: literature survey without derivations or predictions

full rationale

The paper is a review article that classifies existing deep MRI reconstruction methods into data-driven, model-driven, and integrated categories, summarizes their architectures, notes common elements and differences, and discusses signal-processing considerations. No original equations, derivations, parameter fits, or quantitative predictions are advanced whose validity could reduce to the paper's own inputs or self-citations. The taxonomy and discussion are descriptive; the comprehensiveness assumption is definitional for any survey rather than a testable load-bearing premise. No steps meet the circularity criteria.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

This is a review paper; the central claim rests on the completeness and accuracy of the literature summary rather than any new free parameters, axioms, or invented entities introduced by the authors.

pith-pipeline@v0.9.0 · 5671 in / 1043 out tokens · 26356 ms · 2026-05-24T15:27:31.517910+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

40 extracted references · 40 canonical work pages

  1. [1]

    Constrained reconstruction methods in MR imaging,

    Z.-P. Liang, F. E. Boada, R. T. Constable, E. M. Haacke, P. C. Lauterbur, and M. R. Smith, "Constrained reconstruction methods in MR imaging," Rev. Magn. Reson. Med., vol. 4, pp. 67-185, 1992

    work page 1992

  2. [2]

    Regularization Parameter Selection for Nonlinear Iterative Image Restoration and MRI Reconstruction Using GCV and SURE -Based Methods,

    S. Ramani, Z. H. Liu, J. Rosen, J. F. Nielsen, and J. A. Fessler, "Regularization Parameter Selection for Nonlinear Iterative Image Restoration and MRI Reconstruction Using GCV and SURE -Based Methods," IEEE Trans Imag Process, vol. 21, pp. 3659-3672, Aug 2012

    work page 2012

  3. [3]

    Reducing the dimensionality of data with neural networks,

    G. E. Hinton and R. R. Salakhutdinov, "Reducing the dimensionality of data with neural networks," Science, vol. 313, pp. 504-7, Jul 28 2006

    work page 2006

  4. [4]

    Machine learning: Trends, perspectives, and prospects,

    M. I. Jordan and T. M. Mitchell, "Machine learning: Trends, perspectives, and prospects," Science, vol. 349, pp. 255-60, Jul 17 2015

    work page 2015

  5. [5]

    Deep learning,

    Y. LeCun, Y. Bengio, and G. Hinton, "Deep learning," Nature, vol. 521, pp. 436-444, May 28 2015

    work page 2015

  6. [6]

    Accelerating Magnetic Resonance Imaging via Dee p Learning,

    S. Wang, Z. Su, L. Ying, X. Peng, S. Zhu, F. Liang , et al., "Accelerating Magnetic Resonance Imaging via Dee p Learning," IEEE Conference on International Symposium on Biomedical Imaging (ISBI), pp. 514 -517, 2016

    work page 2016

  7. [7]

    A parallel MR imaging method using multilayer perceptron,

    K. Kwon, D. Kim, and H. Park, "A parallel MR imaging method using multilayer perceptron," Med Phys, vol. 44, pp. 6209-6224, Dec 2017

    work page 2017

  8. [8]

    Image reconstruction by domain-transform manifold learning,

    B. Zhu, J . Z. Liu, S. F. Cauley, B. R. Rosen, and M. S. Rosen, "Image reconstruction by domain-transform manifold learning," Nature, vol. 555, pp. 487-492, Mar 21 2018

    work page 2018

  9. [9]

    Compressed Sensing MRI Reconstruction Using a G enerative Adversarial Network With a Cyclic Loss,

    T. M. Quan, T. Nguyen-Duc, and W. K. Jeong, "Compressed Sensing MRI Reconstruction Using a G enerative Adversarial Network With a Cyclic Loss," IEEE Trans Med Imaging , vol. 37, pp. 1488-1497, Jun 2018

    work page 2018

  10. [10]

    Deep Generative Adversarial Neural Networks for Compressive Sensing MRI,

    M. Mardani, E. Gong, J. Y. Cheng, S. S. Vasanawala, G. Zaharchuk, L. Xing, et al. , "Deep Generative Adversarial Neural Networks for Compressive Sensing MRI," IEEE Trans Med Imaging, vol. 38, pp. 167-179, Jan 2019

    work page 2019

  11. [11]

    Deep learning with domain adaptation for accelerated projection-reconstruction MR,

    Y. Han, J. Yoo, H. H. Kim, H. J. Shin, K. Sung, and J. C. Ye, "Deep learning with domain adaptation for accelerated projection-reconstruction MR," Magn Reson Med, vol. 80, pp. 1189-1205, Sep 2018

    work page 2018

  12. [12]

    Deep Convolutional Framelets: A General Deep Learning Framework for Inverse Problems,

    J. C. Ye, Y. Han, and E. Cha, "Deep Convolutional Framelets: A General Deep Learning Framework for Inverse Problems," SIAM J Imag Sci, vol. 11, pp. 991-1048, 2018

    work page 2018

  13. [13]

    Scan-specific robust artificial -neural-networks for k -space interpolation (RAKI) reconstruction: Database- free deep learning for fast imaging,

    M. Akcakaya, S. Moeller, S. Weingartner, and K. Ugurbil, "Scan-specific robust artificial -neural-networks for k -space interpolation (RAKI) reconstruction: Database- free deep learning for fast imaging," Magn Reson Med , vol. 81, pp. 439-453, Jan 2019

    work page 2019

  14. [14]

    Deep Residual Learning for Accelerated MRI Using Magnitude and Phase Networks,

    D. W. Lee, J. J. Yoo, S. H. Tak, and J. C. Ye, "Deep Residual Learning for Accelerated MRI Using Magnitude and Phase Networks," I EEE Trans Bio-Med Eng, vol. 65, pp. 1985-1995, Sep 2018

    work page 1985

  15. [15]

    Real-time cardiovascular MR with spatio -temporal artifact suppression using deep learning-proof of concept in congenital heart disease,

    A. Hauptmann, S. Arridge, F. Lucka, V. Muthurangu, and J. A. Stee den, "Real-time cardiovascular MR with spatio -temporal artifact suppression using deep learning-proof of concept in congenital heart disease," Magn Reson Med , vol. 81, pp. 1143-1156, Feb 2019

    work page 2019

  16. [16]

    DIMENSION: Dynamic MR Imaging with Both K -space and Spatial Prior Knowledge Obtained via Multi-Supervised Network Training,

    S. Wang, Z. Ke, H. Cheng, S. Jia, L. Ying, H. Zheng, et al., "DIMENSION: Dynamic MR Imaging with Both K -space and Spatial Prior Knowledge Obtained via Multi-Supervised Network Training," NMR Biomed, 2019

    work page 2019

  17. [17]

    A Deep Cascade of Convolutional Neural Networks for Dynamic MR Image Reconstruction,

    J. Schlemper, J. Caballero, J. V. Hajnal, A. N. Price, and D. Rueckert, "A Deep Cascade of Convolutional Neural Networks for Dynamic MR Image Reconstruction," IEEE Trans Med Imaging, vol. 37, pp. 491 -503, Feb 2018

    work page 2018

  18. [18]

    KIKI -net: cross-domain convolutional neural networks for reconstructing undersampled magnetic resonance images,

    T. Eo, Y. Jun, T. Kim, J. Jang, H. J . Lee, and D. Hwang, "KIKI -net: cross-domain convolutional neural networks for reconstructing undersampled magnetic resonance images," Magn Reson Med, vol. 80, pp. 2188-2201, Nov 2018

    work page 2018

  19. [19]

    MoDL: Model -Based Deep Learning Architecture for Inverse Problems,

    H. K. Aggarwal, M. P. Mani, and M. Jacob, "MoDL: Model -Based Deep Learning Architecture for Inverse Problems," IEEE Trans Med Imaging, vol. 38, pp. 394-405, Feb 2019

    work page 2019

  20. [20]

    Convolutional Recurrent Neural Networks for Dynamic MR Image Reconstruction,

    C. Qin, J. Schlemper, J. Caballero, A. N. Price, J. V. Hajnal, and D. Rueckert, "Convolutional Recurrent Neural Networks for Dynamic MR Image Reconstruction," IEEE Trans Med Imaging, vol. 38, pp. 280 -290, Jan 2019

    work page 2019

  21. [21]

    Learned Primal -Dual Reconstruction,

    J. Adler and O. Oktem, "Learned Primal -Dual Reconstruction," IEEE Trans Med Imaging, vol. 37, pp. 1322-1332, Jun 2018

    work page 2018

  22. [22]

    Learning Primal Dua l Network for Fast MR Imaging,

    J. Cheng, H. Wang, L. Ying, and D. Liang, "Learning Primal Dua l Network for Fast MR Imaging," presented at the Proc. 27th Annual Meeting of ISMRM, Montreal, QC, Canada, 2019

    work page 2019

  23. [23]

    Deep ADMM -Net for Compressive Sensing MRI,

    Y. Yang, J. Sun, H. Li, and Z. Xu, "Deep ADMM -Net for Compressive Sensing MRI," Neural Information Processing Systems, pp. 10-18, 2016

    work page 2016

  24. [24]

    ADMM -CSNet: A Deep Learning Approach for Image Compressive Sensing,

    Y. Yang, J. Sun, H. Li, and Z. Xu, "ADMM -CSNet: A Deep Learning Approach for Image Compressive Sensing," IEEE Trans Pattern Anal Mach Intell, Nov 28 2018

    work page 2018

  25. [25]

    Learning a variational network for reconstruction of accelerated MRI data,

    K. Hammernik, T. Klatzer, E. Kobler, M. P. Recht, D. K. Sodickson, T. Pock, et al. , "Learning a variational network for reconstruction of accelerated MRI data," Magn Reson Med, vol. 79, pp. 3055 -3071, Jun 2018

    work page 2018

  26. [26]

    ISTA-Net: Interpretable Optimization-Inspired Deep Network for Image Compressive Sensing,

    J. Zhang and B. Ghanem, "ISTA-Net: Interpretable Optimization-Inspired Deep Network for Image Compressive Sensing," IEEE Conference o n Computer Vision and Pattern Recognition (CVPR), pp. 1828-1837, 2018

    work page 2018

  27. [27]

    The perceptron: a probabilistic model for information storage and organization in the brain,

    F. Rosenblatt, "The perceptron: a probabilistic model for information storage and organization in the brain," Psychol Rev, vol. 65, pp. 386-408, Nov 1958

    work page 1958

  28. [28]

    Liang and P

    Z.-P. Liang and P. C . Lauterbur, Principles of Magnetic Resonance Imaging: A Signal Processing Perspective: Wiley -IEEE Press, 1999

    work page 1999

  29. [29]

    Image Reconstruction: An Overview for Clinicians,

    M. S. Hansen and P. Kellman, "Image Reconstruction: An Overview for Clinicians," J Magn Reson Imaging, vol. 41, pp. 573-585, Mar 2015

    work page 2015

  30. [30]

    Single-shot T2 mapping using overlapping -echo detachment planar imaging and a deep convolutional neural network,

    C. Cai, C. Wang, Y. Zeng, S. Cai, D. Liang, Y. Wu, et al., "Single-shot T2 mapping using overlapping -echo detachment planar imaging and a deep convolutional neural network," Magn Reson Med, vol. 80, pp. 2202-2214, Nov 2018

    work page 2018

  31. [31]

    MANTIS: Model-Augmented Neural neTwork with Incoherent k -space Sampling for efficient MR parameter mapping,

    F. Liu, L. Feng, and R. Ki jowski, "MANTIS: Model-Augmented Neural neTwork with Incoherent k -space Sampling for efficient MR parameter mapping," Magn Reson Med, Mar 12 2019

    work page 2019

  32. [32]

    Quantitative susceptibility mapping using deep neural network: QSMnet,

    J. Yoon, E. H. Gong, I. Chatnuntawech, B. Bilgic, J. Lee, W. Jung, et al., "Quantitative susceptibility mapping using deep neural network: QSMnet," NeuroImage, vol. 179, pp. 199-206, Oct 1 2018

    work page 2018

  33. [33]

    MR fingerprinting Deep RecOnstruction NEtwork (DRONE),

    O. Cohen, B. Zhu, and M. S. Rosen, "MR fingerprinting Deep RecOnstruction NEtwork (DRONE)," Magn Reson Med, vol. 80, pp. 885-894, Sep 2018

    work page 2018

  34. [34]

    Assessment of the generalization of learned image reconstruction and the potential for transfer learning,

    F. Knoll, K. Hammernik, E. Kobler, T. Pock, M. P. Recht, and D. K. Sodickson, "Assessment of the generalization of learned image reconstruction and the potential for transfer learning," Magn Reson Med, vol. 81, pp. 116-128, Jan 2019

    work page 2019

  35. [35]

    MR image reconstruction from highly undersampled k -space data by dictionary learning,

    S. Ravishankar and Y. Bresler, " MR image reconstruction from highly undersampled k -space data by dictionary learning," IEEE Trans Med Imaging, vol. 30, pp. 1028-41, May 2011

    work page 2011

  36. [36]

    Adaptive Dictionary Learning in Sparse Gradient Do main for Image Recovery,

    Q. Liu, S. Wang, L. Ying, X. Peng, Y. Zhu, and D. Liang, "Adaptive Dictionary Learning in Sparse Gradient Do main for Image Recovery," IEEE Trans Image Process, vol. 22, pp. 4652-4663, Dec 2013

    work page 2013

  37. [37]

    Dynamic MRI Using SmooThness Regularization on Manifolds (SToRM),

    S. Poddar and M. Jacob, "Dynamic MRI Using SmooThness Regularization on Manifolds (SToRM)," IEEE Trans Med Imaging, vol. 35, pp. 1106-1126, Apr 2016

    work page 2016

  38. [38]

    A Kernel -Based Low-Rank (KLR) Model for Low -Dimensional Manifold Recovery in Highly Accelerated Dynamic MRI,

    U. Nakarmi, Y. Wang, J. Lyu, D. Liang, and L. Ying, "A Kernel -Based Low-Rank (KLR) Model for Low -Dimensional Manifold Recovery in Highly Accelerated Dynamic MRI," IEEE Trans Med Imaging, vol. 36, pp. 2297-2307, Nov 2017

    work page 2017

  39. [39]

    Dermatologist-level classification of skin cancer with deep neural networks,

    A. Esteva, B. Kuprel, R. A. Novoa, J. Ko, S. M. Swetter, H. M. Blau, et al., "Dermatologist-level classification of skin cancer with deep neural networks," Nature, vol. 542, pp. 115-118, Feb 2 2017

    work page 2017

  40. [40]

    Radiomics in lung cancer: Its time is here,

    M. Kalra, G. Wang, and C. G. Orton, "Radiomics in lung cancer: Its time is here," Med Phys , vol. 45, pp. 997-1000, Mar 2018

    work page 2018