Pith. sign in

REVIEW 3 major objections 5 minor 23 references

Deep Slice Interpolation via Marginal Super-Resolution, Fusion and Refinement

T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Deep slice interpolation can be done with 2D networks alone, and the 2D pipeline beats both 2D and 3D CNN baselines on anisotropic brain MRI.

desk verdict Genuinely novel marginal super-resolution decomposition, but the 3D CNN comparison is capacity-uncontrolled; still worth refereeing after addressing that and missing statistics. read the letter →

arxiv 1908.05599 v1 pith:NE5EIRB6 submitted 2019-08-15 eess.IV cs.CV

Cheng Peng , Wei-An Lin , Haofu Liao , Rama Chellappa , S. Kevin Zhou This is my paper · ORCID
classification eess.IVcs.CV
keywords sliceinterpolationmarginalsuper-resolution2DconvolutionalneuralnetworkanisotropicMRItwo-viewfusionbrainvolumeestimationsemanticsegmentationimagerefinement
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that interpolating missing axial slices in an anisotropic brain MR scan does not require a 3D neural network. The proposed pipeline treats the undersampled volume as two stacks of 2D slices, sagittal and coronal, and super-resolves each stack with a 2D convolutional network, producing line-by-line estimates of the missing axial slices. A fusion network combines the two estimates, and a refinement network improves consistency along the axial axis. The payoff is that the fully 2D method beats 2D and 3D CNN baselines on reconstruction quality and gray/white-matter segmentation while needing less GPU memory and fewer training examples.

What carries the argument

The load-bearing object is marginal super-resolution (MSR): super-resolving the downsampled volume independently from the sagittal and coronal directions, using a residual dense network that takes a slab of $s=3$ slices and outputs a single high-resolution slice. The second component is two-view fusion, a small network that takes the re-sliced sagittal and coronal super-resolved estimates of each axial slice and combines them into one slice, removing the horizontal and vertical streak artifacts each view leaves behind. The third component is a refinement network that processes a slab of $k+1$ slices along the axial direction to enforce between-slice consistency; the paper uses an $L^1$ loss at every stage and reports that adversarial loss at refinement produces fake anatomy.

What would settle it

Take isotropic brain volumes, keep every $k$-th axial slice and discard the rest, then run the pipeline. If for some volumes the reconstruction PSNR falls below that of a 3D CNN while the 3D CNN stays high, and the failures concentrate where sagittal and coronal views are smooth but axial anatomy changes sharply, the marginal-super-resolution sufficiency assumption would be violated.

Watch

Extended reading notes

Core claim

The central claim is that 3D slice interpolation can be decomposed into two 2D super-resolution problems followed by fusion and refinement, and that this decomposition is not just a memory-saving approximation but a better predictor. On 120 T1 brain scans downsampled by factors of 4 and 8, the full pipeline reports the best PSNR/SSIM (32.22 dB/0.9441 at 4x, 28.87 dB/0.8808 at 8x) and the best gray/white-matter Dice coefficients (0.9021/0.8593 at 4x, 0.8189/0.7828 at 8x) among the compared methods. The paper interprets the consistent margin over the 3D CNN as evidence that exploiting full 2D context through orthogonal views is more valuable than a direct 3D mapping trained on small patches.

Load-bearing premise

The pipeline assumes that 2D super-resolution of the sagittal and coronal views captures enough 3D anatomical context to reconstruct the missing axial slices, even when the anatomy changes sharply between observed slices.

Editorial extensions

If this is right

  • At 4x sparsity the full pipeline reaches 32.22 dB PSNR and 0.9441 SSIM, above the 2D CNN (31.24 dB, 0.9313) and the 3D CNN (31.34 dB, 0.9292).
  • At 8x sparsity the gap grows: 28.87 dB and 0.8808 SSIM versus 27.88 dB/0.8444 for the 2D CNN and 27.38 dB/0.8390 for the 3D CNN.
  • Interpolated slices segment gray and white matter more accurately, with GM/WM Dice of 0.9021/0.8593 at 4x and 0.8189/0.7828 at 8x.
  • The two-view fusion removes the horizontal and vertical streak artifacts visible in single-view MSR, and the refinement step adds between-slice consistency.
  • Because every operation is 2D, models can be trained on full slices rather than small patches, which the paper credits for the edge over the 3D CNN.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If marginal super-resolution is as sufficient as the paper's results suggest, the same decompose-fuse-refine recipe should transfer to other anisotropic volumetric modalities, such as CT or fetal ultrasound, where orthogonal views carry complementary anatomy.
  • A direct stress test would be to run the pipeline on slices whose gaps are much larger than 8x, or on volumes where the under-sampled axis is not axis-aligned, since the orthogonality of the two marginal views weakens in that setting.
  • The improved segmentation dice on interpolated slices implies that volumetric biomarkers, such as gray-matter volume, could be estimated reliably from heavily under-sampled scans without acquiring an isotropic volume.
  • The reported failure of adversarial loss at the refinement stage suggests a natural follow-up: replace the GAN objective with a structure-preserving or consistency-based loss that sharpens details without inventing anatomy.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a deep-learning pipeline for interpolating missing axial slices in anisotropic brain MRI. The first stage, marginal super-resolution (MSR), applies a 2D residual dense network to slabs of sagittal and coronal slices and reformats the outputs into two estimates of the axial volume. The second stage, two-view fusion and refinement (TFR), fuses these two axial estimates and refines the fused volume along the axial direction. Experiments on 120 ADNI T1 volumes with 4x and 8x under-sampling compare the method with linear interpolation and with 2D and 3D CNN baselines, using PSNR/SSIM and gray/white-matter segmentation Dice and Hausdorff distances. The paper also reports an ablation at 4x sparsity. The reported results show the proposed method achieving the best numbers on all metrics.

Significance. If the main comparison were fully controlled, the paper would make a genuinely useful practical contribution: it shows a memory-efficient 2D-only strategy for slice interpolation, with a clear experimental protocol, a held-out split, an ablation, and downstream-segmentation evaluation. The MSR+TFR pipeline is intuitive and the ablation indicates that each component adds value. However, the headline claim of consistent superiority over 3D CNNs is weakened by an unbalanced 3D baseline and by the absence of any statistical uncertainty estimates, so the central comparison is not yet established at the level required by a journal.

major comments (3)
  1. [Sec. 5.1 and Table 1] The comparison against the 3D CNN baseline is not controlled for model capacity or receptive field. The paper states in Sec. 5.1 that "due to memory constraint, 3D CNN only uses one RDB", while the proposed pipeline uses two RDBs, and the 3D CNN is evaluated on a limited 144x144x256 region whereas the proposed method processes full 2D slices. The reported PSNR/SSIM advantages over the 3D CNN (0.88 dB at 4x and 1.49 dB at 8x in PSNR from Table 1) could therefore be attributed to the weaker baseline rather than to the marginal super-resolution decomposition. The authors should add a 3D baseline with matched parameter count and effective receptive field (e.g., via gradient accumulation or overlapping sub-volume inference), or explicitly state this as a limitation and provide a separate justification for the claimed advantage over 3D processing.
  2. [Sec. 5.2 and Table 1] All quantitative claims are based on single point estimates. With only 20 test volumes, the phrase "consistently outperforms" (Sec. 5.2) is not supported by a statistical test. The authors should report per-subject means with standard deviations or confidence intervals, and apply a paired significance test (e.g., Wilcoxon signed-rank or paired t-test) across the test volumes for PSNR, SSIM, DICE, and HD. Without this, the reported gains of roughly 0.9-1.5 dB PSNR and the DICE differences may not be significant.
  3. [Sec. 4.1] The paper assumes that 2D super-resolution of sagittal and coronal views is informationally sufficient for axial slice interpolation, but this assumption is never validated against a strong 3D model. Because the only 3D comparator is capacity-limited (see the first major comment), the experiments cannot distinguish between the merit of the marginal decomposition and the effect of using a more powerful 2D network. A concrete test would be to compare the MSR-only stage (Table 2) with a full-resolution 3D network of matched capacity on a subset, or to measure the information loss via a consistency check between the fused volume and the observed axial slices. At minimum, the paper should discuss this limitation explicitly.
minor comments (5)
  1. [Figure numbering] Several figures are numbered Fig. 4 in Secs. 5.3 and 5.4, which makes cross-referencing confusing; the figures should be renumbered.
  2. [Sec. 4.2] There are typos such as "super-resovled" and inconsistent use of "saggital"/"sagittal"; the manuscript should be proofread.
  3. [Sec. 5.1] The test-set restriction to slices containing "mostly brain tissues" should be quantified (how many slices per volume) and its effect on the reported metrics should be discussed.
  4. [Secs. 4.2 and 5.1] Architectural details for the baseline 2D CNN and the fusion/refinement networks are underspecified; the text says only that the upsampling network is replaced by one convolutional layer, but the number of filters, kernel sizes, activations, and strides should be reported for reproducibility.
  5. [Sec. 5.2] The statement that at 8x sparsity linear interpolation's HD scores are "comparable" to the CNN baselines is not clearly supported by Table 1, where the WM HD values are 10.59 for LI versus 9.017 for 3D CNN.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: empirical held-out evaluation with independently trained MSR, fusion, and refinement stages.

full rationale

All claimed derivations are empirical and self-contained. The MSR/TFR pipeline is trained from isotropic ground-truth slices via L1 losses (Eqs. 4-6) on a training split and evaluated on a held-out test split from ADNI; no predicted quantity is defined in terms of the target, and no fitted parameter is relabeled as a prediction. Sagittal and coronal super-resolution outputs are fused and refined using axial ground truth, but the fusion and refinement networks are separately trained with L1 loss against axial ground truth, so the final PSNR/SSIM and Dice numbers are not forced by construction. The only self-citation is reference [10] in the general related-work sentence about deep CNNs outperforming traditional approaches; it is not load-bearing and does not justify the central claim. The possible critique that the 3D CNN baseline has reduced capacity and receptive field is an experimental fairness concern, not circularity. Therefore no circular step is found.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The method rests on domain assumptions about learnability and evaluation validity rather than on mathematical axioms. The key unproven premise is that the marginal 2D super-resolution views are informationally sufficient for axial slice reconstruction; this is tested only indirectly through end-task metrics.

free parameters (3)
  • Input slab size s = 3
    Number of sagittal/coronal slices used as MSR network input. The ablation compares s=1 vs s=3 and shows improvement, but the value 3 is chosen without a full search.
  • Number of residual dense blocks = 2 (2D networks), 1 (3D baseline)
    Network capacity is a hand-chosen hyperparameter; the 3D baseline is deliberately smaller due to memory, making the comparison uneven.
  • Learning rate and optimizer = Adam, lr=0.0001, momentum 0.5
    Standard training choices; not searched.
assumptions (3)
  • domain assumption 2D CNN super-resolution of sagittal/coronal views produces axial slices that are a good initial estimate.
    Section 4.1 states that super-resolving sagittal and coronal views is 'equivalent to applying a sequence of 2D super-resolution', but the information-theoretic sufficiency of this decomposition is not proven.
  • domain assumption The U-Net segmentation network trained on OASIS provides an unbiased evaluation of clinical utility when applied to ADNI interpolated slices.
    Section 5.1: the segmentation network is trained on OASIS and applied to the interpolated ADNI slices; domain shift between datasets is not analyzed.
  • domain assumption ADNI 120 T1 scans are representative of general brain MRI for the claims made.
    Section 5.1: only 120 scans from ADNI are used; the split is 95/5/20. Generalization to other scanners, field strengths, or pathologies is not demonstrated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Deep Slice Interpolation via Marginal Super-Resolution, Fusion and Refinement." pith.science (2026). https://pith.science/paper/NE5EIRB6

@misc{pith2026190805599,
  author       = {Pith},
  title        = {Pith review of: Deep Slice Interpolation via Marginal Super-Resolution, Fusion and Refinement},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NE5EIRB6}},
  note         = {Machine review of arXiv:1908.05599}
}
read the original abstract

We propose a marginal super-resolution (MSR) approach based on 2D convolutional neural networks (CNNs) for interpolating an anisotropic brain magnetic resonance scan along the highly under-sampled direction, which is assumed to axial without loss of generality. Previous methods for slice interpolation only consider data from pairs of adjacent 2D slices. The possibility of fusing information from the direction orthogonal to the 2D slices remains unexplored. Our approach performs MSR in both sagittal and coronal directions, which provides an initial estimate for slice interpolation. The interpolated slices are then fused and refined in the axial direction for improved consistency. Since MSR consists of only 2D operations, it is more feasible in terms of GPU memory consumption and requires fewer training samples compared to 3D CNNs. Our experiments demonstrate that the proposed method outperforms traditional linear interpolation and baseline 2D/3D CNN-based approaches. We conclude by showcasing the method's practical utility in estimating brain volumes from under-sampled brain MR scans through semantic segmentation.

Figures

Figures reproduced from arXiv: 1908.05599 by the authors.

Figure 1
Figure 1. The axial, coronal, and sagittal views of an anisotropic MR volume are [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Marginal Super-Resolution Pipeline. 4.1 Marginal Super-Resolution [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Two-view Fusion Pipeline. The TFR stage is the counterpart of MSR which further improves the qual￾ity of slice interpolation by learning the structural variations along the axial direction. As shown in [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Refinement Pipeline. After fusion, the interpolated slices already have visually pleasing qualities. Finally, to improve between-slice consistency along the axial axis, a refinement network Rψ takes a slab of k+ 1 slices I z,k+1 fuse as input and generates a consistent…
Figure 4
Figure 4. Figure 4: Visual comparisons of slice interpolation approaches. For 4x sparsity, the [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 4
Figure 4. Figure 4: Visual comparison of gray matter (Green)/white matter (Blue) segmenta [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 4
Figure 4. Figure 4: Visual comparison for the proposed components. [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

23 extracted references · 17 canonical work pages

  1. [1]

    IEEE Trans

    Ravishankar, S., Bresler, Y.: MR image reconstruction from highly undersampled k-space data by dictionary learning. IEEE Trans. Med. Imaging 30(5) (2011) 1028–1041

    work page 2011

  2. [2]

    Lustig, M., Donoho, D., Pauly, J.M.: Sparse mri: The application of compressed sensing for rapid mr imaging. Magnetic Resonance in Medicine: An Official Journal of the International Society for Magnetic Resonance in Medicine58(6) (2007) 1182– 1195 12 Authors Suppressed Due to Excessive Length

    work page 2007

  3. [3]

    In: 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2008), 24-26 June 2008, Anchorage, Alaska, USA

    Ma, S., Yin, W., Zhang, Y., Chakraborty, A.: An efficient algorithm for compressed MR imaging using total variation and wavelets. In: 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2008), 24-26 June 2008, Anchorage, Alaska, USA. (2008)

    work page 2008

  4. [4]

    IEEE Trans

    Schlemper, J., Caballero, J., Hajnal, J.V., Price, A.N., Rueckert, D.: A deep cas- cade of convolutional neural networks for dynamic MR image reconstruction. IEEE Trans. Med. Imaging 37(2) (2018) 491–503

    work page 2018

  5. [5]

    IEEE Trans

    Goshtasby, A.A., Turner, D.A., Ackerman, L.V.: Matching of tomographic slices for interpolation. IEEE Trans. Med. Imaging 11(4) (1992) 507–516

    work page 1992

  6. [6]

    IEEE Trans

    Grevera, G.J., Udupa, J.K.: Shape-based interpolation of multidimensional grey- level images. IEEE Trans. Med. Imaging 15(6) (1996) 881–892

    work page 1996

  7. [7]

    IEEE Trans

    Lee, T., Wang, W.: Morphology-based three-dimensional interpolation. IEEE Trans. Med. Imaging 19(7) (2000) 711–721

    work page 2000

  8. [8]

    IEEE Trans

    Penney, G.P., Schnabel, J.A., Rueckert, D., Viergever, M.A., Niessen, W.J.: Registration-based interpolation. IEEE Trans. Med. Imaging 23(7) (2004) 922–926

    work page 2004

Show all 23 references
  1. [9]

    In: Medical Image Computing and Computer-Assisted Intervention - MICCAI 2015 - 18th International Conference Munich, Germany, October 5 - 9, 2015, Proceedings, Part III

    Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomed- ical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention - MICCAI 2015 - 18th International Conference Munich, Germany, October 5 - 9, 2015, Proceedings, Part III. ...

    2015

  2. [10]

    CoRR abs/1711.08580 (2017)

    Liu, S., Xu, D., Zhou, S.K., Mertelmeier, T., Wicklein, J., Jerebko, A.K., Grbic, S., Pauly, O., Cai, W., Comaniciu, D.: 3d anisotropic hybrid network: Trans- ferring convolutional features from 2d images to 3d anisotropic volumes. CoRR abs/1711.08580 (2017)

    2017 arXiv

  3. [11]

    CoRR abs/1501.00092 (2015)

    Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convo- lutional networks. CoRR abs/1501.00092 (2015)

    2015 arXiv

  4. [12]

    CoRR abs/1511.04587 (2015)

    Kim, J., Lee, J.K., Lee, K.M.: Accurate image super-resolution using very deep convolutional networks. CoRR abs/1511.04587 (2015)

    2015 arXiv

  5. [13]

    In: 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017

    Zhang, K., Zuo, W., Gu, S., Zhang, L.: Learning deep CNN denoiser prior for image restoration. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017. (2017) 2808–2817

    2017

  6. [14]

    In: 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016

    Kim, J., Lee, J.K., Lee, K.M.: Deeply-recursive convolutional network for image super-resolution. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016. (2016) 1637– 1645

    2016

  7. [15]

    CoRR abs/1608.00367 (2016)

    Dong, C., Loy, C.C., Tang, X.: Accelerating the super-resolution convolutional neural network. CoRR abs/1608.00367 (2016)

    2016 arXiv

  8. [16]

    CoRR abs/1707.02921 (2017)

    Lim, B., Son, S., Kim, H., Nah, S., Lee, K.M.: Enhanced deep residual networks for single image super-resolution. CoRR abs/1707.02921 (2017)

    2017 arXiv

  9. [17]

    In: 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017

    Ledig, C., Theis, L., Huszar, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A.P., Tejani, A., Totz, J., Wang, Z., Shi, W.: Photo-realistic single image super- resolution using a generative adversarial network. In: 2017 IEEE Conference on Computer Vision and Pattern Re...

    2017

  10. [18]

    CoRR abs/1802.08797 (2018)

    Zhang, Y., Tian, Y., Kong, Y., Zhong, B., Fu, Y.: Residual dense network for image super-resolution. CoRR abs/1802.08797 (2018)

    2018 arXiv

  11. [19]

    CoRR abs/1608.06993 (2016)

    Huang, G., Liu, Z., Weinberger, K.Q.: Densely connected convolutional networks. CoRR abs/1608.06993 (2016)

    2016 arXiv

  12. [20]

    CoRR abs/1406.2661 (2014) Title Suppressed Due to Excessive Length 13

    Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A.C., Bengio, Y.: Generative adversarial networks. CoRR abs/1406.2661 (2014) Title Suppressed Due to Excessive Length 13

    2014 arXiv

  13. [21]

    CoRR abs/1803.01417 (2018)

    Chen, Y., Shi, F., Christodoulou, A.G., Zhou, Z., Xie, Y., Li, D.: Efficient and accurate MRI super-resolution using a generative adversarial network and 3d multi- level densely connected network. CoRR abs/1803.01417 (2018)

    2018 arXiv

  14. [22]

    Akkus, Z., Galimzianova, A., Hoogi, A., Rubin, D.L., Erickson, B.J.: Deep learning for brain MRI segmentation: State of the art and future directions. J. Digital Imaging 30(4) (2017) 449–459

    2017

  15. [23]

    Marcus, D.S., Wang, T.H., Parker, J., Csernansky, J.G., Morris, J.C., Buckner, R.L.: Open access series of imaging studies (OASIS): cross-sectional MRI data in young, middle aged, nondemented, and demented older adults. J. Cognitive Neuroscience 19(9) (2007) 1498–1507

    2007

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.