REVIEW 5 major objections 5 minor 19 references
A Multimodal Deep Network for the Reconstruction of T2W MR Images
T0 review · 5 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read A multimodal Dense U-Net reconstructs fourfold-undersampled T2-weighted MR images with SSIM 0.94, using the patient's FLAIR image to preserve brain-lesion detail.
desk verdict Plausible multimodal T2/FLAIR reconstruction idea, but the central SSIM claim is not yet backed by a described test protocol. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the Multimodal Dense U-Net paired with a custom static k-space subsampling mask. The mask, applied at acceleration factor four along the phase-encoding direction, places 80 percent of the sampled lines in the central low-frequency region of k-space and the remaining lines equidistantly to retain high-frequency edge information. The network has two input branches, one for the subsampled T2 image and one for the FLAIR image, each undergoing convolutional contraction before their feature maps are merged and processed by dense blocks built from batch normalization, ELU activations, and 3x3 convolutions, followed by deconvolutive expansion and a final dense block with a 1x1 convolution that outputs the reconstructed T2 image. Training minimizes the sum of mean squared error and structural dissimilarity, so the optimization target directly combines pixel accuracy with perceptual similarity.
What would settle it
Retrain or re-run the network with an explicit train/validation/test split that holds out entire patients, recompute SSIM and MSE on the held-out images, and compare against the unimodal Dense U-Net; if the held-out SSIM falls to 0.86 or below, or if deliberately misaligned FLAIR inputs erase the gain, the claimed multimodal advantage is not established.
Extended reading notes
Core claim
On the paper's own terms, the central discovery is that FLAIR images carry enough structural and lesion information correlated with T2-weighted images to drive high-quality reconstruction of fourfold-subsampled T2 data. A network that processes the subsampled T2 image and the FLAIR image in separate branches, merges their features, and follows the U-Net encoding-decoding layout with dense blocks achieves SSIM 0.94, against 0.86 for the unimodal Dense U-Net. A supporting discovery is that the k-space sampling pattern matters: a custom mask that takes 80 percent of its samples from the low-frequency center of k-space and spreads the remaining 20 percent equidistantly along the phase-encoding direction outperforms a center-only mask, with SSIM 0.86 versus 0.71. The paper presents this as the first attempt to reconstruct T2-weighted images from subsampled T2 and FLAIR images with particular attention to the lesion region.
Load-bearing premise
The central claim stands on the reported 0.94 SSIM having been computed on images the network never saw during training, and on the FLAIR and T2 slices being spatially aligned, yet the paper documents neither a train/test split nor a registration check.
Editorial extensions
If this is right
- If the reported result is correct, T2-weighted MRI acquisition can be accelerated fourfold while retaining structural similarity of 0.94, reducing scan time and motion-artifact exposure for patients with multiple sclerosis.
- The 80/20 center-to-periphery sampling split implies that keeping some high-frequency k-space lines, rather than sampling only the low-frequency center, is what preserves edges and lesion boundaries.
- The gain from 0.86 to 0.94 SSIM over the unimodal Dense U-Net implies that FLAIR images, already acquired in MS protocols, can be reused as free side information to improve reconstruction without additional scan time.
- Because the loss includes structural dissimilarity and the reported gains appear in the lesion area, the method is aimed at preserving diagnostically relevant detail rather than merely improving pixel statistics.
Reading between the lines
- A natural extension the paper does not pursue is applying the same two-branch fusion to other correlated MRI contrast pairs, such as T1-to-T2 or PD-to-FLAIR, where one contrast could be acquired faster and the other used as guidance.
- The paper never defines a train/validation/test split, so the decisive test for its central claim is whether the 0.94 SSIM survives on held-out patients; if it does not, the reported number would reflect memorization rather than reconstruction ability.
- The FLAIR-to-T2 spatial alignment is assumed after reslicing but never quantified, so a practical deployment would need a registration step or a network made robust to misalignment before the multimodal gain could be trusted in the clinic.
- A clinically stronger evaluation would supplement image-similarity metrics with lesion-segmentation scores, for example comparing automated lesion masks on reconstructed and fully sampled images, since whole-image SSIM can be dominated by non-lesion tissue.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a multimodal deep network, termed Multimodal Dense U-Net, to reconstruct T2-weighted MR images that have been subsampled in k-space by a factor of 4. The network takes as input a subsampled T2 image and a FLAIR image, exploiting the anatomical correlation between the two modalities. The authors also introduce a custom k-space sampling mask that keeps 80% of samples from the center of k-space and distributes the rest equidistantly. The loss is a combination of MSE and DSSIM. Experiments on a public dataset of 30 multiple sclerosis patients are reported, with the main claim being that the proposed architecture achieves 0.94 SSIM versus 0.86 for a unimodal Dense U-Net baseline, and that the custom mask outperforms a center-only mask. The paper concludes that fourfold acceleration is feasible while preserving reconstruction quality in lesion areas.
Significance. If the reported results are properly validated, the contribution is potentially useful for fast MRI in multiple sclerosis imaging: exploiting FLAIR-T2 correlation is a sensible direction, the custom mask has a clear rationale, and the proposed architecture is simple and reproducible. The paper uses a public dataset and reports a concrete acceleration factor. However, the significance depends entirely on whether the quantitative claims are computed on held-out data; the current manuscript does not provide that evidence. The comparison is also based on very few images with no statistical support, and the loss curve shown is only a training loss. With proper test-set evaluation, the method could be a modest but valid empirical contribution; without it, the central claim is unsupported.
major comments (5)
- [Section 4.1, Figures 3 and 4] The paper never defines a train/validation/test split or a patient-level separation. Section 4.1 states only that "for each patient we provide the network with 150 FLAIR and T2WIsub images using the T2WIs as target," and the MSE/SSIM values in Figures 3 and 4 are reported for single predicted images with no statement that those images were held out from training. If the displayed and scored slices belong to the training set, the central 0.94 versus 0.86 SSIM comparison would measure memorization of patient anatomy rather than reconstruction ability. The authors must specify the exact number of test slices, perform a patient-level split, and report aggregate metrics (mean and standard deviation) computed on data never used for training.
- [Section 4.3, Figure 4] The quantitative evaluation of the proposed architecture rests on a single displayed image and the text's assertion that "the degree of similarity with respect to the target is 94% rather than 86% of the Dense U-Net." No error bars, number of evaluated images, or per-patient statistics are provided. Two single-image SSIM values do not establish that the multimodal network robustly outperforms the unimodal baseline; the authors need to report distributions over a held-out test set and, if possible, a paired statistical test.
- [Figure 5] The loss curve shown in Figure 5 is the training loss only. No validation loss is plotted, and the early stopping mentioned in Section 4.1 is not tied to any validation criterion or performance metric. A training loss curve provides no evidence about generalization and therefore cannot support the reconstruction-quality claim made in the paper.
- [Section 2.2] The proposed custom mask fixes the central fraction at 80% of the samples and the subsampling factor at 4, but the paper does not describe how these parameters were selected or whether they were chosen before inspecting the evaluation cases. If the mask parameters were tuned on the same images later scored in Figures 3 and 4, the reported improvement over the center mask is subject to selection bias. The authors should state whether the mask was fixed a priori and, ideally, validate the mask choice on an independent set.
- [Section 4.3] The proposed architecture is called a Dense U-Net even though the growth rate of its dense blocks is set to zero, which removes the defining growth mechanism of dense blocks. In addition, the unimodal Dense U-Net baseline is not fully specified: the reader is not told whether it uses the same loss, growth rate, number of feature maps, training schedule, or preprocessing. Without these details, the 0.94 versus 0.86 comparison cannot be attributed to the multimodal design rather than to differences in architecture or training setup.
minor comments (5)
- [Section 1] There is a typo in the introduction: "caused by to slow movements" should read "caused by slow movements."
- [Figure 4] Figure 4 has no axis labels and the caption does not state which image is the target, which is the unimodal prediction, and which is the multimodal prediction; the SSIM values are only mentioned in the body text.
- [Equation (4)] The typesetting of Eq. (4) is ambiguous: the DSSIM expression should be clearly parenthesized as (1 - SSIM)/2, with the denominator unambiguously separated from the leading 1/2 term.
- [Section 4.1] The preprocessing description says voxel sizes were made isotropic to 0.8 x 0.8 x 0.8 mm^3, but it does not state whether FLAIR and T2 slices were co-registered or whether their spatial alignment was verified after reslicing; the multimodal approach assumes voxelwise correspondence between the two inputs.
- [References] Reference 18 has a garbled author name ("ˇZiga, L.") and Reference 19 contains a typo ("deep leaning" instead of "deep learning"); the reference list should be checked for formatting and completeness.
Circularity Check
No circularity: the multimodal reconstruction is a standard supervised fit, and no reported result reduces to the training objective or to a self-citation by construction.
full rationale
The paper's chain is empirical rather than derivational: it defines a target Y_T2, a subsampled input Y_T2sub = M·Y_T2 (Eq. 1), trains a Multimodal Dense U-Net to minimize MSE + DSSIM (Eq. 2), and reports SSIM/MSE on reconstructed images. There is no fitted parameter later renamed as a prediction, no 'uniqueness theorem' imported from the authors' prior work, and no self-citation used as load-bearing evidence; all architectural components (U-Net, dense blocks, ELU, Adam) are cited to external sources. The custom subsampling mask is a fixed design choice, not a parameter fitted to the reported images. The only validity concern is that Section 4.1 describes the training data ('for each patient we provide the network with 150 FLAIR and T2WIsub images using the T2WIs as target') without defining a train/validation/test split, so the 0.94 SSIM figure in Section 4.3 cannot be confirmed from the text to be out-of-sample. That is a reporting gap, not a circularity: the paper does not state that the scored slices were training slices, and no equation makes the reported SSIM equal to the training loss by construction. Therefore the circularity score is 0.
Assumptions & free parameters
free parameters (6)
- Center fraction of custom k-space mask =
0.8
- Subsampling factor =
4
- Dense block growth rate =
0
- Dense block depth and feature maps =
5 levels, 64 maps
- Training schedule =
80 epochs, batch size 4, Adam, early stopping
- Loss weighting =
Equal weights for MSE and DSSIM
assumptions (4)
- standard math Inverse Fourier transform maps subsampled k-space to a degraded image domain (Eq. (1)).
- domain assumption FLAIR and T2 images are spatially co-registered after preprocessing.
- domain assumption FLAIR images are informative for T2 reconstruction in the lesion region.
- domain assumption The public dataset [18] contains paired T2 and FLAIR slices that can be split into independent train/test patients.
Cite this review
Pith. "Pith review of A Multimodal Deep Network for the Reconstruction of T2W MR Images." pith.science (2026). https://pith.science/paper/MTL64Q6W
@misc{pith2026190803009,
author = {Pith},
title = {Pith review of: A Multimodal Deep Network for the Reconstruction of T2W MR Images},
year = {2026},
howpublished = {\url{https://pith.science/paper/MTL64Q6W}},
note = {Machine review of arXiv:1908.03009}
}
read the original abstract
Multiple sclerosis is one of the most common chronic neurological diseases affecting the central nervous system. Lesions produced by the MS can be observed through two modalities of magnetic resonance (MR), known as T2W and FLAIR sequences, both providing useful information for formulating a diagnosis. However, long acquisition time makes the acquired MR image vulnerable to motion artifacts. This leads to the need of accelerating the execution of the MR analysis. In this paper, we present a deep learning method that is able to reconstruct subsampled MR images obtained by reducing the k-space data, while maintaining a high image quality that can be used to observe brain lesions. The proposed method exploits the multimodal approach of neural networks and it also focuses on the data acquisition and processing stages to reduce execution time of the MR analysis. Results prove the effectiveness of the proposed method in reconstructing subsampled MR images while saving execution time.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[1]
Pergamon Books Inc., Elmsford, NY (1984)
Beall, P.T., Amtey, S.R., Kasturi, S.R.: NMR Data Handbook for Biomedical Ap- plications. Pergamon Books Inc., Elmsford, NY (1984)
work page 1984
-
[2]
In: International Conference on Learn- ing Representations (ICLR)
Clevert, D.A., Unterthiner, T., Hochreiter, S.: Fast and accurate deep network learning by exponential linear units (ELUs). In: International Conference on Learn- ing Representations (ICLR). pp. 1–14. San Juan, Puerto Rico (May 2016)
work page 2016
-
[3]
Mag- netic Resonance in Medicine 59(2), 365–373 (Feb 2008)
Gamper, U., Boesiger, P., Kozerke, S.: Compressed sensing in dynamic MRI. Mag- netic Resonance in Medicine 59(2), 365–373 (Feb 2008)
work page 2008
-
[4]
Haacke, E.M., Brown, R.W., Thompson, M.R., Venkatesan, R.: Magnetic Reso- nance Imaging: Physical Principles and Sequence Design, vol. 82. Wiley-Liss, New York, NY (1999)
work page 1999
-
[5]
In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Huang, G., Liu, Z., van der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 2261–2269. Honolulu, HI (Jul 2017)
work page 2017
-
[6]
Magnetic Resonance Imaging 32(10), 1344–1352 (Dec 2014)
Huang, J., Chen, C., Axel, L.: Fast multi-contrast MRI reconstruction. Magnetic Resonance Imaging 32(10), 1344–1352 (Dec 2014)
work page 2014
-
[7]
The British Journal of Radiology 88(1056) (Dec 2015)
Jaspan, O.N., Fleysher, R., Lipton, M.L.: Compressed sensing MRI: A review of the clinical literature. The British Journal of Radiology 88(1056) (Dec 2015)
work page 2015
-
[8]
IEEE Transactions on Image Processing 26(9), 4509–4522 (Sep 2017)
Jin, K.H., McCann, M.T., Froustey, E., Unser, M.: Deep convolutional neural net- work for inverse problems in imaging. IEEE Transactions on Image Processing 26(9), 4509–4522 (Sep 2017)
work page 2017
Show all 19 references
-
[9]
Medical Physics 45(7), 3120–3131 (Jul 2018)
Kim, K.H., Do, W.J., Park, S.H.: Improving resolution of MR images with an adversarial network incorporating images with different contrast. Medical Physics 45(7), 3120–3131 (Jul 2018)
2018
-
[10]
A Signal Processing Perspective
Liang, Z.P., Lauterbur, P.C.: Principles of Magnetic Resonance Imaging. A Signal Processing Perspective. The Institute of electrical and Electronics Engineers, New York, NY (2000)
2000
-
[11]
Magnetic Resonance in Medicine 58, 1182–1195 (Oct 2007)
Lustig, M., Donoho, D., Pauly, J.M.: Sparse MRI: The application of compressed sensing for rapid MR imaging. Magnetic Resonance in Medicine 58, 1182–1195 (Oct 2007)
2007
-
[12]
IEEE Signal Processing Magazine 25(2), 72–82 (Mar 2008) Multimodal Deep Network 9
Lustig, M., Donoho, D.L., Santos, J.M., Pauly, J.M.: Compressed sensing MRI. IEEE Signal Processing Magazine 25(2), 72–82 (Mar 2008) Multimodal Deep Network 9
2008
-
[13]
IEEE Signal Processing Magazine 34(6), 85–95 (Nov 2017)
McCann, M.T., Jin, K.H., Unser, M.: Convolutional neural networks for inverse problems in imaging: A review. IEEE Signal Processing Magazine 34(6), 85–95 (Nov 2017)
2017
-
[14]
IEEE Transactions on Medical Imaging 38(1), 280–290 (Jan 2019)
Qin, C., Schlemper, J., Caballero, J., Price, A.N., Hajnal, J.V., Rueckert, D.: Con- volutional recurrent neural networks for dynamic MR image reconstruction. IEEE Transactions on Medical Imaging 38(1), 280–290 (Jan 2019)
2019
-
[15]
In: International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI)
Ronnenberger, O., Fischer, P., Brox, T.: U-Net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI). Lecture Notes in Computer Science, vol. 9351, pp. 234–241. Springer, C...
2015
-
[16]
arXiv preprint arXiv:1803.09172v1 (Mar 2018)
Roy, S., Butman, J.A., Reich, D.S., Calabresi, P.A., Pham, D.L.: Multiple sclerosis lesion segmentation from brain MRI via fully convolutional neural networks. arXiv preprint arXiv:1803.09172v1 (Mar 2018)
2018 arXiv
-
[17]
IEEE Transactions on Medical Imaging 37(2), 491–503 (Feb 2018)
Schlemper, J., Caballero, J., Hajnal, J.V., Price, A.N., Rueckert, D.: A deep cas- cade of convolutional neural networks for dynamic MR image reconstruction. IEEE Transactions on Medical Imaging 37(2), 491–503 (Feb 2018)
2018
-
[18]
Neuroinformatics 16(1), 51–63 (Jan 2018)
ˇZiga, L., Galimzianova, A., Koren, A., Lukin, M., Pernuˇ s, F., Likar, B., ˇSpiclin, v.: A novel public MR image dataset of multiple sclerosis patients with lesion segmentations based on multi-rater consensus. Neuroinformatics 16(1), 51–63 (Jan 2018)
2018
-
[19]
IEEE Transactions on Biomedical Engineering (Early Access) (2018)
Xiang, L., Chen, Y., Chang, W., Zhan, Y., Lin, W., Wang, Q., Shen, D.: Deep leaning based multi-modal fusion for fast MR reconstruction. IEEE Transactions on Biomedical Engineering (Early Access) (2018)
2018
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.