REVIEW 4 major objections 6 minor 28 references
Cross-modal Medical Image Generation Based on Pyramid Convolutional Attention Network
T0 review · 4 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read A GAN that generates PET images from structural MRI beats previous methods on ADNI quality and AD diagnosis.
desk verdict A plausible architecture package, but the headline numbers are effectively unreviewable: no held-out split, an adversarial loss written as L1, and baselines only partially reproduced. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the PCCA block (pyramid convolution plus channel attention) in the generator's contraction path, followed by a self-attention layer before the deconvolution stages. Pyramid convolution runs several 3D convolutions with different kernel sizes in parallel and concatenates their output, so the network sees both fine details and broad context; channel attention reweights those multi-scale features to suppress redundancy. Self-attention then computes global correlations across spatial positions of the deepest feature map, which the authors argue preserves the overall structure of the brain. The generator is trained with a joint loss combining adversarial loss, voxel-wise L1 loss, and multi-scale SSIM loss, each responsible for a different aspect of fidelity.
What would settle it
Rerun the three main models (PCSA-GAN, HGAN, and GLA-GAN) on a fixed public train/test split of the 356 paired ADNI subjects, using the authors' preprocessed 64x64x64 gray-matter masks, and check whether PCSA-GAN still gets the best MAE, PSNR, and SSIM on the same held-out test set; if it does not, the outperformance claim collapses.
Extended reading notes
Core claim
The central claim is that a U-Net-style conditional GAN whose encoder uses pyramid convolution (parallel 3D convolutions with kernels of size 7, 5, and 3) plus channel attention, and whose decoder begins with a self-attention layer, produces PET images that quantitatively and visually match real FDG-PET scans better than earlier CNN-, U-Net-, and CycleGAN-based sMRI-to-PET methods. The authors report MAE 0.0194, PSNR 29.65, and SSIM 0.9486 on the ADNI dataset, and they report that these synthesized PET images, paired with the source sMRI, give 94.21% accuracy in an AD-versus-normal classification task. They also demonstrate via ablation that removing either the pyramid-channel-attention module or the self-attention module degrades the metrics, and that adding L1 and multi-scale SSIM losses on top of adversarial loss improves output quality.
Load-bearing premise
The headline numbers rest on the assumption that the test subjects were not used in training and that HGAN and GLA-GAN were run under the same conditions, but the paper admits in Section 4.8 that it did not fully reimplement those methods.
Editorial extensions
If this is right
- Generated PET could substitute for real PET in downstream multimodal classification, raising accuracy from 90.48% with sMRI alone to 94.21% in the paper's experiments.
- Clinics without PET capacity could still obtain PET-equivalent metabolic information from the MRI scans they already collect.
- The architecture suggests that multi-scale local features plus global self-attention is a workable recipe for cross-modal medical image synthesis.
- The joint loss result implies that voxel-level and structural losses add complementary training signal to adversarial loss for medical image generation.
Reading between the lines
- Editorial inference: if the reported results replicate on an independent patient-disjoint split, the method could reduce the need for real PET scans in AD research cohorts, since the synthesized PET carries most of the classification information.
- Editorial inference: the same pyramid-plus-attention generator could be tried for other missing-modality pairs such as MRI-to-CT or across PET tracers, since the method is not tied to AD-specific image content.
- Editorial inference: the strongest public test would be to release train/test subject identifiers and the reimplemented baseline code, allowing the outperformance claim to be checked without relying on the paper's in-house reproduction.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes PCSA-GAN, a conditional GAN with a U-Net style generator that augments pyramid convolution with channel attention (PCCA blocks) and a self-attention layer for generating PET images from sMRI in Alzheimer's disease research. The method is evaluated on ADNI, reporting MAE 0.0194, PSNR 29.65, and SSIM 0.9486 for image generation, and 94.21% accuracy for AD classification when the generated PET is combined with sMRI. The authors claim these results outperform prior sMRI-to-PET synthesis methods.
Significance. If properly validated, the proposed architecture would be a modest but useful contribution to cross-modal medical image synthesis, combining multi-scale local feature extraction with global self-attention. The paper includes ablation studies over the PCA and SA modules and over loss functions, which is a strength, and it reports standard deviations for these ablations. However, the empirical claims are currently not supported because the evaluation protocol lacks a held-out test split, the baseline comparisons are incomplete, and the training objective is ambiguously specified. The paper also does not provide code or data to enable independent verification, limiting its reproducibility.
major comments (4)
- [Section 4.2, 4.3] The paper never specifies a train/test split for the 356 paired subjects. Section 4.3 reports the headline metrics 'during training on the publicly available ADNI dataset,' which suggests the MAE, PSNR, and SSIM values may be computed on the training set. Without a held-out test partition, these numbers do not demonstrate generalization, and the claimed superiority over prior methods cannot be assessed. This is a load-bearing issue for the central empirical claim.
- [Section 3.4.1, 3.4.2, Eq. (7)-(12)] Equation (7) is labeled 'adversarial loss' but is identical to the voxel-wise L1 reconstruction loss in Eq. (8). The discriminator loss L_D that appears in Eq. (12) is never defined. The training objective is therefore ambiguous, and it is unclear how adversarial training contributes to the reported results. The authors should state the actual adversarial loss and the full training objective.
- [Section 4.7, 4.8] The comparison with HGAN and GLA-GAN is not verifiable. The authors state that they 'attempted to reproduce' these models and admit in Section 4.8 that it is 'not easy to implement these methods completely.' Without a complete, common implementation and a shared evaluation protocol, the reported superiority is unsupported. Additionally, the 328 sMRI-only subjects used for the classification experiment are not shown to be disjoint from the 356 paired subjects used to train the generator; if there is overlap, the classification results are optimistically biased.
- [Table 3, Section 4.8] Table 3 shows that the proposed method has a lower AUC (94.91%) than Pan et al. (96.95%) and Gao et al. (95.60%), yet the text in Section 4.8 and the abstract claim the method outperforms previous studies in AD diagnosis. The claim should be restricted to the metrics where the method is actually best (ACC, SPE, SEN, F1), or the AUC discrepancy must be explained. As written, the claim is internally inconsistent with the reported numbers.
minor comments (6)
- [Table 2 caption] The caption of Table 2 repeats the caption of Table 1 and refers to 'the impact of the self-attention mechanism,' but the table rows list loss functions. The caption should be corrected.
- [Section 4.8] The text contains 'as shown in Table ??' instead of a proper reference to Table 3.
- [Section 3.4.2, Eq. (3), (9), (10)] There are typesetting errors in equations: Eq. (3) is missing the pooling and activation symbols, and Eqs. (9) and (10) have misplaced or mismatched brackets. These should be cleaned up.
- [References] Reference [24] is cited for the N3 bias-correction algorithm but points to a paper about the anterior commissure; reference [27] is cited for the HAMMER registration algorithm but the cited work is on histopathological image registration. The citations should be corrected to the original methodological sources.
- [Throughout] The paper uses both 'MS-SSIM' and 'MM-SSIM' (the latter in Table 2). The abbreviation should be unified to 'MS-SSIM'.
- [Table 1, Section 4.5] The rows of Table 1 are labeled 'Patch Size,' but the text describes applying self-attention at different scales of features. Please clarify what 'Patch Size' means and how it relates to the feature scale.
Circularity Check
No circularity: this is an empirical GAN benchmarking paper whose claims rest on measured comparisons against ground-truth PET on ADNI, not on a derivation chain that reduces to its own inputs.
full rationale
The paper contains no derivation chain of the kind that could be circular. The central claims—that PCSA-GAN produces PET images with MAE 0.0194, PSNR 29.65, and SSIM 0.9486, and that generated PET plus sMRI yields 94.21% AD classification accuracy—are empirical measurements against real PET ground truth and a classification task, not quantities forced by construction from the method's definitions. The identical equations for adversarial loss (Eq. 7) and voxel-wise L1 loss (Eq. 8) are a typographical duplication of the same formula, not a case of a fitted input being renamed as a prediction. The self-attention patch size and loss-function choices were compared on the evaluation set (Tables 1 and 2), which is a selection-bias or reporting-validity concern, not circularity under the enumerated definitions. No load-bearing argument relies on a self-citation or on an imported uniqueness theorem; references to prior GAN methods are contextual comparisons, and Section 4.8's admission that previous methods were not fully reimplemented weakens the comparative claim but does not make the paper's own result identical to its assumptions. The paper is self-contained as an empirical study, so the honest finding is no significant circularity.
Assumptions & free parameters
free parameters (3)
- Loss weighting hyperparameters alpha, beta, gamma =
not reported
- Self-attention patch size =
4x4x4
- Number of training epochs =
1000
assumptions (3)
- domain assumption Gray matter (GM) masks of sMRI contain sufficient information to predict FDG-PET uptake.
- domain assumption ADNI sMRI and FDG-PET pairs are correctly co-registered and preprocessed into a common 64x64x64 space so that voxel-wise losses are meaningful.
- domain assumption A single train/test split exists and is consistent across all compared methods.
Cite this review
Pith. "Pith review of Cross-modal Medical Image Generation Based on Pyramid Convolutional Attention Network." pith.science (2026). https://pith.science/paper/CJ2UXMSQ
@misc{pith2026241117420,
author = {Pith},
title = {Pith review of: Cross-modal Medical Image Generation Based on Pyramid Convolutional Attention Network},
year = {2026},
howpublished = {\url{https://pith.science/paper/CJ2UXMSQ}},
note = {Machine review of arXiv:2411.17420}
}
read the original abstract
The integration of multimodal medical imaging can provide complementary and comprehensive information for the diagnosis of Alzheimer's disease (AD). However, in clinical practice, since positron emission tomography (PET) is often missing, multimodal images might be incomplete. To address this problem, we propose a method that can efficiently utilize structural magnetic resonance imaging (sMRI) image information to generate high-quality PET images. Our generation model efficiently utilizes pyramid convolution combined with channel attention mechanism to extract multi-scale local features in sMRI, and injects global correlation information into these features using self-attention mechanism to ensure the restoration of the generated PET image on local texture and global structure. Additionally, we introduce additional loss functions to guide the generation model in producing higher-quality PET images. Through experiments conducted on publicly available ADNI databases, the generated images outperform previous research methods in various performance indicators (average absolute error: 0.0194, peak signal-to-noise ratio: 29.65, structural similarity: 0.9486) and are close to real images. In promoting AD diagnosis, the generated images combined with their corresponding sMRI also showed excellent performance in AD diagnosis tasks (classification accuracy: 94.21 %), and outperformed previous research methods of the same type. The experimental results demonstrate that our method outperforms other competing methods in quantitative metrics, qualitative visualization, and evaluation criteria.
Reference graph
Works this paper leans on
-
[24]
In: Seminars in Cell & Developmental Biology, vol
Fenlon, L.R., Suarez, R., Lynton, Z., Richards, L.J.: The evolution, formation and connectivity of the anterior commissure. In: Seminars in Cell & Developmental Biology, vol. 118, pp. 50–59 (2021). Elsevier
work page 2021
-
[27]
Pattern Recognition Letters 152, 210–217 (2021)
Chen, Z., Zhao, S., Hu, K., Han, J., Ji, Y., Ling, S., Gao, X.: A hierarchical and multi-view registration of serial histopathological images. Pattern Recognition Letters 152, 210–217 (2021)
work page 2021
-
[28]
Brain Structure and Function 228(5), 1045–1067 (2023) 18
He, C., Guan, X., Zhang, W., Li, J., Liu, C., Wei, H., Xu, X., Zhang, Y.: Quantitative susceptibility atlas construction in montreal neurological institute space: towards histological-consistent iron-rich deep brain nucleus subregion identification. Brain Structure and Function 228(5), 1045–1067 (2023) 18
work page 2023
-
[1]
Neurocomputing 392, 296–304 (2020)
Bi, X., Li, S., Xiao, B., Li, Y., Wang, G., Ma, X.: Computer aided alzheimer’s disease diagnosis by an unsupervised deep learning technology. Neurocomputing 392, 296–304 (2020)
work page 2020
-
[2]
Frontiers in bioengineering and biotechnology 8, 534592 (2020) 15
Folego, G., Weiler, M., Casseb, R.F., Pires, R., Rocha, A.: Alzheimer’s dis- ease detection through whole-brain 3d-cnn mri. Frontiers in bioengineering and biotechnology 8, 534592 (2020) 15
work page 2020
-
[3]
Liu, M., Li, F., Yan, H., Wang, K., Ma, Y., Shen, L., Xu, M., Initiative, A.D.N.,et al.: A multi-model deep convolutional neural network for automatic hippocampus segmentation and classification in alzheimer’s disease. Neuroimage 208, 116459 (2020)
work page 2020
-
[4]
Biomedical Signal Processing and Control 80, 104312 (2023)
Rallabandi, V.S., Seetharaman, K.: Deep learning-based classification of healthy aging controls, mild cognitive impairment and alzheimer’s disease using fusion of mri-pet imaging. Biomedical Signal Processing and Control 80, 104312 (2023)
work page 2023
-
[5]
Nature communications 13(1), 3404 (2022)
Qiu, S., Miller, M.I., Joshi, P.S., Lee, J.C., Xue, C., Ni, Y., Wang, Y., De Anda- Duran, I., Hwang, P.H., Cramer, J.A., et al.: Multimodal deep learning for alzheimer’s disease dementia assessment. Nature communications 13(1), 3404 (2022)
work page 2022
Show all 28 references
-
[6]
Machine Learning and Knowledge Extraction 5(2), 512–538 (2023)
Shukla, A., Tiwari, R., Tiwari, S.: Alzheimer’s disease detection from fused pet and mri modalities using an ensemble classifier. Machine Learning and Knowledge Extraction 5(2), 512–538 (2023)
2023
-
[7]
Journal of clinical epidemiology 59(10), 1087–1091 (2006)
Donders, A.R.T., Van Der Heijden, G.J., Stijnen, T., Moons, K.G.: A gentle introduction to imputation of missing values. Journal of clinical epidemiology 59(10), 1087–1091 (2006)
2006
-
[8]
In: Medi- cal Image Computing and Computer-Assisted Intervention–MICCAI 2014: 17th International Conference, Boston, MA, USA, September 14-18, 2014, Proceedings, Part III 17, pp
Li, R., Zhang, W., Suk, H.-I., Wang, L., Li, J., Shen, D., Ji, S.: Deep learning based imaging data completion for improved brain disease diagnosis. In: Medi- cal Image Computing and Computer-Assisted Intervention–MICCAI 2014: 17th International Conference, Boston, MA, USA, Se...
2014
-
[9]
Sikka, A., Peri, S.V., Bathula, D.R.: Mri to fdg-pet: cross-modal synthesis using 3d u-net for multi-modal alzheimer’s classification. In: Simulation and Synthesis in Medical Imaging: Third International Workshop, SASHIMI 2018, Held in Con- junction with MICCAI 2018, Granada, ...
2018
-
[10]
IEEE journal of biomedical and health informatics 24(1), 39–49 (2019)
Gao, F., Wu, T., Chu, X., Yoon, H., Xu, Y., Patel, B.: Deep residual incep- tion encoder–decoder network for medical imaging synthesis. IEEE journal of biomedical and health informatics 24(1), 39–49 (2019)
2019
-
[11]
In: 2019 International Conference on Medical Imaging Physics and Engineering (ICMIPE), pp
Hu, S., Yuan, J., Wang, S.: Cross-modality synthesis from mri to pet using adver- sarial u-net with different normalization. In: 2019 International Conference on Medical Imaging Physics and Engineering (ICMIPE), pp. 1–5 (2019). IEEE
2019
-
[12]
In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp
Isola, P., Zhu, J.-Y., Zhou, T., Efros, A.A.: Image-to-image translation with condi- tional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1125–1134 (2017)
2017
-
[13]
Communications of the ACM 63(11), 139–144 (2020)
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, 16 S., Courville, A., Bengio, Y.: Generative adversarial networks. Communications of the ACM 63(11), 139–144 (2020)
2020
-
[14]
IEEE transactions on medical imaging 37(6), 1348–1357 (2018)
Yang, Q., Yan, P., Zhang, Y., Yu, H., Shi, Y., Mou, X., Kalra, M.K., Zhang, Y., Sun, L., Wang, G.: Low-dose ct image denoising using a generative adversarial network with wasserstein distance and perceptual loss. IEEE transactions on medical imaging 37(6), 1348–1357 (2018)
2018
-
[15]
Wolterink, J.M., Dinkla, A.M., Savenije, M.H., Seevinck, P.R., Berg, C.A., Iˇ sgum, I.: Deep mr to ct synthesis using unpaired data. In: Simulation and Synthesis in Medical Imaging: Second International Workshop, SASHIMI 2017, Held in Conjunction with MICCAI 2017, Qu´ ebec Cit...
2017
-
[16]
In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2018: 21st International Conference, Granada, Spain, September 16-20, 2018, Proceedings, Part II 11, pp
Jiang, J., Hu, Y.-C., Tyagi, N., Zhang, P., Rimner, A., Mageras, G.S., Deasy, J.O., Veeraraghavan, H.: Tumor-aware, adversarial domain adaptation from ct to mri for lung cancer segmentation. In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2018: 21st Inter...
2018
-
[17]
arXiv preprint arXiv:1411.1784 (2014)
Mirza, M.: Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784 (2014)
2014 arXiv
-
[18]
In: Proceedings of the IEEE International Conference on Computer Vision, pp
Zhu, J.-Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image transla- tion using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2223–2232 (2017)
2017
-
[19]
IEEE Transactions on Medical Imaging 41(1), 145–157 (2021)
Hu, S., Lei, B., Wang, S., Wang, Y., Feng, Z., Shen, Y.: Bidirectional mapping generative adversarial networks for brain mr to pet synthesis. IEEE Transactions on Medical Imaging 41(1), 145–157 (2021)
2021
-
[20]
Frontiers in Neuroscience 15, 646013 (2021)
Lin, W., Lin, W., Chen, G., Zhang, H., Gao, Q., Huang, Y., Tong, T., Du, M., Initiative, A.D.N.: Bidirectional mapping of brain mri and pet with 3d reversible gan for the diagnosis of alzheimer’s disease. Frontiers in Neuroscience 15, 646013 (2021)
2021
-
[21]
arXiv preprint arXiv:2108.02160 (2021)
Sikka, A., Virk, J.S., Bathula, D.R., et al.: Mri to pet cross-modality transla- tion using globally and locally aware gan (gla-gan) for multi-modal diagnosis of alzheimer’s disease. arXiv preprint arXiv:2108.02160 (2021)
2021 arXiv
-
[22]
IEEE journal of biomedical and health informatics 26(1), 36–43 (2021)
Gao, X., Shi, F., Shen, D., Liu, M.: Task-induced pyramid and attention gan for multimodal brain image imputation and classification in alzheimer’s disease. IEEE journal of biomedical and health informatics 26(1), 36–43 (2021)
2021
-
[23]
In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2018: 21st International Conference, Granada, Spain, September 16-20, 2018, Proceedings, Part III 11, pp
Pan, Y., Liu, M., Lian, C., Zhou, T., Xia, Y., Shen, D.: Synthesizing miss- ing pet from mri with cycle-consistent generative adversarial networks for 17 alzheimer’s disease diagnosis. In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2018: 21st Internation...
2018
-
[25]
Neuroimage 237, 118206 (2021)
Iglesias, J.E., Billot, B., Balbastre, Y., Tabari, A., Conklin, J., Gonz´ alez, R.G., Alexander, D.C., Golland, P., Edlow, B.L., Fischl, B.,et al.: Joint super-resolution and synthesis of 1 mm isotropic mp-rage volumes from clinical mri exams with scans of different orientatio...
2021
-
[26]
Pediatric Radiology 51, 2009–2017 (2021)
Tabari, A., Conklin, J., Figueiro Longo, M.G., Jaimes, C., Setsompop, K., Cauley, S.F., Kirsch, J.E., Huang, S.Y., Rapalino, O., Gee, M.S., et al.: Comparison of ultrafast wave-controlled aliasing in parallel imaging (caipi) magnetization- prepared rapid acquisition gradient e...
2021
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.