REVIEW 5 major objections 4 minor 20 references
LegoPET: Hierarchical Feature Guided Conditional Diffusion for PET Image Reconstruction
T0 review · 5 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read LegoPET injects hierarchical learned features into a sinogram-conditioned diffusion model to improve PET reconstruction quality and training efficiency.
desk verdict LegoPET is a plausible incremental method for diffusion-based PET reconstruction, but its headline gain over cDPM is not cleanly attributable to the proposed feature guidance because the main comparison and the ablation use different inference settings. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
PnPNet is a conditional convolutional U-Net using the same backbone as the diffusion denoiser, trained end-to-end with $\mathcal{L}_{MSE} + 0.1\,\mathcal{L}_{DWT}$ so that its latent features include high-frequency information. From its latent space, two lists of feature maps, $b_d$ from downsampling blocks and $b_m$ from middle blocks, are added as extra biases to the corresponding cDPM blocks. The diffusion model is trained with the standard noise-prediction objective and classifier-free guidance, randomly dropping conditioning information with probability $p_{dp}$; at inference it denoises with the extrapolated prediction $\tilde{\epsilon}_\theta = (1+\lambda_2)\epsilon_\theta(x_t,t,C) - \lambda_2\epsilon_\theta(x_t,t)$. The role of this machinery is to give the denoiser structured prior information at every scale instead of a single-domain conditioning signal.
What would settle it
Run the LegoPET-versus-cDPM comparison over at least five independent training seeds and report the mean and standard deviation of PSNR/SSIM; if the 0.59 dB gap falls within one standard deviation of the two methods' distributions, the claimed advantage is not established. A second test is to evaluate both methods on held-out clinical or 3D PET volumes and check whether the visual and SSIM gains persist outside the simulated 2D phantom setup.
Extended reading notes
Core claim
LegoPET's central claim is that a diffusion model conditioned only on a sinogram benefits measurably from being biased, at multiple resolutions, by features extracted from a pretrained regression network. PnPNet is trained with an MSE loss plus a discrete-wavelet-transform loss that emphasizes the high-frequency spectrum, so its downsampling and middle-block features carry both coarse anatomy and sharp edges. Added as biases to the cDPM's encoder and middle blocks, these features act as a multiscale prior that keeps the reconstructed image consistent with the sinogram and preserves fine detail. The paper reports the best SSIM among all compared methods (0.956 vs 0.945 for cDPM), a 0.59 dB PSNR improvement over cDPM, and visually cleaner images than DeepPET, cGAN, Pix2Pix, and cDPM, all with the same 35.71 M parameter count as the vanilla cDPM.
Load-bearing premise
The load-bearing premise is that the 0.59 dB PSNR and 0.011 SSIM gains measured on 550 simulated 2D FDG brain slices from two phantom volumes reflect a real, reproducible improvement rather than run-to-run variation, and that they will carry over to 3D and patient data.
Editorial extensions
If this is right
- Because LegoPET matches cDPM's 35.71 M parameter count, the reported gains are attributed to the injected features rather than to added capacity, so the method is a drop-in guidance upgrade.
- The ablation study shows LegoPET outperforming cDPM at every checkpoint across 500 epochs, which implies the feature biases accelerate convergence and may allow shorter training schedules.
- The highest SSIM and best visual quality among the compared methods position feature-guided diffusion as a baseline for future sinogram-to-PET reconstruction work.
- The DWT loss in PnPNet training ties edge and texture preservation directly to the guidance, so the method's high-frequency advantage is a designed consequence rather than an accident.
- The paper states that extending the model to an efficient 3D version and validating on patient data are the next steps, meaning clinical relevance depends on those results.
Reading between the lines
- If the PSNR/SSIM gains survive repeated training runs, the same feature-bias injection could transfer to other cross-domain inverse problems where conditioning and output live in different spaces, such as low-dose CT or MRI reconstruction.
- The discrete-wavelet-transform term in PnPNet's loss is the most likely carrier of the edge-preservation benefit; an ablation that removes only that term would isolate its contribution, and the paper does not report one.
- Because the evaluation is slice-wise on simulated 2D phantoms, the design leaves axial correlations unexploited; a 3D extension could either amplify the gains through richer context or erode them through higher memory and data demands.
- The identical parameter count suggests the improvement is geometric rather than capacity-driven, implying the guidance could potentially be distilled into a single network to avoid running PnPNet at inference, which the paper does not explore.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes LegoPET, a conditional diffusion model for PET image reconstruction from sinograms, augmented by hierarchical features extracted from a separately trained convolutional U-Net (PnPNet). The features are injected as biases into the diffusion U-Net, and training uses classifier-free guidance with random conditioning dropout. Experiments on simulated 2D 18F-FDG BrainWeb slices report PSNR/SSIM improvements over a cDPM baseline and over cGAN/Pix2Pix, with visual quality claimed to be better than that of DeepPET. An ablation compares LegoPET with cDPM over 500 epochs with classifier-free guidance disabled.
Significance. If the reported gains are robust, the work offers a simple recipe for injecting learned multiscale image priors into cDPMs for sinogram-to-image reconstruction, with potential applicability to other inverse problems. The code is publicly available, and the ablation directly addresses the contribution of feature guidance separately from classifier-free guidance. However, the significance is currently limited by the absence of statistical analysis, incomplete hyperparameter reporting, and an overstated comparison to DeepPET on PSNR.
major comments (5)
- [Section 3.4, Table 1] The abstract and Section 1 claim that LegoPET 'surpasses recent DL-based PET image reconstruction techniques in terms of visual quality and pixel-level PSNR/SSIM metrics,' but Table 1 shows DeepPET achieves higher PSNR (28.30 dB) than LegoPET (27.59 dB). The text in Section 3.4 correctly softens this to 'comparable PSNR values,' but the abstract and introduction should be revised to avoid a claim that the table contradicts.
- [Sections 3.2 and 3.5] The paper reports that LegoPET models were trained with pdp in {0, 0.1, 0.2, 0.5} and that inference uses a guidance scale lambda_2, but it never states which pdp and lambda_2 produced the Table 1 results. Because the ablation in Section 3.5 disables classifier-free guidance (pdp = 0, and presumably lambda_2 = 0) on both sides, the 0.59 dB PSNR gain in Table 1 cannot be attributed to the PnPNet feature guidance; it could be driven by the classifier-free guidance hyperparameters. The final pdp and lambda_2 values, and an ablation with guidance enabled on both cDPM and LegoPET, must be reported.
- [Sections 3.4 and 3.5] All quantitative claims rest on a single evaluation over 550 slices from two BrainWeb phantoms, with no error bars, multiple training runs, or significance tests. The 0.59 dB PSNR improvement over cDPM and the early-epoch advantage shown in Fig. 3 may be within slice-to-slice or run-to-run variation. Report per-subject metrics and means with standard deviations or confidence intervals, and ideally results from multiple seeds.
- [Sections 2.1 and 2.2] The mechanism for injecting bd and bm into the cDPM is described only as adding them 'as extra biases' to the encoder and middle blocks. No equation specifies how the feature maps are transformed, at which resolutions they are added, whether the PnPNet is frozen during diffusion fine-tuning, or how the layers are time-conditioned. This underspecification prevents reproduction and should be formalized.
- [Table 1] LegoPET is reported with the same 35.71 M parameters as cDPM, but LegoPET also contains the pretrained PnPNet. If the PnPNet parameters are excluded from the count, the table is misleading and the efficiency comparison with baselines is incomplete; clarify whether PnPNet parameters are included or excluded and report both trainable and total parameter counts.
minor comments (4)
- [Abstract and Introduction] There are grammatical errors and typographical artifacts, such as 'challenges such as maintain correspondence and consistency' and 'hierarchicaL' in the abstract; these should be corrected.
- [Figure 2] The statement that LegoPET has 'the second lightest color in the squared error maps' is ambiguous without a colorbar or scale; please add a colorbar or describe the error maps quantitatively.
- [Section 3.3] The baselines are said to be trained 'until full convergence,' but no convergence criterion, number of epochs, or early-stopping rule is provided; please report these details for each baseline.
- [References] Reference [18] is a conference abstract without a DOI or full peer-reviewed details; the authors should either provide a complete citation or reduce reliance on it.
Circularity Check
No significant circularity: LegoPET's reported improvement over cDPM is an empirical result, not a quantity forced by construction or by self-citation.
full rationale
The derivation chain is empirical rather than definitional. The method trains a standalone PnPNet on sinogram-PET pairs, extracts hierarchical feature maps, and injects them as biases into a sinogram-conditioned diffusion model; the cDPM baseline is the same diffusion formulation without those feature biases. Nowhere does the paper define the reported PSNR/SSIM improvement in terms of the fitted PnPNet outputs: the 0.59 dB gain in Table 1 is measured on held-out BrainWeb slices, not computed from a training loss or from the feature maps. The PnPNet is fit to the training data, but the claim is tested on a separate test set, so this is not a fitted input renamed as a prediction. The self-citation to DiffPET [18] appears only as background ("building on previous work [15, 18, 20]") and is not the load-bearing justification for the central improvement claim; the feature-injection mechanism is attributed to external ControlNet-style work [20] and multiscale guided diffusion [15], and the baselines include independently developed methods (DeepPET, cGAN, Pix2Pix, cDPM). The ablation in Section 3.5 compares cDPM and LegoPET under identical training settings, which isolates the feature-guidance contribution as a controlled comparison. The under-reporting of pdp/lambda2 values for Table 1 and lack of error bars are legitimate reproducibility and statistical-concern issues, but they are not circularity: nothing in the paper's equations reduces the claimed result to its own inputs. Accordingly the appropriate circularity score is 0.
Assumptions & free parameters
free parameters (3)
- lambda_1 (DWT loss weight) =
0.1
- pdp (conditioning dropout probability) =
{0, 0.1, 0.2, 0.5}
- lambda_2 (guidance scale) =
not reported
assumptions (4)
- standard math Gaussian forward diffusion process
- domain assumption 2D slices from simulated phantoms represent PET reconstruction
- domain assumption PnPNet features are useful priors for cDPM
- domain assumption Classifier-free guidance improves fidelity
Cite this review
Pith. "Pith review of LegoPET: Hierarchical Feature Guided Conditional Diffusion for PET Image Reconstruction." pith.science (2026). https://pith.science/paper/RG5C4YDI
@misc{pith2026241116629,
author = {Pith},
title = {Pith review of: LegoPET: Hierarchical Feature Guided Conditional Diffusion for PET Image Reconstruction},
year = {2026},
howpublished = {\url{https://pith.science/paper/RG5C4YDI}},
note = {Machine review of arXiv:2411.16629}
}
read the original abstract
Positron emission tomography (PET) is widely utilized for cancer detection due to its ability to visualize functional and biological processes in vivo. PET images are usually reconstructed from histogrammed raw data (sinograms) using traditional iterative techniques (e.g., OSEM, MLEM). Recently, deep learning (DL) methods have shown promise by directly mapping raw sinogram data to PET images. However, DL approaches that are regression-based or GAN-based often produce overly smoothed images or introduce various artifacts respectively. Image-conditioned diffusion probabilistic models (cDPMs) are another class of likelihood-based DL techniques capable of generating highly realistic and controllable images. While cDPMs have notable strengths, they still face challenges such as maintain correspondence and consistency between input and output images when they are from different domains (e.g., sinogram vs. image domain) as well as slow convergence rates. To address these limitations, we introduce LegoPET, a hierarchical feature guided conditional diffusion model for high-perceptual quality PET image reconstruction from sinograms. We conducted several experiments demonstrating that LegoPET not only improves the performance of cDPMs but also surpasses recent DL-based PET image reconstruction techniques in terms of visual quality and pixel-level PSNR/SSIM metrics. Our code is available at https://github.com/yransun/LegoPET.
Figures
Reference graph
Works this paper leans on
-
[1]
Design and construction of a realistic digi- tal brain phantom
D Louis Collins, Alex P Zijdenbos, Vasken Kollokian, John G Sled, Noor Jehan Kabani, Colin J Holmes, and Alan C Evans. Design and construction of a realistic digi- tal brain phantom. IEEE transactions on medical imaging , 17(3):463–468, 1998. 3
work page 1998
-
[2]
Data acquisition in pet imaging
Frederic H Fahey. Data acquisition in pet imaging. Journal of nuclear medicine technology, 30(2):39–49, 2002. 1
work page 2002
-
[3]
Generative adversarial networks
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Commu- nications of the ACM, 63(11):139–144, 2020. 1
2020
-
[4]
Ida H ¨aggstr¨om, C Ross Schmidtlein, Gabriele Campanella, and Thomas J Fuchs. Deeppet: A deep encoder–decoder network for directly solving the pet image reconstruction in- verse problem. Medical image analysis , 54:253–262, 2019. 1, 3, 4
work page 2019
-
[5]
Contrastive diffusion model with auxiliary guidance for coarse-to-fine pet reconstruction
Zeyu Han, Yuhan Wang, Luping Zhou, Peng Wang, Binyu Yan, Jiliu Zhou, Yan Wang, and Dinggang Shen. Contrastive diffusion model with auxiliary guidance for coarse-to-fine pet reconstruction. In International Conference on Medi- cal Image Computing and Computer-Assisted Intervention , pages 239–249. Springer, 2023. 1
work page 2023
-
[6]
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 2
2016
-
[7]
Denoising dif- fusion probabilistic models
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. 1
2020
-
[8]
Classifier-free diffusion guidance
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022. 1, 2, 3, 4
arXiv 2022
Show all 20 references
-
[9]
Image quality metrics: Psnr vs
Alain Hore and Djemel Ziou. Image quality metrics: Psnr vs. ssim. In 2010 20th international conference on pattern recognition, pages 2366–2369. IEEE, 2010. 3
2010
-
[10]
Image-to-image translation with conditional adver- sarial networks
Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adver- sarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 1125–1134,
-
[11]
Photo- realistic single image super-resolution using a generative ad- versarial network
Christian Ledig, Lucas Theis, Ferenc Husz´ar, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew Aitken, Alykhan Tejani, Johannes Totz, Zehan Wang, et al. Photo- realistic single image super-resolution using a generative ad- versarial network. In Proceedings of the IE...
-
[12]
Conditional generative adversarial nets
Mehdi Mirza and Simon Osindero. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784, 2014. 3, 4
2014 arXiv
-
[13]
The appli- cations of discrete wavelet transform in image processing: A review
Gheyath Othman and Diyar Qader Zeebaree. The appli- cations of discrete wavelet transform in image processing: A review. Journal of Soft Computing and Data Mining , 1(2):31–43, 2020. 2
2020
-
[14]
Pytorch: An im- perative style, high-performance deep learning library
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An im- perative style, high-performance deep learning library. Ad- vances in neural information processing systems ...
2019
-
[15]
Multiscale structure guided diffusion for image deblurring
Mengwei Ren, Mauricio Delbracio, Hossein Talebi, Guido Gerig, and Peyman Milanfar. Multiscale structure guided diffusion for image deblurring. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 10721–10733, 2023. 1, 3
2023
-
[16]
Palette: Image-to-image diffusion models
Chitwan Saharia, William Chan, Huiwen Chang, Chris Lee, Jonathan Ho, Tim Salimans, David Fleet, and Mohammad Norouzi. Palette: Image-to-image diffusion models. In ACM SIGGRAPH 2022 conference proceedings, pages 1–10,
2022
-
[17]
Improved techniques for training gans
Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. Advances in neural information processing systems, 29, 2016. 1
2016
-
[18]
Diffpet: A fine tuned sinogram-to-pet conditional diffusion model
Yiran Sun and Osama R Mawlawi. Diffpet: A fine tuned sinogram-to-pet conditional diffusion model. In AAPM 66th Annual Meeting & Exhibition . AAPM, 2024. 1
2024
-
[19]
Improving train- ing efficiency of diffusion models via multi-stage framework and tailored multi-decoder architecture
Huijie Zhang, Yifu Lu, Ismail Alkhouri, Saiprasad Ravis- hankar, Dogyoon Song, and Qing Qu. Improving train- ing efficiency of diffusion models via multi-stage framework and tailored multi-decoder architecture. In Proceedings of the IEEE/CVF Conference on Computer Vision and P...
2024
-
[20]
Adding conditional control to text-to-image diffusion models
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3836–3847, 2023. 1
2023
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.