Pith. sign in

REVIEW 5 major objections 4 minor 20 references

LegoPET: Hierarchical Feature Guided Conditional Diffusion for PET Image Reconstruction

T0 review · 5 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Pith's one-line read LegoPET injects hierarchical learned features into a sinogram-conditioned diffusion model to improve PET reconstruction quality and training efficiency.

desk verdict LegoPET is a plausible incremental method for diffusion-based PET reconstruction, but its headline gain over cDPM is not cleanly attributable to the proposed feature guidance because the main comparison and the ablation use different inference settings. read the letter →

arxiv 2411.16629 v1 pith:RG5C4YDI submitted 2024-11-25 eess.IV cs.CV

classification eess.IVcs.CV
keywords PETreconstructionconditionaldiffusionmodelshierarchicalfeatureguidancesinogram-to-imageclassifier-freeplug-and-playpriorhigh-frequencydetailgenerativemedicalimaging
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tackles a specific failure mode of conditional diffusion models (cDPMs) in PET: when the conditioning input is a sinogram and the target is an image, standard cDPMs struggle to keep the output aligned with the input, tend to miss high-frequency detail, and train slowly. LegoPET's proposal is to train a separate U-Net (PnPNet) on sinogram-PET pairs, then add its multilevel feature maps as biases into a standard cDPM's encoder and middle blocks. On simulated 2D FDG brain-slice data, the paper reports that this guidance lifts PSNR/SSIM from 27.00 dB/0.945 for a vanilla cDPM to 27.59 dB/0.956, yields the most visually realistic reconstructions among the compared methods, and shows strong performance even early in training. The purpose is to show that structured feature guidance is a cheap, effective upgrade for diffusion-based PET reconstruction.

What carries the argument

PnPNet is a conditional convolutional U-Net using the same backbone as the diffusion denoiser, trained end-to-end with $\mathcal{L}_{MSE} + 0.1\,\mathcal{L}_{DWT}$ so that its latent features include high-frequency information. From its latent space, two lists of feature maps, $b_d$ from downsampling blocks and $b_m$ from middle blocks, are added as extra biases to the corresponding cDPM blocks. The diffusion model is trained with the standard noise-prediction objective and classifier-free guidance, randomly dropping conditioning information with probability $p_{dp}$; at inference it denoises with the extrapolated prediction $\tilde{\epsilon}_\theta = (1+\lambda_2)\epsilon_\theta(x_t,t,C) - \lambda_2\epsilon_\theta(x_t,t)$. The role of this machinery is to give the denoiser structured prior information at every scale instead of a single-domain conditioning signal.

What would settle it

Run the LegoPET-versus-cDPM comparison over at least five independent training seeds and report the mean and standard deviation of PSNR/SSIM; if the 0.59 dB gap falls within one standard deviation of the two methods' distributions, the claimed advantage is not established. A second test is to evaluate both methods on held-out clinical or 3D PET volumes and check whether the visual and SSIM gains persist outside the simulated 2D phantom setup.

Watch

Extended reading notes

Core claim

LegoPET's central claim is that a diffusion model conditioned only on a sinogram benefits measurably from being biased, at multiple resolutions, by features extracted from a pretrained regression network. PnPNet is trained with an MSE loss plus a discrete-wavelet-transform loss that emphasizes the high-frequency spectrum, so its downsampling and middle-block features carry both coarse anatomy and sharp edges. Added as biases to the cDPM's encoder and middle blocks, these features act as a multiscale prior that keeps the reconstructed image consistent with the sinogram and preserves fine detail. The paper reports the best SSIM among all compared methods (0.956 vs 0.945 for cDPM), a 0.59 dB PSNR improvement over cDPM, and visually cleaner images than DeepPET, cGAN, Pix2Pix, and cDPM, all with the same 35.71 M parameter count as the vanilla cDPM.

Load-bearing premise

The load-bearing premise is that the 0.59 dB PSNR and 0.011 SSIM gains measured on 550 simulated 2D FDG brain slices from two phantom volumes reflect a real, reproducible improvement rather than run-to-run variation, and that they will carry over to 3D and patient data.

Editorial extensions

If this is right

  • Because LegoPET matches cDPM's 35.71 M parameter count, the reported gains are attributed to the injected features rather than to added capacity, so the method is a drop-in guidance upgrade.
  • The ablation study shows LegoPET outperforming cDPM at every checkpoint across 500 epochs, which implies the feature biases accelerate convergence and may allow shorter training schedules.
  • The highest SSIM and best visual quality among the compared methods position feature-guided diffusion as a baseline for future sinogram-to-PET reconstruction work.
  • The DWT loss in PnPNet training ties edge and texture preservation directly to the guidance, so the method's high-frequency advantage is a designed consequence rather than an accident.
  • The paper states that extending the model to an efficient 3D version and validating on patient data are the next steps, meaning clinical relevance depends on those results.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the PSNR/SSIM gains survive repeated training runs, the same feature-bias injection could transfer to other cross-domain inverse problems where conditioning and output live in different spaces, such as low-dose CT or MRI reconstruction.
  • The discrete-wavelet-transform term in PnPNet's loss is the most likely carrier of the edge-preservation benefit; an ablation that removes only that term would isolate its contribution, and the paper does not report one.
  • Because the evaluation is slice-wise on simulated 2D phantoms, the design leaves axial correlations unexploited; a 3D extension could either amplify the gains through richer context or erode them through higher memory and data demands.
  • The identical parameter count suggests the improvement is geometric rather than capacity-driven, implying the guidance could potentially be distilled into a single network to avoid running PnPNet at inference, which the paper does not explore.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The manuscript proposes LegoPET, a conditional diffusion model for PET image reconstruction from sinograms, augmented by hierarchical features extracted from a separately trained convolutional U-Net (PnPNet). The features are injected as biases into the diffusion U-Net, and training uses classifier-free guidance with random conditioning dropout. Experiments on simulated 2D 18F-FDG BrainWeb slices report PSNR/SSIM improvements over a cDPM baseline and over cGAN/Pix2Pix, with visual quality claimed to be better than that of DeepPET. An ablation compares LegoPET with cDPM over 500 epochs with classifier-free guidance disabled.

Significance. If the reported gains are robust, the work offers a simple recipe for injecting learned multiscale image priors into cDPMs for sinogram-to-image reconstruction, with potential applicability to other inverse problems. The code is publicly available, and the ablation directly addresses the contribution of feature guidance separately from classifier-free guidance. However, the significance is currently limited by the absence of statistical analysis, incomplete hyperparameter reporting, and an overstated comparison to DeepPET on PSNR.

major comments (5)
  1. [Section 3.4, Table 1] The abstract and Section 1 claim that LegoPET 'surpasses recent DL-based PET image reconstruction techniques in terms of visual quality and pixel-level PSNR/SSIM metrics,' but Table 1 shows DeepPET achieves higher PSNR (28.30 dB) than LegoPET (27.59 dB). The text in Section 3.4 correctly softens this to 'comparable PSNR values,' but the abstract and introduction should be revised to avoid a claim that the table contradicts.
  2. [Sections 3.2 and 3.5] The paper reports that LegoPET models were trained with pdp in {0, 0.1, 0.2, 0.5} and that inference uses a guidance scale lambda_2, but it never states which pdp and lambda_2 produced the Table 1 results. Because the ablation in Section 3.5 disables classifier-free guidance (pdp = 0, and presumably lambda_2 = 0) on both sides, the 0.59 dB PSNR gain in Table 1 cannot be attributed to the PnPNet feature guidance; it could be driven by the classifier-free guidance hyperparameters. The final pdp and lambda_2 values, and an ablation with guidance enabled on both cDPM and LegoPET, must be reported.
  3. [Sections 3.4 and 3.5] All quantitative claims rest on a single evaluation over 550 slices from two BrainWeb phantoms, with no error bars, multiple training runs, or significance tests. The 0.59 dB PSNR improvement over cDPM and the early-epoch advantage shown in Fig. 3 may be within slice-to-slice or run-to-run variation. Report per-subject metrics and means with standard deviations or confidence intervals, and ideally results from multiple seeds.
  4. [Sections 2.1 and 2.2] The mechanism for injecting bd and bm into the cDPM is described only as adding them 'as extra biases' to the encoder and middle blocks. No equation specifies how the feature maps are transformed, at which resolutions they are added, whether the PnPNet is frozen during diffusion fine-tuning, or how the layers are time-conditioned. This underspecification prevents reproduction and should be formalized.
  5. [Table 1] LegoPET is reported with the same 35.71 M parameters as cDPM, but LegoPET also contains the pretrained PnPNet. If the PnPNet parameters are excluded from the count, the table is misleading and the efficiency comparison with baselines is incomplete; clarify whether PnPNet parameters are included or excluded and report both trainable and total parameter counts.
minor comments (4)
  1. [Abstract and Introduction] There are grammatical errors and typographical artifacts, such as 'challenges such as maintain correspondence and consistency' and 'hierarchicaL' in the abstract; these should be corrected.
  2. [Figure 2] The statement that LegoPET has 'the second lightest color in the squared error maps' is ambiguous without a colorbar or scale; please add a colorbar or describe the error maps quantitatively.
  3. [Section 3.3] The baselines are said to be trained 'until full convergence,' but no convergence criterion, number of epochs, or early-stopping rule is provided; please report these details for each baseline.
  4. [References] Reference [18] is a conference abstract without a DOI or full peer-reviewed details; the authors should either provide a complete citation or reduce reliance on it.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: LegoPET's reported improvement over cDPM is an empirical result, not a quantity forced by construction or by self-citation.

full rationale

The derivation chain is empirical rather than definitional. The method trains a standalone PnPNet on sinogram-PET pairs, extracts hierarchical feature maps, and injects them as biases into a sinogram-conditioned diffusion model; the cDPM baseline is the same diffusion formulation without those feature biases. Nowhere does the paper define the reported PSNR/SSIM improvement in terms of the fitted PnPNet outputs: the 0.59 dB gain in Table 1 is measured on held-out BrainWeb slices, not computed from a training loss or from the feature maps. The PnPNet is fit to the training data, but the claim is tested on a separate test set, so this is not a fitted input renamed as a prediction. The self-citation to DiffPET [18] appears only as background ("building on previous work [15, 18, 20]") and is not the load-bearing justification for the central improvement claim; the feature-injection mechanism is attributed to external ControlNet-style work [20] and multiscale guided diffusion [15], and the baselines include independently developed methods (DeepPET, cGAN, Pix2Pix, cDPM). The ablation in Section 3.5 compares cDPM and LegoPET under identical training settings, which isolates the feature-guidance contribution as a controlled comparison. The under-reporting of pdp/lambda2 values for Table 1 and lack of error bars are legitimate reproducibility and statistical-concern issues, but they are not circularity: nothing in the paper's equations reduces the claimed result to its own inputs. Accordingly the appropriate circularity score is 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The method introduces no new physical entities or forces. It relies on standard diffusion theory, a domain assumption about simulated phantoms, and the unproven but plausible hypothesis that regression-network features aid diffusion. The main free parameters are hyperparameters, two of which (lambda_2 and the final pdp) are not fully specified in the paper.

free parameters (3)
  • lambda_1 (DWT loss weight) = 0.1
    Weight for the discrete wavelet transform loss in PnPNet training (Eq. 1); chosen by hand with no sensitivity analysis.
  • pdp (conditioning dropout probability) = {0, 0.1, 0.2, 0.5}
    Dropout probability for classifier-free guidance; the value used for the reported final results is not stated, creating potential selection bias (Section 3.2).
  • lambda_2 (guidance scale) = not reported
    Inference-time guidance coefficient in the denoising expression (Section 2.3); its value is never given, so reproducing the reported results requires guessing it.
assumptions (4)
  • standard math Gaussian forward diffusion process
    Eqs. (2)-(3) define the noising process as in DDPM; treated as standard background.
  • domain assumption 2D slices from simulated phantoms represent PET reconstruction
    The method is trained and evaluated on 2D slices of 18F-FDG BrainWeb phantoms; real patient data are not used (Section 3.1).
  • domain assumption PnPNet features are useful priors for cDPM
    The core hypothesis is that hierarchical features from a regression network trained with MSE and DWT losses help the diffusion model; this is not proven, only empirically asserted (Section 2.1).
  • domain assumption Classifier-free guidance improves fidelity
    The training randomly drops conditioning with probability pdp, following Ho and Salimans [8]; relies on that method's effectiveness (Section 2.3).

how reviews work

0 comments
Cite this review

Pith. "Pith review of LegoPET: Hierarchical Feature Guided Conditional Diffusion for PET Image Reconstruction." pith.science (2026). https://pith.science/paper/RG5C4YDI

@misc{pith2026241116629,
  author       = {Pith},
  title        = {Pith review of: LegoPET: Hierarchical Feature Guided Conditional Diffusion for PET Image Reconstruction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RG5C4YDI}},
  note         = {Machine review of arXiv:2411.16629}
}
read the original abstract

Positron emission tomography (PET) is widely utilized for cancer detection due to its ability to visualize functional and biological processes in vivo. PET images are usually reconstructed from histogrammed raw data (sinograms) using traditional iterative techniques (e.g., OSEM, MLEM). Recently, deep learning (DL) methods have shown promise by directly mapping raw sinogram data to PET images. However, DL approaches that are regression-based or GAN-based often produce overly smoothed images or introduce various artifacts respectively. Image-conditioned diffusion probabilistic models (cDPMs) are another class of likelihood-based DL techniques capable of generating highly realistic and controllable images. While cDPMs have notable strengths, they still face challenges such as maintain correspondence and consistency between input and output images when they are from different domains (e.g., sinogram vs. image domain) as well as slow convergence rates. To address these limitations, we introduce LegoPET, a hierarchical feature guided conditional diffusion model for high-perceptual quality PET image reconstruction from sinograms. We conducted several experiments demonstrating that LegoPET not only improves the performance of cDPMs but also surpasses recent DL-based PET image reconstruction techniques in terms of visual quality and pixel-level PSNR/SSIM metrics. Our code is available at https://github.com/yransun/LegoPET.

Figures

Figures reproduced from arXiv: 2411.16629 by the authors.

Figure 1
Figure 1. Overview of LegoPET. LegoPET includes two parts: a learned PnPNet for hierarchical feature maps extraction (right), and a standard 2D sinogram-conditioned DPM for PET image re￾construction (left). Two lists of structured layered feature maps, bd and bm, from the pre-trained PnPNet are added to the cDPM as biases. the latent space of learned PnPNet. We connect the pre￾trained PnPNet to cDPM by incorporating bd and bm… view at source ↗
Figure 2
Figure 2. Comparison of LegoPET with Four Baselines on Two Example Reconstructed Slices. The first column shows the input sinogram images, and the second column shows the reference images reconstructed using OSEM algorithm. The third to sixth columns correspond to the four baselines (labeled above each image), and the final column shows the reconstructed PET image using proposed LegoPET method. PSNR/SSIM values are reported b… view at source ↗
Figure 3
Figure 3. Effectiveness of Hierarchical Feature Guidance. We compare the performance of LegoPET and cDPM within 500 epochs in terms of PSNR and SSIM. We regard cDPM as “LegoPET w/o guidance”, and LegoPET as “cDPM w/ guidance” [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

20 extracted references · 12 canonical work pages

  1. [1]

    Design and construction of a realistic digi- tal brain phantom

    D Louis Collins, Alex P Zijdenbos, Vasken Kollokian, John G Sled, Noor Jehan Kabani, Colin J Holmes, and Alan C Evans. Design and construction of a realistic digi- tal brain phantom. IEEE transactions on medical imaging , 17(3):463–468, 1998. 3

  2. [2]

    Data acquisition in pet imaging

    Frederic H Fahey. Data acquisition in pet imaging. Journal of nuclear medicine technology, 30(2):39–49, 2002. 1

  3. [3]

    Generative adversarial networks

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Commu- nications of the ACM, 63(11):139–144, 2020. 1

  4. [4]

    Deeppet: A deep encoder–decoder network for directly solving the pet image reconstruction in- verse problem

    Ida H ¨aggstr¨om, C Ross Schmidtlein, Gabriele Campanella, and Thomas J Fuchs. Deeppet: A deep encoder–decoder network for directly solving the pet image reconstruction in- verse problem. Medical image analysis , 54:253–262, 2019. 1, 3, 4

  5. [5]

    Contrastive diffusion model with auxiliary guidance for coarse-to-fine pet reconstruction

    Zeyu Han, Yuhan Wang, Luping Zhou, Peng Wang, Binyu Yan, Jiliu Zhou, Yan Wang, and Dinggang Shen. Contrastive diffusion model with auxiliary guidance for coarse-to-fine pet reconstruction. In International Conference on Medi- cal Image Computing and Computer-Assisted Intervention , pages 239–249. Springer, 2023. 1

  6. [6]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 2

  7. [7]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. 1

  8. [8]

    Classifier-free diffusion guidance

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022. 1, 2, 3, 4

Show all 20 references
  1. [9]

    Image quality metrics: Psnr vs

    Alain Hore and Djemel Ziou. Image quality metrics: Psnr vs. ssim. In 2010 20th international conference on pattern recognition, pages 2366–2369. IEEE, 2010. 3

  2. [10]

    Image-to-image translation with conditional adver- sarial networks

    Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adver- sarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 1125–1134,

  3. [11]

    Photo- realistic single image super-resolution using a generative ad- versarial network

    Christian Ledig, Lucas Theis, Ferenc Husz´ar, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew Aitken, Alykhan Tejani, Johannes Totz, Zehan Wang, et al. Photo- realistic single image super-resolution using a generative ad- versarial network. In Proceedings of the IE...

  4. [12]

    Conditional generative adversarial nets

    Mehdi Mirza and Simon Osindero. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784, 2014. 3, 4

  5. [13]

    The appli- cations of discrete wavelet transform in image processing: A review

    Gheyath Othman and Diyar Qader Zeebaree. The appli- cations of discrete wavelet transform in image processing: A review. Journal of Soft Computing and Data Mining , 1(2):31–43, 2020. 2

  6. [14]

    Pytorch: An im- perative style, high-performance deep learning library

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An im- perative style, high-performance deep learning library. Ad- vances in neural information processing systems ...

  7. [15]

    Multiscale structure guided diffusion for image deblurring

    Mengwei Ren, Mauricio Delbracio, Hossein Talebi, Guido Gerig, and Peyman Milanfar. Multiscale structure guided diffusion for image deblurring. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 10721–10733, 2023. 1, 3

  8. [16]

    Palette: Image-to-image diffusion models

    Chitwan Saharia, William Chan, Huiwen Chang, Chris Lee, Jonathan Ho, Tim Salimans, David Fleet, and Mohammad Norouzi. Palette: Image-to-image diffusion models. In ACM SIGGRAPH 2022 conference proceedings, pages 1–10,

  9. [17]

    Improved techniques for training gans

    Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. Advances in neural information processing systems, 29, 2016. 1

  10. [18]

    Diffpet: A fine tuned sinogram-to-pet conditional diffusion model

    Yiran Sun and Osama R Mawlawi. Diffpet: A fine tuned sinogram-to-pet conditional diffusion model. In AAPM 66th Annual Meeting & Exhibition . AAPM, 2024. 1

  11. [19]

    Improving train- ing efficiency of diffusion models via multi-stage framework and tailored multi-decoder architecture

    Huijie Zhang, Yifu Lu, Ismail Alkhouri, Saiprasad Ravis- hankar, Dogyoon Song, and Qing Qu. Improving train- ing efficiency of diffusion models via multi-stage framework and tailored multi-decoder architecture. In Proceedings of the IEEE/CVF Conference on Computer Vision and P...

  12. [20]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3836–3847, 2023. 1

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.