Pith. sign in

REVIEW 3 major objections 6 minor 23 references

Spatiotemporal PET reconstruction using ML-EM with learned diffeomorphic deformation

T0 review · 3 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A PET motion-correction algorithm that costs about the same as standard ML-EM, recovering 46% of the maximal image-quality gain from using all gated data.

desk verdict A clean algorithmic contribution combining M-ML-EM with learned registration, but the single synthetic evaluation is too thin to support the scalability claim as stated. read the letter →

arxiv 1908.09515 v1 pith:BZFKQOJZ submitted 2019-08-26 eess.IV cs.CVcs.LG

Ozan Öktem , Camille Pouchol , Olivier Verdier This is my paper · ORCID
classification eess.IVcs.CVcs.LG
keywords PETreconstructionmotioncorrectiongatedimagingML-EMdiffeomorphicregistrationdeeplearningunsupervisedtraining
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes a joint reconstruction-and-motion-estimation method for gated PET that combines an enhanced ML-EM update with a pre-trained deep-learning registration network. The central claim is that one can correct for inter-gate motion while keeping the per-iteration cost of the order of the standard ML-EM algorithm, and that this yields a significant reduction in noise compared to using only a single gate. On a Derenzo phantom with four gates, the method gains 1.0 dB in PSNR over the best uncorrected single-gate reconstruction, capturing about 46% of the 2.2 dB gain that would be achieved with no motion at all. The authors emphasize that the training is unsupervised, uses synthetic data, and that the approach is intended to scale to 3D and higher resolutions.

What carries the argument

The central object is the compound ML-EM update for multiple gates, f^(n+1)_0 = f^(n)_0 / (Σ_i A_i^T 1) Σ_i A_i^T ( g_i / (A_i f^(n)_0 ) ), where A_i = A W_φ_i and φ_i is the composition of learned diffeomorphisms mapping gate 0 to gate i. The motion between consecutive gates is estimated by a pre-trained voxelmorph-style network that maps a pair of images to a stationary vector field v, with the diffeomorphism given by exp(v); the network is trained unsupervised on synthetic images of random ellipsoids deformed by Gaussian random vector fields. Crucially, the cost of this network evaluation is negligible compared to the ML-EM iterations, and the transposed operator A_i^T is computed through the identity W_φ^T = ~W_{$φ^{{-1}}$} using the mass-preserving action.

What would settle it

Run the same pipeline on a digital phantom whose deformations come from a different distribution (e.g., a realistic breathing or cardiac motion model) and check whether the PSNR gain of roughly 1.0 dB persists; if the gain drops to near zero or the reconstruction degrades, the learned registration component is the bottleneck.

Watch

Extended reading notes

Core claim

The paper's central claim is that spatiotemporal PET reconstruction can be performed by alternating a motion-estimation step, where a learned diffeomorphic registration network matches consecutive gate images, with a generalized ML-EM reconstruction step, where all gated data are combined through a compound forward operator. The key finding is that this joint scheme, called M-ML-EM, recovers a substantial fraction of the information lost when reconstructing from a single gate, while its computational cost is only marginally greater than that of standard ML-EM. Concretely, with four gates of the Derenzo phantom, the optimal strategy of six initial ML-EM iterations followed by 42 M-ML-EM iterations yields a PSNR gain of 1.0 dB over the best single-gate reconstruction, which is about 46% of the 2.2 dB gain obtainable if there were no motion. The authors also note that one outer iteration of their algorithm closely resembles a previously proposed deep-learning-based motion-correction method, and that their framework generalizes it as an optimization problem.

Load-bearing premise

The pre-trained registration network, trained on synthetic images of random ellipsoids deformed by Gaussian random vector fields, must register the actual test phantom accurately; the paper only validates deformations drawn from the same distribution used for training.

Editorial extensions

If this is right

  • The per-iteration cost of M-ML-EM scales linearly with the number of gates, without any additional burden from motion estimation, so the method could plausibly handle the large data volumes of clinical 3D gated PET.
  • The framework naturally accommodates other deformations, such as the mass-preserving action, and can incorporate attenuation correction, extending the approach to more physically realistic settings.
  • Because the method recovers a large fraction of the single-gate quality loss, it could reduce the need for long acquisitions or multiple gates, potentially lowering patient scan time or radiation dose without sacrificing image quality.
  • The fact that a single outer iteration captures most of the improvement suggests that a simple two-stage pipeline—reconstruct each gate, register, then jointly refine—is nearly optimal, simplifying implementation.
  • The variational formulation provides a principled interpretation of prior deep-learning-based motion-correction heuristics, which could guide further algorithmic improvements.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The method's practical value depends on whether the synthetic-training regime—random ellipsoids deformed by Gaussian random fields—transfers to real anatomical motion; the paper validates only in-distribution deformations, and a natural test is to apply the same pipeline to clinical or realistic digital phantoms with known motion.
  • Because the network is trained on intensity-preserving deformations, applying the method to real PET data, where activity changes with tracer uptake as well as motion, may require switching to the mass-preserving action or retraining on time-varying activity; the paper identifies this as future work.
  • The reported 1.0 dB gain is measured against the best single-gate reconstruction; aggregating the same data with simple averaging can also recover some information, so comparing M-ML-EM against that alternative would clarify where the gain actually comes from.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes a joint motion estimation and reconstruction algorithm for gated PET. It alternates between a motion estimation step, in which deformations between gate images are obtained by a pretrained voxelmorph network (trained on synthetic ellipsoid images deformed by Gaussian random vector fields), and a reconstruction step, in which all gates are combined through a modified ML-EM update (M-ML-EM) that incorporates the estimated deformations via composite forward operators. The main claims are that the method yields a 1.0 dB PSNR improvement over gate-zero-only ML-EM on a Derenzo phantom, recovering about 46% of the theoretical 2.2 dB gain from using four times the acquisition time, at a computational cost only marginally greater than that of standard ML-EM.

Significance. The algorithmic derivation is a genuine contribution: the M-ML-EM update (10) is a valid extension of ML-EM, the adjoint identity W_phi^T = ~W_{phi^-1} is correct and clearly stated, and the complexity argument in Section 2.6 convincingly shows that the per-iteration cost remains comparable to standard ML-EM. These properties are important and, if the registration was reliable, would make the method attractive for scaling to 3D and clinical 4D PET. The paper also deserves credit for being self-contained in its derivation and for using unsupervised training on synthetic data, which avoids the need for paired clinical motion fields. However, the experimental evidence is currently too narrow to support the headline claims: it consists of a single synthetic phantom, a single noise level, and test deformations drawn from the same distribution used to train the network. The claimed robustness is stated but not quantified by error bars or repeated trials, and the input distribution shift between clean training images and noisy reconstructions is not analyzed.

major comments (3)
  1. [Section 3.3, Figure 4] The reported 1.0 dB gain is obtained by selecting the iteration counts (6 initial ML-EM iterations and 42 M-ML-EM iterations) that maximize PSNR on the test data. No error bars or repeated realizations are provided, despite the sentence in Section 3.3 claiming that the improvement and the optimal iteration counts are 'extremely robust' to the random deformations and Poisson noise. Without a distribution of PSNR values over multiple independent draws of deformations and noise, the 1.0 dB gain could be an optimistic point estimate. Please report mean and standard deviation, and if possible select the iteration counts on a separate validation set.
  2. [Section 2.3 and Algorithm 1] The pretrained voxelmorph network is trained on clean synthetic ellipsoid images (f1 clean, f2 = W_exp(v) f1), but in Algorithm 1 it is deployed on images that are outputs of only n_init = 6 ML-EM iterations on Poisson-corrupted data. This clean-to-noisy input distribution shift is not addressed, and the deformations estimated from noisy inputs are inserted into the M-ML-EM update (10) and fully trusted. If the network registers noise patterns rather than anatomical structure, the resulting bias propagates directly into the reconstructed f0. Please provide evidence that registration remains accurate on noisy reconstructions, for example by evaluating registration error on ML-EM outputs at varying iteration counts or noise levels, and report results for at least one additional noise level (e.g., t = 30 or t = 120) to show that the gain does not vanish in the low-count regime that motivates the method.
  3. [Section 3.1 and Section 4] The test deformations are drawn from the same distribution used to train the network, so the validation is entirely in-distribution. The paper acknowledges in Section 4 that clinical data and the mass-preserving action are future work, but the claimed 'significantly decrease the noise' (abstract) is based solely on this in-distribution test. Since the method's central component is a learned registration network, a demonstration on out-of-distribution deformations (for example, vector fields with different smoothness scales, or deformations mimicking cardiac or respiratory motion) is needed to assess whether the 1.0 dB gain is an artifact of matching the training distribution or a substantive property of the algorithm.
minor comments (6)
  1. [Section 2.4] The adjoint identity W_phi^T = ~W_{phi^-1} is stated without proof or citation; a short derivation or a reference to a standard text would help the reader verify this load-bearing step.
  2. [Section 2.5, Algorithm 1] The notation 'W_i <- W_{psi_i} W_{i-1}' is ambiguous about composition order; please clarify whether this means W_i = W_{psi_i} ∘ W_{i-1} and how it relates to the definition of φ_i = ψ_i ∘ ... ∘ ψ_1.
  3. [Section 3.2, Figure 3] The caption should state explicitly that PSNR is computed against the gate-zero phantom and should indicate the number of iterations used for the 'no-movement' reference reconstruction.
  4. [Abstract and Section 3.3] The phrase 'significantly decrease the noise' is vague; consider quantifying the claim as 'reduce noise, as measured by PSNR, by 1.0 dB in a synthetic Derenzo phantom study' to avoid over-generalization.
  5. [Section 1.2] The statement that 'one single outer iteration of our algorithm is close to the recently proposed approach [17]' is not elaborated; please specify what [17] does and how it differs from the proposed method.
  6. [Section 2.1] The intensity-preserving action W_ψ is defined in the text, but the mass-preserving action ~W_ψ is only introduced later in (2) within the adjoint discussion; consider defining both actions together in Section 2.1 for consistency.

Circularity Check

0 steps flagged · score 2.0 of 10

No circularity in the derivation; the caveats are evaluational (in-distribution deformations, test-set hyperparameter choice), not constructional reductions.

full rationale

The claimed derivation chain is self-contained. The variational objective (4) is built from the stated Poisson likelihoods and the deformation model f_i = W_{psi_i} f_{i-1} + e_i; the alternating scheme solves (5) with the pre-trained voxelmorph network and (6) with M-ML-EM (10), which is exactly ML-EM for the compound operator A = (A_0, ..., A_N). No equation is defined in terms of the reported PSNR gain. The network is trained on synthetic pairs (f_1, W_{exp(v)} f_1) and is not trained on Derenzo data or on the final reconstruction objective; self-citations [2], [3], [4] are software and prior methodology citations and are not load-bearing. The main caveats are evaluational rather than circular: Section 3.1 states that the test deformations are drawn from the same distribution used to train the network, so the registration is never tested out-of-distribution, and Section 4 defers clinical data and mass-preserving action to future work. Additionally, the 'optimal' iteration counts (6 and 42) are selected from the same phantom on which the 1.0 dB gain is reported. These choices limit external validity and may make the reported gain optimistic, but they do not make the derivation equivalent to its inputs. The score is therefore 2, reflecting only minor non-load-bearing self-citations.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The algorithm introduces no new physical entities. The load-bearing components the reader must accept are the tuned iteration counts, the borrowed registration regularization, the hand-chosen acquisition time, the diffeomorphic intensity-preserving motion model, and the assumption that the network trained on synthetic ellipsoids generalizes to the test phantom. Only in-distribution evidence supports that last assumption.

free parameters (3)
  • ML-EM and M-ML-EM iteration counts (n_init, n_inner, n_outer) = 6, 42, 1
    Chosen as the 'optimal strategy' by maximizing PSNR on the test phantom (Section 3.3); the reported 1.0 dB gain depends on this tuned choice.
  • Registration regularization weight lambda = not reported; adopted from voxelmorph [6]
    Borrowed unchanged from [6]; the motion estimates, and hence the reconstruction gain, depend on this hyperparameter.
  • Acquisition time factor t = 60
    Set by hand (Section 3.1) to mimic clinical noise levels; the noise level determines the magnitude of the PSNR gap and the attainable gain.
assumptions (4)
  • domain assumption Gated PET data follow g_i ~ Poisson(A f_i) for each gate i.
    Standard statistical model for PET (Section 2.2); the entire likelihood term rests on this assumption.
  • domain assumption Consecutive gate images are related by f_i = W_{ψ_i} f_{i-1} + e_i with the intensity-preserving diffeomorphic action W_ψ f(x) = f(ψ^{-1}(x)).
    Motion model in Section 2.2; the paper notes that the mass-preserving action (2) is more physically relevant for PET but uses the intensity-preserving one for both training and test.
  • domain assumption The pre-trained voxelmorph network γ(f1,f2) accurately solves the registration problem (8) for the test phantom.
    The motion-estimation step is delegated entirely to the network; validity is only shown in-distribution (test deformations drawn from the training distribution, Section 3.1).
  • standard math Adjoint identity W_T_φ = ~W_{φ^{-1}} holds and is computable.
    Used in Section 2.4 to compute A_i^T; standard result for diffeomorphic image actions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Spatiotemporal PET reconstruction using ML-EM with learned diffeomorphic deformation." pith.science (2026). https://pith.science/paper/BZFKQOJZ

@misc{pith2026190809515,
  author       = {Pith},
  title        = {Pith review of: Spatiotemporal PET reconstruction using ML-EM with learned diffeomorphic deformation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BZFKQOJZ}},
  note         = {Machine review of arXiv:1908.09515}
}
read the original abstract

Patient movement in emission tomography deteriorates reconstruction quality because of motion blur. Gating the data improves the situation somewhat: each gate contains a movement phase which is approximately stationary. A standard method is to use only the data from a few gates, with little movement between them. However, the corresponding loss of data entails an increase of noise. Motion correction algorithms have been implemented to take into account all the gated data, but they do not scale well, especially not in 3D. We propose a novel motion correction algorithm which addresses the scalability issue. Our approach is to combine an enhanced ML-EM algorithm with deep learning based movement registration. The training is unsupervised, and with artificial data. We expect this approach to scale very well to higher resolutions and to 3D, as the overall cost of our algorithm is only marginally greater than that of a standard ML-EM algorithm. We show that we can significantly decrease the noise corresponding to a limited number of gates.

Figures

Figures reproduced from arXiv: 1908.09515 by the authors.

Figure 1
Figure 1. Example of a 2D synthetic tuple of images [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Derenzo phantom in four different gates. [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. PSNR for different ML-EM strategies without motion correction, and [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: PSNR for various choices of number em iter of initial ML-EM iterates and number diff iter of M-ML-EM iterates. The gain in PSNR is 1.0 dB, which makes up for about 46 % of the maximal gain of 2.2 dB. Reconstructions obtained from the optimal uncorrected (n = 29 iterati…
Figure 5
Figure 5. Figure 5: Optimal reconstructions of the gate zero (measured in PSNR). [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

23 extracted references · 20 canonical work pages

  1. [1]

    Abadi, M., et al.: TensorFlow: Large-scale machine learning on heterogeneous sys- tems (2015), http://tensorflow.org/, software available from tensorflow.org

    work page 2015

  2. [2]

    Royal Institute of Technology (2017)

    Adler, J., Kohr, H., ¨Oktem, O.: ODL-a Python framework for rapid prototyping in inverse problems. Royal Institute of Technology (2017)

    work page 2017

  3. [3]

    Inverse Problems 33(12), 124007 (2017)

    Adler, J., ¨Oktem, O.: Solving ill-posed inverse problems using iterative deep neural networks. Inverse Problems 33(12), 124007 (2017)

    work page 2017

  4. [4]

    IEEE transactions on medical imaging 37(6), 1322–1332 (2018)

    Adler, J., ¨Oktem, O.: Learned primal-dual reconstruction. IEEE transactions on medical imaging 37(6), 1322–1332 (2018)

    2018

  5. [5]

    IEEE Transac- tions on Medical Imaging 29(11), 1892—1906 (2010)

    Blume, M., Martinez-Moller, A., Keil, A., Navab, N., Rafecas, M.: Joint reconstruc- tion of image and motion in gated positron emission tomography. IEEE Transac- tions on Medical Imaging 29(11), 1892—1906 (2010)

    work page 2010

  6. [6]

    In: International Conference on Medical Image Computing and Computer-Assisted Intervention

    Dalca, A.V., Balakrishnan, G., Guttag, J., Sabuncu, M.R.: Unsupervised learn- ing for fast probabilistic diffeomorphic registration. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 729–738. Springer (2018)

    2018

  7. [7]

    In: Proceedings of the IEEE Con- ference on Computer Vision and Pattern Recognition

    Dalca, A.V., Guttag, J., Sabuncu, M.R.: Anatomical priors in convolutional net- works for unsupervised biomedical segmentation. In: Proceedings of the IEEE Con- ference on Computer Vision and Pattern Recognition. pp. 9290–9299 (2018)

    work page 2018

  8. [8]

    (eds.): Correction Techniques in Emission Tomography

    Dawood, M., Jiang, X., Sch¨ afers, K.P. (eds.): Correction Techniques in Emission Tomography. Series in Medical Physics and Biomedical Engineering, CRC Press (2008) 12 O. ¨Oktem et al

    work page 2008

Show all 23 references
  1. [9]

    In: Li, S., Tavares, J.M.T.S

    Farag, A.A., Shalaby, A., El Munim, H.A., Farag, A.: Variational shape repre- sentation for modeling, elastic registration and segmentation. In: Li, S., Tavares, J.M.T.S. (eds.) Shape Analysis in Medical Image Analysis, Lecture Notes in Com- putational Vision and Biomechanics,...

    2014

  2. [10]

    Springer-Verlag (2015)

    Gigengack, F., Jiang, X., Dawood, M., Sch¨ afers, K.P.: Motion Correction in Tho- racic Positron Emission Tomography. Springer-Verlag (2015)

    2015

  3. [11]

    IEEE Transactions in Nuclear Science 49(5), 2344—2349 (2002)

    Gilland, D.R., Mair, B.A., Bowsher, J.E., Jaszczak, R.J.: Simultaneous reconstruc- tion and motion estimation for gated cardiac ECT. IEEE Transactions in Nuclear Science 49(5), 2344—2349 (2002)

    2002

  4. [12]

    Physics in Medicine and Biology 51(18), 4603—4619

    Gravier, E., Yang, Y., King, M.A., Jin, M.: Fully 4D motion-compensated re- construction of cardiac SPECT images. Physics in Medicine and Biology 51(18), 4603—4619. (2006)

    2006

  5. [13]

    Medical image analysis 16(6), 1307–1316 (2012)

    Hinkle, J., Szegedi, M., Wang, B., Salter, B., Joshi, S.: 4d ct image reconstruction with diffeomorphic motion model. Medical image analysis 16(6), 1307–1316 (2012)

    2012

  6. [14]

    IEEE transactions on medical imaging 13(4), 601–609 (1994)

    Hudson, H.M., Larkin, R.S.: Accelerated image reconstruction using ordered sub- sets of projection data. IEEE transactions on medical imaging 13(4), 601–609 (1994)

    1994

  7. [15]

    In: 2003 IEEE Nuclear Science Symposium Con- ference Record

    Jacobson, M.W., Fessler, J.A.: Joint estimation of image and deformation param- eters in motion-corrected PET. In: 2003 IEEE Nuclear Science Symposium Con- ference Record. pp. 3290—3294 (2003)

    2003

  8. [16]

    In: 3rd IEEE International Symposium on Biomedical Imaging: Nano to Macro

    Jacobson, M.W., Fessler, J.A.: Joint estimation of respiratory motion and activity in 4D PET using CT side information. In: 3rd IEEE International Symposium on Biomedical Imaging: Nano to Macro. Arlington, VA, April 6–9, 2006. pp. 275—278 (2006)

    2006

  9. [17]

    In: 15th Inter- national Meeting on Fully Three-Dimensional Image Reconstruction in Radiology and Nuclear Medicine

    Li, T., Zhang, M., Qi, W., Asma, E., Qi, J.: Motion correction of respiratory-gated pet image using deep learning based image registration framework. In: 15th Inter- national Meeting on Fully Three-Dimensional Image Reconstruction in Radiology and Nuclear Medicine. vol. 11072,...

    2019

  10. [18]

    PET Clinics 8(1), 51–67 (2013)

    Rahmim, A., Tang, J., Zaidi, H.: Four-dimensional image reconstruction strategies in cardiac-gated and respiratory- gated PET imaging. PET Clinics 8(1), 51–67 (2013)

    2013

  11. [19]

    Physics in Medicine and Biology 59(22), R371–R418 (2014)

    Reader, A.J., Verhaeghe, J.: 4D image reconstruction for emission tomography. Physics in Medicine and Biology 59(22), R371–R418 (2014)

    2014

  12. [20]

    In: International Conference on Medical image computing and computer-assisted intervention

    Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedi- cal image segmentation. In: International Conference on Medical image computing and computer-assisted intervention. pp. 234–241. Springer (2015)

    2015

  13. [21]

    IEEE transactions on medical imaging 1(2), 113–122 (1982)

    Shepp, L.A., Vardi, Y.: Maximum likelihood reconstruction for emission tomogra- phy. IEEE transactions on medical imaging 1(2), 113–122 (1982)

    1982

  14. [22]

    Younes, L.: Shapes and Diffeomorphisms, Applied Mathematical Sciences, vol. 171. Springer-Verlag (2010)

    2010

  15. [23]

    Inverse Problems 21(1), 357–382 (2005)

    Zhang, Y., Ghodrati, A., Brooks, D.H.: An analytical comparison of three spatio- temporal regularization methods for dynamic linear inverse problems in a common statistical framework. Inverse Problems 21(1), 357–382 (2005)

    2005

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.