Joint Motion Estimation and Segmentation from Undersampled Cardiac MR Image

Chen Qin; Daniel Rueckert; Jo Schlemper; Stefan K. Piechnik; Stefan Neubauer; Steffen E. Petersen; Wenjia Bai

REVIEW 4 major objections 6 minor 13 references

Joint Motion Estimation and Segmentation from Undersampled Cardiac MR Image

T0 review · 4 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A joint deep network can estimate cardiac motion and segmentation directly from undersampled MR data, bypassing image reconstruction, with accuracy close to fully-sampled predictions.

desk verdict A sensible engineering extension of the authors' own joint motion-segmentation model to undersampled cardiac MR, but the simulation-only evaluation leaves the main clinical claim unproven. read the letter →

arxiv 1908.07623 v1 pith:2XXEGUJK submitted 2019-08-20 eess.IV cs.CV

Chen Qin , Wenjia Bai , Jo Schlemper , Steffen E. Petersen , Stefan K. Piechnik , Stefan Neubauer , Daniel Rueckert This is my paper · ORCID

classification eess.IVcs.CV

keywords cardiacMRImotionestimationimagesegmentationundersampledk-spacedeeplearningjointejectionfractionbypassingreconstruction

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that cardiac motion fields and segmentation maps can be predicted directly from undersampled MR data, without first reconstructing the images. The motivation is that clinical measurements such as ejection fraction may stay reliable even when the images themselves are imperfect, so reconstruction can be skipped when the goal is quantitative analysis. The authors build a joint network with a motion-estimation branch and a segmentation branch, and train it with a parallel fully-sampled sub-network that guides the undersampled branch. On 220 cardiac MR sequences at 3x, 6x, and 8x radial undersampling, the reported motion and segmentation accuracy is close to the fully-sampled baseline, with ejection fraction errors of 2.68%, 3.56%, and 4.16%. If this holds, faster cardiac scans could feed directly into motion and contour analysis for diagnosis.

What carries the argument

The central object is the joint Motion-Seg network, a shared feature encoder with a motion-estimation branch and a segmentation branch, extended for undersampled inputs by a parallel fully-sampled sub-network that operates only during training. The motion branch estimates a pixel-wise 2D displacement field between two cardiac frames and uses it to warp the fully-sampled source image; the segmentation branch outputs label maps for the left-ventricular cavity, myocardium, and right-ventricular cavity. The load-bearing supervisory mechanism is the loss in Eq. (2), which combines the intensity error between the warped fully-sampled source and target, a gradient-based smoothness term on the displacement field, and a masked mean-squared error between the undersampled and fully-sampled displacement fields restricted to the region of interest. This lets the undersampled network learn from the fully-sampled network without any ground-truth motion fields.

What would settle it

Acquire prospectively undersampled cardiac MR images with fully sampled references for the same subjects and compare the network's motion and segmentation accuracy, trained only on the synthetic phase-map simulation, against the fully sampled predictions. If the accuracy gap at the same acceleration factors is materially larger than the 3x, 6x, and 8x gaps reported in the paper, the central claim is not supported.

Watch

Extended reading notes

Core claim

The paper's central claim is that one shared deep network can estimate cardiac motion and segmentation jointly from undersampled k-space data and produce results close to those obtained from fully-sampled data. During training, a pre-trained sub-network on corresponding fully-sampled image pairs supervises the undersampled path: the undersampled motion field warps the fully-sampled source image, and the difference between undersampled and fully-sampled displacement fields is penalized inside a region of interest derived from segmentation. Segmentation supervision combines manual labels where available with predictions from the fully-sampled sub-network elsewhere. At test time only the undersampled branch is used. Reported contour distances and Dice scores degrade mildly from 3x to 8x acceleration, and the clinical parameter ejection fraction stays within about four percentage points of the fully-sampled value.

Load-bearing premise

The training pipeline assumes that undersampled k-space data can be realistically simulated by adding smooth 2D sinusoid phase maps to magnitude images; if real cardiac phase varies more rapidly or structurally, a network trained on these simulations may not transfer to actual accelerated acquisitions.

Editorial extensions

If this is right

At 8x acceleration, left-ventricular Dice is 0.9141 versus 0.9348 on fully-sampled data, and ejection fraction error is 4.16%, so for these clinical measures the reconstruction stage may be safely omitted.
The displacement-field supervision term (Proposed-B) mainly improves motion accuracy at 6x and 8x, indicating that stronger guidance is needed as aliasing increases.
Because the fully-sampled sub-network is only needed at training time, the same model can be deployed with undersampled acquisitions alone.
The paper's decision to drop the warped-segmentation consistency loss suggests that cross-task consistency constraints can add noise under undersampling, and should be re-examined when designing joint models for accelerated data.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the synthetic phase-map simulation is a faithful proxy for real cardiac phase, the same guided-training scheme could transfer to other dynamic imaging settings where a fully-sampled reference is available during training.
A natural extension is to re-enable the warped-segmentation consistency loss with uncertainty weighting for low acceleration factors, since the paper found it harmful at high undersampling.
The reported ceiling is set by the fully-sampled network's own accuracy; at very high acceleration, the displacement-field supervision may need to be replaced by a learned prior independent of the fully-sampled reference.
The current evidence is retrospective and simulated; a prospective study with real accelerated acquisitions would test whether the training transfer holds.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The paper does one useful thing: it takes the joint motion-segmentation network from [6] and adapts it to work directly on undersampled k-space data by adding a fully-sampled teacher branch and a displacement-field loss. That is a modest but legitimate extension, and the training scheme is reasonable: the teacher provides supervision for motion and segmentation, while the final evaluation is against manual annotations, so the self-referential teacher does not contaminate the metrics. On 220 UK Biobank subjects, the method holds up well at 3x–8x radial undersampling, with small drops in Dice and motion errors, and the Proposed-B loss clearly helps at higher acceleration factors. The paper is clearly written and the authors are honest that this is simulation-based.

The soft spots are real but not fatal. The stress-test note is on target: the undersampled data are generated by adding smooth sinusoid phase maps to magnitude images, then retrospectively undersampling k-space. Real cardiac phase maps contain flow-induced phase, B0 inhomogeneity, and eddy currents, so the network may be learning to handle an unrealistically smooth artifact. This is the main reason the central claim is not yet established for clinical use. The paper also never compares against Schlemper et al. [10], the closest segmentation-from-undersampled baseline, nor against a reconstruction-then-analysis pipeline, so it is hard to say what is actually gained by bypassing reconstruction. There is no code or data release, and hyperparameter choices (alpha, beta, lambda) are reported but not analyzed beyond a validation-set selection.

The citation pattern is fine; the authors rely heavily on their own prior work, but that is legitimate since the method is a direct extension. The statistical testing (Wilcoxon) is a plus. Overall, this is a competent conference paper with a clear contribution to the application-driven MRI line of work. It deserves a serious peer review, but a reviewer should push for evaluation on retrospectively undersampled real complex k-space data (which the UK Biobank complex data would allow) or at least on more realistic simulated artifacts, before accepting the claim that the model is robust to actual accelerated acquisitions.

Referee Report

4 major / 6 minor

Summary. The manuscript proposes a joint deep learning model for cardiac motion estimation and segmentation directly from undersampled cardiac MR images, extending the Motion-Seg Net of [6]. The model has motion and segmentation branches sharing a feature encoder, and it is trained with a composite loss that incorporates supervision from fully-sampled image pairs, including a displacement-field matching term (Eq. 2). Experiments on 220 UK Biobank subjects simulate radial undersampling by synthesizing smooth 2D sinusoid phase maps on magnitude images and then back-transforming to k-space. Results in Tables 1 and 2 show that the proposed method outperforms a B-spline FFD baseline and yields Dice and motion errors close to fully-sampled predictions, with the Proposed-B loss improving high-acceleration motion accuracy. The central claim is that clinically relevant motion and segmentation can be obtained directly from undersampled data, bypassing the image reconstruction stage.

Significance. If the results hold, the paper addresses a practically important problem: extracting motion and segmentation from accelerated cardiac MR without an explicit reconstruction step could reduce acquisition time and computational cost. The strengths include the large dataset (220 subjects), evaluation against manual annotations for segmentation accuracy, use of standard metrics (Dice, MCD, HD), and the ablation of the additional displacement-field loss (Proposed-A vs. Proposed-B). The paper also clearly states that at test time only the undersampled network is needed, and it provides a useful comparison with FFD across multiple acceleration factors. However, the significance of the contribution depends on whether the simulated undersampling reflects real accelerated acquisitions and on whether direct estimation is compared with the conventional reconstruct-then-analyze pipeline.

major comments (4)

[Section 3, paragraph 1] The evaluation is entirely based on synthetic complex-valued images generated by adding smooth 2D sinusoid phase maps to magnitude images (following [10]), and all undersampling artifacts in both training and testing arise from this simulation. Real cardiac phase maps contain B0 inhomogeneity, flow-induced phase, eddy currents, and tissue-dependent susceptibility, which are not smooth sinusoids. Therefore the central claim that the model is robust to undersampled data and produces results close to fully-sampled ones is not yet established for real accelerated cardiac MR. I request additional experiments on retrospectively undersampled real complex k-space data, or, if such data are unavailable, a clear statement of this limitation and a discussion of how the smooth-phase simulation might affect transferability.
[Section 3, Tables 1 and 2] The claim of bypassing the reconstruction stage is not supported by a reconstruction-then-analysis comparison. The paper compares only against FFD [8] and the fully-sampled joint model [6]; a natural and important baseline is to reconstruct undersampled k-space with a standard method and then run the fully-sampled Motion-Seg Net on the reconstructed images, reporting the same metrics (MCD, HD, Dice, EF). Without such a baseline, the reader cannot judge whether direct joint prediction is preferable to the conventional pipeline. In addition, the segmentation results are not compared with the closest prior work [10], which predicts segmentation maps directly from undersampled k-space; this comparison is needed to situate the segmentation contribution.
[Section 2.1, Eq. (2)] The motion supervision in Eq. (2) uses displacement fields ∆FS predicted by the fully-sampled sub-network, which is the authors' own network from [6]. Because ∆FS is not an independent ground truth, biases in the fully-sampled network are inherited by the undersampled network, and the statement that 'no ground truth displacement fields are required' is potentially misleading: this is a form of knowledge distillation rather than purely unsupervised learning. The ROI mask M_t is also generated from the fully-sampled segmentation predictions, so the supervision itself is not independent of the teacher, even though the evaluation metrics are computed against manual annotations. Please clarify the role of the teacher, discuss the potential effect of teacher bias, and consider whether this weakens the 'unsupervised' claim.
[Section 3, Tables 1 and 2] The claim that undersampled results are 'close to' fully-sampled results is not supported by statistical tests. For example, Table 1 shows Proposed-B at 8x with LV MCD 1.79 mm versus 1.30 mm for the fully-sampled Joint Model [6], and Table 2 shows Myo Dice dropping from 0.8640 (FS) to 0.8260 at 8x. These differences may be clinically meaningful. Please report whether the differences between undersampled and fully-sampled results are statistically significant, and discuss their clinical relevance in the context of cardiac function assessment.

minor comments (6)

[Table 2] The EF error is computed relative to fully-sampled predictions rather than to manual ground truth; consider also reporting EF accuracy against manual segmentations, since the fully-sampled predictions themselves contain errors.
[Section 3, paragraph 2] The description of the data split (100/100/20) should explicitly state that the split is at the subject level and that no slices from the same subject appear in both training and test sets.
[Equations (1) and (2)] The warp operator Γ is used in the equations but is not defined; please provide a definition or a reference for the bilinear interpolation warping operation.
[Section 2.2] The text mentions that the loss Lw is not employed, but Lw is never defined; please define it or rephrase the sentence to be self-contained.
[References] References [6] and [7] are arXiv preprints; please update them to the peer-reviewed versions if they are now published.
[Figure 2] The labels '8 × 1 15 20 25' in Fig. 2 are unclear; please clarify which numbers denote acceleration factor and which denote time points.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central evaluations are grounded in manual annotations and external comparison methods, not in the method's own training targets.

full rationale

The paper's central claim is that joint motion estimation and segmentation can be performed directly from undersampled cardiac MR data with results close to fully-sampled predictions. The evaluation supporting this claim is independently grounded: segmentation Dice scores are computed against manual annotations (Table 2), and motion accuracy is measured via mean contour distance and Hausdorff distance between warped and target segmentations (Table 1), with a comparison to a B-spline FFD baseline. The fully-sampled sub-network loaded from the authors' prior work [6] provides supervision during training, but this is a teacher-student training design rather than a derived prediction, and it is not the evaluation endpoint. The reported closeness to fully-sampled results is therefore not equivalent to the training loss by construction. The synthetic phase-map simulation adopted from [10] is an explicit limitation of the experimental setup, not a circular step: the paper does not redefine undersampling robustness in terms of the simulation, and no equation or claim reduces to the simulation's own output. Self-citations to [6] and [10] are present, but they are not load-bearing in a circular way because the cited models are benchmarked against manual annotations within the paper and because the central quantitative comparisons are made against external references. Correctness risks such as whether smooth sinusoid phase maps transfer to real cardiac phase behavior are validity concerns, not circularity concerns. Overall, the derivation chain is self-contained and the empirical evaluation does not depend on the method predicting its own training targets.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new physical entities. Its dependencies are hyperparameters tuned on validation data and two domain assumptions: the realism of synthetic phase maps and the reliability of the fully-sampled teacher network.

free parameters (3)

alpha = 0.001
Regularization weight for Huber smoothness on displacement gradients, chosen via validation set.
beta = 1
Weight for the displacement-field loss between undersampled and fully-sampled predictions, chosen via validation set.
lambda = 0.01
Weight for the segmentation loss in the joint objective, chosen via validation set.

assumptions (3)

domain assumption Synthetic phase maps created from smooth 2D sinusoids are representative of real k-space phase.
The paper has only magnitude images and synthesizes phase to simulate undersampling; the validity of all experiments depends on this approximation.
domain assumption The fully-sampled sub-network from [6] provides reliable pseudo-labels for training.
The teacher-student setup uses fully-sampled predictions as targets for motion and segmentation; if the teacher is biased, the student inherits that bias.
domain assumption A network trained on simulated undersampling will generalize to real undersampled acquisitions.
The evaluation is entirely on simulated radial undersampling of synthesized k-space, with no real accelerated data tested.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Joint Motion Estimation and Segmentation from Undersampled Cardiac MR Image." pith.science (2026). https://pith.science/paper/2XXEGUJK

@misc{pith2026190807623,
  author       = {Pith},
  title        = {Pith review of: Joint Motion Estimation and Segmentation from Undersampled Cardiac MR Image},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2XXEGUJK}},
  note         = {Machine review of arXiv:1908.07623}
}

read the original abstract

Accelerating the acquisition of magnetic resonance imaging (MRI) is a challenging problem, and many works have been proposed to reconstruct images from undersampled k-space data. However, if the main purpose is to extract certain quantitative measures from the images, perfect reconstructions may not always be necessary as long as the images enable the means of extracting the clinically relevant measures. In this paper, we work on jointly predicting cardiac motion estimation and segmentation directly from undersampled data, which are two important steps in quantitatively assessing cardiac function and diagnosing cardiovascular diseases. In particular, a unified model consisting of both motion estimation branch and segmentation branch is learned by optimising the two tasks simultaneously. Additional corresponding fully-sampled images are incorporated into the network as a parallel sub-network to enhance and guide the learning during the training process. Experimental results using cardiac MR images from 220 subjects show that the proposed model is robust to undersampled data and is capable of predicting results that are close to that from fully-sampled ones, while bypassing the usual image reconstruction stage.

Figures

Figures reproduced from arXiv: 1908.07623 by the authors.

**Figure 2.** Comparison visualisation results for simultaneous prediction of motion [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

13 extracted references · 13 canonical work pages

[6]

Joint Learning of Motion Estimation and Segmentation for Cardiac MR Image Sequences

Qin, C., Bai, W., Schlemper, J., Petersen, S.E., Piechnik, S.K., Neubauer, S., Rueckert, D.: Joint learning of motion estimation and segmentation for cardiac MR image sequences. arXiv preprint arXiv:1806.04066 (2018)

work page Pith review arXiv 2018
[10]

In: MICCAI (2018)

Schlemper, J., Oktay, O., Bai, W., et al.: Cardiac MR segmentation from under- sampled k-space using deep latent representation learning. In: MICCAI (2018)

work page 2018
[8]

IEEE transactions on medical imaging 18(8), 712–721 (1999)

Rueckert, D., Sonoda, L.I., Hayes, C., et al.: Nonrigid registration using free-form deformations: application to breast MR images. IEEE transactions on medical imaging 18(8), 712–721 (1999)

work page 1999
[1]

Journal of Cardiovascular Magnetic Resonance (2018)

Bai, W., Sinclair, M., Tarroni, G., et al.: Automated cardiovascular magnetic reso- nance image analysis with fully convolutional networks. Journal of Cardiovascular Magnetic Resonance (2018)

work page 2018
[2]

In: International Con- ference on Medical Image Computing and Computer-Assisted Intervention

Caballero, J., Bai, W., Price, A.N., et al.: Application-driven MRI: Joint recon- struction and segmentation from undersampled MRI data. In: International Con- ference on Medical Image Computing and Computer-Assisted Intervention. pp. 106–113. Springer (2014)

work page 2014
[3]

In: CVPR (2017)

Caballero, J., Ledig, C., Aitken, A., et al.: Real-time video super-resolution with spatio-temporal networks and motion compensation. In: CVPR (2017)

work page 2017
[4]

In: ICCV

Cheng, J., Tsai, Y.H., Wang, S., Yang, M.H.: Segﬂow: Joint learning for video object segmentation and optical ﬂow. In: ICCV. pp. 686–695 (2017)

work page 2017
[5]

In: International Workshop on Statistical Atlases and Computational Models of the Heart

Oksuz, I., Dharmakumar, R., Tsaftaris, S.A.: Joint myocardial registration and segmentation of cardiac BOLD MRI. In: International Workshop on Statistical Atlases and Computational Models of the Heart. pp. 12–20. Springer (2017)

work page 2017

Show all 13 references

[7]

arXiv preprint arXiv:1712.01751 (2017)

Qin, C., Schlemper, J., Caballero, J., Price, A., Hajnal, J.V., Rueckert, D.: Con- volutional recurrent neural networks for dynamic MR image reconstruction. arXiv preprint arXiv:1712.01751 (2017)

2017 arXiv
[9]

IEEE Transactions on Medical Imaging 37(2) (2018)

Schlemper, J., Caballero, J., Hajnal, J.V., Price, A., Rueckert, D.: A deep cascade of convolutional neural networks for dynamic MR image reconstruction. IEEE Transactions on Medical Imaging 37(2) (2018)

2018
[11]

IEEE transactions on medical imaging 31(6), 1263–1275 (2012)

Shi, W., Zhuang, X., Wang, H., et al.: A comprehensive cardiac motion estimation framework using both untagged and 3-D tagged MR images based on nonrigid registration. IEEE transactions on medical imaging 31(6), 1263–1275 (2012)

2012
[12]

Medical image analysis 17(6), 632–648 (2013)

Tobon-Gomez, C., De Craene, M., Mcleod, K., et al.: Benchmarking framework for myocardial tracking and deformation algorithms: An open access database. Medical image analysis 17(6), 632–648 (2013)

2013
[13]

In: CVPR

Tsai, Y.H., Yang, M.H., Black, M.J.: Video segmentation via object ﬂow. In: CVPR. pp. 3899–3908 (2016)

2016

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [6]

Joint Learning of Motion Estimation and Segmentation for Cardiac MR Image Sequences

Qin, C., Bai, W., Schlemper, J., Petersen, S.E., Piechnik, S.K., Neubauer, S., Rueckert, D.: Joint learning of motion estimation and segmentation for cardiac MR image sequences. arXiv preprint arXiv:1806.04066 (2018)

work page Pith review arXiv 2018

[2] [10]

In: MICCAI (2018)

Schlemper, J., Oktay, O., Bai, W., et al.: Cardiac MR segmentation from under- sampled k-space using deep latent representation learning. In: MICCAI (2018)

work page 2018

[3] [8]

IEEE transactions on medical imaging 18(8), 712–721 (1999)

Rueckert, D., Sonoda, L.I., Hayes, C., et al.: Nonrigid registration using free-form deformations: application to breast MR images. IEEE transactions on medical imaging 18(8), 712–721 (1999)

work page 1999

[4] [1]

Journal of Cardiovascular Magnetic Resonance (2018)

Bai, W., Sinclair, M., Tarroni, G., et al.: Automated cardiovascular magnetic reso- nance image analysis with fully convolutional networks. Journal of Cardiovascular Magnetic Resonance (2018)

work page 2018

[5] [2]

In: International Con- ference on Medical Image Computing and Computer-Assisted Intervention

Caballero, J., Bai, W., Price, A.N., et al.: Application-driven MRI: Joint recon- struction and segmentation from undersampled MRI data. In: International Con- ference on Medical Image Computing and Computer-Assisted Intervention. pp. 106–113. Springer (2014)

work page 2014

[6] [3]

In: CVPR (2017)

Caballero, J., Ledig, C., Aitken, A., et al.: Real-time video super-resolution with spatio-temporal networks and motion compensation. In: CVPR (2017)

work page 2017

[7] [4]

In: ICCV

Cheng, J., Tsai, Y.H., Wang, S., Yang, M.H.: Segﬂow: Joint learning for video object segmentation and optical ﬂow. In: ICCV. pp. 686–695 (2017)

work page 2017

[8] [5]

In: International Workshop on Statistical Atlases and Computational Models of the Heart

Oksuz, I., Dharmakumar, R., Tsaftaris, S.A.: Joint myocardial registration and segmentation of cardiac BOLD MRI. In: International Workshop on Statistical Atlases and Computational Models of the Heart. pp. 12–20. Springer (2017)

work page 2017