REVIEW 4 major objections 4 minor 38 references
Are Registration Uncertainty and Error Monotonically Associated
T0 review · 4 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read Gaussian process registration uncertainty and non-rigid registration error are weakly to moderately monotonically correlated, and the association varies by how landmarks are chosen.
desk verdict A genuinely new empirical question about GP registration uncertainty and error, but the missing kernel length-scale parameter makes the headline correlations non-reproducible and the qualitative claim conditional. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The object carrying the argument is the posterior covariance of a Gaussian process displacement field. Registration uncertainty at a voxel is the diagonal entry of $\Sigma_* = K_{**} - K_*^T K^{-1} K_*$, computed with kernel $k(x,x') = \exp(-x^2/a)$; larger posterior variance means less confidence in the interpolated displacement. The authors compare this uncertainty to error using Spearman's rank correlation $\rho_s$, chosen because it tests monotonic rather than linear relationships and is robust to outliers. The comparison is done point-wise, using known landmark displacements as ground truth, and patch-wise, using appearance-based dissimilarity metrics (histogram intersection and SSD) as error surrogates.
What would settle it
Recompute the point-wise Spearman's rho on the same landmarks while varying the kernel length-scale $a$ over a wide range; if the correlation swings from 0.29-0.40 to near zero or negative, the weak-to-moderate association is an artifact of the unstated kernel calibration rather than a robust property of GP registration uncertainty.
Extended reading notes
Core claim
The central claim is that there is a positive monotonic association between point-wise GP registration uncertainty and non-rigid registration error, but it is weak-to-moderate, not strong enough to support clinical use as a surrogate. In the point-wise posterior predictive checking experiment, the mean Spearman's rho is 0.2899 for manual landmarks from the RESECT dataset and 0.4014 for automatically extracted landmarks from the MIBS dataset. In the patch-wise experiment, where error is approximated by histogram intersection and sum of squared differences over image patches, the correlations are consistently low for both datasets. The authors infer that transformation uncertainty and appearance-based error may be inherently weakly coupled, and that the distribution of landmarks matters for the strength of the association.
Load-bearing premise
The load-bearing premise is that the Gaussian process posterior variance used as 'registration uncertainty' is a faithful measure of confidence; the kernel length-scale $a$ is not reported, so if a different $a$ re-ranks the uncertainties, the reported correlations could be an artifact of calibration rather than a property of the method.
Editorial extensions
If this is right
- Surgeons cannot yet treat GP uncertainty maps as reliable indicators of where non-rigid registration may be wrong, particularly in patch-level assessments.
- Point-wise uncertainty carries some ranking information about error, but the strength is too low to set confidence thresholds for clinical decisions.
- The higher correlation for automatic landmarks suggests that the spatial distribution of features used for interpolation modulates the uncertainty-error link.
- Registration uncertainty should not be used as a proxy for error without further calibration or validation against dense ground-truth deformation.
Reading between the lines
- The paper studies only Gaussian process registration; other probabilistic registration approaches may show a different uncertainty-error association, so the practical verdict should not be read as a general statement about registration uncertainty.
- Because the kernel length-scale is not reported, varying it in a follow-up experiment would show how much of the measured correlation is intrinsic to GP uncertainty and how much is an artifact of calibration.
- A synthetic phantom with a known dense deformation field could test the same question without relying on sparse landmarks or appearance-based patch error, giving a cleaner ground-truth comparison.
- The weak patch-wise correlation may reflect the difference between transformation uncertainty and label uncertainty; a model that estimates label uncertainty directly might yield clinically useful error maps where this one does not.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper asks whether Gaussian-process (GP) registration uncertainty is monotonically associated with non-rigid registration error in image-guided neurosurgery. Using two clinical ultrasound datasets (RESECT and MIBS), the authors compute Spearman rank correlations between point-wise GP posterior uncertainties and landmark displacement errors, as well as patch-wise correlations using intensity dissimilarity measures (HI and SSD). They report weak-to-moderate positive point-wise correlations (mean rho = 0.2899 for manual landmarks, 0.4014 for automatic landmarks) and consistently low patch-wise correlations, concluding that GP uncertainty is not yet a reliable surrogate for registration error.
Significance. If the empirical result holds, it provides a useful cautionary quantification for clinicians considering GP uncertainty maps as error surrogates, and it opens a question that is indeed overlooked in the registration literature. The study has clear strengths: the point-wise test is direct, uses ground-truth displacements, and involves no circularity because the GP posterior variance is computed from landmark positions and kernel choices independent of the error measurement. The use of two clinical datasets is also a plus. However, the quantitative conclusions are conditional on an unstated kernel hyperparameter and lack any statistical uncertainty assessment, so the current weight of the evidence is limited.
major comments (4)
- [Section 2.1, Eq. (3)] The covariance kernel is written as k(x, x') = exp(-x^2/a), but no value or range for a is given anywhere in the manuscript. The posterior variance in Eq. (3) depends on a and the landmark positions, but not on the observed displacements D; consequently the rank ordering of uncertainty values and every Spearman coefficient in Fig. 5 is a function of this unstated parameter. The authors must report the exact value of a used and provide a sensitivity analysis over a range of a, since the headline means (0.2899 and 0.4014) cannot otherwise be reproduced and the influence of this hyperparameter on the conclusions cannot be assessed.
- [Section 3.1, Fig. 5] The central claim of weak-to-moderate positive monotonic correlation rests only on mean rho values across image pairs. The authors do not report the number of image pairs and landmarks, the per-pair correlation values, confidence intervals, or significance tests, so the reader cannot judge whether the correlations are statistically distinguishable from zero or how stable they are across patients. Please provide this information, for instance with bootstrap confidence intervals for the mean rho and per-case results.
- [Section 3.2] The patch-wise experiment is interpreted as showing consistently low correlation, but the authors themselves state that the intensity-based HI/SSD dissimilarity is 'sub-optimal' for estimating registration error under large deformation. Because the error surrogate is acknowledged to be flawed, the low patch-wise rho values cannot be cleanly attributed to the uncertainty-error relationship. The patch-wise conclusion should either be supported with a validated local error measure or explicitly downweighted and removed from the paper's conclusions.
- [Section 3] The experimental setup is not fully specified: the manuscript does not state how many interpolating landmarks were used per image, how they were selected, how many test points were evaluated, or how the automatic landmark detection was performed in MIBS. These details materially affect both the GP uncertainty and the error distribution and are needed for reproducibility of the reported correlations.
minor comments (4)
- [Section 2.1] The kernel notation k(x, x') = exp(-x^2/a) is ambiguous because k takes two spatial arguments; the exponent should be written as e.g. exp(-||x - x'||^2 / a).
- [Section 2.2] The rank-transformation example is correct, but the authors should state how tied ranks are handled when computing Spearman's rho.
- [Section 3.2] The sentence 'while intensities over patches is label uncertainty' is unclear; patch intensities are observations used to define the dissimilarity, not label uncertainty, and the connection to the transformation-versus-label uncertainty distinction should be explained more carefully.
- [Section 3.1] The text says the automatic landmarks 'achieved an average rho of 0.4014, which can be categorized as a moderate-to-strong correlation,' but the correlation categories are not defined; the reader would benefit from an explicit threshold or reference for the labels 'weak-to-moderate' and 'moderate-to-strong.'
Circularity Check
No circularity: GP uncertainty comes from the posterior covariance while error is measured against ground-truth displacements, from independent sources; no fitted parameter or self-citation chain forces the result.
full rationale
The paper's central claim is an empirical association, not a derivation. The uncertainty u is taken from the diagonal of the GP posterior covariance matrix (Eq. 3), which depends on the kernel and landmark geometry but not on the observed displacements. The error is computed independently as the Euclidean distance between the ground-truth displacement and the GP posterior mean (Eq. 2, Sec. 2.3). No parameter was fitted to maximize the Spearman correlation, and the test points are explicitly stated not to have been used for GP interpolation (Sec. 3). The self-citations to earlier GP registration work and to the transformation-versus-label uncertainty taxonomy are contextual and are not load-bearing for the empirical correlation result; there is no imported uniqueness theorem or ansatz presented as a derived constraint. One reproducibility concern exists: the GP kernel length-scale a in k(x,x') = exp(-x^2/a) is never specified, so the numerical correlation values are conditional on an unstated hyperparameter and lack a sensitivity analysis. That is a calibration/reproducibility gap, not circularity, because the uncertainty measure is still computed from a distinct source than the error, and the paper does not claim to derive the association from that kernel choice.
Assumptions & free parameters
free parameters (1)
- GP kernel length-scale a =
not reported
assumptions (4)
- domain assumption The displacement field d(x) is well modeled as a zero-mean Gaussian process with kernel exp(-x^2/a).
- standard math Spearman's rank correlation measures monotonic association.
- domain assumption Intensity-based dissimilarity (SSD, HI) is a valid proxy for patch-wise registration error.
- domain assumption Tested landmarks were not used for GP interpolation, so the error and uncertainty are measured at independent points.
Cite this review
Pith. "Pith review of Are Registration Uncertainty and Error Monotonically Associated." pith.science (2026). https://pith.science/paper/XHZ6RNUW
@misc{pith2026190807709,
author = {Pith},
title = {Pith review of: Are Registration Uncertainty and Error Monotonically Associated},
year = {2026},
howpublished = {\url{https://pith.science/paper/XHZ6RNUW}},
note = {Machine review of arXiv:1908.07709}
}
read the original abstract
In image-guided neurosurgery, current commercial systems usually provide only rigid registration, partly because it is harder to predict, validate and understand non-rigid registration error. For instance, when surgeons see a discrepancy in aligned image features, they may not be able to distinguish between registration error and actual tissue deformation caused by tumor resection. In this case, the spatial distribution of registration error could help them make more informed decisions, e.g., ignoring the registration where the estimated error is high. However, error estimates are difficult to acquire. Probabilistic image registration (PIR) methods provide measures of registration uncertainty, which could be a surrogate for assessing the registration error. It is intuitive and believed by many clinicians that high uncertainty indicates a large error. However, the monotonic association between uncertainty and error has not been examined in image registration literature. In this pilot study, we attempt to address this fundamental problem by looking at one PIR method, the Gaussian process (GP) registration. We systematically investigate the relation between GP uncertainty and error based on clinical data and show empirically that there is a weak-to-moderate positive monotonic correlation between point-wise GP registration uncertainty and non-rigid registration error.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[1]
Maintz, J.B.A., Viergever, M.A.: A Survey of Medical Imag e Registration. Med. Image Anal. 2(1): 1-36 (1998)
work page 1998
-
[2]
Sotiras, A., Davatzikos, C.: Deformable Medical Image Re gistration: A Survey. IEEE Trans. Med. Imaging 32(7), 1153-1190 (2013)
work page 2013
-
[3]
Gerard, I.J., et al: Brain shift in neuronavigation of bra in tumors: A review. Med. Image Anal. 35: 403-420 (2017)
work page 2017
-
[4]
Morin, F., et al.: Brain-shift compensation using intrao perative ultrasound and constraint-based biomechanical simulation. Med. Image An al. 40: 133-153 (2017)
work page 2017
-
[5]
Luo M., Larson P.S., Martin A.J., Konrad P.E., Miga M.I.: A n Integrated Multi- physics Finite Element Modeling Framework for Deep Brain St imulation: Prelimi- nary Study on Impact of Brain Shift on Neuronal Pathways. In: Shen D. et al. (eds). MICCAI 2019. LNCS, vol 11768. pp. 682-690. Springer, Cham (2 019)
work page 2019
-
[6]
Rivaz, H., Collins, D. L.: Deformable registration of pre operative MR, pre-resection ultrasound, and post-resection ultrasound images of neuro surgery. IJCARS (2015)
work page 2015
-
[7]
Kybic L.: Bootstrap Resampling for Image Registration Un certainty Estimation Without Ground Truth. IEEE Trans. Image Processing 19(1), 6 4-73 (2010)
work page 2010
-
[8]
et al.: Assessment of Rigid Registration Quality Measures in Ultrasound- Guided Radiotherapy
Shams R. et al.: Assessment of Rigid Registration Quality Measures in Ultrasound- Guided Radiotherapy. IEEE Trans. Med. Imaging 37(2), 428-4 37 (2018)
work page 2018
Show all 38 references
-
[9]
In: LNCS, vol
Datteri R.D., Dawant B.M.: Automatic detection of the mag nitude and spatial location of error in non-rigid registration. In: LNCS, vol. 7359, pp. 2130. (2012)
2012
-
[10]
IEEE Trans
Hub, M., et al.: A stochastic approach to estimate the unc ertainty involved in B- spline image registration. IEEE Trans. Med. Imaging 28(11) , pp. 17081716 (2009)
2009
-
[11]
Hub M., Karger C.P.: Estimation of the uncertainty of ela stic image registration with the demons algorithm. Phys. Med. & Bio, 58(9): (2013)
2013
-
[12]
IEEE Trans
Saygili G., et al.: Confidence Estimation for Medical Ima ge Registration Based On Stereo Confidences. IEEE Trans. Med. Imaging 35(2), 539-549 (2016)
2016
-
[13]
In: MICCAI’16
Sokooti H., et al.: Accuracy Estimation for Medical Imag e Registration Using Re- gression Forests. In: MICCAI’16. LNCS, vol 9902, pp. 107-11 5 (2016)
2016
-
[14]
Sokooti, H., Saygili, G., Glocker, B., Lelieveldt, B.P. F. and Staring, M.: Quantita- tive error prediction of medical image registration using r egression forests. Medical Image Analysis, 56:110121, 2019
2019
-
[15]
Saygili G.: Local-search based prediction of medical im age registration error. Proc. SPIE 10577, Medical Imaging’18. doi: 10.1117/12.2293740 ( 2018)
2018 doi
-
[16]
a nd Navab, N.: Optical flow estimation with uncertainties through dynamic mrfs
Glocker, B., Paragios, N., Komodakis, N., Tziritas, G. a nd Navab, N.: Optical flow estimation with uncertainties through dynamic mrfs. In Computer Vision and Pattern Recgonition (CVPR), pages 18, 2008
2008
-
[17]
In: MIC- CAI’13
Popuri, K., et al.: A Variational Formulation for Discre te Registration. In: MIC- CAI’13. LNCS, vol. 8151, pp. 187-194. Springer, Berlin, Hei delberg (2013)
2013
-
[18]
Risholm, et al.: Bayesian Characterization of Uncertai nty in Intra-subject Non- rigid Registration. Med. Image Anal. 17(5), 538-555 (2013)
2013
-
[19]
In: LNCS, vol
Lotfi, T., et al.: Improving Probabilistic Image Registr ation via Reinforcement Learning and Uncertainty Evaluation. In: LNCS, vol. 8184, p p. 187-194. (2013)
2013
-
[20]
In: WBIR’14
Wassermann D., et al.: Probabilistic Diffeomorphic Regi stration: Representing Un- certainty. In: WBIR’14. LNCS, vol. 8545. pp. 72-82. Springe r, Cham (2014)
2014
-
[21]
In: MICCAI’15
Yang, X., Niethammer, M.: Uncertainty Quantification fo r LDDMM Using a Low- rank Hessian Approximation. In: MICCAI’15. LNCS, vol. 9350 , pp. 289-296. (2015)
2015
-
[22]
Simpson, I.J.A., et al.: Probabilistic non-linear regi stration with spatially adaptive regularisation. Med. Image Anal. 26(1): 203-216 (2015) 10 j. Luo et al
2015
-
[23]
Heinrich, M.P., et al.: Deformable Image Registration b y Combining Uncertainty Estimates From Supervoxel Belief Propagation. Med. Image A nal. 27, 57-71 (2016)
2016
-
[24]
IEEE Trans
Folgoc, et al.: Quantifying Registration Uncertainty W ith Sparse Bayesian Mod- elling. IEEE Trans. Medical Imaging, 36(2): 607-617 (2017)
2017
-
[25]
In: MICCAI’18
Dalca, A.V., et al.: Unsupervised Learning for Fast Prob abilistic Diffeomorphic Registration. In: MICCAI’18. LNCS, vol. 11070, pp. 729-738. Springer, Cham (2018)
2018
-
[26]
In: MICCAI’18
Wang, J., et al.: Efficient Laplace Approximation for Baye sian Registration Uncer- tainty Quantification. In: MICCAI’18. LNCS, vol. 11073, pp. 880-888. (2018)
2018
-
[27]
In: MICCAI’18
Luo, J., et al.: A feature-driven active framework for ul trasound-based brain shift compensation. In: MICCAI’18. LNCS, vol. 11073, pp. 3038. Sp ringer, Cham (2018)
2018
-
[28]
and Wells, W.M
Sedghi,T.Kapur, T., Luo, J., Mousavi, P. and Wells, W.M. : Probabilistic image registration via deep multi-class clas- sification: Charac terizing uncertainty. MIC- CAI’19 UNSURE workshop, LNCS 11840:1222. (2019)
2019
-
[29]
et al.: Investigation of Feature-Based Nonrigi d Image Registration Using Gaussian Process
Bayer S. et al.: Investigation of Feature-Based Nonrigi d Image Registration Using Gaussian Process. In: Tolxdorff T., Deserno T., Handels H., M aier A., Maier-Hein K., Palm C. (eds) Bildverarbeitung fr die Medizin 2020. Info rmatik aktuell. Springer Vieweg, Wiesbaden (2020)
2020
-
[30]
and Leemput, K.V.: Fast Nonparametric Mutual-In formation-based Reg- istration and Uncertainty Estimation
Agn, M. and Leemput, K.V.: Fast Nonparametric Mutual-In formation-based Reg- istration and Uncertainty Estimation. In MICCAI’19 UNSURE workshop, LNCS 11840: 42-51. (2019)
2019
-
[31]
and Frisken, S.: O n the applicability of registration uncer- tainty
Luo, J., Sedghi, A., Popuri, K., Cobzas, D., Zhang, M., Pr eiswerk, F., Toews, M., Golby, A.J., Sugiyama, M., Wells, W.M. and Frisken, S.: O n the applicability of registration uncer- tainty. Medical Imaging Computing a nd Computer Assisted Intervention (MICCAI), 11765:410419, 2019
2019
-
[32]
and Foreman, D.I.: Nonparametric Statisti cs: A Step-by-Step Ap- proach, Wiley
Corder, G.W. and Foreman, D.I.: Nonparametric Statisti cs: A Step-by-Step Ap- proach, Wiley. (2014)
2014
-
[33]
Gelman, A., et al.: Bayesian data analysis, p653, Chapma n&Hall, USA (2004)
2004
-
[34]
Phys.4 4(7): 3875-3882 (2017)
Xiao, Y., et al.: RESECT: A clinical database of preopera tive MRI and intraoper- ative ultrasound in lowgrade glioma surgeries.Med. Phys.4 4(7): 3875-3882 (2017)
2017
-
[35]
Int J CARS, 13 (10): pp.1525-1538
Machado, I., et al.: Non-rigid registration of 3D ultras ound for neurosurgery us- ing automatic feature detection and matching. Int J CARS, 13 (10): pp.1525-1538. (2018)
2018
-
[36]
et al.: Visualization of deformable ima ge registration quality using local image dissimilarity
Schlachter, M,. et al.: Visualization of deformable ima ge registration quality using local image dissimilarity. IEEE TMI 35(10): 2319-2328 (201 6)
-
[37]
Y., et al.: Evaluation of MRI to ultrasound registr ation methods for brain shift correction: The CuRIOUS2018 Challenge, in IEEE Trans actions on Medical Imaging
Xiao. Y., et al.: Evaluation of MRI to ultrasound registr ation methods for brain shift correction: The CuRIOUS2018 Challenge, in IEEE Trans actions on Medical Imaging. (2020)
2020
-
[38]
Toews M., Wells W.M.: (2013) Efficient and robust model-to -image alignment using 3D scale-invariant features, Med Image Anal, Vol 17, 271-28 2
2013
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.