REVIEW 3 major objections 7 minor 47 references
Implicit Deformable Medical Image Registration with Learnable Kernels
T0 review · 3 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Sparse correspondences plus a learnable kernel yield zero-shot registrations that match commercial medical software in accuracy while better preserving anatomy.
desk verdict Solid implicit-registration method with strong NLST numbers, but the paper never validates the keypoint correspondences that feed the kernel, and the clinical claim lacks significance testing. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the learnable kernel $w(x,y)=\mathrm{softmax}(a(x,y))$ over a neighborhood $N(x)$, where the attention score $a(x,y)$ is the sum of a geometric attention head $H_s$, a semantic attention head $H_f$, and a spatial bias $b(x,y)=1/(1+\|x-y\|^2)$. The kernel multiplies the observed displacements $d(y)$ at sparse keypoint correspondences and sums them to produce the displacement at $x$. The correspondences themselves come from a differentiable cost-volume layer applied to learned multi-scale features, and the whole module is integrated into a five-scale coarse-to-fine pipeline, making the reconstruction incremental and allowing targeted updates when the clinician adds a new correspondence.
What would settle it
Take a test pair, mask out all keypoint neighborhoods in a region of interest, and recompute the field; if the warped landmarks in that region deviate sharply from the full prediction, the kernel does not actually generalize beyond keypoint support. Alternatively, compare the predicted correspondences against dense ground-truth displacements from a dataset with many landmarks to measure correspondence accuracy directly.
Extended reading notes
Core claim
The paper's central claim is that a dense displacement field can be reconstructed from a sparse set of point correspondences by a learnable kernel, and that this reconstruction is more generalizable and more regular than predicting the field directly with a network. The kernel computes the displacement at any location $x$ as a weighted sum of displacements at nearby correspondences, with weights $w(x,y)$ produced by a dual-stream attention mechanism applied to geometric and semantic features, plus a spatial bias term. This kernel is conditioned on the local neighborhood $N(x)$ of each query point, using the nearest $K=30$ keypoints and their correspondences, which come from a differentiable cost-volume layer on multi-scale features. The authors report that this design outperforms or matches the strongest learned baselines and a foundation model on zero-shot tasks, and argue that the resulting deformations preserve anatomical relationships better, as reflected in the low standard deviation of the log-Jacobian.
Load-bearing premise
The method's central assumption is that sparse keypoint correspondences are accurate and dense enough to reconstruct the entire displacement field, including regions with no detected keypoints and under out-of-distribution conditions.
Editorial extensions
If this is right
- If the claim holds, learned registration can be used in clinical workflows where deformation reliability matters, such as checking whether an ablation margin around a tumor is sufficient.
- Test-time refinement becomes practical: adding 10 to 40 manually supplied landmarks improves target registration error by 5% to 14%, letting clinicians steer the registration at inference without retraining.
- The smoothness of the predicted deformations on the lung dataset (standard deviation of the log-Jacobian equal to 0.02) implies fewer folding artifacts than other learned approaches, a prerequisite for trusting deformation-based measurements.
- Matching a specialized commercial system on safety margin assessment while running in about 1.6 seconds versus over two minutes would make intra-procedural verification feasible in the operating room.
Reading between the lines
- A direct evaluation of correspondence accuracy, rather than just the final registration error, would reveal how much of the gain comes from the kernel versus the matching layer, and whether the method degrades gracefully when keypoints are sparse or wrong.
- The same kernelized signal-reconstruction idea could be applied to other dense-field estimation tasks in medical imaging, such as ultrasound motion estimation or inter-modality alignment, since the conditioning is not tied to a specific anatomy.
- The reported confidence proxy (variance of the kernel attention scores) could be calibrated into an uncertainty estimate, giving clinicians a visual signal of where the registration is trustworthy.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a new deformable medical image registration method that casts registration as a signal reconstruction problem. The dense displacement field is reconstructed from sparse keypoint correspondences via a learnable kernel, which is conditioned on local neighborhoods through a dual-stream attention mechanism with geometric and semantic heads. The method is integrated into a multi-scale coarse-to-fine pipeline, and also supports interactive test-time refinement by inserting new point correspondences. The authors evaluate on the public NLST lung CT dataset and an internal colorectal liver cancer CT dataset, comparing against several state-of-the-art learning-based methods and a commercial system. They report competitive or superior target registration error (TRE), smoothness (SDlogJ), liver surface distances (ASSD, HD95), and safety margin assessment (SMA) accuracy, with confidence intervals and Bonferroni-corrected significance tests for most metrics. The main claims are that the method bridges the generalization gap between implicit and explicit registration techniques, generates deformations that better preserve anatomical relationships, and matches the performance of specialized commercial systems on SMA.
Significance. If the claims hold, the work is valuable for the medical image registration community. The formulation of dense registration as kernel-based reconstruction from sparse correspondences is novel, and the explicit support for interactive test-time refinement is clinically attractive. The paper is commendable for reporting confidence intervals and Bonferroni corrections for most headline metrics, which is still rare in this literature. The core contribution—conditioning an implicit representation on local keypoint correspondences—is a plausible mechanism for improving zero-shot generalization. However, the strength of the central claim depends on the accuracy and density of the sparse correspondences, which are never directly validated; and the headline clinical claim of matching a commercial system rests on a single metric without statistical support. These gaps currently prevent the paper from being fully convincing.
major comments (3)
- [Section 2.2 and Eq. (1)] The method's central premise is that the sparse keypoint correspondences produced by the differentiable cost-volume layer on learned multi-scale features are accurate and sufficiently dense to reconstruct the entire displacement field. This premise is never directly tested. The paper reports no evaluation of the correspondences themselves: no correspondence-level target registration error against the NLST landmarks, no comparison between the predicted d(y) at keypoint locations and ground-truth displacements, and no sensitivity analysis to keypoint noise or density. The ablation in Table 3 only varies the detector (DISK, SIFT, SuperPoint) and reports final TRE, which can remain low even with biased correspondences because the softmax kernel and diffusion regularizer smooth the field. Since every output displacement in Eq. (1) is a convex combination of the input correspondences, incorrect correspondences propagate to the entire reconstruction. This gap is load-bearing for the reported accuracy and for the safety-margin claim. I request a direct correspondence-accuracy evaluation (e.g., match-level TRE on NLST landmarks, or a perturbation experiment where known noise is injected into the correspondences) before the main claims can be considered supported.
- [Section 4 and Table 2, SMA column] The headline claim that 'ours is the only AI method matching the performance of specialized commercial systems' is based on a single point estimate: 70.59% versus 71.24% for the commercial system (Ablation-fit). Unlike the other metrics in Tables 1 and 2, no confidence interval or significance test is reported for the SMA accuracy. With 32 test cases, a difference of 0.65 percentage points is likely within chance variation. The authors should provide a confidence interval (e.g., bootstrap) and an appropriate significance test (e.g., McNemar's test on the classification outcomes) for SMA, or alternatively temper the claim to a statement that the method does not significantly differ from the commercial system on this metric. Without this, the clinical claim is statistically unsupported.
- [Section 3, comparison with implicit baseline] The claim that the method 'bridges the generalization gap between implicit and explicit registration techniques' is based on a comparison against AM SIREN [48]. However, the baseline is not necessarily the strongest or most standard version: the authors implemented a multi-scale version of the '3l-512' architecture themselves, with 7.8 million trainable parameters, while the original AM SIREN paper may use a different configuration. The paper provides no justification for this choice or evidence that the implemented baseline matches the intended implicit registration approach. If a more carefully tuned or differently sized AM SIREN were used, the relative gain might change. The authors should either justify the baseline configuration, include the original single-scale AM SIREN, or soften the generalization-gap statement.
minor comments (7)
- [Section 2.2] The description of keypoint detection states that 'when using SIFT, DISK, and SuperPoint detectors we process each 2D slice in the input volume separately.' Since the registration operates on 3D volumes, it is unclear how 2D detections from individual slices are combined into 3D keypoint coordinates, especially for correspondences that may cross slices. Please clarify the exact mechanism that lifts 2D detections to a volumetric coordinate frame.
- [Equation (2)] There is a typographical error: the expression contains an extra closing parenthesis after Es(y), reading 'Hs(Es(x), Es(y)))'. This should be corrected.
- [Tables 1 and 2] The notation '1.72±0.43 [1.40, 2.04]' is ambiguous: the first interval likely denotes standard deviation and the bracketed interval the 95% confidence interval. The caption does not state this explicitly. Please define the reported variability measures in the table captions.
- [Table 2, last row] The label 'Ablation-fit [38]' for the commercial software is confusing, as it may be misread as an ablation study row. Consider renaming it 'Commercial software [38]' for clarity.
- [Section 4, Limitations] The limitations paragraph mentions GPU memory consumption but does not acknowledge the absence of direct correspondence validation or the lack of statistical support for the SMA claim. These are more substantive limitations and should be mentioned in the limitations discussion.
- [Abstract and Conclusion] The phrase 'generates deformations that better preserve anatomical relationships' is supported only by SDlogJ, a global regularity statistic. This is a weak proxy for anatomical plausibility. Consider either adding inverse-consistency or biomechanical plausibility checks, or phrasing the claim more modestly as 'smoother deformations'.
- [References] Reference [1] is listed as 'Anonymous'. The identity of this reference should be resolved before publication, and its anonymization appears to be an artifact of the submission pipeline that should be corrected.
Circularity Check
The core kernel-reconstruction pipeline is not circular, but the interactive-refinement evaluation feeds ground-truth landmarks into the model and then reports TRE improvement, making that specific result circular by construction.
-
fitted input called prediction
[Section 4, 'Test-time behavior' paragraph]
"Then, we examine the value of interactive refinement by re-computing the displacements based on the provided ground-truth landmarks: we achieve a TRE improvement of 5, 9, 13, 14 % using 10, 20, 30, 40 uniformly randomly sampled landmarks respectively."
The ground-truth landmarks are used both as the input to the refinement (new point-to-point correspondences / anchors) and as the target for the TRE metric. Since Eq. (1)-(2) reconstruct each displacement as a softmax-weighted combination of observed anchor displacements, injecting the correct landmark displacements forces the field near those points to match the ground truth; evaluating TRE on those same landmarks (the paper does not state that the evaluation set excludes the provided landmarks) measures how well the model memorized the given anchors, not generalization. The reported 5-14% improvement is therefore a consequence of using the evaluation target as the refinement input, not an independent validation of test-time refinement.
full rationale
The central derivation in Sections 2.1-2.2 is self-contained and not circular: d(x) is reconstructed from sparse cost-volume correspondences via a learned softmax kernel w, and both the kernel and the feature encoder F are trained with registration losses (NCC, diffusion, optional Dice and landmark losses). The kernel parameters are not fitted to the final evaluation landmarks, so Tables 1 and 3 are not reduced to a fit by construction. The only true circular step is the interactive-refinement experiment, where ground-truth landmarks are injected into the model and then used to compute the reported TRE improvement; this is a localized result, not the main registration accuracy claim. The self-citations [24] and [25] support the 5 mm margin threshold used in the SMA evaluation; they are peer-reviewed clinical evidence external to the present method and do not enter the derivation chain, so they do not add circularity by themselves. The absence of a direct accuracy evaluation of the sparse keypoint correspondences is a correctness risk in the central claim, but it is not a form of circularity: the correspondences are learned inputs, not outputs derived from the evaluation targets. Overall, the paper's core contribution remains independent, meriting a moderate score due to the one circular refinement evaluation.
Assumptions & free parameters
free parameters (5)
- Neighborhood size K =
30
- Number of coarse-to-fine scales =
5
- Keypoint cap during training =
1024
- Loss weights =
equal
- Spatial bias kernel b(x,y) =
1/(1+||x-y||^2)
assumptions (5)
- domain assumption Sparse keypoint correspondences are sufficient to reconstruct the dense displacement field
- domain assumption Cost-volume matching on learned features returns accurate correspondences in out-of-distribution scans
- domain assumption The local neighborhood of K=30 nearest keypoints covers every query point adequately
- domain assumption The multi-scale UNet features transfer across patients and CT protocols
- standard math Standard mathematical operations (softmax normalization, Euclidean distance) are valid
Cite this review
Pith. "Pith review of Implicit Deformable Medical Image Registration with Learnable Kernels." pith.science (2026). https://pith.science/paper/KJVJWCHH
@misc{pith2026250602150,
author = {Pith},
title = {Pith review of: Implicit Deformable Medical Image Registration with Learnable Kernels},
year = {2026},
howpublished = {\url{https://pith.science/paper/KJVJWCHH}},
note = {Machine review of arXiv:2506.02150}
}
read the original abstract
Deformable medical image registration is an essential task in computer-assisted interventions. This problem is particularly relevant to oncological treatments, where precise image alignment is necessary for tracking tumor growth, assessing treatment response, and ensuring accurate delivery of therapies. Recent AI methods can outperform traditional techniques in accuracy and speed, yet they often produce unreliable deformations that limit their clinical adoption. In this work, we address this challenge and introduce a novel implicit registration framework that can predict accurate and reliable deformations. Our insight is to reformulate image registration as a signal reconstruction problem: we learn a kernel function that can recover the dense displacement field from sparse keypoint correspondences. We integrate our method in a novel hierarchical architecture, and estimate the displacement field in a coarse-to-fine manner. Our formulation also allows for efficient refinement at test time, permitting clinicians to easily adjust registrations when needed. We validate our method on challenging intra-patient thoracic and abdominal zero-shot registration tasks, using public and internal datasets from the local University Hospital. Our method not only shows competitive accuracy to state-of-the-art approaches, but also bridges the generalization gap between implicit and explicit registration techniques. In particular, our method generates deformations that better preserve anatomical relationships and matches the performance of specialized commercial systems, underscoring its potential for clinical adoption.
Figures
Reference graph
Works this paper leans on
-
[48]
Zimmer, V.A., et al.: Towards generalised neural implicit representations for image registration. In: Deep Generative Models. pp. 45–55 (2024)
work page 2024
- [2]
-
[3]
Medical Image Analysis12, 26–41 (2008)
Avants, B., et al.: Symmetric diffeomorphic image registration with cross- correlation. Medical Image Analysis12, 26–41 (2008)
work page 2008
-
[4]
Balakrishnan, G., et al.: Voxelmorph: A learning framework for deformable medical image registration. IEEE TMI38, 1788–1800 (2019)
work page 2019
-
[5]
Medical Image Analysis82, 102615 (2022)
Chen, J., et al.: Transmorph: Transformer for unsupervised medical image regis- tration. Medical Image Analysis82, 102615 (2022)
work page 2022
-
[6]
Medical Image Analysis 100, 103385 (2025)
Chen, J., et al.: A survey on deep learning in medical image registration. Medical Image Analysis 100, 103385 (2025)
work page 2025
- [7]
-
[8]
Cheng, X., et al.: Winet: Wavelet-based incremental learning for efficient medical image registration. In: MICCAI. pp. 761–771 (2024)
work page 2024
Show all 47 references
-
[9]
Christodoulou, E., et al.: Confidence intervals uncovered: Are we ready for real- world medical imaging AI? In: MICCAI. pp. 124 – 132 (2024)
2024
-
[10]
Journal of Digital Imaging 26(6), 1045–1057 (2013)
Clark, K., et al.: The cancer imaging archive (TCIA). Journal of Digital Imaging 26(6), 1045–1057 (2013)
2013
-
[11]
In: Biomedical Image Registration
Demir, B., et al.: Multigradicon: A foundation model for multimodal medical image registration. In: Biomedical Image Registration. pp. 3–18 (2024)
2024
-
[12]
In: CVPRW
DeTone, D., Malisiewicz, T., Rabinovich, A.: Superpoint: Self-supervised interest point detection and description. In: CVPRW. pp. 337–33712 (2018)
2018
-
[13]
In: ICCV (2015)
Dosovitskiy, A., et al.: Flownet: Learning optical flow with convolutional networks. In: ICCV (2015)
2015
-
[14]
In: MICCAI
Fehrentz, M., et al.: Intraoperative Registration by Cross-Modal Inverse Neural Rendering. In: MICCAI. pp. 317 – 327 (2024)
2024
-
[15]
In: CVPR
Ghahremani, M., et al.: H-vit: A hierarchical vision transformer for deformable image registration. In: CVPR. pp. 11513–11523 (2024)
2024
-
[16]
In: ICML
Guo, C., Pleiss, G., Sun, Y., Weinberger, K.Q.: On calibration of modern neural networks. In: ICML. p. 1321–1330 (2017)
2017
-
[17]
IEEE TMI40 (2021)
Hansen, L., Heinrich, M.P.: Graphregnet: Deep graph regularisation networks on sparse keypoints for dense registration of 3D lung CTs. IEEE TMI40 (2021)
2021
-
[18]
In: MICCAI
Hansen, L., Heinrich, M.P.: Revisiting iterative highly efficient optimisation schemes in medical image registration. In: MICCAI. pp. 203–212 (2021)
2021
-
[19]
In: MICCAI
Heinrich, M.P.: Closing the gap between deep and conventional image registration using probabilistic dense displacement networks. In: MICCAI. pp. 50–58 (2019)
2019
-
[20]
IEEE TMI 42, 697–712 (2023)
Hering, A., et al.: Learn2reg: Comprehensive Multi-Task Medical Image Registra- tion Challenge, Dataset and Evaluation in the Era of Deep Learning. IEEE TMI 42, 697–712 (2023)
2023
-
[21]
IEEE TMI 41(3), 543–558 (2022)
Hoffmann, M., Billot, B., Greve, D.N., Iglesias, J.E., Fischl, B., Dalca, A.V.: Syn- thmorph: Learning contrast-invariant registration without acquired images. IEEE TMI 41(3), 543–558 (2022)
2022
-
[22]
In: NIPS
Jaderberg, M., Simonyan, K., Zisserman, A., Kavukcuoglu, K.: Spatial transformer networks. In: NIPS. p. 2017–2025 (2015)
2015
-
[23]
IEEE TMI29, 196–205 (2010) Implicit Deformable Medical Image Registration with Learnable Kernels 11
Klein, S., Staring, M., Murphy, K., Viergever, M., Pluim, J.: elastix: A toolbox for intensity-based medical image registration. IEEE TMI29, 196–205 (2010) Implicit Deformable Medical Image Registration with Learnable Kernels 11
2010
-
[24]
European Radiology 30(5), 2463–2472 (2020)
Laimer, G., et al.: Minimal ablative margin (MAM) assessment with image fusion. European Radiology 30(5), 2463–2472 (2020)
2020
-
[25]
European Radiology35(2), 1046–1056 (2025)
Laimer, G., et al.: Multicenter and inter-software evaluation of ablative margins after thermal ablation of colorectal liver metastases. European Radiology35(2), 1046–1056 (2025)
2025
-
[26]
IEEE Journal of Biomedical and Health Informatics28(1), 415–426 (2024)
Liu, D., et al.: Local contractive registration with biomechanical model. IEEE Journal of Biomedical and Health Informatics28(1), 415–426 (2024)
2024
-
[27]
Interna- tional Journal of Computer Vision60, 91–110 (2004)
Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Interna- tional Journal of Computer Vision60, 91–110 (2004)
2004
-
[28]
In: ICCV
Mehta, I., et al.: Modulated periodic activations for generalizable local functional representations. In: ICCV. pp. 14194–14203 (2021)
2021
-
[29]
In: CVPR
Meng, M., Feng, D., Bi, L., Kim, J.: Correlation-aware coarse-to-fine mlps for deformable medical image registration. In: CVPR. pp. 9645–9654 (2024)
2024
-
[30]
In: MICCAI
Meng, M., et al.: Non-iterative coarse-to-fine transformer networks for joint affine and deformable image registration. In: MICCAI. pp. 750–760 (2023)
2023
-
[31]
In: ECCV (2020)
Mildenhall, B., et al.: Nerf: Representing scenes as neural radiance fields for view synthesis. In: ECCV (2020)
2020
-
[32]
Com- puter Methods and Programs in Biomedicine98(3), 278–284 (2010)
Modat, M., et al.: Fast free-form deformation using graphics processing units. Com- puter Methods and Programs in Biomedicine98(3), 278–284 (2010)
2010
-
[33]
In: MICCAI
Mok, T.C.W., Chung, A.C.S.: Large Deformation Diffeomorphic Image Registra- tion with Laplacian Pyramid Networks. In: MICCAI. pp. 211–221 (2020)
2020
-
[34]
British Journal of Surgery111(9) (2024)
Paolucci, I., et al.: Identification of a0 minimum ablative margins for colorectal liver metastases. British Journal of Surgery111(9) (2024)
2024
-
[35]
In: MICCAI
Ronneberger, O., et al.: U-net. In: MICCAI. pp. 234–241 (2015)
2015
-
[36]
IEEE TMI44(2), 738–748 (2025)
Siebert, H., et al.: Convexadam. IEEE TMI44(2), 738–748 (2025)
2025
-
[37]
In: NIPS
Sitzmann, V., et al.: Implicit neural representations with periodic activation func- tions. In: NIPS. pp. 7462–7473 (2020)
2020
-
[38]
International Journal of Hyperthermia36, 336–342 (2019)
Solbiati, M., et al.: A novel software platform for volumetric assessment of ablation completeness. International Journal of Hyperthermia36, 336–342 (2019)
2019
-
[39]
In: MICCAI
Tian, L., et al.: uniGradICON: A foundation model for medical image registration. In: MICCAI. pp. 749–760 (2024)
2024
-
[40]
In: NIPS
Tyszkiewicz, M., Fua, P., Trulls, E.: Disk: Learning local features with policy gra- dient. In: NIPS. vol. 33, pp. 14254–14265 (2020)
2020
-
[41]
In: NIPS
Vaswani, A., et al.: Attention is all you need. In: NIPS. vol. 30 (2017)
2017
-
[42]
In: MICCAI
Wang, H., Ni, D., Wang, Y.: Modet: Learning deformable image registration via motion decomposition transformer. In: MICCAI. pp. 740–749 (2023)
2023
-
[43]
IEEE TMI43(6), 2229–2240 (2024)
Wang, H., et al.: Recursive deformable pyramid network for unsupervised medical image registration. IEEE TMI43(6), 2229–2240 (2024)
2024
-
[44]
Radiology: Artificial Intelligence5(5), e230024 (2023)
Wasserthal, J., et al.: Totalsegmentator: Robust segmentation of 104 anatomic structures in ct images. Radiology: Artificial Intelligence5(5), e230024 (2023)
2023
-
[45]
Surgical Endoscopy38(3), 1379–1389 (2024)
Wise, P.A., et al.: Intraoperative liver deformation and organ motion caused by ventilation, laparotomy, and pneumoperitoneum in a porcine model for image- guided liver surgery. Surgical Endoscopy38(3), 1379–1389 (2024)
2024
-
[46]
In: PMLR
Wolterink, J.M., et al.: Implicit neural representations for deformable image reg- istration. In: PMLR. pp. 1349–1359 (2022)
2022
-
[47]
In: ICCV
Zhao, S., et al.: Recursive cascaded networks for unsupervised medical image reg- istration. In: ICCV. pp. 10599–10609 (2019)
2019
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.