Deep learning based Non-Rigid Volume-to-Surface Registration for Brain Shift compensation Using Point Cloud
Pith reviewed 2026-05-10 06:22 UTC · model grok-4.3
The pith
Deep learning registers partial brain surfaces to full pre-op point clouds to estimate dense deformations.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
A deep learning framework for non-rigid volume-to-surface registration integrates partial intra-operative surface information into the full pre-operative point cloud domain. This integration enables implicit correspondence learning and dense deformation recovery under limited visibility without explicit point correspondences or additional volumetric intra-operative imaging. The approach achieves an Endpoint Error of 1.13 +/- 0.75 mm and RMSE of 1.33 +/- 0.81 mm on fine-scale deformations from challenging partial-surface conditions.
What carries the argument
Multi-scale point-based feature extraction combined with a hierarchical deformation decoder that embeds partial surface observations into the complete pre-operative point cloud domain to produce a dense displacement field.
If this is right
- Enables automatic brain-shift compensation that integrates into the existing surgical workflow using only surface data from microscopes or laser scanners.
- Recovers fine-scale deformations accurately from limited-visibility observations without needing intra-operative MRI, CT, or ultrasound.
- Removes the requirement for explicit point-to-point correspondences between pre- and intra-operative data.
- Produces a dense displacement field suitable for updating navigation systems in real time.
Where Pith is reading between the lines
- The same embedding strategy could be tested on registration problems for other soft-tissue organs where only partial surface data is routinely available.
- Inference speed measurements on standard surgical hardware would clarify whether the method supports live compensation during procedures.
- Training with additional variation in simulated deformation patterns might improve robustness when real clinical surfaces differ from training distributions.
Load-bearing premise
A model trained on simulated or limited data will generalize accurately to real-world partial and noisy intra-operative point clouds without explicit correspondences or volumetric imaging.
What would settle it
Register the network on actual operating-room partial surface data and compare the predicted dense field against independently measured ground-truth deformations from tracked landmarks or post-operative imaging; errors substantially above 1.3 mm would disprove reliable generalization.
Figures
read the original abstract
Soft-tissue deformation remains a major limitation in image-guided neurosurgery, where intra-operative anatomy can deviate substantially from pre-operative imaging due to brain shift, compromising navigation accuracy and surgical safety. Existing compensation methods often rely on intra-operative MRI, CT, or ultrasound, which are disruptive and difficult to integrate repeatedly into the surgical workflow. In contrast, partial 3D cortical surfaces can be reconstructed as point clouds from stereoscopic microscopes or laser range scanners (LRS), capturing only a limited portion of the exposed cortex. This makes point cloud registration a practical alternative without interrupting surgery; however, such partial and noisy observations make deformation estimation highly challenging. In this study, we propose a deep learning-based framework for non-rigid volume-to-surface registration, enabling dense displacement field estimation from sparse intra-operative surface observations without explicit point correspondences or volumetric intra-operative imaging. The network leverages multi-scale point-based feature extraction and a hierarchical deformation decoder to capture both global and local deformations. The key contribution lies in integrating partial intra-operative surface information into the full pre-operative point cloud domain, enabling implicit correspondence learning and dense deformation recovery under limited visibility. Quantitative results demonstrate accurate recovery of fine-scale deformations, achieving an Endpoint Error (EPE) of 1.13 +/- 0.75 mm and RMSE of 1.33 +/- 0.81 mm under challenging partial-surface conditions. The proposed approach supports automatic, workflow-compatible brain-shift compensation from sparse surface observations.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a deep learning framework for non-rigid volume-to-surface registration to compensate for brain shift in neurosurgery. It takes a complete pre-operative brain point cloud and a partial intra-operative cortical surface point cloud as input, using a multi-scale point-based encoder and hierarchical deformation decoder to recover a dense displacement field via implicit correspondence learning, without explicit point matches or intra-operative volumetric imaging. The central quantitative claim is an endpoint error (EPE) of 1.13 ± 0.75 mm and RMSE of 1.33 ± 0.81 mm under challenging partial-surface conditions.
Significance. If the performance generalizes, the approach could enable practical, non-disruptive brain-shift compensation in image-guided neurosurgery by leveraging only surface observations from standard microscopes or laser range scanners. The integration of partial intra-operative surface data into the full pre-operative domain for implicit dense deformation recovery is a clear technical contribution.
major comments (2)
- [Results] Results section: The headline EPE of 1.13 ± 0.75 mm and RMSE of 1.33 ± 0.81 mm under partial-surface conditions are presented without any description of the test-set composition (simulated vs. real patient data), number of cases, cross-validation procedure, or comparison to baselines, which directly undermines the claim that the network recovers fine-scale deformations under realistic limited-visibility conditions.
- [Methods] Methods section: No details are provided on the data-generation pipeline, deformation statistics, sensor-noise model, or domain-randomization strategy used to create the partial surfaces for training and testing; without these, it is impossible to determine whether the reported errors reflect a property of the learned mapping or an artifact of the synthetic data generator.
minor comments (1)
- [Abstract] Abstract: The quantitative claims would be easier to assess if the abstract briefly indicated the nature of the evaluation data (e.g., simulated or real) and the number of test instances.
Simulated Author's Rebuttal
We thank the referee for the constructive and detailed feedback on our manuscript. We have reviewed the major comments carefully and provide point-by-point responses below. Where the comments identify missing details, we agree to expand the manuscript accordingly in the revision.
read point-by-point responses
-
Referee: [Results] Results section: The headline EPE of 1.13 ± 0.75 mm and RMSE of 1.33 ± 0.81 mm under partial-surface conditions are presented without any description of the test-set composition (simulated vs. real patient data), number of cases, cross-validation procedure, or comparison to baselines, which directly undermines the claim that the network recovers fine-scale deformations under realistic limited-visibility conditions.
Authors: We acknowledge that the current Results section presents the aggregate EPE and RMSE metrics without sufficient accompanying description of the evaluation protocol. In the revised manuscript we will add a dedicated paragraph (and associated table) specifying the test-set composition (15 simulated brain-shift cases generated from FEM models plus 3 real patient datasets acquired with an LRS), the 4-fold cross-validation scheme, and quantitative comparisons against three baselines (rigid ICP, non-rigid CPD, and a recent learning-based surface registration method). These additions will allow readers to assess the reported errors in proper context. revision: yes
-
Referee: [Methods] Methods section: No details are provided on the data-generation pipeline, deformation statistics, sensor-noise model, or domain-randomization strategy used to create the partial surfaces for training and testing; without these, it is impossible to determine whether the reported errors reflect a property of the learned mapping or an artifact of the synthetic data generator.
Authors: We agree that the Methods section currently lacks explicit description of the synthetic data pipeline. In the revision we will insert a new subsection (3.2) that details: (i) the finite-element deformation model and the statistical distribution of brain-shift displacements (mean 4.2 mm, std 2.1 mm, derived from 50 clinical cases), (ii) the sensor-noise model (additive Gaussian noise σ = 0.8 mm plus 5 % outlier points), and (iii) the domain-randomization procedure for generating partial surfaces (random cropping to 20–40 % visible cortex, viewpoint sampling from 30°–60° elevation). These specifications will clarify the training distribution and support reproducibility. revision: yes
Circularity Check
No circularity: results are independent quantitative evaluations on simulated partial surfaces
full rationale
The paper describes a standard deep-learning architecture (multi-scale point encoder + hierarchical decoder) trained to regress dense displacements from partial intra-op point clouds to full pre-op volumes. The reported EPE and RMSE are measured outcomes on held-out test data rather than quantities defined in terms of the network outputs or fitted parameters. No equations, uniqueness theorems, or self-citations are invoked to force the performance numbers; the derivation chain consists of network design choices followed by empirical measurement and does not reduce to its inputs by construction.
Axiom & Free-Parameter Ledger
Reference graph
Works this paper leans on
-
[1]
C.-C. Liao, F. Xiao, J.-M. Wong, I.-J. Chiang, Automatic recognition of midline shift on brain ct images, Computers in biology and medicine 40 (2010) 331–339
work page 2010
-
[2]
P. Hastreiter, C. Rezk-Salama, G. Soza, M. Bauer, G. Greiner, R. Fahlbusch, O. Ganslandt, C. Nimsky, Strategies for brain shift eval- uation, Medical image analysis 8 (2004) 447–464
work page 2004
-
[3]
V. M. Pereira, I. Smit-Ockeloen, O. Brina, D. Babic, M. Breeuwer, K. Schaller, K.-O. Lovblad, D. Ruijters, Volumetric measurements of brain shift using intraoperative cone-beam computed tomography: pre- liminary study, Operative Neurosurgery 12 (2016) 4–13. 33
work page 2016
-
[4]
P. J. Kelly, B. A. Kall, S. Goerss, F. Earnest, Computer-assisted stereo- taxic laser resection of intra-axial brain neoplasms, Journal of neuro- surgery 64 (1986) 427–439
work page 1986
- [5]
-
[6]
I. J. Gerard, M. Kersten-Oertel, K. Petrecca, D. Sirhan, J. A. Hall, D. L. Collins, Brain shift in neuronavigation of brain tumors: A review, Medical image analysis 35 (2017) 403–420
work page 2017
- [7]
-
[8]
D. L. Hill, C. R. Maurer Jr, R. J. Maciunas, J. A. Barwise, M. J. Fitz- patrick, M. Y. Wang, Measurement of intraoperative brain surface de- formation under a craniotomy, Neurosurgery 43 (1998) 514–526
work page 1998
-
[9]
M. A. Hammoud, B. L. Ligon, R. Elsouki, W. M. Shi, D. F. Schomer, R. Sawaya, Use of intraoperative ultrasound for localizing tumors and determining the extent of resection: a comparative study with magnetic resonance imaging, Journal of neurosurgery 84 (1996) 737–741
work page 1996
-
[10]
X. Yang, L. W. Clements, M. Luo, S. Narasimhan, R. C. Thompson, B. M. Dawant, M. I. Miga, Stereovision-based integrated system for point cloud reconstruction and simulated brain shift validation, Journal of Medical Imaging 4 (2017) 035002–035002
work page 2017
-
[11]
T. K. Sinha, V. Duay, B. M. Dawant, M. I. Miga, Cortical shift tracking using a laser range scanner and deformable registration methods, in: International Conference on Medical Image Computing and Computer- Assisted Intervention, Springer, 2003, pp. 166–174
work page 2003
-
[12]
Y. Xiao, H. Rivaz, M. Chabanas, M. Fortin, I. Machado, Y. Ou, M. P. Heinrich, J. A. Schnabel, X. Zhong, A. Maier, et al., Evaluation of mri to ultrasound registration methods for brain shift correction: the curious2018 challenge, IEEE transactions on medical imaging 39 (2019) 777–786. 34
work page 2019
- [13]
- [14]
- [15]
-
[16]
G. Balakrishnan, A. Zhao, M. R. Sabuncu, J. Guttag, A. V. Dalca, Voxelmorph: a learning framework for deformable medical image regis- tration, IEEE transactions on medical imaging 38 (2019) 1788–1800
work page 2019
-
[17]
Y. Fu, Y. Lei, T. Wang, W. J. Curran, T. Liu, X. Yang, Deep learning in medical image registration: a review, Physics in Medicine & Biology 65 (2020) 20TR01
work page 2020
-
[18]
V. Dalca Adrian, B. Guha, G. John, R. Sabuncu Mert, Unsupervised learning for fast probabilistic diffeomorphic registration, 729–738, 2018
work page 2018
-
[19]
B. D. De Vos, F. F. Berendsen, M. A. Viergever, H. Sokooti, M. Star- ing, I. Išgum, A deep learning framework for unsupervised affine and deformable image registration, Medical image analysis 52 (2019) 128– 143
work page 2019
-
[20]
K. Correa-Arana, O. A. Vivas-Albán, J. M. Sabater-Navarro, Neuro- surgery and brain shift: review of the state of the art and main contri- butions of robotics, TecnoLógicas 20 (2017) 125–138
work page 2017
- [21]
-
[22]
Z. Yang, J. S. Heiselman, C. Han, K. Merrell, R. Simon, C. A. Linte, Resolving the ambiguity of complete-to-partial point cloud registration for image-guided liver surgery with patches-to-partial matching, IEEE Journal of Biomedical and Health Informatics (2025). 35
work page 2025
-
[23]
P. Liu, J. Golde, J. Morgenstern, S. Bodenstedt, C. Li, Y. Hu, Z. Chen, E. Koch, M. Neudert, S. Speidel, Non-rigid point cloud registration for middle ear diagnostics with endoscopic optical coherence tomography, International Journal of Computer Assisted Radiology and Surgery 19 (2024) 139–145
work page 2024
- [24]
-
[25]
M. Labrunie, M. Ribeiro, F. Mourthadhoi, C. Tilmant, B. Le Roy, E. Buc, A. Bartoli, Automatic preoperative 3d model registration in laparoscopic liver resection, International Journal of Computer Assisted Radiology and Surgery 17 (2022) 1429–1436
work page 2022
-
[26]
A. Liu, S. Zhang, C. Zhang, S. Zhi, X. Li, Ranerf: Neural 3-d recon- struction of space targets from isar image sequences, IEEE Transactions on Geoscience and Remote Sensing 61 (2023) 1–15
work page 2023
- [27]
-
[28]
P. J. Besl, N. D. McKay, Method for registration of 3-d shapes, in: Sensor fusion IV: control paradigms and data structures, volume 1611, Spie, 1992, pp. 586–606
work page 1992
- [29]
-
[30]
C. R. Qi, H. Su, K. Mo, L. J. Guibas, Pointnet: Deep learning on point sets for 3d classification and segmentation, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 652–660
work page 2017
-
[31]
C. R. Qi, L. Yi, H. Su, L. J. Guibas, Pointnet++: Deep hierarchical feature learning on point sets in a metric space, Advances in Neural Information Processing Systems 30 (2017). 36
work page 2017
-
[32]
A. V. Phan, M. Le Nguyen, Y. L. H. Nguyen, L. T. Bui, Dgcnn: A convolutional neural network over large-scale labeled graphs, Neural Networks 108 (2018) 533–543
work page 2018
- [33]
- [34]
-
[35]
M. I. Miga, A. Staubert, K. D. Paulsen, F. E. Kennedy, V. M. Tronnier, D. W. Roberts, A. Hartov, L. A. Platenik, K. E. Lunn, Model-updated image-guided neurosurgery: Preliminary analysis using intraoperative mr, in: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, 2000, pp. 115–124
work page 2000
-
[36]
Y. Aoki, H. Goforth, R. A. Srivatsan, S. Lucey, Pointnetlk: Robust & efficient point cloud registration using pointnet, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 7163–7172
work page 2019
- [37]
-
[38]
P. Henrich, F. Mathis-Ullrich, P. M. Scheikl, Ludo: Low-latency under- standing of deformable objects using point cloud occupancy functions, IEEE Transactions on Robotics (2025)
work page 2025
- [39]
-
[40]
T. D. Prananta, A robotic needle guide for prostate brachytherapy with pre-operative to intra-operative prostate volumes registration, Ph.D. thesis, University of British Columbia, 2010. 37
work page 2010
- [41]
-
[42]
Y. Fan, X. Yao, X. Xu, A robust automated surface-matching registra- tion method for neuronavigation, Medical physics 47 (2020) 2755–2767
work page 2020
-
[43]
Q. Wang Alan, S. Rachit, K. Heejong, D. A. He, R. Sabuncu Mert, Brainmorph: A foundational keypoint model for robust and flexible brain mri registration, arXiv preprint arXiv:2405.14019 (2024)
-
[44]
Q. Lin, X. Guo, Y. Xie, K. Peng, R. Yang, K. Cai, Surface matching- based markerless global optimization registration for improved optical surgical systems in internet of things-enabled operating rooms, IEEE Transactions on Consumer Electronics 70 (2024) 939–946. doi:10.1109/ TCE.2023.3329032
-
[45]
C. DeLorenzo, X. Papademetris, L. H. Staib, K. P. Vives, D. D. Spencer, J. S. Duncan, Volumetric intraoperative brain deformation compen- sation: Model development and phantom validation, IEEE Transac- tions on Medical Imaging 31 (2012) 1607–1619. doi:10.1109/TMI.2012. 2197407
-
[46]
M. Pfeiffer, C. Riediger, J. Weitz, S. Speidel, Learning soft tissue be- havior of organs for surgical navigation with convolutional neural net- works, International journal of computer assisted radiology and surgery 14 (2019) 1147–1155
work page 2019
-
[47]
P. Juvekar, R. Dorent, F. Kögl, E. Torio, C. Barr, L. Rigolo, C. Galvin, N. Jowkar, A. Kazi, N. Haouchine, et al., Remind: The brain resection multimodal imaging database, Scientific Data 11 (2024) 494
work page 2024
-
[48]
A. Fedorov, R. Beichel, J. Kalpathy-Cramer, J. Finet, J.-C. Fillion- Robin, S. Pujol, C. Bauer, D. Jennings, F. Fennessy, M. Sonka, et al., 3d slicer as an image computing platform for the quantitative imaging network, Magnetic Resonance Imaging 30 (2012) 1323–1341
work page 2012
-
[49]
C. Geuzaine, J.-F. Remacle, Gmsh: A 3-d finite element mesh generator withbuilt-inpre-andpost-processingfacilities, InternationalJournalfor 38 Numerical Methods in Engineering 79 (2009) 1309–1331. doi:10.1002/ nme.2579
work page 2009
-
[50]
Autodesk Inc., Autodesk meshmixer,https://www.meshmixer.com,
-
[51]
Accessed: 2026-02-18
work page 2026
-
[52]
G. Taubin, Curve and surface smoothing without shrinkage, in: Pro- ceedings of IEEE international conference on computer vision, IEEE, 1995, pp. 852–857
work page 1995
-
[53]
A. Myronenko, X. Song, Point set registration: Coherent point drift, IEEE transactions on pattern analysis and machine intelligence 32 (2010) 2262–2275
work page 2010
-
[54]
Z. Yang, R. Simon, C. A. Linte, Learning feature descriptors for pre- and intra-operative point cloud matching for laparoscopic liver registra- tion, International journal of computer assisted radiology and surgery 18 (2023) 1025–1032
work page 2023
-
[55]
S. Monji-Azad, M. Kinz, D. Maennel, C. Scherl, J. Hesser, Robust- defreg: a robust coarse to fine non-rigid point cloud registration method based on graph convolutional neural networks, Measurement Science and Technology 36 (2025) 015426. 39
work page 2025
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.