REVIEW 5 major objections 5 minor 20 references
Toward Patient-specific Partial Point Cloud to Surface Completion for Pre- to Intra-operative Registration in Image-guided Liver Interventions
T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Completing a partially visible liver surface with a rotation-equivariant occupancy network reduces rigid-registration error from roughly 33–40 mm to 3–5 mm in phantom tests.
desk verdict Patient-specific VN-OccNet completion plus Go-ICP cuts TRE from ~30-40 mm to ~3-5 mm on two phantoms, but the test partial clouds come from the same synthetic pipeline used in training, so clinical transfer is unproven. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
VN-OccNet is a vector-neuron occupancy network: the encoder represents point-cloud features as vectors in SO(3)-equivariant layers, and the decoder predicts occupancy probabilities for query points, with the architecture guaranteeing that a rotation of the input point cloud merely rotates the latent features rather than changing the reconstruction. The decoder outputs occupancy for a query grid; the Multiresolution Isosurface Extraction (MISE) algorithm adaptively subdivides surface-intersecting voxels down to a 128³ resolution, and Marching Cubes turns those voxels into a watertight mesh. A threshold c = 0.4 converts occupancy probabilities into inside/outside labels. The completed mesh's vertices are sampled as a dense target point cloud, and Go-ICP performs globally optimal rigid registration against the pre-operative source cloud. The full chain—completion in about 0.25 s plus registration in about 16 s—is what the experiments evaluate.
What would settle it
Take a real intra-operative liver surface where the true deformation is known from postoperative imaging or tracked fiducials, feed the partial cloud to the patient-specific network, and compare the completed surface to the true deformed surface; if the Chamfer distance or per-point error on real data is much larger than the in silico test values, or if Go-ICP on the completed surface gives TRE comparable to the raw partial cloud, the central claim would be contradicted.
Extended reading notes
Core claim
The central claim is that surface completion, rather than a better registration objective, is the effective remedy for partial intra-operative visibility. Using VN-OccNet—a rotation-equivariant occupancy network—the authors generate a complete watertight liver mesh from a partial point cloud, extract its vertices, and feed them to the Go-ICP rigid registration algorithm. On two in vitro phantoms, this reduces target registration error from roughly 33–40 mm at 30% visibility to 3–5 mm, a statistically significant improvement that persists even when the raw partial cloud is given more visibility (40% and 50%). The same network maintains its reconstruction accuracy across I/I, Z/Z, Z/SO(3), and SO(3)/SO(3) train/test configurations, whereas the non-equivariant OccNet baseline fails in the cross-rotation settings. The paper presents the result as a patient-specific pipeline: each patient's pre-operative model is deformed in simulation to create training data, so the network learns that patient's geometry and deformation patterns rather than a generic liver shape.
Load-bearing premise
The load-bearing premise is that the simulated deformations used for training—a soft-elastic finite-element model of the patient's liver, with stiffness 2–5 kPa and applied forces up to 3 N—match the way a real liver deforms during surgery; if real deformations fall outside this simulated range, the completed surfaces will be wrong and the registration gains will not transfer to patients.
Editorial extensions
If this is right
- At 30% intra-operative visibility, registration with a completed surface achieves TRE of 5.19 ± 1.34 mm (phantom No. 1) and 3.35 ± 0.61 mm (phantom No. 3), several times smaller than the 33–40 mm TRE obtained with the raw partial cloud.
- Increasing the raw cloud's visibility from 30% to 50% only reduces TRE to about 26 mm, still far worse than the completed-surface result, so completion buys more accuracy than acquiring more surface.
- Because VN-OccNet's accuracy is stable across Z-axis and SO(3) rotations without rotation data augmentation, the method removes the need to pre-align intra-operative point clouds before completion.
- The watertight mesh output enables uniform surface sampling, which the authors state is important for registration methods that assume consistent point density.
- The patient-specific training strategy, using simulated neo-Hookean deformations of the pre-operative model, is what allows the network to generalize to the in vitro phantom deformations without generic liver training data.
Reading between the lines
- If the completion step transfers to real intra-operative acquisition, initial rigid registration could become fully automatic: no manual correspondences, no fiducial initialization, and no pre-alignment, because the reconstruction is rotation-agnostic.
- The same recipe—simulate deformations of the patient's own pre-operative anatomy, train an occupancy network, complete, then register—could be applied to other partially visible organs or to laparoscopic video surfaces, though the FEM simulation fidelity would be the main risk.
- The paper itself acknowledges (Discussion) that real intra-operative clouds contain noise, holes, and occlusions absent from the training simulation, and that training a fresh network per patient may limit practicality; these caveats define the main risk to the central claim.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a patient-specific pipeline for rigid registration of pre-operative liver surfaces to partial intra-operative point clouds. A VN-OccNet occupancy network is trained on simulated deformations of a patient-specific liver model to complete partial point clouds into full watertight surfaces, and the completed surface is registered to the pre-operative source with Go-ICP. The authors evaluate the rotation equivariance of VN-OccNet against OccNet under four train/test rotation settings, and report Target Registration Error (TRE) on two in vitro phantoms, showing large improvements when using completed surfaces (5.19 ± 1.34 mm and 3.35 ± 0.61 mm for phantoms No. 1 and No. 3) over the original partial clouds at 30% visibility (33.13 ± 13.89 mm and 39.87 ± 15.07 mm).
Significance. If the reported TRE gains hold under realistic intra-operative conditions, the approach would offer a useful step toward addressing partial visibility in image-guided liver interventions without manual initialization or rigid pre-registration. The systematic comparison of VN-OccNet and OccNet under different rotation settings is a genuine empirical contribution, and the patient-specific simulation strategy is well motivated. However, the empirical support is narrow: only two in vitro phantoms are used, the training and test partial-cloud generation pipelines are matched, and no comparison is made with prior completion-based registration methods. These limitations do not invalidate the proof-of-concept, but they substantially constrain the strength of the clinical claim as currently stated.
major comments (5)
- [§3.4, Table 2] The TRE values in Table 2 and the Wilcoxon Rank Sum test in Section 4.2 are computed over 500 rotated target clouds per phantom, but these 500 clouds are generated by applying 100 random rotations to only five viewpoint-specific partial clouds. The repeated rotations are not independent samples; treating them as such inflates the effective sample size and the reported significance. Please report statistics per unique viewpoint or per phantom (n = 5 per phantom), or use an analysis that accounts for the repeated-measures structure, and state the number of independent cases explicitly.
- [§3.2 and §3.4] The partial point clouds used to obtain the strong TRE reductions in Table 2 are generated by the same pipeline at training and test time: posterior cropping, downsampling to D points, and selecting the N nearest points to a viewpoint. The in vitro test uses the same five-viewpoint procedure, so the 30% visibility condition is exactly the distribution the network was trained on. The Discussion concedes that real intra-operative data may contain noise, holes, and occlusions not represented in this simulation. To support the claim that completion improves registration in the targeted clinical setting, the authors should evaluate on partial inputs produced by a different occlusion/noise model or on real intra-operative data, or explicitly delimit the claim to the matched synthetic occlusion setting.
- [§5 and Related Work] The paper positions its contribution relative to Jia et al. [5] and Foti et al. [2], stating that those methods require rigid initialization or manual correspondences, but it does not compare against them empirically. Since the central claim is that the proposed completion-based registration is an improvement over existing approaches, the absence of any such baseline makes the comparison claim unsubstantiated. At minimum, run the earlier methods on the same phantom data or include a learning-based registration baseline; otherwise soften the claim.
- [§3.1 and §4.2] The registration evaluation is limited to two in vitro phantoms (No. 1 and No. 3), and no real patient data are used. The two phantoms may not capture the anatomical and deformation variability needed to support the general statement that completion 'significantly enhance[s] registration performance.' Please add per-phantom variability, justify the choice of these two phantoms, and discuss how the results would transfer to other patients.
- [§3.1 and §5] The network is trained on deformations from a neo-Hookean FEM simulation with stiffness 2-5 kPa and forces up to 3 N, while the in vitro phantoms are deformed by physical wedges. The manuscript does not quantify the similarity between the simulated deformation space and the actual phantom deformations, so the statement in Section 5 that the model 'generalize[s] effectively to realistic deformations observed in in vitro phantoms' is supported only by indirect TRE measurements on completed surfaces. Provide a direct comparison (e.g., deformation magnitudes, surface displacement fields) or acknowledge this as an assumption.
minor comments (5)
- [§3.5, Eq. (1)] In the second term of the Chamfer Distance definition, 'min_{hat t in P}' should be 'min_{hat t in hat T}'; the set P is undefined.
- [§2.2] In the MISE description, the notation '313 voxels' and '323' should be formatted as '31^3 voxels' and '32^3' for readability.
- [Table 1] Several numerical entries have an errant space after the decimal point (e.g., '0 .47', '3 .51', '0 .86'); these formatting issues should be corrected.
- [Figure 3 caption] The caption refers to 'five target point clouds generated for each intra-operative liver model during testing,' but it is unclear whether the figure shows in silico or in vitro models; clarify how the five viewpoints are selected and reused in Section 3.4.
- [§3.4] The phrase '100 random SO(3) rotations within the range [-pi/2, pi/2]' is imprecise; specify whether the range applies to Euler angles or axis-angle parameters.
Circularity Check
No circularity found: the reported TRE gains are measured on held-out physical phantoms using a completion network trained on simulated deformations; the shared viewpoint-crop protocol is a distributional assumption, not a circular reduction.
full rationale
The paper's central derivation chain is: (1) VN-OccNet is trained on in-silico deformed liver surfaces synthesized from a patient-specific model; (2) at test time, a partial target point cloud is passed through the trained network to produce occupancy probabilities, from which a mesh is extracted via MISE and Marching Cubes; (3) the completed mesh is registered to a source point cloud using Go-ICP; and (4) TRE is measured against fiducials on physical in-vitro phantoms. No step equates the predicted completed surface with the training input by construction. The in-vitro phantom surfaces are segmented from CT and were not used to fit the network weights; only the occupancy threshold c is tuned on the validation set, which is ordinary hyperparameter selection and does not force the reported registration errors. The fact that test-time partial clouds are generated with the same viewpoint-cropping procedure as training makes the evaluation in-distribution with respect to occlusion pattern, but that is an acknowledged generalization limitation stated in the Discussion and Future Work, not a mathematical identity between the completed output and the input. The self-citations to [12], [19], and [17] provide motivation and datasets, but they are not used as uniqueness theorems or as the derivation of the TRE values. Since there is no equation-to-equation reduction and no fitted parameter renamed as a prediction, there is no specific circular step to report.
Assumptions & free parameters
free parameters (5)
- Occupancy threshold c =
0.4
- Anterior surface downsample D =
1000
- Nearest-point count N =
300
- Random rotation range for augmentation =
[-pi/2, pi/2]
- Training epochs =
604
assumptions (5)
- domain assumption Simulated deformations from a neo-Hookean FEM model (Young's modulus 2-5 kPa, Poisson's ratio 0.35, forces up to 3 N) are representative of real intra-operative liver deformations
- domain assumption The pre-operative model is an accurate geometric representation of the intra-operative anatomy
- domain assumption VN-OccNet's rotation-equivariant architecture provides robustness to arbitrary SO(3) rotations
- standard math Go-ICP is a globally optimal and appropriate baseline for rigid registration
- domain assumption The in vitro gelatin phantoms are a valid testbed for registration
Cite this review
Pith. "Pith review of Toward Patient-specific Partial Point Cloud to Surface Completion for Pre- to Intra-operative Registration in Image-guided Liver Interventions." pith.science (2026). https://pith.science/paper/WTKNWI2J
@misc{pith2026250519518,
author = {Pith},
title = {Pith review of: Toward Patient-specific Partial Point Cloud to Surface Completion for Pre- to Intra-operative Registration in Image-guided Liver Interventions},
year = {2026},
howpublished = {\url{https://pith.science/paper/WTKNWI2J}},
note = {Machine review of arXiv:2505.19518}
}
read the original abstract
Intra-operative data captured during image-guided surgery lacks sub-surface information, where key regions of interest, such as vessels and tumors, reside. Image-to-physical registration enables the fusion of pre-operative information and intra-operative data, typically represented as a point cloud. However, this registration process struggles due to partial visibility of the intra-operative point cloud. In this research, we propose a patient-specific point cloud completion approach to assist with the registration process. Specifically, we leverage VN-OccNet to generate a complete liver surface from a partial intra-operative point cloud. The network is trained in a patient-specific manner, where simulated deformations from the pre-operative model are used to train the model. First, we conduct an in-depth analysis of VN-OccNet's rotation-equivariant property and its effectiveness in recovering complete surfaces from partial intra-operative surfaces. Next, we integrate the completed intra-operative surface into the Go-ICP registration algorithm to demonstrate its utility in improving initial rigid registration outcomes. Our results highlight the promise of this patient-specific completion approach in mitigating the challenges posed by partial intra-operative visibility. The rotation equivariant and surface generation capabilities of VN-OccNet hold strong promise for developing robust registration frameworks for variations of the intra-operative point cloud.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[5]
Jia, M., Kyan, M.: Improving intraoperative liver registration in image-guided surgery with learning-based reconstruction. In: ICASSP 2021-2021 IEEE Inter- national Conference on Acoustics, Speech and Signal Processing (ICASSP). pp. 1230–1234. IEEE (2021)
work page 2021
-
[2]
Foti, S., Koo, B., Dowrick, T., Ramalhinho, J., Allam, M., Davidson, B., Stoy- anov, D., Clarkson, M.J.: Intraoperative liver surface completion with graph con- volutional vae. In: Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, and Graphs in Biomedical Image Analysis: Second International Work- shop, UNSURE 2020, and Third Intern...
work page 2020
-
[1]
Deng, C., Litany, O., Duan, Y., Poulenard, A., Tagliasacchi, A., Guibas, L.J.: Vec- tor neurons: A general framework for so (3)-equivariant networks. In: Proceedings 14 N. Poudel et al. of the IEEE/CVF International Conference on Computer Vision. pp. 12200–12209 (2021)
work page 2021
-
[3]
Journal of medical imaging5(2), 021203–021203 (2018)
Heiselman, J.S., Clements, L.W., Collins, J.A., Weis, J.A., Simpson, A.L., Gee- varghese, S.K., Kingham, T.P., Jarnagin, W.R., Miga, M.I.: Characterization and correction of intraoperative soft tissue deformation in image-guided laparoscopic liver surgery. Journal of medical imaging5(2), 021203–021203 (2018)
work page 2018
-
[4]
Journal of Medical Imaging 11(1), 015001–015001 (2024)
Heiselman, J.S., Collins, J.A., Ringel, M.J., Peter Kingham, T., Jarnagin, W.R., Miga, M.I.: The image-to-physical liver registration sparse data challenge: com- parison of state-of-the-art using a common dataset. Journal of Medical Imaging 11(1), 015001–015001 (2024)
work page 2024
-
[6]
Surgical endoscopy29, 3338–3347 (2015)
Kenngott, H.G., Wünscher, J., Wagner, M., Preukschas, A., Wekerle, A.L., Neher, P., Suwelack, S., Speidel, S., Nickel, F., Oladokun, D., et al.: Openhelp (heidel- berg laparoscopy phantom): development of an open-source surgical evaluation and training tool. Surgical endoscopy29, 3338–3347 (2015)
work page 2015
-
[7]
The International Journal of Medical Robotics and Computer Assisted Surgery12(2), 158–178 (2016)
Lin, B., Sun, Y., Qian, X., Goldgof, D., Gitlin, R., You, Y.: Video-based 3D recon- struction, laparoscope localization and deformation recovery for abdominal min- imally invasive surgery: a survey. The International Journal of Medical Robotics and Computer Assisted Surgery12(2), 158–178 (2016)
work page 2016
-
[8]
In: Proceedings of the 14th Annual Conference on Computer Graphics and Interactive Techniques
Lorensen, W.E., Cline, H.E.: Marching cubes: A high resolution 3D sur- face construction algorithm. In: Proceedings of the 14th Annual Conference on Computer Graphics and Interactive Techniques. p. 163–169. SIGGRAPH ’87, Association for Computing Machinery, New York, NY, USA (1987). https://doi.org/10.1145/37401.37422, https://doi.org/10.1145/37401.37422
arXiv 1987
Show all 20 references
-
[9]
In: Medical Imaging 2023: Image-Guided Procedures, Robotic Inter- ventions, and Modeling
Merrell, K., Jackson, P., Simon, R., Linte, C.: Developing and evaluating the fi- delity of patient specific kidney emulating phantoms for image-guided intervention applications. In: Medical Imaging 2023: Image-Guided Procedures, Robotic Inter- ventions, and Modeling. vol. 124...
2023
-
[10]
In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition
Mescheder, L., Oechsle, M., Niemeyer, M., Nowozin, S., Geiger, A.: Occupancy networks: Learning 3D reconstruction in function space. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 4460–4470 (2019)
2019
-
[11]
In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, 2020, Proceedings, Part IV 23
Pfeiffer, M., Riediger, C., Leger, S., Kühn, J.P., Seppelt, D., Hoffmann, R.T., Weitz, J., Speidel, S.: Non-rigid volume to surface registration using a data- driven biomechanical model. In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd Internati...
2020
-
[12]
In: Medical Imaging 2025: Image-Guided Procedures, Robotic Interventions, and Modeling
Poudel, N., Yang, Z., Merrell, K., Simon, R., Linte, C.A.: Evaluation of intraop- erative patient-specific methods for point cloud completion for minimally invasive liver interventions. In: Medical Imaging 2025: Image-Guided Procedures, Robotic Interventions, and Modeling. vol...
2025
-
[13]
International Journal of Computer Assisted Radiology and Surgery 13, 947–956 (2018)
Robu, M.R., Ramalhinho, J., Thompson, S., Gurusamy, K., Davidson, B., Hawkes, D., Stoyanov, D., Clarkson, M.J.: Global rigid registration of ct to video in laparo- scopic liver surgery. International Journal of Computer Assisted Radiology and Surgery 13, 947–956 (2018)
2018
-
[14]
https://doi.org/10.34788/0S3G-QD15, https://www.rit.edu/researchcomputing/
Rochester Institute of Technology: Research computing services (2025). https://doi.org/10.34788/0S3G-QD15, https://www.rit.edu/researchcomputing/
2025 doi
-
[15]
In- ternational Journal of Computer Vision128, 1162–1181 (2020)
Stutz, D., Geiger, A.: Learning 3D shape completion under weak supervision. In- ternational Journal of Computer Vision128, 1162–1181 (2020)
2020
-
[16]
IEEE transactions on pattern analysis and machine intelligence 38(11), 2241–2254 (2015)
Yang, J., Li, H., Campbell, D., Jia, Y.: Go-ICP: A globally optimal solution to 3D icp point-set registration. IEEE transactions on pattern analysis and machine intelligence 38(11), 2241–2254 (2015)
2015
-
[17]
arXiv preprint arXiv:2412.19328 (2024)
Yang, Z., Heiselman, J.S., Han, C., Merrell, K., Simon, R., Linte, C., et al.: Resolv- ing the ambiguity of complete-to-partial point cloud registration for image-guided liver surgery with patches-to-partial matching. arXiv preprint arXiv:2412.19328 (2024)
2024 arXiv
-
[18]
International journal of computer assisted radiology and surgery18(6), 1025–1032 (2023)
Yang, Z., Simon, R., Linte, C.A.: Learning feature descriptors for pre-and intra- operative point cloud matching for laparoscopic liver registration. International journal of computer assisted radiology and surgery18(6), 1025–1032 (2023)
2023
-
[19]
IEEE Transactions on Medical Imaging pp
Yang, Z., Simon, R., Merrell, K., Linte, C.A.: Boundary constraint-free biomechanical model-based surface matching for intraoperative liver defor- mation correction. IEEE Transactions on Medical Imaging pp. 1–1 (2024). https://doi.org/10.1109/TMI.2024.3515632
2024
-
[20]
In: Proceedings of the IEEE/CVF international conference on computer vision
Yu, X., Rao, Y., Wang, Z., Liu, Z., Lu, J., Zhou, J.: Pointr: Diverse point cloud completion with geometry-aware transformers. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 12498–12507 (2021)
2021
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.