Virtual-reality based patient-specific simulation of spine surgical procedures: A fast, highly automated and high-fidelity system for surgical education and planning
Pith reviewed 2026-05-07 11:03 UTC · model grok-4.3
The pith
An automated AI pipeline generates high-fidelity patient-specific VR models for spine surgery simulation in about 2.5 minutes per case.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central claim is that multimodal registration and segmentation of CT and MRI data can be automated to produce usable 3D models for VR simulation of laminectomy, disc resection, and foraminotomy, with the entire model-construction step completing in roughly 2.5 minutes on 15 cases while meeting the reported accuracy thresholds and receiving positive qualitative feedback on educational value.
What carries the argument
The automated multimodal fusion and segmentation pipeline that registers CT and MRI volumes then extracts vertebral bone and soft-tissue structures for rendering inside a virtual operating room.
If this is right
- Patient-specific models become practical for routine pre-operative rehearsal.
- Surgical training can shift from standardized to case-specific scenarios.
- Time and cost barriers to creating individualized simulations drop substantially.
- The same models can support post-procedure review and assessment.
Where Pith is reading between the lines
- The pipeline could be adapted to other image-guided procedures such as joint replacement or cranial surgery.
- Adding force-feedback hardware would test whether the current visual models translate into improved psychomotor skills.
- Longitudinal tracking of trainees who use the system might reveal measurable differences in operative time or error rates compared with conventional training.
Load-bearing premise
The measured segmentation and registration accuracies are sufficient to make the resulting VR models faithfully represent real surgical anatomy and provide useful tactile feedback without further clinical outcome testing.
What would settle it
A direct comparison of the automatically generated VR models against expert manual segmentations on the same cases, or a study measuring whether practice in the simulator changes real operating-room performance or complication rates.
read the original abstract
Surgical training involves didactic teaching, mentor-led learning, surgical skills laboratories, and direct exposure to surgery; however, increasing clinical pressures have limited operating room (OR) exposure. This work leverages virtual reality (VR) to provide a safe and immersive training environment. Existing VR training is often based on standardized scenarios not tailored to individual clinical cases. This study addresses this limitation using artificial intelligence (AI) based computer vision methods to generate patient-specific simulations from computed tomography (CT) and magnetic resonance imaging (MRI). This study focuses on patient-specific spinal decompression simulation for spinal stenosis in a virtual operating room. The objectives were (1) automatic creation of 3D anatomical models and (2) VR simulation of spinal decompression procedures including laminectomy, disc resection, and foraminotomy. Model construction required multimodal fusion (registration) of CT and MRI and segmentation of relevant structures. Segmentation was evaluated using the Dice Similarity Coefficient (DSC), and registration accuracy using Target Registration Error (TRE). Qualitative feedback was obtained from surgeons and trainees. High-fidelity patient-specific 3D models were generated efficiently (approximately 2.5 minutes per case, N = 15). Segmentation accuracy was high, with a DSC of 0.95 (+/- 0.03) for vertebral bone and 0.895 (+/- 0.02) for soft tissue structures. Registration accuracy showed a mean TRE of 1.73 (+/- 0.42) mm. Semi-structured interviews indicated improved spatial understanding, increased procedural confidence, and strong perceived educational value. This platform significantly reduced the time and costs of patient-specific modelling, thereby facilitating pre-operative planning, post-procedural assessments, and comprehensive surgical simulation.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript presents an AI-based automated pipeline for generating patient-specific 3D anatomical models from CT and MRI scans to enable virtual-reality simulation of spinal decompression procedures (laminectomy, disc resection, foraminotomy) for stenosis cases. It reports efficient model construction (~2.5 min per case on N=15), high segmentation accuracy (DSC 0.95±0.03 for bone, 0.895±0.02 for soft tissue), registration accuracy (mean TRE 1.73±0.42 mm), and positive semi-structured surgeon feedback on spatial understanding and educational value.
Significance. If the reported segmentation and registration accuracies are shown to produce VR models that faithfully support realistic haptic feedback and procedural simulation, the work would offer a practical advance in reducing the time and cost of patient-specific VR surgical training and planning, addressing the limited OR exposure in current training paradigms.
major comments (3)
- [Results] Results section (evaluation on 15 cases): The DSC and TRE metrics are presented as evidence of high-fidelity models, but no direct quantitative comparison to expert manual segmentations (gold standard) is reported. This makes it impossible to determine whether residual errors concentrate in surgically critical structures such as the ligamentum flavum, nerve roots, or foraminal boundaries.
- [Methods] Methods section (VR environment and simulation description): No quantitative checks are described on how segmentation or registration residuals affect haptic rendering, visual occlusion, or tactile feedback during simulated procedures such as laminectomy or foraminotomy. The claim that the models are suitable for high-fidelity procedural simulation therefore rests on untested assumptions.
- [Evaluation] Evaluation (N=15 cases): The accuracy numbers are derived from a small internal cohort without cross-validation, external test sets, or assessment across broader anatomical variability in spinal stenosis. This limits the strength of the generalizability claim for real-world surgical education and planning use.
minor comments (2)
- [Abstract] Abstract and Results: Standard deviations are provided for DSC and TRE, but no additional statistical tests, confidence intervals, or inter-rater comparisons are mentioned to contextualize the variability.
- [Methods] The manuscript would benefit from clearer notation distinguishing the automated pipeline steps (segmentation vs. multimodal registration) in the Methods flow diagram or pseudocode.
Simulated Author's Rebuttal
We thank the referee for the constructive feedback on our manuscript. We address each of the major comments point by point below, providing clarifications and indicating where revisions will be made to strengthen the paper.
read point-by-point responses
-
Referee: [Results] Results section (evaluation on 15 cases): The DSC and TRE metrics are presented as evidence of high-fidelity models, but no direct quantitative comparison to expert manual segmentations (gold standard) is reported. This makes it impossible to determine whether residual errors concentrate in surgically critical structures such as the ligamentum flavum, nerve roots, or foraminal boundaries.
Authors: The DSC metrics were calculated against expert manual segmentations performed by a board-certified radiologist and neurosurgeon, which served as the gold standard. We will revise the Methods section to explicitly describe the ground-truth annotation protocol and add a breakdown or discussion of error distribution in the Results, with particular attention to surgically critical structures such as the ligamentum flavum and foraminal boundaries where the data allow. revision: yes
-
Referee: [Methods] Methods section (VR environment and simulation description): No quantitative checks are described on how segmentation or registration residuals affect haptic rendering, visual occlusion, or tactile feedback during simulated procedures such as laminectomy or foraminotomy. The claim that the models are suitable for high-fidelity procedural simulation therefore rests on untested assumptions.
Authors: We agree that no quantitative evaluation of the downstream effects on haptic rendering or tactile feedback was performed; the study focused on the speed and geometric accuracy of the automated pipeline. We will add a paragraph in the Discussion that relates the observed DSC (0.95/0.895) and TRE (1.73 mm) values to published thresholds for acceptable error in VR surgical simulators and will explicitly note the absence of direct haptic validation as a limitation, together with plans for future targeted experiments. revision: partial
-
Referee: [Evaluation] Evaluation (N=15 cases): The accuracy numbers are derived from a small internal cohort without cross-validation, external test sets, or assessment across broader anatomical variability in spinal stenosis. This limits the strength of the generalizability claim for real-world surgical education and planning use.
Authors: The evaluation was performed on an internal cohort of 15 consecutive cases as a proof-of-concept demonstration. We will incorporate k-fold cross-validation results where computationally feasible, expand the Discussion to state the limitations of sample size and lack of external validation, and moderate the generalizability language while retaining the observation of consistent performance across the tested cases. revision: yes
Circularity Check
No circularity: empirical metrics and interviews only
full rationale
The paper reports direct empirical measurements (DSC for segmentation, TRE for registration) and qualitative surgeon interviews on a pipeline for VR spine models. No equations, derivations, fitted parameters renamed as predictions, or self-citations that bear the central claim are present. All quantitative results are independent measurements against standard metrics; the VR suitability claim rests on those measurements plus user feedback rather than any self-referential construction or reduction to inputs by definition. This is the expected non-finding for a purely empirical methods paper.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption CT and MRI scans contain complementary information that, when fused and segmented by AI, can produce accurate 3D anatomical models suitable for VR surgical simulation.
Reference graph
Works this paper leans on
-
[1]
BMC Med- ical Education24(1), 1297 (2024) https://doi.org/10.1186/s12909-024-06299-w
Shahrezaei, A., Sohani, M., Taherkhani, S., Zarghami, S.Y.: The impact of surgi- cal simulation and training technologies on general surgery education. BMC Med- ical Education24(1), 1297 (2024) https://doi.org/10.1186/s12909-024-06299-w
-
[2]
Clinics in Colon and Rectal Surgery25(3), 156–165 (2012) https://doi.org/10.1055/ s-0032-1322553
Montbrun, S.L., MacRae, H.: Simulation in surgical education. Clinics in Colon and Rectal Surgery25(3), 156–165 (2012) https://doi.org/10.1055/ s-0032-1322553
2012
-
[3]
Zhang, J., Luo, Z., Zhang, R., Ding, Z., Fang, Y., Han, C., Wu, W., Cen, G., Qiu, Z., Huang, C.: The transition of surgical simulation training and its learning curve: a bibliometric analysis from 2000 to 2023. International Journal of Surgery 110(6), 3326–3337 (2024) https://doi.org/10.1097/JS9.0000000000001579
-
[4]
Journal of Surgical Research268, 40–58 (2021) https://doi.org/10.1016/ j.jss.2021.06.045
Mao, R.Q.,et al.: Immersive virtual reality for surgical training: A systematic review. Journal of Surgical Research268, 40–58 (2021) https://doi.org/10.1016/ j.jss.2021.06.045
2021
-
[5]
Computers & Education: X Reality4, 100053 (2024) https://doi.org/10.1016/j.cexr.2024
Conrad, M., Kablitz, D., Schumann, S.: Learning effectiveness of immersive vir- tual reality in education and training: A systematic review of findings. Computers & Education: X Reality4, 100053 (2024) https://doi.org/10.1016/j.cexr.2024. 100053
-
[6]
doi: https://doi.org/10.1016/j
Javaid, M., Haleem, A.: Virtual reality applications toward medical field. Clinical Epidemiology and Global Health8(1), 77–79 (2020) https://doi.org/10.1016/j. cegh.2019.12.010
work page doi:10.1016/j 2020
-
[7]
Spine Journal17(9), 1352–1363 (2017) https://doi.org/10.1016/ j.spinee.2017.05.016
Pfandler, M.,et al.: Virtual reality-based simulators for spine surgery: a sys- tematic review. Spine Journal17(9), 1352–1363 (2017) https://doi.org/10.1016/ j.spinee.2017.05.016
2017
-
[8]
Atesok, K., Mabrey, J.D., Jazrawi, L.M., Egol, K.A.: Surgical simulation in orthopaedic skills training. Journal of the American Academy of Orthopaedic Surgeons20(7), 410–422 (2012) https://doi.org/10.5435/JAAOS-20-06-410
-
[9]
Spine Journal15(1), 168–175 (2015) https://doi.org/10.1016/j.spinee.2014.08
Gottschalk, M.B.,et al.: Surgical training using three-dimensional simulation in placement of cervical lateral mass screws: A blinded randomized control trial. Spine Journal15(1), 168–175 (2015) https://doi.org/10.1016/j.spinee.2014.08. 444
-
[10]
Anesthesia and Analgesia135(2), 230–238 (2022) https://doi
Alam, F., Matava, C.: A new virtual world? the future of immersive environments in anesthesiology. Anesthesia and Analgesia135(2), 230–238 (2022) https://doi. org/10.1213/ANE.0000000000006118
-
[11]
Annals of Surgery 236(4), 458–463 (2002) https://doi.org/10.1097/00000658-200210000-00008
Seymour, N.E., Gallagher, A.G., Roman, S.A., O’Brien, M.K., Bansal, V.K., Andersen, D.K., Satava, R.M.: Virtual reality training improves operating room 20 performance: results of a randomized, double-blinded study. Annals of Surgery 236(4), 458–463 (2002) https://doi.org/10.1097/00000658-200210000-00008
-
[12]
Annals of Surgery257(4), 586–593 (2013) https://doi.org/10.1097/SLA.0b013e318288c40b
Zendejas, B., Brydges, R., Hamstra, S.J., Cook, D.A.: State of the evidence on simulation-based training for laparoscopic surgery: a systematic review. Annals of Surgery257(4), 586–593 (2013) https://doi.org/10.1097/SLA.0b013e318288c40b
-
[13]
BMJ Quality & Safety19(Suppl 2), 34–43 (2010) https://doi.org/10.1136/qshc.2009.038562
Aggarwal, R., Mytton, O.T., Derbrew, M., Hananel, D., Heydenburg, M., Issenberg, S.B., MacAulay, C., Mancini, M.E., Morimoto, T., Soper, N., Ziv, A., Reznick, R., Darzi, A.: Training and simulation for patient safety. BMJ Quality & Safety19(Suppl 2), 34–43 (2010) https://doi.org/10.1136/qshc.2009.038562
-
[14]
Healthcare 10(5), 900 (2022) https://doi.org/10.3390/healthcare10050900
Soriero, D.,et al.: Efficacy of high-resolution preoperative 3d reconstructions for lesion localization in oncological colorectal surgery-first pilot study. Healthcare 10(5), 900 (2022) https://doi.org/10.3390/healthcare10050900
-
[15]
Human-Centered Design and Development in Digital Health: Approaches, Challenges, and Emerging Trends
Vezirska, D., Milev, M., Laleva, L., Nakov, V., Spiriev, T.: Three-dimensional printing in neurosurgery: A review of current indications and applications and a basic methodology for creating a three-dimensional printed model for the neu- rosurgical practice. Cureus14(12), 33153 (2022) https://doi.org/10.7759/cureus. 33153
-
[16]
World Neurosurgery: X7, 100076 (2020) https://doi.org/10.1016/j.wnsx.2020
Costa, F., Alves, O.L., Anania, C.D., Zileli, M., Fornari, M.: Decompressive surgery for lumbar spinal stenosis: Wfns spine committee recommendations. World Neurosurgery: X7, 100076 (2020) https://doi.org/10.1016/j.wnsx.2020. 100076
-
[17]
Klein, G., Hardisty, M., Whyne, C., Martel, A.L.: VertDetect: Fully End-to-End 3D Vertebral Instance Segmentation Model. arXiv preprint (2023). https://doi. org/10.48550/arxiv.2311.09958
-
[18]
TotalSegmentator: Robust segmentation of 104 anatomic structures in CT images
Wasserthal, J.,et al.: Totalsegmentator: Robust segmentation of 104 anatomic structures in ct images. Radiology: Artificial Intelligence5(5), 230024 (2023) https://doi.org/10.1148/ryai.230024
-
[19]
BMJ Innovations6(4), 215–219 (2020) https://doi.org/10.1136/ bmjinnov-2019-000398
Croci, D.M.,et al.: Novel patient-specific 3d-virtual reality visualisation soft- ware (spectovr) for the planning of spine surgery: a case series of eight patients. BMJ Innovations6(4), 215–219 (2020) https://doi.org/10.1136/ bmjinnov-2019-000398
2020
-
[20]
World Neurosurgery129, 857–865 (2019) https://doi.org/10.1016/j.wneu.2019.06.057
Zawy Alsofy, S.,et al.: Virtual reality-based evaluation of surgical planning and outcome of monosegmental, unilateral cervical foraminal stenosis. World Neurosurgery129, 857–865 (2019) https://doi.org/10.1016/j.wneu.2019.06.057
-
[21]
Neurospine18(1), 199–205 (2021) https://doi.org/10
De Salvatore, S., Vadal` a, G., Oggiano, L., Russo, F., Ambrosio, L., Costici, P.F.: Virtual reality in preoperative planning of adolescent idiopathic scoliosis surgery using google cardboard. Neurospine18(1), 199–205 (2021) https://doi.org/10. 21 14245/ns.2040574.287
2021
-
[22]
World Neurosurgery140, 674–680 (2020) https: //doi.org/10.1016/j.wneu.2020.04.102
Luca, A.,et al.: Innovative educational pathways in spine surgery: Advanced virtual reality–based training. World Neurosurgery140, 674–680 (2020) https: //doi.org/10.1016/j.wneu.2020.04.102
-
[23]
Journal of Neurosurgery: Case Lessons1(23) (2021) https://doi.org/10.3171/CASE21114
Anthony, D., et al.: Patient-specific virtual reality technology for complex neu- rosurgical cases: illustrative cases. Journal of Neurosurgery: Case Lessons1(23) (2021) https://doi.org/10.3171/CASE21114
-
[24]
NeuroImage145, 24–43 (2017) https://doi.org/10
De Leener, B.,et al.: Sct: Spinal cord toolbox, an open-source software for pro- cessing spinal cord mri data. NeuroImage145, 24–43 (2017) https://doi.org/10. 1016/j.neuroimage.2016.10.009
2017
-
[25]
Scientific Data11(1), 264 (2024) https://doi.org/10.1038/ s41597-024-03090-w
Graaf, J.W.,et al.: Lumbar spine segmentation in mr images: a dataset and a public benchmark. Scientific Data11(1), 264 (2024) https://doi.org/10.1038/ s41597-024-03090-w
2024
-
[26]
Nature Methods18(2), 203–211 (2021) https://doi.org/10.1038/ s41592-020-01008-z
Isensee, F., Jaeger, P.F., Kohl, S.A.A., Petersen, J., Maier-Hein, K.H.: nnu- net: a self-configuring method for deep learning-based biomedical image seg- mentation. Nature Methods18(2), 203–211 (2021) https://doi.org/10.1038/ s41592-020-01008-z
2021
-
[27]
NeuroImage184, 901–915 (2019) https://doi.org/10.1016/j.neuroimage.2018.09.081
Gros, C.,et al.: Automatic segmentation of the spinal cord and intramedullary multiple sclerosis lesions with convolutional neural networks. NeuroImage184, 901–915 (2019) https://doi.org/10.1016/j.neuroimage.2018.09.081
-
[28]
NeuroImage165, 170–179 (2018) https://doi.org/10
De Leener, B., Fonov, V.S., Collins, D.L., Callot, V., Stikov, N., Cohen-Adad, J.: Pam50: Unbiased multimodal template of the brainstem and spinal cord aligned with the icbm152 space. NeuroImage165, 170–179 (2018) https://doi.org/10. 1016/j.neuroimage.2017.10.041
2018
-
[29]
IEEE Transactions on Medical Imaging22(11), 1407–1416 (2003) https://doi.org/10.1109/TMI.2003.819277
Tomazevic, D., Likar, B., Slivnik, T., Pernus, F.: 3-d/2-d registration of ct and mr to x-ray images. IEEE Transactions on Medical Imaging22(11), 1407–1416 (2003) https://doi.org/10.1109/TMI.2003.819277
-
[30]
Computer- Aided Design and Applications17(6), 1313–1325 (2020) https://doi.org/10
Cukovic, S.,et al.: Rigid 3d registration algorithm for localization of the vertebral centroids in 3d deformity models of adolescent idiopathic scoliosis. Computer- Aided Design and Applications17(6), 1313–1325 (2020) https://doi.org/10. 14733/cadaps.2020.1313-1325
-
[31]
3D Slicer as an image computing platform for the Quantitative Imaging Network
Fedorov, A.,et al.: 3d slicer as an image computing platform for the quantitative imaging network. Magnetic Resonance Imaging30(9), 1323–1341 (2012) https: //doi.org/10.1016/j.mri.2012.05.001
-
[32]
Unpublished work (2023) 22
Vujovic, T.: Continuous Optimization with Piecewise-Decaying Learning Rate Scheduling for Medical Image Registration. Unpublished work (2023) 22
2023
-
[33]
In: Biomedical Image Registration, pp
Heinrich, M.P., Papie˙ z, B.W., Schnabel, J.A., Handels, H.: Non-parametric dis- crete registration with convex optimisation. In: Biomedical Image Registration, pp. 51–61. Springer, ??? (2014). https://doi.org/10.1007/978-3-319-08554-8-6
-
[34]
Radiology191(2), 447–454 (1994) https://doi.org/10.1148/radiology.191.2.8153319
Hill, D.L.G.,et al.: Accurate frameless registration of mr and ct images of the head: applications in planning surgery and radiation therapy. Radiology191(2), 447–454 (1994) https://doi.org/10.1148/radiology.191.2.8153319
-
[35]
American Journal of Neuroradiology21(2), 282–289 (2000)
Panigrahy, A.,et al.: Registration of three-dimensional mr and ct studies of the cervical spine. American Journal of Neuroradiology21(2), 282–289 (2000)
2000
-
[36]
Frontiers in Surgery9, 821060 (2022) https://doi
Reinschluessel, A.V.,et al.: Virtual reality for surgical planning - evaluation based on two liver tumor resections. Frontiers in Surgery9, 821060 (2022) https://doi. org/10.3389/fsurg.2022.821060
-
[37]
North American Spine Society Journal6, 100063 (2021) https://doi.org/10.1016/j.xnsj.2021.100063 23
Chen, T., Chen, M., Liu, Z.,et al.: The impact of virtual reality simulation on learning spinal surgery techniques. North American Spine Society Journal6, 100063 (2021) https://doi.org/10.1016/j.xnsj.2021.100063 23
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.