pith. sign in

arxiv: 2604.26781 · v1 · submitted 2026-04-29 · 💻 cs.CV

Virtual-reality based patient-specific simulation of spine surgical procedures: A fast, highly automated and high-fidelity system for surgical education and planning

Pith reviewed 2026-05-07 11:03 UTC · model grok-4.3

classification 💻 cs.CV
keywords virtual realitypatient-specific modelingspine surgery simulationimage segmentationCT-MRI registrationsurgical educationspinal decompression
0
0 comments X p. Extension

The pith

An automated AI pipeline generates high-fidelity patient-specific VR models for spine surgery simulation in about 2.5 minutes per case.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper presents a system that uses computer vision to automatically segment and fuse CT and MRI scans into 3D anatomical models for virtual-reality training in spinal decompression procedures. It reports that these models can be built rapidly with Dice similarity coefficients of 0.95 for bone and 0.895 for soft tissue plus a target registration error of 1.73 mm. Surgeon and trainee interviews indicated gains in spatial understanding and procedural confidence. If the approach holds, it would let training and pre-operative planning move away from generic scenarios toward individualized cases without large manual effort or high costs.

Core claim

The central claim is that multimodal registration and segmentation of CT and MRI data can be automated to produce usable 3D models for VR simulation of laminectomy, disc resection, and foraminotomy, with the entire model-construction step completing in roughly 2.5 minutes on 15 cases while meeting the reported accuracy thresholds and receiving positive qualitative feedback on educational value.

What carries the argument

The automated multimodal fusion and segmentation pipeline that registers CT and MRI volumes then extracts vertebral bone and soft-tissue structures for rendering inside a virtual operating room.

If this is right

  • Patient-specific models become practical for routine pre-operative rehearsal.
  • Surgical training can shift from standardized to case-specific scenarios.
  • Time and cost barriers to creating individualized simulations drop substantially.
  • The same models can support post-procedure review and assessment.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The pipeline could be adapted to other image-guided procedures such as joint replacement or cranial surgery.
  • Adding force-feedback hardware would test whether the current visual models translate into improved psychomotor skills.
  • Longitudinal tracking of trainees who use the system might reveal measurable differences in operative time or error rates compared with conventional training.

Load-bearing premise

The measured segmentation and registration accuracies are sufficient to make the resulting VR models faithfully represent real surgical anatomy and provide useful tactile feedback without further clinical outcome testing.

What would settle it

A direct comparison of the automatically generated VR models against expert manual segmentations on the same cases, or a study measuring whether practice in the simulator changes real operating-room performance or complication rates.

read the original abstract

Surgical training involves didactic teaching, mentor-led learning, surgical skills laboratories, and direct exposure to surgery; however, increasing clinical pressures have limited operating room (OR) exposure. This work leverages virtual reality (VR) to provide a safe and immersive training environment. Existing VR training is often based on standardized scenarios not tailored to individual clinical cases. This study addresses this limitation using artificial intelligence (AI) based computer vision methods to generate patient-specific simulations from computed tomography (CT) and magnetic resonance imaging (MRI). This study focuses on patient-specific spinal decompression simulation for spinal stenosis in a virtual operating room. The objectives were (1) automatic creation of 3D anatomical models and (2) VR simulation of spinal decompression procedures including laminectomy, disc resection, and foraminotomy. Model construction required multimodal fusion (registration) of CT and MRI and segmentation of relevant structures. Segmentation was evaluated using the Dice Similarity Coefficient (DSC), and registration accuracy using Target Registration Error (TRE). Qualitative feedback was obtained from surgeons and trainees. High-fidelity patient-specific 3D models were generated efficiently (approximately 2.5 minutes per case, N = 15). Segmentation accuracy was high, with a DSC of 0.95 (+/- 0.03) for vertebral bone and 0.895 (+/- 0.02) for soft tissue structures. Registration accuracy showed a mean TRE of 1.73 (+/- 0.42) mm. Semi-structured interviews indicated improved spatial understanding, increased procedural confidence, and strong perceived educational value. This platform significantly reduced the time and costs of patient-specific modelling, thereby facilitating pre-operative planning, post-procedural assessments, and comprehensive surgical simulation.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 2 minor

Summary. The manuscript presents an AI-based automated pipeline for generating patient-specific 3D anatomical models from CT and MRI scans to enable virtual-reality simulation of spinal decompression procedures (laminectomy, disc resection, foraminotomy) for stenosis cases. It reports efficient model construction (~2.5 min per case on N=15), high segmentation accuracy (DSC 0.95±0.03 for bone, 0.895±0.02 for soft tissue), registration accuracy (mean TRE 1.73±0.42 mm), and positive semi-structured surgeon feedback on spatial understanding and educational value.

Significance. If the reported segmentation and registration accuracies are shown to produce VR models that faithfully support realistic haptic feedback and procedural simulation, the work would offer a practical advance in reducing the time and cost of patient-specific VR surgical training and planning, addressing the limited OR exposure in current training paradigms.

major comments (3)
  1. [Results] Results section (evaluation on 15 cases): The DSC and TRE metrics are presented as evidence of high-fidelity models, but no direct quantitative comparison to expert manual segmentations (gold standard) is reported. This makes it impossible to determine whether residual errors concentrate in surgically critical structures such as the ligamentum flavum, nerve roots, or foraminal boundaries.
  2. [Methods] Methods section (VR environment and simulation description): No quantitative checks are described on how segmentation or registration residuals affect haptic rendering, visual occlusion, or tactile feedback during simulated procedures such as laminectomy or foraminotomy. The claim that the models are suitable for high-fidelity procedural simulation therefore rests on untested assumptions.
  3. [Evaluation] Evaluation (N=15 cases): The accuracy numbers are derived from a small internal cohort without cross-validation, external test sets, or assessment across broader anatomical variability in spinal stenosis. This limits the strength of the generalizability claim for real-world surgical education and planning use.
minor comments (2)
  1. [Abstract] Abstract and Results: Standard deviations are provided for DSC and TRE, but no additional statistical tests, confidence intervals, or inter-rater comparisons are mentioned to contextualize the variability.
  2. [Methods] The manuscript would benefit from clearer notation distinguishing the automated pipeline steps (segmentation vs. multimodal registration) in the Methods flow diagram or pseudocode.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive feedback on our manuscript. We address each of the major comments point by point below, providing clarifications and indicating where revisions will be made to strengthen the paper.

read point-by-point responses
  1. Referee: [Results] Results section (evaluation on 15 cases): The DSC and TRE metrics are presented as evidence of high-fidelity models, but no direct quantitative comparison to expert manual segmentations (gold standard) is reported. This makes it impossible to determine whether residual errors concentrate in surgically critical structures such as the ligamentum flavum, nerve roots, or foraminal boundaries.

    Authors: The DSC metrics were calculated against expert manual segmentations performed by a board-certified radiologist and neurosurgeon, which served as the gold standard. We will revise the Methods section to explicitly describe the ground-truth annotation protocol and add a breakdown or discussion of error distribution in the Results, with particular attention to surgically critical structures such as the ligamentum flavum and foraminal boundaries where the data allow. revision: yes

  2. Referee: [Methods] Methods section (VR environment and simulation description): No quantitative checks are described on how segmentation or registration residuals affect haptic rendering, visual occlusion, or tactile feedback during simulated procedures such as laminectomy or foraminotomy. The claim that the models are suitable for high-fidelity procedural simulation therefore rests on untested assumptions.

    Authors: We agree that no quantitative evaluation of the downstream effects on haptic rendering or tactile feedback was performed; the study focused on the speed and geometric accuracy of the automated pipeline. We will add a paragraph in the Discussion that relates the observed DSC (0.95/0.895) and TRE (1.73 mm) values to published thresholds for acceptable error in VR surgical simulators and will explicitly note the absence of direct haptic validation as a limitation, together with plans for future targeted experiments. revision: partial

  3. Referee: [Evaluation] Evaluation (N=15 cases): The accuracy numbers are derived from a small internal cohort without cross-validation, external test sets, or assessment across broader anatomical variability in spinal stenosis. This limits the strength of the generalizability claim for real-world surgical education and planning use.

    Authors: The evaluation was performed on an internal cohort of 15 consecutive cases as a proof-of-concept demonstration. We will incorporate k-fold cross-validation results where computationally feasible, expand the Discussion to state the limitations of sample size and lack of external validation, and moderate the generalizability language while retaining the observation of consistent performance across the tested cases. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical metrics and interviews only

full rationale

The paper reports direct empirical measurements (DSC for segmentation, TRE for registration) and qualitative surgeon interviews on a pipeline for VR spine models. No equations, derivations, fitted parameters renamed as predictions, or self-citations that bear the central claim are present. All quantitative results are independent measurements against standard metrics; the VR suitability claim rests on those measurements plus user feedback rather than any self-referential construction or reduction to inputs by definition. This is the expected non-finding for a purely empirical methods paper.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on standard domain assumptions in medical imaging rather than new postulates or fitted parameters; no free parameters or invented entities are explicitly introduced beyond routine AI segmentation models.

axioms (1)
  • domain assumption CT and MRI scans contain complementary information that, when fused and segmented by AI, can produce accurate 3D anatomical models suitable for VR surgical simulation.
    Invoked in the model construction step for all 15 cases.

pith-pipeline@v0.9.0 · 5638 in / 1523 out tokens · 79236 ms · 2026-05-07T11:03:24.985281+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

37 extracted references · 26 canonical work pages

  1. [1]

    BMC Med- ical Education24(1), 1297 (2024) https://doi.org/10.1186/s12909-024-06299-w

    Shahrezaei, A., Sohani, M., Taherkhani, S., Zarghami, S.Y.: The impact of surgi- cal simulation and training technologies on general surgery education. BMC Med- ical Education24(1), 1297 (2024) https://doi.org/10.1186/s12909-024-06299-w

  2. [2]

    Clinics in Colon and Rectal Surgery25(3), 156–165 (2012) https://doi.org/10.1055/ s-0032-1322553

    Montbrun, S.L., MacRae, H.: Simulation in surgical education. Clinics in Colon and Rectal Surgery25(3), 156–165 (2012) https://doi.org/10.1055/ s-0032-1322553

  3. [3]

    International Journal of Surgery 110(6), 3326–3337 (2024) https://doi.org/10.1097/JS9.0000000000001579

    Zhang, J., Luo, Z., Zhang, R., Ding, Z., Fang, Y., Han, C., Wu, W., Cen, G., Qiu, Z., Huang, C.: The transition of surgical simulation training and its learning curve: a bibliometric analysis from 2000 to 2023. International Journal of Surgery 110(6), 3326–3337 (2024) https://doi.org/10.1097/JS9.0000000000001579

  4. [4]

    Journal of Surgical Research268, 40–58 (2021) https://doi.org/10.1016/ j.jss.2021.06.045

    Mao, R.Q.,et al.: Immersive virtual reality for surgical training: A systematic review. Journal of Surgical Research268, 40–58 (2021) https://doi.org/10.1016/ j.jss.2021.06.045

  5. [5]

    Computers & Education: X Reality4, 100053 (2024) https://doi.org/10.1016/j.cexr.2024

    Conrad, M., Kablitz, D., Schumann, S.: Learning effectiveness of immersive vir- tual reality in education and training: A systematic review of findings. Computers & Education: X Reality4, 100053 (2024) https://doi.org/10.1016/j.cexr.2024. 100053

  6. [6]

    doi: https://doi.org/10.1016/j

    Javaid, M., Haleem, A.: Virtual reality applications toward medical field. Clinical Epidemiology and Global Health8(1), 77–79 (2020) https://doi.org/10.1016/j. cegh.2019.12.010

  7. [7]

    Spine Journal17(9), 1352–1363 (2017) https://doi.org/10.1016/ j.spinee.2017.05.016

    Pfandler, M.,et al.: Virtual reality-based simulators for spine surgery: a sys- tematic review. Spine Journal17(9), 1352–1363 (2017) https://doi.org/10.1016/ j.spinee.2017.05.016

  8. [8]

    Journal of the American Academy of Orthopaedic Surgeons20(7), 410–422 (2012) https://doi.org/10.5435/JAAOS-20-06-410

    Atesok, K., Mabrey, J.D., Jazrawi, L.M., Egol, K.A.: Surgical simulation in orthopaedic skills training. Journal of the American Academy of Orthopaedic Surgeons20(7), 410–422 (2012) https://doi.org/10.5435/JAAOS-20-06-410

  9. [9]

    Spine Journal15(1), 168–175 (2015) https://doi.org/10.1016/j.spinee.2014.08

    Gottschalk, M.B.,et al.: Surgical training using three-dimensional simulation in placement of cervical lateral mass screws: A blinded randomized control trial. Spine Journal15(1), 168–175 (2015) https://doi.org/10.1016/j.spinee.2014.08. 444

  10. [10]

    Anesthesia and Analgesia135(2), 230–238 (2022) https://doi

    Alam, F., Matava, C.: A new virtual world? the future of immersive environments in anesthesiology. Anesthesia and Analgesia135(2), 230–238 (2022) https://doi. org/10.1213/ANE.0000000000006118

  11. [11]

    Annals of Surgery 236(4), 458–463 (2002) https://doi.org/10.1097/00000658-200210000-00008

    Seymour, N.E., Gallagher, A.G., Roman, S.A., O’Brien, M.K., Bansal, V.K., Andersen, D.K., Satava, R.M.: Virtual reality training improves operating room 20 performance: results of a randomized, double-blinded study. Annals of Surgery 236(4), 458–463 (2002) https://doi.org/10.1097/00000658-200210000-00008

  12. [12]

    Annals of Surgery257(4), 586–593 (2013) https://doi.org/10.1097/SLA.0b013e318288c40b

    Zendejas, B., Brydges, R., Hamstra, S.J., Cook, D.A.: State of the evidence on simulation-based training for laparoscopic surgery: a systematic review. Annals of Surgery257(4), 586–593 (2013) https://doi.org/10.1097/SLA.0b013e318288c40b

  13. [13]

    BMJ Quality & Safety19(Suppl 2), 34–43 (2010) https://doi.org/10.1136/qshc.2009.038562

    Aggarwal, R., Mytton, O.T., Derbrew, M., Hananel, D., Heydenburg, M., Issenberg, S.B., MacAulay, C., Mancini, M.E., Morimoto, T., Soper, N., Ziv, A., Reznick, R., Darzi, A.: Training and simulation for patient safety. BMJ Quality & Safety19(Suppl 2), 34–43 (2010) https://doi.org/10.1136/qshc.2009.038562

  14. [14]

    Healthcare 10(5), 900 (2022) https://doi.org/10.3390/healthcare10050900

    Soriero, D.,et al.: Efficacy of high-resolution preoperative 3d reconstructions for lesion localization in oncological colorectal surgery-first pilot study. Healthcare 10(5), 900 (2022) https://doi.org/10.3390/healthcare10050900

  15. [15]

    Human-Centered Design and Development in Digital Health: Approaches, Challenges, and Emerging Trends

    Vezirska, D., Milev, M., Laleva, L., Nakov, V., Spiriev, T.: Three-dimensional printing in neurosurgery: A review of current indications and applications and a basic methodology for creating a three-dimensional printed model for the neu- rosurgical practice. Cureus14(12), 33153 (2022) https://doi.org/10.7759/cureus. 33153

  16. [16]

    World Neurosurgery: X7, 100076 (2020) https://doi.org/10.1016/j.wnsx.2020

    Costa, F., Alves, O.L., Anania, C.D., Zileli, M., Fornari, M.: Decompressive surgery for lumbar spinal stenosis: Wfns spine committee recommendations. World Neurosurgery: X7, 100076 (2020) https://doi.org/10.1016/j.wnsx.2020. 100076

  17. [17]

    arXiv preprint (2023)

    Klein, G., Hardisty, M., Whyne, C., Martel, A.L.: VertDetect: Fully End-to-End 3D Vertebral Instance Segmentation Model. arXiv preprint (2023). https://doi. org/10.48550/arxiv.2311.09958

  18. [18]

    TotalSegmentator: Robust segmentation of 104 anatomic structures in CT images

    Wasserthal, J.,et al.: Totalsegmentator: Robust segmentation of 104 anatomic structures in ct images. Radiology: Artificial Intelligence5(5), 230024 (2023) https://doi.org/10.1148/ryai.230024

  19. [19]

    BMJ Innovations6(4), 215–219 (2020) https://doi.org/10.1136/ bmjinnov-2019-000398

    Croci, D.M.,et al.: Novel patient-specific 3d-virtual reality visualisation soft- ware (spectovr) for the planning of spine surgery: a case series of eight patients. BMJ Innovations6(4), 215–219 (2020) https://doi.org/10.1136/ bmjinnov-2019-000398

  20. [20]

    World Neurosurgery129, 857–865 (2019) https://doi.org/10.1016/j.wneu.2019.06.057

    Zawy Alsofy, S.,et al.: Virtual reality-based evaluation of surgical planning and outcome of monosegmental, unilateral cervical foraminal stenosis. World Neurosurgery129, 857–865 (2019) https://doi.org/10.1016/j.wneu.2019.06.057

  21. [21]

    Neurospine18(1), 199–205 (2021) https://doi.org/10

    De Salvatore, S., Vadal` a, G., Oggiano, L., Russo, F., Ambrosio, L., Costici, P.F.: Virtual reality in preoperative planning of adolescent idiopathic scoliosis surgery using google cardboard. Neurospine18(1), 199–205 (2021) https://doi.org/10. 21 14245/ns.2040574.287

  22. [22]

    World Neurosurgery140, 674–680 (2020) https: //doi.org/10.1016/j.wneu.2020.04.102

    Luca, A.,et al.: Innovative educational pathways in spine surgery: Advanced virtual reality–based training. World Neurosurgery140, 674–680 (2020) https: //doi.org/10.1016/j.wneu.2020.04.102

  23. [23]

    Journal of Neurosurgery: Case Lessons1(23) (2021) https://doi.org/10.3171/CASE21114

    Anthony, D., et al.: Patient-specific virtual reality technology for complex neu- rosurgical cases: illustrative cases. Journal of Neurosurgery: Case Lessons1(23) (2021) https://doi.org/10.3171/CASE21114

  24. [24]

    NeuroImage145, 24–43 (2017) https://doi.org/10

    De Leener, B.,et al.: Sct: Spinal cord toolbox, an open-source software for pro- cessing spinal cord mri data. NeuroImage145, 24–43 (2017) https://doi.org/10. 1016/j.neuroimage.2016.10.009

  25. [25]

    Scientific Data11(1), 264 (2024) https://doi.org/10.1038/ s41597-024-03090-w

    Graaf, J.W.,et al.: Lumbar spine segmentation in mr images: a dataset and a public benchmark. Scientific Data11(1), 264 (2024) https://doi.org/10.1038/ s41597-024-03090-w

  26. [26]

    Nature Methods18(2), 203–211 (2021) https://doi.org/10.1038/ s41592-020-01008-z

    Isensee, F., Jaeger, P.F., Kohl, S.A.A., Petersen, J., Maier-Hein, K.H.: nnu- net: a self-configuring method for deep learning-based biomedical image seg- mentation. Nature Methods18(2), 203–211 (2021) https://doi.org/10.1038/ s41592-020-01008-z

  27. [27]

    NeuroImage184, 901–915 (2019) https://doi.org/10.1016/j.neuroimage.2018.09.081

    Gros, C.,et al.: Automatic segmentation of the spinal cord and intramedullary multiple sclerosis lesions with convolutional neural networks. NeuroImage184, 901–915 (2019) https://doi.org/10.1016/j.neuroimage.2018.09.081

  28. [28]

    NeuroImage165, 170–179 (2018) https://doi.org/10

    De Leener, B., Fonov, V.S., Collins, D.L., Callot, V., Stikov, N., Cohen-Adad, J.: Pam50: Unbiased multimodal template of the brainstem and spinal cord aligned with the icbm152 space. NeuroImage165, 170–179 (2018) https://doi.org/10. 1016/j.neuroimage.2017.10.041

  29. [29]

    IEEE Transactions on Medical Imaging22(11), 1407–1416 (2003) https://doi.org/10.1109/TMI.2003.819277

    Tomazevic, D., Likar, B., Slivnik, T., Pernus, F.: 3-d/2-d registration of ct and mr to x-ray images. IEEE Transactions on Medical Imaging22(11), 1407–1416 (2003) https://doi.org/10.1109/TMI.2003.819277

  30. [30]

    Computer- Aided Design and Applications17(6), 1313–1325 (2020) https://doi.org/10

    Cukovic, S.,et al.: Rigid 3d registration algorithm for localization of the vertebral centroids in 3d deformity models of adolescent idiopathic scoliosis. Computer- Aided Design and Applications17(6), 1313–1325 (2020) https://doi.org/10. 14733/cadaps.2020.1313-1325

  31. [31]

    3D Slicer as an image computing platform for the Quantitative Imaging Network

    Fedorov, A.,et al.: 3d slicer as an image computing platform for the quantitative imaging network. Magnetic Resonance Imaging30(9), 1323–1341 (2012) https: //doi.org/10.1016/j.mri.2012.05.001

  32. [32]

    Unpublished work (2023) 22

    Vujovic, T.: Continuous Optimization with Piecewise-Decaying Learning Rate Scheduling for Medical Image Registration. Unpublished work (2023) 22

  33. [33]

    In: Biomedical Image Registration, pp

    Heinrich, M.P., Papie˙ z, B.W., Schnabel, J.A., Handels, H.: Non-parametric dis- crete registration with convex optimisation. In: Biomedical Image Registration, pp. 51–61. Springer, ??? (2014). https://doi.org/10.1007/978-3-319-08554-8-6

  34. [34]

    Radiology191(2), 447–454 (1994) https://doi.org/10.1148/radiology.191.2.8153319

    Hill, D.L.G.,et al.: Accurate frameless registration of mr and ct images of the head: applications in planning surgery and radiation therapy. Radiology191(2), 447–454 (1994) https://doi.org/10.1148/radiology.191.2.8153319

  35. [35]

    American Journal of Neuroradiology21(2), 282–289 (2000)

    Panigrahy, A.,et al.: Registration of three-dimensional mr and ct studies of the cervical spine. American Journal of Neuroradiology21(2), 282–289 (2000)

  36. [36]

    Frontiers in Surgery9, 821060 (2022) https://doi

    Reinschluessel, A.V.,et al.: Virtual reality for surgical planning - evaluation based on two liver tumor resections. Frontiers in Surgery9, 821060 (2022) https://doi. org/10.3389/fsurg.2022.821060

  37. [37]

    North American Spine Society Journal6, 100063 (2021) https://doi.org/10.1016/j.xnsj.2021.100063 23

    Chen, T., Chen, M., Liu, Z.,et al.: The impact of virtual reality simulation on learning spinal surgery techniques. North American Spine Society Journal6, 100063 (2021) https://doi.org/10.1016/j.xnsj.2021.100063 23