Fully Differentiable Ultrasound Simulation Utilizing Ray-Tracing
Pith reviewed 2026-05-10 09:27 UTC · model grok-4.3
The pith
A fully differentiable end-to-end ultrasound simulation framework based on full-path Monte Carlo ray tracing enables gradient-based optimization over scene and acquisition parameters.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
We present a fully differentiable end-to-end ultrasound simulation framework based on full-path Monte Carlo ray tracing. Building on UltraRay, the method propagates gradients from image-space losses back through acoustic transport, beamforming, and post-processing, enabling gradient-based optimization over scene and acquisition parameters. The framework combines differentiable ray transport in Mitsuba 3/Dr.Jit with a custom differentiable bridge through the ultrasound image-formation pipeline.
What carries the argument
Full-path Monte Carlo ray tracing with a custom differentiable bridge through the ultrasound image-formation pipeline
Load-bearing premise
The Monte Carlo ray-tracing approximation plus the custom differentiable bridge through beamforming must accurately capture the physics needed for the reported inverse problems.
What would settle it
A mismatch between the framework's computed gradients and independent finite-difference calculations on a controlled phantom, or failure to recover injected ground-truth parameters in a simulation-to-simulation inverse test.
Figures
read the original abstract
Ultrasound imaging tasks such as calibration, inverse parameter estimation, and acquisition design require models that are physically grounded, efficient, and differentiable with respect to meaningful material and system parameters. While full-wave solvers offer high fidelity, they are often too expensive for iterative optimization, and existing ray-based methods have mostly been limited to forward simulation. In this work, we present a fully differentiable end-to-end ultrasound simulation framework based on full-path Monte Carlo ray tracing. Building on UltraRay, the method propagates gradients from image-space losses back through acoustic transport, beamforming, and post-processing, enabling gradient-based optimization over scene and acquisition parameters. The framework combines differentiable ray transport in Mitsuba 3/Dr.Jit with a custom differentiable bridge through the ultrasound image-formation pipeline. Forward examples reproduce expected geometric image features and capture more complex anatomical structures. In inverse problems, the method recovers known parameters in a simulated-reference setting and identifies effective parameters that improve agreement between simulated and experimental B-mode images in a simulation-to-real setting. Finite-difference comparisons further support the consistency of the computed gradients. Overall, this work provides a practical foundation for differentiable, physics-based ultrasound simulation and optimization.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript presents a fully differentiable end-to-end ultrasound simulation framework based on full-path Monte Carlo ray tracing, extending UltraRay with Mitsuba 3/Dr.Jit. Gradients from image-space losses are propagated back through acoustic transport, beamforming, and post-processing to enable optimization over scene and acquisition parameters. Forward simulations reproduce expected geometric features and complex anatomical structures; inverse tasks recover parameters in sim-to-sim and sim-to-real settings; finite-difference checks confirm gradient consistency.
Significance. If the central claims hold, the work supplies a practical, physics-based differentiable simulator for ultrasound tasks including calibration, inverse parameter estimation, and acquisition design. It advances beyond non-differentiable ray-based methods by integrating a custom bridge through the image-formation pipeline, supporting gradient-based optimization. Strengths include the forward reproduction of geometric and anatomical features, demonstration of parameter recovery across sim-to-sim and sim-to-real regimes, and explicit finite-difference validation of gradients.
major comments (1)
- Abstract and inverse-problems section: the claims that the method 'recovers known parameters in a simulated-reference setting' and 'identifies effective parameters that improve agreement' in sim-to-real are not accompanied by quantitative error metrics (e.g., MSE, SSIM, or parameter error norms), ablation studies, or analysis of how post-hoc choices affect differentiability. Without these, it is difficult to evaluate whether the recovered parameters are accurate or merely fitted, undermining assessment of the framework's utility for the stated inverse problems.
minor comments (2)
- The description of the custom differentiable bridge through beamforming would benefit from a diagram or explicit pseudocode showing the gradient flow, as the current text leaves the implementation details somewhat opaque.
- Notation for acoustic transport parameters and the Monte Carlo sampling strategy should be unified across the methods and results sections to avoid ambiguity when readers attempt to reproduce the gradient checks.
Simulated Author's Rebuttal
We thank the referee for the constructive review and for recognizing the potential utility of the differentiable ultrasound simulation framework. We address the single major comment below and will revise the manuscript to incorporate the suggested improvements.
read point-by-point responses
-
Referee: Abstract and inverse-problems section: the claims that the method 'recovers known parameters in a simulated-reference setting' and 'identifies effective parameters that improve agreement' in sim-to-real are not accompanied by quantitative error metrics (e.g., MSE, SSIM, or parameter error norms), ablation studies, or analysis of how post-hoc choices affect differentiability. Without these, it is difficult to evaluate whether the recovered parameters are accurate or merely fitted, undermining assessment of the framework's utility for the stated inverse problems.
Authors: We agree that the inverse-problem results would benefit from quantitative support. The current manuscript demonstrates recovery primarily via visual comparison of B-mode images and parameter values. In the revised version we will add explicit quantitative metrics: MSE and SSIM between optimized and reference images, together with L2 norms on the recovered material and system parameters, for both the sim-to-sim and sim-to-real cases. We will also include ablation studies on post-processing choices (e.g., envelope detection, log compression, and scan-conversion parameters) and analyze their influence on gradient flow and optimization stability. These additions will allow readers to assess the accuracy of the recovered parameters more rigorously. revision: yes
Circularity Check
No significant circularity
full rationale
The paper presents an engineering implementation of a differentiable ultrasound simulator by integrating Mitsuba/Dr.Jit ray tracing with UltraRay and a custom differentiable beamforming bridge. Forward images are shown to reproduce geometric features, inverse tasks recover parameters via optimization, and gradients are validated by finite differences. No derivation chain, equation, or result is shown to reduce to its own inputs by construction, self-definition, or a load-bearing self-citation that forces the outcome. The sim-to-real effective-parameter identification is an application of the optimization capability rather than a renamed fit presented as an independent prediction. The work is self-contained as a practical framework with external validation steps.
Axiom & Free-Parameter Ledger
axioms (2)
- domain assumption Monte Carlo ray tracing sufficiently approximates acoustic propagation for the anatomical structures and frequencies considered
- domain assumption Gradients computed through the custom bridge remain numerically stable and physically meaningful
Reference graph
Works this paper leans on
-
[1]
What is ultrasound?Progress in biophysics and molecular biology, 93(1-3):3–83, 2007
Timothy G Leighton. What is ultrasound?Progress in biophysics and molecular biology, 93(1-3):3–83, 2007
work page 2007
-
[2]
Olivier Villemain, Jérôme Baranger, Mark K Friedberg, Clément Papadacci, Alexandre Dizeux, Emmanuel Messas, Mickael Tanter, Mathieu Pernot, and Luc Mertens. Ultrafast ultrasound imaging in pediatric and adult cardiology: techniques, applications, and perspectives.JACC: Cardiovascular Imaging, 13(8):1771–1791, 2020
work page 2020
-
[3]
Joseph Woo. A short history of the development of ultrasound in obstetrics and gynecology.History of Ultrasound in Obstetrics and Gynecology, 3:1–25, 2002
work page 2002
-
[4]
Arya Kadakia, Junhang Zhang, Xincheng Yao, Qifa Zhou, and Michael J Heiferman. Ultrasound in ocular oncology: Technical advances, clinical applications, and limitations.Experimental Biology and Medicine, 248(5):371–379, 2023
work page 2023
-
[5]
Joseph Osterwalder, Effie Polyzogopoulou, and Beatrice Hoffmann. Point-of-care ultrasound—history, current and evolving clinical concepts in emergency medicine.Medicina, 59(12):2179, 2023
work page 2023
-
[6]
Pascal Laugier. Physical principles of ultrasound propagation and image formation.Echography of the Eye and Orbit, pages 1–13, 2024
work page 2024
-
[7]
Machine learning for medical imaging.radiographics, 37(2):505–515, 2017
Bradley J Erickson, Panagiotis Korfiatis, Zeynettin Akkus, and Timothy L Kline. Machine learning for medical imaging.radiographics, 37(2):505–515, 2017
work page 2017
-
[8]
Anthony Paproki, Olivier Salvado, and Clinton Fookes. Synthetic data for deep learning in computer vision & medical imaging: A means to reduce data bias.ACM Computing Surveys, 56(11):1–37, 2024
work page 2024
-
[9]
A real-time medical ultrasound simulator based on a generative adversarial network model
B Peng, X Huang, S Wang, and J Jiang. A real-time medical ultrasound simulator based on a generative adversarial network model. In2019 IEEE International Conference on Image Processing (ICIP), pages 4629–4633. IEEE, 2019
work page 2019
-
[10]
Ultrasound image simulation with generative adversarial network
Grace Pigeau, Lydia Elbatarny, Victoria Wu, Abigael Schonewille, Gabor Fichtinger, and Tamas Ungi. Ultrasound image simulation with generative adversarial network. InMedical Imaging 2020: Image-Guided Procedures, Robotic Interventions, and Modeling, volume 11315, pages 54–60. SPIE, 2020
work page 2020
-
[11]
Hadrien Reynaud, Alberto Gomez, Paul Leeson, Qingjie Meng, and Bernhard Kainz. Echoflow: a foundation model for cardiac ultrasound image and video generation.arXiv preprint arXiv:2503.22357, 2025
-
[12]
Lin Zhang, Tiziano Portenier, and Orcun Goksel. Learning ultrasound rendering from cross-sectional model slices for simulated training.International journal of computer assisted radiology and surgery, 16(5):721–730, 2021. 18 APREPRINT- APRIL17, 2026
work page 2021
-
[13]
Ziwen Guo, Zi Fang, and Zhuang Fu. Ulre-nerf: 3d ultrasound imaging through neural rendering with ultrasound reflection direction parameterization.arXiv preprint arXiv:2408.00860, 2024
-
[14]
Ben Luijten, Nishith Chennakeshava, Yonina C Eldar, Massimo Mischi, and Ruud JG van Sloun. Ultrasound signal processing: From models to deep learning.Ultrasound in medicine & biology, 49(3):677–698, 2023
work page 2023
-
[15]
Ian Goodfellow, Yoshua Bengio, Aaron Courville, and Yoshua Bengio.Deep learning, volume 1. MIT press Cambridge, 2016
work page 2016
-
[16]
Solving inverse problems using data-driven models.Acta numerica, 28:1–174, 2019
Simon Arridge, Peter Maass, Ozan Öktem, and Carola-Bibiane Schönlieb. Solving inverse problems using data-driven models.Acta numerica, 28:1–174, 2019
work page 2019
-
[17]
Gregory Ongie, Ajil Jalal, Christopher A Metzler, Richard G Baraniuk, Alexandros G Dimakis, and Rebecca Willett. Deep learning techniques for inverse problems in imaging.IEEE Journal on Selected Areas in Information Theory, 1(1):39–56, 2020
work page 2020
-
[18]
GT Clement and Kullervo Hynynen. Field characterization of therapeutic ultrasound phased arrays through forward and backward planar projection.The Journal of the Acoustical Society of America, 108(1):441–446, 2000
work page 2000
-
[19]
M S Canney, M R Bailey, L A Crum, V A Khokhlova, and O A Sapozhnikov. Acoustic characterization of high intensity focused ultrasound fields: A combined measurement and modeling approach.The Journal of the Acoustical Society of America, 124(4):2406–2420, 2008
work page 2008
-
[20]
Ultrasound imaging and its modeling
Jørgen A Jensen. Ultrasound imaging and its modeling. InImaging of complex media with acoustic and seismic waves, pages 135–166. Springer, 2002
work page 2002
-
[21]
Johan Fredrik Synnevag, Andreas Austeng, and Sverre Holm. Adaptive beamforming applied to medical ultrasound imaging.IEEE transactions on ultrasonics, ferroelectrics, and frequency control, 54(8):1606–1613, 2007
work page 2007
-
[22]
Ni Chen, Liangcai Cao, Ting-Chung Poon, Byoungho Lee, and Edmund Y Lam. Differentiable imaging: A new tool for computational optical imaging.Advanced Physics Research, 2(6):2200118, 2023
work page 2023
-
[23]
Kristoffer Virta and Ken Mattsson. Acoustic wave propagation in complicated geometries and heterogeneous media.Journal of Scientific Computing, 61(1):90–118, 2014
work page 2014
-
[24]
N Dai, A Vafidis, and ER Kanasewich. Wave propagation in heterogeneous, porous media; a velocity-stress, finite-difference method.Geophysics, 60(2):327–340, 1995
work page 1995
-
[25]
Guillaume Chiavassa and Bruno Lombard. Wave propagation across acoustic/biot’s media: a finite-difference method.Communications in Computational Physics, 13(4):985–1012, 2013
work page 2013
-
[26]
Anton Van Pamel, Gaofeng Sha, Stanislav I Rokhlin, and Michael JS Lowe. Finite-element modelling of elastic wave propagation and scattering within heterogeneous media.Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 473(2197), 2017
work page 2017
-
[27]
Universite de Liege (Belgium), 2022
Anthony Royer.Efficient finite element methods for solving high-frequency time-harmonic acoustic wave problems in heterogeneous media. Universite de Liege (Belgium), 2022
work page 2022
-
[28]
B E Treeby and B T Cox. k-wave: Matlab toolbox for the simulation and reconstruction of photoacoustic wave fields.Journal of Biomedical Optics, 15(2):021314–021314, 2010
work page 2010
-
[29]
Modelling elastic wave propagation using the k-wave matlab toolbox
Bradley E Treeby, Jiri Jaros, Daniel Rohrbach, and BT Cox. Modelling elastic wave propagation using the k-wave matlab toolbox. In2014 IEEE international ultrasonics symposium, pages 146–149. IEEE, 2014
work page 2014
-
[30]
Juanjuan Gu and Yun Jing. msound: An open source toolbox for modeling acoustic wave propagation in heterogeneous media.IEEE transactions on ultrasonics, ferroelectrics, and frequency control, 68(5):1476–1486, 2021
work page 2021
-
[31]
Andrew S Glassner.An introduction to ray tracing. Morgan Kaufmann, 1989
work page 1989
- [32]
-
[33]
Peter Shirley and R Keith Morley.Realistic ray tracing. AK Peters, Ltd., 2008
work page 2008
-
[34]
Ultraray: Introducing full-path ray tracing in physics-based ultrasound simulation
Felix Duelmer, Mohammad Farid Azampour, Magdalena Wysocki, and Nassir Navab. Ultraray: Introducing full-path ray tracing in physics-based ultrasound simulation. InMedical Image Computing and Computer Assisted Intervention – MICCAI 2025, volume 15961 ofLecture Notes in Computer Science, pages 653–662. Springer, Cham, 2026
work page 2025
-
[35]
Wenzel Jakob, Sébastien Speierer, Nicolas Roussel, Merlin Nimier-David, Delio Vicini, Tizian Zeltner, Baptiste Nicolet, Miguel Crespo, Vincent Leroy, and Ziyi Zhang. Mitsuba 3 renderer, 2022. https://mitsuba-renderer.org
work page 2022
-
[36]
Tzu-Mao Li, Miika Aittala, Frédo Durand, and Jaakko Lehtinen. Differentiable monte carlo ray tracing through edge sampling.ACM Transactions on Graphics (TOG), 37(6):1–11, 2018. 19 APREPRINT- APRIL17, 2026
work page 2018
-
[37]
Eric Veach.Robust Monte Carlo methods for light transport simulation. Stanford University, 1998
work page 1998
-
[38]
Matt Pharr, Wenzel Jakob, and Greg Humphreys.Physically based rendering: From theory to implementation. MIT Press, 2023
work page 2023
-
[39]
M Skwarek, J Hreczecha, M Dudziak, and M Grzybiak. The morphology of the right atrioventricular valve in the adult human heart.Folia Morphologica, 65(3):200–208, 2006
work page 2006
-
[40]
Meador, Marcin Malinowski, Tomasz Jazwiec, Tomasz A
Mrudang Mathur, William D. Meador, Marcin Malinowski, Tomasz Jazwiec, Tomasz A. Timek, and Manuel K. Rausch. Texas TriValve 1.0 : a reverse-engineered, open model of the human tricuspid valve.Engineering with Computers, may 2022
work page 2022
-
[41]
Grace N Bechtel, Colton J Kostelnik, and Manuel K Rausch. How well do 3d-printed tissue mimics represent the complex mechanics of biological soft tissues? an example study with stratasys’ cardiovascular tissuematrix materials.Journal of Biomedical Materials Research Part A, 113(1):e37787, 2025. 20
work page 2025
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.