pith. sign in

arxiv: 2604.15017 · v1 · submitted 2026-04-16 · 💻 cs.CE

Fully Differentiable Ultrasound Simulation Utilizing Ray-Tracing

Pith reviewed 2026-05-10 09:27 UTC · model grok-4.3

classification 💻 cs.CE
keywords ultrasound simulationdifferentiable renderingMonte Carlo ray tracinginverse problemsB-mode imaginggradient-based optimizationacoustic transport
0
0 comments X

The pith

A fully differentiable end-to-end ultrasound simulation framework based on full-path Monte Carlo ray tracing enables gradient-based optimization over scene and acquisition parameters.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper develops a simulation method for ultrasound that lets users optimize physical parameters such as tissue properties or probe settings by following gradients from image comparisons. It achieves this by making the complete pipeline from sound-wave propagation to final B-mode image differentiable, using Monte Carlo ray tracing for acoustic transport and a custom bridge for beamforming and post-processing. A sympathetic reader would care because prior ultrasound models were either too slow for repeated optimization or could not supply the gradient information needed for calibration, inverse estimation, and acquisition design tasks. Forward examples match expected geometric features, while inverse tests recover parameters from both synthetic references and real experimental images, with finite-difference checks supporting gradient accuracy.

Core claim

We present a fully differentiable end-to-end ultrasound simulation framework based on full-path Monte Carlo ray tracing. Building on UltraRay, the method propagates gradients from image-space losses back through acoustic transport, beamforming, and post-processing, enabling gradient-based optimization over scene and acquisition parameters. The framework combines differentiable ray transport in Mitsuba 3/Dr.Jit with a custom differentiable bridge through the ultrasound image-formation pipeline.

What carries the argument

Full-path Monte Carlo ray tracing with a custom differentiable bridge through the ultrasound image-formation pipeline

Load-bearing premise

The Monte Carlo ray-tracing approximation plus the custom differentiable bridge through beamforming must accurately capture the physics needed for the reported inverse problems.

What would settle it

A mismatch between the framework's computed gradients and independent finite-difference calculations on a controlled phantom, or failure to recover injected ground-truth parameters in a simulation-to-simulation inverse test.

Figures

Figures reproduced from arXiv: 2604.15017 by Collin E. Haese, Felix Kreidel, Issam Moussa, Jan N. Fuhg, L. River Spencer, Manuel K. Rausch, Reagan A. Cardoza, Vijay K. Dubey.

Figure 1
Figure 1. Figure 1: B-mode images for the hollow cylinder test case. [PITH_FULL_IMAGE:figures/full_fig_p010_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Evolution of the right atrioventricular valve from the open to the closed state, shown from top down [ [PITH_FULL_IMAGE:figures/full_fig_p011_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Evolution of the optimized cylinder parameters in the simulated-reference experiment. The impedance [PITH_FULL_IMAGE:figures/full_fig_p012_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: History of the weighted ℓ1 + ℓ2 image-space loss for the simulated-reference optimization problem. The loss decreases rapidly over the course of the optimization and approaches a near-plateau at a very low value, indicating that the differentiable pipeline successfully identifies a parameter set that closely reproduces the simulated reference image. From Figures 4 and 5, the radius exhibits the strongest i… view at source ↗
Figure 5
Figure 5. Figure 5: Automatic-differentiation (AD) and finite-difference (FD) gradient comparisons for the three optimized [PITH_FULL_IMAGE:figures/full_fig_p013_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Comparison of the initial simulated image, the optimized simulated image, and the simulated reference image [PITH_FULL_IMAGE:figures/full_fig_p014_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: (a) Experimental setup used to capture B-mode images of a 3D-printed cylinder. The cylinder and Philips [PITH_FULL_IMAGE:figures/full_fig_p015_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Evolution of the optimized cylinder parameters in the simulation-to-real experiment. The impedance [PITH_FULL_IMAGE:figures/full_fig_p015_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: History of the weighted ℓ1 + ℓ2 image-space loss for the simulation-to-real optimization problem. The loss exhibits a pronounced decrease during the early iterations, followed by gradual stabilization, indicating that the optimizer identifies a parameter regime with substantially improved agreement between the simulated and experimental B-mode images. 15 [PITH_FULL_IMAGE:figures/full_fig_p015_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Automatic-differentiation (AD) and finite-difference (FD) gradient comparisons for the three optimized [PITH_FULL_IMAGE:figures/full_fig_p016_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Overlay comparison of the simulated image (green) and the experimental target (red) before and after [PITH_FULL_IMAGE:figures/full_fig_p017_11.png] view at source ↗
read the original abstract

Ultrasound imaging tasks such as calibration, inverse parameter estimation, and acquisition design require models that are physically grounded, efficient, and differentiable with respect to meaningful material and system parameters. While full-wave solvers offer high fidelity, they are often too expensive for iterative optimization, and existing ray-based methods have mostly been limited to forward simulation. In this work, we present a fully differentiable end-to-end ultrasound simulation framework based on full-path Monte Carlo ray tracing. Building on UltraRay, the method propagates gradients from image-space losses back through acoustic transport, beamforming, and post-processing, enabling gradient-based optimization over scene and acquisition parameters. The framework combines differentiable ray transport in Mitsuba 3/Dr.Jit with a custom differentiable bridge through the ultrasound image-formation pipeline. Forward examples reproduce expected geometric image features and capture more complex anatomical structures. In inverse problems, the method recovers known parameters in a simulated-reference setting and identifies effective parameters that improve agreement between simulated and experimental B-mode images in a simulation-to-real setting. Finite-difference comparisons further support the consistency of the computed gradients. Overall, this work provides a practical foundation for differentiable, physics-based ultrasound simulation and optimization.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 2 minor

Summary. The manuscript presents a fully differentiable end-to-end ultrasound simulation framework based on full-path Monte Carlo ray tracing, extending UltraRay with Mitsuba 3/Dr.Jit. Gradients from image-space losses are propagated back through acoustic transport, beamforming, and post-processing to enable optimization over scene and acquisition parameters. Forward simulations reproduce expected geometric features and complex anatomical structures; inverse tasks recover parameters in sim-to-sim and sim-to-real settings; finite-difference checks confirm gradient consistency.

Significance. If the central claims hold, the work supplies a practical, physics-based differentiable simulator for ultrasound tasks including calibration, inverse parameter estimation, and acquisition design. It advances beyond non-differentiable ray-based methods by integrating a custom bridge through the image-formation pipeline, supporting gradient-based optimization. Strengths include the forward reproduction of geometric and anatomical features, demonstration of parameter recovery across sim-to-sim and sim-to-real regimes, and explicit finite-difference validation of gradients.

major comments (1)
  1. Abstract and inverse-problems section: the claims that the method 'recovers known parameters in a simulated-reference setting' and 'identifies effective parameters that improve agreement' in sim-to-real are not accompanied by quantitative error metrics (e.g., MSE, SSIM, or parameter error norms), ablation studies, or analysis of how post-hoc choices affect differentiability. Without these, it is difficult to evaluate whether the recovered parameters are accurate or merely fitted, undermining assessment of the framework's utility for the stated inverse problems.
minor comments (2)
  1. The description of the custom differentiable bridge through beamforming would benefit from a diagram or explicit pseudocode showing the gradient flow, as the current text leaves the implementation details somewhat opaque.
  2. Notation for acoustic transport parameters and the Monte Carlo sampling strategy should be unified across the methods and results sections to avoid ambiguity when readers attempt to reproduce the gradient checks.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the constructive review and for recognizing the potential utility of the differentiable ultrasound simulation framework. We address the single major comment below and will revise the manuscript to incorporate the suggested improvements.

read point-by-point responses
  1. Referee: Abstract and inverse-problems section: the claims that the method 'recovers known parameters in a simulated-reference setting' and 'identifies effective parameters that improve agreement' in sim-to-real are not accompanied by quantitative error metrics (e.g., MSE, SSIM, or parameter error norms), ablation studies, or analysis of how post-hoc choices affect differentiability. Without these, it is difficult to evaluate whether the recovered parameters are accurate or merely fitted, undermining assessment of the framework's utility for the stated inverse problems.

    Authors: We agree that the inverse-problem results would benefit from quantitative support. The current manuscript demonstrates recovery primarily via visual comparison of B-mode images and parameter values. In the revised version we will add explicit quantitative metrics: MSE and SSIM between optimized and reference images, together with L2 norms on the recovered material and system parameters, for both the sim-to-sim and sim-to-real cases. We will also include ablation studies on post-processing choices (e.g., envelope detection, log compression, and scan-conversion parameters) and analyze their influence on gradient flow and optimization stability. These additions will allow readers to assess the accuracy of the recovered parameters more rigorously. revision: yes

Circularity Check

0 steps flagged

No significant circularity

full rationale

The paper presents an engineering implementation of a differentiable ultrasound simulator by integrating Mitsuba/Dr.Jit ray tracing with UltraRay and a custom differentiable beamforming bridge. Forward images are shown to reproduce geometric features, inverse tasks recover parameters via optimization, and gradients are validated by finite differences. No derivation chain, equation, or result is shown to reduce to its own inputs by construction, self-definition, or a load-bearing self-citation that forces the outcome. The sim-to-real effective-parameter identification is an application of the optimization capability rather than a renamed fit presented as an independent prediction. The work is self-contained as a practical framework with external validation steps.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The framework rests on the validity of ray-tracing for ultrasound and the existence of a differentiable interface through the image pipeline; no new physical entities are introduced.

axioms (2)
  • domain assumption Monte Carlo ray tracing sufficiently approximates acoustic propagation for the anatomical structures and frequencies considered
    Invoked when stating that forward examples reproduce expected geometric features.
  • domain assumption Gradients computed through the custom bridge remain numerically stable and physically meaningful
    Required for the claim that finite-difference comparisons support consistency.

pith-pipeline@v0.9.0 · 5535 in / 1237 out tokens · 36713 ms · 2026-05-10T09:27:41.636249+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

41 extracted references · 41 canonical work pages

  1. [1]

    What is ultrasound?Progress in biophysics and molecular biology, 93(1-3):3–83, 2007

    Timothy G Leighton. What is ultrasound?Progress in biophysics and molecular biology, 93(1-3):3–83, 2007

  2. [2]

    Ultrafast ultrasound imaging in pediatric and adult cardiology: techniques, applications, and perspectives.JACC: Cardiovascular Imaging, 13(8):1771–1791, 2020

    Olivier Villemain, Jérôme Baranger, Mark K Friedberg, Clément Papadacci, Alexandre Dizeux, Emmanuel Messas, Mickael Tanter, Mathieu Pernot, and Luc Mertens. Ultrafast ultrasound imaging in pediatric and adult cardiology: techniques, applications, and perspectives.JACC: Cardiovascular Imaging, 13(8):1771–1791, 2020

  3. [3]

    A short history of the development of ultrasound in obstetrics and gynecology.History of Ultrasound in Obstetrics and Gynecology, 3:1–25, 2002

    Joseph Woo. A short history of the development of ultrasound in obstetrics and gynecology.History of Ultrasound in Obstetrics and Gynecology, 3:1–25, 2002

  4. [4]

    Ultrasound in ocular oncology: Technical advances, clinical applications, and limitations.Experimental Biology and Medicine, 248(5):371–379, 2023

    Arya Kadakia, Junhang Zhang, Xincheng Yao, Qifa Zhou, and Michael J Heiferman. Ultrasound in ocular oncology: Technical advances, clinical applications, and limitations.Experimental Biology and Medicine, 248(5):371–379, 2023

  5. [5]

    Point-of-care ultrasound—history, current and evolving clinical concepts in emergency medicine.Medicina, 59(12):2179, 2023

    Joseph Osterwalder, Effie Polyzogopoulou, and Beatrice Hoffmann. Point-of-care ultrasound—history, current and evolving clinical concepts in emergency medicine.Medicina, 59(12):2179, 2023

  6. [6]

    Physical principles of ultrasound propagation and image formation.Echography of the Eye and Orbit, pages 1–13, 2024

    Pascal Laugier. Physical principles of ultrasound propagation and image formation.Echography of the Eye and Orbit, pages 1–13, 2024

  7. [7]

    Machine learning for medical imaging.radiographics, 37(2):505–515, 2017

    Bradley J Erickson, Panagiotis Korfiatis, Zeynettin Akkus, and Timothy L Kline. Machine learning for medical imaging.radiographics, 37(2):505–515, 2017

  8. [8]

    Synthetic data for deep learning in computer vision & medical imaging: A means to reduce data bias.ACM Computing Surveys, 56(11):1–37, 2024

    Anthony Paproki, Olivier Salvado, and Clinton Fookes. Synthetic data for deep learning in computer vision & medical imaging: A means to reduce data bias.ACM Computing Surveys, 56(11):1–37, 2024

  9. [9]

    A real-time medical ultrasound simulator based on a generative adversarial network model

    B Peng, X Huang, S Wang, and J Jiang. A real-time medical ultrasound simulator based on a generative adversarial network model. In2019 IEEE International Conference on Image Processing (ICIP), pages 4629–4633. IEEE, 2019

  10. [10]

    Ultrasound image simulation with generative adversarial network

    Grace Pigeau, Lydia Elbatarny, Victoria Wu, Abigael Schonewille, Gabor Fichtinger, and Tamas Ungi. Ultrasound image simulation with generative adversarial network. InMedical Imaging 2020: Image-Guided Procedures, Robotic Interventions, and Modeling, volume 11315, pages 54–60. SPIE, 2020

  11. [11]

    Echoflow: a foundation model for cardiac ultrasound image and video generation.arXiv preprint arXiv:2503.22357, 2025

    Hadrien Reynaud, Alberto Gomez, Paul Leeson, Qingjie Meng, and Bernhard Kainz. Echoflow: a foundation model for cardiac ultrasound image and video generation.arXiv preprint arXiv:2503.22357, 2025

  12. [12]

    Learning ultrasound rendering from cross-sectional model slices for simulated training.International journal of computer assisted radiology and surgery, 16(5):721–730, 2021

    Lin Zhang, Tiziano Portenier, and Orcun Goksel. Learning ultrasound rendering from cross-sectional model slices for simulated training.International journal of computer assisted radiology and surgery, 16(5):721–730, 2021. 18 APREPRINT- APRIL17, 2026

  13. [13]

    Ulre-nerf: 3d ultrasound imaging through neural rendering with ultrasound reflection direction parameterization,

    Ziwen Guo, Zi Fang, and Zhuang Fu. Ulre-nerf: 3d ultrasound imaging through neural rendering with ultrasound reflection direction parameterization.arXiv preprint arXiv:2408.00860, 2024

  14. [14]

    Ultrasound signal processing: From models to deep learning.Ultrasound in medicine & biology, 49(3):677–698, 2023

    Ben Luijten, Nishith Chennakeshava, Yonina C Eldar, Massimo Mischi, and Ruud JG van Sloun. Ultrasound signal processing: From models to deep learning.Ultrasound in medicine & biology, 49(3):677–698, 2023

  15. [15]

    MIT press Cambridge, 2016

    Ian Goodfellow, Yoshua Bengio, Aaron Courville, and Yoshua Bengio.Deep learning, volume 1. MIT press Cambridge, 2016

  16. [16]

    Solving inverse problems using data-driven models.Acta numerica, 28:1–174, 2019

    Simon Arridge, Peter Maass, Ozan Öktem, and Carola-Bibiane Schönlieb. Solving inverse problems using data-driven models.Acta numerica, 28:1–174, 2019

  17. [17]

    Deep learning techniques for inverse problems in imaging.IEEE Journal on Selected Areas in Information Theory, 1(1):39–56, 2020

    Gregory Ongie, Ajil Jalal, Christopher A Metzler, Richard G Baraniuk, Alexandros G Dimakis, and Rebecca Willett. Deep learning techniques for inverse problems in imaging.IEEE Journal on Selected Areas in Information Theory, 1(1):39–56, 2020

  18. [18]

    Field characterization of therapeutic ultrasound phased arrays through forward and backward planar projection.The Journal of the Acoustical Society of America, 108(1):441–446, 2000

    GT Clement and Kullervo Hynynen. Field characterization of therapeutic ultrasound phased arrays through forward and backward planar projection.The Journal of the Acoustical Society of America, 108(1):441–446, 2000

  19. [19]

    M S Canney, M R Bailey, L A Crum, V A Khokhlova, and O A Sapozhnikov. Acoustic characterization of high intensity focused ultrasound fields: A combined measurement and modeling approach.The Journal of the Acoustical Society of America, 124(4):2406–2420, 2008

  20. [20]

    Ultrasound imaging and its modeling

    Jørgen A Jensen. Ultrasound imaging and its modeling. InImaging of complex media with acoustic and seismic waves, pages 135–166. Springer, 2002

  21. [21]

    Adaptive beamforming applied to medical ultrasound imaging.IEEE transactions on ultrasonics, ferroelectrics, and frequency control, 54(8):1606–1613, 2007

    Johan Fredrik Synnevag, Andreas Austeng, and Sverre Holm. Adaptive beamforming applied to medical ultrasound imaging.IEEE transactions on ultrasonics, ferroelectrics, and frequency control, 54(8):1606–1613, 2007

  22. [22]

    Differentiable imaging: A new tool for computational optical imaging.Advanced Physics Research, 2(6):2200118, 2023

    Ni Chen, Liangcai Cao, Ting-Chung Poon, Byoungho Lee, and Edmund Y Lam. Differentiable imaging: A new tool for computational optical imaging.Advanced Physics Research, 2(6):2200118, 2023

  23. [23]

    Acoustic wave propagation in complicated geometries and heterogeneous media.Journal of Scientific Computing, 61(1):90–118, 2014

    Kristoffer Virta and Ken Mattsson. Acoustic wave propagation in complicated geometries and heterogeneous media.Journal of Scientific Computing, 61(1):90–118, 2014

  24. [24]

    Wave propagation in heterogeneous, porous media; a velocity-stress, finite-difference method.Geophysics, 60(2):327–340, 1995

    N Dai, A Vafidis, and ER Kanasewich. Wave propagation in heterogeneous, porous media; a velocity-stress, finite-difference method.Geophysics, 60(2):327–340, 1995

  25. [25]

    Wave propagation across acoustic/biot’s media: a finite-difference method.Communications in Computational Physics, 13(4):985–1012, 2013

    Guillaume Chiavassa and Bruno Lombard. Wave propagation across acoustic/biot’s media: a finite-difference method.Communications in Computational Physics, 13(4):985–1012, 2013

  26. [26]

    Anton Van Pamel, Gaofeng Sha, Stanislav I Rokhlin, and Michael JS Lowe. Finite-element modelling of elastic wave propagation and scattering within heterogeneous media.Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 473(2197), 2017

  27. [27]

    Universite de Liege (Belgium), 2022

    Anthony Royer.Efficient finite element methods for solving high-frequency time-harmonic acoustic wave problems in heterogeneous media. Universite de Liege (Belgium), 2022

  28. [28]

    k-wave: Matlab toolbox for the simulation and reconstruction of photoacoustic wave fields.Journal of Biomedical Optics, 15(2):021314–021314, 2010

    B E Treeby and B T Cox. k-wave: Matlab toolbox for the simulation and reconstruction of photoacoustic wave fields.Journal of Biomedical Optics, 15(2):021314–021314, 2010

  29. [29]

    Modelling elastic wave propagation using the k-wave matlab toolbox

    Bradley E Treeby, Jiri Jaros, Daniel Rohrbach, and BT Cox. Modelling elastic wave propagation using the k-wave matlab toolbox. In2014 IEEE international ultrasonics symposium, pages 146–149. IEEE, 2014

  30. [30]

    Juanjuan Gu and Yun Jing. msound: An open source toolbox for modeling acoustic wave propagation in heterogeneous media.IEEE transactions on ultrasonics, ferroelectrics, and frequency control, 68(5):1476–1486, 2021

  31. [31]

    Morgan Kaufmann, 1989

    Andrew S Glassner.An introduction to ray tracing. Morgan Kaufmann, 1989

  32. [32]

    Springer, 2019

    Jon Peddie.Ray tracing: a tool for all, volume 2. Springer, 2019

  33. [33]

    AK Peters, Ltd., 2008

    Peter Shirley and R Keith Morley.Realistic ray tracing. AK Peters, Ltd., 2008

  34. [34]

    Ultraray: Introducing full-path ray tracing in physics-based ultrasound simulation

    Felix Duelmer, Mohammad Farid Azampour, Magdalena Wysocki, and Nassir Navab. Ultraray: Introducing full-path ray tracing in physics-based ultrasound simulation. InMedical Image Computing and Computer Assisted Intervention – MICCAI 2025, volume 15961 ofLecture Notes in Computer Science, pages 653–662. Springer, Cham, 2026

  35. [35]

    Mitsuba 3 renderer, 2022

    Wenzel Jakob, Sébastien Speierer, Nicolas Roussel, Merlin Nimier-David, Delio Vicini, Tizian Zeltner, Baptiste Nicolet, Miguel Crespo, Vincent Leroy, and Ziyi Zhang. Mitsuba 3 renderer, 2022. https://mitsuba-renderer.org

  36. [36]

    Differentiable monte carlo ray tracing through edge sampling.ACM Transactions on Graphics (TOG), 37(6):1–11, 2018

    Tzu-Mao Li, Miika Aittala, Frédo Durand, and Jaakko Lehtinen. Differentiable monte carlo ray tracing through edge sampling.ACM Transactions on Graphics (TOG), 37(6):1–11, 2018. 19 APREPRINT- APRIL17, 2026

  37. [37]

    Stanford University, 1998

    Eric Veach.Robust Monte Carlo methods for light transport simulation. Stanford University, 1998

  38. [38]

    MIT Press, 2023

    Matt Pharr, Wenzel Jakob, and Greg Humphreys.Physically based rendering: From theory to implementation. MIT Press, 2023

  39. [39]

    The morphology of the right atrioventricular valve in the adult human heart.Folia Morphologica, 65(3):200–208, 2006

    M Skwarek, J Hreczecha, M Dudziak, and M Grzybiak. The morphology of the right atrioventricular valve in the adult human heart.Folia Morphologica, 65(3):200–208, 2006

  40. [40]

    Meador, Marcin Malinowski, Tomasz Jazwiec, Tomasz A

    Mrudang Mathur, William D. Meador, Marcin Malinowski, Tomasz Jazwiec, Tomasz A. Timek, and Manuel K. Rausch. Texas TriValve 1.0 : a reverse-engineered, open model of the human tricuspid valve.Engineering with Computers, may 2022

  41. [41]

    Grace N Bechtel, Colton J Kostelnik, and Manuel K Rausch. How well do 3d-printed tissue mimics represent the complex mechanics of biological soft tissues? an example study with stratasys’ cardiovascular tissuematrix materials.Journal of Biomedical Materials Research Part A, 113(1):e37787, 2025. 20