REVIEW 3 major objections 1 minor 52 references
Neural Galerkin Normalizing Flows for Bayesian Inference of Diffusions with Inaccessible Boundaries
T0 review · 3 major / 1 minor · reviewed 2026-06-28 · grok-4.3
Pith's one-line read Neural Galerkin normalizing flows approximate transition densities of diffusions with inaccessible boundaries to enable efficient Bayesian inference via MCMC.
desk verdict The paper trains a Neural Galerkin normalizing flow offline over initial data and coefficients to approximate transition densities for diffusions with vanishing diffusion at inaccessible boundaries, but the accuracy and stability of that approximation under the singular Dirac initial condition still need concrete checks. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
Neural Galerkin Normalizing Flow architecture for solving the Fokker-Planck equation with Dirac initial condition.
What would settle it
A direct comparison showing that the MCMC posterior obtained using the approximated likelihood differs substantially from the true posterior computed using exact transition densities or highly accurate numerical methods would falsify the claim.
Extended reading notes
Core claim
By solving the Fokker-Planck equation in a Neural Galerkin framework using a new normalizing flow architecture, the method learns the transition density function for diffusions between observation times, with the product of these densities serving as an approximation to the likelihood function that permits efficient Markov chain Monte Carlo sampling for posterior inference.
Load-bearing premise
The proposed architecture accurately solves the Fokker-Planck equation with Dirac initial condition for the considered diffusions across the specified training distribution.
Editorial extensions
If this is right
- Enables cheap posterior sampling via MCMC after offline training.
- Avoids real-time solution of the FP equation for each MCMC proposal.
- Avoids repeated simulation of diffusion bridges required in other likelihood-free methods.
- Handles diffusions where the diffusion matrix vanishes at inaccessible boundaries.
Reading between the lines
- The approach may generalize to other stochastic processes beyond the trained distribution if the architecture captures the underlying dynamics well.
- It could facilitate parameter estimation in high-frequency data scenarios where many transitions are observed.
- Integration with other inference techniques like variational methods might further speed up computations.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a Neural Galerkin Normalizing Flow architecture to solve the Fokker-Planck equation with Dirac initial condition over a training distribution of initial data and diffusion coefficients. This is applied to diffusions with vanishing diffusion matrix at inaccessible boundaries (e.g., Feller-condition stochastic volatility models). The resulting transition densities are multiplied along observed trajectories to approximate the likelihood, enabling efficient MCMC posterior sampling without real-time PDE solves or repeated simulations.
Significance. If the approximation is accurate with controlled per-step error and generalizes beyond the training distribution, the offline-training approach would provide a computationally efficient route to Bayesian inference for parameter-dependent diffusions where analytic transitions are unavailable. The focus on degenerate FP operators is a targeted strength, but the lack of any reported validation leaves the practical significance unestablished.
major comments (3)
- [Abstract] Abstract: the central claim that the product of transition densities 'approximates the likelihood function' and enables 'cheap posterior sampling via MCMC' is load-bearing, yet the abstract supplies no numerical validation, error metrics, baseline comparisons, or tests on vanishing-diffusion cases, so the claim cannot be assessed.
- [Method description] Method (Neural Galerkin framework): no stability analysis of the residual for the degenerate FP operator or a priori error bounds for the Dirac-initial-condition solve are provided; without these, it is unclear whether the learned density remains positive and integrates to one when coefficients lie outside the training support.
- [Training] Training and generalization: the architecture is trained over a distribution of initial data and coefficients, but no out-of-distribution tests or accumulation-of-error analysis for the product along trajectories are described; coherent O(Δt) errors per step would bias the MCMC posterior even if individual solves appear plausible.
minor comments (1)
- Notation for the transition density p(x_{t+Δt}|x_t, θ) and the precise form of the Neural Galerkin loss could be clarified to make the connection to the standard Fokker-Planck operator explicit.
Simulated Author's Rebuttal
We are grateful to the referee for the constructive feedback. We address each major comment below and will revise the manuscript accordingly.
read point-by-point responses
-
Referee: [Abstract] Abstract: the central claim that the product of transition densities 'approximates the likelihood function' and enables 'cheap posterior sampling via MCMC' is load-bearing, yet the abstract supplies no numerical validation, error metrics, baseline comparisons, or tests on vanishing-diffusion cases, so the claim cannot be assessed.
Authors: We agree that the abstract, as currently written, does not reference supporting numerical evidence. In the revision we will add a concise statement summarizing the key validation results (error metrics, baseline comparisons, and performance on vanishing-diffusion models) to make the central claim assessable from the abstract alone. revision: yes
-
Referee: [Method description] Method (Neural Galerkin framework): no stability analysis of the residual for the degenerate FP operator or a priori error bounds for the Dirac-initial-condition solve are provided; without these, it is unclear whether the learned density remains positive and integrates to one when coefficients lie outside the training support.
Authors: The normalizing-flow parameterization guarantees positivity and unit integral by construction for any coefficient values. We acknowledge that the manuscript currently lacks a dedicated stability analysis of the residual for the degenerate operator and a priori bounds. We will add a short subsection providing a stability argument for the Neural Galerkin residual on degenerate Fokker-Planck operators together with preliminary a priori considerations. revision: yes
-
Referee: [Training] Training and generalization: the architecture is trained over a distribution of initial data and coefficients, but no out-of-distribution tests or accumulation-of-error analysis for the product along trajectories are described; coherent O(Δt) errors per step would bias the MCMC posterior even if individual solves appear plausible.
Authors: We agree that the current manuscript does not report explicit out-of-distribution tests or a dedicated accumulation-of-error analysis along trajectories. We will include both in the revised version: additional numerical experiments on parameter/initial-condition values outside the training distribution, and a quantitative study of per-step error propagation and its effect on the MCMC posterior. revision: yes
Circularity Check
No significant circularity; derivation is a standard numerical approximation technique.
full rationale
The paper trains a Neural Galerkin Normalizing Flow to approximate solutions of the Fokker-Planck equation over a distribution of initial data and coefficients, then forms the likelihood as the product of the resulting transition densities along a trajectory. This follows directly from the Markov property of the diffusion and does not reduce any prediction to a fitted input by construction, invoke load-bearing self-citations for uniqueness, or smuggle ansatzes. No quoted steps exhibit self-definition or renaming of known results; the method remains an independent forward solver whose accuracy is an empirical question separate from the derivation chain.
Assumptions & free parameters
assumptions (2)
- domain assumption The Fokker-Planck equation describes the time evolution of the probability density for the diffusion process.
- domain assumption Normalizing flows can represent solutions to the Fokker-Planck PDE with Dirac initial conditions over the training distribution.
Cite this review
Pith. "Pith review of Neural Galerkin Normalizing Flows for Bayesian Inference of Diffusions with Inaccessible Boundaries." pith.science (2026). https://pith.science/paper/JTNSXGA4
@misc{pith2026260604324,
author = {Pith},
title = {Pith review of: Neural Galerkin Normalizing Flows for Bayesian Inference of Diffusions with Inaccessible Boundaries},
year = {2026},
howpublished = {\url{https://pith.science/paper/JTNSXGA4}},
note = {Machine review of arXiv:2606.04324}
}
read the original abstract
One of the primary challenges in Bayesian inference on the parameters of a diffusion model from discrete observations is the unavailability of an analytical expression for the transition density function between consecutive observation times, which is needed to derive the likelihood function. Extending previous studies that solve Fokker-Planck (FP) type partial differential equations with Normalizing Flows, we propose a new Normalizing Flow architecture to learn the transition density function of the diffusion process between two observation times. We do so by solving in a Neural Galerkin framework the associated FP equation with a Dirac mass as initial condition, over a specified training distribution of the initial datum and the coefficients of the diffusion. We specifically focus on processes whose diffusion matrix vanishes in certain inaccessible boundary regions, such as Stochastic Volatility models that satisfy a Feller condition. The product of the obtained transition densities evaluated along the observed trajectory approximates the likelihood function, thereby enabling cheap posterior sampling via Markov chain Monte Carlo (MCMC). After the offline training phase, inference becomes significantly more efficient, as it avoids the need to solve the FP equation in real time for each parameter proposed by the MCMC sampler or to rely on other likelihood-free methods for Bayesian inference that involve repeated simulation of diffusion bridges.
Figures
Figures from the paper (9 more)
Reference graph
Works this paper leans on
-
[1]
Roberts, and Paul Fearnhead
Alexandros Beskos, Omiros Papaspiliopoulos, Gareth O. Roberts, and Paul Fearnhead. Exact and Computationally Efficient Likelihood-Based Estimation for Discretely Observed Diffusion Processes (with Discussion).Journal of the Royal Statistical Society: Series B (Statistical Methodology), 68(3):333–382, 2006
2006
-
[2]
Roberts, Alexandros Beskos, and Paul Fearnhead
Giorgos Sermaidis, Omiros Papaspiliopoulos, Gareth O. Roberts, Alexandros Beskos, and Paul Fearnhead. Markov Chain Monte Carlo for Exact Inference for Diffusions.Scandinavian Journal of Statistics, 40(2):294–321, 2013
2013
-
[3]
MCMC Analysis of Diffusion Models with Application to Finance.Journal of Business & Economic Statistics, 19(2):177–191, 2001
Bjorn Eraker. MCMC Analysis of Diffusion Models with Application to Finance.Journal of Business & Economic Statistics, 19(2):177–191, 2001
2001
-
[4]
Likelihood Inference for Discretely Observed Nonlinear Diffusions.Econometrica, 69(4):959–993, 2001
Omar Elerian, Siddhartha Chib, and Neil Shephard. Likelihood Inference for Discretely Observed Nonlinear Diffusions.Econometrica, 69(4):959–993, 2001
2001
-
[5]
Christophe Andrieu and Gareth O. Roberts. The Pseudo-Marginal Approach for Efficient Monte Carlo Computations.The Annals of Statistics, 37(2):697–725, 2009
2009
-
[6]
Particle Markov chain Monte Carlo methods.Journal of the Royal Statistical Society: Series B (Statistical Methodology), 72(3):269–342, 2010
Christophe Andrieu, Arnaud Doucet, and Roman Holenstein. Particle Markov chain Monte Carlo methods.Journal of the Royal Statistical Society: Series B (Statistical Methodology), 72(3):269–342, 2010
2010
-
[7]
Bayesian Inference for Diffusions with Low-Frequency Observations
Christiane Fuchs. Bayesian Inference for Diffusions with Low-Frequency Observations. In Inference for Diffusion Processes: With Applications in Life Sciences, pages 171–278. Springer, 2013
2013
-
[8]
Roberts and Osnat Stramer
Gareth O. Roberts and Osnat Stramer. On Inference for Non-linear Diffusion Models Using Metropolis-Hastings Algorithms.Biometrika, 88(3):603–621, 2001
2001
Show all 52 references
-
[9]
Springer-Verlag, New York, Heidelberg, 1972
Iosif Ilyich Gikhman and Anatoli Vladimirovich Skorokhod.Stochastic Differential Equations. Springer-Verlag, New York, Heidelberg, 1972
1972
-
[10]
Maziar Raissi, Paris Perdikaris, and George Em Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations.Journal of Computational Physics, 378:686–707, 2019
2019
-
[11]
DGM: A deep learning algorithm for solving partial differential equations.Journal of Computational Physics, 375:1339–1364, 2018
Justin Sirignano and Konstantinos Spiliopoulos. DGM: A deep learning algorithm for solving partial differential equations.Journal of Computational Physics, 375:1339–1364, 2018
2018
-
[12]
Solving high-dimensional partial differential equations using deep learning.Proceedings of the National Academy of Sciences, 115(34):8505– 8510, 2018
Jiequn Han, Arnulf Jentzen, and Weinan E. Solving high-dimensional partial differential equations using deep learning.Proceedings of the National Academy of Sciences, 115(34):8505– 8510, 2018
2018
-
[13]
Kevrekidis, Lu Lu, Paris Perdikaris, Sifan Wang, and Liu Yang
George Em Karniadakis, Ioannis G. Kevrekidis, Lu Lu, Paris Perdikaris, Sifan Wang, and Liu Yang. Physics-informed machine learning.Nature Reviews Physics, 3(6):422–440, 2021
2021
-
[14]
Alfio Quarteroni, Paola Gervasio, and Francesco Regazzoni. Combining physics-based and data-driven models: advancing the frontiers of research with scientific machine learning.Math- ematical Models and Methods in Applied Sciences, page 1–167, March 2025
2025
-
[15]
Haozhe Su, M. V . Tretyakov, and David P. Newton. Deep Learning of Transition Probability Densities for Stochastic Asset Models with Applications in Option Pricing.Management Science, 70(7):4428–4449, 2024
2024
-
[16]
Learning the temporal evolution of multivariate densities via normalizing flows.Chaos: An Interdisciplinary Journal of Nonlinear Science, 32(3), 2022
Yubin Lu, Romit Maulik, Ting Gao, Felix Dietrich, Ioannis G Kevrekidis, and Jinqiao Duan. Learning the temporal evolution of multivariate densities via normalizing flows.Chaos: An Interdisciplinary Journal of Nonlinear Science, 32(3), 2022
2022
-
[17]
Adaptive deep density approximation for Fokker-Planck equations.Journal of Computational Physics, 457:111080, 2022
Kejun Tang, Xiaoliang Wan, and Qifeng Liao. Adaptive deep density approximation for Fokker-Planck equations.Journal of Computational Physics, 457:111080, 2022. 10
2022
-
[18]
Solving Time Dependent Fokker-Planck Equations via Temporal Normalizing Flow.Communications in Computational Physics, 32(2):401–423, January 2022
Xiaodong Feng, Tao Zhou, and Li Zeng. Solving Time Dependent Fokker-Planck Equations via Temporal Normalizing Flow.Communications in Computational Physics, 32(2):401–423, January 2022
2022
-
[19]
Characterizing possible failure modes in physics-informed neural networks.Advances in neural information processing systems, 34:26548–26560, 2021
Aditi Krishnapriyan, Amir Gholami, Shandian Zhe, Robert Kirby, and Michael W Mahoney. Characterizing possible failure modes in physics-informed neural networks.Advances in neural information processing systems, 34:26548–26560, 2021
2021
-
[20]
Mitigating Propagation Failures in Physics-informed Neural Networks using Retain-Resample-Release (R3) Sampling
Arka Daw, Jie Bu, Sifan Wang, Paris Perdikaris, and Anuj Karpatne. Mitigating Propagation Failures in Physics-informed Neural Networks using Retain-Resample-Release (R3) Sampling. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan ...
2023
-
[21]
Neural Galerkin Normalizing Flow for Transition Probabil- ity Density Functions of Diffusion Models.arXiv preprint arXiv:2603.18907, 2026
Riccardo Saporiti and Fabio Nobile. Neural Galerkin Normalizing Flow for Transition Probabil- ity Density Functions of Diffusion Models.arXiv preprint arXiv:2603.18907, 2026
2026
-
[22]
Neural Galerkin schemes with active learning for high-dimensional evolution equations.Journal of Computational Physics, 496:112588, 2024
Joan Bruna, Benjamin Peherstorfer, and Eric Vanden-Eijnden. Neural Galerkin schemes with active learning for high-dimensional evolution equations.Journal of Computational Physics, 496:112588, 2024
2024
-
[23]
Graham, Alexandre H
Matthew M. Graham, Alexandre H. Thiery, and Alexandros Beskos. Manifold Markov Chain Monte Carlo Methods for Bayesian Inference in Diffusion Models.Journal of the Royal Statistical Society Series B: Statistical Methodology, 84(4):1229–1256, 04 2022
2022
-
[24]
Augmented pseudo-marginal Metropolis–Hastings for partially observed diffusion processes.Statistics and Computing, 32(1):21, Feb 2022
Andrew Golightly and Chris Sherlock. Augmented pseudo-marginal Metropolis–Hastings for partially observed diffusion processes.Statistics and Computing, 32(1):21, Feb 2022
2022
-
[25]
Bayesian inference for non-synchronously observed diffusions.SIAM/ASA Journal on Uncertainty Quantification, 14(2):369–393, 2026
Ajay Jasra, Kengo Kamatani, and Amin Wu. Bayesian inference for non-synchronously observed diffusions.SIAM/ASA Journal on Uncertainty Quantification, 14(2):369–393, 2026
2026
-
[26]
Bayesian inference from time se- ries of allele frequency data using exact simulation techniques.arXiv preprint arXiv:2502.12279, 2025
Jaromir Sant, Paul A Jenkins, Jere Koskela, and Dario Spano. Bayesian inference from time se- ries of allele frequency data using exact simulation techniques.arXiv preprint arXiv:2502.12279, 2025
2025
-
[27]
Unbiased Likelihood Estimation of Wright–Fisher Diffusion Processes
Celia García-Pareja and Fabio Nobile. Unbiased Likelihood Estimation of Wright–Fisher Diffusion Processes. InInternational Conference on Monte Carlo and Quasi-Monte Carlo Methods in Scientific Computing, pages 259–275. Springer, 2022
2022
-
[28]
Inference for SDE Models via Approximate Bayesian Computation.Journal of Computational and Graphical Statistics, 23(4):1080–1100, 2014
Umberto Picchini. Inference for SDE Models via Approximate Bayesian Computation.Journal of Computational and Graphical Statistics, 23(4):1080–1100, 2014
2014
-
[29]
Towards Data-Conditional Simula- tion for ABC Inference in Stochastic Differential Equations.Bayesian Analysis, 21(1):309 – 339, 2026
Petar Jovanovski, Andrew Golightly, and Umberto Picchini. Towards Data-Conditional Simula- tion for ABC Inference in Stochastic Differential Equations.Bayesian Analysis, 21(1):309 – 339, 2026
2026
-
[30]
Black-box variational inference for stochastic differential equations
Tom Ryder, Andrew Golightly, A Stephen McGough, and Dennis Prangle. Black-box variational inference for stochastic differential equations. InInternational Conference on Machine Learning, pages 4423–4432. PMLR, 2018
2018
-
[31]
Closed-form likelihood expansions for multivariate diffusions.The Annals of Statistics, 36(2):906–937, 2008
Yacine Aït-Sahalia. Closed-form likelihood expansions for multivariate diffusions.The Annals of Statistics, 36(2):906–937, 2008
2008
-
[32]
Explicit form of approximate transition probability density functions of diffusion processes.Journal of Econometrics, 187(1):57–73, 2015
Seungmoon Choi. Explicit form of approximate transition probability density functions of diffusion processes.Journal of Econometrics, 187(1):57–73, 2015
2015
-
[33]
A new delta expansion for multivariate diffusions via the Itô-Taylor expansion.Journal of Econometrics, 209(2):256–288, 2019
Nian Yang, Nan Chen, and Xiangwei Wan. A new delta expansion for multivariate diffusions via the Itô-Taylor expansion.Journal of Econometrics, 209(2):256–288, 2019
2019
-
[34]
A closed-form transition density expansion for elliptic and hypo-elliptic SDEs.arXiv preprint arXiv:2502.07047, 2025
Yuga Iguchi and Alexandros Beskos. A closed-form transition density expansion for elliptic and hypo-elliptic SDEs.arXiv preprint arXiv:2502.07047, 2025
2025
-
[35]
Pavliotis.Stochastic Processes and Applications
Grigorios A. Pavliotis.Stochastic Processes and Applications. Springer New York, NY , 2014. 11
2014
-
[36]
Boundary conditions for computing densities in hybrid models via PDE methods.Stochastics, 84(5-6):705–718, 2012
Vladimir Lucic. Boundary conditions for computing densities in hybrid models via PDE methods.Stochastics, 84(5-6):705–718, 2012
2012
-
[37]
Calibration of the Heston stochastic local volatility model: A finite volume scheme.International Journal of Financial Engineering, 08(01):2050048, 2021
Bernd Engelmann, Frank Koster, and Daniel Oeltz. Calibration of the Heston stochastic local volatility model: A finite volume scheme.International Journal of Financial Engineering, 08(01):2050048, 2021
2021
-
[38]
LSMR: An Iterative Algorithm for Sparse Least-Squares Problems.SIAM Journal on Scientific Computing, 33(5):2950–2971, 2011
David Chin-Lung Fong and Michael Saunders. LSMR: An Iterative Algorithm for Sparse Least-Squares Problems.SIAM Journal on Scientific Computing, 33(5):2950–2971, 2011
2011
-
[39]
Johnson, Samuel Kotz, and Narayanaswamy Balakrishnan.Continuous Univariate Distributions, Volume 2
Norman L. Johnson, Samuel Kotz, and Narayanaswamy Balakrishnan.Continuous Univariate Distributions, Volume 2. John Wiley & Sons, 1995
1995
-
[40]
Maximum likelihood estimation of stochastic volatility models.Journal of Financial Economics, 83(2):413–452, 2007
Yacine Aït-Sahalia and Robert Kimmel. Maximum likelihood estimation of stochastic volatility models.Journal of Financial Economics, 83(2):413–452, 2007
2007
-
[41]
Cox, Jonathan E
John C. Cox, Jonathan E. Ingersoll, and Stephen A. Ross. A Theory of the Term Structure of Interest Rates.Econometrica, 53(2):385–407, 1985
1985
-
[42]
Lamoureux and A
C. Lamoureux and A. Paseka. Information in Option Prices and the Underlying Asset Dynamics. Working paper, Eller School of Business, University of Arizona, 2005
2005
-
[43]
Radford M. Neal. Slice sampling.The Annals of Statistics, 31(3):705–767, 2003
2003
-
[44]
Springer, Berlin, Heidelberg, 2009
Cédric Villani.Optimal Transport: Old and New, volume 338 ofGrundlehren der Mathematis- chen Wissenschaften. Springer, Berlin, Heidelberg, 2009
2009
-
[45]
Ronald Gallant
Garland B Durham and A. Ronald Gallant. Numerical Techniques for Maximum Likelihood Estimation of Continuous-Time Diffusion Processes.Journal of Business & Economic Statistics, 20(3):297–338, 2002
2002
-
[46]
Bayesian Inference for Irreducible Diffusion Processes Using the Pseudo-Marginal Approach.Bayesian Analysis, 6(2):231–258, 2011
Osnat Stramer and Matthew Bognar. Bayesian Inference for Irreducible Diffusion Processes Using the Pseudo-Marginal Approach.Bayesian Analysis, 6(2):231–258, 2011
2011
-
[47]
Ilya I. Gikhman. A Short Remark on Feller’s Square Root Condition. SSRN Working Paper, 2011
2011
-
[48]
Pytorch: An imperative style, high-performance deep learning library.Advances in neural information processing systems, 32, 2019
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library.Advances in neural information processing systems, 32, 2019
2019
-
[49]
Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows
George Papamakarios, David Sterratt, and Iain Murray. Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows. In Kamalika Chaudhuri and Masashi Sugiyama, editors,Proceedings of the Twenty-Second International Conference on Artificial Intelligenc...
2019
-
[50]
Das, Silverio Foresi, and Rangarajan K
Pierluigi Balduzzi, Sanjiv R. Das, Silverio Foresi, and Rangarajan K. Sundaram. A Simple Approach to Three-Factor Affine Term Structure Models.Journal of Fixed Income, 6(1):43–53, 1996
1996
-
[51]
H. E. Daniels. Saddlepoint Approximations in Statistics.The Annals of Mathematical Statistics, 25(4):631–650, 1954
1954
-
[52]
∂n2,m θl (τ)(x1 m) ∂x1m ≃δ x0((x0)m)
Yaniv Leviathan, Matan Kalman, and Yossi Matias. Fast inference from transformers via speculative decoding. InInternational Conference on Machine Learning, pages 19274–19286. PMLR, 2023. 12 A Structure of the Normalizing Flow The multi-layer schematic of the bounded Normalizin...
2023
Reviewed June 28, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.