REVIEW 3 major objections 5 minor 40 references
Deep Operator Networks for Bayesian Parameter Estimation in PDEs
T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read DeepBayONet claims to solve PDEs, estimate unknown parameters, and quantify both aleatoric and epistemic uncertainty in a single training run, with regression coverage of 89.05%.
desk verdict New architecture, but the Bayesian claim is unsupported: the implemented loss is not the derived ELBO. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The engine is the deep operator network with latent modulation: $M_\theta(t,x,\lambda) = \sigma\big(W_0\,(b(t,x)\odot \tau(\lambda)) + b_0\big)$, where the branch network $b$ encodes the spatiotemporal input, the trunk network $\tau$ maps independent standard-normal draws into a distribution over the unknown parameter $\lambda$, and the element-wise product $\odot$ couples the two before the output layer. This coupling lets uncertainty in the parameter flow into the forward solution, and the variational loss (data likelihood plus PDE-residual likelihood plus a KL-type term) trains $\tau$ to be an approximate posterior over $\lambda$. The simplification that replaces the exact KL divergence with $L_{\mathrm{STD}} = \log \sigma_y^2$ is what makes the training practical, and it is the component on which the Bayesian interpretation rests.
What would settle it
Generate many synthetic datasets from a PDE with known parameter $\lambda_{\mathrm{true}}$, train DeepBayONet on each, and measure how often the trunk's 95% credible interval for $\lambda$ contains the true value. If the empirical coverage is far from 95%—especially if it is systematically overconfident—the 'posterior' is not calibrated and the Bayesian claim does not hold.
Extended reading notes
Core claim
The central claim is that modulating a DeepONet's branch activations by a trunk-generated latent variable $\lambda \sim \mathcal{N}(0, I)$ and training with an ELBO-style loss yields an approximate posterior over the PDE parameters rather than a point estimate. The paper argues that this posterior carries both data uncertainty, through a learned observation-noise variance $\sigma_y^2$, and model uncertainty, through the spread of the trunk's output. It demonstrates the claim on a regression benchmark, a 1D unsteady heat equation, a 2D reaction-diffusion equation, and a 3D eigenvalue problem, with the regression experiment showing 89.05% coverage of 95% confidence intervals overall and 97% out-of-distribution coverage.
Load-bearing premise
The load-bearing premise is that drawing $\lambda \sim \mathcal{N}(0,I)$ and multiplying the branch network's hidden activations by $\tau(\lambda)$ produces a valid approximate posterior over the unknown PDE parameters, and that replacing the exact KL divergence with $L_{\mathrm{STD}} = \log \sigma_y^2$ leaves the variational Bayesian interpretation intact.
Editorial extensions
If this is right
- A single training run yields both a forward PDE solution and a posterior over unknown parameters, so the method shortcuts the usual two-stage solve-then-invert pipeline.
- In the regression benchmark, DeepBayONet achieves 89.05% coverage of 95% confidence intervals overall and 97% out-of-distribution, exceeding every tested baseline.
- For the 2D reaction-diffusion problem, the mean recovered reaction rate is $0.999$ with standard deviation $5.4 \times 10^{-4}$, matching the true $k=1$ to three decimal places.
- Withdrawing boundary conditions from the loss widens the posterior and increases residuals at the boundary, indicating the network can express its reduced confidence.
- The 3D eigenvalue experiment suggests the method can pick out the first dipole eigenmode and its eigenvalue in a curved domain without using spherical coordinates.
Reading between the lines
- The trunk's 'posterior' is only as trustworthy as the simplified STD loss; a direct check would be to retrain with the exact KL divergence and compare posterior widths and coverage.
- Because the model does not learn a parametrized solution operator (the paper states this explicitly), an untested regime is extrapolation to PDE coefficients or source terms that differ from the training instance.
- The multiplicative perturbation of branch features is formally similar to multiplicative noise in dropout, so a fair cost/benefit test would compare against MC-dropout with a learned input-dependent variance on the same benchmarks.
- For the eigenvalue problem, the normalization term $(E[u^2]-1)^2$ is the only thing preventing collapse to $u\equiv 0$; a stressed test would be searching for higher or degenerate eigenvalues where mode separation is harder.
Formalized claims in Lean
-
Claim #1: The central claim is that modulating a DeepONet's branch activations by a trunk-generated latent variable $\lambda \sim \mathcal{N}(0, I)$ and training with an ELBO-style loss yields an approximate posterior over the PDE parameters rather than a point estimate. The paper argues that this posterior carries both data uncertainty, through a learned observation-noise variance $\sigma_y^2$, and model u
/-- @claim 1 The central claim is that modulating a DeepONet's branch activations by a trunk-generated latent variable $\lambda \sim \mathcal{N}(0, I)$ and training with an ELBO-style loss yields an approximate posterior over the PDE parameters rather than a point estimate. The paper argues that this posterior carries both data uncertainty, through a learned observation-noise variance $\sigma_y^2$, and model u -/ def central_claim : Prop :=
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes DeepBayONet, a DeepONet-based architecture with physics-informed loss terms for forward PDE solving and parameter estimation. The central claim is that Bayesian training is implemented through variational inference, yielding posterior distributions over PDE parameters and quantifying both aleatoric and epistemic uncertainties. The authors derive an ELBO in Section 2.1, describe a trunk network that maps samples from a standard normal to an approximate posterior, and present experiments on regression, 1D function approximation, the 1D heat equation, a 2D reaction-diffusion equation, and a 3D eigenvalue problem. Comparisons against SNN, BNN, MCDO, DENN, and B-PINN variants are reported, with DeepBayONet claiming higher 95% CI coverage and accurate parameter recovery.
Significance. If the claims were supported, the method would be a useful contribution: it combines operator learning with physics-informed losses, provides a unified architecture for forward and inverse problems, and ships reproducible code. However, as written, the central Bayesian claim is not supported by the paper's own equations. The optimized loss is not the derived ELBO, and the reported parameter distributions are not posterior distributions conditioned on the data. The paper is therefore not yet a reliable basis for Bayesian UQ in PDE surrogate modeling.
major comments (3)
- [Sec. 2.4, Eqs. (17)-(18e), footnote 1] The training objective implemented in the paper is not the ELBO derived in Eq. (12). The ELBO contains the KL divergence term of Eq. (13), but the actual loss in Eq. (17) replaces this term with L_STD = log sigma_y^2 (Eq. 18e), and footnote 1 states this was chosen empirically as simpler than evaluating the exact KL divergence. Consequently, the optimized objective is not a variational lower bound, and the q(lambda) represented by the trunk network is not trained as a variational posterior. This directly undermines the abstract's claim that 'Bayesian training is implemented through variational inference.'
- [Sec. 2.3, Eqs. (16a)-(16b)] The 'posterior' histograms in Figures 4, 7, 11, and 15 are not samples from p(lambda | D). Equation (16a) draws lambda from a fixed standard normal independent of the data, and Eq. (16b) computes a deterministic transformation tau(lambda) through the trunk network. The resulting distribution over tau(lambda) is a pushforward of the fixed prior N(0, I) through a learned map; it is not conditioned on observations through Bayes' rule. Nothing in the loss anchors this distribution to the posterior, so the reported uncertainty intervals are not Bayesian credible intervals.
- [Sec. 3.6 and Fig. 15] The 3D eigenvalue experiment contains an internal inconsistency: the text states the ground-truth eigenvalue is lambda_11 = 4.49341^2 ≈ 20.19 (dipole mode), while Figure 15a is described as showing convergence to 'lambda ≈ 4.493.' If 4.493 is the first zero of the spherical Bessel function, it would correspond to a different quantity. As written, the reader cannot tell which eigenvalue was actually estimated, and this undermines the claim of accurate parameter recovery in the 3D example.
minor comments (5)
- [Sec. 3.2, Fig. 2 caption] The caption refers to 'ODD dataset' where it should say 'OOD dataset.'
- [Sec. 3.4, Fig. 5 caption] The caption says 'eq. (21) eq. (21)' for the 1D heat equation, which is a duplicated reference.
- [Sec. 3.6, Fig. 15] Figure 15 labels the panels as '(a) 3D Axis Plot' and '(b) 3D Histogram,' but the text refers to Figure 15a as the histogram; the panel labels and the textual references should be aligned.
- [Sec. 3.2, Table 2] The BNN baseline reports a total 95% CI coverage of 13.30%, which is unusually low for a Bayesian method; the authors should clarify the baseline configuration and diagnostic checks to ensure the comparison is not affected by a poorly tuned prior or variational posterior.
- [Sec. 2.3, paragraph on data uncertainty] The paper claims sigma_y^2 can be learned as an input-dependent function, but the loss terms in Eqs. (18d) and (18e) use a single scalar sigma_y^2. Either the implementation should be described consistently or the claim of input-dependent aleatoric uncertainty should be moderated.
Circularity Check
The Bayesian posterior claim is not derived: the KL term is replaced by a heuristic log-variance loss, so the reported parameter distributions are pushforwards of a fixed Gaussian through the fitted trunk network.
-
self definitional
[Section 2.4, Eq. (17)-(18e) and footnote 1]
"The loss component in eq. (18e) was found empirically to be simpler than evaluating the exact KL-divergence. The intuition behind it is that the log of the variance of the predicted solution needs to be of moderate magnitude."
Eq. (12) derives an ELBO whose optimization includes the analytic KL divergence D_KL(q(λ)||p0(λ)). Eq. (17) defines the actual training loss with L_STD = |log σ_y^2| in place of that KL term. The optimized objective is therefore not the ELBO, so q(λ) is not trained to approximate p(λ|D). The claimed variational inference reduces, by the paper's own equations and footnote, to fitting a heuristic log-variance penalty; the 'posterior' is whatever the trunk network produces, not the result of a Bayesian update.
-
fitted input called prediction
[Section 2.3, Eqs. (16a)-(16b); Section 3.5, Table 3]
"λ ∼ N(0,I), ˜h = τ(λ)⊙h. ... The posterior distribution q(λ) over the latent parameters can be empirically estimated using the trunk network to understand the model’s confidence in the predicted parameter."
Because λ is always sampled from the fixed distribution N(0,I), the 'posterior' histogram is, by construction, the pushforward of the fixed Gaussian through the trained trunk network τ(λ); no likelihood update or KL regularization enters. The reported parameter estimates (e.g., k = 0.999 ± 5.4e-4 in Table 3) are the moments of this fitted modulation, so the uncertainty and parameter-recovery claims are readings of the fitted network rather than predictions from an inferred posterior.
full rationale
The forward PDE solving and regression experiments are self-contained and benchmarked independently, so the paper is not wholly circular. However, the central Bayesian UQ claim is not derived: Eq. (12) derives an ELBO with a KL term, but Eq. (17)/(18e) replaces it with L_STD = |log σ_y^2|, as footnote 1 admits. Since the optimized objective is not the ELBO, the trunk-network output τ(λ) for λ∼N(0,I) is not an approximate posterior p(λ|D); the histograms and coverage intervals are properties of the fitted stochastic perturbation. This makes the 'posterior distributions' and the reported parameter uncertainties reduce, by the paper's own equations, to the learned modulation rather than to Bayesian inference. The 3D eigenvalue section also contains an internal inconsistency (the text gives the eigenvalue λ_11 = 20.19, while Fig. 15a is said to converge to λ≈4.493, which is the Bessel zero, not the eigenvalue), further undermining the parameter-estimation claim, though this is a correctness issue rather than circularity. No load-bearing self-citation chain was found; the cited B-PINN and DeepONet works are external.
Assumptions & free parameters
free parameters (3)
- Loss weights w_Interior, w_IC, w_BC, w_STD, w_Data =
heat: wIC=3, wData=6, wBC=1, wInterior=1, wSTD=1; reaction-diffusion: wInterior=20000, wData=60000, wBC=100, wSTD=20…
- Observation noise variance σ_y^2 =
learned, values not reported
- Residual noise variance σ_R^2 =
constant or learned, not specified in experiments
assumptions (5)
- domain assumption The joint likelihood factorizes as a product over data points of the data likelihood and the PDE residual likelihood (Eq. 4).
- domain assumption The PDE in Eq. 1 is well posed and y(t,x;λ) exists uniquely with the stated initial and boundary conditions.
- standard math Both p0(λ) and q(λ) are Gaussian with σ0^2 = 1, enabling the closed-form KL in Eq. 13.
- standard math DeepONet can approximate the solution operator M_θ(t,x,λ) as a product of branch and trunk outputs (Eq. 14).
- ad hoc to paper L_STD = log σ_y^2 is an acceptable replacement for the exact KL divergence (Eq. 18e, footnote 1).
Cite this review
Pith. "Pith review of Deep Operator Networks for Bayesian Parameter Estimation in PDEs." pith.science (2026). https://pith.science/paper/S3W3NOAT
@misc{pith2026250110684,
author = {Pith},
title = {Pith review of: Deep Operator Networks for Bayesian Parameter Estimation in PDEs},
year = {2026},
howpublished = {\url{https://pith.science/paper/S3W3NOAT}},
note = {Machine review of arXiv:2501.10684}
}
read the original abstract
We present a novel framework combining Deep Operator Networks (DeepONets) with Physics-Informed Neural Networks (PINNs) to solve partial differential equations (PDEs) and estimate their unknown parameters. By integrating data-driven learning with physical constraints, our method achieves robust and accurate solutions across diverse scenarios. Bayesian training is implemented through variational inference, allowing for comprehensive uncertainty quantification for both aleatoric and epistemic uncertainties. This ensures reliable predictions and parameter estimates even in noisy conditions or when some of the physical equations governing the problem are missing. The framework demonstrates its efficacy in solving forward and inverse problems, including the 1D unsteady heat equation and 2D reaction-diffusion equations, as well as regression tasks with sparse, noisy observations. This approach provides a computationally efficient and generalizable method for addressing uncertainty quantification in PDE surrogate modeling.
Figures
Figures from the paper (12 more)
Reference graph
Works this paper leans on
-
[1]
S. Roberts, A. A. Popov, A. Sarshar, A. Sandu, A fast time-stepping strategy for dynamical systems equipped with a surrogate model, SIAM Journal on Scientific Computing 44 (3) (2022) A1405–A1427. doi:10.1137/20M1386281
-
[2]
A. Sarshar, P. Tranquilli, B. Pickering, A. McCall, C. J. Roy, A. Sandu, A numerical investigation of matrix-free implicit time-stepping meth- ods for large CFD simulations, Computers & Fluids 159 (2017) 53–63. doi:10.1016/j.compfluid.2017.09.014
-
[3]
Y. Chen, Y. Shi, B. Zhang, Optimal Control Via Neural Networks: A Convex Approach (2019). doi:10.48550/arxiv.1805.11835
-
[4]
Solving PDE-constrained Control Problems Using Operator Learning
R. Hwang, J. Y. Lee, J. Y. Shin, H. J. Hwang, Solving PDE-constrained Control Problems using Operator Learning (2021). doi:10.48550/arxiv.2111.04941. 25
work page Pith review arXiv doi:10.48550/arxiv.2111.04941 2021
-
[5]
S. Wang, M. A. Bhouri, P. Perdikaris, Fast PDE-constrained optimiza- tion via self-supervised operator learning, CoRR abs/2110.13297 (2021). doi:10.48550/arxiv.2110.13297
-
[6]
A. Bhattacharjee, A. A. Popov, A. Sarshar, A. Sandu, Improving adam through an implicit-explicit (imex) time-stepping approach, Jour- nal of Machine Learning for Modeling and Computing 5 (3) (2024). doi:10.1615/jmachlearnmodelcomput.2024053508
-
[7]
M.Raissi, P.Perdikaris, G.E.Karniadakis, Physics-informedneuralnet- works: A deep learning framework for solving forward and inverse prob- lems involving nonlinear partial differential equations, Journal of Com- putational Physics 378 (2019) 686–707. doi:10.1016/j.jcp.2018.10.045
-
[8]
L. Lu, X. Meng, Z. Mao, G. Karniadakis, DeepXDE: A deep learning library for solving differential equations, SIAM Review 63 (1) (2021) 208–228. doi:10.1137/19M1274067
Show all 40 references
-
[9]
doi:10.1007/s10915-022-01939-z
S.Cuomo, V.S.DiCola, F.Giampaolo, G.Rozza, M.Raissi, F.Piccialli, Scientific machine learning through physics–informed neural networks: Where we are and what’s next, Journal of Scientific Computing 92 (3) (2022) 88. doi:10.1007/s10915-022-01939-z
2022 doi
-
[10]
Z. Mao, A. D. Jagtap, G. E. Karniadakis, Physics-informed neural net- works for high-speed flows, Computer Methods in Applied Mechanics and Engineering 360 (2020) 112789. doi:10.1016/j.cma.2019.112789
2020
-
[11]
doi:10.1137/18m1229845
G.Pang, L.Lu, G.E.Karniadakis, fPINNs: FractionalPhysics-Informed Neural Networks, SIAM Journal on Scientific Computing 41 (4) (2019) A2603–A2626. doi:10.1137/18m1229845
2019 doi
-
[12]
Y. Chen, L. Lu, G. E. Karniadakis, L. Dal Negro, Physics-informed neural networks for inverse problems in nano-optics and metamaterials, Optics Express 28 (8) (2020) 11618. doi:10.1364/oe.384875
2020 doi
-
[13]
Q. He, D. Barajas-Solano, G. Tartakovsky, A. M. Tartakovsky, Physics- informed neural networks for multiphysics data assimilation with appli- cation to subsurface transport, Advances in Water Resources 141 (2020) 103610. doi:10.1016/j.advwatres.2020.103610. 26
2020
- [14]
- [15]
- [16]
-
[17]
Zhang, L
D. Zhang, L. Lu, L. Guo, G. E. Karniadakis, Quantifying total uncer- tainty in physics-informed neural networks for solving forward and in- verse stochastic problems, Journal of Computational Physics 397 (2019) 108850. doi:10.1016/j.jcp.2019.07.048
2019 doi
-
[18]
Paszke, S
A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A.Desmaison, L.Antiga, A.Lerer, AutomaticdifferentiationinPyTorch (Oct. 2017). URLhttps://openreview.net/forum?id=BJJsrmfCZ
2017
-
[19]
Bradbury, R
J. Bradbury, R. Frostig, P. Hawkins, M. J. Johnson, C. Leary, D. Maclaurin, S. Wanderman-Milne, JAX: composable transformations of Python+NumPy programs, version 0.2.26 (2018). URLhttp://github.com/google/jax
2018
-
[20]
Abadi, et al., Tensorflow: Large-scale machine learning on heteroge- neous systems, software available from tensorflow.org (2015)
M. Abadi, et al., Tensorflow: Large-scale machine learning on heteroge- neous systems, software available from tensorflow.org (2015). URLhttps://www.tensorflow.org/
2015
-
[21]
A. G. Baydin, B. A. Pearlmutter, A. A. Radul, J. M. Siskind, Auto- matic differentiation in machine learning: A survey, Journal of Machine Learning Research 18 (153) (2018) 1–43. doi:10.5555/3327757.3327758
2018
-
[22]
C. C. Margossian, A review of automatic differentiation and its efficient implementation, WIREs Data Mining and Knowledge Discovery 9 (4) (2019) e1305. doi:10.1002/widm.1305
2019 doi
-
[23]
F. A. C. Viana, A. K. Subramaniyan, A survey of bayesian calibration and physics-informed neural networks in scientific modeling, Archives 27 of Computational Methods in Engineering 28 (5) (2021) 3801–3830. doi:10.1007/s11831-021-09539-0
2021 doi
-
[24]
J. P. Molnar, S. J. Grauer, Flow field tomography with uncertainty quantification using a Bayesian physics-informed neural network, Mea- surement Science and Technology 33 (6) (2022) 065305, publisher: IOP Publishing. doi:10.1088/1361-6501/ac5437
2022 doi
-
[25]
Y. Li, Y. Wang, L. Yan, Surrogate modeling for Bayesian inverse prob- lems based on physics-informed neural networks, Journal of Computa- tional Physics 475 (2023) 111841. doi:10.1016/j.jcp.2022.111841
2023
-
[26]
Wang, D.-Y
H. Wang, D.-Y. Yeung, A survey on bayesian deep learning, ACM Com- puting Surveys 53 (5) (2020) 108:1–108:37. doi:10.1145/3409383
2020 doi
-
[27]
Bonneville, C
C. Bonneville, C. Earls, Bayesian deep learning for partial dif- ferential equation parameter discovery with sparse and noisy data, Journal of Computational Physics: X 16 (2022) 100115. doi:10.1016/j.jcpx.2022.100115
2022
- [28]
-
[29]
L. Yang, X. Meng, G. E. Karniadakis, B-PINNs: Bayesian physics- informed neural networks for forward and inverse PDE problems with noisy data, Journal of Computational Physics 425 (Jan. 2021). doi:10.1016/j.jcp.2020.109913
2021
-
[30]
L. Lu, P. Jin, G. E. Karniadakis, DeepONet: Learning nonlinear op- erators for identifying differential equations based on the universal ap- proximation theorem of operators (Oct. 2019). doi:10.1038/s42256-021- 00302-5
2019 doi
-
[31]
G. Lin, C. Moya, Z. Zhang, B-deeponet: An enhanced bayesian deeponet for solving noisy parametric pdes using accelerated replica exchange sgld, Journal of Computational Physics 473 (2023) 111713. doi:https://doi.org/10.1016/j.jcp.2022.111713. URLhttps://www.sciencedirect.com/s...
2023
- [32]
-
[33]
T. Chen, H. Chen, Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems, IEEE Transactions on Neural Networks 6 (4) (1995) 911–917. doi:10.1109/72.392253
1995 doi
-
[34]
Kendall, Y
A. Kendall, Y. Gal, What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?, in: Advances in Neural Infor- mation Processing Systems, Vol. 30, Curran Associates, Inc., 2017. doi:10.5555/3295222.3295309
2017
-
[35]
Srivastava, G
N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, R. Salakhutdinov, Dropout: A simple way to prevent neural networks from overfitting, Journal of Machine Learning Research 15 (1) (2014) 1929–1958
2014
-
[36]
Xiang, W
Z. Xiang, W. Peng, X. Liu, W. Yao, Self-adaptive loss balanced physics-informed neural networks, Neurocomputing 496 (2022) 11–34. doi:10.1016/j.neucom.2022.05.015
2022 doi
-
[37]
L. D. McClenny, U. M. Braga-Neto, Self-adaptive physics-informed neu- ral networks, Journal of Computational Physics 474 (2023) 111722. doi:10.1016/j.jcp.2022.111722
2023
-
[38]
URLhttps://is.gd/ZvwCFr
DeepFindr, How to handle uncertainty in deep learning, [Online; ac- cessed 2024-12-26] (2024). URLhttps://is.gd/ZvwCFr
2024
-
[39]
Abdar, F
M. Abdar, F. Pourpanah, S. Hussain, D. Rezazadegan, L. Liu, M. Ghavamzadeh, P. Fieguth, X. Cao, A. Khosravi, U. R. Acharya, et al., A review of uncertainty quantification in deep learning: Tech- niques, applications and challenges, Information fusion 76 (2021) 243–
2021
-
[297]
doi:10.1016/j.inffus.2021.05.008. 29
2021 doi
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.