Robust optimal design of large-scale Bayesian nonlinear inverse problems
Pith reviewed 2026-05-23 20:30 UTC · model grok-4.3
The pith
A framework finds designs for nonlinear Bayesian PDE inverse problems that stay optimal despite uncertainties in the model or prior.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central claim is that a worst-case scenario formulation, paired with efficient expected-information-gain approximations and eigenvalue-sensitivity gradient evaluations, yields a scalable method that solves the robust optimal design problem for large-scale PDE-constrained Bayesian inversions and returns designs stable to variations in the inverse-problem elements, as shown for elliptic-PDE sensor placement.
What carries the argument
Eigenvalue sensitivity techniques that supply analytical forms and fast evaluation of the gradient of the expected information gain utility with respect to the uncertain inverse-problem elements inside a worst-case max-min formulation.
If this is right
- The method scales to infinite-dimensional problems governed by PDEs without requiring full discretization of the design space at every step.
- Analytical gradients obtained from eigenvalue sensitivities allow gradient-based optimization of the otherwise combinatorial max-min problem.
- The probabilistic optimization paradigm converts the robust design task into a tractable stochastic program whose solution yields a design stable to the considered uncertainties.
- The framework is demonstrated to produce effective robust sensor placements for an elliptic PDE inverse problem.
Where Pith is reading between the lines
- The same sensitivity machinery could be reused to quantify how much each uncertain element contributes to design degradation, guiding which uncertainties deserve the most modeling effort.
- If the EIG approximations extend without retuning, the framework could be applied directly to time-dependent or highly nonlinear PDEs common in fluid or structural inverse problems.
- Embedding the robust optimizer inside existing Bayesian inversion libraries would let practitioners request a design that is already protected against their own prior and model uncertainties.
Load-bearing premise
The EIG approximations and eigenvalue sensitivity calculations remain accurate enough that the worst-case designs stay near-optimal when the actual nonlinear PDE model, prior, or noise model differs from the ones used in the design computation.
What would settle it
A numerical test that perturbs the forward model, prior covariance, or noise level within realistic ranges, recomputes the true expected information gain for both the robust design and a non-robust design, and checks whether the robust design retains higher utility in the worst-case perturbations.
Figures
read the original abstract
We consider robust optimal experimental design (ROED) for nonlinear Bayesian inverse problems governed by partial differential equations (PDEs). An optimal design is one that maximizes some utility quantifying the quality of the solution of an inverse problem. However, the optimal design is dependent on elements of the inverse problem such as the simulation model, the prior, or the measurement error model. ROED aims to produce an optimal design that is aware of the additional uncertainties encoded in the inverse problem and remains optimal even after variations in them. We follow a worst-case scenario approach to develop a new framework for robust optimal design of nonlinear Bayesian inverse problems. The proposed framework a) is scalable and designed for infinite-dimensional Bayesian nonlinear inverse problems constrained by PDEs; b) develops efficient approximations of the utility, namely, the expected information gain; c) employs eigenvalue sensitivity techniques to develop analytical forms and efficient evaluation methods of the gradient of the utility with respect to the uncertainties we wish to be robust against; and d) employs a probabilistic optimization paradigm that properly defines and efficiently solves the resulting combinatorial max-min optimization problem. The effectiveness of the proposed approach is illustrated for optimal sensor placement problem in an inverse problem governed by an elliptic PDE.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes a framework for robust optimal experimental design (ROED) of nonlinear Bayesian inverse problems governed by PDEs. It adopts a worst-case scenario formulation to produce designs robust to uncertainties in the model, prior, or noise model. The framework develops efficient approximations to the expected information gain (EIG) utility, employs eigenvalue sensitivity analysis to obtain analytical gradients of the utility with respect to the uncertain elements, and uses a probabilistic optimization approach to solve the resulting combinatorial max-min problem. Scalability to infinite-dimensional settings is claimed, with effectiveness illustrated via an optimal sensor placement example governed by an elliptic PDE.
Significance. If the EIG approximations and eigenvalue sensitivity techniques remain accurate under worst-case perturbations, the work would provide a computationally tractable route to robust designs in large-scale PDE-constrained inverse problems. The probabilistic treatment of the max-min problem and the use of sensitivity methods for gradient evaluation are strengths that could improve efficiency over naive sampling approaches. The elliptic PDE demonstration shows practical applicability in a common setting, though broader impact depends on validation for strongly nonlinear forward maps.
major comments (2)
- [Abstract and illustration] Abstract and elliptic PDE illustration: the central claim of effectiveness for nonlinear Bayesian inverse problems rests on the EIG approximations and eigenvalue sensitivity producing gradients accurate enough for the max-min optimizer. However, the provided illustration uses an elliptic PDE, for which the parameter-to-observation map is often effectively linear; this does not test whether first-order eigenvalue perturbation analysis misses higher-order effects when worst-case uncertainties push the problem far from the nominal point.
- [Method and numerical results] The worst-case formulation combined with the proposed approximations is load-bearing for the robustness claim, yet no quantitative assessment (e.g., comparison of approximate vs. exact EIG gradients or bias in the resulting designs) is supplied for cases where the forward map exhibits clear nonlinearity.
minor comments (2)
- Notation for the uncertain elements (model, prior, noise) and their parameterization in the max-min problem could be introduced earlier and used consistently to improve readability.
- [Abstract] The abstract states the framework is 'scalable' for infinite-dimensional problems; a brief statement on how the low-rank or Laplace-type EIG approximations scale with mesh size or parameter dimension would strengthen the claim.
Simulated Author's Rebuttal
We thank the referee for the constructive comments and the opportunity to clarify aspects of our work. Below we respond point-by-point to the major comments.
read point-by-point responses
-
Referee: [Abstract and illustration] Abstract and elliptic PDE illustration: the central claim of effectiveness for nonlinear Bayesian inverse problems rests on the EIG approximations and eigenvalue sensitivity producing gradients accurate enough for the max-min optimizer. However, the provided illustration uses an elliptic PDE, for which the parameter-to-observation map is often effectively linear; this does not test whether first-order eigenvalue perturbation analysis misses higher-order effects when worst-case uncertainties push the problem far from the nominal point.
Authors: The elliptic PDE example employs an uncertain conductivity field as the inversion parameter, rendering the parameter-to-observation map nonlinear. Nevertheless, we accept that the chosen regime may not exhibit strong nonlinearity and that first-order eigenvalue perturbation may miss higher-order effects under large worst-case shifts. We will revise the numerical results section to include an explicit discussion of the degree of nonlinearity present in the example, the range of perturbations considered, and the conditions under which the first-order sensitivity remains reliable. revision: partial
-
Referee: [Method and numerical results] The worst-case formulation combined with the proposed approximations is load-bearing for the robustness claim, yet no quantitative assessment (e.g., comparison of approximate vs. exact EIG gradients or bias in the resulting designs) is supplied for cases where the forward map exhibits clear nonlinearity.
Authors: We agree that the manuscript does not supply a direct quantitative comparison of approximate versus exact EIG gradients or resulting design bias for forward maps with clear nonlinearity. The current numerical study is limited to the elliptic PDE setting as a demonstration of scalability. We will add a statement in the conclusions section acknowledging this limitation and identifying the provision of such assessments for strongly nonlinear maps as an important direction for future work. revision: yes
Circularity Check
No circularity; derivation chain self-contained against external benchmarks
full rationale
The abstract and description outline a framework using EIG approximations, eigenvalue sensitivity, and probabilistic optimization for ROED, but contain no equations, fitted parameters renamed as predictions, or self-citations that bear the central load. No self-definitional steps, uniqueness theorems imported from authors, or ansatzes smuggled via citation are present. The claims rest on proposed methods whose validity is to be assessed externally (e.g., via the elliptic PDE illustration), not by reduction to the inputs themselves. This is the normal case of an independent derivation.
Axiom & Free-Parameter Ledger
Reference graph
Works this paper leans on
-
[1]
A. Alexanderian , Optimal experimental design for infinite-dimensional Bayesian inverse problems governed by PDEs: a review , Inverse Problems, 37 (2021), p. 043001
work page 2021
-
[2]
A. Alexanderian, P. J. Gloor, and O. Ghattas , On Bayesian A- and D-optimal experi- ROED OF INFINITE-DIMENSIONAL BAYESIAN NONLINEAR INVERSE PROBLEMS 25 mental designs in infinite dimensions , Bayesian Analysis, 11 (2016), pp. 671 – 695
work page 2016
-
[3]
A. Alexanderian, R. Nicholson, and N. Petra , Optimal design of large-scale nonlinear Bayesian inverse problems under model uncertainty , Inverse Problems, (2024)
work page 2024
-
[4]
A. Alexanderian, N. Petra, G. Stadler, and I. Sunseri , Optimal design of large-scale Bayesian linear inverse problems under reducible model uncertainty: Good to know what you don’t know, SIAM/ASA Journal on Uncertainty Quantification, 9 (2021), pp. 163–184
work page 2021
-
[5]
A. Alexanderian and A. K. Saibaba , Efficient D-optimal design of experiments for infinite- dimensional Bayesian linear inverse problems , SIAM Journal on Scientific Computing, 40 (2018), pp. A2956–A2985
work page 2018
-
[6]
A. C. Atkinson and A. N. Donev , Optimum Experimental Designs , Oxford, 1992
work page 1992
-
[7]
A. Attia , Probabilistic approach to black-box binary optimization with budget constraints: Application to sensor placement , arXiv preprint arXiv:2406.05830, (2024)
-
[8]
A. Attia and E. Constantinescu , Optimal experimental design for inverse problems in the presence of observation correlations , SIAM Journal on Scientific Computing, 44 (2022), pp. A2808–A2842
work page 2022
- [9]
- [10]
-
[11]
W. Bangerth, A framework for the adaptive finite element solution of large-scale inverse problems, SIAM Journal on Scientific Computing, 30 (2008), pp. 2965–2989
work page 2008
-
[12]
A. Bartuska, L. Espath, and R. Tempone, Small-noise approximation for Bayesian optimal experimental design with nuisance uncertainty, Comput. Methods Appl. Mech. Engrg., 399 (2022), p. 115320
work page 2022
-
[13]
S. Biedermann and H. Dette , A note on maximin and Bayesian D-optimal designs in weighted polynomial regression, Mathematical Methods of Statistics, 12 (2003), p. 358–370
work page 2003
-
[14]
T. Bui-Thanh, O. Ghattas, J. Martin, and G. Stadler , A computational framework for infinite-dimensional Bayesian inverse problems part i: The linearized case, with appli- cation to global seismic inversion , SIAM Journal on Scientific Computing, 35 (2013), pp. A2494–A2523
work page 2013
-
[15]
K. Chaloner and I. Verdinelli, Bayesian experimental design: A review , Statistical Science, 10 (1995)
work page 1995
-
[16]
P. Chen, U. Villa, and O. Ghattas , Taylor approximation and variance reduction for pde- constrained optimal control under uncertainty , Journal of Computational Physics, 385 (2019), p. 163–186
work page 2019
-
[17]
A. Chowdhary, Scalable Uncertainty Quantification for Infinite-Dimensional Bayesian In- verse Problems., PhD thesis, Feb. 2025, https://www.lib.ncsu.edu/resolver/1840.20/45106
work page 2025
-
[18]
A. Chowdhary, S. E. Ahmed, and A. Attia , PyOED: An extensible suite for data assimila- tion and model-constrained optimal design of experiments , ACM Trans. Math. Softw., 50 (2024)
work page 2024
-
[19]
A. Chowdhary, S. Tong, G. Stadler, and A. Alexanderian , Sensitivity analysis of the information gain in infinite-dimensional bayesian linear inverse problems , International Journal for Uncertainty Quantification, 14 (2024)
work page 2024
-
[20]
D. R. Cox , Planning of Experiments , Wiley Classics Library, John Wiley & Sons, Nashville, TN, Apr. 1992
work page 1992
-
[21]
Y. Daon and G. Stadler, Mitigating the influence of the boundary on PDE-based covariance operators, 2018
work page 2018
- [22]
- [23]
-
[24]
Fedorov , Optimal experimental design , WIREs Computational Statistics, 2 (2010), pp
V. Fedorov , Optimal experimental design , WIREs Computational Statistics, 2 (2010), pp. 581–589
work page 2010
-
[25]
J. Go and T. Isaac , Robust expected information gain for optimal Bayesian experimental design using ambiguity sets, in Proceedings of the Thirty-Eighth Conference on Uncertainty in Artificial Intelligence, J. Cussens and K. Zhang, eds., vol. 180 of Proceedings of Machine Learning Research, PMLR, 01–05 Aug 2022, pp. 728–737
work page 2022
- [26]
-
[27]
C. R. Harris, K. J. Millman, S. J. van der Walt, R. Gommers, P. Virtanen, D. Courna- peau, E. Wieser, J. Taylor, S. Berg, N. J. Smith, R. Kern, M. Picus, S. Hoyer, M. H. van Kerkwijk, M. Brett, A. Haldane, J. F. del R ´ıo, M. Wiebe, P. Pe- terson, P. G ´erard-Marchant, K. Sheppard, T. Reddy, W. Weckesser, H. Ab- basi, C. Gohlke, and T. E. Oliphant , Array...
-
[28]
X. Huan and Y. M. Marzouk , Simulation-based optimal bayesian experimental design for nonlinear systems, Journal of Computational Physics, 232 (2013), p. 288–317, https://doi. org/10.1016/j.jcp.2012.08.013
-
[29]
J. Kaipio and V. Kolehmainen , Approximate marginalization over modeling errors and un- certainties in inverse problems , Bayesian Theory and Applications, (2013), pp. 644–672
work page 2013
-
[30]
V. Kolehmainen, T. Tarvainen, S. R. Arridge, and J. P. Kaipio, Marginalization of unin- teresting distributed parameters in inverse problems-application to diffuse optical tomog- raphy, International Journal for Uncertainty Quantification, 1 (2011)
work page 2011
- [31]
-
[32]
S. Kullback and R. A. Leibler, On information and sufficiency, The Annals of Mathematical Statistics, 22 (1951), pp. 79–86
work page 1951
-
[33]
L. C. Lau and H. Zhou , A local search framework for experimental design , (2020)
work page 2020
-
[34]
P. D. Lax, Linear algebra and its applications, Pure and Applied Mathematics: A Wiley Series of Texts, Monographs and Tracts, Wiley-Blackwell, Chichester, England, 2 ed., Aug. 2007
work page 2007
-
[35]
E. Levitin and B. Polyak, Constrained minimization methods, USSR Computational Math- ematics and Mathematical Physics, 6 (1966), pp. 1–50
work page 1966
-
[36]
D. C. Liu and J. Nocedal, On the limited memory BFGS method for large scale optimization , Mathematical Programming, 45 (1989), p. 503–528
work page 1989
-
[37]
A. Logg, K.-A. Mardal, and G. Wells , Automated Solution of Differential Equations by the Finite Element Method: The FEniCS Book , vol. 84 of Lecture Notes in Computational Science and Engineering, Springer Berlin Heidelberg, Berlin, Heidelberg, 2012
work page 2012
-
[38]
M. Mozumder, T. Tarvainen, S. Arridge, J. P. Kaipio, C. D’Andrea, and V. Kolehmainen, Approximate marginalization of absorption and scattering in fluores- cence diffuse optical tomography, Inverse Problems & Imaging, 10 (2016), p. 227
work page 2016
-
[39]
T. O’Leary-Roseberry, U. Villa, P. Chen, and O. Ghattas , Derivative-informed pro- jected neural networks for high-dimensional parametric maps governed by PDEs , Com- puter Methods in Applied Mechanics and Engineering, 388 (2022), p. 114199
work page 2022
-
[40]
R.-E. Plessix, A review of the adjoint-state method for computing the gradient of a functional with geophysical applications, Geophysical Journal International, 167 (2006), pp. 495–503
work page 2006
-
[41]
L. Pronzato and E. Walter , Robust experiment design via maximin optimization , Mathe- matical Biosciences, 89 (1988), pp. 161–176
work page 1988
-
[42]
T. Rainforth, A. Foster, D. R. Ivanova, and F. B. Smith, Modern Bayesian Experimental Design, Statistical Science, 39 (2024), pp. 100 – 114
work page 2024
-
[43]
C. R. Rojas, J. S. Welsh, G. C. Goodwin, and A. Feuer, Robust optimal experiment design for system identification, Automatica, 43 (2007), pp. 993–1008
work page 2007
-
[44]
A. M. Stuart , Inverse problems: A Bayesian perspective , Acta Numerica, 19 (2010), p. 451–559
work page 2010
-
[45]
I. Sunseri, A. Alexanderian, J. Hart, and B. v. B. Waanders, Hyper-differential sensitivity analysis for nonlinear Bayesian inverse problems , International Journal for Uncertainty Quantification, 14 (2024), p. 1–20
work page 2024
- [46]
- [47]
-
[48]
D. Uci ´nski, Optimal measurement methods for distributed parameter system identification , Systems and Control Series, CRC Press, Boca Raton, FL, 2005
work page 2005
- [49]
-
[50]
P. Virtanen, R. Gommers, T. E. Oliphant, M. Haberland, T. Reddy, D. Cournapeau, E. Burovski, P. Peterson, W. Weckesser, J. Bright, S. J. van der Walt, M. Brett, J. Wilson, K. J. Millman, N. Mayorov, A. R. J. Nelson, E. Jones, R. Kern, E. Lar- ROED OF INFINITE-DIMENSIONAL BAYESIAN NONLINEAR INVERSE PROBLEMS 27 son, C. J. Carey, ˙I. Polat, Y. Feng, E. W. Mo...
-
[51]
A. Wald, Statistical decision functions which minimize the maximum risk , Annals of Mathe- matics, 46 (1945), p. 265–280
work page 1945
-
[52]
K. Wu, P. Chen, and O. Ghattas , A fast and scalable computational framework for large- scale high-dimensional bayesian optimal experimental design , SIAM/ASA Journal on Un- certainty Quantification, 11 (2023), pp. 235–261. Appendix A. Variational Tools. The gradient and the Hessian of the cost functional Φ (3.10a) are essential components of our proposed...
work page 2023
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.