pith. sign in

arxiv: 1907.04022 · v1 · pith:Y2J3DVRBnew · submitted 2019-07-09 · 🧮 math.NA · cs.NA

PDE/PDF-informed adaptive sampling for efficient non-intrusive surrogate modelling

Yous van Halder , Benjamin Sanderse , Barry Koren This is my paper

Pith reviewed 2026-05-25 00:22 UTC · model grok-4.3

classification 🧮 math.NA cs.NA
keywords adaptive samplingsurrogate modellingPDE residualempirical interpolationuncertain parametersnon-intrusive methodsneural network solvers
0
0 comments X

The pith

A PDE residual combined with the parameter probability density guides adaptive sampling for non-intrusive surrogate models of PDEs.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces a refinement measure for adaptive sampling when building non-intrusive surrogate models of PDEs that depend on uncertain parameters. The measure is computed inside an empirical interpolation procedure and combines the PDE residual, evaluated term by term for each partial derivative, with the probability density function of the uncertain parameters. It further restricts attention to those parts of the solution that affect the quantity of interest. The approach applies to arbitrary parameter domains, including non-hypercube shapes, and to any discretization method, including neural-network PDE solvers. Readers would care because the method promises to reduce the number of expensive PDE solves needed to reach a target surrogate accuracy.

Core claim

The central claim is that a refinement measure based on the PDE residual and the PDF of uncertain parameters, computed inside an empirical interpolation procedure and excluding solution components irrelevant to the quantity of interest, produces efficient non-intrusive surrogates even when the parameter space is non-hypercube and when the underlying PDE is discretized by neural-network solvers.

What carries the argument

The PDE/PDF-informed refinement measure inside the empirical interpolation procedure, which weights local residual contributions by parameter probability and restricts them to quantity-of-interest relevant solution parts.

If this is right

  • Surrogate construction is possible for parameter domains that are not hypercubes.
  • The method works with any discretization, including neural-network PDE solvers.
  • Only solution components that affect the quantity of interest enter the refinement decisions.
  • Fewer PDE evaluations are needed to reach a target surrogate accuracy.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same residual-plus-density idea could be tested inside other non-intrusive surrogate constructions such as polynomial chaos expansions.
  • The approach may reduce computational cost in downstream tasks such as uncertainty propagation or optimization under uncertainty.
  • High-dimensional parameter spaces remain an open test case for the method.

Load-bearing premise

The PDE residual, when computed separately for each partial-derivative term, supplies a reliable indicator for where to place new sample points inside the empirical interpolation procedure, and this indicator remains valid for neural-network discretizations.

What would settle it

A test problem in which uniform sampling reaches a prescribed accuracy in the quantity of interest with fewer PDE solves than the proposed adaptive measure would falsify the efficiency claim.

Figures

Figures reproduced from arXiv: 1907.04022 by Barry Koren, Benjamin Sanderse, Yous van Halder.

Figure 1
Figure 1. Figure 1: Schematic representation of the methodology. between general empirical interpolation and NIPPAS is the way in which the residual is constructed. In empirical interpolation the residual is based on the entire spatial/temporal surrogate v˜, rather than focussing on the QoI u˜. Consequently, the residual in empirical interpolation is defined as (3.22) REI(z) = nl ∑ l=1 gl(z,X) ◦Ll(ev(z)) , which has the advan… view at source ↗
Figure 2
Figure 2. Figure 2: Example solutions for different Reynolds number. The solution are computed on a computational grid with ∆x = 10−3 . In order for the discretisation to produce stable results, the cell Reynolds number Re∆x = Re∆x should satisfy Re∆x < 2, which is satisfied by picking ∆x sufficiently small, which is ∆x = 10−3 in our case (NPDE −1000). 13 [PITH_FULL_IMAGE:figures/full_fig_p013_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: The results for the steady-state advection diffusion equation for the two refinement measures R and R ∗ and three different Reynolds functions in the random space. The solutions are computed using ∆x = 10−3 . The error in the surrogate converges exponentially fast to machine precision for both refinement measures without any significant difference. Samples are placed at similar locations for both refinemen… view at source ↗
Figure 4
Figure 4. Figure 4: Convergence comparison for NIPPAS method and conventional empirical interpolation for the steady-state advection diffusion equation. The NIPPAS enhances the surrogate by focusing on locations that are relevant for the QoI. This leads to faster conver￾gence for the NIPPAS when compared to conventional empirical interpolation. However, in case more solution values from the solution vector v would be incorpor… view at source ↗
Figure 5
Figure 5. Figure 5: Reference solution for the advection-diffusion equation for the quantity u(z1,z2) = vNx (1) based on 5000 samples. The choice of the time discretisation method affects the accuracy of the black-box solver, and changes the shape of the 16 [PITH_FULL_IMAGE:figures/full_fig_p016_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Error (6.2) comparison between the Smolyak sparse grid surrogate and the NIPPAS surrogate for different time discreti￾sation methods with ∆t = 10−5 and Nx = 256. The errors are plotted with respect to both the discrete solution of (6.10) and the exact solution of (6.15). The error with respect to the discrete solution converges to zero, with a convergence rate that is significantly faster than the Smolyak … view at source ↗
Figure 7
Figure 7. Figure 7: Convergence in advection-diffusion surrogate, mean and variance for the two refinement measures (3.16) and (3.20). The error in the surrogate is computed with (6.2) with 5000 Monte-Carlo samples. The errors in the mean and variance are calculated with (6.1) [PITH_FULL_IMAGE:figures/full_fig_p018_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Convergence of surrogate for three different non-hypercube domains. The results show an exponential convergence behaviour for all three different geometries, but the convergence rates slightly differ, which is likely caused by the choice of basis, which is suboptimal for all domains. Furthermore, 19 [PITH_FULL_IMAGE:figures/full_fig_p019_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Dirichlet PDF on the scaled unit simplex for (α1,α2,α3) = (5,2,2). NIPPAS method in combination with a more recently developed numerical method. Instead of using a Riemann solver, a neural network is used to solve the SWEs [20]. An advantage of using neural networks for solving PDEs is that the solution is given in terms of a functional form, from which derivatives can be directly computed analytically. Th… view at source ↗
Figure 10
Figure 10. Figure 10: (left) Surrogate model based on 300 samples for refinement measure (3.16). (right) Sample locations for refinement measures (3.16) and (3.20), respectively. The surrogate and sample locations for both refinement measures (3.16) and (3.20) are shown in figure 10. The sample locations show clustering at the boundaries, to produce a stable interpolant. As mentioned before, if the surrogate tends to become un… view at source ↗
Figure 11
Figure 11. Figure 11: Results for the dambreak problem with random inputs. The convergence plot shows a comparison of the two different refinement measure (3.16) and (3.20). The error in the surrogate is computed with (6.2) with 5000 Monte-Carlo samples. The errors in the mean and variance are calculated with (6.1). The results show indeed faster convergence in statistical quantities when accounting for the PDF in the refineme… view at source ↗
read the original abstract

A novel refinement measure for non-intrusive surrogate modelling of partial differential equations (PDEs) with uncertain parameters is proposed. Our approach uses an empirical interpolation procedure, where the proposed refinement measure is based on a PDE residual and probability density function of the uncertain parameters, and excludes parts of the PDE solution that are not used to compute the quantity of interest. The PDE residual used in the refinement measure is computed by using all the partial derivatives that enter the PDE separately. The proposed refinement measure is suited for efficient parametric surrogate construction when the underlying PDE is known, even when the parameter space is non-hypercube, and has no restrictions on the type of the discretisation method. Therefore, we are not restricted to conventional discretisation techniques, e.g., finite elements and finite volumes, and the proposed method is shown to be effective when used in combination with recently introduced neural network PDE solvers. We present several numerical examples with increasing complexity that demonstrate accuracy, efficiency and generality of the method.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The manuscript proposes a novel refinement measure for non-intrusive surrogate modelling of PDEs with uncertain parameters. The approach employs an empirical interpolation procedure in which the refinement indicator is constructed from a term-by-term PDE residual and the given parameter PDF; solution components irrelevant to the quantity of interest are excluded. The method is asserted to impose no restrictions on the underlying discretisation (including neural-network solvers) and to remain valid on non-hypercube parameter domains. Several numerical examples of increasing complexity are presented to illustrate accuracy and efficiency.

Significance. If the residual-based indicator can be shown to produce reliable adaptive sampling without introducing bias or requiring problem-specific calibration, the work would supply a practical, discretisation-agnostic tool for efficient parametric surrogate construction, especially when combined with emerging neural-network PDE solvers. The explicit incorporation of the parameter PDF and the exclusion of QoI-irrelevant solution regions are potentially useful features.

major comments (2)
  1. [Abstract] Abstract and numerical-examples section: the central claim that the term-by-term PDE residual (combined with the parameter PDF) supplies a reliable, discretisation-agnostic refinement indicator is supported solely by numerical demonstrations; no a-priori error analysis, monotonicity argument, or counter-example study is supplied to confirm that this indicator remains monotone with the true surrogate error when the solver is a neural network or when only a subset of the solution enters the QoI. This assumption is load-bearing for the generality and efficiency assertions.
  2. [Abstract] Abstract: the statement that the method 'has no restrictions on the type of the discretisation method' is not accompanied by any analysis of how the residual is evaluated inside a neural-network solver (where derivatives are obtained via automatic differentiation) or by any comparison against conventional residual estimators on the same test problems.
minor comments (1)
  1. The precise algorithmic steps for excluding QoI-irrelevant solution components inside the empirical-interpolation loop are not described; a short pseudocode block or explicit formula would improve reproducibility.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the careful reading and constructive comments on the manuscript. We respond to each major comment below.

read point-by-point responses

  1. Referee: [Abstract] Abstract and numerical-examples section: the central claim that the term-by-term PDE residual (combined with the parameter PDF) supplies a reliable, discretisation-agnostic refinement indicator is supported solely by numerical demonstrations; no a-priori error analysis, monotonicity argument, or counter-example study is supplied to confirm that this indicator remains monotone with the true surrogate error when the solver is a neural network or when only a subset of the solution enters the QoI. This assumption is load-bearing for the generality and efficiency assertions.

    Authors: We agree that the manuscript supports the refinement indicator solely through numerical demonstrations across examples of increasing complexity, including neural-network discretizations and cases where only a subset of the solution contributes to the QoI. No a-priori error analysis, monotonicity argument, or counter-example study is present. We will revise the abstract and add a limitations paragraph in the conclusions to state explicitly that the indicator is a heuristic validated empirically on the presented test problems, without a general guarantee of monotonicity with the true surrogate error. revision: yes

  2. Referee: [Abstract] Abstract: the statement that the method 'has no restrictions on the type of the discretisation method' is not accompanied by any analysis of how the residual is evaluated inside a neural-network solver (where derivatives are obtained via automatic differentiation) or by any comparison against conventional residual estimators on the same test problems.

    Authors: The numerical examples apply the method to neural-network solvers by computing the term-by-term residual via automatic differentiation of the network outputs. The manuscript contains no dedicated analysis of this evaluation process or side-by-side comparisons with conventional residual estimators. We will insert a short explanatory paragraph in the methodology section describing the automatic-differentiation approach for neural-network residuals and will note the absence of comparative benchmarks as an item for future investigation. revision: yes

Circularity Check

0 steps flagged

No significant circularity detected

full rationale

The paper constructs its refinement measure directly from the external PDE residual (computed term-by-term) and the supplied parameter PDF, then uses this measure inside an empirical interpolation loop to select new samples. No equation or procedure defines the measure in terms of quantities already fitted from the surrogate model itself, nor does any central claim reduce to a self-citation chain or a fitted input renamed as a prediction. The derivation therefore remains self-contained against external benchmarks (the known PDE and PDF) and receives the default non-circularity finding.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review yields no identifiable free parameters, axioms, or invented entities; the method is described as building on standard empirical interpolation and PDE residual concepts without introducing new postulated objects.

pith-pipeline@v0.9.0 · 5702 in / 1262 out tokens · 26405 ms · 2026-05-25T00:22:58.233469+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

39 extracted references · 39 canonical work pages · 1 internal anchor

  1. [1]

    H. Xiao, J. L. Wu, J. X. Wang, R. Sun, C. J. Roy, Quantifying and reducing model-form uncertainties in Reynolds-averaged Navier-Stokes simulations: a data-driven, physics-informed Bayesian approach, Journal of Computational Physics 324 (2016) 115–136

    work page 2016

  2. [2]

    W. N. Edeling, R. P. Dwight, P. Cinnella, Simplex-stochastic collocation method with improved scalability, Journal of Computational Physics 310 (2016) 301–328

    work page 2016

  3. [3]

    Simon, P

    F. Simon, P. Guillen, P. Sagaut, D. Lucor, A gPC-based approach to uncertain transonic aerodynamics, Computer Methods in Applied Mechanics and Engineering 199 (2010) 1091–1099

    work page 2010

  4. [4]

    Cho, S.-Y

    K.-H. Cho, S.-Y . Shin, W. Kolch, O. Wolkenhauer, Experimental design in systems biology, based on parameter sensitivity analysis using a Monte Carlo method: a case study for the TNF-mediated NF- B signal transduction pathway, Simulation 79 (2003) 726–739

    work page 2003

  5. [5]

    Abagyan, M

    R. Abagyan, M. Totrov, Biased probability Monte Carlo conformational searches and electrostatic calculations for peptides and proteins, Journal of Molecular Biology 235 (3) (1994) 983–1002

    work page 1994

  6. [6]

    E. M. Constantinescu, V . M. Zavala, M. Rocklin, S. Lee, M. Anitescu, A computational framework for uncertainty quantification and stochastic optimization in unit commitment with wind power generation, IEEE Transactions on Power Systems 26 (1) (2011) 431–441

    work page 2011

  7. [7]

    Mateos, T

    J. Mateos, T. Gonzalez, D. Pardo, V . Hoel, A. Cappy, Monte Carlo simulator for the design optimization of low-noise HEMTs, IEEE Transactions on Electron Devices 47 (10) (2000) 1950–1956

    work page 2000

  8. [8]

    Papadrakakis, N

    M. Papadrakakis, N. D. Lagaros, Reliability-based structural optimization using neural networks and Monte Carlo simulation, Computer Methods in Applied Mechanics and Engineering 191 (32) (2002) 3491–3507

    work page 2002

  9. [9]

    Xiu, Numerical Methods for Stochastic Computations: A Spectral Method Approach, Princeton University Press, 2010

    D. Xiu, Numerical Methods for Stochastic Computations: A Spectral Method Approach, Princeton University Press, 2010

    work page 2010

  10. [10]

    J. P. Boyd, Chebyshev and Fourier Spectral Methods: Second Revised Edition, Courier Corporation, 2013

    work page 2013

  11. [11]

    Narayan, J

    A. Narayan, J. Jakeman, Adaptive Leja sparse grid constructions for stochastic collocation and high-dimensional approximation, SIAM Journal on Scientific Computing 36 (6) (2014) A2952–A2983

    work page 2014

  12. [12]

    L. M. M. van den Bos, B. Sanderse, W. A. A. M. Bierbooms, G. J. W. van Bussel, Bayesian model calibration with interpolating polynomials based on adaptively weighted Leja nodes, arXiv e-prints (2018) arXiv:1802.02035

    work page arXiv 2018

  13. [13]

    Barrault, Y

    M. Barrault, Y . Maday, N. C. Nguyen, A. T. Patera, An empirical interpolation method: application to efficient reduced-basis discretization of partial differential equations, Comptes Rendus Mathematique 339 (9) (2004) 667 – 672

    work page 2004

  14. [14]

    J. S. Hesthaven, G. Rozza, B. Stamm, The Empirical Interpolation Method, in: Certified Reduced Basis Methods for Parametrized Partial Differential Equations, Springer Briefs in Mathematics, Springer, Cham, 2016, pp. 67–85

    work page 2016

  15. [15]

    J. S. Hesthaven, B. Stamm, S. Zhang, Efficient greedy algorithms for high-dimensional parameter spaces with applications to empirical interpolation and reduced basis methods, ESAIM: Mathematical Modelling and Numerical Analysis 48 (1) (2014) 259–283

    work page 2014

  16. [16]

    Ohlberger, S

    M. Ohlberger, S. Rave, Reduced Basis Methods: Success, Limitations and Future Challenges, arXiv:1511.02021 [math]

    work page arXiv

  17. [17]

    N. C. Nguyen, A. T. Patera, J. Peraire, A best points interpolation method for efficient approximation of parametrized functions, International Journal for Numerical Methods in Engineering 73 (4) (2008) 521–543

    work page 2008

  18. [18]

    Nobile, R

    F. Nobile, R. Tempone, C. Webster, A sparse grid stochastic collocation method for partial differential equations with random input data, SIAM Journal on Numerical Analysis 46 (5) (2008) 2309–2345

    work page 2008

  19. [19]

    Nobile, R

    F. Nobile, R. Tempone, C. Webster, An Anisotropic Sparse Grid Stochastic Collocation Method for Partial Differential Equations with Random Input Data, SIAM Journal on Numerical Analysis 46 (5) (2008) 2411–2442

    work page 2008

  20. [20]

    Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations

    M. Raissi, P. Perdikaris, G. E. Karniadakis, Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations, arXiv:1711.10561 [cs, math, stat]

    work page internal anchor Pith review Pith/arXiv arXiv

  21. [21]

    Chantrasmi, Pade-legendre Method for Uncertainty Quantification with Fluid Dynamics Applications, Stanford University, 2011

    T. Chantrasmi, Pade-legendre Method for Uncertainty Quantification with Fluid Dynamics Applications, Stanford University, 2011

    work page 2011

  22. [22]

    L. N. Trefethen, Approximation Theory and Approximation Practice, SIAM, 2013

    work page 2013

  23. [23]

    Loukrezis, U

    D. Loukrezis, U. Rmer, H. De Gersem, Numerical Comparison of Leja and Clenshaw-Curtis Dimension-Adaptive Collocation for Stochastic Parametric Electromagnetic Field Problems, arXiv:1712.07223 [cs]

    work page arXiv

  24. [24]

    Higham, Accuracy and Stability of Numerical Algorithms, Other Titles in Applied Mathematics, Society for Industrial and Applied Mathematics, 2002

    N. Higham, Accuracy and Stability of Numerical Algorithms, Other Titles in Applied Mathematics, Society for Industrial and Applied Mathematics, 2002

    work page 2002

  25. [25]

    Narayan, D

    A. Narayan, D. Xiu, Stochastic collocation methods on unstructured grids in high dimensions via interpolation, SIAM Journal on Scientific Computing 34 (3) (2012) A1729–A1752

    work page 2012

  26. [26]

    C. d. Boor, A. Ron, On multivariate polynomial interpolation, Constructive Approximation 6 (3) (1990) 287–302

    work page 1990

  27. [27]

    Sauer, Polynomial interpolation of minimal degree, Numerische Mathematik 78 (1) (1997) 59–85

    T. Sauer, Polynomial interpolation of minimal degree, Numerische Mathematik 78 (1) (1997) 59–85

    work page 1997

  28. [28]

    Neumaier, Solving Ill-Conditioned and Singular Linear Systems: A Tutorial on Regularization, SIAM Review 40 (3) (1998) 636–666

    A. Neumaier, Solving Ill-Conditioned and Singular Linear Systems: A Tutorial on Regularization, SIAM Review 40 (3) (1998) 636–666

    work page 1998

  29. [29]

    Varah, On the Numerical Solution of Ill-Conditioned Linear Systems with Applications to Ill-Posed Problems, SIAM Journal on Numerical Analysis 10 (2) (1973) 257–267

    J. Varah, On the Numerical Solution of Ill-Conditioned Linear Systems with Applications to Ill-Posed Problems, SIAM Journal on Numerical Analysis 10 (2) (1973) 257–267

    work page 1973

  30. [30]

    Higham, The Accuracy of Solutions to Triangular Systems, SIAM Journal on Numerical Analysis 26 (5) (1989) 1252–1265

    N. Higham, The Accuracy of Solutions to Triangular Systems, SIAM Journal on Numerical Analysis 26 (5) (1989) 1252–1265

    work page 1989

  31. [31]

    Klinger, Approximate pseudoinverse solutions to ill-conditioned linear systems, Journal of Optimization Theory and Applications 2 (2) (1968) 117–124

    A. Klinger, Approximate pseudoinverse solutions to ill-conditioned linear systems, Journal of Optimization Theory and Applications 2 (2) (1968) 117–124

    work page 1968

  32. [32]

    Mohideen, V

    S. Mohideen, V . Cherkassky, On Recursive Calculation of the Generalized Inverse of a Matrix, ACM Transactions on Mathematical Software 17 (1) (1991) 130–147

    work page 1991

  33. [33]

    Shinozaki, M

    N. Shinozaki, M. Sibuya, K. Tanabe, Numerical algorithms for the Moore-Penrose inverse of a matrix: Direct methods, Annals of the Institute of Statistical Mathematics 24 (1) (1972) 193–203

    work page 1972

  34. [34]

    Canuto, M

    C. Canuto, M. Y . Hussaini, A. M. Quarteroni, T. A. Zang, Jr, Spectral Methods in Fluid Dynamics, Springer Science & Business Media, 2012. 23

    work page 2012

  35. [35]

    Klimke, B

    A. Klimke, B. Wohlmuth, Algorithm 847: Spinterp: Piecewise multilinear hierarchical sparse grid interpolation in MATLAB, ACM Trans- actions on Mathematical Software 31 (2005) 561–579

    work page 2005

  36. [36]

    C. B. Vreugdenhil, Numerical Methods for Shallow-Water Flow, Springer Verlag, 2013

    work page 2013

  37. [37]

    Hazewinkel, Encyclopaedia of Mathematics, Springer Science & Business Media, 1990

    M. Hazewinkel, Encyclopaedia of Mathematics, Springer Science & Business Media, 1990

    work page 1990

  38. [38]

    R. J. LeVeque, Finite V olume Methods for Hyperbolic Problems, Cambridge University Press, 2002

    work page 2002

  39. [39]

    Goodfellow, Y

    I. Goodfellow, Y . Bengio, A. Courville, Deep Learning, MIT Press, 2016. 24

    work page 2016