pith. sign in

arxiv: 2606.09291 · v2 · pith:V56AGEJJnew · submitted 2026-06-08 · 🧮 math.OC · cs.NA· math.NA

Multilevel Stochastic Gradient Descent for Risk-Averse PDE-Constrained Optimization

Niklas Baumgarten , Philipp A. Guth , David Schneiderhan , Tommaso Vanzan This is my paper

Pith reviewed 2026-06-27 15:46 UTC · model grok-4.3

classification 🧮 math.OC cs.NAmath.NA
keywords multilevel stochastic gradient descentrisk-averse optimizationPDE-constrained optimizationmultilevel Monte Carloadaptive gradient estimatesconvergence ratesthree-dimensional elliptic problemsparallel scalability
0
0 comments X

The pith

Multilevel stochastic gradient descent with adaptive Monte Carlo estimates outperforms standard batched methods for risk-averse PDE optimization.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper develops a multilevel stochastic gradient descent algorithm for optimization problems constrained by three-dimensional partial differential equations where the objective includes a risk-averse measure. It replaces ordinary gradient estimates with adaptive multilevel Monte Carlo versions that aim to control variance and bias while retaining parallel scalability. A sympathetic reader would care because these problems are expensive to solve and arise in applications such as engineering design under uncertainty. The method is examined on elliptic diffusion problems with large risk-aversion parameters to show gains in convergence rate and computational complexity over standard batched stochastic gradient descent.

Core claim

The algorithm uses adaptive multilevel Monte Carlo gradient estimates, provides parallel scalability as well as improved convergence rates and computational complexity compared to standard batched stochastic gradient descent methods. We study the method in computationally demanding settings using three-dimensional elliptic diffusion problems and large risk-aversion parameters.

What carries the argument

Adaptive multilevel Monte Carlo gradient estimates for risk-averse objectives in PDE-constrained optimization.

If this is right

  • The algorithm achieves improved convergence rates compared to standard batched stochastic gradient descent.
  • Computational complexity is reduced while preserving parallel scalability.
  • The approach remains effective for three-dimensional elliptic diffusion problems with large risk-aversion parameters.
  • Gradient estimates can be constructed adaptively without unacceptable bias or variance growth.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same adaptive estimation strategy might extend to other risk measures or uncertainty-quantification tasks if variance control remains stable.
  • Reducing batch sizes through multilevel sampling could lower memory requirements in large-scale PDE optimization.
  • Similar multilevel ideas could be tested on time-dependent or nonlinear PDE constraints to check whether the complexity gains persist.

Load-bearing premise

Adaptive multilevel Monte Carlo estimates can be constructed for risk-averse objectives without introducing unacceptable bias or variance growth in three-dimensional PDE settings with large risk-aversion parameters.

What would settle it

A numerical experiment in which the variance of the gradient estimates grows unbounded or bias appears as the risk-aversion parameter increases in a three-dimensional elliptic problem would falsify the central claim.

Figures

Figures reproduced from arXiv: 2606.09291 by David Schneiderhan, Niklas Baumgarten, Philipp A. Guth, Tommaso Vanzan.

Figure 1
Figure 1. Figure 1: Update procedure for Mk,ℓ=1 = 16 parallel samples on level ℓ = 1. The result is accumulated onto one domain distributed data structure on level ℓ = 3 [PITH_FULL_IMAGE:figures/full_fig_p013_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Update procedure for Mk,ℓ=2 = 8 and Mk,ℓ=3 = 4 parallel sam￾ples. The result is accumulated onto one domain distributed data structure on level ℓ = 3 [PITH_FULL_IMAGE:figures/full_fig_p013_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Two samples ˜a1 and ˜a2 of the random fields used for A. which is based on (16), however, note that fixed step sizes satisfying τ ∈ (0, c 2L2 ), as derived in the proof of Theorem 3.1, work as well. The goals of the numerical experiments are threefold: (i) To explore the impact of θ, par￾ticularly in high risk-aversion regimes and in demanding computational settings. (ii) To vali￾date the presented converg… view at source ↗
Figure 4
Figure 4. Figure 4: Objective (1) (left), norm of estimate to gradient (7) (center), total number of optimization steps taken in four hours (right) plotted over an increasing risk-aversion parameter θ. on θ as it is also influenced by the estimates (23) and (24), both of which involve constants that depend on θ. To verify this, we collected measurements from the final gradient batch across all three experiments and plotted th… view at source ↗
Figure 5
Figure 5. Figure 5: Total number of samples { P k Mk,ℓ} Lk ℓ=1 (top left), verification of (27) (bottom left), verification of (23) and (24) (center column), verification of (25) and (26) (right column) for three different risk-aversion parameters θ ∈ {8.0, 32.0, 72.0} The increased computational capacity enables the exploration of more samples and addi￾tional levels, thereby reducing the uncertainty in the final objective (1… view at source ↗
Figure 6
Figure 6. Figure 6: Total number of samples { P k Mk,ℓ} Lk ℓ=1 (top left), estimated objective (1) with root of MSE error bars (37) plotted over the iteration k (top center), norm of estimated gradient (9) plotted over the total computing time (top right), communication split (39) over the used levels in the final iteration (bottom left), verification of (24)-(27) over all iterations (bottom center), norm of estimated gradien… view at source ↗
read the original abstract

We present recent advances in applying and analyzing multilevel stochastic gradient descent algorithms to risk-averse, three-dimensional PDE-constrained optimization problems. The algorithm uses adaptive multilevel Monte Carlo gradient estimates, provides parallel scalability as well as improved convergence rates and computational complexity compared to standard batched stochastic gradient descent methods. We study the method in computationally demanding settings using three-dimensional elliptic diffusion problems and large risk-aversion parameters.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 0 minor

Summary. The manuscript presents advances in multilevel stochastic gradient descent algorithms for risk-averse PDE-constrained optimization problems in three dimensions. It employs adaptive multilevel Monte Carlo gradient estimates and claims parallel scalability along with improved convergence rates and computational complexity over standard batched stochastic gradient descent, demonstrated on three-dimensional elliptic diffusion problems with large risk-aversion parameters.

Significance. If the adaptive MLMC gradient estimates achieve the required variance reduction for risk-averse objectives, the approach could deliver meaningful efficiency gains in high-dimensional stochastic PDE optimization, extending multilevel techniques to settings where risk measures inflate sample variance.

major comments (2)
  1. [Abstract] Abstract: The central claim of improved convergence rates and computational complexity depends on the adaptive MLMC estimates attaining variance decay with β > 1 (per standard MLMC complexity theorems) even after risk-aversion inflates gradient variance; no error analysis, complexity theorem, or explicit rate derivation is supplied to confirm this holds for large risk-aversion parameters.
  2. [Abstract] Abstract: The interaction between the O(h^{2}) PDE discretization error in 3D elliptic problems and the additional variance growth from risk-aversion is not analyzed, leaving open whether the multilevel correction terms can still deliver the asserted complexity improvement over batched SGD.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the careful reading and valuable comments on our manuscript. We address the major comments point by point below. We agree that the abstract claims would be strengthened by additional explicit theoretical support and will revise the manuscript accordingly.

read point-by-point responses

  1. Referee: [Abstract] Abstract: The central claim of improved convergence rates and computational complexity depends on the adaptive MLMC estimates attaining variance decay with β > 1 (per standard MLMC complexity theorems) even after risk-aversion inflates gradient variance; no error analysis, complexity theorem, or explicit rate derivation is supplied to confirm this holds for large risk-aversion parameters.

    Authors: We agree with the referee that an explicit error analysis and complexity theorem confirming variance decay with β > 1 for the adaptive MLMC gradient estimates under risk-aversion (including for large risk-aversion parameters) is not supplied in the current manuscript. The work emphasizes algorithmic development and numerical demonstration on 3D elliptic problems. In the revised version we will add a dedicated section deriving the relevant complexity bounds, extending standard MLMC theory to the risk-averse setting. revision: yes

  2. Referee: [Abstract] Abstract: The interaction between the O(h^{2}) PDE discretization error in 3D elliptic problems and the additional variance growth from risk-aversion is not analyzed, leaving open whether the multilevel correction terms can still deliver the asserted complexity improvement over batched SGD.

    Authors: We acknowledge that the manuscript does not provide an explicit analysis of the interaction between the O(h²) discretization error and risk-aversion-induced variance growth. The current results rely on numerical evidence from three-dimensional elliptic diffusion problems. We will incorporate this analysis in the revision to rigorously confirm that the multilevel corrections achieve the claimed complexity gains. revision: yes

Circularity Check

0 steps flagged

No circularity detected; claims rest on external MLMC theory

full rationale

The provided abstract and context contain no equations, parameter fits, or derivation steps that reduce by construction to the paper's own inputs. The central claims of improved convergence and complexity via adaptive multilevel Monte Carlo gradient estimates for risk-averse PDE problems are presented as building on standard multilevel Monte Carlo theory rather than self-defining or self-citing in a load-bearing way. No self-definitional, fitted-input-called-prediction, or uniqueness-imported patterns are exhibited. The derivation chain is therefore self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract provides no explicit free parameters, axioms, or invented entities; assessment limited by lack of full text.

pith-pipeline@v0.9.1-grok · 5597 in / 1013 out tokens · 16242 ms · 2026-06-27T15:46:32.518209+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

41 extracted references · 2 linked inside Pith

  1. [1]

    Optimization problems governed by systems of PDEs with uncertain- ties,

    M. Heinkenschloss and D. P. Kouri, “Optimization problems governed by systems of PDEs with uncertain- ties,”Acta Numer., vol. 34, pp. 491–577, 2025

    2025

  2. [2]

    A trust-region algorithm with adaptive stochastic collocation for PDE optimization under uncertainty,

    D. P. Kouri, M. Heinkenschloss, D. Ridzal, and B. G. van Bloemen Waanders, “A trust-region algorithm with adaptive stochastic collocation for PDE optimization under uncertainty,”SIAM J. Sci. Comput., vol. 35, no. 4, pp. A1847–A1879, 2013

    2013

  3. [3]

    A quasi-Monte Carlo method for optimal control under uncertainty,

    P. A. Guth, V. Kaarnioja, F. Y. Kuo, C. Schillings, and I. H. Sloan, “A quasi-Monte Carlo method for optimal control under uncertainty,”SIAM/ASA J. Uncertain. Quantif., vol. 9, no. 2, pp. 354–383, 2021

    2021

  4. [4]

    Parabolic PDE-constrained optimal control under uncertainty with entropic risk measure using quasi-Monte Carlo integration,

    P. A. Guth, V. Kaarnioja, F. Y. Kuo, C. Schillings, and I. H. Sloan, “Parabolic PDE-constrained optimal control under uncertainty with entropic risk measure using quasi-Monte Carlo integration,”Numer. Math., vol. 156, no. 2, pp. 565–608, 2024

    2024

  5. [5]

    Sample size estimates for risk-neutral semilinear PDE-constrained optimization,

    J. Milz and M. Ulbrich, “Sample size estimates for risk-neutral semilinear PDE-constrained optimization,” SIAM J. Optim., vol. 34, no. 1, pp. 844–869, 2024

    2024

  6. [6]

    A combination technique for optimal control problems constrained by random PDEs,

    F. Nobile and T. Vanzan, “A combination technique for optimal control problems constrained by random PDEs,”SIAM/ASA J. Uncertain. Quantif., vol. 12, no. 2, pp. 693–721, 2024

    2024

  7. [7]

    Projected stochastic gradients for convex constrained problems in Hilbert spaces,

    C. Geiersbach and G. C. Pflug, “Projected stochastic gradients for convex constrained problems in Hilbert spaces,”SIAM J. Optim., vol. 29, no. 3, pp. 2079–2099, 2019

    2079

  8. [8]

    A stochastic gradient method with mesh refinement for PDE-constrained optimization under uncertainty,

    C. Geiersbach and W. Wollner, “A stochastic gradient method with mesh refinement for PDE-constrained optimization under uncertainty,”SIAM J. Sci. Comput., vol. 42, no. 5, pp. A2750–A2772, 2020

    2020

  9. [9]

    Complexity analysis of stochastic gradient methods for PDE- constrained optimal control problems with uncertain parameters,

    M. Martin, S. Krumscheid, and F. Nobile, “Complexity analysis of stochastic gradient methods for PDE- constrained optimal control problems with uncertain parameters,”ESAIM Math. Model. Numer. Anal., vol. 55, no. 4, pp. 1599–1633, 2021

    2021

  10. [10]

    Adaptive sampling strategies for stochastic optimization,

    R. Bollapragada, R. Byrd, and J. Nocedal, “Adaptive sampling strategies for stochastic optimization,” SIAM J. Optim., vol. 28, no. 4, pp. 3312–3343, 2018

    2018

  11. [11]

    Adaptive sampling strategies for risk-averse sto- chastic optimization with constraints,

    F. Beiser, B. Keith, S. Urbainczyk, and B. Wohlmuth, “Adaptive sampling strategies for risk-averse sto- chastic optimization with constraints,”IMA J. Numer. Anal., vol. 43, no. 6, pp. 3729–3765, 2023

    2023

  12. [12]

    An adaptive importance sampling algorithm for risk-averse optimization,

    S. Pieraccini and T. Vanzan, “An adaptive importance sampling algorithm for risk-averse optimization,” J. Comput. Phys., 2026

    2026

  13. [13]

    Stochastic gradient with least-squares control variates,

    F. Nobile, M. Raviola, and N. Schaeffer, “Stochastic gradient with least-squares control variates,” 2025

    2025

  14. [14]

    Multilevel stochastic gradient descent for optimal control under uncertainty,

    N. Baumgarten and D. Schneiderhan, “Multilevel stochastic gradient descent for optimal control under uncertainty,”arXiv preprint arXiv:2506.02647, 2025

    Pith/arXiv arXiv 2025

  15. [15]

    Multilevel Monte Carlo methods,

    M. B. Giles, “Multilevel Monte Carlo methods,”Acta Numer., vol. 24, pp. 259––328, 2015

    2015

  16. [16]

    Multilevel Monte Carlo methods and applications to elliptic PDEs with random coefficients,

    K. A. Cliffe, M. B. Giles, R. Scheichl, and A. L. Teckentrup, “Multilevel Monte Carlo methods and applications to elliptic PDEs with random coefficients,”Comput. Vis. Sci., vol. 14, no. 1, p. 3, 2011. 22 NIKLAS BAUMGARTEN, PHILIPP A. GUTH, DAVID SCHNEIDERHAN, AND TOMMASO VANZAN

    2011

  17. [17]

    Further analysis of multilevel Monte Carlo methods for elliptic PDEs with random coefficients,

    A. L. Teckentrup, R. Scheichl, M. B. Giles, and E. Ullmann, “Further analysis of multilevel Monte Carlo methods for elliptic PDEs with random coefficients,”Numer. Math., vol. 125, no. 3, pp. 569–600, 2013

    2013

  18. [18]

    Multi-level Monte Carlo finite element method for elliptic PDEs with stochastic coefficients,

    A. Barth, C. Schwab, and N. Zollinger, “Multi-level Monte Carlo finite element method for elliptic PDEs with stochastic coefficients,”Numer. Math., vol. 119, no. 1, pp. 123–161, 2011

    2011

  19. [19]

    Multilevel Monte Carlo for scalable Bayesian computations,

    M. Giles, T. Nagapetyan, L. Szpruch, S. Vollmer, and K. Zygalakis, “Multilevel Monte Carlo for scalable Bayesian computations,”arXiv preprint arXiv:1609.06144, 2016

    Pith/arXiv arXiv 2016

  20. [20]

    Multilevel Monte Carlo analysis for optimal control of elliptic PDEs with random coefficients,

    A. A. Ali, E. Ullmann, and M. Hinze, “Multilevel Monte Carlo analysis for optimal control of elliptic PDEs with random coefficients,”SIAM/ASA J. Uncertain. Quantif., vol. 5, no. 1, pp. 466–492, 2017

    2017

  21. [21]

    A multigrid solver for PDE-constrained optimization with uncertain inputs,

    G. Ciaramella, F. Nobile, and T. Vanzan, “A multigrid solver for PDE-constrained optimization with uncertain inputs,”J. Sci. Comput., vol. 101, no. 1, p. 13, 2024

    2024

  22. [22]

    Robust optimization of PDEs with random coefficients using a multilevel Monte Carlo method,

    A. Van Barel and S. Vandewalle, “Robust optimization of PDEs with random coefficients using a multilevel Monte Carlo method,”SIAM/ASA J. Uncertain. Quantif., vol. 7, no. 1, pp. 174–202, 2019

    2019

  23. [23]

    Multilevel quasi-Monte Carlo for optimization under uncertainty,

    P. A. Guth and A. Van Barel, “Multilevel quasi-Monte Carlo for optimization under uncertainty,”Numer. Math, vol. 154, no. 3-4, pp. 443–484, 2023

    2023

  24. [24]

    Multilevel quadrature formulae for the optimal control of random PDEs,

    F. Nobile and T. Vanzan, “Multilevel quadrature formulae for the optimal control of random PDEs,” Numer. Math, vol. 157, no. 6, pp. 2291–2322, 2025

    2025

  25. [25]

    Gradient-based optimisation of the conditional-value-at-risk using the multi-level Monte Carlo method,

    S. Ganesh and F. Nobile, “Gradient-based optimisation of the conditional-value-at-risk using the multi-level Monte Carlo method,”J. Comput. Phys., vol. 495, p. 112523, 2023

    2023

  26. [26]

    A budgeted multi-level Monte Carlo method for full field estimates of multi-PDE problems,

    N. Baumgarten, R. Kutri, and R. Scheichl, “A budgeted multi-level Monte Carlo method for full field estimates of multi-PDE problems,” 2025

    2025

  27. [27]

    Existence and optimality conditions for risk-averse PDE-constrained optimization,

    D. P. Kouri and T. M. Surowiec, “Existence and optimality conditions for risk-averse PDE-constrained optimization,”SIAM/ASA J. Uncertain. Quantif., vol. 6, no. 2, pp. 787–815, 2018

    2018

  28. [28]

    Tr¨ oltzsch,Optimal Control of Partial Differential Equations: Theory, Methods and Applications, vol

    F. Tr¨ oltzsch,Optimal Control of Partial Differential Equations: Theory, Methods and Applications, vol. 112. Providence, RI: American Mathematical Society, 2010

    2010

  29. [29]

    Hinze, R

    M. Hinze, R. Pinnau, M. Ulbrich, and S. Ulbrich,Optimization with PDE constraints, vol. 23. Springer Science & Business Media, 2008

    2008

  30. [30]

    F¨ ollmer and T

    H. F¨ ollmer and T. Knispel,Convex risk measures: Basic facts, law-invariance and beyond, asymptotics for large portfolios, ch. Chapter 30, pp. 507–554

  31. [31]

    Nesterov,Lectures on convex optimization, vol

    Y. Nesterov,Lectures on convex optimization, vol. 137. Springer, 2018

    2018

  32. [32]

    Multilevel Monte Carlo path simulation,

    M. B. Giles, “Multilevel Monte Carlo path simulation,”Oper. Res., vol. 56, no. 3, pp. 607–617, 2008

    2008

  33. [33]

    Updating formulae and a pairwise algorithm for computing sample variances,

    T. F. Chan, G. H. Golub, and R. J. LeVeque, “Updating formulae and a pairwise algorithm for computing sample variances,” 1982

    1982

  34. [34]

    Numerically stable, scalable formulas for parallel and online computation of higher-order multivariate central moments with arbitrary weights,

    P. P´ ebay, T. B. Terriberry, H. Kolla, and J. Bennett, “Numerically stable, scalable formulas for parallel and online computation of higher-order multivariate central moments with arbitrary weights,”Comput. Statist., vol. 31, no. 4, pp. 1305–1325, 2016

    2016

  35. [35]

    Note on a method for calculating corrected sums of squares and products,

    B. P. Welford, “Note on a method for calculating corrected sums of squares and products,”Technometrics, vol. 4, pp. 419–420, 1962

    1962

  36. [36]

    A fully parallelized and budgeted multilevel Monte Carlo method and the application to acoustic waves,

    N. Baumgarten, S. Krumscheid, and C. Wieners, “A fully parallelized and budgeted multilevel Monte Carlo method and the application to acoustic waves,”SIAM/ASA J. Uncertain. Quantif., vol. 12, no. 3, pp. 901–931, 2024

    2024

  37. [37]

    G. J. Lord, C. E. Powell, and T. Shardlow,An Introduction to Computational Stochastic PDEs. Cambridge Texts in Applied Mathematics, Cambridge University Press, 2014

    2014

  38. [38]

    Adaptive step sizes for preconditioned stochastic gradient descent,

    F. K¨ ohne, L. Kreis, A. Schiela, and R. Herzog, “Adaptive step sizes for preconditioned stochastic gradient descent,” 2024

    2024

  39. [39]

    Finite element error analysis of elliptic PDEs with random coefficients and its application to multilevel Monte Carlo methods,

    J. Charrier, R. Scheichl, and A. L. Teckentrup, “Finite element error analysis of elliptic PDEs with random coefficients and its application to multilevel Monte Carlo methods,”SIAM J. Numer. Anal., vol. 51, no. 1, pp. 322–352, 2013

    2013

  40. [40]

    Optimization of conditional value-at-risk,

    R. T. Rockafellar, S. Uryasev,et al., “Optimization of conditional value-at-risk,”J. Risk, vol. 2, pp. 21–42, 2000

    2000

  41. [41]

    Ginkgo: A modern linear operator algebra framework for high performance computing,

    H. Anzt, T. Cojean, G. Flegar, F. G¨ obel, T. Gr¨ utzmacher, P. Nayak, T. Ribizel, Y. M. Tsai, and E. S. Quintana-Ort´ ı, “Ginkgo: A modern linear operator algebra framework for high performance computing,” ACM Trans. Math. Softw., vol. 48, no. 1, pp. 1–33, 2022

    2022