Pith. sign in

REVIEW 4 major objections 4 minor 50 references

Derivative-free stochastic bilevel optimization for inverse problems

T0 review · 4 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that regularization parameters and measurement designs in inverse problems can be learned by a stochastic bilevel method that only queries objective values, with total oracle and lower-level complexity O(ε^{-3}) to reach…

desk verdict Original exact-case theory, but the inexact lower-level bias analysis relies on an unstated independence of ξ1 and ξ2 that the paper's own data model violates. read the letter →

arxiv 2411.18100 v1 pith:POPJJX44 submitted 2024-11-27 math.OC

Mathias Staudigl , Simon Weissmann , Tristan van Leeuwen This is my paper · ORCID
classification math.OC MSC 90C1590C2690C5665K10
keywords stochasticbileveloptimizationzeroth-ordermethodsderivative-freeinverseproblemsregularizationparameterlearningGaussiansmoothingGoldsteinsubdifferentialcomplexityanalysis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a stochastic bilevel optimization method for learning regularization parameters (and, in optimal experimental design, measurement policies) in inverse problems from data, when the lower-level problem is a convex nonsmooth variational reconstruction. Its central claim is that this task can be solved without gradients: the algorithm estimates the gradient of a Gaussian-smoothed version of the hyperobjective using only function values, allows the lower-level problem to be solved only inexactly, and still provably converges. The proof supplies finite-time complexity bounds: $O(\varepsilon^{-2})$ iterations and $O(\varepsilon^{-3})$ total calls to the function oracle and lower-level solver to reach small expected prox-gradient norm in the nonconvex case, an explicit convergence rate for the convex case, and a relaxed Goldstein stationarity guarantee for the nonconvex case. If correct, this gives a disciplined, oracle-cheap alternative to hyperparameter grids and manual tuning in data-driven inverse problems.

What carries the argument

The load-bearing object is the Gaussian-smoothed hyperobjective $h_\eta(y)=\mathbb{E}[h(y+\eta U)]$ with $U$ standard Gaussian, whose gradient $\nabla h_\eta(y)=\mathbb{E}[(H(y+\eta U,\xi)-H(y,\xi))/\eta\, BU]$ can be sampled without derivatives. Finite differencing in random Gaussian directions gives the zeroth-order estimator $V_{\eta,m}$, whose variance is controlled by the square-integrable random Lipschitz modulus of $H(\cdot,\xi)$ (Lemma 4.7). The prox-gradient mapping $G_{\eta,t}(y)=(y-\mathrm{prox}_{t r_1}(y-t\nabla h_\eta(y)))/t$ serves as the stationarity measure, and its stochastic analogue is what Algorithm 1 drives to zero. Proposition 4.5 converts smallness of $\nabla h_\eta$ into membership in a Goldstein $\delta$-subdifferential of the original nonsmooth $h$, which is what turns the complexity bounds into a meaningful relaxed-stationarity guarantee.

What would settle it

Compute the Lipschitz modulus of the solution map $x^*(y,\xi_2)$ for the lower-level problem (8.1) as the Tikhonov parameter $\lambda\to 0$ at fixed data; the theory requires $\|\mathrm{lip}_0(H(\cdot,\xi))\|_2<\infty$. If data and parameters exist with $\lambda\to 0$ where this modulus is unbounded or not square-integrable, then the variance bound $s^2=(4+n)^2\|\mathrm{lip}_0(H(\cdot,\xi))\|_2^2/m$ is infinite and the $O(\varepsilon^{-3})$ guarantee collapses; a direct numerical check measures the empirical variance of $V_{\eta,m}(y)$ along such a sequence and observes it diverging as $\lambda\to 0$.

Watch

Extended reading notes

Core claim

The central discovery is a derivative-free proximal-gradient scheme (Algorithm 1) for the hyperobjective $\Psi(y)=\mathbb{E}[F(x^*(y,\xi_2),\xi_1)]+r_1(y)$, where $x^*(y,\xi_2)$ is the unique solution of the lower-level variational problem. The method replaces the unavailable hypergradient by a multi-point finite-difference estimator built from Gaussian random directions and i.i.d. data samples; Gaussian smoothing turns the nonsmooth expected hyperobjective into a $C^{1,1}$ function $h_\eta$, so the prox-gradient machinery applies. With step sizes $\alpha_k=2\beta/(\mathrm{lip}_1(h_\eta)\sqrt{k})$ and batch sizes $m_{k+1}=a\sqrt{k}$, the paper proves $\mathbb{E}\|G_{\eta,\alpha_\kappa}(y_\kappa)\|^2\le\varepsilon$ after $O(\varepsilon^{-2})$ iterations and $O(\varepsilon^{-3})$ function and lower-level oracle calls (Corollary 5.4), and the same rate with controlled inexact lower-level solves (Corollary 5.7). In the convex case it bounds the expected optimality gap explicitly in terms of the smoothing parameter $\eta$ and the inexactness schedule (Theorem 6.1), and in the nonconvex case it shows the output is an $(\varepsilon_1,\varepsilon_2)$-stationary point in the Goldstein sense with $O(\varepsilon_1^{-4}\varepsilon_2^{-2})$ iteration complexity (Theorem 7.1).

Load-bearing premise

The proof hinges on the lower-level solution changing Lipschitz continuously with the learned parameters, with the random rate of change having finite second moment; this can break down when the regularization parameter approaches zero, which is precisely the regime of greatest interest in inverse problems.

Editorial extensions

If this is right

  • If the bounds hold, users can learn regularization parameters and experimental designs with no hypergradient computation: only objective-function samples and an inner solver for the lower-level problem are needed.
  • For the recommended decaying step-size and growing batch-size schedule, reaching expected squared prox-gradient norm $\varepsilon$ requires $O(\varepsilon^{-2})$ iterations and $O(\varepsilon^{-3})$ total inner solves.
  • In the convex case the expected optimality gap vanishes at a rate explicit in $\eta$ and the $\beta_k$ schedule, letting the user trade inner-solve accuracy against total cost.
  • In the nonconvex case the output is certified as a relaxed Goldstein stationary point, a necessary relaxation because no first-order method can certify ordinary $\varepsilon$-stationarity in finite time.
  • Constant step sizes and constant batch sizes converge only to a plateau, so the schedule is essential for the guarantee.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A consequence the authors leave implicit: the theory requires the Lipschitz modulus of the lower-level solution map to be square-integrable over the whole parameter space, which is not verified for the TV/Tikhonov lower-level problem; restricting $y$ to a compact box bounded away from zero, as the experiments do, is a practical but unproven workaround.
  • Because the variance constant grows with the dimension $n$ of the parameter space $Y$, the method is most attractive when few hyperparameters are learned, such as a handful of regularization parameters or a softmax-parametrized measurement policy.
  • A testable extension is to replace i.i.d. Gaussian directions with quasi-Monte Carlo or multilevel Monte Carlo sampling, which the paper flags as future work; the complexity bounds here predict the logarithmic factor would shrink if the cumulative batch sum were reduced.
  • The complexity transfer to Goldstein stationarity suggests the same algorithm could handle other nonsmooth composite upper-level problems, such as group-sparse hyperparameter learning, provided the same smoothing and square-integrability conditions hold.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper develops a zeroth-order (derivative-free) stochastic proximal-gradient method for bilevel optimization problems arising in inverse problems, where the lower-level problem is a convex, possibly nonsmooth variational problem and the upper-level objective is an expected risk over regularization or design parameters. The method uses Gaussian smoothing and finite-difference estimators of the hypergradient, and treats both exact and inexact lower-level solutions. The paper proves iteration and oracle complexity bounds for the prox-gradient mapping in the nonconvex case, an optimality-gap bound in the convex case, a Goldstein-stationarity complexity statement, and reports numerical experiments on one-dimensional signal denoising and on Radon-based experimental design.

Significance. If correct, this is a useful contribution to derivative-free stochastic bilevel optimization under nonsmooth lower-level problems. The paper gives explicit convergence rates, handles inexact lower-level solves as a bias term, and connects the smoothed complexity analysis to a Goldstein-type stationarity criterion. The exact-case analysis is largely standard and plausibly correct, and the numerical experiments illustrate the intended application. However, several load-bearing assumptions are not satisfied by the paper's own numerical model, and the inexact-bias analysis contains an unjustified independence factorization. These issues affect the theoretical claims that cover the reported experiments.

major comments (4)
  1. [§4.7, Lemma 4.10 and Appendix B.4] The proof of Lemma 4.10 factors conditional expectations such as E[lip0(F(·,ξ1))^2 ||xβ−x*||^2 ||U||^2 | F_k] into a product of marginal expectations after invoking "independence of the triple (ξ1, ξ2, U)". The paper does not assume ξ1 and ξ2 are independent; indeed Example 1.1 and Section 8 take ξ1=X and ξ2=D=KX+Z, which are dependent. The same factorization appears in the proof of Lemma 4.9. Since Lemma 4.10 is the only control of the bias terms a_{k+1}, b_{k+1} in decomposition (4.19), Theorems 5.6 and 6.1 are not proved for the paper's own data model. This is load-bearing because Section 8 implements exactly the inexact estimator (4.9).
  2. [§8.1 and Assumption 3] Assumption 3 requires F(·, ξ1) to be globally Lipschitz with finite first moment of the Lipschitz modulus. The upper-level loss used in both experiments is F(x, X)=||x−X||^2 on X=R^{n_x}, which is not globally Lipschitz. No truncation or boundedness of X is stated in Section 8. Consequently the variance bound in Lemma 4.7, the estimate (4.3), and all subsequent complexity results do not apply to the numerical setting as written. The authors should either state a modified assumption (e.g., bounded X with a truncated loss) or prove the needed bounds under the quadratic loss plus boundedness of the iterates.
  3. [Assumptions 5 and 6; §8.1] Assumption 5(S.2) requires y↦x*(y, ξ2) to be Lipschitz uniformly in ξ2, and Assumption 6 requires square-integrability of the Lipschitz modulus of H(·, ξ). The paper does not establish either property for the lower-level problem (8.1) with TV and Tikhonov regularization, and these properties are not consequences of the stated µ-strong convexity. In particular, as λ or τ approach zero the solution map can become highly sensitive, and the theorems are stated on all of Y while the experiments constrain y to a bounded box. The hypotheses of the convergence theory are therefore not verified for the numerical example.
  4. [Appendix B.4, bound following eq. (4.20)] When bounding the b_{k+1} term, the proof uses E[||U||^{2p/(p−2)}]^{(p−2)/p} ≤ n. For a standard Gaussian vector this is false for p>2; Lemma A.1 yields an upper bound of order n + p/(p−2) after raising to the appropriate power. The resulting constant C_F should depend on p and n. This is repairable and does not change the rates, but as printed the lemma is not correct.
minor comments (4)
  1. [Appendix B.2, proof of Lemma 5.1] The proof contains a repeated typo: the displayed bound for ||tilde G_{η,α_k}(y_k) − G_{η,α_k}(y_k)|| ends with "= ||∆W_{k+1}||_*^2", but it should be ||∆W_{k+1}||_*.
  2. [§8.1.1] The sentence "the regularization parameters λ, ν and the smoothing parameter ν" repeats ν; it should presumably read λ, τ and ν.
  3. [§3, after eq. (1.3)] The notation is inconsistent: the lower-level regularizer is r2(x) in (1.3), but two paragraphs below the text refers to "a regularizer/penalty function r2(y)" in the upper level; this should be r1(y).
  4. [§8.2] The numerical sections state that Algorithm 1 is run with inexact lower-level solutions, but Definition 4.6 requires an L^p error bound over the sample. The paper should explain how the implemented stopping criterion enforces the L^p condition, or state that the experiments are heuristic and not covered by the inexact theory.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity is found; the complexity analysis is derived from stated assumptions and external smoothing and stationarity results.

full rationale

The paper's derivation chain is not circular. The hyperobjective h(y)=E[F(x*(y,ξ2),ξ1)] and its Gaussian smoothing h_η are defined independently of the algorithm's output, and the zeroth-order estimator is built from function queries of H; its unbiasedness in Lemma 4.7 follows from Gaussian integration rather than from any fitted constant. The complexity bounds in Theorems 5.2, 5.6, 6.1, and 7.1 are explicit functions of stated quantities such as lip0(H), lip1(h_η), s^2, C_F, β_k, m_k, and η, with no parameter fitted to the quantity being predicted. Proposition 4.5 is imported from Lei et al. [35], and the Gaussian-smoothing lemmas from Nesterov and Spokoiny [40], both external sources; the paper's self-citations ([12], [17], [49]) are contextual or experimental and are not load-bearing for any theorem. The reviewer-flagged weakness in Lemma 4.10 is a genuine assumption gap: the Appendix B.4 proof factors E[lip0(F)^2 ||xβ−x*||^2] into a product using the 'independence of the triple (ξ1,ξ2,U)', which can fail for the dependent pair (X,D)=ξ in the paper's own Example 1.1, and the quadratic loss in Section 8 is not globally Lipschitz as Assumption 3 requires. However, an unverified independence or Lipschitz hypothesis is a correctness risk, not a circular reduction of the conclusion to the inputs. Therefore no circular step is established, and the score is 0.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

The central complexity theorems rely on a set of regularity assumptions on the lower-level solution map and the hyperobjective. Most are standard stochastic-programming regularity conditions; the most fragile is Assumption 5(S.2)/Assumption 6, the uniform Lipschitz property of x*(.,xi_2) with a square-integrable Lipschitz constant of H. The theorems also use the Lei et al. Goldstein relation as an external input. No invented physical or mathematical entities are introduced; the only hand-chosen numbers are the algorithm hyperparameters used in the numerical experiments.

free parameters (6)
  • smoothing parameter eta = eta = 0.01 (signal denoising); eta_k = k^{-1/2} (OED)
    Controls the bias-variance trade-off in the zeroth-order estimator. The theory requires eta shrinking with the target accuracy, but experiments use a fixed or slowly decaying value.
  • step-size scale alpha_0 = alpha_0 = 1 (signal denoising), alpha_0 = 0.2 (OED), alpha_k = alpha_0/sqrt(k)
    Hand-chosen to fit the theory's step-size policy; affects the observed convergence speed in the experiments.
  • lower-level tolerance scale beta_0 = beta_0 = 0.01 (signal denoising), beta_0 = 0.1 (OED), beta_k = beta_0/sqrt(k)
    Hand-chosen inexactness schedule that is intended to satisfy the Lp error condition of Definition 4.6.
  • mini-batch scale m_0 = m_0 = 1 in both experiments, m_k = m_0*sqrt(k)
    Number of Gaussian directions per iteration; theory requires m_k growing to control estimator variance.
  • Tikhonov regularization matrix scale = L2 = 0.012*Delta^{-1} (signal denoising); L = I (OED)
    Problem-specific choice that influences conditioning of the lower-level problem; not fitted to data.
  • upper-level penalty weight = 10^{-6} (signal denoising)
    Added to the upper-level objective to encourage well-conditioned lower-level problems; hand-tuned.
assumptions (6)
  • domain assumption Lower-level problem has a unique solution x*(y,xi_2) for each (y,xi_2), measurable in xi_2 and Lipschitz in y on int dom(r1).
    Standing Hypothesis and Assumption 5 (Sections 1 and 3.1). Without it, the hyperobjective H(y,xi)=F(x*(y,xi_2),xi_1) is not a well-defined Lipschitz function and the entire estimator and algorithm are undefined.
  • domain assumption The random Lipschitz modulus of H(.,xi) has finite second moment.
    Assumption 6. Needed to bound the variance of the gradient estimator (Lemma 4.7); not implied by Assumptions 3-5.
  • domain assumption F is Caratheodory and Lipschitz in x with integrable modulus; f(x)=E[F(x,xi_1)] is finite and Lipschitz.
    Assumptions 2-3; standard measurability and integrability conditions for stochastic programming.
  • domain assumption Lower-level objective is convex and continuously differentiable in x; r2 is proper closed convex.
    Assumption 4; ensures prox and subdifferential calculus applies to the lower-level problem.
  • domain assumption r1 is closed convex proper; dom(r1) is bounded for the convex and stationarity theorems.
    Assumptions 1 and 7. Boundedness is used for the diameter M in Theorem 6.1 and Theorem 7.1.
  • standard math Goldstein-subdifferential membership of smoothed gradients: for small eta, grad h_eta(y) is in the Goldstein delta-subdifferential of h at y up to an epsilon ball.
    Proposition 4.5, borrowed from Lei, Pong, Sun, and Yue (2024), arXiv:2405.04150. This bridge is essential for the relaxed stationarity result in Theorem 7.1 and is not proved in this paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Derivative-free stochastic bilevel optimization for inverse problems." pith.science (2026). https://pith.science/paper/POPJJX44

@misc{pith2026241118100,
  author       = {Pith},
  title        = {Pith review of: Derivative-free stochastic bilevel optimization for inverse problems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/POPJJX44}},
  note         = {Machine review of arXiv:2411.18100}
}
read the original abstract

Inverse problems are key issues in several scientific areas, including signal processing and medical imaging. Data-driven approaches for inverse problems aim for learning model and regularization parameters from observed data samples, and investigate their generalization properties when confronted with unseen data. This approach dictates a statistical approach to inverse problems, calling for stochastic optimization methods. In order to learn model and regularisation parameters simultaneously, we develop in this paper a stochastic bilevel optimization approach in which the lower level problem represents a variational reconstruction method formulated as a convex non-smooth optimization problem, depending on the observed sample. The upper level problem represents the learning task of the regularisation parameters. Combining the lower level and the upper level problem leads to a stochastic non-smooth and non-convex optimization problem, for which standard gradient-based methods are not straightforward to implement. Instead, we develop a unified and flexible methodology, building on a derivative-free approach, which allows us to solve the bilevel optimization problem only with samples of the objective function values. We perform a complete complexity analysis of this scheme. Numerical results on signal denoising and experimental design demonstrate the computational efficiency and the generalization properties of our method.

Figures

Figures reproduced from arXiv: 2411.18100 by the authors.

Figure 1
Figure 1. (a)-(c) Learned regularization parameters (y (1) k , y (2) k , y (3) k ) resulting from Algorithm 1 and (d) sum￾mation over the random operators G˜ βt η,t . 0 0.2 0.4 0.6 0.8 1 -0.2 0 0.2 0.4 0.6 0.8 1 1.2 (a) ground truth signal (noisefree) 0 0.2 0.4 0.6 0.8 1 -0.2 0 0.2 0.4 0.6 0.8 1 1.2 (f) recon (high Tik, high TV) signal (noisy) signal (noisefree) estimate 0 0.2 0.4 0.6 0.8 1 -0.2 0 0.2 0.4 0.6 0.8 1 1.2 (e) re… view at source ↗
Figure 2
Figure 2. (a) Ground truth signal and (b) reconstruction of the signal using the learned regularization parameters (λ, τ, ν)(yN) after N = 300 iterations. As comparison we show the reconstruction (c) using low Tikhonov regularization with λ = 10−3 and low TV regularization with τ = 10−3 , (d) using high Tikhonov regularization with λ = 10−1 and low TV regularization with τ = 10−3 , (e) using low Tikhonov regularization with λ… view at source ↗
Figure 3
Figure 3. Pointwise generalization error in the upper level ei(λ, τ, ν) over the validation data set (x val i , d val i ) mval i=1 , mval = 50. We plot the errors for the different choices of regularization parameters from [PITH_FULL_IMAGE:figures/full_fig_p030_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Realizations of the random triangles in Example 8.3.2 In order to enhance the reconstruction accuracy we have implemented the OED problem of choosing the best possible policy over the set of all possible angles [49]. The regularization parameters are again parametrized…
Figure 5
Figure 5. Figure 5: (a) Ground truth image and (b) reconstruction of the image using the learned regularization parameters (λ, τ, ν)(yN) and the learned policy p(yN) after N = 2000 iterations. As comparison we show the reconstruction of the image (c) using the learned regularization param…
Figure 6
Figure 6. Figure 6: Pointwise generalization error in the upper level ei(p, λ, τ, ν) over the validation data set (x val i , d val i ) mval i=1 , mval = 10. We plot the errors for the different choices of regularization parameters from [PITH_FULL_IMAGE:figures/full_fig_p033_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

50 extracted references · 45 canonical work pages

  1. [1]

    Optimal algorit hms for online convex optimization with multi-point bandit feedback

    Alekh Agarwal, Ofer Dekel, and Lin Xiao. Optimal algorit hms for online convex optimization with multi-point bandit feedback. In 23rd Conference on Learning Theory , pages 28–40, 2010

    work page 2010

  2. [2]

    Solving inverse problems using data- driven models

    Simon Arridge, Peter Maass, Ozan Öktem, and Carola-Bibi ane Schönlieb. Solving inverse problems using data- driven models. Acta Numerica, 28:1–174, 2019

    work page 2019

  3. [3]

    Zerot h-order nonconvex stochastic optimization: Handling constraints, high dimensionality , and saddle points

    Krishnakumar Balasubramanian and Saeed Ghadimi. Zerot h-order nonconvex stochastic optimization: Handling constraints, high dimensionality , and saddle points. Foundations of Computational Mathematics , 22(1):35–76, 2022

    work page 2022

  4. [4]

    Bauschke and Patrick L

    Heinz H. Bauschke and Patrick L. Combettes. Convex Analysis and Monotone Operator Theory in Hilbert Spa ces. Springer - CMS Books in Mathematics, 2016

    work page 2016

  5. [5]

    Modern regularizatio n methods for inverse problems

    Martin Benning and Martin Burger. Modern regularizatio n methods for inverse problems. Acta Numerica, 27:1–111, 2018

    work page 2018

  6. [6]

    Berahas, Liyuan Cao, Krzysztof Choromanski, a nd Katya Scheinberg

    Albert S. Berahas, Liyuan Cao, Krzysztof Choromanski, a nd Katya Scheinberg. A theoretical and empirical comparison of gradient approximations in derivative-free optimization. Foundations of Computational Mathematics , 22(2):507–560, 2022

    work page 2022

  7. [7]

    Bottou, F

    L. Bottou, F. Curtis, and J. Nocedal. Optimization metho ds for large-scale machine learning. SIAM Review , 60(2):223–311, 2018 /11/11 2018

    work page 2018

  8. [8]

    Prediction, Learning, and Games

    Nicolo Cesa-Bianchi and Gabor Lugosi. Prediction, Learning, and Games . Cambridge University Press, Cambridge, 2006

    work page 2006

Show all 50 references
  1. [9]

    Bayesian Exp erimental Design: A Review

    Kathryn Chaloner and Isabella V erdinelli. Bayesian Exp erimental Design: A Review. Statistical Science, 10(3):273 – 304, 1995

    1995

  2. [10]

    Optimization and nonsmooth analysis

    Frank H Clarke. Optimization and nonsmooth analysis . SIAM, 1990

    1990

  3. [11]

    Introduction to derivative-free optimization

    Andrew R Conn, Katya Scheinberg, and Luis N Vicente. Introduction to derivative-free optimization . SIAM, 2009

    2009

  4. [12]

    A r egularized variance-reduced modified extragradient method for stochastic hierarchical games

    Shisheng Cui, Uday V Shanbhag, and Mathias Staudigl. A r egularized variance-reduced modified extragradient method for stochastic hierarchical games. arXiv preprint arXiv:2302.06497, 2023

    2023 arXiv

  5. [13]

    Shanbhag, and Farzad Yousefian

    Shisheng Cui, Uday V . Shanbhag, and Farzad Yousefian. Co mplexity guarantees for an implicit smoothing-enabled method for stochastic mpecs. Mathematical Programming, 2022. 40

    2022

  6. [14]

    Stochastic mode l-based minimization of weakly convex functions

    Damek Davis and Dmitriy Drusvyatskiy . Stochastic mode l-based minimization of weakly convex functions. SIAM Journal on Optimization , 29(1):207–239, 2019

    2019

  7. [15]

    Proximally guided stochastic subgradient method for nonsmooth, nonconvex problems

    Damek Davis and Benjamin Grimmer. Proximally guided stochastic subgradient method for nonsmooth, nonconvex problems. SIAM Journal on Optimization , 29(3):1908–1930, 2024 /04/21 2019

    1908

  8. [16]

    Optimal rates for zero-order convex optimization: The power of two function evaluations

    John C Duchi, Michael I Jordan, Martin J Wainwright, and Andre Wibisono. Optimal rates for zero-order convex optimization: The power of two function evaluations. IEEE T ransactions on Information Theory , 61(5):2788–2806, 2015

    2015

  9. [17]

    Multiagent online learning in time-varying games

    Benoit Duvocelle, Panayotis Mertikopoulos, Mathias S taudigl, and Dries V ermeulen. Multiagent online learning in time-varying games. Mathematics of Operations Research , 2023/01/31 2022

    2023

  10. [18]

    Ehrhardt and Lindon Roberts

    Matthias J. Ehrhardt and Lindon Roberts. Inexact deriv ative-free optimization for bilevel learning. Journal of Mathematical Imaging and Vision, 63(5):580–600, 2021

    2021

  11. [19]

    Analyzing inex act hypergradients for bilevel learning

    Matthias J Ehrhardt and Lindon Roberts. Analyzing inex act hypergradients for bilevel learning. IMA Journal of Applied Mathematics, 89(1):254–278, 8 /13/2024 2024

    2024

  12. [20]

    H.W. Engl, M. Hanke, and G. Neubauer. Regularization of Inverse Problems . Mathematics and Its Applications. Springer Netherlands, 1996

    1996

  13. [21]

    Bilevel programming for hyperparameter optimization and meta-learning

    Luca Franceschi, Paolo Frasconi, Saverio Salzo, Ricca rdo Grazzi, and Massimiliano Pontil. Bilevel programming for hyperparameter optimization and meta-learning. In International conference on machine learning , pages 1568–1577. PMLR, 2018

    2018

  14. [22]

    Bayesian Optimization

    Roman Garnett. Bayesian Optimization. Cambridge University Press, 2023

    2023

  15. [23]

    Stochastic first- and ze roth-order methods for nonconvex stochastic program- ming

    Saeed Ghadimi and Guanghui Lan. Stochastic first- and ze roth-order methods for nonconvex stochastic program- ming. SIAM Journal on Optimization , 23(4):2341–2368, 2024 /04/21 2013

    2024

  16. [24]

    Mini- batch stochastic approximation methods for nonconvex stochastic composite optimization

    Saeed Ghadimi, Guanghui Lan, and Hongchao Zhang. Mini- batch stochastic approximation methods for nonconvex stochastic composite optimization. Mathematical Programming, 155(1-2):267–305, 2016

    2016

  17. [25]

    Michael B. Giles. Multilevel monte carlo methods. Acta Numerica, 24:259–328, 2015

    2015

  18. [26]

    Optimization of lipschitz continuo us functions

    Allen A Goldstein. Optimization of lipschitz continuo us functions. Mathematical Programming, 13:14–22, 1977

    1977

  19. [27]

    On the iteration complexity of hyper- gradient computation

    Riccardo Grazzi, Luca Franceschi, Massimiliano Ponti l, and Saverio Salzo. On the iteration complexity of hyper- gradient computation. International Conference on Machine Learning , pages 3748–3758, 2020

    2020

  20. [28]

    Learning regularization functionals—a supervised training approach

    Eldad Haber and Luis Tenorio. Learning regularization functionals—a supervised training approach. Inverse Problems, 19(3):611, 2003

    2003

  21. [29]

    Per Christian Hansen, Jakob Jørgensen, and William R. B . Lionheart. Computed T omography: Algorithms, Insight, and Just Enough Theory. Society for Industrial and Applied Mathematics, Philadel phia, P A, 2021

    2021

  22. [30]

    A bilevel approach for parameter learning in inverse problems

    Gernot Holler, Karl Kunisch, and Richard C Barnard. A bilevel approach for parameter learning in inverse problems. Inverse Problems, 34(11):115012, 2018

    2018

  23. [31]

    A two-timescale stochastic algorithm framework for bilevel optimization: Complexity analysis and applicatio n to actor-critic

    Mingyi Hong, Hoi-To Wai, Zhaoran Wang, and Zhuoran Yang. A two-timescale stochastic algorithm framework for bilevel optimization: Complexity analysis and applicatio n to actor-critic. SIAM Journal on Optimization , 33(1):147– 180, 2023

    2023

  24. [32]

    Zeroth-order optimization with orthogonal random directions

    David Kozak, Cesare Molinari, Lorenzo Rosasco, Luis Te norio, and Silvia Villa. Zeroth-order optimization with orthogonal random directions. Mathematical Programming, 2022

    2022

  25. [33]

    A bilevel optimization ap proach for parameter learning in variational models

    Karl Kunisch and Thomas Pock. A bilevel optimization ap proach for parameter learning in variational models. SIAM Journal on Imaging Sciences , 6(2):938–983, 2024 /10/14 2013

    2024

  26. [34]

    A fully first-order method for stochastic bilevel optimization

    Jeongyeol Kwon, Dohyun Kwon, Stephen Wright, and Rober t D Nowak. A fully first-order method for stochastic bilevel optimization. PMLR, 2023

    2023

  27. [35]

    S ubdifferentially polynomially bounded functions and gaussian smoothing-based zeroth-order optimization

    Ming Lei, Ting Kei Pong, Shuqin Sun, and Man-Chung Yue. S ubdifferentially polynomially bounded functions and gaussian smoothing-based zeroth-order optimization. arXiv preprint arXiv:2405.04150, 2024

    2024 arXiv

  28. [36]

    D. V . Lindley . On a Measure of the Information Provided b y an Experiment. The Annals of Mathematical Statistics , 27(4):986 – 1005, 1956

    1956

  29. [37]

    R. Liu, J. Gao, J. Zhang, D. Meng, and Z. Lin. Investigati ng bi-level optimization for learning and vision from a uni- fied perspective: A survey and beyond. IEEE T ransactions on Pattern Analysis and Machine Intelligence, 44(12):10045– 10067, dec 2022

    2022

  30. [38]

    First-order penalty method s for bilevel optimization

    Zhaosong Lu and Sanyou Mei. First-order penalty method s for bilevel optimization. SIAM Journal on Optimization, 34(2):1937–1969, 2024

    1937

  31. [39]

    Lectures on Convex Optimization , volume 137 of Springer Optimization and Its Applications

    Yurii Nesterov . Lectures on Convex Optimization , volume 137 of Springer Optimization and Its Applications . Springer International Publishing, 2018. 41

    2018

  32. [40]

    Random gradient -free minimization of convex functions

    Yurii Nesterov and Vladimir Spokoiny . Random gradient -free minimization of convex functions. Foundations of Computational Mathematics, 17(2):527–566, 2017

    2017

  33. [41]

    T echniques for gradient-based bilevel optimization with non-smooth lower level problems

    Peter Ochs, René Ranftl, Thomas Brox, and Thomas Pock. T echniques for gradient-based bilevel optimization with non-smooth lower level problems. Journal of Mathematical Imaging and Vision , 56:175–194, 2016

    2016

  34. [42]

    A zero th-order proximal stochastic gradient method for weakly convex stochastic optimization

    Spyridon Pougkakiotis and Dionysis Kalogerias. A zero th-order proximal stochastic gradient method for weakly convex stochastic optimization. SIAM Journal on Scientific Computing , 45(5):A2679–A2702, 2024 /04/30 2023

    2024

  35. [43]

    Ivanova, and Freddi e Bickford Smith

    Tom Rainforth, Adam Foster, Desi R. Ivanova, and Freddi e Bickford Smith. Modern Bayesian Experimental Design. Statistical Science, 39(1):100 – 114, 2024

    2024

  36. [44]

    Meta-learning with implicit gradients

    Aravind Rajeswaran, Chelsea Finn, Sham M Kakade, and Se rgey Levine. Meta-learning with implicit gradients. Advances in neural information processing systems , 32, 2019

    2019

  37. [45]

    Opt imal experimental design for inverse problems with state constraints

    Lars Ruthotto, Julianne Chung, and Matthias Chung. Opt imal experimental design for inverse problems with state constraints. SIAM Journal on Scientific Computing , 40(4):B1080–B1100, 2018

    2018

  38. [46]

    Shapiro, D

    A. Shapiro, D. Dentcheva, and A. Ruszczy ´ nski. Lectures on Stochastic Programming . Society for Industrial and Applied Mathematics, 2017/12/29 2009

    2017

  39. [47]

    A gradi ent-based bilevel optimization approach for tuning regularization hyperparameters

    Ankur Sinha, Tanmay Khandait, and Raja Mohanty . A gradi ent-based bilevel optimization approach for tuning regularization hyperparameters. Optimization Letters, 18(6):1383–1404, 2024

    2024

  40. [48]

    Introduction to stochastic search and optimization: estim ation, simulation, and control

    James C Spall. Introduction to stochastic search and optimization: estim ation, simulation, and control. John Wiley & Sons, 2005

    2005

  41. [49]

    Se quential experimental design for x-ray ct using deep reinforcement learning

    Tianyuan Wang, Felix Lucka, and Tristan van Leeuwen. Se quential experimental design for x-ray ct using deep reinforcement learning. IEEE T ransactions on Computational Imaging, 10:953–968, 2024

    2024

  42. [50]

    Complexity of finding stationary points of nonconvex nonsmooth functions

    Jingzhao Zhang, Hongzhou Lin, Stefanie Jegelka, Suvri t Sra, and Ali Jadbabaie. Complexity of finding stationary points of nonconvex nonsmooth functions. International Conference on Machine Learning , pages 11173–11182, 2020. 42

    2020

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.