Pith. sign in

REVIEW 2 major objections 4 minor 23 references

A Stochastic Gradient Descent Method for Globally Minimizing Nearly Convex Functions

T0 review · 2 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Adding Gaussian noise whose strength tracks the gap between the current function value and a lower bound makes stochastic gradient descent converge linearly to the global minimum of 'nearly convex' nonconvex functions.

desk verdict A real new result—linear-rate global convergence for a nonconvex class—but the 'first' claim overshoots and Assumption 4.1 needs a conditional-unbiasedness patch. read the letter →

arxiv 2505.03222 v2 pith:EV3GIDKM submitted 2025-05-06 math.OC

Chenglong Bao , Liang Chen , Weizhi Shao This is my paper · ORCID
classification math.OC MSC 65K0590C2690C06
keywords stochasticgradientdescentglobaloptimizationlinearconvergencenearlyconvexfunctionsadaptiveGaussiannoiseone-pointstrongconvexitydouble-loopalgorithmRastriginfunction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Plain gradient descent on a nonconvex function can stop forever at a local minimum. The paper introduces a class of nonconvex functions called nearly convex functions — small perturbations of strongly convex functions that may still contain many strict local minima and saddle points — and proves that a gradient-descent scheme with adaptive Gaussian noise finds their unique global minimizer. The twist is the noise rule: at each step the noise variance is proportional to the gap between the current function value and a lower bound on the optimum, so the algorithm is noisy exactly when it needs to escape a basin and nearly quiet when it approaches the global minimum. With exact gradients and a valid lower bound, convergence to the global minimizer is linear; with stochastic gradients it is linear to a neighborhood whose size is set by the gradient variance and the lower-bound error. The paper also gives a double-loop variant that learns the lower bound on the fly, reaching an $\varepsilon$-neighborhood in $O(\ln^2(1/\varepsilon))$ full-gradient steps.

What carries the argument

The machinery has three parts. First, the new regularity condition nearly convex (Definition 3.3): $f$ has a unique global minimizer $x^*$, satisfies $\|\nabla f(x)\| \le L\|x - x^*\|$, and its perturbation distance $\beta(f, x^*, \alpha)$ to the family of one-point strongly convex functions (gradient pointing toward the minimizer: $\langle \nabla f(x), x - \bar x\rangle \ge \alpha\|x - \bar x\|^2$) is at most $\frac14\sqrt{\alpha^5/(dL^3)}$. That small-perturbation bound carries the argument: Proposition 3.1 sandwiches $f$ between two quadratics around $x^*$, and Proposition 3.2 caps the barrier height of any local basin, so no basin is deep enough to trap the noise-driven process for long. Second, the adaptive noise rule $\sigma_t := \sqrt{\eta s\,(f(x_{t+1/2}) - f_{\mathrm{lb}})_+}$ injected after each gradient step: large far from the lower bound, negligible near it. Third, the contraction lemma (Lemma 5.4), which shows that one step of this update reduces the expected squared distance by the factor $(1 - \eta\lambda/100)$ plus an error proportional to $(f^* - f_{\mathrm{lb}})$; the proof propagates the Gaussian expectation through the gradient map $G(x) = x - \eta\nabla f(x)$ and uses the perturbation bounds to control the cross terms.

What would settle it

A direct test of the theorem's quantitative prediction: take a provably $(\alpha,L)$-nearly convex function, for instance $J^2_{\varepsilon,R}$ with $4\varepsilon(1+\varepsilon\sqrt{1+R^2})^{3/2} = 1$ in $d = 1$, run GND with $\eta = 2\alpha/(5L^2)$, $s = \lambda/(3L)$, and $f_{\mathrm{lb}} = f^*$ over many random trajectories, and compare the empirical contraction factor of $E[\|y_t - x^*\|^2]$ with the predicted $1 - \eta\lambda/100$ and the zero asymptotic error. A second check is dimensional: determine whether any choice of $(\varepsilon, R, \alpha, L)$ satisfies (3.2) at $d = 10$; if the nearly convex class is empty in ten dimensions, then Theorem 4.1 does not cover the 10-dimensional Rastrigin experiments reported in the paper, and the claim that the guarantees scale to high dimension is unsupported.

Watch

Extended reading notes

Core claim

The central result, Theorem 4.1, states that if $f$ is $(\alpha,L)$-nearly convex and the stochastic gradient oracle has variance at most $r^2$, then with $\eta = 2\alpha/(5L^2)$, $s = \lambda/(3L)$, and any lower bound $f_{\mathrm{lb}} \le f^*$, the GND iterates satisfy $$E[\|y_t - x^*\|^2] \le \bigl(1 - \tfrac{\eta\$\lambda$}{100}\bigr)^t \|y_0 - x^*\|^2 + 100b,$$ where $\lambda = 2\alpha - \eta L^2$ and $b = \frac{\eta r^2}{2\alpha-\eta L^2} + \frac{5\eta\lambda+14}{42L}(f^* - f_{\mathrm{lb}})$. When the full gradient is used and $f_{\mathrm{lb}} = f^*$, the error term vanishes ($b = 0$) and the iterates converge linearly to the unique global minimizer $x^*$, a regime the paper describes as the first guarantee that a noisy gradient method efficiently escapes local minima and converges globally. The variable $y_t = x_t - \eta\nabla f(x_t)$ is the point after the full-gradient step; the stochastic-gradient point $x_{t+1/2} = y_t - \eta\omega_t$ (with oracle noise $\omega_t$) receives the Gaussian perturbation $\sigma_t \xi_t$, and the next tracked point is $y_{t+1} = G(x_{t+1})$, so the Gaussian convolution of $f$ is what tames the perturbation. Theorem 4.2 extends the guarantee to the double-loop DL-GND algorithm, which alternately refines the lower-bound estimate and runs GND so the user need not know $f^*$ in advance; with full gradients it reaches an $\varepsilon$-neighborhood of $x^*$ in $O(\ln^2(1/\varepsilon))$ iterations with high probability.

Load-bearing premise

The argument collapses if the perturbation bound $\beta(f, x^*, \alpha) \le \frac14\sqrt{\alpha^5/(dL^3)}$ fails: the paper sets this constant for analytical convenience, admits it could be relaxed but gives no quantitative relaxation, and since it shrinks as $1/\sqrt{d}$, the class of provably covered functions thins out as dimension grows.

Editorial extensions

If this is right

  • On any $(\alpha,L)$-nearly convex function with exact gradients, choosing $\eta = 2\alpha/(5L^2)$, $s = \lambda/(3L)$, and $f_{\mathrm{lb}} = f^*$ yields linear convergence to the global minimizer, with $O(\ln(1/\varepsilon))$ iterations needed to reach squared distance $\varepsilon$ with high probability.
  • With stochastic gradients of variance $r^2$, the same algorithm converges linearly to a neighborhood of radius proportional to $\eta r^2/\lambda$ (plus a lower-bound error term), so oracle noise, not the landscape's local minima, limits the achievable accuracy.
  • The double-loop DL-GND reaches an $\varepsilon$-neighborhood of the global minimizer in $O(\ln^2(1/\varepsilon))$ full-gradient iterations with high probability and without knowing $f^*$ in advance, because it refines the lower bound from the best function values seen along the run.
  • Because nearly convex functions may contain infinitely many strict local minima and saddle points, the guarantees cover a landscape class strictly outside the reach of standard global-convergence theory for first-order methods, which typically excludes non-global stationary points.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: the dimension-shrinking factor $1/\sqrt{d}$ in (3.2) is the part most likely to limit the theory's reach; if the threshold could be relaxed to a dimension-independent constant, the class would plausibly cover high-dimensional objectives of the Rastrigin type, and one can test this by checking whether the contraction condition in Remark 4 still holds for such functions.
  • Editorial extension: the rule 'noise variance proportional to the value gap' is a transferable design principle; the same schedule could be applied to derivative-free methods, evolutionary strategies, or reinforcement learning, where the objective value (or return) is observed but the gradient is not, suggesting a family of global optimizers beyond SGD.
  • Editorial extension: the paper proves PL implies nearly convexity in one dimension but leaves the higher-dimensional implication open; if it holds under mild smoothness assumptions, the nearly-convex theory would automatically cover many over-parameterized machine-learning objectives that satisfy PL-type conditions.
  • Editorial extension: the double-loop lower-bound update is effectively a running best-value tracker; replacing the convex combination with a soft-max over recent best values, or with a schedule that never raises $f_{\mathrm{lb}}$ above $f^*$, could improve the constants in Theorem 4.2 while preserving the $O(\ln^2(1/\varepsilon))$ rate - a testable algorithmic variant.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper introduces a new regularity class called nearly convex functions, defined as small perturbations of one-point strongly convex functions, where the perturbation is controlled by the dimension-dependent bound in Definition 3.3(c). For this class the authors propose the Gaussian Noise Descent (GND) algorithm, which adds an adaptive Gaussian noise whose variance depends on the difference between the current function value and a lower bound estimate f_lb. The main result, Theorem 4.1, states that with step size eta = 2alpha/(5L^2), noise factor s = lambda/(3L), and f_lb <= f*, the iterates y_t = x_t - eta grad f(x_t) satisfy E||y_t - x*||^2 <= (1 - eta lambda/100)^t ||y_0 - x*||^2 + 100b, where b measures the stochastic gradient variance and the lower-bound gap. In particular, with full gradients and f_lb = f*, linear convergence to the unique global minimizer is obtained. A double-loop variant, DL-GND, adaptively updates f_lb and is shown to reach an epsilon-neighborhood in O(ln^2(1/epsilon)) steps in the full-gradient case and O((1/epsilon) ln(1/epsilon)) steps in the stochastic case. The paper also contains examples of nearly convex functions and numerical experiments on a univariate nearly convex function and on Rastrigin functions.

Significance. If the main theorem is correct as stated, this is a genuine theoretical contribution: it gives the first linear-rate global convergence guarantee for a noisy gradient descent method on a nonconvex function class with multiple strict local minima. The derivation is self-contained and the constants eta and s are explicit rather than fitted to the conclusion. The proof of Lemma 5.4 is a real contraction calculation, and the double-loop lower-bound estimation in Theorem 4.2 is a nontrivial extension. The paper's claims about the breadth of the nearly convex class, however, are partly undercut by the dimension-dependent small-perturbation bound (3.2) and by the fact that the stochastic-oracle assumption is not stated in the conditional form actually used in the proof. The numerical experiments are reasonable but do not verify near convexity for the Rastrigin test functions.

major comments (2)
  1. [Assumption 4.1 and the proof of Theorem 4.1 in Section 5.2] Assumption 4.1 states unbiasedness and variance bounds only pointwise for fixed x: E[SG(x)] = grad f(x) and E||SG(x) - grad f(x)||^2 <= r^2. In the proof of Theorem 4.1, however, the step after the centered display following Eq. (5.16) uses E[omega_t | F_t] = 0 and E||omega_t||^2 | F_t] <= r^2 to obtain E[||y_t - eta omega_t - x*||^2 | F_t] <= ||y_t - x*||^2 + eta^2 r^2. This conditional step is not justified by Assumption 4.1 alone: E[omega_t | F_t] can be a nonzero F_t-measurable vector aligned with y_t - x*, which would add a positive cross term of order eta * delta * ||y_t - x*||^2 and can destroy the contraction factor (1 - eta lambda/100). The same conditional unbiasedness is reused in the probability estimates of Theorem 4.1 and in the proof of Theorem 4.2. This is a patchable assumption gap rather than an error in the subsequent algebra: the authors should either add to Assumption 4.1 the requirements E[SG(x_t) | F_t] = grad f(x_t) and E[||SG(x_t) - grad f(x_t)||^2 | F_t] <= r^2 for the induced filtration, or explicitly state that the oracle noises are independent of the past. Without such a strengthening, Theorem 4.1 is not derivable as written.
  2. [Definition 3.3(c), Eq. (3.2)] The small-perturbation bound beta(f,x*,alpha) <= (1/4) sqrt(alpha^5/(d L^3)) shrinks as d^{-1/2} with dimension, and the text immediately after Definition 3.3 says the constant can be relaxed but gives no quantitative relaxation. This bound is load-bearing: Lemma 5.4's contraction factor in Eq. (5.14) is derived from the specific constant 1/4 through the estimate C1 < 19/28 lambda in Eq. (5.13). For a reader evaluating whether the class of nearly convex functions is broad enough to be significant, the dimension dependence and the lack of a relaxed version are important. The authors should either prove a quantitative relaxation of (3.2) or state explicitly that the linear global-convergence guarantee applies only to perturbations of order O(d^{-1/2}) relative to the underlying quadratic term.
minor comments (4)
  1. [Display before Eq. (5.11)] The inequality sigma(x) <= sqrt(eta s L/2) ||x - x*||^2 + eta s (f* - f_lb) is dimensionally inconsistent: the first term scales as length squared and the second as a function value. What is used in the subsequent derivation is sigma(x)^2 <= (eta s L/2) ||x - x*||^2 + eta s (f* - f_lb), or equivalently sigma(x) <= sqrt(eta s L/2) ||x - x*|| + sqrt(eta s (f* - f_lb)). Please correct this display.
  2. [Proofs of Theorems 4.1 and 4.2, Section 5] The proof of Theorem 4.1 refers to 'by using Lemma 5.4, (4.2) and (5.2)' and the proof of Theorem 4.2 repeatedly says 'we know that (5.3) holds' and 'by using (5.3)'; these references do not correspond to numbered equations in the manuscript and should be renumbered to point to the specific displayed inequalities being used.
  3. [Section 6.2, Rastrigin experiments] The Rastrigin experiments are presented as validating the theoretical findings, but the paper does not verify that the Rastrigin functions with the chosen parameters satisfy Definition 3.3(c), and near convexity is not established for them. The text should state explicitly that these experiments are illustrative rather than a direct verification of the assumptions of Theorem 4.1.
  4. [Introduction, first paragraph of Section 1] The phrase 'first theoretical guarantee that noisy gradient descent can efficiently escape local minima and achieve global convergence' is stronger than necessary, since Ref. [5] already provides an algebraic global-convergence guarantee for an adaptive-noise scheme; the new contribution is the linear rate under the nearly convex class. Please qualify the claim accordingly.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: convergence theorem is derived from explicit assumptions; the admitted convenience constant is an assumption-strength issue, not a fitted conclusion.

full rationale

The paper's central claim — linear convergence of Gaussian-Noise Descent for (alpha,L)-nearly convex functions — is obtained from a standard proof chain: Lemma 5.4 proves a one-step contraction using only Definition 3.3, Assumption 4.1, Lemma 5.1, and Lemma 5.2, and Theorem 4.1 then applies this contraction recursively. No step of the derivation reuses the conclusion as an input. The constants eta = 2alpha/(5L^2) and s = lambda/(3L) are chosen in the proof to satisfy the sufficient inequality (4.6); the paper states this openly: 'As will be observed from (5.12) ... we typically set eta = 2alpha/(5L^2) and s = lambda/(3L).' This is a proof-design choice, not a fitted prediction. The lower bound flb is an explicit algorithmic input; the neighborhood size 100b depends on the gradient variance r^2 and on f* - flb, so the convergence-to-a-neighborhood statement follows from the analysis rather than being assumed. The small-perturbation bound in Definition 3.3(c), with the admitted remark that 'the constant ... is set for ease of the forthcoming convergence analysis of algorithms. This bound may be relaxed to a larger value,' is a stated assumption on the function class; it does not define the conclusion, and the theorem is explicitly conditional on it. If the bound is too restrictive in practice, that is a strength-of-assumption concern, not a circularity. The conditioning of Assumption 4.1 along the randomized filtration is a technical proof gap (the paper's conditional-expectation step is not fully justified by the stated oracle assumption), but it is an assumption gap, not a reduction of the claimed result to its inputs. Self-citations such as [14] and [5] are used for motivation and context, not as load-bearing uniqueness theorems. Numerical experiments use tuned hyperparameters, but they do not feed back into the theoretical derivation. Overall, the derivation is self-contained against the stated assumptions, and no circular step is present.

Assumptions & free parameters 4 free parameters · 5 assumptions · 1 invented entities

The central claim rests on a newly defined function class and several standard stochastic-oracle assumptions. The proof introduces no fitted constants drawn from data; eta and s are chosen for the analysis, and flb must be a valid lower bound. The main structural input is the small-perturbation bound (3.2), which is the paper's real restriction.

free parameters (4)
  • step size eta = 2*alpha/(5*L^2)
    Chosen in Theorem 4.1 and Lemma 5.4 so that the contraction factor (1 - eta*lambda/100) is less than 1; in general, eta < 2*alpha/L^2 is required. The specific value 2/5 is an ad hoc choice to make inequality (4.6) hold.
  • noise factor s = lambda/(3L)
    Chosen in Theorem 4.1 and Lemma 5.4 to satisfy inequality (4.6); s is otherwise a hyperparameter controlling noise strength.
  • lower bound flb = must be <= f*; ideally flb = f*
    The convergence neighborhood in Theorem 4.1(b) is proportional to b, which grows linearly with (f* - flb). Numerical experiments (Figure 7) show that even for a 2D Rastrigin function, a gap of 1e-8 or smaller is needed for convergence to 1e-3. This is not a fitted number, but a problem-dependent input that must be sharp.
  • combination coefficient gamma (DL-GND) = (1-etaL)^2*eps / ((1-etaL)^2*eps + 100*L*b_eps)
    In Theorem 4.2, gamma is set to this formula to force the lower-bound update to progress; in practice it must be tuned (Figure 8).
assumptions (5)
  • domain assumption The objective function f is (alpha,L)-nearly convex (Definition 3.3), including unique global minimizer, ||grad f(x)|| <= L||x-x*||, and beta(f,x*,alpha) <= (1/4) sqrt(alpha^5/(d L^3)).
    This is the class for which all convergence theorems are proved; it is introduced in this paper and is much more restrictive than L-smoothness or the PL condition.
  • domain assumption Stochastic gradient oracle SG(x) satisfies E[SG(x)] = grad f(x) and E[||SG(x)-grad f(x)||^2] <= r^2 (Assumption 4.1).
    Standard SGD assumption; the proof of Theorem 4.1 additionally uses conditional unbiasedness E[omega_t|F_t]=0, which is not explicitly stated.
  • domain assumption flb <= f* for Algorithm 1 (Remark 3).
    The noise term uses (f(x)-flb)_+, and the proof relies on the identity (f(x)-flb)_+ = f(x)-flb; if the lower bound overshoots, the guarantees of Theorem 4.1 do not apply (Remark 6).
  • standard math Gaussian concentration and moment identities for xi ~ N(0, I_d/d) (Lemma 5.1).
    Used throughout the proof of Lemma 5.2 and Lemma 5.4.
  • ad hoc to paper The constants eta = 2alpha/(5L^2) and s = lambda/(3L) satisfy inequality (4.6).
    The proof requires beta*sqrt(2d)(sqrt(2/(pi eta s (alpha-beta)))+1)(1+eta s L) <= 2alpha - eta L^2 - sL/2; the specific choice of eta and s is made to satisfy this bound. The paper notes the constant in (3.2) can be relaxed but gives no quantitative relaxation.
invented entities (1)
  • nearly convex function class (Definition 3.3) independent evidence
    purpose: Defines the set of functions for which GND is guaranteed to converge to the global minimizer; it generalizes one-point strong convexity to a slightly perturbed nonconvex class with bounded local-minimum depth.
    The paper gives explicit nonconvex examples (Examples 3.2 and 3.3) satisfying the definition, so the class is nonempty; the convergence theorems provide testable predictions (linear decay of MSE and N-CP) that the numerical experiments confirm.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Stochastic Gradient Descent Method for Globally Minimizing Nearly Convex Functions." pith.science (2026). https://pith.science/paper/EV3GIDKM

@misc{pith2026250503222,
  author       = {Pith},
  title        = {Pith review of: A Stochastic Gradient Descent Method for Globally Minimizing Nearly Convex Functions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EV3GIDKM}},
  note         = {Machine review of arXiv:2505.03222}
}
read the original abstract

This paper proposes a stochastic gradient descent method with an adaptive Gaussian noise term for the global minimization of nearly convex functions, which are nonconvex and possess multiple strict local minimizers. The noise term, independent of the gradient, is determined by the difference between the current function value and a lower bound estimate of the optimal value. In both probability space and state space, we show that the proposed algorithm converges linearly to a neighborhood of the global optimal solution. The size of this neighborhood depends on the variance of the gradient and the deviation between the estimated lower bound and the optimal value. In particular, when full gradient information is available and a sharp lower bound of the objective function is provided, the algorithm achieves linear convergence to the global optimum. Furthermore, we introduce a double-loop scheme that alternately updates the lower bound estimate and the optimization sequence, enabling convergence to a neighborhood of the global optimum that depends solely on the gradient variance. Numerical experiments on several benchmark problems demonstrate the effectiveness of the proposed algorithm.

Figures

Figures reproduced from arXiv: 2505.03222 by the authors.

Figure 1
Figure 1. The relationships among RSI (3.9), PL Condition (3.10), Nearly Convexity (Definition 3.3), and QG Condition (3.11), under the assumption that f is L-smooth and admits a unique global minimizer. Definition 3.6 (Quadratic Growth (QG)). There exists a constant µq > 0 such that f(x) − f ∗ ≥ µq 2 ∥x − x ∗ p∥ 2 = µq 2 dist(x, S) 2 ∀x ∈ R d , x∗ p ∈ ΠS (x). (3.11) Note that all of the conditions in the above definitions ar… view at source ↗
Figure 2
Figure 2. Plots of the functions J 1 n,k(x) and J 2 ε,R(x) defined in Examples 3.2 and 3.3, together with their derivatives. For J 1 n,k(x), we choose k = 1 and n = n1 = 199. For J 2 ε,R(x), we set ε = 2 27 and R = √ 18161 8 , such that ε √ 1 + R2 = 5 4 and 4ε [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. The MSE and N-CP versus iterations of the GD, GND, and DL-GND algorithms are evaluated by minimizing J 1 7,1(x) and J 1 112,2(x) given by Equation (6.2). 6 Numerical experiments In this section, we conduct numerical experiments to test the performance of the proposed GND (Algo￾rithm 1) and DL-GND (Algorithm 2) algorithms. By sampling 10,000 initial points, we run the algorithms and evaluate their performance using t… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: The convergence behavior for minimizing the two dimensional Rastrigin function [PITH_FULL_IMAGE:figures/full_fig_p022_4.png]
Figure 5
Figure 5. Figure 5: One trajectory for minimizing the two dimensional Rastrigin function [PITH_FULL_IMAGE:figures/full_fig_p022_5.png]
Figure 6
Figure 6. Figure 6: Minimizing the two dimensional Rastrigin function [PITH_FULL_IMAGE:figures/full_fig_p023_6.png]
Figure 7
Figure 7. Figure 7: Minimizing the two-dimensional Rastrigin function [PITH_FULL_IMAGE:figures/full_fig_p023_7.png]
Figure 8
Figure 8. Figure 8: Minimizing the two-dimensional Rastrigin function [PITH_FULL_IMAGE:figures/full_fig_p023_8.png]
Figure 9
Figure 9. Figure 9: The convergence behavior for minimizing the ten-dimensional Rastrigin function [PITH_FULL_IMAGE:figures/full_fig_p025_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

23 extracted references · 17 canonical work pages

  1. [1]

    On exponential convergence of sgd in non-convex over-parametrized learning

    Raef Bassily, Mikhail Belkin, and Siyuan Ma. On exponential convergence of sgd in non-convex over-parametrized learning. arXiv preprint arXiv:1811.02564 , 2018

    arXiv 2018

  2. [2]

    Large-scale machine learning with stochastic gradient descent

    L´ eon Bottou. Large-scale machine learning with stochastic gradient descent. In Proceedings of COMPSTAT’2010: 19th International Conference on Computational StatisticsParis France, August 22-27, 2010 Keynote, Invited and Contributed Papers , pages 177–186. Springer, 2010

    work page 2010

  3. [3]

    Diffusion for global optimization in Rn

    Tzuu-Shuh Chiang, Chii-Ruey Hwang, and Shuenn Jyi Sheu. Diffusion for global optimization in Rn. SIAM Journal on Control and Optimization , 25(3):737–753, 1987

    work page 1987

  4. [4]

    Large Deviations Techniques and Applications

    Amir Dembo. Large Deviations Techniques and Applications . Springer, 2009

    work page 2009

  5. [5]

    An algebraically converging stochastic gradient descent algorithm for global optimization

    Bj¨ orn Engquist, Kui Ren, and Yunan Yang. An algebraically converging stochastic gradient descent algorithm for global optimization. arXiv preprint arXiv:2204.05923 , 2022

    arXiv 2022

  6. [6]

    Adaptive state-dependent diffusion for derivative-free optimization

    Bj¨ orn Engquist, Kui Ren, and Yunan Yang. Adaptive state-dependent diffusion for derivative-free optimization. Communications on Applied Mathematics and Computation , 6:1241–1269, 2024

    work page 2024

  7. [7]

    Escaping from saddle points — online stochastic gradient for tensor decomposition

    Rong Ge, Furong Huang, Chi Jin, and Yang Yuan. Escaping from saddle points — online stochastic gradient for tensor decomposition. In Peter Gr¨ unwald, Elad Hazan, and Satyen Kale, editors, Proceedings of The 28th Conference on Learning Theory , volume 40 of Proceedings of Machine Learning Research, pages 797–842, Paris, France, 03–06 Jul 2015. PMLR

    work page 2015

  8. [8]

    Recursive stochastic algorithms for global optimization in Rd

    Saul B Gelfand and Sanjoy K Mitter. Recursive stochastic algorithms for global optimization in Rd. SIAM Journal on Control and Optimization , 29(5):999–1018, 1991

    work page 1991

Show all 23 references
  1. [9]

    Diffusions for global optimization

    Stuart Geman and Chii-Ruey Hwang. Diffusions for global optimization. SIAM Journal on Control and Optimization, 24(5):1031–1043, 1986

    1986

  2. [10]

    Gradient descent is optimal under lower restricted secant inequality and upper error bound

    Charles Guille-Escuret, Adam Ibrahim, Baptiste Goujaud, and Ioannis Mitliagkas. Gradient descent is optimal under lower restricted secant inequality and upper error bound. Advances in Neural Information Processing Systems, 35:24893–24904, 2022

    2022

  3. [11]

    Large-time behavior of perturbed diffusion markov pro- cesses with applications to the second eigenvalue problem for fokker-planck operators and simulated annealing

    Chii-Ruey Hwang and Shuenn-Jyi Sheu. Large-time behavior of perturbed diffusion markov pro- cesses with applications to the second eigenvalue problem for fokker-planck operators and simulated annealing. Acta Applicandae Mathematica, 19(3):253–295, 1990

    1990

  4. [12]

    Kakade, and Michael I

    Chi Jin, Rong Ge, Praneeth Netrapalli, Sham M. Kakade, and Michael I. Jordan. How to escape saddle points efficiently. In Doina Precup and Yee Whye Teh, editors, Proceedings of the 34th International Conference on Machine Learning , volume 70 of Proceedings of Machine Learning...

    2017

  5. [13]

    Linear convergence of gradient and proximal- gradient methods under the polyak-/suppress lojasiewicz condition

    Hamed Karimi, Julie Nutini, and Mark Schmidt. Linear convergence of gradient and proximal- gradient methods under the polyak-/suppress lojasiewicz condition. InJoint European conference on machine learning and knowledge discovery in databases , pages 795–811. Springer, 2016

    2016

  6. [14]

    Bobby Kleinberg, Yuanzhi Li, and Yang Yuan. An alternative view: When does SGD escape local minima? In Jennifer Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning , volume 80 of Proceedings of Machine Learning Research, pages 2...

    2018

  7. [15]

    Asymptotic global behavior for stochastic approximation and diffusions with slowly decreasing noise effects: global minimization via monte carlo

    Harold J Kushner. Asymptotic global behavior for stochastic approximation and diffusions with slowly decreasing noise effects: global minimization via monte carlo. SIAM Journal on Applied Mathematics, 47(1):169–185, 1987

    1987

  8. [16]

    Error bounds, PL condition, and quadratic growth for weakly convex functions, and linear convergences of proximal point methods

    Feng-Yi Liao, Lijun Ding, and Yang Zheng. Error bounds, PL condition, and quadratic growth for weakly convex functions, and linear convergences of proximal point methods. In 6th Annual Learning for Dynamics & Control Conference , pages 993–1005. PMLR, 2024. 29

    2024

  9. [17]

    SGDR: Stochastic gradient descent with warm restarts

    Ilya Loshchilov and Frank Hutter. SGDR: Stochastic gradient descent with warm restarts. In International Conference on Learning Representations, 2017

    2017

  10. [18]

    Linear convergence of first order methods for non-strongly convex optimization

    Ion Necoara, Yu Nesterov, and Francois Glineur. Linear convergence of first order methods for non-strongly convex optimization. Mathematical Programming, 175:69–107, 2019

    2019

  11. [19]

    Gradient methods for the minimisation of functionals

    Boris T Polyak. Gradient methods for the minimisation of functionals. USSR Computational Mathematics and Mathematical Physics , 3(4):864–878, 1963

    1963

  12. [20]

    Systems of extremal control

    Leonard Andreeviˇ c Rastrigin. Systems of extremal control. Nauka, 1974

    1974

  13. [21]

    Stochastic convex optimization

    Shai Shalev-Shwartz, Ohad Shamir, Nathan Srebro, and Karthik Sridharan. Stochastic convex optimization. In Proceedings of the 22nd Conference on Learning Theory , 2009

    2009

  14. [22]

    How does learning rate decay help modern neural networks? arXiv preprint arXiv:1908.01878 , 2019

    Kaichao You, Mingsheng Long, Jianmin Wang, and Michael I Jordan. How does learning rate decay help modern neural networks? arXiv preprint arXiv:1908.01878 , 2019

    1908 arXiv

  15. [23]

    Gradient methods for convex minimization: better rates under weaker conditions

    Hui Zhang and Wotao Yin. Gradient methods for convex minimization: better rates under weaker conditions. arXiv preprint arXiv:1303.4645 , 2013. 30

    2013 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.