Pith. sign in

REVIEW 3 major objections 5 minor 127 references

Stability of first-order methods in tame optimization

T0 review · 3 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read In tame nonsmooth optimization, stable points of first-order methods must be local minima, strict local minima are stable, and coercive objectives draw iterates to their critical set.

desk verdict Solid compilation of previously published stability results for the subgradient method on tame functions; the advertised extension to momentum, reshuffling, and coordinate methods is asserted rather than proved. read the letter →

arxiv 2412.00640 v1 pith:RDGVOUQV submitted 2024-12-01 math.OC

Lexiao Lai This is my paper · ORCID
classification math.OC MSC 49J5290C3037B2503C64
keywords first-ordermethodssubgradientmethodconstantstepsizetamefunctionso-minimalstructuresLyapunovstabilitynonsmoothoptimizationspuriouslocalminima
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper develops a discrete Lyapunov stability theory for first-order optimization methods with constant step size on tame nonsmooth objectives, meaning functions definable in an o-minimal structure, a class that covers most objectives used in statistical learning. It establishes a local dichotomy: a point is stable for any such method only if it is a local minimum, and it is stable whenever it is a strict local minimum. For coercive tame functions, it proves the set of critical points is globally stable, so iterates from any bounded initialization eventually enter every prescribed neighborhood of the critical set once the step size is small enough. It also shows that some spurious local minima are strongly unstable for the subgradient method without added noise, under geometric conditions (higher-order metric subregularity and the Verdier condition) verified on a ReLU neural-network loss and robust PCA. The practical consequence is that for highly irregular objectives, whether first-order iterates stay near a point is governed by the tame structure of the objective rather than by convexity or smoothness.

What carries the argument

The carrying mechanism is the approximation property (Definition 2.1): a method is approximated by subgradient trajectories when, for every compact initialization and finite time horizon, all sufficiently small constant step sizes produce iterates that stay within $\epsilon$ of some absolutely continuous solution of $x'(t)\in -c\,\partial f(x(t))$. This transfers the discrete iteration into the continuous subgradient flow, where the chain rule for tame functions, $(f\circ x)'=-\|x'\|^2$, guarantees monotone decrease until a critical point is reached. Around a local minimum, strictness plus a Łojasiewicz-type inequality creates a connected sublevel-set barrier that keeps the shadowing trajectory inside a small ball; around a spurious critical point, a Chetaev function whose values grow along iterates, together with metric subregularity and the Verdier condition, prevents the distance to the critical manifold from shrinking, forcing escape.

What would settle it

Take a semialgebraic locally Lipschitz function with a strict local minimum $x^*$ and simulate the subgradient method for step sizes $\alpha=2^{-j}$ from initializations in $B(x^*,2^{-j})$; if for some fixed $\epsilon>0$ the iterates leave $B(x^*,\epsilon)$ for arbitrarily small such pairs, the sufficiency direction of the stability theorem would be refuted, while any stable point that is not a local minimum would refute the necessity direction.

Watch

Extended reading notes

Core claim

The central claim is that first-order methods that are approximated by subgradient trajectories behave, over finite horizons, like the continuous subgradient flow $x'\in -c\partial f(x)$, and that this approximation turns discrete Lyapunov stability into a local-optimality statement. The paper proves that a stable point of a locally Lipschitz tame function must be a local minimum (Theorem 3.5), that every strict local minimum is stable (Theorem 3.6), and that for coercive tame functions the set of critical points is globally stable (Theorem 4.2): for every $\epsilon>0$ and every bounded set of initial points, a sufficiently small constant step size forces the iterates to eventually lie in $B(\mathrm{crit}(f),\epsilon)$. For the subgradient method, it further proves that a spurious local minimum is strongly unstable when a Chetaev function increases along the iterates and the geometry satisfies metric $\theta_2$-subregularity of $\partial f$ with $\theta_2>1$ together with the Verdier condition along the manifold of critical points. Applications verify these conditions for a ReLU neural-network loss with $\ell^1$ error and for robust principal component analysis with data matrices having zero rows or columns, so the escape is deterministic and almost sure for nearby initializations.

Load-bearing premise

The results rest on the approximation property for each method—the claim that over every finite horizon its iterates, with arbitrarily small constant step size, shadow a subgradient trajectory; for the subgradient method this is proved, whereas for the momentum, random-reshuffling, and coordinate-descent variants it is only asserted to follow from a cited proof.

Editorial extensions

If this is right

  • Stable non-minimizing points are impossible for tame objectives: for any method satisfying the approximation property, stability of a point forces it to be a local minimum.
  • Every strict local minimum of a locally Lipschitz tame function is stable, so sufficiently close initialization and sufficiently small step size confine all iterates to any prescribed neighborhood.
  • For coercive tame functions, the critical set is globally attracting: from any bounded initialization and with small enough constant step size, iterates eventually enter every neighborhood of the critical set.
  • Spurious local minima can be left without stochastic noise: under the geometric conditions of Theorem 5.10, the subgradient method escapes from almost every nearby initialization for all but finitely many step sizes.
  • Function values along the iterates settle near a critical value, so the global behavior is a value-level stabilization and not merely set-wise convergence.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The approximation framework is likely portable to other first-order schemes, such as proximal or adaptive step methods, because the proof of shadowing uses only upper semicontinuity of the Clarke subdifferential, boundedness of iterates, and the tame chain rule; this extension is not carried out in the paper.
  • The instability criteria suggest a preprocessing certificate: on semialgebraic objectives one could algorithmically search stratifications for a Chetaev function and Verdier condition and flag spurious minima before running optimization; the paper does not propose such an algorithm.
  • If the approximation property extends to nonsummable diminishing step sizes, as the introduction indicates for most results, the local stability dichotomy would be schedule-independent; this would be a testable strengthening for momentum and reshuffling variants.
  • The verified examples hint that noise-free escape from sharp spurious minima may be common in real training objectives; a broader numerical survey across benchmark losses would test how often the Verdier condition holds in practice.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This manuscript studies constant-step-size first-order methods for minimizing locally Lipschitz tame functions. It introduces Definition 2.1, an approximation property requiring that discrete iterates be uniformly close, over finite horizons, to some subgradient trajectory of the objective. Theorem 2.2 proves this property for the vanilla subgradient method. The thesis then defines local M-stability (Definition 3.1) and global stability (Definition 4.1), and proves that stable points are necessarily local minima (Theorem 3.5), that strict local minima are stable for every method satisfying Definition 2.1 (Theorem 3.6), and that the set of critical points of a coercive locally Lipschitz tame function is globally stable (Theorem 4.2). Chapter 5 gives sufficient conditions for strong instability of the subgradient method, using Chetaev functions, metric theta-subregularity, and the Verdier condition, with applications to a ReLU-network loss and robust PCA. The central results are conditional on Definition 2.1; Table 2.1 asserts this property for momentum, random reshuffling with momentum, and random-permutations coordinate descent, but the derivations are delegated to the author's earlier work rather than reproduced.

Significance. If the approximation property is established for all four methods, the paper provides a coherent discrete Lyapunov theory for nonsmooth tame optimization: local stability coincides with local minimality modulo strictness, and the critical set is globally attracting in the coercive case. The proofs of Theorems 3.5, 3.6, and 4.2 use the Kurdyka-Lojasiewicz inequality and the chain rule for subgradient trajectories carefully, and Theorem 2.2 for the subgradient method is fully proved in the text. The instability criteria in Chapter 5 are concrete and are verified on two nontrivial applications, which is a strength. However, the advertised scope over 'common first-order methods' is not supported by the manuscript itself: Definition 2.1 is stronger than the approximation notion used in the cited earlier work, and for Algorithms 2-4 the paper only states that the proof 'can be adapted' without giving the required estimates. The stability theorems are therefore rigorously established only for the subgradient method within this document.

major comments (3)
  1. [Chapter 2, Table 2.1 and Remark 2.4] Definition 2.1 requires uniform finite-horizon approximation for every pair of compact sets X0, X1 and every starting index k0, and the text explicitly notes that this is stronger than [83, Definition 3]. For Algorithms 2-4 the manuscript only says that 'using the techniques in [83], the same proof can be adapted' and Remark 2.4 delegates the argument to [83, Section 4]. No explicit estimates involving the momentum parameters beta, gamma, the boundedness of iterates, or the O(alpha) momentum state are provided. Since Theorems 3.5, 3.6, and 4.2 are all formulated for an arbitrary method satisfying Definition 2.1, the local and global stability claims for Algorithms 2-4 are not supported by the arguments given in this manuscript.
  2. [Definitions 2.1 and 3.1 with Algorithm 2] Algorithm 2 is a two-step method whose state includes x_{k-1}, and its initialization requires ||x_{-1}-x0|| <= delta*alpha. Definition 3.1, however, constrains only x0 when testing local stability. Definition 2.1 quantifies over all k0 and over initial segments x0,...,x_{k0} in X1, but the stability condition gives no information about x_{-1}. The manuscript does not show that a uniform approximation property holds when the momentum state is initialized only through the condition ||x_{-1}-x0|| <= delta*alpha, nor does it explain how Definition 2.1's shifted quantifier over k0 is compatible with this state condition. This gap must be resolved before Theorem 3.6 can be applied to Algorithm 2.
  3. [Definition 2.1 with Algorithms 3 and 4] For the random reshuffling and random-permutations coordinate descent methods, Definition 2.1 is written deterministically: it requires the approximation for every sequence in M(f, alpha, X0, k0). The manuscript does not clarify whether this property is meant to hold for every random permutation realization or only with high probability. If [83, Section 4] provides only a probabilistic or in-expectation approximation, the deterministic statements of Theorems 3.5, 3.6, and 4.2 do not follow from that reference. The author should state the exact mode of approximation and either prove the deterministic version or restrict the scope of the theorems accordingly.
minor comments (5)
  1. [Introduction, page 2] The sentence 'Examples of such functions are can be found in Chapter 3' is grammatically broken and should be corrected.
  2. [Theorem 1.6 vs. Theorems 3.5 and 3.6] Theorem 1.6 defines the desingularizing function on [0, infinity), while Theorems 3.5 and 3.6 use it on [0, rho); the notation should be harmonized to avoid a mismatch of domains.
  3. [Definition 2.1] The roles of X0 and X1 could be stated more explicitly: X0 contains the k0-th term and X1 contains the initial segment, but the condition x(0) in X0 does not require x(0) to equal x_{k0}; this is intentional for shifted trajectories but should be clarified, especially because Theorem 3.6 invokes Definition 2.1 with a new initial compact set.
  4. [Proof of Theorem 3.6, around Eq. (3.7)] The proof says 'Consider a sequence generated by the subgradient method' even though the theorem is stated for an arbitrary method M satisfying Definition 2.1; this should read 'by M' to avoid a logical mismatch.
  5. [Lemma 5.9 and Section 5.1] The function sign is defined as a set-valued mapping with sign(0) = [-1,1], but the text repeatedly calls it a function; the set-valued convention should be stated consistently in the proof and in the application to robust PCA.

Circularity Check

1 steps flagged · score 4.0 of 10

Approximation property for Algorithms 2–4 is imported from the authors' own prior weaker result; the universal stability theorems otherwise derive from their inputs without definitional circularity.

  1. self citation load bearing [Chapter 2, paragraph following Table 2.1 (p. 10); see also Remark 2.4]
    "Definition 2.1 is different from [83, Definition 3] as it asks the approximation to hold over any given time interval, instead of over a certain one. ... Despite the fact that we need the approximation to hold in a stronger sense, Definition 2.1 continues to be satisfied by the algorithms considered in [83], with the additional assumption that the objective function is tame and lower bounded. In the remainder of this chapter, we prove the subgradient method (Algorithm 1) satisfies Definition 2.1."

    Theorems 3.5, 3.6, and 4.2 are universal statements: they take as input any method satisfying Definition 2.1. For Algorithms 2–4, the thesis supplies no derivation of Definition 2.1; it only asserts that the proof in [83] can be adapted. The cited article is co-authored by the thesis author, and the thesis itself notes that Definition 2.1 is stronger than [83, Definition 3]. Thus the exact strengthened approximation property—finite-horizon uniform approximation for every compact set, with the shifted k0 quantifier needed by the local-stability theorems—is not extracted from an independent proof in the thesis.

full rationale

The central stability characterizations are not circular by construction. Theorem 3.5 derives necessity from the Kurdyka-Łojasiewicz inequality and the approximation property; Theorem 3.6 derives sufficiency from the Łojasiewicz and Kurdyka-Łojasiewicz inequalities; Theorem 4.2 is proved from the same approximation property and tame-geometry finiteness arguments. The subgradient-method case is proved in Theorem 2.2. None of these steps defines a conclusion in terms of itself, fits a parameter and renames it a prediction, or imports a uniqueness theorem to forbid alternatives. The instability results of Chapter 5 are likewise verified directly through explicitly checked Chetaev functions, metric subregularity, and Verdier-type estimates. The only circularity-type burden is the delegation of Definition 2.1 for Algorithms 2–4 to the authors' own prior work [83], which the thesis itself describes as proving a weaker approximation notion. Because the stronger property is precisely the input on which the local and global stability theorems hinge, the scope claim for those algorithms is partially supported by self-citation rather than by a self-contained proof. This warrants a score of 4, not higher, since the subgradient-method results and the general stability theory have independent mathematical content.

Assumptions & free parameters 0 free parameters · 6 assumptions · 0 invented entities

No free parameters are fitted to data; the constants in the applications are constructed, not estimated. The central claims rest on standard tame-geometry machinery and on the approximation property, the latter being partly delegated to prior self-authored work. No new physical or mathematical entities are postulated.

assumptions (6)
  • domain assumption The objective is tame in a fixed o-minimal structure on the real field.
    All stability and convergence results assume locally Lipschitz tame f; this excludes pathological functions such as Rockafellar's nowhere-monotonic Lipschitz function in Example 3.4.
  • standard math Kurdyka-Lojasiewicz inequality for locally Lipschitz tame functions (Theorem 1.7, citing Bolte et al. 2007).
    Used in the proofs of Theorems 3.5, 3.6, and 4.2 to lower-bound subgradient norms near noncritical points.
  • standard math Definable Morse-Sard theorem gives finitely many critical values over bounded sets.
    Used in Chapter 4 to partition sublevel sets and in Chapter 5 to show critical sets are manifolds in applications.
  • standard math Chain rule for subgradient trajectories of tame functions (Proposition 1.8, citing Davis et al.).
    Provides the energy identity d/dt f(x(t)) = -||x'(t)||^2 that drives the continuous-time arguments in Chapters 2-4.
  • ad hoc to paper Definition 2.1 approximation holds for Algorithms 2-4 under the assumptions in Table 2.1.
    The thesis proves the approximation for the vanilla subgradient method in Theorem 2.2, but for momentum, random reshuffling, and coordinate descent it only asserts that the proof in [83] can be adapted.
  • domain assumption In Theorem 5.10, existence of a Chetaev function C with C(x_{k+1})-C(x_k) >= c1 d(x_k,S)^theta1, metric theta2-subregularity, and the Verdier condition.
    These are sufficient conditions imposed on the local geometry around a non-strict local minimum; they are verified case-by-case in Section 5.1.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Stability of first-order methods in tame optimization." pith.science (2026). https://pith.science/paper/RDGVOUQV

@misc{pith2026241200640,
  author       = {Pith},
  title        = {Pith review of: Stability of first-order methods in tame optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RDGVOUQV}},
  note         = {Machine review of arXiv:2412.00640}
}
read the original abstract

Modern data science applications demand solving large-scale optimization problems. The prevalent approaches are first-order methods, valued for their scalability. These methods are implemented to tackle highly irregular problems where assumptions of convexity and smoothness are untenable. Seeking to deepen the understanding of these methods, we study first-order methods with constant step size for minimizing locally Lipschitz tame functions. To do so, we propose notions of discrete Lyapunov stability for optimization methods. Concerning common first-order methods, we provide necessary and sufficient conditions for stability. We also show that certain local minima can be unstable, without additional noise in the method. Our analysis relies on the connection between the iterates of the first-order methods and continuous-time dynamics.

Figures

Figures reproduced from arXiv: 2412.00640 by the authors.

Figure 3.1
Figure 3.1. Iterates of the subgradient method with constant step size. [PITH_FULL_IMAGE:figures/full_fig_p023_3_1.png] view at source ↗
Figure 3.2
Figure 3.2. Illustration of a Rockafellar function. The two examples above call for additional structures on the objective function in order to characterize stability of local minima. We thus confine our investigation to locally Lipschitz tame functions. When 𝑓 is locally Lipschitz and tame, the set of stable points and local minima coincide in two cases. The first is when 𝑓 is continuously differentiable with a locally Lipschi… view at source ↗
Figure 3.3
Figure 3.3. Induction step For the remainder of the proof, we seek to show that 𝑥𝐾 ∈ [ 𝑓 ⩽ Δ𝜖]𝑥 ∗ . In order to do so, we prove that 𝐵(𝑥(𝐾𝛼), 𝜖′ ) is a connected subset of [ 𝑓 ⩽ Δ𝜖] that has nonempty intersection with [ 𝑓 ⩽ Δ𝜖]𝑥 ∗ . Since [ 𝑓 ⩽ Δ𝜖]𝑥 ∗ is a connected component of [ 𝑓 ⩽ Δ𝜖], by maximality and (3.8) we then have 𝑥𝐾 ∈ 𝐵(𝑥(𝐾𝛼), 𝜖′ ) ⊂ [ 𝑓 ⩽ Δ𝜖]𝑥 ∗ . For all 𝑥˜ ∈ 𝐵(𝑥(𝐾𝛼), 𝜖′ ), we have 𝑓 (𝑥˜) − 𝑓 (𝑥 ∗ ) = 𝑓 (𝑥˜) − 𝑓 … view at source ↗
Figures from the paper (10 more)
Figure 3.4
Figure 3.4. Figure 3.4: Continuous and discrete subgradient trajectories in magenta and yellow respectively. [PITH_FULL_IMAGE:figures/full_fig_p027_3_4.png]
Figure 3
Figure 3. Figure 3: for a realization of the subgradient method. Note that [PITH_FULL_IMAGE:figures/full_fig_p027_3.png]
Figure 3.5
Figure 3.5. Figure 3.5: Subgradient method randomly initialized in the unit ball (100 trials with different step [PITH_FULL_IMAGE:figures/full_fig_p028_3_5.png]
Figure 3.6
Figure 3.6. Figure 3.6: Visualization of the critical points. Having chosen 𝛿 and 𝛼¯, let us fix some 𝛼 ∈ (0, 𝛼¯] from now on. Consider a sequence 𝑥0, 𝑥1, 𝑥2, . . . generated by the subgradient method with constant step size 𝛼 and initialized in [−𝛿, 𝛿]. Our goal is to show that all the ite…
Figure 4.1
Figure 4.1. Figure 4.1: The subgradient method with momentum, random reshuffling with momentum, and [PITH_FULL_IMAGE:figures/full_fig_p033_4_1.png]
Figure 5.1
Figure 5.1. Figure 5.1: Verdier stratification of the domain of two continuous semi-algebraic functions. [PITH_FULL_IMAGE:figures/full_fig_p042_5_1.png]
Figure 5.2
Figure 5.2. Figure 5.2: Illustration of 𝑑(𝑥𝑘+1, 𝑆) ⩾ 𝑑(𝑥𝑘 , 𝑆) for 𝑑(𝑥𝑘 , 𝑆) sufficiently small. Since (𝑥𝑘 )𝑘∈N is generated by the subgradient method with constant step size 𝛼, for all 𝑘 ∈ N there exists 𝑣𝑘 ∈ 𝜕 𝑓 (𝑥𝑘 ) such that 𝑥𝑘+1 = 𝑥𝑘 − 𝛼𝑣𝑘 . As illustrated in [PITH_FULL_IMAGE:figures…
Figure 5.3
Figure 5.3. Figure 5.3: Subgradient method randomly initialized near a spurious local minimum of a ReLU [PITH_FULL_IMAGE:figures/full_fig_p048_5_3.png]
Figure 5.4
Figure 5.4. Figure 5.4: Subgradient method randomly initialized near a spurious local minimum of robust [PITH_FULL_IMAGE:figures/full_fig_p050_5_4.png]
Figure 5.5
Figure 5.5. Figure 5.5: Output after 500 iterations of the subgradient method with step size [PITH_FULL_IMAGE:figures/full_fig_p050_5_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

127 extracted references · 70 canonical work pages

  1. [83]

    Global stability of first-order methods for coercive tame functions,

    C. Josz and L. Lai, “Global stability of first-order methods for coercive tame functions,” Mathematical Programming, pp. 1–26, 2023

    work page 2023

  2. [1]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 770–778

    2016

  3. [2]

    An image is worth 16x16 words: Transformers for image recogni- tion at scale,

    A. Dosovitskiy et al., “An image is worth 16x16 words: Transformers for image recogni- tion at scale,” arXiv preprint arXiv:2010.11929, 2020

    arXiv 2010

  4. [3]

    Attention is all you need,

    A. Vaswani et al., “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017

    2017

  5. [4]

    Bert: Pre-training of deep bidirec- tional transformers for language understanding,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirec- tional transformers for language understanding,” arXiv preprint arXiv:1810.04805, 2018

    arXiv 2018

  6. [5]

    Language models are few-shot learners,

    T. Brown et al., “Language models are few-shot learners,”Advances in neural information processing systems, vol. 33, pp. 1877–1901, 2020

    1901

  7. [6]

    High-resolution image synthesis with latent diffusion models,

    R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-resolution image synthesis with latent diffusion models,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 10 684–10 695

    2022

  8. [7]

    A stochastic approximation method,

    H. Robbins and S. Monro, “A stochastic approximation method,” The annals of mathemat- ical statistics, pp. 400–407, 1951

    1951

Show all 127 references
  1. [8]

    Adam: A method for stochastic optimization,

    D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014

    2014 arXiv

  2. [9]

    An overview of gradient descent optimization algorithms,

    S. Ruder, “An overview of gradient descent optimization algorithms,” arXiv preprint arXiv:1609.04747, 2016

    2016 arXiv

  3. [10]

    Large scale distributed deep networks,

    J. Dean et al., “Large scale distributed deep networks,” Advances in neural information processing systems, vol. 25, 2012

    2012

  4. [11]

    Problème général de la stabilité du mouvement,

    A. Liapounoff, “Problème général de la stabilité du mouvement,” in Annales de la Faculté des sciences de Toulouse: Mathématiques, vol. 9, 1907, pp. 203–474

    1907

  5. [12]

    Sastry, Nonlinear systems: analysis, stability, and control

    S. Sastry, Nonlinear systems: analysis, stability, and control. Springer Science & Business Media, 2013, vol. 10

    2013

  6. [13]

    van den Dries, Tame topology and o-minimal structures

    L. van den Dries, Tame topology and o-minimal structures . Cambridge university press, 1998, vol. 248

    1998

  7. [14]

    An invitation to tame optimization,

    A. D. Ioffe, “An invitation to tame optimization,” SIAM Journal on Optimization, vol. 19, no. 4, pp. 1894–1917, 2009

    1917

  8. [15]

    Subdifferentiability of real functions,

    X. Wang, “Subdifferentiability of real functions,” Real Analysis Exchange, vol. 30, no. 1, pp. 137–171, 2005

    2005

  9. [16]

    Friedman, T

    J. Friedman, T. Hastie, R. Tibshirani, et al., The elements of statistical learning . Springer series in statistics New York, 2001, vol. 1. 46

    2001

  10. [17]

    Generalized gradients and applications,

    F. H. Clarke, “Generalized gradients and applications,” Transactions of the American Math- ematical Society, 1975

    1975

  11. [18]

    F. H. Clarke, Optimization and Nonsmooth Analysis . SIAM Classics in Applied Mathe- matics, 1990

    1990

  12. [19]

    Aubin and A

    J.-P. Aubin and A. Cellina, Differential inclusions: set-valued maps and viability theory . Springer-Verlag, 1984, vol. 264

    1984

  13. [20]

    O. A. Nielsen, An introduction to integration and measure theory . Wiley-Interscience, 1997, vol. 17

    1997

  14. [21]

    Remarks on Tarski’s problem concerning ( R,+,∗, exp),

    L. van den Dries, “Remarks on Tarski’s problem concerning ( R,+,∗, exp),” in Studies in Logic and the Foundations of Mathematics, vol. 112, Elsevier, 1984, pp. 97–121

    1984

  15. [22]

    Definable sets in ordered structures. i,

    A. Pillay and C. Steinhorn, “Definable sets in ordered structures. i,” Transactions of the American Mathematical Society, vol. 295, no. 2, pp. 565–592, 1986

    1986

  16. [23]

    Bochnak, M

    J. Bochnak, M. Coste, and M.-F. Roy, Real algebraic geometry. Springer Science & Busi- ness Media, 2013, vol. 36

    2013

  17. [24]

    Geometric categories and o-minimal structures,

    L. van den Dries and C. Miller, “Geometric categories and o-minimal structures,” Duke Mathematical Journal, vol. 84, no. 2, pp. 497–540, 1996

    1996

  18. [25]

    Division d’une distribution par une fonction analytique de variables réelles,

    S. Łojasiewicz, “Division d’une distribution par une fonction analytique de variables réelles,” Comptes rendus hebdomadaires des séances de l’Académie des sciences. Paris , pp. 683– 686, 246 1958

    1958

  19. [26]

    Sur le problème de la division,

    S. Łojasiewicz, “Sur le problème de la division,” Studia Mathematica, 87–136, 1959

    1959

  20. [27]

    On the division of distributions by polynomials,

    L. Hörmander, “On the division of distributions by polynomials,” Arkiv för matematik , vol. 3, no. 6, pp. 555–568, 1958

    1958

  21. [28]

    On gradients of functions definable in o-minimal structures,

    K. Kurdyka, “On gradients of functions definable in o-minimal structures,” in Annales de l’institut Fourier, vol. 48, 1998, pp. 769–783

    1998

  22. [29]

    Clarke subgradients of stratifiable func- tions,

    J. Bolte, A. Daniilidis, A. Lewis, and M. Shiota, “Clarke subgradients of stratifiable func- tions,” SIAM Journal on Optimization, vol. 18, no. 2, pp. 556–572, 2007

    2007

  23. [30]

    Proximal alternating minimization and projection methods for nonconvex problems: An approach based on the Kurdyka- Łojasiewicz inequality,

    H. Attouch, J. Bolte, P. Redont, and A. Soubeyran, “Proximal alternating minimization and projection methods for nonconvex problems: An approach based on the Kurdyka- Łojasiewicz inequality,”Mathematics of operations research, vol. 35, pp. 438–457, 2010

    2010

  24. [31]

    Global convergence of the gradient method for functions definable in o-minimal structures,

    C. Josz, “Global convergence of the gradient method for functions definable in o-minimal structures,” Mathematical Programming, pp. 1–29, 2023

    2023

  25. [32]

    Curves of descent,

    D. Drusvyatskiy, A. D. Ioffe, and A. S. Lewis, “Curves of descent,” SIAM Journal on Control and Optimization, vol. 53, no. 1, pp. 114–138, 2015

    2015

  26. [33]

    Stochastic subgradient method converges on tame functions,

    D. Davis, D. Drusvyatskiy, S. Kakade, and J. D. Lee, “Stochastic subgradient method converges on tame functions,” Foundations of computational mathematics, vol. 20, no. 1, pp. 119–154, 2020

    2020

  27. [34]

    Optimization methods for large-scale machine learning,

    L. Bottou, F. E. Curtis, and J. Nocedal, “Optimization methods for large-scale machine learning,” Siam Review, vol. 60, no. 2, pp. 223–311, 2018. 47

    2018

  28. [35]

    Nonconvex Robust Low-Rank Matrix Recov- ery,

    X. Li, Z. Zhu, A. M.-C. So, and R. Vidal, “Nonconvex Robust Low-Rank Matrix Recov- ery,”SIAM Journal on Optimization, 2019

    2019

  29. [36]

    Global convergence of sub-gradient method for robust matrix re- covery: Small initialization, noisy measurements, and over-parameterization,

    J. Ma and S. Fattahi, “Global convergence of sub-gradient method for robust matrix re- covery: Small initialization, noisy measurements, and over-parameterization,” Journal of Machine Learning Research, vol. 24, pp. 1–84, 2023

    2023

  30. [37]

    Accelerating SGD for Highly Ill-Conditioned Huge-Scale Online Matrix Completion,

    G. Zhang, H.-M. Chiu, and R. Y . Zhang, “Accelerating SGD for Highly Ill-Conditioned Huge-Scale Online Matrix Completion,” Advances in Neural Information Processing Sys- tems, vol. 35, 2022

    2022

  31. [38]

    Deep learning,

    Y . LeCun, Y . Bengio, and G. Hinton, “Deep learning,”Nature, vol. 521, no. 7553, pp. 436– 444, 2015

    2015

  32. [39]

    Sur l’Equation à l’Aide de Laquelle on Détermine les Inégalités Sécu- laires,

    A. L. Cauchy, “Sur l’Equation à l’Aide de Laquelle on Détermine les Inégalités Sécu- laires,” Oeuvres Complètes, pp. 174–195, 1829

  33. [40]

    Continuous time analysis of momentum methods,

    N. B. Kovachki and A. M. Stuart, “Continuous time analysis of momentum methods,” Journal of Machine Learning Research, vol. 22, no. 17, pp. 1–40, 2021

    2021

  34. [41]

    Some methods of speeding up the convergence of iteration methods,

    B. T. Polyak, “Some methods of speeding up the convergence of iteration methods,” USSR Computational Mathematics and Mathematical Physics, vol. 4, no. 5, pp. 1–17, 1964

    1964

  35. [42]

    Nesterov, Lectures on Convex Optimization

    Y . Nesterov, Lectures on Convex Optimization. Springer Science & Business Media, 2018, vol. 137

    2018

  36. [43]

    Shor, “Application of the gradient method for the solution of network transportation problems

    N. Shor, “Application of the gradient method for the solution of network transportation problems. fjotes, scientific seminar on theory and applications of cybernetics and opera- tions research,” Kiev: Academy of Sciences USSR, 1962

    1962

  37. [44]

    On the structure of algorithms for numerical solution of problems of optimal planning and design,

    N. Shor, “On the structure of algorithms for numerical solution of problems of optimal planning and design,” Diss. Doctor Philos. Kiev, 1964

    1964

  38. [45]

    Bertsekas, Convex optimization algorithms

    D. Bertsekas, Convex optimization algorithms. Athena Scientific, 2015

    2015

  39. [46]

    A method for solving the convex programming problem with convergence rate𝑂(1/𝑘2),

    Y . E. Nesterov, “A method for solving the convex programming problem with convergence rate𝑂(1/𝑘2),” in Dokl. akad. nauk Sssr, vol. 269, 1983, pp. 543–547

    1983

  40. [47]

    Heavy-ball method in nonconvex optimization problems,

    S. Zavriev and F. Kostyuk, “Heavy-ball method in nonconvex optimization problems,” Computational Mathematics and Modeling, vol. 4, no. 4, pp. 336–341, 1993

    1993

  41. [48]

    iPiano: Inertial proximal algorithm for nonconvex optimization,

    P. Ochs, Y . Chen, T. Brox, and T. Pock, “iPiano: Inertial proximal algorithm for nonconvex optimization,” SIAM Journal on Imaging Sciences, vol. 7, no. 2, pp. 1388–1419, 2014

    2014

  42. [49]

    Adaptive switching circuits,

    B. Widrow and M. E. Hoff, “Adaptive switching circuits,” Stanford Univ Ca Stanford Elec- tronics Labs, Tech. Rep., 1960

    1960

  43. [50]

    An adaptive associative memory principle,

    T. Kohonen, “An adaptive associative memory principle,” IEEE Transactions on Comput- ers, vol. 100, no. 4, pp. 444–445, 1974

    1974

  44. [51]

    On the convergence of the lms algorithm with adaptive learning rate for linear feedforward networks,

    Z.-Q. Luo, “On the convergence of the lms algorithm with adaptive learning rate for linear feedforward networks,” Neural Computation, vol. 3, no. 2, pp. 226–245, 1991

    1991

  45. [52]

    Incremental subgradient methods for nondifferentiable op- timization,

    A. Nedic and D. P. Bertsekas, “Incremental subgradient methods for nondifferentiable op- timization,” SIAM Journal on Optimization, vol. 12, no. 1, pp. 109–138, 2001. 48

    2001

  46. [53]

    Incremental gradient, subgradient, and proximal methods for convex optimization: A survey,

    D. P. Bertsekas et al., “Incremental gradient, subgradient, and proximal methods for convex optimization: A survey,” Optimization for Machine Learning , vol. 2010, no. 1-38, p. 3, 2011

    2010

  47. [54]

    Why random reshuffling beats stochas- tic gradient descent,

    M. Gürbüzbalaban, A. Ozdaglar, and P. A. Parrilo, “Why random reshuffling beats stochas- tic gradient descent,” Mathematical Programming, vol. 186, no. 1, pp. 49–84, 2021

    2021

  48. [55]

    Random reshuffling: Simple analysis with vast improvements,

    K. Mishchenko, A. Khaled, and P. Richtárik, “Random reshuffling: Simple analysis with vast improvements,”Advances in Neural Information Processing Systems, vol. 33, pp. 17 309– 17 320, 2020

    2020

  49. [56]

    A unified convergence analysis for shuffling-type gradient methods,

    L. M. Nguyen, Q. Tran-Dinh, D. T. Phan, P. H. Nguyen, and M. van Dijk, “A unified convergence analysis for shuffling-type gradient methods,” Journal of Machine Learning Research, vol. 22, no. 207, pp. 1–44, 2021

    2021

  50. [57]

    Incremental without replacement sampling in nonconvex optimization,

    E. Pauwels, “Incremental without replacement sampling in nonconvex optimization,” Jour- nal of Optimization Theory and Applications, pp. 1–26, 2021

    2021

  51. [58]

    Convergence of random reshuffling under the kurdyka– łojasiewicz inequality,

    X. Li, A. Milzarek, and J. Qiu, “Convergence of random reshuffling under the kurdyka– łojasiewicz inequality,” SIAM Journal on Optimization , vol. 33, no. 2, pp. 1092–1120, 2023

    2023

  52. [59]

    On the importance of initialization and momentum in deep learning,

    I. Sutskever, J. Martens, G. Dahl, and G. Hinton, “On the importance of initialization and momentum in deep learning,” in International conference on machine learning , PMLR, 2013, pp. 1139–1147

    2013

  53. [60]

    Smg: A shuffling gradient-based method with momentum,

    T. H. Tran, L. M. Nguyen, and Q. Tran-Dinh, “Smg: A shuffling gradient-based method with momentum,” inInternational Conference on Machine Learning, PMLR, 2021, pp. 10 379– 10 389

    2021

  54. [61]

    Nesterov accelerated shuffling gradi- ent method for convex optimization,

    T. H. Tran, K. Scheinberg, and L. M. Nguyen, “Nesterov accelerated shuffling gradi- ent method for convex optimization,” in International Conference on Machine Learning , PMLR, 2022, pp. 21 703–21 732

    2022

  55. [62]

    Coordinate descent algorithms,

    S. J. Wright, “Coordinate descent algorithms,” Mathematical Programming, vol. 151, no. 1, pp. 3–34, 2015

    2015

  56. [63]

    J. M. Ortega and W. C. Rheinboldt, Iterative solution of nonlinear equations in several variables. Academic Press, New York, 1970

    1970

  57. [64]

    On the convergence of the coordinate descent method for convex differentiable minimization,

    Z.-Q. Luo and P. Tseng, “On the convergence of the coordinate descent method for convex differentiable minimization,” Journal of Optimization Theory and Applications , vol. 72, no. 1, pp. 7–35, 1992

    1992

  58. [65]

    Efficiency of coordinate descent methods on huge-scale optimization prob- lems,

    Y . Nesterov, “Efficiency of coordinate descent methods on huge-scale optimization prob- lems,” SIAM Journal on Optimization, vol. 22, no. 2, pp. 341–362, 2012

    2012

  59. [66]

    Randomness and per- mutations in coordinate descent methods,

    M. Gürbüzbalaban, A. Ozdaglar, N. D. Vanli, and S. J. Wright, “Randomness and per- mutations in coordinate descent methods,” Mathematical Programming, vol. 181, no. 2, pp. 349–376, 2020

    2020

  60. [67]

    On the convergence of block coordinate descent type meth- ods,

    A. Beck and L. Tetruashvili, “On the convergence of block coordinate descent type meth- ods,” SIAM journal on Optimization, vol. 23, no. 4, pp. 2037–2060, 2013. 49

    2013

  61. [68]

    Random permutations fix a worst case for cyclic coordinate descent,

    C.-P. Lee and S. J. Wright, “Random permutations fix a worst case for cyclic coordinate descent,” IMA Journal of Numerical Analysis, vol. 39, no. 3, pp. 1246–1275, 2019

    2019

  62. [69]

    Euler, Institutiones calculi integralis

    L. Euler, Institutiones calculi integralis. impensis Academiae imperialis scientiarum, 1792, vol. 1

  63. [70]

    Blanton, Foundations of Differential Calculus

    J. Blanton, Foundations of Differential Calculus . Springer Science & Business Media, 2006

    2006

  64. [71]

    E. A. Coddington and N. Levinson, Theory of ordinary differential equations. Tata McGraw- Hill Education, 1955

    1955

  65. [72]

    Analysis of recursive stochastic algorithms,

    L. Ljung, “Analysis of recursive stochastic algorithms,” IEEE transactions on automatic control, vol. 22, no. 4, pp. 551–575, 1977

    1977

  66. [73]

    General convergence results for stochastic approximations via weak con- vergence theory,

    H. J. Kushner, “General convergence results for stochastic approximations via weak con- vergence theory,”Journal of mathematical analysis and applications, vol. 61, no. 2, pp. 490– 503, 1977

    1977

  67. [74]

    Convergence of recursive adaptive and identification procedures via weak convergence theory,

    H Kushner, “Convergence of recursive adaptive and identification procedures via weak convergence theory,” IEEE Transactions on Automatic Control , vol. 22, no. 6, pp. 921– 930, 1977

    1977

  68. [75]

    Stochastic approximations and differential inclu- sions,

    M. Benaïm, J. Hofbauer, and S. Sorin, “Stochastic approximations and differential inclu- sions,” SIAM Journal on Control and Optimization, vol. 44, no. 1, pp. 328–348, 2005

    2005

  69. [76]

    Stochastic approximations and differential inclu- sions, part ii: Applications,

    M. Benaïm, J. Hofbauer, and S. Sorin, “Stochastic approximations and differential inclu- sions, part ii: Applications,” Mathematics of Operations Research, vol. 31, no. 4, pp. 673– 695, 2006

    2006

  70. [77]

    Dynamics of stochastic approximation algorithms,

    M. Benaïm, “Dynamics of stochastic approximation algorithms,” in Seminaire de proba- bilites XXXIII, Springer, 2006, pp. 1–68

    2006

  71. [78]

    V . S. Borkar, Stochastic approximation: a dynamical systems viewpoint . Springer, 2009, vol. 48

    2009

  72. [79]

    Stochastic methods for composite and weakly convex optimiza- tion problems,

    J. C. Duchi and F. Ruan, “Stochastic methods for composite and weakly convex optimiza- tion problems,” SIAM Journal on Optimization, vol. 28, no. 4, pp. 3229–3259, 2018

    2018

  73. [80]

    Conservative set valued fields, automatic differentiation, stochas- tic gradient methods and deep learning,

    J. Bolte and E. Pauwels, “Conservative set valued fields, automatic differentiation, stochas- tic gradient methods and deep learning,” Mathematical Programming, pp. 1–33, 2020

    2020

  74. [81]

    Random monotone operators and application to stochastic optimization,

    A. Salim, “Random monotone operators and application to stochastic optimization,” Ph.D. dissertation, Université Paris-Saclay (ComUE), 2018

    2018

  75. [82]

    Adam-family methods for nonsmooth optimiza- tion with convergence guarantees,

    N. Xiao, X. Hu, X. Liu, and K.-C. Toh, “Adam-family methods for nonsmooth optimiza- tion with convergence guarantees,”Journal of Machine Learning Research, vol. 25, no. 48, pp. 1–53, 2024

    2024

  76. [84]

    Converging multistep methods for initial value problems involving multival- ued maps,

    K. Taubert, “Converging multistep methods for initial value problems involving multival- ued maps,” Computing, vol. 27, no. 2, pp. 123–136, 1981. 50

    1981

  77. [85]

    A. F. Filippov, Differential equations with discontinuous righthand sides: control systems. Springer Science & Business Media, 2013, vol. 18

    2013

  78. [86]

    Difference methods for differential inclusions: A survey,

    A. Dontchev and F. Lempio, “Difference methods for differential inclusions: A survey,” SIAM review, vol. 34, no. 2, pp. 263–294, 1992

    1992

  79. [87]

    Subgradient methods for sharp weakly convex functions,

    D. Davis, D. Drusvyatskiy, K. J. MacPhee, and C. Paquette, “Subgradient methods for sharp weakly convex functions,”Journal of Optimization Theory and Applications, vol. 179, no. 3, pp. 962–982, 2018

    2018

  80. [88]

    Stochastic algorithms with geometric step decay converge linearly on sharp functions,

    D. Davis, D. Drusvyatskiy, and V . Charisopoulos, “Stochastic algorithms with geometric step decay converge linearly on sharp functions,”arXiv preprint arXiv:1907.09547, 2019

    1907 arXiv

  81. [89]

    Low-rank matrix recovery with composite optimization: Good conditioning and rapid convergence,

    V . Charisopoulos, Y . Chen, D. Davis, M. Díaz, L. Ding, and D. Drusvyatskiy, “Low-rank matrix recovery with composite optimization: Good conditioning and rapid convergence,” Foundations of Computational Mathematics, pp. 1–89, 2021

    2021

  82. [90]

    On the stable equilibrium points of gradient systems,

    P.-A. Absil and K. Kurdyka, “On the stable equilibrium points of gradient systems,” Sys- tems & control letters, vol. 55, no. 7, pp. 573–577, 2006

    2006

  83. [91]

    Favorable classes of lipschitz continuous functions in subgradient opti- mization,

    R. T. Rockafellar, “Favorable classes of lipschitz continuous functions in subgradient opti- mization,” WP-81-001, 1981

    1981

  84. [92]

    Generic differentiability of lipschitzian functions,

    G. Lebourg, “Generic differentiability of lipschitzian functions,” Transactions of the Amer- ican Mathematical Society, vol. 256, pp. 125–144, 1979

    1979

  85. [93]

    Convergence of the iterates of descent methods for analytic cost functions,

    P.-A. Absil, R. Mahony, and B. Andrews, “Convergence of the iterates of descent methods for analytic cost functions,” SIAM Journal on Optimization , vol. 16, no. 2, pp. 531–547, 2005

    2005

  86. [94]

    Abadi et al

    M. Abadi et al. , TensorFlow: Large-scale machine learning on heterogeneous systems , Software available from tensorflow.org, 2015

    2015

  87. [95]

    Pytorch: An imperative style, high-performance deep learning library,

    A. Paszke et al., “Pytorch: An imperative style, high-performance deep learning library,” Advances in neural information processing systems, vol. 32, 2019

    2019

  88. [96]

    A simple weight decay can improve generalization,

    A. Krogh and J. Hertz, “A simple weight decay can improve generalization,” Advances in neural information processing systems, vol. 4, 1991

    1991

  89. [97]

    Regression shrinkage and selection via the lasso,

    R. Tibshirani, “Regression shrinkage and selection via the lasso,” Journal of the Royal Statistical Society Series B: Statistical Methodology, vol. 58, no. 1, pp. 267–288, 1996

    1996

  90. [98]

    Matrix Completion has No Spurious Local Minimum,

    R. Ge, J. D. Lee, and T. Ma, “Matrix Completion has No Spurious Local Minimum,” NIPS, 2016

    2016

  91. [99]

    Beck, First-order methods in optimization

    A. Beck, First-order methods in optimization. SIAM, 2017

    2017

  92. [100]

    N. G. Chetaev, The stability of motion. Pergamon Press, 1961

    1961

  93. [101]

    Holder metric subregularity with applications to proximal point method,

    G. Li and B. S. Mordukhovich, “Holder metric subregularity with applications to proximal point method,” SIAM Journal on Optimization, vol. 22, no. 4, pp. 1655–1684, 2012

    2012

  94. [102]

    Higher-order metric subregularity and its applica- tions,

    B. S. Mordukhovich and W. Ouyang, “Higher-order metric subregularity and its applica- tions,” Journal of Global Optimization, vol. 63, no. 4, pp. 777–795, 2015. 51

    2015

  95. [103]

    Hölder stable minimizers, tilt stability, and Hölder metric regularity of subdifferentials,

    X. Y . Zheng and K. F. Ng, “Hölder stable minimizers, tilt stability, and Hölder metric regularity of subdifferentials,”SIAM Journal on Optimization, vol. 25, no. 1, pp. 416–438, 2015

    2015

  96. [104]

    Stratifications de Whitney et théoreme de Bertini-Sard,

    J.-L. Verdier, “Stratifications de Whitney et théoreme de Bertini-Sard,” Inventiones math- ematicae, vol. 36, no. 1, pp. 295–312, 1976

    1976

  97. [105]

    Stochastic subgradient descent escapes active strict saddles on weakly convex functions,

    P. Bianchi, W. Hachem, and S. Schechtman, “Stochastic subgradient descent escapes active strict saddles on weakly convex functions,”Mathematics of Operations Research, 2023

    2023

  98. [106]

    Active manifolds, stratifications, and conver- gence to local minima in nonsmooth optimization,

    D. Davis, D. Drusvyatskiy, and L. Jiang, “Active manifolds, stratifications, and conver- gence to local minima in nonsmooth optimization,” arXiv preprint arXiv:2108.11832v2 , 2023

    2023 arXiv

  99. [107]

    Nonconvergence to unstable points in urn models and stochastic approxima- tions,

    R. Pemantle, “Nonconvergence to unstable points in urn models and stochastic approxima- tions,” The Annals of Probability, vol. 18, no. 2, pp. 698–712, 1990

    1990

  100. [108]

    Gradient Descent Only Converges to Minimizers,

    J. D. Lee, M. Simchowitz, M. I. Jordan, and B. Recht, “Gradient Descent Only Converges to Minimizers,” COLT, 2016

    2016

  101. [109]

    Gradient Descent Only Converges to Minimizers: Non- Isolated Critical Points and Invariant Regions,

    I. Panageas and G. Piliouras, “Gradient Descent Only Converges to Minimizers: Non- Isolated Critical Points and Invariant Regions,”ITCS, 2017

    2017

  102. [110]

    An alternative view: When does sgd escape local min- ima?

    B. Kleinberg, Y . Li, and Y . Yuan, “An alternative view: When does sgd escape local min- ima?” In International Conference on Machine Learning, PMLR, 2018, pp. 2698–2707

    2018

  103. [111]

    On large-batch training for deep learning: Generalization gap and sharp minima,

    N. S. Keskar, D. Mudigere, J. Nocedal, M. Smelyanskiy, and P. T. P. Tang, “On large-batch training for deep learning: Generalization gap and sharp minima,” ICRL, 2017

    2017

  104. [112]

    R. T. Rockafellar and R. J.-B. Wets, Variational analysis. Springer Science & Business Media, 2009, vol. 317

    2009

  105. [113]

    Metric subregularity of multifunctions: First and second order infinitesimal characterizations,

    H. Van Ngai and P. N. Tinh, “Metric subregularity of multifunctions: First and second order infinitesimal characterizations,” Mathematics of Operations Research, pp. 703–724, 2015

    2015

  106. [114]

    Characterization of metric regularity of subdifferen- tials,

    F. A. Artacho and M. H. Geoffroy, “Characterization of metric regularity of subdifferen- tials,” Journal of Convex Analysis, vol. 15, no. 2, p. 365, 2008

    2008

  107. [115]

    Regularity and conditioning of solution mappings in variational analysis,

    A. L. Dontchev and R. T. Rockafellar, “Regularity and conditioning of solution mappings in variational analysis,” Set-Valued Analysis, vol. 12, no. 1, pp. 79–109, 2004

    2004

  108. [116]

    Boumal, An introduction to optimization on smooth manifolds

    N. Boumal, An introduction to optimization on smooth manifolds . Cambridge University Press, 2023

    2023

  109. [117]

    Tangents to an analytic variety,

    H. Whitney, “Tangents to an analytic variety,” Annals of Mathematics , vol. 81, pp. 496– 549, 3 1965

    1965

  110. [118]

    Verdier and strict Thom stratifications in o-minimal structures,

    T. Lê Loi, “Verdier and strict Thom stratifications in o-minimal structures,” Illinois Journal of Mathematics, vol. 42, no. 2, pp. 347–356, 1998

    1998

  111. [119]

    A mathematical model for automatic differentiation in machine learning,

    J. Bolte and E. Pauwels, “A mathematical model for automatic differentiation in machine learning,” Advances in Neural Information Processing Systems, vol. 33, pp. 10 809–10 819, 2020

    2020

  112. [120]

    Smooth manifolds,

    J. M. Lee, “Smooth manifolds,” in Introduction to Smooth Manifolds , Springer, 2013, pp. 1–31. 52

    2013

  113. [121]

    Local differentiability of distance functions,

    R. Poliquin, R Rockafellar, and L. Thibault, “Local differentiability of distance functions,” Transactions of the American mathematical Society, vol. 352, no. 11, pp. 5231–5249, 2000

    2000

  114. [122]

    Braun, L

    P. Braun, L. Grüne, and C. M. Kellett, (In-)Stability of Differential Inclusions: Notions, Equivalences, and Lyapunov-like Characterizations. Springer Nature, 2021

    2021

  115. [123]

    Uniting control laws: On obstacle avoidance and global stabilization of underactuated linear systems,

    P. Braun, C. M. Kellett, and L. Zaccarian, “Uniting control laws: On obstacle avoidance and global stabilization of underactuated linear systems,” in 2019 IEEE 58th Conference on Decision and Control (CDC), IEEE, 2019, pp. 8154–8159

    2019

  116. [124]

    Complete instability of differential inclusions using Lyapunov methods,

    P. Braun, L. Grüne, and C. M. Kellett, “Complete instability of differential inclusions using Lyapunov methods,” in 2018 IEEE Conference on Decision and Control (CDC) , IEEE, 2018, pp. 718–724

    2018

  117. [125]

    On the complexity of robust PCA and l1-norm low-rank matrix approximation,

    N. Gillis and S. A. Vavasis, “On the complexity of robust PCA and l1-norm low-rank matrix approximation,” Mathematics of Operations Research , vol. 43, no. 4, pp. 1072– 1084, 2018

    2018

  118. [126]

    Non-convex Robust PCA,

    P. Netrapalli, N. U. N, S. Sanghavi, A. Anandkumar, and P. Jain, “Non-convex Robust PCA,” NeurIPS, 2014

    2014

  119. [127]

    Robust principal component analysis?

    E. J. Candès, X. Li, Y . Ma, and J. Wright, “Robust principal component analysis?” Journal of the ACM (JACM), vol. 58, no. 3, pp. 1–37, 2011. 53

    2011

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.