Pith. sign in

REVIEW 4 major objections 5 minor 44 references

Single-loop $\mathcal{O}(\epsilon^{-3})$ stochastic smoothing algorithms for nonsmooth Riemannian optimization

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A single-loop stochastic smoothing algorithm reaches an $\varepsilon$-stationary point of a nonsmooth Riemannian problem in $O(\varepsilon^{-3})$ iterations for Lipschitz nonsmooth terms, and in…

desk verdict Plausible and well-structured attempt at an optimal single-loop rate for nonsmooth Riemannian problems, but the central Lemma 4.4 currently rests on an unstated vector-transport isometry and a squared-smoothness condition that is not part of the assumptions. read the letter →

arxiv 2505.09485 v2 pith:TCOMGYXV submitted 2025-05-14 math.OC

Kangkang Deng , Zheng Peng , Weihe Wu This is my paper · ORCID
classification math.OC MSC 65K0565K1090C0590C2690C30
keywords nonsmoothRiemannianoptimizationstochasticsmoothingMoreauenveloperecursivemomentumvariancereductionsingle-loopalgorithmiterationcomplexityerrorboundcondition
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper establishes worst-case iteration guarantees for solving nonsmooth stochastic optimization problems on compact Riemannian manifolds, where the objective is an expected smooth term plus a nonsmooth term composed with a nonlinear map. Its first algorithm targets Lipschitz nonsmooth terms and proves the optimal $O(\epsilon^{-3})$ iteration complexity to reach an $\epsilon$-stationary point, improving on earlier stochastic Riemannian rates of $O(\epsilon^{-5})$ and $\tilde{O}(\epsilon^{-3.5})$. Its second algorithm handles constraints encoded as an indicator function of a convex set and, under an error-bound condition with parameter $\theta\ge 1$, reaches $\tilde{O}(\epsilon^{-\max\{\theta+2,2\theta\}})$, matching the best-known Euclidean results. Both methods are single-loop, use only $O(1)$ samples per iteration, and dynamically shrink the smoothing parameter, so iteration complexity and sample complexity coincide. A sympathetic reader would care because these rates are the first to match deterministic benchmarks in the stochastic Riemannian nonsmooth setting without nested loops or large batches.

What carries the argument

The load-bearing mechanism is the interaction of three ingredients. First, the Moreau envelope $h_\mu$ with proximal map $\mathrm{prox}_{\mu h}$ turns the nonsmooth term into a smooth surrogate whose gradient is $(x-\mathrm{prox}_{\mu h}(x))/\mu$, with the smoothing parameter $\mu_k$ decreased over time. Second, a recursive momentum estimator $\delta_k=\mathrm{grad}\,\tilde f(x_k,\xi_k)+(1-a_k)T_{x_k}^{x_{k-1}}(\delta_{k-1}-\mathrm{grad}\,\tilde f(x_{k-1},\xi_k))$ transports previous gradient information across tangent spaces to reduce variance without inner loops or large batches. Third, the adaptive step $\tau_k=1/(\sum_{i=1}^k\|G_i\|^2/a_{k+1})^{1/3}$ in Algorithm 3.1 couples step size to accumulated gradient norms; Algorithm 3.2 instead uses a truncated momentum estimator projected onto a ball of radius $L_f$ plus polynomially decaying step and smoothing sequences. Retraction smoothness, established in Lemma 4.3 for both cases, is what lets the algorithms take a single retraction step per iteration and still control function descent.

What would settle it

Give a compact submanifold and a retraction/vector-transport pair satisfying Definition 2.3 but not preserving tangent-vector norms, and compute the left and right sides of inequality (4.7) on a one-step example; if the inequality fails, the estimator-error analysis needs an additional isometry assumption. Alternatively, implement Algorithm 3.1 on a sphere or Stiefel manifold with a factory vector transport, measure $\|T_x^y v\|/\|v\|$, and test whether the empirical error recursions obey the claimed bounds.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central discovery is that dynamic Moreau smoothing can be combined with a recursive-momentum variance-reduction estimator to make stochastic Riemannian smoothing algorithms match the best deterministic complexity in the Lipschitz case and the best Euclidean complexity in the constrained case. The stationarity measure is a KKT-type condition: a point $x$ is $\epsilon$-stationary if there is $y$ and $z\in\partial h(y)$ with $\mathbb{E}\|P_{T_xM}(\nabla f(x)+\nabla c(x)^\top z)\|\le\epsilon$ and $\|c(x)-y\|\le\epsilon$. For Lipschitz $h$, Algorithm 3.1 with smoothing parameter $\mu_k=k^{-1/3}$, momentum $a_{k+1}=k^{-2/3}$, and adaptive step $\tau_k$ achieves the $O(\epsilon^{-3})$ rate. For $h=\delta_C$, Algorithm 3.2 with a truncated momentum estimator achieves $\tilde{O}(\epsilon^{-\max\{\theta+2,2\theta\}})$ under the geometric error-bound assumption $\|\mathrm{grad}\,g(x)\|\ge\zeta\,\mathrm{dist}^\theta(c(x),C)$. A direct corollary is that in both cases the sample complexity equals the iteration complexity because each iteration draws one fresh sample and performs a constant number of gradient evaluations.

Load-bearing premise

The rate proof assumes the vector transport that moves gradients between tangent spaces preserves norms exactly, although the formal definition only requires linearity and the identity property; under a transport that shrinks or stretches lengths, the momentum-error recursion no longer telescopes and the $O(\epsilon^{-3})$ rate loses its support.

Editorial extensions

If this is right

  • If the central claim is right, stochastic nonsmooth Riemannian optimization with Lipschitz composite terms needs no more than $O(\epsilon^{-3})$ iterations, matching the deterministic Riemannian bound and closing the gap left by the $O(\epsilon^{-5})$ and $\tilde{O}(\epsilon^{-3.5})$ stochastic methods.
  • The single-loop, one-sample design means the method is practical for online settings: sample complexity and iteration complexity coincide, with no nested subroutines and no periodic restart schedule.
  • For constraints written as $c(x)\in C$, the $\tilde{O}(\epsilon^{-\max\{\theta+2,2\theta\}})$ rate under the error-bound condition transfers the Euclidean variance-reduced constrained-optimization rate to compact submanifolds.
  • Setting $h\equiv 0$ recovers the smooth stochastic Riemannian setting, and the analysis removes a logarithmic factor relative to earlier Riemannian recursive-momentum methods.
  • The stationarity definition requires an exact subgradient $z\in\partial h(y)$ and feasibility gap $\|c(x)-y\|\le\epsilon$, so an $\epsilon$-stationary point in this paper is a stronger output guarantee than relaxed KKT notions used by prior augmented Lagrangian analyses.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the proof's use of the isometry property of vector transport at Eq. (4.9) is not entailed by Definition 2.3, so Algorithm 3.1's advertised rate rests on an implicit assumption that transported vectors keep their norms; a counterexample transport with bounded distortion would test whether the recursion still holds.
  • Editorial inference: the parameter-free adaptive step and the Moreau smoothing schedule suggest a template for other nonsmooth composite structures, such as weakly convex $h$ with H\"older-type growth, where the achievable rate would be governed by the growth exponent in place of $1/2$.
  • Editorial inference: the truncated momentum estimator in Algorithm 3.2, which projects onto a ball of radius $L_f$, indicates that boundedness of the estimator is the key replacement for convexity of the constraint, so a similar truncation could extend the rate to nonconvex constraint sets satisfying the same geometric error bound.
  • Editorial note: the paper reports no numerical experiments and defers them to future work, so the practical behavior of the adaptive step and truncation on, say, sparse PCA or robust matrix completion remains untested; a small-scale experiment comparing Algorithm 3.1 with the earlier $\tilde{O}(\epsilon^{-3.5})$ method would be the natural next check.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes two single-loop stochastic smoothing algorithms for the composite nonsmooth Riemannian problem (1.1). For Lipschitz h, Algorithm 3.1 uses a STORM+-style recursive momentum estimator for grad f and the Moreau envelope for h, with an adaptive step size τ_k; the paper claims an optimal O(ε^{-3}) iteration complexity at an ε-stationary point in the sense of Definition 2.9. For h equal to the indicator of a convex set, Algorithm 3.2 uses a truncated momentum estimator and, under Assumption 4 with parameter θ≥1, claims a complexity of O~(ε^{-max{θ+2,2θ}}). The analysis relies on retraction smoothness of the smoothed problem (Lemma 4.3) and on variance bounds for the momentum estimator (Lemmas 4.4–4.5 and 4.8), together with an error-bound induction for the constraint-violation measure in the constrained case.

Significance. If the central proofs are made rigorous, the results would be a meaningful advance: an optimal, fully single-loop, O(1)-samples-per-iteration rate for nonsmooth stochastic Riemannian optimization, extending Euclidean STORM+ results to compact submanifolds and improving on the O(ε^{-5}) and O(ε^{-3.5}) bounds of [33] and [14]. The unified treatment of two nonsmooth regimes, the explicit stationary-point definitions, and the comparison with previous oracle bounds are useful contributions. However, as written the paper is not self-contained: three load-bearing gaps (isometry of the vector transport, a missing second-moment smoothness condition, and undefined initialization of the momentum estimator) currently prevent verification of the stated rates.

major comments (4)
  1. [Lemma 4.4 / Eq. (4.9)] The first equality in (4.9) is justified by 'the isometry property of vector transport', but Definition 2.3 and Assumption 1 only postulate linearity and the zero-transport identity, not norm preservation. Without an isometry condition (or a uniform bound on ||T_{x_k}^{x_{k-1}}(v)||/||v||), the conditional variance recursion carries an extra multiplicative constant in front of (1-a_k)^2 E||ε_{k-1}||^2; Lemma 4.5's telescoping in (4.12) then fails, and the two-case argument in Theorem 3.1 no longer yields the O(ε^{-3}) bound. The proof requires either an explicit isometric vector transport assumption or an additional uniform norm-bound assumption on T.
  2. [Assumption 1.D / Eq. (4.9)] The last inequality in (4.9) uses E||grad \tilde f(x_k, ξ_k) - T_{x_k}^{x_{k-1}} grad \tilde f(x_{k-1}, ξ_k)||^2 ≤ \tilde L^2 τ_{k-1}^2 ||G_{k-1}||^2, but Assumption 1.D only bounds the first moment E||·|| ≤ \tilde L||ζ||. A first-moment bound does not imply the corresponding second-moment bound without additional assumptions, such as a squared mean-smoothness condition or a uniform bound on the stochastic gradients. Since this step is used immediately in the base variance bound (4.7), the rate statement in Theorem 3.1 depends on an unstated strengthening of Assumption 1.D.
  3. [Algorithm 3.1 / Algorithm 3.2] Both algorithms are formally incomplete at k=1: the momentum estimator δ_1 is never computed or initialized, yet step 2 of Algorithm 3.1 uses G_1 = δ_1 + P(∇c^T ∇h_{μ1}) before any rule producing δ_1; the same holds for δ_1 in Algorithm 3.2. The convergence proofs further use ε_0 = δ_0 - grad f(x_0) via Lemma 4.4, but δ_0 is undefined. The statement needs an explicit initialization (for example, sample ξ_1 and set δ_1 = grad \tilde f(x_1, ξ_1), or set δ_1 = grad \tilde f(x_0, ξ_1)) and a corresponding base-case treatment of the error recursion.
  4. [Theorem 3.4 / proof of Theorem 3.4] The theorem's displayed Lyapunov function Φ_k = E[F_{μ_k}(x_k)] + ||δ_k - grad f(x_k)||/(16τ_0 \tilde L^2) is inconsistent with the proof, which uses Φ_k = E[F_{μ_k}(x_k)] + C E||δ_k - grad f(x_k)||^2/τ_{k-1} with C = 1/(16\tilde L^2). The theorem statement also references τ_0, which is never defined, and omits the square on the norm. Since the telescoping argument in (4.42)–(4.46) uses the proof's version, the statement of Theorem 3.4 as written is not well defined.
minor comments (5)
  1. [Abstract and Section 1] The phrases 'an stochastic algorithm' and 'for exist algorithms' are typographical errors; the latter should presumably read 'for existing algorithms'.
  2. [Section 1.2, Notations] The line 'We denote l(t) = O(l(t))' appears to be a typo; the intended statement is presumably l(t) = O(\bar l(t)) or a similar asymptotic comparison between two different functions.
  3. [Corollary 3.2, Eq. (3.10)] The constant 863√3 in (3.10) conflicts with the 864√3 in (3.6) and with the doubling of (4.24); the displayed constants should be reconciled.
  4. [Theorem 3.1 and Lemma 4.5] The indexing of the momentum parameters is not stated consistently: Theorem 3.1 sets a_1=1 and a_{k+1}=k^{-2/3} for k≥1, while Lemma 4.5's summations use a_k and a_{k-1} without defining a_0; the base-case index should be spelled out.
  5. [Remark 2] Calling Algorithm 3.1 'parameter-free' is confusing in view of the explicit schedules for a_k and μ_k; the intended meaning (no manually tuned step-size sequence) should be clarified in the text.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: theorem rates are derived from explicit assumptions; self-citations are baselines only.

full rationale

The derivation is self-contained: the central complexity bounds in Theorem 3.1 and Theorem 3.4 are proven from the stated Assumptions 1-4 using standard external lemmas (e.g., Propositions 2.5 and 2.8 from [6] and [8], Lemma 4.1 from [28]), not imported from prior work by the same authors. The momentum estimators (3.5) and (3.16) are defined in the paper, and their error bounds are derived in Lemmas 4.4 and 4.8 rather than assumed. The adaptive step-size (3.4) is analyzed through Lemmas 4.1 and 4.5. The O(epsilon^{-3}) and O-tilde(epsilon^{-max{theta+2,2theta}}) rates follow by telescoping, Jensen's inequality, and the two-case argument in the proof of Theorem 3.1. The self-citations [14], [15], [16], and [33] are used for comparison baselines, KKT terminology, or attribution of the smoothing framework alongside the external work [4]; none of these citations supplies the target rate or the constrained-case conclusion. No parameter is fitted to data, and no prediction is a renamed input. The proof-support issue concerning an isometric vector transport in Lemma 4.4, where Definition 2.3 only postulates linearity and the identity property, is a correctness gap rather than circularity: the proof does not define isometry in terms of the O(epsilon^{-3}) conclusion, and the gap is not the result of the paper presupposing its own theorem.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The central rates rest on standard smoothness and momentum results plus problem-specific assumptions: convex Lipschitz h, a convex constraint indicator with an error-bound condition, and an unstated isometric vector-transport property. No new physical or mathematical entities are invented; the main uncharged assumption is vector-transport isometry.

free parameters (2)
  • Smoothing and momentum schedule for Algorithm 3.1 = mu_k = k^{-1/3}; a_{k+1} = k^{-2/3}; adaptive tau_k = 1/(sum_{i=1}^k ||G_i||^2 / a_{k+1})^{1/3}
    Chosen by hand so that smoothing error and momentum-variance terms balance in the proof of Theorem 3.1; the O(epsilon^-3) exponent is tied to these schedules.
  • Smoothing, step-size, and momentum schedule for Algorithm 3.2 = omega = min{theta/(theta+2), 1/2}; mu_k = k^{-omega}; tau_k = c_tau (k+1)^{-omega}; a_k = c_a k^{-2omega}
    Chosen to close the error-bound and Lyapunov telescoping in Theorem 3.4; the constrained-case rate depends on this choice of omega.
assumptions (5)
  • ad hoc to paper Unstated isometric property of the vector transport T
    Lemma 4.4 invokes isometry of T around Eq. (4.9), but Definition 2.3 only requires linearity and the identity property, and Assumption 1 does not state isometry.
  • domain assumption Assumption 1.D average smoothness of stochastic gradients (Eq. 3.1)
    Needed for the variance bounds in Lemmas 4.4 and 4.8; it is an oracle-condition assumption that is not verified for the listed applications.
  • domain assumption Assumption 2: h is convex and ell_h-Lipschitz continuous
    Used for the Moreau-envelope gradient bounds and Lemma 2.6; the problem statement in Eq. (1.1) says weakly-convex, which is stronger than what is analyzed.
  • domain assumption Assumptions 3 and 4: h is the indicator of a convex set C and ||grad g(x)|| >= zeta dist^theta(c(x),C)
    Powers the constrained-case rate in Theorem 3.4; the error-bound condition is a regularity assumption on the constraint geometry that is not established for the applications listed.
  • standard math External results from [6], [8], and [28]
    Moreau-envelope properties, retraction-smoothness transfer, and the summation lemma are cited standard results used as black boxes; they do not contain the target result.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Single-loop $\mathcal{O}(\epsilon^{-3})$ stochastic smoothing algorithms for nonsmooth Riemannian optimization." pith.science (2026). https://pith.science/paper/TCOMGYXV

@misc{pith2026250509485,
  author       = {Pith},
  title        = {Pith review of: Single-loop $\mathcalO(\epsilon^-3)$ stochastic smoothing algorithms for nonsmooth Riemannian optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TCOMGYXV}},
  note         = {Machine review of arXiv:2505.09485}
}
abstract

In this paper, we develop two Riemannian stochastic smoothing algorithms for nonsmooth optimization problems on Riemannian manifolds, addressing distinct forms of the nonsmooth term \( h \). Both methods combine dynamic smoothing with a momentum-based variance reduction scheme in a fully online manner. When \( h \) is Lipschitz continuous, we propose an stochastic algorithm under adaptive parameter that achieves the optimal iteration complexity of \( \mathcal{O}(\epsilon^{-3}) \), improving upon the best-known rates for exist algorithms. When \( h \) is the indicator function of a convex set, we design a new algorithm using truncated momentum, and under a mild error bound condition with parameter \( \theta \geq 1 \), we establish a complexity of \( \tilde{\mathcal{O}}(\epsilon^{-\max\{\theta+2, 2\theta\}}) \), in line with the best-known results in the Euclidean setting. Both algorithms feature a single-loop design with low per-iteration cost and require only \( \mathcal{O}(1) \) samples per iteration, ensuring that sample and iteration complexities coincide. Our framework encompasses a broad class of problems and recovers or matches optimal complexity guarantees in several important settings, including smooth stochastic Riemannian optimization, composite problems in Euclidean space, and constrained optimization via indicator functions.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

44 extracted references · 36 canonical work pages

  1. [33]

    Z. Peng, W. Wu, J. Hu, and K. Deng , Riemannian smoothing gradient type algorithms for nonsmooth optimization problem on compact Riemannian submanifold embedded in Euclidean space, Applied Mathematics & Optimization, 88 (2023), p. 85

    work page 2023

  2. [14]

    K. Deng, J. Hu, J. Wu, and Z. Wen , Oracle complexities of augmented Lagrangian methods for nonsmooth manifold optimization , arXiv preprint arXiv:2404.05121, (2024)

    arXiv 2024

  3. [1]

    Absil and S

    P.-A. Absil and S. Hosseini, A collection of nonsmooth Riemannian optimization problems , Non- smooth Optimization and Its Applications, (2019), pp. 1–15

    work page 2019

  4. [2]

    Absil, R

    P.-A. Absil, R. Mahony, and R. Sepulchre , Optimization Algorithms on Matrix Manifolds , Princeton University Press, Princeton, NJ, 2008

    2008

  5. [3]

    Alacaoglu and S

    A. Alacaoglu and S. J. Wright, Complexity of single loop algorithms for nonlinear programming with stochastic objective and constraints , in International Conference on Artificial Intelligence and Statistics, PMLR, 2024, pp. 4627–4635

    work page 2024

  6. [4]

    Beck and I

    A. Beck and I. Rosset , A dynamic smoothing technique for a class of nonsmooth optimization problems on manifolds, SIAM Journal on Optimization, 33 (2023), pp. 1473–1493, https://doi. org/10.1137/22M1489447. 26 DENG AND PENG AND WU

    work page doi:10.1137/22m1489447 2023

  7. [5]

    A. S. Berahas, F. E. Curtis, D. Robinson, and B. Zhou , Sequential quadratic optimization for nonlinear equality constrained stochastic optimization , SIAM Journal on Optimization, 31 (2021), pp. 1352–1379

    work page 2021

  8. [6]

    B¨ohm and S

    A. B¨ohm and S. J. Wright , Variable smoothing for weakly convex composite functions , Journal of Optimization Theory and Applications, 188 (2021), pp. 628–649

    work page 2021

Show all 44 references
  1. [7]

    Bonnabel, Stochastic gradient descent on Riemannian manifolds , IEEE Transactions on Auto- matic Control, 58 (2013), pp

    S. Bonnabel, Stochastic gradient descent on Riemannian manifolds , IEEE Transactions on Auto- matic Control, 58 (2013), pp. 2217–2229

    2013

  2. [8]

    Boumal, P.-A

    N. Boumal, P.-A. Absil, and C. Cartis , Global rates of convergence for nonconvex optimization on manifolds, IMA Journal of Numerical Analysis, 39 (2019), pp. 1–33

    2019

  3. [9]

    Cambier and P.-A

    L. Cambier and P.-A. Absil , Robust low-rank matrix completion by Riemannian optimization , SIAM Journal on Scientific Computing, 38 (2016), pp. S440–S460

    2016

  4. [10]

    F. E. Curtis, M. J. O’Neill, and D. P. Robinson , Worst-case complexity of an SQP method for nonlinear equality constrained stochastic optimization , Mathematical Programming, 205 (2024), pp. 431–483

    2024

  5. [11]

    F. E. Curtis, D. P. Robinson, and B. Zhou , Inexact sequential quadratic optimization for min- imizing a stochastic objective function subject to deterministic nonlinear equality constraints , arXiv preprint arXiv:2107.03512, (2021)

    2021 arXiv

  6. [12]

    Cutkosky and F

    A. Cutkosky and F. Orabona, Momentum-based variance reduction in non-convex sgd, Advances in neural information processing systems, 32 (2019)

    2019

  7. [13]

    Demidovich, G

    Y. Demidovich, G. Malinovsky, and P. Richt ´arik, Streamlining in the Riemannian realm: Efficient Riemannian optimization with loopless variance reduction , arXiv preprint arXiv:2403.06677, (2024)

    2024 arXiv

  8. [15]

    Deng and Z

    K. Deng and Z. Peng , A manifold inexact augmented Lagrangian method for nonsmooth opti- mization on Riemannian submanifolds in Euclidean space, IMA Journal of Numerical Analysis, (2022), https://doi.org/10.1093/imanum/drac018

    2022 doi

  9. [16]

    K. Deng, S. Zhang, B. Wang, J. Jin, J. Zhou, and H. Wang , Stochastic momentum ADMM for nonconvex and nonsmooth optimization with application to PnP algorithm , arXiv preprint arXiv:2504.08223, (2025)

    2025 arXiv

  10. [17]

    Y. Gao, A. Rodomanov, and S. U. Stich , Non-convex stochastic composite optimization with Polyak momentum, arXiv preprint arXiv:2403.02967, (2024)

    2024 arXiv

  11. [18]

    Ghadimi and G

    S. Ghadimi and G. Lan , Accelerated gradient methods for nonconvex nonlinear and stochastic programming, Mathematical Programming, 156 (2016), pp. 59–99

    2016

  12. [19]

    Han and J

    A. Han and J. Gao, Riemannian stochastic recursive momentum method for non-convex optimiza- tion, arXiv preprint arXiv:2008.04555, (2020)

    2020 arXiv

  13. [20]

    Han and J

    A. Han and J. Gao , Improved variance reduction methods for Riemannian non-convex optimiza- tion, IEEE Transactions on Pattern Analysis and Machine Intelligence, 44 (2021), pp. 7610– 7623

    2021

  14. [21]

    Han and J

    A. Han and J. Gao, Riemannian stochastic recursive momentum method for non-convex optimiza- tion, in Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21, 2021, pp. 2505–2511

    2021

  15. [22]

    Hosseini and A

    S. Hosseini and A. Uschmajew , A Riemannian gradient sampling algorithm for nonsmooth opti- mization on manifolds , SIAM Journal on Optimization, 27 (2017), pp. 173–189

    2017

  16. [23]

    J. Hu, X. Liu, Z.-W. Wen, and Y.-X. Yuan , A brief introduction to manifold optimization , Journal of the Operations Research Society of China, 8 (2020), pp. 199–248

    2020

  17. [24]

    Huang, S

    F. Huang, S. Chen, and H. Huang , Faster stochastic alternating direction method of multipliers for nonconvex optimization , in International conference on machine learning, PMLR, 2019, pp. 2839–2848

    2019

  18. [25]

    Huang, S

    F. Huang, S. Chen, and Z. Lu, Stochastic alternating direction method of multipliers with variance reduction for nonconvex optimization , arXiv preprint arXiv:1610.02758, (2016)

    2016 arXiv

  19. [26]

    I. T. Jolliffe, N. T. Trendafilov, and M. Uddin , A modified principal component technique based on the LASSO , Journal of computational and Graphical Statistics, 12 (2003), pp. 531– 547

    2003

  20. [27]

    Kasai, H

    H. Kasai, H. Sato, and B. Mishra , Riemannian stochastic recursive gradient algorithm, in Inter- national conference on machine learning, PMLR, 2018, pp. 2516–2524

    2018

  21. [28]

    K. Levy, A. Kavis, and V. Cevher , Storm+: Fully adaptive sgd with recursive momentum for nonconvex optimization , Advances in Neural Information Processing Systems, 34 (2021), A SINGLE LOOPO(ϵ−3) STOCHASTIC SMOOTHING ALGORITHM 27 pp. 20571–20582

    2021

  22. [29]

    X. Li, S. Chen, Z. Deng, Q. Qu, Z. Zhu, and A. Man-Cho So , Weakly convex optimization over Stiefel manifold using Riemannian subgradient-type methods , SIAM Journal on Optimization, 31 (2021), pp. 1605–1634

    2021

  23. [30]

    Li, P.-Y

    Z. Li, P.-Y. Chen, S. Liu, S. Lu, and Y. Xu, Rate-improved inexact augmented Lagrangian method for constrained nonconvex optimization , in International Conference on Artificial Intelligence and Statistics, PMLR, 2021, pp. 2170–2178

    2021

  24. [31]

    Li, P.-Y

    Z. Li, P.-Y. Chen, S. Liu, S. Lu, and Y. Xu , Stochastic inexact augmented Lagrangian method for nonconvex expectation constrained optimization , Computational Optimization and Appli- cations, 87 (2024), pp. 117–147

    2024

  25. [32]

    Z. Lu, S. Mei, and Y. Xiao , Variance-reduced first-order methods for deterministically con- strained stochastic nonconvex optimization with strong convergence guarantees, arXiv preprint arXiv:2409.09906, (2024)

    2024 arXiv

  26. [34]

    M. F. Sahin, A. Eftekhari, A. Alacaoglu, F. L. G´omez, and V. Cevher, An inexact augmented Lagrangian framework for nonconvex optimization with nonlinear constraints , in Proceedings of NeurIPS 2019, 2019

    2019

  27. [35]

    H. Sato, H. Kasai, and B. Mishra , Riemannian stochastic variance reduced gradient algorithm with retraction and vector transport, SIAM Journal on Optimization, 29 (2019), pp. 1444–1472

    2019

  28. [36]

    S. E. Selvan, P. B. Borckmans, A. Chattopadhyay, and P.-A. Absil, Spherical mesh adaptive direct search for separating quasi-uncorrelated sources by range-based independent component analysis, Neural computation, 25 (2013), pp. 2486–2522

    2013

  29. [37]

    S. E. Selvan, S. T. George, and R. Balakrishnan , Range-based ICA using a nonsmooth quasi- Newton optimizer for electroencephalographic source localization in focal epilepsy, Neural com- putation, 27 (2015), pp. 628–671

    2015

  30. [38]

    Q. Shi, X. Wang, and H. Wang, A momentum-based linearized augmented Lagrangian method for nonconvex constrained stochastic optimization, Mathematics of Operations Research, (2025)

    2025

  31. [39]

    Tran-Dinh, N

    Q. Tran-Dinh, N. H. Pham, D. T. Phan, and L. M. Nguyen , A hybrid stochastic optimization framework for composite nonconvex optimization , Mathematical Programming, 191 (2022), pp. 1005–1071

    2022

  32. [40]

    B. Wang, S. Ma, and L. Xue , Riemannian stochastic proximal gradient methods for nonsmooth optimization over the Stiefel manifold , Journal of Machine Learning Research, 23 (2022), pp. 1– 33

    2022

  33. [41]

    X. Wang, S. Ma, and Y.-x. Yuan , Penalty methods with stochastic approximation for stochastic nonlinear programming, Mathematics of computation, 86 (2017), pp. 1793–1820

    2017

  34. [42]

    Z. Wang, K. Ji, Y. Zhou, Y. Liang, and V. Tarokh, Spiderboost and momentum: Faster variance reduction algorithms, Advances in Neural Information Processing Systems, 32 (2019)

    2019

  35. [43]

    Xu and Y

    Y. Xu and Y. Xu, Momentum-based variance-reduced proximal stochastic gradient method for com- posite nonconvex stochastic optimization , Journal of Optimization Theory and Applications, 196 (2023), pp. 266–297

    2023

  36. [44]

    Zhou, X.-T

    P. Zhou, X.-T. Yuan, and J. Feng, Faster first-order methods for stochastic non-convex optimiza- tion on Riemannian manifolds , in The 22nd international conference on artificial intelligence and statistics, PMLR, 2019, pp. 138–147

    2019

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.