Pith. sign in

REVIEW 2 major objections 5 minor 37 references

Revisiting Convergence: Shuffling Complexity Beyond Lipschitz Smoothness

T0 review · 2 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that shuffling-type gradient methods converge at the best-known rates without Lipschitz smoothness, under the weaker ℓ-smoothness assumption and a general bounded-variance condition.

desk verdict First ℓ-smoothness analysis of shuffling methods; nonconvex rates look right, but the no-variance theorems and one technical lemma need repair. read the letter →

arxiv 2507.08913 v1 pith:R5DS4VCE submitted 2025-07-11 cs.LG math.OCstat.ML

Qi He , Peiran Yu , Ziyi Chen , Heng Huang This is my paper
classification cs.LGmath.OCstat.ML MSC 90C2690C1590C25
keywords shuffling-typegradientmethodsrandomreshufflingℓ-smoothnessrelaxedsmoothnessnonconvexoptimizationstochasticdescentfinite-summinimizationconvergencerates
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Most convergence guarantees for shuffling-type gradient methods assume Lipschitz-bounded gradients, an assumption known to fail for several common machine-learning objectives. This paper replaces it with ℓ-smoothness, where the Hessian norm may grow subquadratically with the gradient norm, and adds a general bounded-variance condition on the per-sample gradients. Under these weaker hypotheses, the paper proves that the shuffling gradient algorithm still converges, with total gradient-evaluation complexity $O(n^{(p+1)/2}\epsilon^{-3})$ for nonconvex random reshuffling, matching the previous Lipschitz-smooth rate when $p=0$. It also supplies rates for strongly convex and non-strongly convex objectives, for both random and arbitrary shuffling schemes, and reports experiments on phase retrieval, distributionally robust optimization, and image classification where the shuffling algorithm converges faster than SGD.

What carries the argument

The proof is carried by a stopping-time argument. Let $\tau$ be the first epoch at which the function value exceeds a fixed threshold $H$. Conditioned on $t<\tau$, an induction lemma (Lemma A.4) shows that every iterate within the epoch stays inside a ball of radius $G'/\ell(2G')$ around the epoch start, so ordinary Lipschitz smoothness holds along the whole trajectory; the bounded-variance condition (Assumption 4.3) is what controls the component gradients inside that ball through Lemma A.2. A variance decomposition for sampling without replacement then bounds the within-epoch permutation-sum drift, and the optional stopping theorem turns the per-epoch descent inequality into a high-probability bound on the accumulated squared gradient norms. The step-size rule $\eta_t \le \tfrac{1}{2L}\sqrt{\tfrac{n}{A+n}}$ together with a sum-of-cubes budget $\sum_t \eta_t^3 \le n\Delta_1/(L^2\sigma^2)$ makes the remainder terms telescope.

What would settle it

Take $n=2$ with $f_1(x)=x^4$ and $f_2(x)=-x^4+2x^2$, so $F(x)=x^2$. Every component is ℓ-smooth, but the component-gradient deviation grows like $|x|^3$ while $\|\nabla F(x)\|^2=4x^2$, so Assumption 4.3 is violated. Run Algorithm 1 with random reshuffling and a step size obeying Theorem 4.4, and record whether the average squared gradient norm reaches $\epsilon^2$ within the predicted epoch count; seeing the rate persist would show the assumption is not necessary, while divergence or stalling would confirm that the bounded-variance condition is doing the work the proof assigns to it.

Watch

Extended reading notes

Core claim

Under Assumptions 4.1 to 4.3, Algorithm 1 with random reshuffling finds a point whose average squared gradient norm is at most $\epsilon^2$, with probability at least $1-\delta$, using $O(n^{(p+1)/2}\epsilon^{-3})$ total gradient evaluations, where $p\in[0,2)$ is the degree of the ℓ-smoothness function. With an arbitrary (including fixed) permutation scheme the same guarantee costs $O(n^{p/2+1}\epsilon^{-3})$ evaluations. For strongly convex objectives the paper proves $ ildem{O}(n^{(p+1)/2}\epsilon^{-1/2})$ evaluations under random reshuffling, and for non-strongly convex objectives $O(n^{(p+1)/2}\epsilon^{-3/2})$, with analogous arbitrary-scheme variants. When $p=0$, ℓ-smoothness degenerates to ordinary Lipschitz smoothness and the rates coincide with the previously known shuffling-type rates, so the result extends the best-known guarantees to a strictly larger function class.

Load-bearing premise

The nonconvex rates rest on Assumption 4.3, which says the average squared discrepancy between per-sample gradients and the full gradient is bounded by a constant times the squared full-gradient norm plus a constant; if component gradients are so heavy-tailed that this ratio grows without bound, the stated rates do not follow.

Editorial extensions

If this is right

  • When $p=0$ the nonconvex random-reshuffling complexity is $O(\sqrt{n}\,\epsilon^{-3})$, exactly the rate previously proven under Lipschitz smoothness.
  • Strongly convex random reshuffling needs $\widetilde{O}(n^{(p+1)/2}\epsilon^{-1/2})$ gradient evaluations, and non-strongly convex random reshuffling needs $O(n^{(p+1)/2}\epsilon^{-3/2})$.
  • Arbitrary shuffling schemes cost an extra factor of about $\sqrt{n}$ in total gradient evaluations compared with random reshuffling in each setting.
  • The concrete step-size choice $\eta = \Theta(n^{(1-p)/2}\epsilon)$ realizes the stated rates, giving a practical rule for non-Lipschitz objectives.
  • The results extend convergence guarantees to problems such as phase retrieval and distributionally robust optimization, where Lipschitz smoothness fails but ℓ-smoothness holds.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If Assumption 4.3 is truly the hinge, then clipping or robust variance estimators might extend these rates to heavy-tailed component gradients; the paper does not test that extension.
  • The confidence parameter enters polynomially ($T=O(\delta^{-3/2-p/(2-p)})$ in the nonconvex random-reshuffling case), so improving the probability argument to a logarithmic dependence appears to be a natural next step.
  • The same stopping-time framework may transfer to variance-reduced shuffling methods or single-shuffling schemes, which the authors list as future work.
  • The arbitrary-shuffling results suggest deterministic data order is only polynomially worse than random order under ℓ-smoothness; whether the $\sqrt{n}$ gap appears at practical $n$ and $\epsilon$ could be tested empirically.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper studies shuffling-type gradient methods (incremental, shuffle-once, and random reshuffling) for finite-sum problems in which the component functions are ℓ-smooth rather than Lipschitz smooth. The authors give counterexamples (DRO, phase retrieval), propose explicit step-size rules, and claim high-probability convergence rates for nonconvex, strongly convex, and non-strongly convex objectives, under a general bounded-gradient-variance condition and, in two theorems, without that condition. For p=0, the stated nonconvex rate O(n^{(p+1)/2} ε^{-3}) is claimed to match Nguyen et al. (2021), and similar matching claims are made for the convex settings. The appendix contains detailed proofs built around a stopping-time construction, and Section 4.3 candidly lists limitations, including the potentially large constant G' in Theorems 4.9 and 4.12.

Significance. If the proofs are completed, this is a valuable extension of shuffling-method theory: it removes the Lipschitz-smoothness assumption while preserving the same n-dependence as the Lipschitz-smooth rates in the p=0 special case. The paper has real strengths: explicit step-size choices, a detailed appendix, a clearly stated general variance condition, and honest limitation statements. The central nonconvex-rate claim is plausible and the main proof strategy is inventive, but two load-bearing steps in the appendix are not justified as written; both appear locally repairable, which is why I do not recommend rejection. No code or machine-checked proofs are provided, so verification is by reading the supplied derivations.

major comments (2)
  1. [A.1.1, Lemma A.5, inequality (8)] The optional-stopping step is not justified as written. The process Z_t is defined so that Z_t - Z_{t-1} equals -η_t^3 L^2/(2n)(A‖∇F(w_0^t)‖² + σ²) + (η_t L^2/(2n))Σ_{k=0}^{n-1} A1(t-1,k), and both terms are measurable with respect to F_{t-1}. Hence E[Z_t | F_{t-1}] = Z_t, which does not give the supermartingale inequality E[Z_t | F_{t-1}] ≤ Z_{t-1}; with the shifted filtration F_{t-2}, Z_t is not adapted. Since inequality (8) is what absorbs the permutation-variance terms and feeds directly into Theorems 4.4, 4.8, and 4.11, this is load-bearing. The gap appears repairable: because {t<τ} ∈ F_{t-1}, one can insert the indicator 1_{t<τ} inside the conditional-expectation bound and sum over t without invoking optional stopping, so I view this as a proof gap rather than a disproof of the theorem.
  2. [A.3, proof of Theorem 4.12] The invariance of the sublevel set S is not established. The induction assumes w_0^1, ..., w_0^{t-1} ∈ S in order to apply inequality (13) locally. In the case F(w_0^{t-1}) - F* > ε, the displayed argument only gives ‖w_0^t - w*‖² ≤ ‖w_0^{t-1} - w*‖². For a general convex function, remaining no farther from w* does not imply remaining in the sublevel set {F ≤ F(w_0^1)}; for example, with F(x,y) = x² + 100y² and w_0^1 = (1,0), the point (0.8,0.6) has the same distance to (0,0) but a larger function value. Therefore w_0^t ∈ S is not guaranteed, and the uniform L-smoothness needed for (13) for all t is not available. This needs a different argument or a strengthened assumption; the theorem may still be true, but the proof as written is incomplete.
minor comments (5)
  1. [Theorem 4.12 statement] The conclusion says 'we have ‖∇F(w_0^t)‖ ≤ G' but G is not defined in that theorem; the intended bound is presumably G'.
  2. [Assumption 4.3] The comparison with the standard stochastic-gradient condition has a typo: it should be E‖∇F(w;ξ) − ∇F(w)‖² ≤ σ², not E[|∇F(w;ξ) − ∇F(w)|] ≤ σ². Also, since the finite-sum inequality (3) is deterministic, the phrase 'a.s.' is unnecessary.
  3. [Definition of p, Section 4.1] The definition 'p = sup{p ≥ 0 | lim_{w→∞} ℓ(w)/w^p > 0}' uses the same symbol p on both sides; it should be written with a different dummy variable, e.g., p = sup{ q ≥ 0 : lim_{w→∞} ℓ(w)/w^q > 0 }.
  4. [Theorem 4.4 display] The step-size condition 'η_t ≤ 1/(2L) sqrt(A/n + 1)' is ambiguous; it should read η_t ≤ (1/(2L))√(A/n + 1). Similar parenthesization would help in Theorem 4.6.
  5. [Section 5] The experiments compare Algorithm 1 with SGD on average learning curves, but they do not directly test the predicted rates or the dependence on δ, n, and p; this limits the extent to which the numerics validate the theory, though the experiments are otherwise reasonable.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the claimed rates are derived by forward analysis from explicit assumptions, and the only flagged issue in Lemma A.5 is a correctness gap, not a circular reduction.

full rationale

The derivation is self-contained rather than circular. Assumptions 4.1-4.3 and Definition 2.2 fix the objective class; the auxiliary quantities G, G', and L are defined from problem constants (Δ1, δ, n, A, σ, ℓ) through the fixed-point inequality u² ≤ 2ℓ(2u)H, and the step-size conditions in Theorems 4.4, 4.6, 4.8, and 4.11 are stated as constraints on η_t and T in terms of those constants. No parameter is fitted to data or to the target ε-rate, and the advertised O(n^{(p+1)/2}ε^{-3}) total gradient complexity is obtained by algebraically substituting the displayed choice η = O(∛(n^{1-p}/T)) into the epoch constraint T ≥ 32Δ1/(η_T δ ε²), not by assuming the conclusion. The imported lemmas (Li et al. Lemma 3.5; Mishchenko et al. Lemma 1; Nguyen et al. Lemmas 6/7 and Theorem 1) are external prior work, and the authors' self-citations (Xian et al.; Chen et al. 2023) appear only as related-work context or experimental setup, so no load-bearing self-citation chain is present. The paper's own Limitations section concedes that G' in Theorems 4.9/4.12 is potentially large and hard to verify, which weakens those unconditional-sounding complexity claims but does not make them circular. One non-circular proof concern should be flagged: in Lemma A.5, the optional-stopping argument for the A1 terms claims "We know {Z_t} is a supermartingale," but with the filtration F_t = σ(π(1),...,π(t)) used in the proof, both terms defining Z_t − Z_{t−1} are F_{t−1}-measurable, so E[Z_t|F_{t−1}] = Z_t; inequality (8) is therefore not justified as written. This is a repairable proof gap, not an equivalence between output and input, so it does not raise the circularity score.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

No free parameters are fitted to data. The axioms are the standard and domain-specific assumptions stated in Section 4.1 plus an implicit compactness/membership assumption in Theorems 4.9 and 4.12 that the proofs do not fully establish. No new entities are introduced.

assumptions (5)
  • domain assumption Assumption 4.1: F is proper and bounded below on its domain.
    Standard well-posedness assumption for the finite-sum problem (P).
  • domain assumption Assumption 4.2: F and all f(·;i) are ℓ-smooth for a common sub-quadratic ℓ, with 0 ≤ p < 2.
    Defines the relaxed smoothness class. The sub-quadratic requirement (p<2) determines the achievable step sizes and rates, e.g., η = O(n^{(1-p)/2} ε).
  • domain assumption Assumption 4.3: component gradient variance satisfies (1/n)Σ||∇f(w;i)-∇F(w)||² ≤ A||∇F(w)||² + σ².
    Load-bearing: used in Lemma A.2 to bound component gradients by G' and in Lemma A.5 to control permutation variance. Without it the nonconvex rates in Theorems 4.4 and 4.6 fail.
  • domain assumption Assumption 4.7 (µ-strong convexity) and Assumption 4.10 (convexity of components).
    Define the strongly convex and non-strongly convex regimes used in Theorems 4.8, 4.9, 4.11, and 4.12.
  • domain assumption Implicit in Theorems 4.9 and 4.12: the sublevel set S is compact, G' = max_{w∈S,i}||∇f(w;i)|| is finite, and the iterates remain inside the L-smooth ball around each epoch start.
    The theorem statements define G' but do not justify that the iterates remain in S; Theorem 4.12's proof asserts this from a distance-to-optimum bound, which does not imply membership in S for general convex functions. This is an unproven assumption that the no-variance rates rest on.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Revisiting Convergence: Shuffling Complexity Beyond Lipschitz Smoothness." pith.science (2026). https://pith.science/paper/R5DS4VCE

@misc{pith2026250708913,
  author       = {Pith},
  title        = {Pith review of: Revisiting Convergence: Shuffling Complexity Beyond Lipschitz Smoothness},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/R5DS4VCE}},
  note         = {Machine review of arXiv:2507.08913}
}
read the original abstract

Shuffling-type gradient methods are favored in practice for their simplicity and rapid empirical performance. Despite extensive development of convergence guarantees under various assumptions in recent years, most require the Lipschitz smoothness condition, which is often not met in common machine learning models. We highlight this issue with specific counterexamples. To address this gap, we revisit the convergence rates of shuffling-type gradient methods without assuming Lipschitz smoothness. Using our stepsize strategy, the shuffling-type gradient algorithm not only converges under weaker assumptions but also match the current best-known convergence rates, thereby broadening its applicability. We prove the convergence rates for nonconvex, strongly convex, and non-strongly convex cases, each under both random reshuffling and arbitrary shuffling schemes, under a general bounded variance condition. Numerical experiments further validate the performance of our shuffling-type gradient algorithm, underscoring its practical efficacy.

Figures

Figures reproduced from arXiv: 2507.08913 by the authors.

Figure 1
Figure 1. Experimental Results on Convex (up) and Strongly-convex (down) Objective Functions. their sample average below. f(x) = 1 1050 X (k,i)∈E fi,k(x) = 1 2 ||x||2+ exp(n + 1) − exp(−n) 1050[exp(1) − 1] X 50 j=1 [exp(xj ) + exp(−xj )]. All these functions fj,k and f are 1-strongly convex and ℓ￾smooth (with ℓ(u) = 5u + 5) but not Lipschitz-smooth. We repeat the experiment in the same procedure above, except that all the ste… view at source ↗
Figure 3
Figure 3. Experimental Results on Cifar 10 Dataset. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

37 extracted references · 26 canonical work pages

  1. [1]

    K. Ahn, C. Yun, and S. Sra. Sgd with shuffling: optimal rates without component convexity and large epoch requirements. Advances in Neural Information Processing Systems, 33: 0 17526--17535, 2020

    work page 2020

  2. [2]

    Arjevani, Y

    Y. Arjevani, Y. Carmon, J. C. Duchi, D. J. Foster, N. Srebro, and B. Woodworth. Lower bounds for non-convex stochastic optimization. Mathematical Programming, 199 0 (1): 0 165--214, 2023

    2023

  3. [3]

    L. Bottou. Curiously fast convergence of some stochastic gradient descent algorithms. 2009. URL https://api.semanticscholar.org/CorpusID:16822133

    work page 2009

  4. [4]

    L. Bottou. Stochastic Gradient Descent Tricks, pages 421--436. Springer Berlin Heidelberg, Berlin, Heidelberg, 2012. ISBN 978-3-642-35289-8. doi:10.1007/978-3-642-35289-8_25. URL https://doi.org/10.1007/978-3-642-35289-8_25

    doi:10.1007/978-3-642-35289-8_25 2012

  5. [5]

    Z. Chen, Y. Zhou, Y. Liang, and Z. Lu. Generalized-smooth nonconvex optimization is as efficient as smooth nonconvex optimization. arXiv preprint arXiv:2303.02854, 2023

    work page Pith review arXiv 2023

  6. [6]

    Crawshaw, M

    M. Crawshaw, M. Liu, F. Orabona, W. Zhang, and Z. Zhuang. Robustness to unbounded smoothness of generalized signsgd. Advances in neural information processing systems, 35: 0 9955--9968, 2022

    work page 2022

  7. [7]

    De and T

    S. De and T. Goldstein. Efficient distributed SGD with variance reduction. In F. Bonchi, J. Domingo - Ferrer, R. Baeza - Yates, Z. Zhou, and X. Wu, editors, IEEE 16th International Conference on Data Mining, ICDM 2016, December 12-15, 2016, Barcelona, Spain , pages 111--120. IEEE Computer Society, 2016. doi:10.1109/ICDM.2016.0022. URL https://doi.org/10.1...

    arXiv 2016

  8. [8]

    J. Drenth. Principles of protein X-ray crystallography. Springer Science & Business Media, 2007

    work page 2007

Show all 37 references
  1. [9]

    d’Aspremont, D

    A. d’Aspremont, D. Scieur, A. Taylor, et al. Acceleration methods. Foundations and Trends in Optimization , 5 0 (1-2): 0 1--245, 2021

    2021

  2. [10]

    Feurer and F

    M. Feurer and F. Hutter. Hyperparameter optimization. Automated machine learning: Methods, systems, challenges, pages 3--33, 2019

    2019

  3. [11]

    Ghadimi and G

    S. Ghadimi and G. Lan. Stochastic first-and zeroth-order methods for nonconvex stochastic programming. SIAM journal on optimization, 23 0 (4): 0 2341--2368, 2013

    2013

  4. [12]

    G \" u rb \" u zbalaban, A

    M. G \" u rb \" u zbalaban, A. E. Ozdaglar, and P. A. Parrilo. Why random reshuffling beats stochastic gradient descent. Math. Program., 186 0 (1): 0 49--84, 2021. doi:10.1007/S10107-019-01440-W. URL https://doi.org/10.1007/s10107-019-01440-w

    2021 doi

  5. [13]

    J. Z. HaoChen and S. Sra. Random shuffling beats SGD after finite epochs. In K. Chaudhuri and R. Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA , volume 97 of Proceedings of M...

    2019

  6. [14]

    K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770--778, 2016

    2016

  7. [15]

    J. Jin, B. Zhang, H. Wang, and L. Wang. Non-convex distributionally robust optimization: Non-asymptotic analysis. Advances in Neural Information Processing Systems, 34: 0 2771--2782, 2021

    2021

  8. [16]

    Koloskova, N

    A. Koloskova, N. Doikov, S. U. Stich, and M. Jaggi. Shuffle sgd is always better than sgd: improved analysis of sgd with arbitrary data orders. arXiv preprint arXiv:2305.19259, 2023

    2023 arXiv

  9. [17]

    Krizhevsky

    A. Krizhevsky. Learning multiple layers of features from tiny images. Master's thesis, University of Toronto, 2009

    2009

  10. [18]

    D. Levy, Y. Carmon, J. C. Duchi, and A. Sidford. Large-scale methods for distributionally robust optimization. Advances in Neural Information Processing Systems, 33: 0 8847--8860, 2020

    2020

  11. [19]

    H. Li, J. Qian, Y. Tian, A. Rakhlin, and A. Jadbabaie. Convex and non-convex optimization under generalized smoothness. In Thirty-seventh Conference on Neural Information Processing Systems, 2023 a

    2023

  12. [20]

    H. Li, A. Rakhlin, and A. Jadbabaie. Convergence of adam under relaxed assumptions, 2023 b

    2023

  13. [21]

    Malinovsky, A

    G. Malinovsky, A. Sailanbayev, and P. Richt \'a rik. Random reshuffling with variance reduction: New analysis and better rates. In Uncertainty in Artificial Intelligence, pages 1347--1357. PMLR, 2023

    2023

  14. [22]

    J. Miao, P. Charalambous, J. Kirz, and D. Sayre. Extending the methodology of x-ray crystallography to allow imaging of micrometre-sized non-crystalline specimens. Nature, 400 0 (6742): 0 342--344, 1999

    1999

  15. [23]

    Mishchenko, A

    K. Mishchenko, A. Khaled, and P. Richt \'a rik. Random reshuffling: Simple analysis with vast improvements. Advances in Neural Information Processing Systems, 33: 0 17309--17320, 2020

    2020

  16. [24]

    A. S. Nemirovskij and D. B. Yudin. Problem complexity and method efficiency in optimization. 1983

    1983

  17. [25]

    Nesterov

    Y. Nesterov. Gradient methods for minimizing composite functions. Mathematical programming, 140 0 (1): 0 125--161, 2013

    2013

  18. [26]

    L. M. Nguyen, Q. Tran-Dinh, D. T. Phan, P. H. Nguyen, and M. Van Dijk. A unified convergence analysis for shuffling-type gradient methods. The Journal of Machine Learning Research, 22 0 (1): 0 9397--9440, 2021

    2021

  19. [27]

    Radford, K

    A. Radford, K. Narasimhan, T. Salimans, I. Sutskever, et al. Improving language understanding by generative pre-training. 2018

    2018

  20. [28]

    Reisizadeh, H

    A. Reisizadeh, H. Li, S. Das, and A. Jadbabaie. Variance-reduced clipping for non-convex optimization. arXiv preprint arXiv:2303.00883, 2023

    2023 arXiv

  21. [29]

    Safran and O

    I. Safran and O. Shamir. How good is SGD with random shuffling? In J. D. Abernethy and S. Agarwal, editors, Conference on Learning Theory, COLT 2020, 9-12 July 2020, Virtual Event [Graz, Austria] , volume 125 of Proceedings of Machine Learning Research, pages 3250--3284. PMLR ...

    2020

  22. [30]

    O. Shamir. Without-replacement sampling for stochastic gradient methods: Convergence results and application to distributed optimization. CoRR, abs/1603.00570, 2016. URL http://arxiv.org/abs/1603.00570

    2016 arXiv

  23. [31]

    R. S. Sutton and A. G. Barto. Reinforcement learning: An introduction. MIT press, 2018

    2018

  24. [32]

    B. Wang, Y. Zhang, H. Zhang, Q. Meng, Z.-M. Ma, T.-Y. Liu, and W. Chen. Provable adaptivity in adam. arXiv preprint arXiv:2208.09900, 2022

    2022 arXiv

  25. [33]

    B. Wang, Y. Zhang, H. Zhang, Q. Meng, R. Sun, Z.-M. Ma, T.-Y. Liu, Z.-Q. Luo, and W. Chen. Provable adaptivity of adam under non-uniform smoothness. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 2960--2969, 2024

    2024

  26. [34]

    W. Xian, Z. Chen, and H. Huang. Delving into the convergence of generalized smooth minimax optimization. In Forty-first International Conference on Machine Learning

  27. [35]

    Zhang, J

    B. Zhang, J. Jin, C. Fang, and L. Wang. Improved analysis of clipping algorithms for non-convex optimization. Advances in Neural Information Processing Systems, 33: 0 15511--15521, 2020

    2020

  28. [36]

    Zhang, T

    J. Zhang, T. He, S. Sra, and A. Jadbabaie. Why gradient clipping accelerates training: A theoretical justification for adaptivity. arXiv preprint arXiv:1905.11881, 2019

    1905 arXiv

  29. [37]

    Zhang, P

    Q. Zhang, P. Xiao, K. Ji, and S. Zou. On the convergence of multi-objective optimization under generalized smoothness. arXiv preprint arXiv:2405.19440, 2024

    2024 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.