Pith. sign in

REVIEW 2 major objections 5 minor 44 references

An Efficient Stochastic First-Order Algorithm for Nonconvex--Strongly Concave Minimax Optimization beyond Lipschitz Smoothness

T0 review · 2 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A normalized stochastic descent-ascent method with momentum reaches ε-stationarity in O(ε⁻⁴) stochastic-gradient evaluations for nonconvex–strongly concave minimax problems under generalized smoothness, with a constant batch size.

desk verdict The constant-batchsize idea is sensible, but Lemma 3 is false on compact Y, so the stated main theorems don't cover the constrained case; the unconstrained version may survive. read the letter →

arxiv 2603.04940 v2 pith:I4F6IXXP submitted 2026-03-05 math.OC

Yan Gao , Yongchao Liu This is my paper · ORCID
classification math.OC MSC 90C2690C4765K05
keywords nonconvex-stronglyconcaveminimaxoptimization(L0L1)-generalizedsmoothnessnormalizedstochasticgradientdescentascentmomentumconstantbatchsizeε-stationarypointoraclecomplexitydistributionallyrobust
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that stochastic minimax optimization beyond Lipschitz smoothness does not need large batches. It proposes NSGDA-M, a single-loop method that updates the outer variable by normalized stochastic gradient descent with momentum and the inner variable by projected stochastic gradient ascent, and claims an ε-stationary point of the primal function Φ(x)=max_{y∈Y} L(x,y) in O(ε⁻⁴) stochastic-gradient evaluations in expectation and O(ε⁻⁴(log(1/δ))^{3/2}) with high probability. Under the (L0,L1) generalized smoothness condition, this would match the known ε⁻⁴ oracle lower bound for nonconvex stochastic optimization and remove the Θ(ε⁻²) batch-size requirement of earlier generalized-smooth minimax methods. The theorems cover both unconstrained and compact convex inner feasible sets; the proof's y-tracking recursion, which keeps the inner iterate near the exact maximizer y*(x), is the step on which the rates depend.

What carries the argument

The load-bearing object is the tracking error ∥y_t − y*(x_t)∥ and the two inequalities that control it. Lemma 1 gives Lipschitz continuity of the exact maximizer y*(·) with modulus κ=(Ly,0+Ly,1B)/μ; Lemma 3 bounds the squared inner gradient by 2(Ly,0+Ly,1∥∇yL∥)(L(x,y*(x))−L(x,y)), converting the y-ascent gain into a contraction. Plugging these into the y recursion produces inequality (6) in the expectation proof and inequality (12) in the high-probability proof. On the x side, the normalized momentum step makes the noise enter as a geometrically discounted martingale sum, which is what lets the method work with a single sample per iteration.

What would settle it

Take L(x,y)=f(x)+y−(μ/2)y² on Y=[0,1] with 0<μ<1 and any bounded nonconvex f. The unique y*-maximizer is 1 for every x; at y=1, ∇yL=1−μ>0 while the right-hand side of Lemma 3 is 0, so Lemma 3's conclusion is false. Since Lemma 3 is invoked in inequalities (6) and (12), checking whether NSGDA-M's stated O(ε⁻⁴) tracking bound holds on this instance (or on a simplex-constrained robust regression problem with a boundary maximizer) directly tests the constrained-case claim.

Watch

Extended reading notes

Core claim

The paper's central claim is that NSGDA-M achieves the optimal ε-dependence for nonconvex stochastic optimization in a minimax setting with generalized smoothness: O(ε⁻⁴) stochastic-gradient calls in expectation (Theorem 1) and O(ε⁻⁴(log(1/δ))^{3/2}) with probability at least 1−δ (Theorem 2), with a batch size of one. The x update uses a momentum-accelerated normalized stochastic gradient, and the y update is a projected stochastic ascent step. The only initial-condition requirement is that y0 be within O(√δ/Lx,1) (or O(1/Lx,1) in Theorem 2) of y*(x0). For the constrained case, Assumption 4 bounds the inner gradient at the maximizer, and the analysis bounds the tracking error ∥y_t−y*(x_t)∥ b

Load-bearing premise

The rate depends on Lemma 3, which assumes that moving from a current inner point along its gradient direction (with step size governed by the local smoothness constants) produces a point still inside the feasible set Y; for compact convex Y with a boundary maximizer this can fail, and then the tracking-error contraction used in Theorems 1 and 2 does not follow.

Editorial extensions

If this is right

  • If the guarantee holds, generalized-smooth nonconvex–strongly concave minimax problems are solved with the same ε⁻⁴ oracle complexity as standard Lipschitz-smooth nonconvex stochastic optimization.
  • The constant batch size enables streaming or single-pass use, since one sample per iteration suffices instead of batches of size Θ(ε⁻²).
  • The failure-probability dependence improves to (log(1/δ))^{3/2}, making high-probability certificates practical at small δ.
  • The stated theorems require a warm start y0 near y*(x0); absent that, the tracking-error argument is not covered.
  • Experiments on distributionally robust logistic regression over nine benchmark datasets show the batch-size-one momentum variant converging as reliably as a batch-size-50 normalized ascent baseline.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • (Editorial extension) The proof's dependence on Lemma 3 suggests that the constrained-case theorem is only as solid as the feasibility of the trial point; a repair would likely project the trial step onto the tangent or normal cone of Y, and if that repair works, the same rates should carry over.
  • (Editorial extension) The normalized-momentum mechanism isolates noise handling from sample size; substituting a variance-reduced estimator for the momentum term is a natural next step that could sharpen the log(1/δ) factor in the high-probability bound.
  • (Editorial extension) If the claimed bounds hold, the practical message for distributionally robust optimization is that single-sample updates are enough, making generalized-smooth minimax solvers usable in streaming settings without growing batches.
  • (Editorial extension) Testing the algorithm on an exact boundary-maximizer example would separate a proof gap from a genuine algorithmic failure; if the method still converges at the predicted rate despite Lemma 3 failing, the theorem may be repairable without changing the algorithm.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. This paper studies stochastic nonconvex-strongly concave minimax problems (1) under (L0,L1)-generalized smoothness with Y either R^m or a compact convex set. It proposes NSGDA-M (Algorithm 1), which updates the inner variable by projected stochastic gradient ascent and the outer variable by normalized SGD with momentum. The main theoretical claims are an O(epsilon^-4) expected stochastic-gradient complexity (Theorem 1) and an O(epsilon^-4 (log(1/delta))^{3/2}) high-probability complexity (Theorem 2) with a constant batch size, improving on the O(epsilon^-2) batch size and delta^{-4} factor of Xian et al. An appendix analyzes normalized SGDA. Numerical experiments on simplex-constrained DRO logistic regression are reported.

Significance. If the results were valid, the paper would make a useful contribution to the growing literature on nonconvex minimax optimization under generalized smoothness. The analysis is self-contained, the stepsize choices are explicit, and no fitted parameters or circularly tuned constants are used; the constant-batchsize aspect and the improved delta dependence are meaningful. However, the central lemma used to control the y-tracking error is false for compact convex Y, and the constrained case, including the paper's own experiments, is exactly where the proof fails. The advertised improvement over [34] is therefore not established for the stated problem class.

major comments (2)
  1. [Section 3, Lemma 3] Lemma 3 asserts that for any (x,y) in R^n x R^m, ||grad_y L(x,y)||^2 <= 2(Ly0 + Ly1||grad_y L(x,y)||)(L(x,y*(x))-L(x,y)). The proof sets y' = y + grad_y L(x,y)/(Ly0 + Ly1||grad_y L(x,y)||) and then invokes L(x,y*(x)) >= L(x,y'), which requires y' to be feasible for Y. Under Assumption 2(b), Y may be a compact convex set and this feasibility can fail. A concrete counterexample is L(x,y) = y - (mu/2)y^2 with Y = [0,1] and 0 < mu < 1. Then y*(x)=1, grad_y L(x,1)=1-mu>0, while the gap L(x,y*(x))-L(x,y*(x)) is zero; Lemma 3 would give (1-mu)^2 <= 0. This L satisfies Assumptions 1-4 with Ly0=mu, Ly1=0, and B=1-mu. The proof's key step is therefore invalid for the constrained case.
  2. [Theorems 1 and 2, inequalities (6) and (12)] The false Lemma 3 is the only source of the negative gap term that absorbs eta_y^2||grad_y L||^2 in the y-tracking recursion. In deriving (6) and (12), the term -2 eta_y (1 - eta_y Lbar_y)(L(x_t,y*(x_t))-L(x_t,y_t)) is used to obtain contraction. Without Lemma 3, that term is unavailable, and the recursive estimate leading to ||y_t-y*(x_t)|| <= 1/(8 Lx,1) on the stopping-time event is not established. This bound is load-bearing: it is used to define T0, to bound I2 in the momentum-error estimate, and to justify the final stationarity guarantee. The same defect propagates to the constrained-case claims in the Appendix Theorems 3-4. Since Assumption 2(b) explicitly includes compact convex Y and the experiments use a simplex, the main theorems do not support the paper's stated claims for that setting.
minor comments (5)
  1. [Section 3.1, discussion after Theorem 1] The claim of O(epsilon^-4) in expectation should be qualified. The displayed parameter choice contains delta-dependent terms, and the final complexity expression includes 1/(epsilon^3 sqrt(delta)) and 1/(epsilon^2 delta). The O(epsilon^-4) statement only holds when delta is treated as a fixed constant or when epsilon is sufficiently small relative to delta.
  2. [Lemma 1 proof] The proof cites Lemma 4 before Lemma 4 is introduced in Section 3.2; renumber or add a forward reference.
  3. [Assumption 3 / Lemma 3] Assumption 3 uses 1/Lx,1 and 1/Ly,1, which are undefined when those constants are zero. State the convention for zero constants, and make explicit in Lemma 3 that the constructed point y' must lie in Y when Y is a proper subset of R^m.
  4. [Algorithm 1, eq. (3)] The normalized update x_{t+1} = x_t - eta_x m_{t+1}/||m_{t+1}|| is undefined when m_{t+1}=0; specify a convention, e.g., x_{t+1}=x_t in that case.
  5. [Abstract] The front-matter abstract and the abstract at the start of the full text disagree. The former mentions an expected stationarity bound O(delta^{-3/4}T^{-1/4}) + G_Phi delta and a T^{-1/7} rate that do not appear in the body, and the two abstracts state different logarithmic dependencies for the high-probability result. These should be aligned.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the convergence analysis is self-contained and no fitted quantity is relabeled as a prediction.

full rationale

The paper derives its O(epsilon^-4) guarantees from stated assumptions - (L0,L1) generalized smoothness, mu-strong concavity, unbiased/sub-Gaussian noise, and bounded gradient at y*(x) - and explicit closed-form hyperparameter choices in Theorems 1-2. There are no fitted parameters called predictions: 1-beta, eta_x, eta_y, and T are selected from problem constants, and the stochastic-gradient complexity is then computed, not matched to data. The few imported results (Lemma 4 from [22] and Lemma 5 from [8]) are external concentration and recursive-MGF facts, not self-citations, and they do not presuppose the theorem's conclusion. The comparison to [34] is a claimed improvement, but the paper supplies its own analysis rather than relying on [34] for the main derivation. The skeptical objection about Lemma 3 concerns the feasibility of y' = y + grad_y L(x,y)/(L_y0 + L_y1 ||grad_y L(x,y)||) when Y is compact; if true, this would invalidate the constrained-case guarantees through inequalities (6) and (12). However, an invalid or missing assumption is a correctness risk, not circularity: Lemma 3 does not reduce the theorem's conclusion to its own inputs by construction. No self-definitional step, fitted-input-as-prediction step, or load-bearing self-citation chain is present, so the circularity score is 0.

Assumptions & free parameters 1 free parameters · 6 assumptions · 0 invented entities

The theory rests on standard stochastic/smoothness assumptions plus one ad hoc feasibility assumption in Lemma 3 that is invalid for compact feasible sets.

free parameters (1)
  • Step-size/momentum constants in Theorems 1–2 = 1−β, ηx, ηy as min-expressions in Thm 1/2
    Chosen by hand with large constants (37632, 762048, etc.) to make the proof inequalities hold; not fitted to data. They are standard algorithm-parameter choices rather than empirical free parameters.
assumptions (6)
  • domain assumption Φ is lower bounded (Assumption 1)
    Guarantees finite Φ∗ and use of ΔΦ.
  • domain assumption L(x,·) is µ-strongly concave and Y is R^m or compact convex (Assumption 2)
    Defines y*(x) and the projection step.
  • domain assumption (L0,L1) partial generalized smoothness (Assumption 3)
    Basis for local descent lemmas and y*-Lipschitz bound.
  • domain assumption Gradient at y*(x) bounded by B (Assumption 4)
    Controls y*-Lipschitz constant for compact Y.
  • domain assumption Unbiased stochastic gradients (Assumption 5), variance bounds (Assumptions 6/7)
    Standard stochastic oracle conditions.
  • ad hoc to paper In Lemma 3, y′ = y+∇yL/(Ly0+Ly1∥∇yL∥) is feasible in Y, giving L(x,y*)≥L(x,y′)
    False for compact Y with boundary maximizers; the lemma and therefore the constrained-case theorems fail.

how reviews work

0 comments
Cite this review

Pith. "Pith review of An Efficient Stochastic First-Order Algorithm for Nonconvex--Strongly Concave Minimax Optimization beyond Lipschitz Smoothness." pith.science (2026). https://pith.science/paper/I4F6IXXP

@misc{pith2026260304940,
  author       = {Pith},
  title        = {Pith review of: An Efficient Stochastic First-Order Algorithm for Nonconvex--Strongly Concave Minimax Optimization beyond Lipschitz Smoothness},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/I4F6IXXP}},
  note         = {Machine review of arXiv:2603.04940}
}
abstract

In recent years, nonconvex minimax problems have attracted significant attention because of their broad applications in machine learning, including generative adversarial networks, robust optimization and adversarial training. Most existing algorithms for nonconvex stochastic minimax problems are developed under the standard Lipschitz smoothness assumption. In this paper, we study stochastic minimax problems under a generalized smoothness condition and propose an algorithm, NSGDA-M, which simultaneously updates the inner variable by stochastic gradient ascent and updates the outer variable by normalized stochastic gradient descent with momentum. When the objective function is nonconvex--strongly concave, we show that NSGDA-M finds an \(\epsilon\)-stationary point of the primal function within \(\mathcal O(\epsilon^{-4}\log(1/(\epsilon\delta)))\) stochastic gradient evaluations with probability at least \(1-\delta\). Moreover, we establish an expected stationarity guarantee of \(\mathcal O(\delta^{-3/4}T^{-1/4})+G_\Phi\delta\), which gives a convergence rate \(\mathcal O(T^{-1/7})\). Here \(G_\Phi\) bounds the primal gradient norms along the iterates. Numerical experiments on a distributionally robust optimization problem demonstrate the effectiveness of the proposed algorithm.

Figures

Figures reproduced from arXiv: 2603.04940 by the authors.

Figure 1
Figure 1. Convergence of NSGDA-M, NSGDA and SGDA Acknowledgment The research is supported by the NSFC #12471283 and Fundamental Re￾search Funds for the Central Universities DUT24LK001. 22 [PITH_FULL_IMAGE:figures/full_fig_p022_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

44 extracted references · 2 linked inside Pith

  1. [34]

    Delving into the convergence of generalized smooth minimax optimization

    Wenhan Xian, Ziyi Chen, and Heng Huang. Delving into the convergence of generalized smooth minimax optimization. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 54191– 54211. PMLR, 2024

    2024

  2. [1]

    Duchi, Dylan J

    Yossi Arjevani, Yair Carmon, John C. Duchi, Dylan J. Fost er, Nathan Srebro, and Blake Woodworth. Lower bounds for non-convex stochastic optimization. Mathematical Program- ming, 199(1–2):165–214, 2023

    2023

  3. [2]

    Alternating proximal-gra dient steps for (stochastic) nonconvex-concave minimax problems

    Radu Ioan Bot ¸ and Axel B¨ ohm. Alternating proximal-gra dient steps for (stochastic) nonconvex-concave minimax problems. SIAM Journal on Optimization , 33(3):1884–1913, 2023

    1913

  4. [3]

    An efficient stochastic algorithm for decentralized nonconvex-strongly-concave minimax optimization

    Lesi Chen, Haishan Ye, and Luo Luo. An efficient stochastic algorithm for decentralized nonconvex-strongly-concave minimax optimization. In Proceedings of the 27th International Conference on Artificial Intelligence and Statistics , volume 238 of Proceedings of Machine Learning Research, pages 1990–1998. PMLR, 2024

    1990

  5. [4]

    Genera lized-smooth nonconvex optimization is as efficient as smooth nonconvex optimizatio n

    Zhe Chen, Yi Zhou, Yingbin Liang, and Zhaosong Lu. Genera lized-smooth nonconvex optimization is as efficient as smooth nonconvex optimizatio n. In International Conference on Machine Learning , pages 5396–5427. PMLR, 2023

    2023

  6. [5]

    Robustness to unbounded smoothness of generalized signSGD

    Michael Crawshaw, Mingrui Liu, Francesco Orabona, Wei Z hang, and Zhenxun Zhuang. Robustness to unbounded smoothness of generalized signSGD . In Advances in Neural Information Processing Systems (NeurIPS) , volume 35, pages 9955–9968, 2022

    2022

  7. [6]

    Antonia Creswell, Tom White, Vincent Dumoulin, Kai Arul kumaran, Biswa Sengupta, and Anil A. Bharath. Generative adversarial networks: An overv iew. IEEE Signal Processing Magazine, 35(1):53–65, 2018

    2018

  8. [7]

    Momentum improves norma lized sgd

    Ashok Cutkosky and Harsh Mehta. Momentum improves norma lized sgd. In Proceedings of the 37th International Conference on Machine Learning , volume 119 of Proceedings of Machine Learning Research, pages 2260–2268. PMLR, 2020

    2020

Show all 44 references
  1. [8]

    Stochastic optimization under distributional drift

    Joshua Cutler, Dmitriy Drusvyatskiy, and Zaid Harchaou i. Stochastic optimization under distributional drift. Journal of Machine Learning Research , 24(147):1–56, 2023

    2023

  2. [9]

    Beyond uniform smoothness: A stopped analysis of adaptive sgd

    Matthew Faw, Litu Rout, Constantine Caramanis, and Sanj ay Shakkottai. Beyond uniform smoothness: A stopped analysis of adaptive sgd. In The Thirty Sixth Annual Conference on Learning Theory , pages 89–160. PMLR, 2023

    2023

  3. [10]

    Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio

    Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adve rsarial nets. In Advances in Neural Information Processing Systems , volume 27, pages 2672–2680, 2014

    2014

  4. [11]

    Long short-term memory

    Alex Graves. Long short-term memory. In Supervised sequence labelling with recurrent neural networks, pages 37–45. Springer, 2012

    2012

  5. [12]

    Ac celerated zeroth-order and first-order momentum methods from mini to minimax optimizat ion

    Feihu Huang, Shangqian Gao, Jian Pei, and Heng Huang. Ac celerated zeroth-order and first-order momentum methods from mini to minimax optimizat ion. Journal of Machine Learning Research, 23(36):1–70, 2022. 23

    2022

  6. [13]

    Adagda: Faste r adaptive gradient descent ascent methods for minimax optimization

    Feihu Huang, Xidong Wu, and Zhengmian Hu. Adagda: Faste r adaptive gradient descent ascent methods for minimax optimization. In International Conference on Artificial Intel- ligence and Statistics , pages 2365–2389. PMLR, 2023

    2023

  7. [14]

    Parameter-agnostic optimization un- der relaxed smoothness

    Florian H¨ ubler, Junchi Yang, Xiang Li, and Niao He. Parameter-agnostic optimization un- der relaxed smoothness. In International Conference on Artificial Intelligence and Stati stics, pages 4861–4869. PMLR, 2024

    2024

  8. [15]

    A short note on concentration inequalities for random vectors with subg aussian norm

    Chi Jin, Praneeth Netrapalli, Rong Ge, Sham M Kakade, and Michael I Jordan. A short note on concentration inequalities for random vectors with subg aussian norm. arXiv preprint arXiv:1902.03736, 2019

    1902 arXiv

  9. [16]

    N on-convex distributionally robust optimization: Non-asymptotic analysis

    Jikai Jin, Bohang Zhang, Haiyang Wang, and Liwei Wang. N on-convex distributionally robust optimization: Non-asymptotic analysis. Advances in Neural Information Processing Systems, 34:2771–2782, 2021

    2021

  10. [17]

    Large-scale methods for dis- tributionally robust optimization

    Daniel Levy, Yair Carmon, John C Duchi, and Aaron Sidfor d. Large-scale methods for dis- tributionally robust optimization. In Advances in Neural Information Processing Systems , volume 33, pages 8847–8860. Curran Associates, Inc., 2020

    2020

  11. [18]

    Convex and non-convex optimization under generalized smoothness

    Haochuan Li, Jian Qian, Yi Tian, Alexander Rakhlin, and Ali Jadbabaie. Convex and non-convex optimization under generalized smoothness. Advances in Neural Information Processing Systems, 36:40238–40271, 2023

    2023

  12. [19]

    Con vergence of adam under relaxed assumptions

    Haochuan Li, Alexander Rakhlin, and Ali Jadbabaie. Con vergence of adam under relaxed assumptions. Advances in Neural Information Processing Systems , 36:52166–52196, 2023

    2023

  13. [20]

    Tiada: A time-scale a daptive algorithm for nonconvex minimax optimization

    Xiang Li, Junchi Yang, and Niao He. Tiada: A time-scale a daptive algorithm for nonconvex minimax optimization. In The Eleventh International Conference on Learning Representa - tions, ICLR , 2023

    2023

  14. [21]

    On gradient descent ascent for nonconvex-concave minimax problems

    Tianyi Lin, Chi Jin, and Michael Jordan. On gradient descent ascent for nonconvex-concave minimax problems. In International Conference on Machine Learning , pages 6083–6093. PMLR, 2020

    2020

  15. [22]

    Near-optimal non-convex stochastic optimization under generalized smoothness

    Zijian Liu, Srikanth Jagabathula, and Zhengyuan Zhou. Near-optimal non-convex stochastic optimization under generalized smoothness. arXiv preprint arXiv:2302.06032 , 2023

    2023 arXiv

  16. [23]

    Stoc hastic recursive gradient descent ascent for stochastic nonconvex-strongly-concave minimax problems

    Luo Luo, Haishan Ye, Zhichao Huang, and Tong Zhang. Stoc hastic recursive gradient descent ascent for stochastic nonconvex-strongly-concave minimax problems. In Advances in Neural Information Processing Systems, volume 33, pages 20566–20577. Curran Associates, Inc., 2020

    2020

  17. [24]

    Tight analysis of extra-gradient and optimistic gradient methods for nonc onvex minimax problems

    Pouria Mahdavinia, Yuyang Deng, Haochuan Li, and Mehrd ad Mahdavi. Tight analysis of extra-gradient and optimistic gradient methods for nonc onvex minimax problems. In Advances in Neural Information Processing Systems, volume 35, pages 31213–31225. Curran Associates, Inc., 2022. 24

    2022

  18. [25]

    Hongseok Namkoong and John C. Duchi. Stochastic gradie nt methods for distributionally robust optimization with f-divergences. In Advances in Neural Information Processing Systems, pages 2208–2216, 2016

    2016

  19. [26]

    Weakly-convex–concave min– max optimization: provable algorithms and applications in machine learning

    Hassan Rafique, Mingrui Liu, Qihang Lin, and Tianbao Yang. Weakly-convex–concave min– max optimization: provable algorithms and applications in machine learning. Optimization Methods and Software , 37(3):1087–1121, 2022

    2022

  20. [27]

    Variance-reduced clipping for non-convex optimization

    Amirhossein Reisizadeh, Haochuan Li, Subhro Das, and A li Jadbabaie. Variance-reduced clipping for non-convex optimization. InICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages 1–5. IEEE, 2025

    2025

  21. [28]

    Federated minimax optimization: Improved convergence analyses and algorith ms

    Pranay Sharma, Rohan Panda, Gauri Joshi, and Pramod Var shney. Federated minimax optimization: Improved convergence analyses and algorith ms. In International Conference on Machine Learning , pages 19683–19730. PMLR, 2022

    2022

  22. [29]

    Certifia ble distributional robustness with principled adversarial training

    Aman Sinha, Hongseok Namkoong, and John Duchi. Certifia ble distributional robustness with principled adversarial training. In International Conference on Learning Representa- tions, 2018

    2018

  23. [30]

    Mome ntum ensures convergence of signsgd under weaker assumptions

    Tao Sun, Qingsong Wang, Dongsheng Li, and Bao Wang. Mome ntum ensures convergence of signsgd under weaker assumptions. In Proceedings of the 40th International Conference on Machine Learning , volume 202 of Proceedings of Machine Learning Research , pages 33077–33099. PMLR, 2023

    2023

  24. [31]

    Theoretical ana lysis of adversarial learning: A minimax approach

    Zhuoran Tu, Jun Zhang, and Dacheng Tao. Theoretical ana lysis of adversarial learning: A minimax approach. In Advances in Neural Information Processing Systems , volume 32, pages 12248–12259, 2019

    2019

  25. [32]

    Co nvergence of adagrad for non-convex objectives: Simple proofs and relaxed assumpti ons

    Bohan Wang, Huishuai Zhang, Zhiming Ma, and Wei Chen. Co nvergence of adagrad for non-convex objectives: Simple proofs and relaxed assumpti ons. In The Thirty Sixth Annual Conference on Learning Theory , pages 161–190. PMLR, 2023

    2023

  26. [33]

    Solving a class of non-convex minimax optimization in federated lear ning

    Xidong Wu, Jianhui Sun, Zhengmian Hu, Aidong Zhang, and Heng Huang. Solving a class of non-convex minimax optimization in federated lear ning. In Advances in Neural Information Processing Systems , volume 36, pages 11232–11245, 2023

    2023

  27. [35]

    A stochastic gda method with backtracking for solving nonconvex (strongly) concave minimax problems, 2024

    Qiushui Xu, Xuan Zhang, Necdet Serhat Aybat, and Mert G¨ urb¨ uzbalaban. A stochastic gda method with backtracking for solving nonconvex (strongly) concave minimax problems, 2024

    2024

  28. [36]

    Vincent Poor

    Tengyu Xu, Zhe Wang, Yingbin Liang, and H. Vincent Poor. Enhanced first and zeroth order variance reduced algorithms for min-max optimizatio n, 2021. 25

    2021

  29. [37]

    Nest your adaptive al gorithm for parameter-agnostic nonconvex minimax optimization

    Junchi Yang, Xiang Li, and Niao He. Nest your adaptive al gorithm for parameter-agnostic nonconvex minimax optimization. In Advances in Neural Information Processing Systems , volume 35, pages 11202–11216, 2022

    2022

  30. [38]

    Faster single-loop algo- rithms for minimax optimization without strong concavity

    Junchi Yang, Antonio Orvieto, Aurelien Lucchi, and Nia o He. Faster single-loop algo- rithms for minimax optimization without strong concavity. In International Conference on Artificial Intelligence and Statistics , pages 5485–5517. PMLR, 2022

    2022

  31. [39]

    Impr oved analysis of clipping algo- rithms for non-convex optimization

    Bohang Zhang, Jikai Jin, Cong Fang, and Liwei Wang. Impr oved analysis of clipping algo- rithms for non-convex optimization. Advances in Neural Information Processing Systems , 33:15511–15521, 2020

    2020

  32. [40]

    Why gradient clipping accel- erates training: A theoretical justification for adaptivit y

    Jingzhao Zhang, Tianxing He, Suvrit Sra, and Ali Jadbab aie. Why gradient clipping accel- erates training: A theoretical justification for adaptivit y. In International Conference on Learning Representations (ICLR), 2020

    2020

  33. [41]

    Sapd+: An accelerated stochastic method for nonconvex-concave minimax problems

    Xuan Zhang, Necdet Serhat Aybat, and Mert Gurbuzbalaba n. Sapd+: An accelerated stochastic method for nonconvex-concave minimax problems. In Advances in Neural Infor- mation Processing Systems , volume 35, pages 21668–21681, 2022

    2022

  34. [42]

    Finding second-order stationary points for generalized-smooth nonconvex minimax optimization via gradient-based algorithm, 2025

    Qi Zhu, Yuanyuan Liu, Fanhua Shang, Hongying Liu, and Zh ouchen Lin. Finding second-order stationary points for generalized-smooth nonconvex minimax optimization via gradient-based algorithm, 2025. 26 Appendix In this section, we provide the detailed convergence analys is for ...

    2025

  35. [43]

    Then, by setting ηx = min { ǫ 48˜κ2Lx,0 , √ δ 32˜κ2Lx,1 } , ηy = 1 Ly , bx = 576σ2 x ǫ2 , by = max { 2304˜κσ2 y ǫ2L2 y , 512˜κL2 x,1σ2 y δL2 y } , it holds with probability at least 1 − δ that 1T T −1∑ t=0 E∥∇Φ(xt)∥ ≤ ǫ within at most T = max { 12∆Φ ηxǫ , 96Lx,0˜κ∆y,0 ǫ } iter...

  36. [44]

    Then, by setting ηx ≤ min    ǫ 96Lx,0˜κ2 √ log( 2e δ ) , ǫ 6(˜κ + 1)Lx,0 , 1 80Lx,1˜κ2 √ log (2e δ )    , ηy = 1 Ly , bx ≥ 46656σ2 x log( 4 δ ) ǫ2 , by ≥ max { 8192L2 x,1(cσy)2 log (2e δ ) µ2 , 12288L2 x,0(cσy)2 log (2e δ ) µ2ǫ2 } , it holds with probability at least 1 −...

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.