REVIEW 4 major objections 4 minor 1 cited by
Doubly Smoothed Optimistic Gradients: A Universal Approach for Smooth Minimax Problems
T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read DS-OGDA, a single-loop first-order method, converges on all four smooth minimax classes with one parameter set and reaches optimal rates when the structure is known.
desk verdict A serious C-C analysis undermined by an empty parameter interval in the universality lemma; the paper deserves revision, not desk rejection. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The engine is the regularized function $F(x,y,z,v) = f(x,y) + \frac{r_x}{2}\|x-z\|^2 - \frac{r_y}{2}\|y-v\|^2$, whose extra variables $z, v$ are smoothed 'belief' states updated slowly by $z_{t+1} = z_t + \beta_x(x_t - z_t)$ and $v_{t+1} = v_t + \beta_y(y_t - v_t)$, together with the gradient operator $G_u = [\nabla_x F; -\nabla_y F; \nabla_z F; -\nabla_v F]$. The updates on $(x,y)$ add the optimistic gradient difference $\eta(G^{t-1} - G^t)$, which is what distinguishes the method from its predecessor. The proof runs on two tracks: for convex-concave problems, the updates are read as an inexact proximal point method applied to $G$, and a new boundedness result (Proposition 3.1) keeps the iterates inside a compact set; for nonconvex problems, a Lyapunov function built from primal descent, dual ascent, and the proximal terms is shown to decrease, with the one-sided KL inequalities (for instance $(\max_{y'\in Y} f(x,y') - f(x,y))^\theta \le \frac{1}{\tau_y}\operatorname{dist}(0, -\nabla_y f(x,y) + \partial \iota_Y(y))$) supplying the primal-dual error bound that keeps the two players in balance.
What would settle it
On the paper's own hard instance, $\min_{x\in\mathbb{R}} \max_{y\in[0,1]} \tfrac{1}{2}x^2y$, DS-OGDA should produce an $O(T^{-1/2})$-saddle point under the universal schedule and $O(T^{-1})$ under the tuned schedule; computing the exact rate on this one function would directly test the boundedness argument. The other edge is class coverage: since the paper offers no concrete nonconvex-nonconcave function satisfying its global uniform one-sided KL inequality, writing one down explicitly — or showing that none exists beyond effectively convex-concave cases — would settle how much of the nonconvex-nonconcave landscape the claim actually covers.
Extended reading notes
Core claim
The central result, Theorem 2.1, states that if at least one of the four one-sided structural assumptions holds — concavity in $y$, a one-sided Kurdyka–Łojasiewicz inequality in $y$, convexity in $x$, or the one-sided KL inequality in $x$ — then DS-OGDA, run with the symmetric parameter choice $r := r_x = r_y \ge 74L$, $\eta_x = \eta_y$, and $\beta_x = \beta_y \le O(T^{-1/2})$, produces an $O(T^{-1/4})$-game stationary point after $T$ iterations, and an $O(T^{-1/2})$-saddle point when both convexity and concavity hold. This universal schedule assumes a known positive lower bound $\epsilon$ on the KL exponent $\theta$. With structure verified, Theorem 2.2 improves the convex-concave rate to $O(T^{-1})$ (choosing $r \le O(T^{-1})$ and $\beta = \eta r$), matching the known lower bound for convex-concave saddle-point problems; Theorem 2.3 improves the nonconvex-concave rate to $O(T^{-1/((4\theta-2)_+ + 2)})$ under a one-sided KL exponent $\theta$ and keeps $O(T^{-1/4})$ under plain concavity; Corollary 2.1 states the mirror-image results for convex-nonconcave and KL-nonconcave problems. Section 4 closes the argument by proving that the same scheme without the extrapolation term, DS-GDA, achieves only $O(T^{-1/2})$ on a hard convex-concave instance of the form $\min_{x\in\mathbb{R}} \max_{y\in[0,1]} \tfrac{1}{2}x^2y$, and that this rate is tight — so the optimistic gradient difference is what purchases optimality.
Load-bearing premise
The nonconvex results rest on a global, uniform one-sided Kurdyka–Łojasiewicz inequality — one fixed exponent $\theta$ and one fixed constant $\tau$ must hold at every feasible point, not just near a solution — and the universality theorem additionally requires a known positive lower bound on that exponent, which is itself structural knowledge.
Editorial extensions
If this is right
- A practitioner who cannot verify whether the objective is convex-concave can run DS-OGDA with one fixed schedule and still be guaranteed an $O(T^{-1/4})$-game stationary point after $T$ iterations, without testing which of the four structural conditions holds.
- Once the structure is verified, the same algorithm attains the optimal $O(T^{-1})$ saddle-point rate for convex-concave problems and the best-known single-loop rates for the nonconvex classes.
- The extrapolation step is shown to be necessary: without it the doubly smoothed scheme is stuck at $O(T^{-1/2})$ in the convex-concave case, and the paper proves this rate tight on a concrete instance.
- The symmetric update means every nonconvex-concave theorem transfers to convex-nonconcave problems by swapping the players' roles, so the analysis is not repeated for the mirror class.
- In the nonconvex-concave case the rate improves from $O(T^{-1/4})$ to $O(T^{-1/((4\theta-2)_+ + 2)})$ when the one-sided KL exponent $\theta$ is known, interpolating between $O(T^{-1/2})$ and $O(T^{-1/4})$ as $\theta$ moves from $1/2$ to $1$.
Reading between the lines
- My reading: the headlined 'no prior structural knowledge' is stronger than Theorem 2.1 delivers, because Lemma 3.4 requires the user to know a positive lower bound $\epsilon$ on the KL exponent $\theta$ — a piece of structural information — even though the bound can be crude.
- The global uniform one-sided KL assumption — one exponent and one constant over the whole feasible region — is far stronger than the usual local KL property; the paper gives no concrete nonconvex-nonconcave function that satisfies it, so the size of the nonconvex-nonconcave territory actually covered remains an open question.
- The nonconvex half of the proof delegates heavy lifting: the appendix omits detailed derivations and imports several technical lemmas from the earlier DS-GDA analysis, so a reader checking the nonconvex claims would need to reconstruct those steps from the cited treatment.
- Note that the universal and the structure-tuned regimes are opposite parameter choices — $r \ge 74L$ versus $r \le O(T^{-1})$ — so the same algorithm is universal, but the same parameter set does not simultaneously deliver the optimal rate; an adaptive schedule that interpolates automatically between the two regimes would be the stronger claim.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces DS-OGDA, a single-loop, symmetric first-order method that combines the double smoothing of DS-GDA with an optimistic gradient extrapolation. It claims that with one fixed symmetric parameter choice the method converges to an O(T^{-1/4})-game stationary point across convex-concave, nonconvex-concave, convex-nonconcave, and nonconvex-nonconcave problems satisfying a one-sided KŁ condition, and to an O(T^{-1/2})-saddle point in the convex-concave case. When structure is known, the same algorithm is claimed to achieve optimal or best-known rates, including O(T^{-1}) for convex-concave problems. The convex-concave analysis in Section 3.1 is developed in detail via an inexact proximal-point interpretation. The nonconvex results are asserted to follow from a descent estimate (Proposition 3.2) whose derivation is delegated to previous work and largely omitted, and the universal symmetric-parameter result rests on Lemma 3.4.
Significance. If established, this would be a significant unification of minimax optimization theory: a single-loop method with a single parameter schedule that automatically interpolates between convex-concave and nonconvex regimes would be practically valuable and would simplify a fragmented literature. The algorithmic idea, adding extrapolation to DS-GDA, is natural, and the C-C proof in Section 3.1 is careful and mostly self-contained. The paper is also explicit about the strong global form of the KŁ assumptions it uses. However, the load-bearing universal-parameter lemma is internally inconsistent as printed, the key nonconvex descent estimate is not proved in the manuscript, and the advertised "no prior structural knowledge" property is not what the theorems deliver. Because these issues touch the paper's central claims, the current version does not provide a verifiable basis for its main conclusions.
major comments (4)
- [Section 3.3, Lemma 3.4] The displayed interval for η is inconsistent at the claimed parameter range. With L=1 and r=74, the lower bound in Lemma 3.4 evaluates to (73+2665)/(108·5476)≈4.63×10^{-3}, while the upper bound is min{1/(8√5·74),1/(20·1·7)}≈7.55×10^{-4}; no η satisfies both. The sign in front of the square root in (20) also appears to assign the wrong quadratic root, and with the other root the interval becomes nonempty only for much larger r, if at all within the remaining step-size limits. Since Theorem 2.1's existence of a symmetric parameter set depends entirely on this lemma, the universal applicability claim is not established as written.
- [Section 3.2 / Appendix B, Proposition 3.2] The nonconvex convergence results (Theorem 2.3, Corollary 2.1, and Theorem 2.1(i)) rest on Proposition 3.2, but its proof is not given. Appendix B states "The detailed derivation is omitted" and the proof is reduced to copying the coefficient analysis from Zheng et al. [2023, Theorem 1]; Lemmas A.2 and A.3 are likewise stated with "we omit the detailed proofs." Since Proposition 3.2 is the algorithm-specific descent estimate for the new DS-OGDA update, this is not a routine citation: the extrapolation changes the error terms, and the manuscript itself flags that the derivation is missing. A reader cannot verify the central nonconvex claim from the submitted text.
- [Theorem 2.1 and Lemma 3.4] The abstract's claim that DS-OGDA works without prior structural knowledge is not supported by the theorem as stated. Theorem 2.1 requires the user to know a positive lower bound ε on the KŁ exponent θ, and Lemma 3.4's third bound on β contains (β(r−L))^{1/(2ε)} and diam^{1/ε−2}, so enforcing β≤O(T^{-1/2}) requires a known lower bound on θ. Since θ may be arbitrarily small and is not assumed known, the "universal single set of parameters" is not parameter-free. The displayed β constraint is also self-referential, with β appearing on both sides, so it is not an explicit admissible range.
- [Section 4, Theorem 4.2] The claimed tightness proof for GDA does not deliver the stated lower bound. After setting x0=√(2ε), the proof only shows that the saddle-point gap is at least (1−η)^{2t}ε, which is ≤ε; this gives no contradiction with the requirement that the gap be ≤ε, and it does not imply T=Ω(ε^{-2}). The displayed inference "T=Ω(1/η)=Ω(1/ε^2)" is also algebraically unjustified because η=O(T^{-1/2}) makes 1/η depend on T. This result is not needed for the main convergence theorems, but it is the paper's evidence that the DS-GDA bound is tight and should be corrected or removed.
minor comments (4)
- [Theorem 2.1] The symbol ε is used both for the target accuracy and for the lower bound on the KŁ exponent θ; renaming one of them (for example, using θ0 for the KŁ lower bound) would avoid confusion.
- [Lemma 3.4] The third displayed constraint on β contains β on the right-hand side and therefore is not an explicit upper bound; it should be solved for β or restated as an implicit condition with a clear existence argument.
- [Assumptions 2.2(ii) and 2.3(ii)] These assumptions require a single global KŁ exponent θ and constants τx,τy to hold uniformly over all x∈X and y∈Y. Since Theorem 2.1 includes nonconvex-nonconcave problems satisfying these assumptions, the paper should state whether any genuinely nonconvex-nonconcave example satisfies them and, if so, provide one.
- [Definition 2.1(iii)] The ε-optimization stationary point is defined through prox_{1/r·φ}(x), but φ(x)=max_{y'∈Y} f(x,y')+ι_X(x) is generally nonconvex, so the proximal map may be set-valued or not uniquely defined; the definition should specify how the proximal point is selected or use a distance-to-set formulation.
Circularity Check
Nonconvex universality is imported from the authors' own DS-GDA paper, but the C-C derivation is self-contained; no prediction reduces to a fit.
-
self citation load bearing
[Section 3.2 (after Proposition 3.2) and Appendix B]
"To control the negative term in (18), we introduce a primal-dual error bound, which has been comprehensively studied in our prior work [Zheng et al., 2023, Propositions 2]. For completeness, we state the error bound result below. ... The overall argument parallels that of Zheng et al. [2023, Theorem 1]; the only changes required for DS-OGDA appear in Lemmas A.1, A.2, and A.3. The detailed derivation is omitted, and we directly obtain:"
The nonconvex convergence results (Theorem 2.3, Corollary 2.1, and consequently the universal guarantee Theorem 2.1(i)) rest on Proposition 3.2, the basic descent estimate (18), and Proposition 3.3, the primal-dual error bound. Proposition 3.3 is not proved in this manuscript; it is imported from the authors' own DS-GDA paper. Proposition 3.2 is asserted with 'the detailed derivation is omitted' after stating that the argument parallels [Zheng et al. 2023, Theorem 1]. Thus the new nonconvex derivation chain terminates in a same-author citation rather than in equations derived here. This is load-bearing self-citation.
full rationale
The paper's main universality claim is not circular in the sense of fitting predictions to targets or defining an output as its input. The convex-concave analysis (Theorem 2.2) is derived in the text using the inexact proximal point framework of Mokhtari et al. and is benchmarked against standard OGDA; it does not reduce to a fitted quantity. The universal symmetric-parameter claim (Theorem 2.1) is a union of case analyses rather than a self-referential construction. The principal circularity-adjacent issue is the nonconvex half: Proposition 3.3 is lifted from the authors' own prior work, and Proposition 3.2's proof is partly delegated to the same prior framework with the detailed derivation omitted. That makes the nonconvex guarantees rely on load-bearing self-citation, but it is not an equation-level reduction of a result to its own assumptions. The printed interval in Lemma 3.4 appears internally inconsistent as written, and Theorem 2.1's stated requirement theta in [epsilon,1] conflicts with the abstract's claim of eliminating prior structural knowledge; these are correctness and presentation issues rather than circularity. Overall, the central C-C contribution stands independently, while the nonconvex universality rests on a same-author citation chain, so a moderate score of 4 is appropriate.
Assumptions & free parameters
free parameters (5)
- eta_x, eta_y (step sizes) =
O(1) or O(T^{-1/2}) depending on theorem
- r_x, r_y (smoothing parameters) =
r>=74L in universal setting; O(1) elsewhere
- beta_x, beta_y (smoothing step sizes) =
<= O(T^{-1/2}) universally; beta=eta*r and <= O(T^{-1}) for C-C
- epsilon (lower bound on KL exponent theta) =
any positive number; parameters depend on it
- KL exponent theta and constants tau_x, tau_y =
assumed known for improved rates
assumptions (4)
- domain assumption Assumption 2.1: gradients of f are Lx/Ly-Lipschitz on X x Y with L>=1
- domain assumption Assumption 2.2/2.3: at least one one-sided condition holds (concavity, convexity, or uniform KL inequality)
- domain assumption Assumption 2.4: existence of a saddle point for C-C problems with noncompact sets
- domain assumption Zheng et al. [2023, Theorem 1, Propositions 2 and 6]; Zhang et al. [2020]; Li et al. [2025] results are correct and applicable
Cite this review
Pith. "Pith review of Doubly Smoothed Optimistic Gradients: A Universal Approach for Smooth Minimax Problems." pith.science (2026). https://pith.science/paper/XXD3NETV
@misc{pith2026250607397,
author = {Pith},
title = {Pith review of: Doubly Smoothed Optimistic Gradients: A Universal Approach for Smooth Minimax Problems},
year = {2026},
howpublished = {\url{https://pith.science/paper/XXD3NETV}},
note = {Machine review of arXiv:2506.07397}
}
read the original abstract
Smooth minimax optimization problems play a central role in a wide range of applications, including machine learning, game theory, and operations research. However, existing algorithmic frameworks vary significantly depending on the problem structure -- whether it is convex-concave, nonconvex-concave, convex-nonconcave, or even nonconvex-nonconcave with additional regularity conditions. In particular, this diversity complicates the tuning of step-sizes since even verifying convexity (or concavity) assumptions is challenging and problem-dependent. We introduce a universal and single-loop algorithm, Doubly Smoothed Optimistic Gradient Descent Ascent (DS-OGDA), that applies to a broad class of smooth minimax problems. Specifically, this class includes convex-concave, nonconvex-concave, convex-nonconcave, and nonconvex-nonconcave minimax optimization problems satisfying a one-sided Kurdyka-Lojasiewicz (KL) property. DS-OGDA works with a universal single set of parameters for all problems in this class, eliminating the need for prior structural knowledge to determine step-sizes. Moreover, when a particular problem structure in our class is specified, DS-OGDA achieves optimal or best-known performance guarantees. Overall, our results provide a comprehensive and versatile framework for smooth minimax optimization, bridging the gap between convex and nonconvex problem structures and simplifying the choice of algorithmic strategies across diverse applications.
Forward citations
Cited by 1 Pith paper
-
A first-order method for constrained nonconvex-nonconcave minimax optimization
Under a local Kurdyka-Łojasiewicz condition, the constrained nonconvex-nonconcave minimax value function is locally generalized Hölder smooth, and an interleaved SCP/proximal-gradient method achieves Õ(ε^{−max{1/(1−θ)...
Reference graph
Works this paper leans on
-
[1]
M. Arjovsky, S. Chintala, and L. Bottou. Wasserstein generative adversarial networks. InProceedings of the 34th International Conference on Machine Learning (ICML 2017), pages 214–223. PMLR,
work page 2017
-
[11]
P . Mertikopoulos, B. Lecouat, H. Zenati, C.-S. Foo, V . Chandrasekhar, and G. Piliouras. Optimistic mirror descent in saddle-point problems: Going the extra (gradient) mile. InProceedings of the 6th International Conference on Learning Representations (ICLR 2018),
work page 2018
-
[13]
23 T. Pethick, P . Patrinos, O. Fercoq, V . Cevherå, et al. Escaping limit cycles: Global convergence for constrained nonconvex-nonconcave minimax problems. InProceedings of the 10th International Conference on Learning Representations (ICLR 2022),
work page 2022
-
[14]
Our analysis builds on the framework developed in Li et al
24 Appendix In the Appendix, we provide the missing proofs for nonconvex minimax problems. Our analysis builds on the framework developed in Li et al. [2025], Zheng et al. [2023], and incorporates several key lemmas from Zheng et al. [2023]. Below, we highlight only the components that are specific to our new algorithms. A Some Useful Lemmas Lemma A.1.For...
work page 2025
- [2007]
-
[2011]
J. Blanchet, J. Li, S. Lin, and X. Zhang. Distributionally robust optimization and robust statistics. arXiv preprint arXiv:2401.14655,
-
[2018]
C. Daskalakis, A. Ilyas, V . Syrgkanis, and H. Zeng. Training GANs with optimism. InProceedings of the 6th International Conference on Learning Representations (ICLR 2018),
work page 2018
-
[2019]
S. Omidshafiei, J. Pazis, C. Amato, J. P . How, and J. Vian. Deep decentralized multi-task multi- agent reinforcement learning under partial observability. InProceedings of the 34th International Conference on Machine Learning (ICML 2017), pages 2681–2690. PMLR,
work page 2017
Show all 14 references
-
[2020]
C. Jin, P . Netrapalli, and M. Jordan. What is local optimality in nonconvex-nonconcave minimax optimization? InProceedings of the 37th International Conference on Machine Learning (ICML 2020), pages 4880–4889. PMLR,
2020
-
[2021]
B. Dai, A. Shaw, L. Li, L. Xiao, N. He, Z. Liu, J. Chen, and L. Song. Sbeed: Convergent reinforcement learning with nonlinear function approximation. InProceedings of the 35th International Conference on Machine Learning (ICML 2018), pages 1125–1134. PMLR,
2018
-
[2022]
Y. Cai, A. Oikonomou, and W. Zheng. Accelerated algorithms for constrained nonconvex- nonconcave min-max optimization and comonotone inclusion. InProceedings of the 41st In- ternational Conference on Machine Learning (ICML 2024), pages 5312–5347. PMLR,
2024
-
[2023]
T. Lin, C. Jin, and M. Jordan. On gradient descent ascent for nonconvex-concave minimax problems. InProceedings of the 37th International Conference on Machine Learning (ICML 2020), pages 6083–6093. PMLR, 2020a. T. Lin, C. Jin, and M. I. Jordan. Near-optimal algorithms for min...
2020
-
[2024]
Z. Chen, Y. Zhou, T. Xu, and Y. Liang. Proximal gradient descent-ascent: Variable convergence under kł geometry. InProceedings of the 9th International Conference on Learning Representations (ICLR 2021),
2021
-
[2025]
Li and G
22 T. Li and G. Lan. A simple uniformly optimal method without line search for convex optimization. arXiv preprint arXiv:2310.10082,
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.