REVIEW 3 cited by
ODE Analysis of Stochastic Gradient Methods with Optimism and Anchoring for Minimax Problems
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Despite remarkable empirical success, the training dynamics of generative adversarial networks (GAN), which involves solving a minimax game using stochastic gradients, is still poorly understood. In this work, we analyze last-iterate convergence of simultaneous gradient descent (simGD) and its variants under the assumption of convex-concavity, guided by a continuous-time analysis with differential equations. First, we show that simGD, as is, converges with stochastic sub-gradients under strict convexity in the primal variable. Second, we generalize optimistic simGD to accommodate an optimism rate separate from the learning rate and show its convergence with full gradients. Finally, we present anchored simGD, a new method, and show convergence with stochastic subgradients.
Forward citations
Cited by 3 Pith papers
-
Halpern Iteration Achieves $\tilde{\mathcal{O}}(\epsilon^{-1/p})$ $p$th-Order Oracle Complexity for Monotone Variational Inequalities
A large-step inexact Halpern iteration with an anchored tensor method yields tilde-O(epsilon^{-1/p}) p-th order oracle complexity for smooth monotone variational inequalities for all p >= 2.
-
Direct Acceleration of Stochastic Root-Finding Without Variance Reduction and Regularization
A dual-anchor stochastic root-finding algorithm achieves O(epsilon^{-3}) oracle complexity with constant mini-batching and no variance reduction for cocoercive operators.
-
Last-Iterate Convergence of Single-Loop Stochastic Methods for Constrained Convex-Concave Minimax Problems
Perturbed S-EG and S-OGDA achieve O(T^{-1/4}) last-iterate restricted primal-dual gap rates when T is known and O(T^{-1/5}) anytime rates under standard stochastic oracles.
Discussion (0). Continue with ORCID to comment.