A new quantized optimistic dual averaging algorithm with layer-wise adaptive compression is presented, with theoretical convergence guarantees for monotone variational inequalities and empirical speedups on distributed GAN training.
A Unified Analysis of Extra-gradient and Optimistic Gradient Methods for Saddle Point Problems: Proximal Point Approach
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
In this paper we consider solving saddle point problems using two variants of Gradient Descent-Ascent algorithms, Extra-gradient (EG) and Optimistic Gradient Descent Ascent (OGDA) methods. We show that both of these algorithms admit a unified analysis as approximations of the classical proximal point method for solving saddle point problems. This viewpoint enables us to develop a new framework for analyzing EG and OGDA for bilinear and strongly convex-strongly concave settings. Moreover, we use the proximal point approximation interpretation to generalize the results for OGDA for a wide range of parameters.
fields
cs.LG 1years
2025 1verdicts
REJECT 1representative citing papers
citing papers explorer
-
Layer-wise Quantization for Quantized Optimistic Dual Averaging
A new quantized optimistic dual averaging algorithm with layer-wise adaptive compression is presented, with theoretical convergence guarantees for monotone variational inequalities and empirical speedups on distributed GAN training.