REVIEW 6 cited by
A Variational Inequality Perspective on Generative Adversarial Networks
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
A Variational Inequality Perspective on Generative Adversarial Networks
read the original abstract
Generative adversarial networks (GANs) form a generative modeling approach known for producing appealing samples, but they are notably difficult to train. One common way to tackle this issue has been to propose new formulations of the GAN objective. Yet, surprisingly few studies have looked at optimization methods designed for this adversarial training. In this work, we cast GAN optimization problems in the general variational inequality framework. Tapping into the mathematical programming literature, we counter some common misconceptions about the difficulties of saddle point optimization and propose to extend techniques designed for variational inequalities to the training of GANs. We apply averaging, extrapolation and a computationally cheaper variant that we call extrapolation from the past to the stochastic gradient method (SGD) and Adam.
Forward citations
Cited by 6 Pith papers
-
A path-following framework on fiber bundle for variational inequalities
Path-following algorithm on fixed-point fiber bundle for general finite-dimensional VIs with global linear convergence to nonsingular solutions and linear error reduction for singular ones.
-
Metric-Gradient Projection for Stable Multi-Agent Policy Learning
HPML projects multi-agent update fields onto the closest metric-gradient potential flow via Hodge decomposition, yielding Lyapunov potentials and equilibrium-gap bounds.
-
Training Deep Learning Models with Norm-Constrained LMOs
Scion is a new stochastic LMO-based optimizer family that unifies existing methods, supports unconstrained problems, and delivers hyperparameter transferability plus speedups on nanoGPT training.
-
Why SGD is not Brownian Motion: A New Perspective on Stochastic Dynamics
SGD is reformulated via a master equation from discrete updates, producing a discrete Fokker-Planck equation that predicts non-stationary variance growth proportional to learning rate in flat Hessian directions.
-
Mirror Descent-Type Algorithms for the Variational Inequality Problem with Functional Constraints
Mirror descent algorithms with productive/non-productive step switching achieve optimal convergence rates for bounded monotone operators and Lipschitz convex functional constraints in variational inequalities.
-
Methods for Solving Variational Inequalities with Markovian Stochasticity
Presents a stochastic extragradient algorithm for variational inequalities with Markovian noise, proving convergence under L-Lipschitzness, strong monotonicity, and noise bounded only at the optimum, plus experiments ...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.