Derives ODE limits of Adam-DA showing that first- and second-order momentum parameters reverse their convergence roles in zero-sum games compared to minimization, validated on GAN experiments.
arXiv preprint arXiv:1803.06573 , year=
7 Pith papers cite this work. Polarity classification is still indexing.
7
Pith papers citing it
abstract
We provide a simple proof for the Fenchel duality between strong convexity and Lipschitz continuous gradient. To this end, we first establish equivalent conditions of convexity for a general function that may not be differentiable. By utilizing these equivalent conditions, we can directly obtain equivalent conditions for strong convexity and Lipschitz continuous gradient. Based on these results, we can easily prove Fenchel duality. Beside this main result, we also identify several conditions that are implied by strong convexity or Lipschitz continuous gradient, but are not necessarily equivalent to them. This means that these conditions are more general than strong convexity or Lipschitz continuous gradient themselves.
verdicts
UNVERDICTED 7representative citing papers
The paper establishes the first tilde O(epsilon^{-1}) upper bounds and matching lower bounds for forward-KL-regularized offline contextual bandits under single-policy concentrability in both tabular and general function approximation settings.
Semi-discrete Flow Matching produces terminal assignment regions that are topologically simple (open, simply connected, homeomorphic to the ball under assumption) yet geometrically distinct from optimal transport Laguerre cells, as they can be non-convex with curved boundaries.
The Dual² approach produces iD2A and MiD2A gradient methods that achieve asymptotic convergence under milder conditions on the public function and linear rates with reduced communication and computation complexity.
Introduces a Wasserstein-space multivariate autoregressive model for distributional time series, proving second-order stationarity via iterated random functions and providing a simplex-constrained consistent estimator that enables graph learning.
i-DEQ adds momentum to DEQ fixed-point iterations, yielding convergence guarantees, training stability, and halved inference time while matching state-of-the-art reconstruction quality on inverse problems.
A fully distributed primal-dual algorithm solves nonsmooth strongly convex problems with coupled constraints on time-varying digraphs at O(1/k) rate without communicating primal variables.
citing papers explorer
-
Understanding Dynamics of Adam in Zero-Sum Games: An ODE Approach
Derives ODE limits of Adam-DA showing that first- and second-order momentum parameters reverse their convergence roles in zero-sum games compared to minimization, validated on GAN experiments.
-
Fast Rates for Offline Contextual Bandits with Forward-KL Regularization under Single-Policy Concentrability
The paper establishes the first tilde O(epsilon^{-1}) upper bounds and matching lower bounds for forward-KL-regularized offline contextual bandits under single-policy concentrability in both tabular and general function approximation settings.
-
Tessellations of Semi-Discrete Flow Matching
Semi-discrete Flow Matching produces terminal assignment regions that are topologically simple (open, simply connected, homeomorphic to the ball under assumption) yet geometrically distinct from optimal transport Laguerre cells, as they can be non-convex with curved boundaries.
-
Accelerated Decentralized Constraint-Coupled Optimization: A Dual$^2$ Approach
The Dual² approach produces iD2A and MiD2A gradient methods that achieve asymptotic convergence under milder conditions on the public function and linear rates with reduced communication and computation complexity.
-
Wasserstein multivariate auto-regressive models for modeling distributional time series
Introduces a Wasserstein-space multivariate autoregressive model for distributional time series, proving second-order stationarity via iterated random functions and providing a simplex-constrained consistent estimator that enables graph learning.
-
i-DEQ: A stable inertial deep equilibrium model for image restoration
i-DEQ adds momentum to DEQ fixed-point iterations, yielding convergence guarantees, training stability, and halved inference time while matching state-of-the-art reconstruction quality on inverse problems.
-
Distributed Optimization with Coupled Constraints over Time-Varying Digraph
A fully distributed primal-dual algorithm solves nonsmooth strongly convex problems with coupled constraints on time-varying digraphs at O(1/k) rate without communicating primal variables.