Pith. sign in

REVIEW 3 major objections 4 minor 53 references

Convergence of Gradient Methods on Bilinear Zero-Sum Games

T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read For bilinear zero-sum games, the paper proves exact necessary-and-sufficient conditions for linear convergence of extra-gradient, optimistic gradient descent, and momentum methods, and shows that alternating (Gauss-Seidel) updates…

desk verdict Solid theory paper: exact convergence conditions for gradient methods on bilinear games, but two load-bearing steps rest on unverified computer algebra and a numerical asymptotic. read the letter →

arxiv 1908.05699 v4 pith:ULV25VIV submitted 2019-08-15 cs.LG cs.GTmath.OCstat.ML

Guojun Zhang , Yaoliang Yu This is my paper · ORCID
classification cs.LGcs.GTmath.OCstat.ML MSC 91A0590C4765F1065K05
keywords bilinearzero-sumgamesextra-gradientmethodoptimisticgradientdescentmomentumGauss-SeidelvsJacobiupdatesSchurstabilitylinearconvergenceratessaddlepoint
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper studies the simplest adversarial setting, bilinear zero-sum games min_x max_y x^T E y, and asks exactly when popular gradient algorithms converge. It proves necessary and sufficient conditions for linear convergence of generalized extra-gradient, optimistic gradient descent, momentum, and plain gradient descent, under both simultaneous (Jacobi) and alternating (Gauss-Seidel) parameter updates; the conditions are polynomial inequalities in the singular values of E and the algorithm's step-size and momentum parameters. The main comparative result is that alternating updates converge on a strictly larger set of parameters than simultaneous updates for OGD and momentum, and for extra-gradient under a stated condition. The paper also computes the optimal linear convergence rate for each algorithm as a function of the condition number of E, showing, for example, that Gauss-Seidel OGD improves the rate exponent from 1/($6κ^{2}$) to 1/$κ^{2}$ compared with Jacobi OGD. A reader should care because these are the first exact stability tests for this problem class, and they extend the classical Stein-Rosenberg theorem — that alternating beats simultaneous iteration — from linear systems to bilinear games.

What carries the argument

The load-bearing device is Schur's stability test (Theorem 2.2): a real polynomial has all roots inside the open unit disk — equivalently, the iterates converge linearly — iff a list of determinants constructed from its coefficients are all positive. For quadratic, cubic, and quartic polynomials the paper writes these out as explicit inequalities (Corollary 2.1). The second device is Theorem 2.3, which relates the characteristic polynomial of simultaneous (Jacobi) updates p(λ,1) to that of alternating (Gauss-Seidel) updates p(λ,λ), turning the algorithm comparison into a comparison of two one-variable polynomials. The paper then only needs to check the Schur inequalities for each algorithm.

What would settle it

Run a rigorous independent check of the OGD inclusion: use exact computer algebra (or a certified interval method) to evaluate the universally quantified implication in Appendix C.4.5, or search for a concrete quadruple (α, β1, β2, σ) that satisfies the Jacobi conditions (3.8) but violates the Gauss-Seidel conditions (3.9). Finding such a parameter point would refute Theorem 3.3's containment; reproducing it with a certificate would confirm the paper's claim.

Watch

Extended reading notes

Core claim

The paper's central claim is that for the bilinear zero-sum game min_x max_y x^T E y with invertible E, convergence of generalized gradient-family algorithms is fully governed by Schur stability of a polynomial in one variable derived from the update rule, so the exact convergence region in parameter space is a set of polynomial inequalities in the singular values of E. Specifically, extra-gradient and optimistic gradient descent converge linearly under explicitly stated inequalities (Theorems 3.2 and 3.3), Gauss-Seidel momentum converges exactly when a stated inequality system holds while Jacobi momentum and both variants of plain gradient descent never converge (Theorems 3.1 and 3.4). The paper further shows that for OGD the Gauss-Seidel convergence region strictly contains the Jacobi one, and for EG the containment holds when β1+β2+$α^{2}$ < 2/$σ1^{2}$ (Theorem 3.2). On rates, the paper computes the optimal linear convergence exponents: extra-gradient reaches ($κ^{2}$−1)/($κ^{2}$+1), Gauss-Seidel OGD reaches about 1−1/$κ^{2}$, Jacobi OGD about 1−1/($6κ^{2}$), and Gauss-Seidel momentum about 1−2/($9κ^{2}$) for large condition numbers κ.

Load-bearing premise

The claim that the OGD Gauss-Seidel convergence region strictly contains the Jacobi region (Theorem 3.3) is finished by a computer-algebra verification of a quantified polynomial implication, without a machine-checkable certificate, so the claim depends on that computation (and the faithful reduction to polynomial conditions) being correct.

Editorial extensions

If this is right

  • For OGD, momentum, and EG (under its stated condition), any parameter choice that makes simultaneous updates converge also makes alternating updates converge — and often faster — so practitioners can safely switch to GS updates.
  • Since the conditions are exact, they provide a test for whether a given step-size and momentum choice will converge, without running the algorithm.
  • The optimal-rate results specify parameter settings that achieve the best linear exponent, including cases where the classical choice α = 2β for OGD is suboptimal.
  • The r-Schur scaling lemma gives a numerical procedure to locate the best linear rate for any of the studied algorithms by binary search on the spectral radius.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same Schur-polynomial machinery should transfer to local convergence analysis of smooth non-bilinear games, where the Jacobian at the saddle point plays the role of E; the paper's conditions would then be local and the GS-vs-Jacobi comparison would carry over near stable equilibria.
  • The singular-matrix appendix shows the exact conditions extend to rectangular E by projecting onto the range; one testable consequence is that the same convergence regions hold for the effective nonzero singular values.
  • The finding that momentum requires at least one negative momentum parameter in bilinear games suggests that the success of negative momentum in GAN training is not incidental, and that tuning β1 ≠ β2 asymmetrically could yield rates between the analyzed special cases.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper gives a systematic stability analysis of gradient-type algorithms for bilinear zero-sum games min_x max_y x^T E y. For generalized versions of gradient descent, extra-gradient (EG), optimistic gradient descent (OGD), and heavy-ball momentum, the authors derive characteristic polynomials for both simultaneous (Jacobi) and alternating (Gauss-Seidel) updates, invoke Schur stability conditions to obtain necessary and sufficient convergence conditions, and compare the Jacobi and Gauss-Seidel convergence regions. They also derive optimal linear rates for EG, for special cases of OGD, and for Gauss-Seidel momentum, and support the predictions with experiments on bilinear games and GANs. The advertised central claims are: exact convergence regions for these algorithms (Theorems 3.2-3.4), a Stein-Rosenberg-type containment of the Jacobi region in the Gauss-Seidel region (Theorem 3.3 and Table 1), and improved convergence rates for generalized parameter choices (Theorems 4.1-4.3 and Table 2).

Significance. If the theorems are correct, this is a substantial contribution: it provides the first exact, complete stability characterization for several widely used algorithms on the prototypical bilinear zero-sum game, with explicit characteristic polynomials, necessary and sufficient Schur conditions, and a unified Jacobi/Gauss-Seidel comparison that extends the classical Stein-Rosenberg insight to a non-monotone setting. The paper also ships reproducible experiments and a numerical search method for optimal rates. The main caveats are that one load-bearing containment proof relies on an unverified computer-algebra result and one advertised optimal rate is supported only by numerical observation; these issues are local but must be fixed before the results can be accepted as theorem-level claims.

major comments (3)
  1. [C.4.5 (Theorem 3.3)] The central containment claim that the Jacobi OGD convergence region is contained in the Gauss-Seidel region, advertised as 'yes' in Table 1, is discharged by a single quantified Mathematica Reduce call returning True, together with additional Reduce outputs in C.4.1-C.4.4. No independently checkable certificate is supplied, and the paper only notes that the result could in principle be verified manually via cylindrical algebraic decomposition. Since this containment is a load-bearing advertised result, please provide either a full human-readable algebraic proof of the implication from (3.8) to (3.9) or a machine-checkable certificate, and state the exact assumptions, options, and version of the computer algebra system used.
  2. [Theorem 4.2 and Eq. (D.18)] Equation (4.1) cannot be correct as printed. Setting sigma_1 = 1 and letting sigma_n tend to 0 gives r_*^2 -> 1/2 + (1/4) sqrt(2 sqrt(8)) approximately 1.094, which is larger than 1, while the theorem simultaneously asserts r_* ~ 1 - 1/(6 kappa^2), which requires r_*^2 -> 1 from below. The displayed expression is also dimensionally inconsistent for a squared spectral radius. This is a load-bearing error in the claimed optimal Jacobi OGD rate; please re-derive the third case in D.2.1 and correct (4.1) and (D.18).
  3. [D.3 (Theorem 4.3)] The claimed optimal rate r_* ~ 1 - 2/(9 kappa^2) for Gauss-Seidel momentum is supported only by the sentence 'Numerically, we find...' after Eq. (D.25). No proof of the asymptotic is given, yet Theorem 4.3 is presented without qualification and the rate appears in Table 2. Please provide an analytic derivation of the asymptotic, or explicitly reclassify this result as a numerical conjecture and state the supporting computation and accuracy.
minor comments (4)
  1. [D.2.1] After Eq. (D.15), the text says the minimum is achieved at alpha = 2 beta and then says 'From now on, we assume alpha <= 2 beta'. This is confusing; please clarify that the alpha >= 2 beta branch is minimized at its boundary alpha = 2 beta and that the remaining analysis covers the other branch.
  2. [C.4] The Reduce outputs labelled 'False.' and 'True.' should state that these are exact symbolic quantifier-elimination results rather than numerical evaluations, and should be accompanied by the Mathematica version and any relevant assumptions on the variables.
  3. [Theorem 4.1] The optimal EG rate is described as being achieved at alpha -> 0; since alpha = 0 with beta fixed corresponds to a limiting idealized update, please state explicitly whether the rate is attained by any finite admissible parameter configuration or only approached in the limit, and adjust the wording in Table 2 accordingly.
  4. [Appendix C.1 and Theorem 2.3] The notation 'taking alpha_2 -> lambda alpha_2' in the derivation of the Gauss-Seidel polynomials is terse; a short explanation that this is an application of Theorem 2.3 with the lower block-triangular part shifted would improve readability.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper derives convergence regions and rates from the update rules via characteristic polynomials and Schur stability, with no fitted inputs or load-bearing self-citations.

full rationale

The paper's central claims are self-contained derivations. For each algorithm (GD, EG, OGD, momentum), the authors first write the update as a linear dynamical system, compute the characteristic polynomial (equations 3.1, 3.2, 3.3, 3.6, 3.7, 3.10, 3.11 in the main text, with derivations in Appendix C.1), then apply the standard Schur stability criterion (Theorem 2.2, an external classical result) to obtain necessary and sufficient convergence conditions (Theorems 3.1-3.4). The containment claims compare the resulting Jacobi and Gauss-Seidel regions by logical implication; the proof of Theorem 3.3 uses a Mathematica Reduce quantifier-elimination check (Appendix C.4.5), which is a computational proof step and not an input fitted to reproduce the conclusion. The optimal-rate results (Theorems 4.1-4.3) solve genuine min-max optimizations over the spectral radius formula, again derived from the characteristic polynomials, not fitted to data. Experiments in Section 5 validate rather than define the theoretical predictions. The only weakness is that Theorem 4.3's asymptotic rate r* ~ 1 - 2/(9 kappa^2) is supported by a numerical observation in Appendix D.3 ('Numerically, we find...'), and the containment proof in Theorem 3.3 relies on a non-machine-checkable computer algebra output. These are rigor gaps or correctness risks, not circularity: neither the convergence conditions nor the rates are defined in terms of the target claim, and no parameter is fitted from data and then renamed as a prediction. There are no load-bearing self-citations, and the cited external results (Schur 1917, Cheng & Chiou 2007, Gohberg et al. 1982) are standard theorems with stated assumptions that do not include the paper's conclusions.

Assumptions & free parameters 2 free parameters · 6 assumptions · 0 invented entities

The central claims do not rely on fitted data parameters. The key external inputs are classical spectral and stability theorems. The ledger flags two paper-specific inputs: reliance on Mathematica for a containment proof, and a numerically inferred momentum rate, both of which should be independently verified.

free parameters (2)
  • EG optimal step-size limit = alpha -> 0, gamma -> infinity, beta = 2/(sigma_1^2 + sigma_n^2)
    The optimal EG rate is only attained in this limiting parameter regime, so the rate is a limiting property of the generalized algorithm rather than a practical finite-step recommendation.
  • GS momentum parameter restriction = beta1 = -1/2, beta2 = 0
    The momentum optimal-rate theorem is derived only for this special case inherited from Gidel et al. (2019b); it is not shown to be globally optimal in beta1 and beta2.
assumptions (6)
  • standard math Schur stability conditions (Corollary 2.1 and Theorem C.1) exactly characterize roots in the unit disk.
    Used in all Section 3 proofs to convert spectral radius conditions to coefficient inequalities.
  • standard math A multi-step LDS converges for all initializations iff its characteristic polynomial spectral radius is below 1 (Theorem 2.1).
    This bridges the gradient updates to polynomial root analysis; quoted from Gohberg et al. (1982).
  • standard math Block determinant formula with commuting C and D (Lemma C.1) holds.
    Used to derive all characteristic polynomials in Appendix C.1.
  • domain assumption Mathematica Reduce outputs in Appendices C.4 and C.5 are correct.
    The OGD and momentum region proofs depend on CAS quantifier elimination; the paper provides snippets but no independent certificate.
  • ad hoc to paper The asymptotic expansion in Theorem 4.3 is valid.
    The rate 1 - 2/(9 kappa^2) is observed numerically from root functions, not proven in closed form.
  • domain assumption The game matrix E is invertible in the main text; the singular case is handled in Appendix G.
    The main theorems assume invertibility; the singular extension projects onto range spaces without reworking the proofs.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Convergence of Gradient Methods on Bilinear Zero-Sum Games." pith.science (2026). https://pith.science/paper/ULV25VIV

@misc{pith2026190805699,
  author       = {Pith},
  title        = {Pith review of: Convergence of Gradient Methods on Bilinear Zero-Sum Games},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ULV25VIV}},
  note         = {Machine review of arXiv:1908.05699}
}
read the original abstract

Min-max formulations have attracted great attention in the ML community due to the rise of deep generative models and adversarial methods, while understanding the dynamics of gradient algorithms for solving such formulations has remained a grand challenge. As a first step, we restrict to bilinear zero-sum games and give a systematic analysis of popular gradient updates, for both simultaneous and alternating versions. We provide exact conditions for their convergence and find the optimal parameter setup and convergence rates. In particular, our results offer formal evidence that alternating updates converge "better" than simultaneous ones.

Figures

Figures reproduced from arXiv: 1908.05699 by the authors.

Figure 1
Figure 1. Left: linear convergence of optimal EG, Jacobi OGD, Gauss–Seidel OGD in a bilinear game with the log distance; Middle: comparison among Adam, SGD and EG in learning the mean of a Gaussian with WGAN with the squared distance; Right: Comparison between EG with (α = 0.02, γ = 2.0) and without scaling (α = γ = 0.2). We use the squared distance [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. Heat maps of the spectral radii of different algorithms. We take [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Jacobi vs. GS updates. y-axis: Squared distance ||φ − v||2 . x-axis: Number of epochs. Left: EG with γ = 0.2, α = 0.02; Middle: OGD with α = 0.2, β1 = 0.1, β2 = 0; Right: Momentum with α = 0.08, β = −0.1. We plot only a few epochs for Jacobi if it does not converge [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Test samples from the generator network trained with stochastic GD (step size [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Test samples from the generator network trained with stochastic OGD ( [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Test samples generated from the generator network trained with stochastic Adam. [PITH_FULL_IMAGE:figures/full_fig_p027_6.png]
Figure 7
Figure 7. Figure 7: Contour plot of spectral radius equal to [PITH_FULL_IMAGE:figures/full_fig_p027_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

53 extracted references · 44 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Arjovsky, S

    M. Arjovsky, S. Chintala, and L. Bottou. Wasserstein generative adversarial networks. In International Conference on Machine Learning, 2017

    work page 2017

  3. [3]

    K. J. Arrow, L. Hurwicz, and H. Uzawa. Studies in linear and non-linear programming. Stanford University Press, 1958

    work page 1958

  4. [4]

    J. P. Bailey and G. Piliouras. Multiplicative weights update in zero-sum games. In Proceedings of the 2018 ACM Conference on Economics and Computation, pp.\ 321--338. ACM, 2018

    work page 2018

  5. [5]

    J. P. Bailey, G. Gidel, and G. Piliouras. Finite regret and cycles with fixed step-size via alternating gradient descent-ascent. arXiv preprint arXiv:1907.04392, 2019

    work page Pith review arXiv 1907

  6. [6]

    R. E. Bruck. On the weak convergence of an ergodic iteration for the solution of variational inequalities for monotone operators in Hilbert space https://doi.org/10.1016/0022-247X(77)90152-4. Journal of Mathematical Analysis and Applications, 61 0 (1): 0 159--164, 1977

    work page doi:10.1016/0022-247x(77)90152-4

  7. [7]

    Carmon, Y

    Y. Carmon, Y. Jin, A. Sidford, and K. Tian. Variance reduction for matrix games. In Advances in Neural Information Processing Systems, pp.\ 11377--11388, 2019

    work page 2019

  8. [8]

    X. Chen, X. Deng, and S.-H. Teng. Settling the complexity of computing two-player N ash equilibria. Journal of the ACM, 56 0 (3): 0 14, 2009

    work page 2009

Show all 53 references
  1. [9]

    S. S. Cheng and S. S. Chiou. Exact stability regions for quartic polynomials. Bulletin of the Brazilian Mathematical Society, 38 0 (1): 0 21--38, 2007

    2007

  2. [10]

    B. Dai, A. Shaw, L. Li, L. Xiao, N. He, Z. Liu, J. Chen, and L. Song. Sbeed: Convergent reinforcement learning with nonlinear function approximation. In International Conference on Machine Learning, pp.\ 1125--1134, 2018

    2018

  3. [11]

    Daskalakis and I

    C. Daskalakis and I. Panageas. Last-iterate convergence: Zero-sum games and constrained min-max optimization. In Innovations in Theoretical Computer Science, 2019

    2019

  4. [12]

    Daskalakis, A

    C. Daskalakis, A. Ilyas, V. Syrgkanis, and H. Zeng. Training GAN s with optimism. In International Conference on Learning Representations, 2018

    2018

  5. [13]

    Deligkas, J

    A. Deligkas, J. Fearnley, R. Savani, and P. Spirakis. Computing approximate N ash equilibria in polymatrix games. Algorithmica, 77 0 (2): 0 487--514, 2017

    2017

  6. [14]

    V. F. Dem'yanov and A. B. Pevnyi. Numerical methods for finding saddle points https://doi.org/10.1016/0041-5553(72)90002-X. USSR Computational Mathematics and Mathematical Physics, 12 0 (5): 0 11--52, 1972

    1972 doi

  7. [15]

    S. S. Du, J. Chen, L. Li, L. Xiao, and D. Zhou. Stochastic variance reduction methods for policy evaluation. In International Conference on Machine Learning, pp.\ 1049--1058, 2017

    2017

  8. [16]

    Freund and R

    Y. Freund and R. E. Schapire. Adaptive game playing using multiplicative weights. Games and Economic Behavior, 29 0 (1-2): 0 79--103, 1999

    1999

  9. [17]

    Gidel, H

    G. Gidel, H. Berard, G. Vignoud, P. Vincent, and S. Lacoste-Julien. A variational inequality perspective on generative adversarial networks. In International Conference on Learning Representations, 2019 a

    2019

  10. [18]

    Gidel, R

    G. Gidel, R. A. Hemmat, M. Pezeshki, G. Huang, R. Lepriol, S. Lacoste-Julien, and I. Mitliagkas. Negative momentum for improved game dynamics http://proceedings.mlr.press/v89/gidel19a. In AISTATS, 2019 b

    2019

  11. [19]

    Gohberg, P

    I. Gohberg, P. Lancaster, and L. Rodman. Matrix polynomials. Academic Press, 1982

    1982

  12. [20]

    E. G. Gol'shtein. A generalized gradient method for finding saddlepoints. Ekonomika i matematicheskie metody, 8 0 (4): 0 569--579, 1972

    1972

  13. [21]

    Goodfellow, J

    I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. In Advances in Neural Information Processing Systems, pp.\ 2672--2680, 2014

    2014

  14. [22]

    Hsieh, F

    Y.-G. Hsieh, F. Iutzeler, J. Malick, and P. Mertikopoulos. On the convergence of single-call stochastic extra-gradient methods. In Advances in Neural Information Processing Systems, pp.\ 6936--6946, 2019

    2019

  15. [23]

    D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations, 2015

    2015

  16. [24]

    G. M. Korpelevich. The extragradient method for finding saddle points and other problems. Matecon, 12: 0 747--756, 1976

    1976

  17. [25]

    Liang and J

    T. Liang and J. Stokes. Interaction matters: A note on non-asymptotic local convergence of generative adversarial networks http://proceedings.mlr.press/v89/liang19b. In AISTATS, 2019

    2019

  18. [26]

    P. L. Lions. Une m\'ethode it\'erative de r\'esolution d'une in\'equation variationnelle https://doi.org/10.1007/BF02760552. Israel Journal of Mathematics, 31 0 (2): 0 204--208, 1978

    1978 doi

  19. [27]

    Madry, A

    A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018

    2018

  20. [28]

    M. Mansour. Discrete-time and sampled-data stability tests. In Williams S. Levine (ed.), The Control Handbook: Control System Fundamentals. CRC press, 2nd edition, 2011

    2011

  21. [29]

    Martinet

    B. Martinet. R\'egularisation d'in\'equations variationnelles par approximations successives http://www.numdam.org/item/M2AN_1970__4_3_154_0. ESAIM: Mathematical Modelling and Numerical Analysis: Mod\'elisation Math\'ematique et Analyse Num\'erique, 4 0 (R3): 0 154--158, 1970

    1970

  22. [30]

    Mertikopoulos, C

    P. Mertikopoulos, C. Papadimitriou, and G. Piliouras. Cycles in adversarial regularized learning. In Proceedings of the Twenty-Ninth Annual ACM-SIAM Symposium on Discrete Algorithms, pp.\ 2703--2717. SIAM, 2018

    2018

  23. [31]

    Mertikopoulos, B

    P. Mertikopoulos, B. Lecouat, H. Zenati, C.-S. Foo, V. Chandrasekhar, and G. Piliouras. Optimistic mirror descent in saddle-point problems: Going the extra (gradient) mile. In International Conference on Learning Representations, 2019

    2019

  24. [32]

    Mescheder, S

    L. Mescheder, S. Nowozin, and A. Geiger. The numerics of GAN s. In Advances in Neural Information Processing Systems, pp.\ 1825--1835, 2017

    2017

  25. [33]

    Mescheder, A

    L. Mescheder, A. Geiger, and S. Nowozin. Which training methods for GANs do actually converge? In International Conference on Machine Learning, 2018

    2018

  26. [34]

    Mokhtari, A

    A. Mokhtari, A. Ozdaglar, and S. Pattathil. Proximal point approximations achieving a convergence rate of O (1/k) for smooth convex-concave saddle point problems: Optimistic gradient and extra-gradient methods. arXiv preprint arXiv:1906.01115, 2019 a

    1906 arXiv

  27. [35]

    Mokhtari, A

    A. Mokhtari, A. Ozdaglar, and S. Pattathil. A unified analysis of extra-gradient and optimistic gradient methods for saddle point problems: Proximal point approach. arXiv preprint arXiv:1901.08511, 2019 b

    1901 arXiv

  28. [36]

    R. D. C. Monteiro and B. F. Svaiter. On the complexity of the hybrid proximal extragradient method for the iterates and the ergodic mean. SIAM Journal on Optimization, 20 0 (6): 0 2755--2787, 2010

    2010

  29. [37]

    Nagarajan and J

    V. Nagarajan and J. Z. Kolter. Gradient descent GAN optimization is locally stable. In Advances in Neural Information Processing Systems, pp.\ 5585--5595, 2017

    2017

  30. [38]

    Nedi \'c and A

    A. Nedi \'c and A. Ozdaglar. Subgradient methods for saddle-point problems. Journal of optimization theory and applications, 142 0 (1): 0 205--228, 2009

    2009

  31. [39]

    Nemirovski

    A. Nemirovski. Prox-method with rate of convergence O (1/t) for variational inequalities with lipschitz continuous monotone operators and smooth convex-concave saddle point problems. SIAM Journal on Optimization, 15 0 (1): 0 229--251, 2004

    2004

  32. [40]

    A. S. Nemirovski and D. B. Yudin. Ces \`a ro convergence of the gradient method of approximating saddle points of convex-concave functions. Doklady Akademii Nauk, 239: 0 1056--1059, 1978

    1978

  33. [41]

    A. S. Nemirovski and D. B. Yudin. Problem complexity and method efficiency in optimization. Wiley, 1983

    1983

  34. [42]

    Nesterov

    Y. Nesterov. A method for unconstrained convex minimization problem with the rate of convergence O (1/k^ 2 ) . Doklady Akademii Nauk, 269: 0 543--547, 1983

    1983

  35. [43]

    W. Peng, Y. Dai, H. Zhang, and L. Cheng. Training GANs with centripetal acceleration. arXiv preprint arXiv:1902.08949, 2019

    1902 arXiv

  36. [44]

    B. T. Polyak. Some methods of speeding up the convergence of iteration methods https://doi.org/10.1016/0041-5553(64)90137-5. USSR Computational Mathematics and Mathematical Physics, 4 0 (5): 0 1--17, 1964

    1964 doi

  37. [45]

    L. D. Popov. A modification of the Arrow--Hurwicz method for search of saddle points. Mathematical Notes, 28 0 (5): 0 845--848, 1980

    1980

  38. [46]

    R. T. Rockafellar. Monotone operators and the proximal point algorithm. SIAM journal on control and optimization, 14 0 (5): 0 877--898, 1976

    1976

  39. [47]

    Y. Saad. Iterative methods for sparse linear systems. SIAM , 2nd edition, 2003

    2003

  40. [48]

    U ber P otenzreihen, die im I nnern des E inheitskreises beschr \

    I. Schur. \"U ber P otenzreihen, die im I nnern des E inheitskreises beschr \"a nkt sind. Journal f \"u r die reine und angewandte Mathematik , 147: 0 205--232, 1917

    1917

  41. [49]

    Stein and R

    P. Stein and R. L. Rosenberg. On the solution of linear simultaneous equations by iteration. Journal of the London Mathematical Society, 1 0 (2): 0 111--118, 1948

    1948

  42. [50]

    P. Tseng. On linear convergence of iterative methods for the variational inequality problem. Journal of Computational and Applied Mathematics, 60 0 (1-2): 0 237--252, 1995

    1995

  43. [51]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  44. [52]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  45. [53]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.