Pith. sign in

REVIEW 2 major objections 3 minor 32 references

An Optimistic Gradient Tracking Method for Distributed Minimax Optimization

T0 review · 2 major / 3 minor · reviewed 2026-08-05 · deepseek-v4-flash

Pith's one-line read Distributed minimax can reach the known communication lower bound with an optimistic gradient-tracking method.

desk verdict The optimal-rate claim rests on a false joint convexity step in Lemma 4, so the main theorems are unproven as written. read the letter →

arxiv 2508.21431 v1 pith:LPR37GVZ submitted 2025-08-29 math.OC cs.DC

classification math.OCcs.DC MSC 90C4790C2568W15
keywords distributedminimaxoptimizationgradienttrackingoptimisticdescentascentacceleratedconsensusstronglyconvex-stronglyconcavecommunicationcomplexityheterogeneousdatasaddle-pointproblems
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper studies a network of nodes that jointly solve a minimax problem—minimize over x, maximize over y—where the global objective is the average of local functions. The authors propose DOGT, a decentralized optimistic gradient-tracking algorithm, and prove that for smooth strongly convex-strongly concave objectives it converges linearly to the exact saddle point even when local functions are heterogeneous, without requiring bounded gradients or bounded domains. They then add an accelerated consensus protocol to obtain ADOGT and prove it reaches epsilon-optimality in O(kappa log(1/epsilon)) iterations with O(kappa log(1/epsilon)/sqrt(1-sqrt(rho_W))) communication rounds, matching the known deterministic lower bound for this problem class. If these claims hold, ADOGT is communication-optimal among deterministic decentralized methods for this setting, and DOGT provides a single-loop baseline that is robust to data heterogeneity.

What carries the argument

The surrogate objective adds to each local function quadratic proximal terms plus linear similarity corrections that measure the gap between the local gradient and a tracking variable; its first-order condition yields a proximal update, and a one-step approximation replaces the unknown next gradient by a difference of the two most recent gradients, with error O(gamma^2). This produces the optimistic gradient-descent-ascent step in Algorithm 1. The gradient-tracking variables are updated by consensus-weighted gradient differences, so their averages equal the average gradient while their deviations from consensus contract. The analysis is carried by a Lyapunov function that simultaneously cont

What would settle it

Run DOGT or ADOGT on a ring network with the paper's synthetic objective but set mu=0 on one node, or use different mu values per node, and measure the residual 1/n ||z_k - 1 z*||^2. If the residual stops decreasing linearly or the iteration count to epsilon grows faster than O(kappa log(1/epsilon))/sqrt(1-sqrt(rho_W)), the dependence on Assumption 1 is exposed and the optimal-complexity claim fails.

Watch

Extended reading notes

Core claim

The central claim is that a distributed optimistic method can simultaneously track the global gradient and exploit optimistic extrapolation, so the average trajectory mimics centralized optimistic gradient descent ascent while the network disagreement contracts. The proof uses a Lyapunov function that combines the optimality-gap term, the successive-difference term, the consensus error, and the gradient-tracking error; the authors show each component contracts geometrically under a stepsize bound. For ADOGT, replacing the standard gossip matrix by an accelerated consensus matrix with a carefully chosen number of inner communication steps makes the effective spectral gap at least 1/2, yieldin

Load-bearing premise

All local objectives must be strongly convex in x and strongly concave in y with the same strength mu; if curvature varies across nodes or vanishes (for example in bilinear objectives), the linear-rate proof and the matching lower-bound claim no longer apply.

Editorial extensions

If this is right

  • For smooth strongly convex-strongly concave objectives satisfying the paper's assumptions, DOGT converges linearly with a rate that depends only on the condition number and the network spectral gap, using a single-loop update with no inner subproblem solves.
  • DOGT improves on the gradient-tracking extragradient baseline by a factor of O(kappa^{1/3}) when the network is well connected, meaning the surrogate and optimistic extrapolation provide a real speedup.
  • ADOGT's round complexity matches the deterministic lower bound of the problem class up to constants, so no deterministic decentralized method in this class can be asymptotically faster in communication rounds.
  • The communication complexity scales as 1/sqrt(1-sqrt(rho_W)) rather than 1/(1-rho_W), demonstrating the quantitative benefit of the accelerated consensus protocol.
  • Because the algorithm requires only first-order gradient information plus one consensus round per iteration, it is directly deployable in heterogeneous data settings where plain distributed optimistic methods fail to reach the optimum.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same surrogate-plus-acceleration recipe could plausibly extend to stochastic gradient estimates, but the optimal round-complexity match would likely require additional sample-complexity terms that the paper does not analyze.
  • The accelerated protocol requires prior knowledge of the spectral gap rho_W; in networks where rho_W is unknown, estimating it online could make ADOGT parameter-free while preserving the same complexity.
  • Since the lower bound is already matched for the strongly convex-strongly concave class, further communication improvements would have to come from exploiting additional structure, such as second-order similarity among local functions, rather than from the consensus protocol alone.
  • The dependence on a uniform strong-convexity constant mu suggests a testable extension: deriving rates that explicitly track the spread of per-node curvature values, which would indicate how much heterogeneity the method can tolerate before the linear-rate guarantee fails.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 3 minor

Summary. The paper proposes DOGT, a distributed optimistic gradient tracking algorithm for smooth strongly-convex--strongly-concave minimax problems over networks, and an accelerated variant ADOGT. The claimed contributions are a linear convergence rate for DOGT and an optimal communication complexity for ADOGT that matches the lower bound of Beznosikov et al. (2020). The analysis is Lyapunov-based, with supporting lemmas on the optimality gap, consensus error, and gradient tracking error. Numerical experiments on a bilinear example illustrate behavior but do not provide a quantitative test of the rates.

Significance. If the optimal rate and communication complexity claims were valid, the work would be a meaningful advance in distributed minimax optimization, closing a gap to known lower bounds while maintaining robustness to heterogeneity. The algorithm design, based on a surrogate proximal point approximation, is plausible and the intended comparison with existing GT-EG methods is appropriate. However, the central convergence proof is invalid as written: a key inequality in Lemma 4 does not follow from the stated assumptions and is false for the paper's own model. Because Theorem 1 and Theorem 2 rest on this lemma, the advertised optimality is unproven. The manuscript therefore does not deliver its main claimed contribution.

major comments (2)
  1. [Theorem 1, Eq. (16)–(18)] The proof of Lemma 4 uses the inequality ⟨∇_x f_i(z_i), x_i−x'⟩ + ⟨−∇_y f_i(z_i), y_i−y'⟩ ≥ f_i(z_i)−f_i(z') + (μ/2)‖z_i−z'‖². Assumption 1 gives only separate μ-strong convexity in x and μ-strong concavity in y, not joint strong convexity in z. The inequality fails already for the paper's own model (23) with n=1, a_i=b_i=0, μ=1: f(x,y)=xy+0.5x²−0.5y², z*=(0,0), z=(2,1) gives LHS=5 but RHS=6. This is a load-bearing error: the displayed inequality is used to derive (29), and Theorem 1's contraction (16)–(18) combines Lemma 4 with Lemmas 1–3. The linear-rate proof is therefore invalid, and Theorem 2, which replaces W by M_T and reuses the DOGT rate, inherits the gap.
  2. [Theorem 1, proof step (16)→(18)] The transition from inequality (16) to the claimed Lyapunov contraction (18) is asserted. The text says 'combining Lemma 2 and 3' but does not show how the consensus-error and gradient-tracking-error terms are weighted and absorbed with the selected coefficients (19). This is a presentation gap in the main proof; while less severe than the Lemma 4 flaw, it further prevents verification of the theorem as written.
minor comments (3)
  1. [Assumption 3 and subsequent notation] ρ_W is defined as ‖W−J‖_2², i.e., the squared spectral norm of the disagreement operator, but later it is used as if it were the spectral gap (e.g., in the rate expressions 1−√ρ_W). Please clarify the relationship between ρ_W and the second-largest eigenvalue magnitude, and correct the definitions accordingly.
  2. [Algorithm 1, line 2] The update uses ∇x_{i,k}−∇x_{i,k−1} in the optimistic step. This notation is defined only implicitly; please state explicitly that ∇f_{i,k} = ∇f_i(x_{i,k}, y_{i,k}) and that f_{i,−1} is initialized in Algorithm 1.
  3. [Throughout] Minor typos: 'receptively' in Section II-B, 'convergence rateO(κlog(ε^{-1}))' in Theorem 2 (missing parentheses). The experiments (Fig. 1) show only 100 iterations and do not estimate the linear rate; consider fitting the rate or showing a log-scale plot of the residual to support the theoretical claims.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity; central derivation is self-contained and benchmarked against an external lower bound.

full rationale

The paper's claimed contributions are derived from Assumptions 1-3 (strong convexity-concavity, smoothness, graph connectivity) with a Lyapunov-based analysis. Theorem 1 (linear convergence of DOGT) is proved directly from Lemmas 1-4, which are all proven in the appendix from the stated assumptions. Theorem 2 (optimal rate of ADOGT) follows by substituting the accelerated consensus matrix from [32] into the DOGT iteration complexity; the optimality claim is matched against the external lower bound of [22]. No parameter is fitted to data, and no convergence rate is obtained by renaming an input or by assuming the conclusion. The only self-citation is [11], which appears in the introduction as a related-work reference on adaptive stepsizes and is not used as a premise in any proof. The approximation argument from [19] is motivational rather than load-bearing, since the algorithm is analyzed directly. The potential mathematical issue in Lemma 4 highlighted by the skeptic is an error in the proof (the inequality asserted does not follow from Assumption 1), but an incorrect proof step is a correctness concern, not circularity: the derivation does not reduce to its own inputs by definition or via self-citation. Therefore the paper exhibits no circularity pattern; the score reflects only the incidental self-citation that is not load-bearing.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claims rest on standard smoothness, strong convexity/concavity, and graph mixing assumptions. The only user-chosen quantities are the stepsize and the Lyapunov weights, which are bounded by theory and do not function as fitted degrees of freedom. No new physical or mathematical entities are introduced.

free parameters (3)
  • stepsize γ = γ ≤ min{1/(64L), (1-ρ_W)^2/(144L√ρ_W)}; set to 0.1 in experiments
    User-selected stepsize constrained by Theorem 1. The convergence rates are proven for any γ in this range; the experiments pick γ=0.1.
  • Lyapunov coefficients c1, c2 = c1 = 72γL/(n(1-ρ_W)), c2 = 4608γ^3 L/(n(1-ρ_W)^3)
    Chosen by hand to make the coefficient terms in inequality (18) non-positive. They are proof artifacts, not fitted predictions.
  • acceleration rounds T = T = ⌈ln(2)/√(1-√ρ_W)⌉
    Number of accelerated gossip steps per iteration in ADOGT, set by Theorem 2 to guarantee ρ_M ≤ 1/2. Depends on knowing ρ_W.
assumptions (4)
  • domain assumption Assumption 1: each local fi is µ-strongly convex in x and µ-strongly concave in y for the same µ > 0.
    Used in Lemma 4 to bound the optimality gap contraction; without it the linear rate collapses.
  • domain assumption Assumption 2: each local fi is jointly L-smooth in x and y.
    Used throughout the lemmas to bound differences of gradients.
  • domain assumption Assumption 3: the weight matrix W is doubly stochastic with spectral gap ρ_W := ||W - J||_2^2 < 1.
    Required for consensus contraction and for the accelerated protocol; ADOGT additionally needs ρ_W known to set T.
  • standard math Standard strong convexity-strong concavity property f(x̄) - f(z*) ≥ µ/2 ||x̄ - z*||^2.
    Follows from Assumption 1 and is invoked in Lemma 4.

how reviews work

0 comments
Cite this review

Pith. "Pith review of An Optimistic Gradient Tracking Method for Distributed Minimax Optimization." pith.science (2026). https://pith.science/paper/LPR37GVZ

@misc{pith2026250821431,
  author       = {Pith},
  title        = {Pith review of: An Optimistic Gradient Tracking Method for Distributed Minimax Optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LPR37GVZ}},
  note         = {Machine review of arXiv:2508.21431}
}
abstract

This paper studies the distributed minimax optimization problem over networks. To enhance convergence performance, we propose a distributed optimistic gradient tracking method, termed DOGT, which solves a surrogate function that captures the similarity between local objective functions to approximate a centralized optimistic approach locally. Leveraging a Lyapunov-based analysis, we prove that DOGT achieves linear convergence to the optimal solution for strongly convex-strongly concave objective functions while remaining robust to the heterogeneity among them. Moreover, by integrating an accelerated consensus protocol, the accelerated DOGT (ADOGT) algorithm achieves an optimal convergence rate of $\mathcal{O} \left( \kappa \log \left( \epsilon ^{-1} \right) \right)$ and communication complexity of $\mathcal{O} \left( \kappa \log \left( \epsilon ^{-1} \right) /\sqrt{1-\sqrt{\rho _W}} \right)$ for a suboptimality level of $\epsilon>0$, where $\kappa$ is the condition number of the objective function and $\rho_W$ is the spectrum gap of the network. Numerical experiments illustrate the effectiveness of the proposed algorithms.

Figures

Figures reproduced from arXiv: 2508.21431 by the authors.

Figure 1
Figure 1. Comparison of the convergence performance between D [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

32 extracted references · 27 canonical work pages

  1. [28]

    A decentralized algo rithm for large scale min-max problems,

    S. Mukherjee and M. Chakraborty, “A decentralized algo rithm for large scale min-max problems,” in 2020 59th IEEE Conference on Decision and Control (CDC) , pp. 2967–2972, IEEE, 2020

  2. [1]

    Agnostic federated learning,

    M. Mohri, G. Sivek, and A. T. Suresh, “Agnostic federated learning,” in International Conference on Machine Learning , pp. 4615–4625, PMLR, 2019

  3. [2]

    Certifyin g some distributional robustness with principled adversarial tr aining,

    A. Sinha, H. Namkoong, R. V olpi, and J. Duchi, “Certifyin g some distributional robustness with principled adversarial tr aining,” arXiv preprint arXiv:1710.10571, 2017

  4. [3]

    Game theory for autonomy: From min-max optimization to equilibrium and bounded ratio nality learning,

    K. G. V amvoudakis, F. Fotiadis, J. P . Hespanha, R. Chinchilla, G. Y ang, M. Liu, J. S. Shamma, and L. Pavel, “Game theory for autonomy: From min-max optimization to equilibrium and bounded ratio nality learning,” in 2023 American Control Conference (ACC) , pp. 4363– 4380, IEEE, 2023

  5. [4]

    Generative adversar ial nets,

    I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Ward e-Farley, S. Ozair, A. Courville, and Y . Bengio, “Generative adversar ial nets,” Advances in neural information processing systems , vol. 27, 2014

  6. [5]

    Improved training of wasserstein gans,

    I. Gulrajani, F. Ahmed, M. Arjovsky, V . Dumoulin, and A. C . Courville, “Improved training of wasserstein gans,” Advances in neural information processing systems , vol. 30, 2017

  7. [6]

    Federated optimization: Distributed machine learning for on-device intelligence,

    J. Kone ˇcn`y, H. B. McMahan, D. Ramage, and P . Richtárik, “Federated optimization: Distributed machine learning for on-device intelligence,” arXiv preprint arXiv:1610.02527 , 2016

  8. [7]

    Distributed optimization for control,

    A. Nedi ´c and J. Liu, “Distributed optimization for control,” Annual Review of Control, Robotics, and Autonomous Systems , vol. 1, pp. 77– 103, 2018

Show all 32 references
  1. [8]

    Can decentralized algorithms outperform centralized algorit hms? a case study for decentralized parallel stochastic gradient desc ent,

    X. Lian, C. Zhang, H. Zhang, C.-J. Hsieh, W. Zhang, and J. L iu, “Can decentralized algorithms outperform centralized algorit hms? a case study for decentralized parallel stochastic gradient desc ent,” Advances in neural information processing systems , vol. 30, 2017

  2. [9]

    A survey of distributed optimiza tion and control algorithms for electric power systems,

    D. K. Molzahn, F. Dörfler, H. Sandberg, S. H. Low, S. Chakra barti, R. Baldick, and J. Lavaei, “A survey of distributed optimiza tion and control algorithms for electric power systems,” IEEE Transactions on Smart Grid , vol. 8, no. 6, pp. 2941–2962, 2017

  3. [10]

    A decentralized parallel algorithm for training generati ve adversarial nets,

    M. Liu, W. Zhang, Y . Mroueh, X. Cui, J. Ross, T. Y ang, and P . Das, “A decentralized parallel algorithm for training generati ve adversarial nets,” Advances in Neural Information Processing Systems , vol. 33, pp. 11056–11070, 2020

  4. [11]

    Achieving near -optimal convergence for distributed minimax optimization with ada ptive step- sizes,

    Y . Huang, X. Li, Y . Shen, N. He, and J. Xu, “Achieving near -optimal convergence for distributed minimax optimization with ada ptive step- sizes,” Advances in Neural Information Processing Systems , vol. 37, pp. 19740–19782, 2024

  5. [12]

    Numerical methods for finding saddle points,

    V . F. Dem’yanov and A. B. Pevnyi, “Numerical methods for finding saddle points,” USSR Computational Mathematics and Mathematical Physics, vol. 12, no. 5, pp. 11–52, 1972

  6. [13]

    Tra ining gans with optimism,

    C. Daskalakis, A. Ilyas, V . Syrgkanis, and H. Zeng, “Tra ining gans with optimism,” arXiv preprint arXiv:1711.00141 , 2017

  7. [14]

    The extragradient method for findin g saddle points and other problems,

    G. M. Korpelevich, “The extragradient method for findin g saddle points and other problems,” Matecon, vol. 12, pp. 747–756, 1976

  8. [15]

    Distributed average cons ensus with time-varying metropolis weights,

    L. Xiao, S. Boyd, and S. Lall, “Distributed average cons ensus with time-varying metropolis weights,” Automatica, vol. 1, pp. 1–4, 2006

  9. [16]

    Local stochastic gradient desc ent ascent: Convergence analysis and communication efficiency,

    Y . Deng and M. Mahdavi, “Local stochastic gradient desc ent ascent: Convergence analysis and communication efficiency,” in International Conference on Artificial Intelligence and Statistics , pp. 1387–1395, PMLR, 2021

  10. [17]

    Communication-efficient learning of deep networks from de central- ized data,

    B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Ar cas, “Communication-efficient learning of deep networks from de central- ized data,” in Artificial intelligence and statistics , pp. 1273–1282, PMLR, 2017

  11. [18]

    An efficient stochastic algor ithm for decentralized nonconvex-strongly-concave minimax op timization,

    L. Chen, H. Y e, and L. Luo, “An efficient stochastic algor ithm for decentralized nonconvex-strongly-concave minimax op timization,” in International Conference on Artificial Intelligence and St atistics, pp. 1990–1998, PMLR, 2024

  12. [19]

    A unified an alysis of extra- gradient and optimistic gradient methods for saddle point p roblems: Proximal point approach,

    A. Mokhtari, A. Ozdaglar, and S. Pattathil, “A unified an alysis of extra- gradient and optimistic gradient methods for saddle point p roblems: Proximal point approach,” in International Conference on Artificial Intelligence and Statistics , pp. 1497–1507, PMLR, 2020

  13. [20]

    A tight and unified analysis of gradient-based methods for a whole sp ectrum of differentiable games,

    W. Azizian, I. Mitliagkas, S. Lacoste-Julien, and G. Gi del, “A tight and unified analysis of gradient-based methods for a whole sp ectrum of differentiable games,” in International conference on artificial intelligence and statistics , pp. 2863–2873, PMLR, 2020

  14. [21]

    Interaction matters: A note on n on-asymptotic local convergence of generative adversarial networks,

    T. Liang and J. Stokes, “Interaction matters: A note on n on-asymptotic local convergence of generative adversarial networks,” in The 22nd International Conference on Artificial Intelligence and St atistics, pp. 907–915, PMLR, 2019

  15. [22]

    Distribu ted saddle- point problems: Lower bounds, near-optimal and robust algo rithms,

    A. Beznosikov, V . Samokhin, and A. Gasnikov, “Distribu ted saddle- point problems: Lower bounds, near-optimal and robust algo rithms,” arXiv preprint arXiv:2010.13112 , 2020

  16. [23]

    A decentralized proximal point-type method for saddle poi nt prob- lems,

    W. Liu, A. Mokhtari, A. Ozdaglar, S. Pattathil, Z. Shen, and N. Zheng, “A decentralized proximal point-type method for saddle poi nt prob- lems,” arXiv preprint arXiv:1910.14380 , 2019

  17. [24]

    Diffusion stoc hastic optimization for min-max problems,

    H. Cai, S. A. Alghunaim, and A. H. Sayed, “Diffusion stoc hastic optimization for min-max problems,” IEEE Transactions on Signal Processing, 2024

  18. [25]

    Augmented distribut ed gradient methods for multi-agent optimization under uncoordinated constant stepsizes,

    J. Xu, S. Zhu, Y . C. Soh, and L. Xie, “Augmented distribut ed gradient methods for multi-agent optimization under uncoordinated constant stepsizes,” in 2015 54th IEEE Conference on Decision and Control (CDC), pp. 2055–2060, IEEE, 2015

  19. [26]

    Distributed stochastic gradient tracking methods,

    S. Pu and A. Nedi ´c, “Distributed stochastic gradient tracking methods,” Mathematical Programming, vol. 187, no. 1, pp. 409–457, 2021

  20. [27]

    Multi-agent re in- forcement learning via double averaging primal-dual optim ization,

    H.-T. Wai, Z. Y ang, Z. Wang, and M. Hong, “Multi-agent re in- forcement learning via double averaging primal-dual optim ization,” Advances in neural information processing systems , vol. 31, 2018

  21. [29]

    Distributed opt imization based on gradient tracking revisited: Enhancing convergen ce rate via surrogation,

    Y . Sun, G. Scutari, and A. Daneshmand, “Distributed opt imization based on gradient tracking revisited: Enhancing convergen ce rate via surrogation,” SIAM Journal on Optimization , vol. 32, no. 2, pp. 354– 385, 2022

  22. [30]

    Distributed saddle-point problems under data similarity,

    A. Beznosikov, G. Scutari, A. Rogozin, and A. Gasnikov, “Distributed saddle-point problems under data similarity,” Advances in Neural Information Processing Systems , vol. 34, pp. 8172–8184, 2021

  23. [31]

    Near-optimal distributed mi nimax optimization under the second-order similarity,

    Q. Zhou, H. Y e, and L. Luo, “Near-optimal distributed mi nimax optimization under the second-order similarity,” Advances in Neural Information Processing Systems , vol. 37, pp. 28009–28050, 2024

  24. [32]

    Accelerated linear iterations f or distributed averaging,

    J. Liu and A. S. Morse, “Accelerated linear iterations f or distributed averaging,” Annual Reviews in Control , vol. 35, pp. 160–165, 2011

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.