REVIEW 2 major objections 3 minor 32 references
An Optimistic Gradient Tracking Method for Distributed Minimax Optimization
T0 review · 2 major / 3 minor · reviewed 2026-08-05 · deepseek-v4-flash
Pith's one-line read Distributed minimax can reach the known communication lower bound with an optimistic gradient-tracking method.
desk verdict The optimal-rate claim rests on a false joint convexity step in Lemma 4, so the main theorems are unproven as written. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The surrogate objective adds to each local function quadratic proximal terms plus linear similarity corrections that measure the gap between the local gradient and a tracking variable; its first-order condition yields a proximal update, and a one-step approximation replaces the unknown next gradient by a difference of the two most recent gradients, with error O(gamma^2). This produces the optimistic gradient-descent-ascent step in Algorithm 1. The gradient-tracking variables are updated by consensus-weighted gradient differences, so their averages equal the average gradient while their deviations from consensus contract. The analysis is carried by a Lyapunov function that simultaneously cont
What would settle it
Run DOGT or ADOGT on a ring network with the paper's synthetic objective but set mu=0 on one node, or use different mu values per node, and measure the residual 1/n ||z_k - 1 z*||^2. If the residual stops decreasing linearly or the iteration count to epsilon grows faster than O(kappa log(1/epsilon))/sqrt(1-sqrt(rho_W)), the dependence on Assumption 1 is exposed and the optimal-complexity claim fails.
Extended reading notes
Core claim
The central claim is that a distributed optimistic method can simultaneously track the global gradient and exploit optimistic extrapolation, so the average trajectory mimics centralized optimistic gradient descent ascent while the network disagreement contracts. The proof uses a Lyapunov function that combines the optimality-gap term, the successive-difference term, the consensus error, and the gradient-tracking error; the authors show each component contracts geometrically under a stepsize bound. For ADOGT, replacing the standard gossip matrix by an accelerated consensus matrix with a carefully chosen number of inner communication steps makes the effective spectral gap at least 1/2, yieldin
Load-bearing premise
All local objectives must be strongly convex in x and strongly concave in y with the same strength mu; if curvature varies across nodes or vanishes (for example in bilinear objectives), the linear-rate proof and the matching lower-bound claim no longer apply.
Editorial extensions
If this is right
- For smooth strongly convex-strongly concave objectives satisfying the paper's assumptions, DOGT converges linearly with a rate that depends only on the condition number and the network spectral gap, using a single-loop update with no inner subproblem solves.
- DOGT improves on the gradient-tracking extragradient baseline by a factor of O(kappa^{1/3}) when the network is well connected, meaning the surrogate and optimistic extrapolation provide a real speedup.
- ADOGT's round complexity matches the deterministic lower bound of the problem class up to constants, so no deterministic decentralized method in this class can be asymptotically faster in communication rounds.
- The communication complexity scales as 1/sqrt(1-sqrt(rho_W)) rather than 1/(1-rho_W), demonstrating the quantitative benefit of the accelerated consensus protocol.
- Because the algorithm requires only first-order gradient information plus one consensus round per iteration, it is directly deployable in heterogeneous data settings where plain distributed optimistic methods fail to reach the optimum.
Reading between the lines
- The same surrogate-plus-acceleration recipe could plausibly extend to stochastic gradient estimates, but the optimal round-complexity match would likely require additional sample-complexity terms that the paper does not analyze.
- The accelerated protocol requires prior knowledge of the spectral gap rho_W; in networks where rho_W is unknown, estimating it online could make ADOGT parameter-free while preserving the same complexity.
- Since the lower bound is already matched for the strongly convex-strongly concave class, further communication improvements would have to come from exploiting additional structure, such as second-order similarity among local functions, rather than from the consensus protocol alone.
- The dependence on a uniform strong-convexity constant mu suggests a testable extension: deriving rates that explicitly track the spread of per-node curvature values, which would indicate how much heterogeneity the method can tolerate before the linear-rate guarantee fails.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes DOGT, a distributed optimistic gradient tracking algorithm for smooth strongly-convex--strongly-concave minimax problems over networks, and an accelerated variant ADOGT. The claimed contributions are a linear convergence rate for DOGT and an optimal communication complexity for ADOGT that matches the lower bound of Beznosikov et al. (2020). The analysis is Lyapunov-based, with supporting lemmas on the optimality gap, consensus error, and gradient tracking error. Numerical experiments on a bilinear example illustrate behavior but do not provide a quantitative test of the rates.
Significance. If the optimal rate and communication complexity claims were valid, the work would be a meaningful advance in distributed minimax optimization, closing a gap to known lower bounds while maintaining robustness to heterogeneity. The algorithm design, based on a surrogate proximal point approximation, is plausible and the intended comparison with existing GT-EG methods is appropriate. However, the central convergence proof is invalid as written: a key inequality in Lemma 4 does not follow from the stated assumptions and is false for the paper's own model. Because Theorem 1 and Theorem 2 rest on this lemma, the advertised optimality is unproven. The manuscript therefore does not deliver its main claimed contribution.
major comments (2)
- [Theorem 1, Eq. (16)–(18)] The proof of Lemma 4 uses the inequality ⟨∇_x f_i(z_i), x_i−x'⟩ + ⟨−∇_y f_i(z_i), y_i−y'⟩ ≥ f_i(z_i)−f_i(z') + (μ/2)‖z_i−z'‖². Assumption 1 gives only separate μ-strong convexity in x and μ-strong concavity in y, not joint strong convexity in z. The inequality fails already for the paper's own model (23) with n=1, a_i=b_i=0, μ=1: f(x,y)=xy+0.5x²−0.5y², z*=(0,0), z=(2,1) gives LHS=5 but RHS=6. This is a load-bearing error: the displayed inequality is used to derive (29), and Theorem 1's contraction (16)–(18) combines Lemma 4 with Lemmas 1–3. The linear-rate proof is therefore invalid, and Theorem 2, which replaces W by M_T and reuses the DOGT rate, inherits the gap.
- [Theorem 1, proof step (16)→(18)] The transition from inequality (16) to the claimed Lyapunov contraction (18) is asserted. The text says 'combining Lemma 2 and 3' but does not show how the consensus-error and gradient-tracking-error terms are weighted and absorbed with the selected coefficients (19). This is a presentation gap in the main proof; while less severe than the Lemma 4 flaw, it further prevents verification of the theorem as written.
minor comments (3)
- [Assumption 3 and subsequent notation] ρ_W is defined as ‖W−J‖_2², i.e., the squared spectral norm of the disagreement operator, but later it is used as if it were the spectral gap (e.g., in the rate expressions 1−√ρ_W). Please clarify the relationship between ρ_W and the second-largest eigenvalue magnitude, and correct the definitions accordingly.
- [Algorithm 1, line 2] The update uses ∇x_{i,k}−∇x_{i,k−1} in the optimistic step. This notation is defined only implicitly; please state explicitly that ∇f_{i,k} = ∇f_i(x_{i,k}, y_{i,k}) and that f_{i,−1} is initialized in Algorithm 1.
- [Throughout] Minor typos: 'receptively' in Section II-B, 'convergence rateO(κlog(ε^{-1}))' in Theorem 2 (missing parentheses). The experiments (Fig. 1) show only 100 iterations and do not estimate the linear rate; consider fitting the rate or showing a log-scale plot of the residual to support the theoretical claims.
Circularity Check
No significant circularity; central derivation is self-contained and benchmarked against an external lower bound.
full rationale
The paper's claimed contributions are derived from Assumptions 1-3 (strong convexity-concavity, smoothness, graph connectivity) with a Lyapunov-based analysis. Theorem 1 (linear convergence of DOGT) is proved directly from Lemmas 1-4, which are all proven in the appendix from the stated assumptions. Theorem 2 (optimal rate of ADOGT) follows by substituting the accelerated consensus matrix from [32] into the DOGT iteration complexity; the optimality claim is matched against the external lower bound of [22]. No parameter is fitted to data, and no convergence rate is obtained by renaming an input or by assuming the conclusion. The only self-citation is [11], which appears in the introduction as a related-work reference on adaptive stepsizes and is not used as a premise in any proof. The approximation argument from [19] is motivational rather than load-bearing, since the algorithm is analyzed directly. The potential mathematical issue in Lemma 4 highlighted by the skeptic is an error in the proof (the inequality asserted does not follow from Assumption 1), but an incorrect proof step is a correctness concern, not circularity: the derivation does not reduce to its own inputs by definition or via self-citation. Therefore the paper exhibits no circularity pattern; the score reflects only the incidental self-citation that is not load-bearing.
Assumptions & free parameters
free parameters (3)
- stepsize γ =
γ ≤ min{1/(64L), (1-ρ_W)^2/(144L√ρ_W)}; set to 0.1 in experiments
- Lyapunov coefficients c1, c2 =
c1 = 72γL/(n(1-ρ_W)), c2 = 4608γ^3 L/(n(1-ρ_W)^3)
- acceleration rounds T =
T = ⌈ln(2)/√(1-√ρ_W)⌉
assumptions (4)
- domain assumption Assumption 1: each local fi is µ-strongly convex in x and µ-strongly concave in y for the same µ > 0.
- domain assumption Assumption 2: each local fi is jointly L-smooth in x and y.
- domain assumption Assumption 3: the weight matrix W is doubly stochastic with spectral gap ρ_W := ||W - J||_2^2 < 1.
- standard math Standard strong convexity-strong concavity property f(x̄) - f(z*) ≥ µ/2 ||x̄ - z*||^2.
Cite this review
Pith. "Pith review of An Optimistic Gradient Tracking Method for Distributed Minimax Optimization." pith.science (2026). https://pith.science/paper/LPR37GVZ
@misc{pith2026250821431,
author = {Pith},
title = {Pith review of: An Optimistic Gradient Tracking Method for Distributed Minimax Optimization},
year = {2026},
howpublished = {\url{https://pith.science/paper/LPR37GVZ}},
note = {Machine review of arXiv:2508.21431}
}
abstract
This paper studies the distributed minimax optimization problem over networks. To enhance convergence performance, we propose a distributed optimistic gradient tracking method, termed DOGT, which solves a surrogate function that captures the similarity between local objective functions to approximate a centralized optimistic approach locally. Leveraging a Lyapunov-based analysis, we prove that DOGT achieves linear convergence to the optimal solution for strongly convex-strongly concave objective functions while remaining robust to the heterogeneity among them. Moreover, by integrating an accelerated consensus protocol, the accelerated DOGT (ADOGT) algorithm achieves an optimal convergence rate of $\mathcal{O} \left( \kappa \log \left( \epsilon ^{-1} \right) \right)$ and communication complexity of $\mathcal{O} \left( \kappa \log \left( \epsilon ^{-1} \right) /\sqrt{1-\sqrt{\rho _W}} \right)$ for a suboptimality level of $\epsilon>0$, where $\kappa$ is the condition number of the objective function and $\rho_W$ is the spectrum gap of the network. Numerical experiments illustrate the effectiveness of the proposed algorithms.
Figures
Reference graph
Works this paper leans on
-
[28]
A decentralized algo rithm for large scale min-max problems,
S. Mukherjee and M. Chakraborty, “A decentralized algo rithm for large scale min-max problems,” in 2020 59th IEEE Conference on Decision and Control (CDC) , pp. 2967–2972, IEEE, 2020
work page 2020
-
[1]
M. Mohri, G. Sivek, and A. T. Suresh, “Agnostic federated learning,” in International Conference on Machine Learning , pp. 4615–4625, PMLR, 2019
work page 2019
-
[2]
Certifyin g some distributional robustness with principled adversarial tr aining,
A. Sinha, H. Namkoong, R. V olpi, and J. Duchi, “Certifyin g some distributional robustness with principled adversarial tr aining,” arXiv preprint arXiv:1710.10571, 2017
arXiv 2017
-
[3]
K. G. V amvoudakis, F. Fotiadis, J. P . Hespanha, R. Chinchilla, G. Y ang, M. Liu, J. S. Shamma, and L. Pavel, “Game theory for autonomy: From min-max optimization to equilibrium and bounded ratio nality learning,” in 2023 American Control Conference (ACC) , pp. 4363– 4380, IEEE, 2023
work page 2023
-
[4]
I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Ward e-Farley, S. Ozair, A. Courville, and Y . Bengio, “Generative adversar ial nets,” Advances in neural information processing systems , vol. 27, 2014
work page 2014
-
[5]
Improved training of wasserstein gans,
I. Gulrajani, F. Ahmed, M. Arjovsky, V . Dumoulin, and A. C . Courville, “Improved training of wasserstein gans,” Advances in neural information processing systems , vol. 30, 2017
work page 2017
-
[6]
Federated optimization: Distributed machine learning for on-device intelligence,
J. Kone ˇcn`y, H. B. McMahan, D. Ramage, and P . Richtárik, “Federated optimization: Distributed machine learning for on-device intelligence,” arXiv preprint arXiv:1610.02527 , 2016
arXiv 2016
-
[7]
Distributed optimization for control,
A. Nedi ´c and J. Liu, “Distributed optimization for control,” Annual Review of Control, Robotics, and Autonomous Systems , vol. 1, pp. 77– 103, 2018
work page 2018
Show all 32 references
-
[8]
Can decentralized algorithms outperform centralized algorit hms? a case study for decentralized parallel stochastic gradient desc ent,
X. Lian, C. Zhang, H. Zhang, C.-J. Hsieh, W. Zhang, and J. L iu, “Can decentralized algorithms outperform centralized algorit hms? a case study for decentralized parallel stochastic gradient desc ent,” Advances in neural information processing systems , vol. 30, 2017
2017
-
[9]
A survey of distributed optimiza tion and control algorithms for electric power systems,
D. K. Molzahn, F. Dörfler, H. Sandberg, S. H. Low, S. Chakra barti, R. Baldick, and J. Lavaei, “A survey of distributed optimiza tion and control algorithms for electric power systems,” IEEE Transactions on Smart Grid , vol. 8, no. 6, pp. 2941–2962, 2017
2017
-
[10]
A decentralized parallel algorithm for training generati ve adversarial nets,
M. Liu, W. Zhang, Y . Mroueh, X. Cui, J. Ross, T. Y ang, and P . Das, “A decentralized parallel algorithm for training generati ve adversarial nets,” Advances in Neural Information Processing Systems , vol. 33, pp. 11056–11070, 2020
2020
-
[11]
Achieving near -optimal convergence for distributed minimax optimization with ada ptive step- sizes,
Y . Huang, X. Li, Y . Shen, N. He, and J. Xu, “Achieving near -optimal convergence for distributed minimax optimization with ada ptive step- sizes,” Advances in Neural Information Processing Systems , vol. 37, pp. 19740–19782, 2024
2024
-
[12]
Numerical methods for finding saddle points,
V . F. Dem’yanov and A. B. Pevnyi, “Numerical methods for finding saddle points,” USSR Computational Mathematics and Mathematical Physics, vol. 12, no. 5, pp. 11–52, 1972
1972
-
[13]
Tra ining gans with optimism,
C. Daskalakis, A. Ilyas, V . Syrgkanis, and H. Zeng, “Tra ining gans with optimism,” arXiv preprint arXiv:1711.00141 , 2017
2017 arXiv
-
[14]
The extragradient method for findin g saddle points and other problems,
G. M. Korpelevich, “The extragradient method for findin g saddle points and other problems,” Matecon, vol. 12, pp. 747–756, 1976
1976
-
[15]
Distributed average cons ensus with time-varying metropolis weights,
L. Xiao, S. Boyd, and S. Lall, “Distributed average cons ensus with time-varying metropolis weights,” Automatica, vol. 1, pp. 1–4, 2006
2006
-
[16]
Local stochastic gradient desc ent ascent: Convergence analysis and communication efficiency,
Y . Deng and M. Mahdavi, “Local stochastic gradient desc ent ascent: Convergence analysis and communication efficiency,” in International Conference on Artificial Intelligence and Statistics , pp. 1387–1395, PMLR, 2021
2021
-
[17]
Communication-efficient learning of deep networks from de central- ized data,
B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Ar cas, “Communication-efficient learning of deep networks from de central- ized data,” in Artificial intelligence and statistics , pp. 1273–1282, PMLR, 2017
2017
-
[18]
An efficient stochastic algor ithm for decentralized nonconvex-strongly-concave minimax op timization,
L. Chen, H. Y e, and L. Luo, “An efficient stochastic algor ithm for decentralized nonconvex-strongly-concave minimax op timization,” in International Conference on Artificial Intelligence and St atistics, pp. 1990–1998, PMLR, 2024
1990
-
[19]
A unified an alysis of extra- gradient and optimistic gradient methods for saddle point p roblems: Proximal point approach,
A. Mokhtari, A. Ozdaglar, and S. Pattathil, “A unified an alysis of extra- gradient and optimistic gradient methods for saddle point p roblems: Proximal point approach,” in International Conference on Artificial Intelligence and Statistics , pp. 1497–1507, PMLR, 2020
2020
-
[20]
A tight and unified analysis of gradient-based methods for a whole sp ectrum of differentiable games,
W. Azizian, I. Mitliagkas, S. Lacoste-Julien, and G. Gi del, “A tight and unified analysis of gradient-based methods for a whole sp ectrum of differentiable games,” in International conference on artificial intelligence and statistics , pp. 2863–2873, PMLR, 2020
2020
-
[21]
Interaction matters: A note on n on-asymptotic local convergence of generative adversarial networks,
T. Liang and J. Stokes, “Interaction matters: A note on n on-asymptotic local convergence of generative adversarial networks,” in The 22nd International Conference on Artificial Intelligence and St atistics, pp. 907–915, PMLR, 2019
2019
-
[22]
Distribu ted saddle- point problems: Lower bounds, near-optimal and robust algo rithms,
A. Beznosikov, V . Samokhin, and A. Gasnikov, “Distribu ted saddle- point problems: Lower bounds, near-optimal and robust algo rithms,” arXiv preprint arXiv:2010.13112 , 2020
2010 arXiv
-
[23]
A decentralized proximal point-type method for saddle poi nt prob- lems,
W. Liu, A. Mokhtari, A. Ozdaglar, S. Pattathil, Z. Shen, and N. Zheng, “A decentralized proximal point-type method for saddle poi nt prob- lems,” arXiv preprint arXiv:1910.14380 , 2019
1910 arXiv
-
[24]
Diffusion stoc hastic optimization for min-max problems,
H. Cai, S. A. Alghunaim, and A. H. Sayed, “Diffusion stoc hastic optimization for min-max problems,” IEEE Transactions on Signal Processing, 2024
2024
-
[25]
Augmented distribut ed gradient methods for multi-agent optimization under uncoordinated constant stepsizes,
J. Xu, S. Zhu, Y . C. Soh, and L. Xie, “Augmented distribut ed gradient methods for multi-agent optimization under uncoordinated constant stepsizes,” in 2015 54th IEEE Conference on Decision and Control (CDC), pp. 2055–2060, IEEE, 2015
2015
-
[26]
Distributed stochastic gradient tracking methods,
S. Pu and A. Nedi ´c, “Distributed stochastic gradient tracking methods,” Mathematical Programming, vol. 187, no. 1, pp. 409–457, 2021
2021
-
[27]
Multi-agent re in- forcement learning via double averaging primal-dual optim ization,
H.-T. Wai, Z. Y ang, Z. Wang, and M. Hong, “Multi-agent re in- forcement learning via double averaging primal-dual optim ization,” Advances in neural information processing systems , vol. 31, 2018
2018
-
[29]
Distributed opt imization based on gradient tracking revisited: Enhancing convergen ce rate via surrogation,
Y . Sun, G. Scutari, and A. Daneshmand, “Distributed opt imization based on gradient tracking revisited: Enhancing convergen ce rate via surrogation,” SIAM Journal on Optimization , vol. 32, no. 2, pp. 354– 385, 2022
2022
-
[30]
Distributed saddle-point problems under data similarity,
A. Beznosikov, G. Scutari, A. Rogozin, and A. Gasnikov, “Distributed saddle-point problems under data similarity,” Advances in Neural Information Processing Systems , vol. 34, pp. 8172–8184, 2021
2021
-
[31]
Near-optimal distributed mi nimax optimization under the second-order similarity,
Q. Zhou, H. Y e, and L. Luo, “Near-optimal distributed mi nimax optimization under the second-order similarity,” Advances in Neural Information Processing Systems , vol. 37, pp. 28009–28050, 2024
2024
-
[32]
Accelerated linear iterations f or distributed averaging,
J. Liu and A. S. Morse, “Accelerated linear iterations f or distributed averaging,” Annual Reviews in Control , vol. 35, pp. 160–165, 2011
2011
Reviewed August 5, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.