Pith. sign in

REVIEW 2 major objections 4 minor 76 references

Effective Method with Compression for Distributed and Federated Cocoercive Variational Inequalities

T0 review · 2 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper shows that a MARINA-style compressed update converges linearly for cocoercive strongly monotone variational inequalities, halving the expected squared operator norm every epoch while transmitting only compressed gradient…

desk verdict Solid but narrow: a correct MARINA-for-cocoercive-VI analysis with an overstated communication bound and experiments that do not match the theory. read the letter →

arxiv 2412.14935 v1 pith:QBDRJAM4 submitted 2024-12-19 math.OC

Daniil Medyakov , Gleb Molodtsov , Aleksandr Beznosikov This is my paper · ORCID
classification math.OC MSC 90C3368W1590C25
keywords variationalinequalitiescocoerciveoperatorsstronglymonotonedistributedoptimizationgradientcompressionMARINAlinearconvergencecommunicationcomplexity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper targets variational inequalities solved across many devices, where the per-device cost of sending full operator evaluations dominates training. It adapts the MARINA compression technique from minimization to cocoercive strongly monotone variational inequalities, letting each device transmit a compressed difference of local operators instead of the full operator. The central result is linear convergence: with step size $\gamma = 1/(8\ell(1+\alpha/n))$ and epoch length $K = 30\ell(1+\alpha/n)/\mu$, each epoch halves the expected squared norm of the aggregate operator, so reaching $\varepsilon$-accuracy costs $O((1+\delta(\ell/\mu)(1+\alpha/n))\log(\|F(z_0)\|^2/\varepsilon^2))$ compressed transmissions per device. If correct, this shows compression need not slow the convergence rate for this problem class, giving a practical way to cut communication in distributed saddle-point and game-theoretic training.

What carries the argument

The engine of the method is the recursive estimator $g_k^i = g_{k-1} + C(F_i(z_k) - F_i(z_{k-1}))$ maintained on each device, averaged by the server into $g_k$ and used in the step $z_{k+1} = z_k - \gamma g_k$. Unbiasedness of the compressor $C$ makes the compression noise mean-zero, and the per-device cocoercivity inequality $\|F_i(u)-F_i(v)\|^2 \le \ell\langle F_i(u)-F_i(v), u-v\rangle$ converts the accumulated variance into a contraction of $\mathbb{E}\|g_k\|^2$; strong monotonicity of $F$ supplies the $\mu$-dependent rate. Lemma 1 controls the contraction of the compressed estimator, Lemma 2 bounds the gap between $F(z_K)$ and $g_K$, and their combination yields the factor-$1/2$ shrinkage per epoch that is Theorem 1.

What would settle it

Run Algorithm 1 on a two-device problem where the aggregate operator $F=(F_1+F_2)/2$ is strongly monotone and cocoercive but $F_1$ is not cocoercive, using $\gamma = 1/(8\ell(1+\alpha/n))$ and $K = 30\ell(1+\alpha/n)/\mu$, and check whether $\mathbb{E}\|F(\tilde z_s)\|^2$ halves each epoch. If the contraction fails, the per-device cocoercivity assumption is doing real work rather than being a technical convenience.

Watch

Extended reading notes

Core claim

The paper claims that Algorithm 1, a MARINA-type method for distributed variational inequalities, achieves linear convergence under per-device cocoercivity, strong monotonicity of the aggregate operator, and unbiased compression. Concretely, Theorem 1 states that with $\gamma = 1/(8\ell(1+\alpha/n))$ and $K = 30\ell(1+\alpha/n)/\mu$, one epoch satisfies $\mathbb{E}\|F(\tilde z_s)\|^2 \le \frac12 \mathbb{E}\|F(\tilde z_{s-1})\|^2$, where $\tilde z_s$ is the epoch output and $F$ is the average of the local operators. Corollary 1.1 converts this into a per-device communication complexity of $O((1+\delta(\ell/\mu)(1+\alpha/n))\log(\|F(z_0)\|^2/\varepsilon^2))$ gradient transmissions, where $\delta$ is the fraction of information kept by the compressor. The experiments on a bilinear saddle-point problem illustrate that compressed variants reach the same accuracy with substantially fewer transmitted kilobits.

Load-bearing premise

The load-bearing premise is that every device's local operator satisfies the per-operator cocoercivity inequality; if even one local operator violates it, the proof's variance control in Lemma 1 fails, and the linear-convergence guarantee of Theorem 1 is not established.

Editorial extensions

If this is right

  • One epoch of Algorithm 1 reduces $\mathbb{E}\|F(\tilde z_s)\|^2$ by a factor of $1/2$, so the number of epochs to reach $\varepsilon^2$-accuracy is $O(\log(\|F(z_0)\|^2/\varepsilon^2))$.
  • Per device, the number of transmitted gradient vectors is $O((1+\delta(\ell/\mu)(1+\alpha/n))\log(\|F(z_0)\|^2/\varepsilon^2))$, so smaller compression ratios $\delta$ directly lower communication.
  • With $\delta \le 1/\alpha$ and $\alpha = n$, the bound becomes $O((1+\ell/(\mu n))\log(1/\varepsilon))$, the same communication complexity as the DIANA-based analysis for cocoercive VIs.
  • In the paper's saddle-point experiments, both coordinate-drop and int8-quantized MARINA reach the same accuracy as uncompressed MARINA while transmitting far fewer kilobits per device.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the theoretical guarantee does not cover the common federated setting where only the average operator is cocoercive, because the proof's variance control is per-device; whether the method still converges linearly there is an open empirical question.
  • Beyond the paper: biased compressors with error feedback are a natural next step, but the proof's cancellation of compression noise relies essentially on unbiasedness, so such an extension would need new machinery.
  • Beyond the paper: the epoch length $K$ grows linearly with $\ell/\mu$ times $(1+\alpha/n)$, so for ill-conditioned problems the periodic transmission of the full operator becomes more expensive; the paper's high-condition-number experiments ($\ell\approx10^4$) show slower progress in transmitted kilobits, which is consistent with that dependence.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper studies distributed variational inequalities of the form F(z) = (1/n) \sum_i F_i(z) = 0 under per-operator cocoercivity and aggregate strong monotonicity. It proposes Algorithm 1, a MARINA-type method that periodically recomputes the full operator and otherwise communicates compressed differences F_i(z_k) - F_i(z_{k-1}). The main result (Theorem 1) gives a contraction E||F(\tilde z_s)||^2 <= (1/2) E||F(\tilde z_{s-1})||^2 for a fixed step size and epoch length; Corollary 1.1 translates this into a communication complexity. Experiments on a bilinear saddle-point problem compare compressed and uncompressed variants.

Significance. Under the stated assumptions, the convergence proof is coherent and the constants are explicit; I verified the key steps in Lemmas 1 and 2, including the variance bound for unbiased compressors, the use of cocoercivity and strong monotonicity, and the telescoping arguments. The method is a sensible extension of MARINA to cocoercive variational inequalities and would be a useful contribution if the communication claim is stated accurately. The main weakness is the communication accounting in Corollary 1.1; the convergence theorem itself is not affected.

major comments (2)
  1. [Corollary 1.1 (proof) and Algorithm 1, line 7] The communication count in Corollary 1.1 counts only uplink transmissions. Algorithm 1, line 7 requires the server to broadcast the full uncompressed vector g_{k-1} to every device in each of the K-1 inner iterations. Counting both directions per device gives S(1 + \delta(K-1) + (K-1)) = O((1+\delta)(\ell/\mu)(1+\alpha/n) \log(1/\varepsilon)) gradient transmissions per device, not the stated O((1+\delta(\ell/\mu)(1+\alpha/n)) \log(1/\varepsilon)). For small \delta, the advertised compression saving disappears. Because the paper's headline contribution is communication efficiency, the statement of Corollary 1.1 and the surrounding discussion must be revised, or the algorithm must be modified to avoid full-vector downlink.
  2. [Assumption 1 and Lemma 1] The proof of Lemma 1 uses per-device \ell-cocoercivity to control the variance term (\alpha/n^2) \sum_i ||F_i(z_k) - F_i(z_{k-1})||^2. In federated settings it is often realistic that only the aggregate operator F is cocoercive while individual local operators are not. The paper should explicitly state this restriction and discuss whether the analysis can be relaxed to aggregate cocoercivity; as written, the linear convergence guarantee depends critically on Assumption 1 holding for every device.
minor comments (4)
  1. [Theorem 1 and Algorithm 1] The epoch length K is written as a real number 30\ell(1+\alpha/n)/\mu, although it is used as a number of iterations; a ceiling should be taken, with a remark that the constants are unchanged up to rounding.
  2. [Assumption 4 and Corollary 1.1] Assumption 4, which says that the compressor 'leaves a fraction \delta of information', is informal. The bit-counting in Corollary 1.1 is only exact for coordinate-sampling compressors; for quantization, the relation between \delta and the transmitted bits should be specified.
  3. [Section 6] The experiments do not report the exact \delta used for quantization, nor do they use the theoretical step size and epoch length from Theorem 1, so they should be described as an empirical illustration rather than a validation of the theorem's constants.
  4. [References] The citation 'Medyakov et al.' in Section 2.2 appears without a year or venue; it should be completed or removed.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the convergence proof derives Theorem 1 from stated Assumptions 1–3 with explicit constants, and the MARINA comparison is external benchmark, not a proof input.

full rationale

The paper's derivation chain is self-contained. Lemma 1 bounds the expected norm of the compressed gradient iterate E||g_K||^2 using only Assumptions 1 (per-device cocoercivity), 2 (strong monotonicity), and 3 (unbiased compression), with an explicit step-size condition gamma <= 1/(2 ell (1 + alpha/n)). Lemma 2 bounds the gap E||F(z_K) - g_K||^2 from the same assumptions. Theorem 1 combines these two lemmas with the explicit choices gamma = 1/(8 ell (1 + alpha/n)) and K = 30 ell (1 + alpha/n)/mu to obtain the linear contraction E||F(z_s)||^2 <= 1/2 E||F(z_{s-1})||^2. No parameter is fitted to data and then renamed a prediction; the constants gamma and K are prescribed model-dependent quantities, not calibrated from the experimental runs or from the target accuracy. The cited prior work (MARINA by Gorbunov et al. 2021, DIANA comparison by Beznosikov et al. 2023a) is used for algorithmic inspiration and as a benchmark, not as a load-bearing ingredient of the proof. The authors do cite their own earlier works in the literature review, but those citations are contextual and do not supply the theorem's assumptions or conclusions. The only concern visible in the text is the communication-counting convention in Corollary 1.1, which counts per-device uplink compressed transmissions and omits the full-vector downlink of g_{k-1} on line 7; that is a potential overstatement of efficiency, not a circularity, and it does not affect the convergence derivation. Overall, the central convergence claim has independent mathematical content and does not reduce by construction to its inputs.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The central claim depends on four stated assumptions (cocoercivity of local operators, strong monotonicity of the global operator, unbiasedness of compressors, and information preservation rate) plus an implicit independence assumption on the compressors. No new entities are introduced. There are no free parameters fitted to data; the step size and epoch length are set analytically.

assumptions (5)
  • domain assumption Each local operator F_i is ℓ-cocoercive (Assumption 1).
    This is the main regularity condition on the problem; it is stronger than Lipschitz continuity and is used in Lemmas 1 and 2 to control the variance of the compressed gradient differences. It must hold for every device individually, which is restrictive in heterogeneous federated settings.
  • domain assumption The global operator F is µ-strongly monotone (Assumption 2).
    This provides the contraction used in Lemma 1 and Theorem 1 to obtain linear convergence.
  • domain assumption The compressor is unbiased with bounded variance (Assumption 3).
    E[C(u)]=u and E||C(u)-u||^2 ≤ α||u||^2 are used to make the cross terms vanish and to bound the added variance.
  • domain assumption The compressor preserves a fraction δ of information (Assumption 4).
    Used in Corollary 1.1 to count the communication cost per device.
  • domain assumption Compressors on different devices are independent and the expectation is conditional on the history.
    The proofs require that pairwise scalar products of compression errors vanish in expectation; this is standard in distributed compression but is not stated explicitly in the assumptions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Effective Method with Compression for Distributed and Federated Cocoercive Variational Inequalities." pith.science (2026). https://pith.science/paper/QBDRJAM4

@misc{pith2026241214935,
  author       = {Pith},
  title        = {Pith review of: Effective Method with Compression for Distributed and Federated Cocoercive Variational Inequalities},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QBDRJAM4}},
  note         = {Machine review of arXiv:2412.14935}
}
read the original abstract

Variational inequalities as an effective tool for solving applied problems, including machine learning tasks, have been attracting more and more attention from researchers in recent years. The use of variational inequalities covers a wide range of areas - from reinforcement learning and generative models to traditional applications in economics and game theory. At the same time, it is impossible to imagine the modern world of machine learning without distributed optimization approaches that can significantly speed up the training process on large amounts of data. However, faced with the high costs of communication between devices in a computing network, the scientific community is striving to develop approaches that make computations cheap and stable. In this paper, we investigate the compression technique of transmitted information and its application to the distributed variational inequalities problem. In particular, we present a method based on advanced techniques originally developed for minimization problems. For the new method, we provide an exhaustive theoretical convergence analysis for cocoersive strongly monotone variational inequalities. We conduct experiments that emphasize the high performance of the presented technique and confirm its practical applicability.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

76 extracted references · 47 canonical work pages

  1. [1]

    Stochastic variance reduction for variational inequality methods

    Ahmet Alacaoglu and Yura Malitsky. Stochastic variance reduction for variational inequality methods. arXiv preprint arXiv:2102.08352, 2021

    arXiv 2021

  2. [2]

    Forward-reflected-backward method with variance reduction

    Ahmet Alacaoglu, Yura Malitsky, and Volkan Cevher. Forward-reflected-backward method with variance reduction. Computational Optimization and Applications, 80, 11 2021. doi:10.1007/s10589-021-00305-3

    doi:10.1007/s10589-021-00305-3 2021

  3. [3]

    Qsgd: Communication-efficient sgd via gradient quantization and encoding

    Dan Alistarh, Demjan Grubic, Jerry Li, Ryota Tomioka, and Milan Vojnovic. Qsgd: Communication-efficient sgd via gradient quantization and encoding. Advances in neural information processing systems, 30, 2017

    2017

  4. [4]

    Convex sparse matrix factorizations

    Francis Bach, Julien Mairal, and Jean Ponce. Convex sparse matrix factorizations. arXiv preprint arXiv:0812.1869, 2008

    arXiv 2008

  5. [5]

    Robust optimization, volume 28

    Aharon Ben-Tal, Laurent El Ghaoui, and Arkadi Nemirovski. Robust optimization, volume 28. Princeton university press, 2009

    work page 2009

  6. [6]

    A Unified Analysis of Variational Inequality Methods: Variance Reduction, Sampling, Quantization and Coordinate Descent

    A. Beznosikov, A. Gasnikov, K. Zainulina, A. Maslovskiy, and D. Pasechnyuk. A unified analysis of variational inequality methods: Variance reduction, sampling, quantization andcoordinate descent. arXiv preprint arXiv:2201.12206, 2022 a

    work page Pith review arXiv 2022

  7. [7]

    Compression and data similarity: Combination of two techniques for communication-efficient solving of distributed variational inequalities

    Aleksandr Beznosikov and Alexander Gasnikov. Compression and data similarity: Combination of two techniques for communication-efficient solving of distributed variational inequalities. In International Conference on Optimization and Applications, pages 151--162. Springer, 2022

    work page 2022

  8. [8]

    Sarah-based variance-reduced algorithm for stochastic finite-sum cocoercive variational inequalities

    Aleksandr Beznosikov and Alexander Gasnikov. Sarah-based variance-reduced algorithm for stochastic finite-sum cocoercive variational inequalities. In Data Analysis and Optimization: In Honor of Boris Mirkin's 80th Birthday, pages 47--57. Springer, 2023

    work page 2023

Show all 76 references
  1. [9]

    Random-reshuffled sarah does not need full gradient computations

    Aleksandr Beznosikov and Martin Tak \'a c . Random-reshuffled sarah does not need full gradient computations. Optimization Letters, 18 0 (3): 0 727--749, 2024

    2024

  2. [10]

    Distributed saddle-point problems: Lower bounds, near-optimal and robust algorithms

    Aleksandr Beznosikov, Valentin Samokhin, and Alexander Gasnikov. Distributed saddle-point problems: Lower bounds, near-optimal and robust algorithms. arXiv preprint arXiv:2010.13112, 2020

    2010 arXiv

  3. [11]

    Distributed saddle-point problems under data similarity

    Aleksandr Beznosikov, Gesualdo Scutari, Alexander Rogozin, and Alexander Gasnikov. Distributed saddle-point problems under data similarity. Advances in Neural Information Processing Systems, 34: 0 8172--8184, 2021

    2021

  4. [12]

    Decentralized local stochastic extra-gradient for variational inequalities

    Aleksandr Beznosikov, Pavel Dvurechenskii, Anastasiia Koloskova, Valentin Samokhin, Sebastian U Stich, and Alexander Gasnikov. Decentralized local stochastic extra-gradient for variational inequalities. Advances in Neural Information Processing Systems, 35: 0 38116--38133, 2022 b

    2022

  5. [13]

    Distributed methods with compressed communication for solving variational inequalities, with theoretical guarantees

    Aleksandr Beznosikov, Peter Richt \'a rik, Michael Diskin, Max Ryabinin, and Alexander Gasnikov. Distributed methods with compressed communication for solving variational inequalities, with theoretical guarantees. Advances in Neural Information Processing Systems, 35: 0 14013-...

    2022

  6. [14]

    Stochastic gradient descent-ascent: Unified theory and new efficient methods

    Aleksandr Beznosikov, Eduard Gorbunov, Hugo Berard, and Nicolas Loizou. Stochastic gradient descent-ascent: Unified theory and new efficient methods. In International conference on artificial intelligence and statistics, pages 172--235. PMLR, 2023 a

    2023

  7. [15]

    On biased compression for distributed learning

    Aleksandr Beznosikov, Samuel Horv \'a th, Peter Richt \'a rik, and Mher Safaryan. On biased compression for distributed learning. Journal of Machine Learning Research, 24 0 (276): 0 1--50, 2023 b

    2023

  8. [16]

    Smooth monotone stochastic variational inequalities and saddle point problems: A survey

    Aleksandr Beznosikov, Boris Polyak, Eduard Gorbunov, Dmitry Kovalev, and Alexander Gasnikov. Smooth monotone stochastic variational inequalities and saddle point problems: A survey. European Mathematical Society Magazine, 0 (127): 0 15--28, 2023 c

    2023

  9. [17]

    First order methods with markovian noise: from acceleration to variational inequalities

    Aleksandr Beznosikov, Sergey Samsonov, Marina Sheshukova, Alexander Gasnikov, Alexey Naumov, and Eric Moulines. First order methods with markovian noise: from acceleration to variational inequalities. Advances in Neural Information Processing Systems, 36, 2024 a

    2024

  10. [18]

    Similarity, compression and local steps: three pillars of efficient communications for distributed variational inequalities

    Aleksandr Beznosikov, Martin Tak \'a c, and Alexander Gasnikov. Similarity, compression and local steps: three pillars of efficient communications for distributed variational inequalities. Advances in Neural Information Processing Systems, 36, 2024 b

    2024

  11. [19]

    Nonexpansive nonlinear operators in a banach space

    Felix E Browder. Nonexpansive nonlinear operators in a banach space. Proceedings of the National Academy of Sciences, 54 0 (4): 0 1041--1044, 1965

    1965

  12. [20]

    A first-order primal-dual algorithm for convex problems with applications to imaging

    Antonin Chambolle and Thomas Pock. A first-order primal-dual algorithm for convex problems with applications to imaging. Journal of mathematical imaging and vision, 40: 0 120--145, 2011

    2011

  13. [21]

    Reducing noise in gan training with variance reduced extragradient

    Tatjana Chavdarova, Gauthier Gidel, Fran c ois Fleuret, and Simon Lacoste-Julien. Reducing noise in gan training with variance reduced extragradient. volume 32, pages 393--403, 2019

    2019

  14. [22]

    Faster stochastic algorithms for minimax optimization under polyak- \ L \ ojasiewicz condition

    Lesi Chen, Boyuan Yao, and Luo Luo. Faster stochastic algorithms for minimax optimization under polyak- \ L \ ojasiewicz condition. Advances in Neural Information Processing Systems, 35: 0 13921--13932, 2022

    2022

  15. [23]

    Faster non-convex federated learning via global and local momentum

    Rudrajit Das, Anish Acharya, Abolfazl Hashemi, Sujay Sanghavi, Inderjit S Dhillon, and Ufuk Topcu. Faster non-convex federated learning via global and local momentum. In Uncertainty in Artificial Intelligence, pages 496--506. PMLR, 2022

    2022

  16. [24]

    Saga: A fast incremental gradient method with support for non-strongly convex composite objectives

    Aaron Defazio, Francis Bach, and Simon Lacoste-Julien. Saga: A fast incremental gradient method with support for non-strongly convex composite objectives. Advances in neural information processing systems, 27, 2014

    2014

  17. [25]

    Local stochastic gradient descent ascent: Convergence analysis and communication efficiency

    Yuyang Deng and Mehrdad Mahdavi. Local stochastic gradient descent ascent: Convergence analysis and communication efficiency. In International Conference on Artificial Intelligence and Statistics, pages 1387--1395. PMLR, 2021

    2021

  18. [26]

    A general framework for a class of first order primal-dual algorithms for convex optimization in imaging science

    Ernie Esser, Xiaoqun Zhang, and Tony F Chan. A general framework for a class of first order primal-dual algorithms for convex optimization in imaging science. SIAM Journal on Imaging Sciences, 3 0 (4): 0 1015--1046, 2010

    2010

  19. [27]

    Finite-dimensional variational inequalities and complementarity problems

    Francisco Facchinei and Jong-Shi Pang. Finite-dimensional variational inequalities and complementarity problems. Springer, 2003

    2003

  20. [28]

    A variational inequality perspective on generative adversarial networks

    Gauthier Gidel, Hugo Berard, Ga \"e tan Vignoud, Pascal Vincent, and Simon Lacoste-Julien. A variational inequality perspective on generative adversarial networks. arXiv preprint arXiv:1802.10551, 2018

    2018 arXiv

  21. [29]

    Generative adversarial networks

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Communications of the ACM, 63 0 (11): 0 139--144, 2020

    2020

  22. [30]

    Distributed and stochastic optimization methods with gradient compression and local steps

    Eduard Gorbunov. Distributed and stochastic optimization methods with gradient compression and local steps. arXiv preprint arXiv:2112.10645, 2021

    2021 arXiv

  23. [31]

    A unified theory of sgd: Variance reduction, sampling, quantization and coordinate descent

    Eduard Gorbunov, Filip Hanzely, and Peter Richt \'a rik. A unified theory of sgd: Variance reduction, sampling, quantization and coordinate descent. In International Conference on Artificial Intelligence and Statistics, pages 680--690. PMLR, 2020

    2020

  24. [32]

    Marina: Faster non-convex distributed learning with compression

    Eduard Gorbunov, Konstantin P Burlachenko, Zhize Li, and Peter Richt \'a rik. Marina: Faster non-convex distributed learning with compression. In International Conference on Machine Learning, pages 3788--3798. PMLR, 2021

    2021

  25. [33]

    Stochastic extragradient: General analysis and improved rates

    Eduard Gorbunov, Hugo Berard, Gauthier Gidel, and Nicolas Loizou. Stochastic extragradient: General analysis and improved rates. In International Conference on Artificial Intelligence and Statistics, pages 7865--7901. PMLR, 2022

    2022

  26. [34]

    Federated learning with compression: Unified analysis and sharp guarantees

    Farzin Haddadpour, Mohammad Mahdi Kamani, Aryan Mokhtari, and Mehrdad Mahdavi. Federated learning with compression: Unified analysis and sharp guarantees. In International Conference on Artificial Intelligence and Statistics, pages 2350--2358. PMLR, 2021

    2021

  27. [35]

    Natural compression for distributed deep learning

    Samuel Horv \'o th, Chen-Yu Ho, Ludovit Horvath, Atal Narayan Sahu, Marco Canini, and Peter Richt \'a rik. Natural compression for distributed deep learning. In Mathematical and Scientific Machine Learning, pages 129--141. PMLR, 2022

    2022

  28. [36]

    On the convergence of single-call stochastic extra-gradient methods

    Yu-Guan Hsieh, Franck Iutzeler, J\' e r\^ o me Malick, and Panayotis Mertikopoulos. On the convergence of single-call stochastic extra-gradient methods. In Advances in Neural Information Processing Systems, volume 32, pages 6938--6948. Curran Associates, Inc., 2019

    2019

  29. [37]

    Explore aggressively, update conservatively: Stochastic extragradient methods with variable stepsize scaling

    Yu-Guan Hsieh, Franck Iutzeler, J \'e r \^o me Malick, and Panayotis Mertikopoulos. Explore aggressively, update conservatively: Stochastic extragradient methods with variable stepsize scaling. volume 33, pages 16223--16234, 2020

    2020

  30. [38]

    Quantization and training of neural networks for efficient integer-arithmetic-only inference

    Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, and Dmitry Kalenichenko. Quantization and training of neural networks for efficient integer-arithmetic-only inference. In Proceedings of the IEEE conference on computer vision an...

    2018

  31. [39]

    Accelerating stochastic gradient descent using predictive variance reduction

    Rie Johnson and Tong Zhang. Accelerating stochastic gradient descent using predictive variance reduction. Advances in neural information processing systems, 26, 2013

    2013

  32. [40]

    Solving variational inequalities with stochastic mirror-prox algorithm

    Anatoli Juditsky, Arkadi Nemirovski, and Claire Tauvel. Solving variational inequalities with stochastic mirror-prox algorithm. Stochastic Systems, 1 0 (1): 0 17--58, 2011

    2011

  33. [41]

    Advances and open problems in federated learning

    Peter Kairouz, H Brendan McMahan, Brendan Avent, Aur \'e lien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, et al. Advances and open problems in federated learning. Foundations and trends in machine learning , 1...

    2021

  34. [42]

    Federated optimization: Distributed machine learning for on-device intelligence

    Jakub Kone c n \'y , H Brendan McMahan, Daniel Ramage, and Peter Richt \'a rik. Federated optimization: Distributed machine learning for on-device intelligence. arXiv preprint arXiv:1610.02527, 2016

    2016 arXiv

  35. [43]

    Optimal gradient sliding and its application to optimal distributed optimization under similarity

    Dmitry Kovalev, Aleksandr Beznosikov, Ekaterina Borodich, Alexander Gasnikov, and Gesualdo Scutari. Optimal gradient sliding and its application to optimal distributed optimization under similarity. Advances in Neural Information Processing Systems, 35: 0 33494--33507, 2022 a

    2022

  36. [44]

    Optimal algorithms for decentralized stochastic variational inequalities

    Dmitry Kovalev, Aleksandr Beznosikov, Abdurakhmon Sadiev, Michael Persiianov, Peter Richt \'a rik, and Alexander Gasnikov. Optimal algorithms for decentralized stochastic variational inequalities. Advances in Neural Information Processing Systems, 35: 0 31073--31088, 2022 b

    2022

  37. [45]

    Quantizing deep convolutional networks for efficient inference: A whitepaper

    Raghuraman Krishnamoorthi. Quantizing deep convolutional networks for efficient inference: A whitepaper. arXiv preprint arXiv:1806.08342, 2018

    2018 arXiv

  38. [46]

    Deep learning

    Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. nature, 521 0 (7553): 0 436--444, 2015

    2015

  39. [47]

    Federated learning: Challenges, methods, and future directions

    Tian Li, Anit Kumar Sahu, Ameet Talwalkar, and Virginia Smith. Federated learning: Challenges, methods, and future directions. IEEE signal processing magazine, 37 0 (3): 0 50--60, 2020

    2020

  40. [48]

    A decentralized parallel algorithm for training generative adversarial nets

    Mingrui Liu, Wei Zhang, Youssef Mroueh, Xiaodong Cui, Jarret Ross, Tianbao Yang, and Payel Das. A decentralized parallel algorithm for training generative adversarial nets. Advances in Neural Information Processing Systems, 33: 0 11056--11070, 2020 a

    2020

  41. [49]

    A decentralized proximal point-type method for saddle point problems

    Weijie Liu, Aryan Mokhtari, Asuman Ozdaglar, Sarath Pattathil, Zebang Shen, and Nenggan Zheng. A decentralized proximal point-type method for saddle point problems. arXiv preprint arXiv:1910.14380, 2019

    1910 arXiv

  42. [50]

    Adversarial training for large neural language models

    Xiaodong Liu, Hao Cheng, Pengcheng He, Weizhu Chen, Yu Wang, Hoifung Poon, and Jianfeng Gao. Adversarial training for large neural language models. arXiv preprint arXiv:2004.08994, 2020 b

    2004 arXiv

  43. [51]

    Stochastic gradient descent-ascent and consensus optimization for smooth games: Convergence analysis under expected co-coercivity

    Nicolas Loizou, Hugo Berard, Gauthier Gidel, Ioannis Mitliagkas, and Simon Lacoste-Julien. Stochastic gradient descent-ascent and consensus optimization for smooth games: Convergence analysis under expected co-coercivity. Advances in Neural Information Processing Systems, 34: ...

    2021

  44. [52]

    Communication-efficient learning of deep networks from decentralized data

    Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, pages 1273--1282. PMLR, 2017

    2017

  45. [53]

    Shuffling heuristic in variational inequalities: Establishing new convergence guarantees

    Daniil Medyakov, Gleb Molodtsov, Evseev Grigoriy, Egor Petrov, and Aleksandr Beznosikov. Shuffling heuristic in variational inequalities: Establishing new convergence guarantees. In International Conference on Computational Optimization

  46. [54]

    Optimal data splitting in distributed optimization for machine learning

    Daniil Medyakov, Gleb Molodtsov, Aleksandr Beznosikov, and Alexander Gasnikov. Optimal data splitting in distributed optimization for machine learning. In Doklady Mathematics, volume 108, pages S465--S475. Pleiades Publishing Moscow, 2023

    2023

  47. [55]

    Revisiting stochastic extragradient

    Konstantin Mishchenko, Dmitry Kovalev, Egor Shulgin, Peter Richt \'a rik, and Yura Malitsky. Revisiting stochastic extragradient. In International Conference on Artificial Intelligence and Statistics, pages 4573--4582. PMLR, 2020

    2020

  48. [56]

    Distributed learning with compressed gradient differences

    Konstantin Mishchenko, Eduard Gorbunov, Martin Tak \'a c , and Peter Richt \'a rik. Distributed learning with compressed gradient differences. Optimization Methods and Software, pages 1--16, 2024

    2024

  49. [57]

    Non-asymptotic analysis of stochastic approximation algorithms for machine learning

    Eric Moulines and Francis Bach. Non-asymptotic analysis of stochastic approximation algorithms for machine learning. Advances in neural information processing systems, 24, 2011

    2011

  50. [58]

    Efficiency of coordinate descent methods on huge-scale optimization problems

    Yu Nesterov. Efficiency of coordinate descent methods on huge-scale optimization problems. SIAM Journal on Optimization, 22 0 (2): 0 341--362, 2012

    2012

  51. [59]

    Sarah: A novel method for machine learning problems using stochastic recursive gradient

    Lam M Nguyen, Jie Liu, Katya Scheinberg, and Martin Tak \'a c . Sarah: A novel method for machine learning problems using stochastic recursive gradient. In International conference on machine learning, pages 2613--2621. PMLR, 2017

    2017

  52. [60]

    Stochastic variance reduction methods for saddle-point problems

    Balamurugan Palaniappan and Francis Bach. Stochastic variance reduction methods for saddle-point problems. In Advances in Neural Information Processing Systems, pages 1416--1424, 2016

    2016

  53. [61]

    Optimal analysis of method with batching for monotone stochastic finite-sum variational inequalities

    Alexander Pichugin, Maksim Pechin, Aleksandr Beznosikov, A Savchenko, and A Gasnikov. Optimal analysis of method with batching for monotone stochastic finite-sum variational inequalities. In Doklady Mathematics, volume 108, pages S348--S359. Springer, 2023

    2023

  54. [62]

    Method with batching for stochastic finite-sum variational inequalities in non-euclidean setting

    Alexander Pichugin, Maksim Pechin, Aleksandr Beznosikov, Vasilii Novitskii, and Alexander Gasnikov. Method with batching for stochastic finite-sum variational inequalities in non-euclidean setting. Chaos, Solitons & Fractals, 187: 0 115396, 2024

    2024

  55. [63]

    Distributed coordinate descent method for learning with big data

    Peter Richt \'a rik and Martin Tak \'a c . Distributed coordinate descent method for learning with big data. Journal of Machine Learning Research, 17 0 (75): 0 1--25, 2016

    2016

  56. [64]

    A stochastic approximation method

    Herbert Robbins and Sutton Monro. A stochastic approximation method. The annals of mathematical statistics, pages 400--407, 1951

    1951

  57. [65]

    A stochastic gradient method with an exponential convergence \_rate for finite training sets

    Nicolas Roux, Mark Schmidt, and Francis Bach. A stochastic gradient method with an exponential convergence \_rate for finite training sets. Advances in neural information processing systems, 25, 2012

    2012

  58. [66]

    Federated multi-task learning

    Virginia Smith, Chao-Kai Chiang, Maziar Sanjabi, and Ameet S Talwalkar. Federated multi-task learning. Advances in neural information processing systems, 30, 2017

    2017

  59. [67]

    Methods for optimization problems with markovian stochasticity and non-euclidean geometry

    Vladimir Solodkin, Andrew Veprikov, and Aleksandr Beznosikov. Methods for optimization problems with markovian stochasticity and non-euclidean geometry. arXiv preprint arXiv:2408.01848, 2024

    2024

  60. [68]

    Distributed mean estimation with limited communication

    Ananda Theertha Suresh, X Yu Felix, Sanjiv Kumar, and H Brendan McMahan. Distributed mean estimation with limited communication. In International conference on machine learning, pages 3329--3337. PMLR, 2017

    2017

  61. [69]

    Decentralized min-max optimization: Formulations, algorithms and applications in network poisoning attack

    Ioannis Tsaknakis, Mingyi Hong, and Sijia Liu. Decentralized min-max optimization: Formulations, algorithms and applications in network poisoning attack. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 5755--5759. IEEE, 2020

    2020

  62. [70]

    A survey on distributed machine learning

    Joost Verbraeken, Matthijs Wolting, Jonathan Katzy, Jeroen Kloppenburg, Tim Verbelen, and Jan S Rellermeyer. A survey on distributed machine learning. Acm computing surveys (csur), 53 0 (2): 0 1--33, 2020

    2020

  63. [71]

    Theory of games and economic behavior: by J

    John Von Neumann and Oskar Morgenstern. Theory of games and economic behavior: by J. Von Neumann and O. Morgenstern. Princeton university press, 1953

    1953

  64. [72]

    Gradient sparsification for communication-efficient distributed optimization

    Jianqiao Wangni, Jialei Wang, Ji Liu, and Tong Zhang. Gradient sparsification for communication-efficient distributed optimization. Advances in Neural Information Processing Systems, 31, 2018

    2018

  65. [73]

    Terngrad: Ternary gradients to reduce communication in distributed deep learning

    Wei Wen, Cong Xu, Feng Yan, Chunpeng Wu, Yandan Wang, Yiran Chen, and Hai Li. Terngrad: Ternary gradients to reduce communication in distributed deep learning. Advances in neural information processing systems, 30, 2017

    2017

  66. [74]

    Integer quantization for deep learning inference: Principles and empirical evaluation

    Hao Wu, Patrick Judd, Xiaojie Zhang, Mikhail Isaev, and Paulius Micikevicius. Integer quantization for deep learning inference: Principles and empirical evaluation. arXiv preprint arXiv:2004.09602, 2020

    2004 arXiv

  67. [75]

    Error compensated quantized sgd and its applications to large-scale distributed optimization

    Jiaxiang Wu, Weidong Huang, Junzhou Huang, and Tong Zhang. Error compensated quantized sgd and its applications to large-scale distributed optimization. In International conference on machine learning, pages 5325--5333. PMLR, 2018

    2018

  68. [76]

    Freelb: Enhanced adversarial training for natural language understanding

    Chen Zhu, Yu Cheng, Zhe Gan, Siqi Sun, Tom Goldstein, and Jingjing Liu. Freelb: Enhanced adversarial training for natural language understanding. arXiv preprint arXiv:1909.11764, 2019

    1909 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.