Pith. sign in

REVIEW 2 major objections 5 minor 60 references

GADMM: Fast and Communication Efficient Framework for Distributed Machine Learning

T0 review · 2 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read GADMM claims distributed convex learning reaches the global optimum with workers split into two groups, each talking only to two chain neighbors and only half transmitting per round; a dynamic variant keeps the guarantee under rewiring.

desk verdict The static GADMM result is a solid two-block ADMM on a chain; the D-GADMM extension is overclaimed because its proof fixes a dual saddle point that the rewired chain does not respect. read the letter →

arxiv 1909.00047 v3 pith:BGCOWQMF submitted 2019-08-30 cs.LG cs.DCcs.ITcs.NImath.ITstat.ML

Anis Elgabli , Jihong Park , Amrit S. Bedi , Mehdi Bennis , Vaneet Aggarwal This is my paper · ORCID
classification cs.LGcs.DCcs.ITcs.NImath.ITstat.ML MSC 90C2568W15
keywords GADMMADMMdistributedmachinelearningdecentralizedoptimizationcommunicationefficiencyconvextime-varyingnetworktopologyconsensus
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that distributed machine learning over convex losses can be solved optimally without any central server: workers are arranged in a chain, split into a head group and a tail group, each worker exchanges its model only with its two chain neighbors, and at most half the workers transmit per round. The proposed GADMM algorithm provably drives the consensus residuals and the objective gap to zero, giving the same optimality guarantee as standard ADMM while cutting per-round communication load roughly in half. A dynamic variant, D-GADMM, extends the same guarantee to chains that are rewired over time, and the paper argues and simulates that periodic rewiring actually accelerates convergence toward the speed of a fully connected star topology. A sympathetic reader should care because communication, not computation, is the bottleneck in distributed training at scale, and this is a recipe for removing the server and halving uploads without giving up optimality.

What carries the argument

The argument is carried by the chain-structured augmented Lagrangian $L_\rho = \sum_n f_n(\theta_n) + \sum_{n=1}^{N-1}\langle\lambda_n, \theta_n - \theta_{n+1}\rangle + \frac{\rho}{2}\sum_{n=1}^{N-1}\|\theta_n - \theta_{n+1}\|^2$, the alternating head/tail update steps in (11)--(15), and two residuals: the primal residual $r^k_{n,n+1} = \theta^k_n - \theta^k_{n+1}$ measuring neighbor disagreement, and the dual residual $s^k_n$ measuring how far a head worker's update lags a full block-coordinate sweep. The proof centers on a Lyapunov function $V_k$ (equation 32) built from dual errors and neighbor-model deviations; showing $V_{k+1} \le V_k$ each iteration makes the residuals summable, which forces them to zero and then forces the objective gap to zero through the upper and lower bounds of Lemma 1. For D-GADMM the same Lyapunov structure is asserted to survive chain rewiring.

What would settle it

With four workers and simple convex losses of known optimum, alternate the chain between two orderings every iteration and compute $V_k$ from equation 125 using the saddle-point dual $\lambda^*$ of a single fixed ordering; if $V_{k+1} > V_k$ across any rewiring while the unwired GADMM converges, the monotone-decrease step of the D-GADMM proof fails, and the algorithm converges, if at all, for reasons the proof does not state.

Watch

Extended reading notes

Core claim

GADMM rewrites the distributed learning problem as minimizing $\sum_{n=1}^N f_n(\theta_n)$ subject only to the chain constraints $\theta_n = \theta_{n+1}$ for $n = 1,\dots, N-1$, eliminating the global consensus variable $\Theta$ that parameter-server ADMM must broadcast. The workers at odd indices form the head group and those at even indices the tail group. In each iteration, all head workers minimize their local augmented Lagrangian in parallel using the previous tail values, transmit their new models to their two tail neighbors, all tail workers then update in parallel using the fresh head values, and every worker refreshes its adjacent dual variables locally. Theorem 2 states that when every $f_n$ is closed, proper, and convex and the Lagrangian $L_0$ has a saddle point, the primal residual $r^k_{n,n+1} = \theta^k_n - \theta^k_{n+1}$ and the dual residuals $s^k_n$ of the head workers both converge to zero, and $\sum_n f_n(\theta^k_n) \to \sum_n f_n(\theta^\star)$, the global optimum. The same convergence is claimed for D-GADMM when the logical chain is rewired every $\tau$ iterations.

Load-bearing premise

The dynamic variant's proof holds one set of optimal dual variables fixed across all chain rewirings, yet each rewiring changes which constraints the duals must certify, and the paper does not show its error measure keeps decreasing across a topology change.

Editorial extensions

If this is right

  • Per-round bandwidth demand is halved: only $N/2$ workers transmit, each to two neighbors, so each active worker enjoys twice the resources of parameter-server ADMM.
  • Convex distributed learning is solved exactly, not approximately: Theorem 2 guarantees residuals and objective gap go to zero.
  • Time-varying topologies cost nothing in guarantee: D-GADMM claims identical convergence when workers move and the chain is rebuilt.
  • Frequent rewiring speeds training: the simulations show refresh-rate-1 D-GADMM converging faster than standard ADMM with roughly forty times less communication cost.
  • Any chain construction that spans all workers once suffices; the paper shows the optimal construction is TSP-hard, so heuristics are the intended route.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The rewiring speedup suggests D-GADMM's rate is governed by the mixing time of the randomly rewired chain; measuring convergence against the spectral gap of the chain Laplacian across refresh rates would test this.
  • The fixed-$\lambda^*$ proof gap points to a simple repair, re-synchronizing dual variables along the new chain at each rewiring; the simulations skip this step and still converge, so the algorithm may be stronger than its proof.
  • Because GADMM never forms a global average, the framework carries over to mesh-like settings where no worker has links to all others --- a transfer the paper motivates but does not develop.
  • In the authors' own Shannon-link cost model, halving the number of transmitters per round lowers each worker's energy budget for a target rate, a per-device benefit left implicit.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper proposes GADMM, a decentralized ADMM-type algorithm for distributed machine learning. Workers are arranged in a chain and split into head and tail groups; at each iteration one group updates in parallel and exchanges primal variables with its two neighbors, halving the number of contending transmitters per communication round. The authors prove convergence to the global optimum for closed, proper, convex losses under a static chain (Theorem 2), and claim the same convergence for D-GADMM, a variant that rewires the logical chain periodically to handle time-varying topologies. Numerical experiments on linear and logistic regression compare GADMM favorably with LAG, dual averaging, DGD, and other baselines in iterations, communication cost, and runtime, and show that D-GADMM can accelerate convergence by periodic rewiring.

Significance. The static GADMM result is a plausible and useful contribution: the proof in Appendices A and B is a self-contained adaptation of two-block ADMM arguments, including a Lyapunov argument for the parallel group updates, and the communication pattern is a genuine departure from parameter-server ADMM. The empirical study is extensive and directly measures communication cost, which is appropriate for the paper's stated goal. The D-GADMM claim is the main weakness: the convergence proof in Appendix D is not valid as written because it uses a single dual saddle point across all chain rewirings and telescopes a Lyapunov function whose defining index sets change with time. Since the time-varying guarantee is one of the two headline claims in the abstract and Section 6, this gap is load-bearing. If repaired, the paper would be a solid contribution; as it stands, the D-GADMM convergence result is not established.

major comments (2)
  1. [Appendix D, Eqs. (88) and (108)-(110)] The proof of D-GADMM assumes a fixed dual saddle point λ* that remains valid after every topology change. However, the dual feasibility conditions in Eq. (88) depend on the current left and right neighbors: for interior workers they read 0 ∈ ∂f_n(θ*) − λ*_{n_l,k} + λ*_n, and for the first and last workers the edge duals appear with sign changes. When the chain is reordered, the worker occupying each position changes, so the condition that a given λ* must satisfy changes. For affine functions f_n(θ)=a_n^T θ, the required edge duals are cumulative sums of the a_n in chain order, which are order-dependent. Consequently, a single λ* cannot certify the saddle-point inequality in Eq. (108) for every rewiring, and the lower bound in Eq. (110) is not justified. This invalidates the descent argument for V_k and therefore the claimed convergence of D-GADMM.
  2. [Appendix D, Eqs. (125)-(130)] The Lyapunov function V_k in Eq. (125) is defined using the current head set N^k_h and the current neighbor labels n_l,k and n_r,k. When the topology changes, N^{k+1}_h is generally different from N^k_h, and the terms ρ‖θ^{k+1}_{n_l,k}−θ*‖² and ρ‖θ^{k+1}_{n_r,k}−θ*‖² that enter V_{k+1} are not the same terms that appeared in V_k. The step from Eq. (124) to Eq. (126) pairs terms as if the index sets were fixed from one iteration to the next, and the telescoping sum leading to Eq. (130) also requires a fixed set. Even if one were to choose a new saddle point after each rewire, the comparison of V_{k+1} and V_k would need to account for the change of index sets explicitly. This is an independent gap in the D-GADMM proof, on top of the fixed-λ* issue in Eqs. (108)-(110).
minor comments (5)
  1. [Section 6 / Algorithm 2, line 10] The statement 'sends λ^k_n to its right neighbor (worker n_r,k)' is confusing because the preceding paragraph describes each node transmitting its right dual variable to the new right neighbor; please clarify whether the dual variable is indexed by the sending node or by the edge, since the proof reinterprets λ_n after a rewire.
  2. [Appendix D, Eq. (81)] The augmented Lagrangian in Eq. (81) has a mismatch: the inner product uses ⟨λ_n, θ_n − θ_{n_r,k}⟩, while the quadratic penalty is written as (ρ/2)‖θ_n − θ_{n+1}‖². The second term should use the same neighbor notation n_r,k.
  3. [Throughout] There are several spelling and terminology slips: 'iterativly' in Section 3, 'Standared ADMM' in the caption of Fig. 8, and 'ACV' versus 'AVC' in Section 7 and Fig. 6(c). These should be corrected in a revision.
  4. [Theorem 2 statement] The condition that 'the Lagrangian L0 has a saddle point' is used throughout the proof, but L0 is never defined explicitly. Please define L0 as the Lagrangian with ρ=0 and state the saddle-point assumption precisely.
  5. [Appendix D, Eq. (92)] The notation 0∈∂f_N(θ^{k+1}_N)−λ^{k+1}_{N_l,k} is unclear because N_l,k is not a defined neighbor label; it should be written as N_{l,k} or n_{l,k} with a consistent subscript style.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: GADMM's convergence proof is self-contained; the D-GADMM appendix has a proof gap but does not reduce by construction to its inputs.

full rationale

The paper's central claim is Theorem 2, which states that GADMM iterates drive the primal residuals, dual residuals, and objective optimality gap to zero. The proof in Lemma 1 and Appendix B starts from the first-order optimality conditions of the actual GADMM updates in (11)-(15), derives upper and lower bounds on the objective gap in terms of residuals, and constructs a Lyapunov function whose monotone decrease is established from those same optimality conditions. The only external assumption is the standard existence of a saddle point of the Lagrangian L0; this is an assumption, not a fitted parameter, a renamed empirical pattern, or a load-bearing self-citation. No quantity used in the proof is defined in terms of the convergence conclusion, and no benchmark result is relabeled as a prediction. The numerical sections tune the hyperparameters rho and tau, but these are not inputs to the convergence proof, so the empirical comparisons do not force the theorem. Appendix D's D-GADMM proof does contain a load-bearing rigor gap: it uses a fixed dual saddle point lambda* in the lower bound (110) and in the Lyapunov function (125), while the dual feasibility conditions (88) depend on the current chain neighbors, so the saddle-point inequality L0(theta*,lambda*) <= L0(theta^{k+1},lambda*) need not hold after rewiring. That is a correctness or completeness problem, not a circular reduction: the paper does not fit lambda* to the desired convergence claim, and it does not derive the conclusion from a self-citation. The self-citations in the reference list (e.g., Bedi et al. 2019, Park et al. 2019, Jeong et al. 2018, Sriranga et al. 2019) are background and related-work citations and are not load-bearing for Theorem 2 or for the D-GADMM proof. Therefore the circularity score is 0.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The central results rest on standard convex analysis assumptions and on exact local minimization. The main ad hoc premise is the fixed dual saddle point across topology changes in D-GADMM, which is not established. No new physical entities are introduced.

free parameters (2)
  • ρ (penalty parameter) = Tuned per dataset and task; e.g., 3, 5, 7 for linear regression; 2E-3, 3E-3 for logistic regression
    Convergence is proven for any fixed ρ>0, but the empirical convergence speed and communication cost depend strongly on ρ. The paper selects ρ values that give good performance, without a principled tuning rule.
  • τ (refresh period) = Simulation uses 15 iterations; refresh rates 1, 10, 50 in Fig. 8
    Controls how often D-GADMM rewires the logical chain. The paper does not provide a rule for optimal τ, only empirical demonstrations.
assumptions (5)
  • domain assumption Each local loss function f_n is closed, proper, and convex.
    The convergence theorem in Section 5 states this explicitly before Theorem 2. It is the core convexity assumption.
  • domain assumption The Lagrangian L0 has a saddle point.
    Theorem 2 invokes existence of a saddle point, which is standard for convex problems with affine constraints, but not proven in the paper.
  • ad hoc to paper Each worker solves its local subproblem exactly at every iteration.
    The convergence proof relies on the minimizer satisfying the first-order optimality conditions (18) and (22). No inexact solver or stopping tolerance is analyzed.
  • ad hoc to paper For D-GADMM, a single dual saddle point λ* remains valid across all chain rewirings.
    Appendix D uses a fixed λ* in the Lyapunov function and bounds, but the dual feasibility conditions depend on the current neighbor structure. This is not justified and is a likely gap.
  • domain assumption The number of workers N is even, or can be made even without loss of generality.
    The head/tail grouping with equal sizes assumes even N. The paper says 'without loss of generality' but does not explain the odd-N case.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GADMM: Fast and Communication Efficient Framework for Distributed Machine Learning." pith.science (2026). https://pith.science/paper/BGCOWQMF

@misc{pith2026190900047,
  author       = {Pith},
  title        = {Pith review of: GADMM: Fast and Communication Efficient Framework for Distributed Machine Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BGCOWQMF}},
  note         = {Machine review of arXiv:1909.00047}
}
read the original abstract

When the data is distributed across multiple servers, lowering the communication cost between the servers (or workers) while solving the distributed learning problem is an important problem and is the focus of this paper. In particular, we propose a fast, and communication-efficient decentralized framework to solve the distributed machine learning (DML) problem. The proposed algorithm, Group Alternating Direction Method of Multipliers (GADMM) is based on the Alternating Direction Method of Multipliers (ADMM) framework. The key novelty in GADMM is that it solves the problem in a decentralized topology where at most half of the workers are competing for the limited communication resources at any given time. Moreover, each worker exchanges the locally trained model only with two neighboring workers, thereby training a global model with a lower amount of communication overhead in each exchange. We prove that GADMM converges to the optimal solution for convex loss functions, and numerically show that it converges faster and more communication-efficient than the state-of-the-art communication-efficient algorithms such as the Lazily Aggregated Gradient (LAG) and dual averaging, in linear and logistic regression tasks on synthetic and real datasets. Furthermore, we propose Dynamic GADMM (D-GADMM), a variant of GADMM, and prove its convergence under the time-varying network topology of the workers.

Figures

Figures reproduced from arXiv: 1909.00047 by the authors.

Figure 1
Figure 1. An illustration of (a) distributed gradient descent with a parameter server and (b) [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Objective error, total communication cost, and total running time comparison [PITH_FULL_IMAGE:figures/full_fig_p015_2.png] view at source ↗
Figure 3
Figure 3. Objective error, total communication cost, and total running time comparison [PITH_FULL_IMAGE:figures/full_fig_p015_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Objective error, total communication cost, and total running time comparison [PITH_FULL_IMAGE:figures/full_fig_p016_4.png]
Figure 5
Figure 5. Figure 5: Objective error, total communication cost, and total running time comparison [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]
Figure 6
Figure 6. Figure 6: The cumulative distribution function (CDF) of total communication cost (TC) [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]
Figure 7
Figure 7. Figure 7: Objective error, total communication cost, and total running time of D-GADMM [PITH_FULL_IMAGE:figures/full_fig_p018_7.png]
Figure 8
Figure 8. Figure 8: Objective error, total communication cost, and total running time of D-GADMM, [PITH_FULL_IMAGE:figures/full_fig_p019_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

60 extracted references · 51 canonical work pages

  1. [1]

    write newline

    " write newline "" initialize.prev.this.status FUNCTION begin.bib " write newline preamble empty 'skip preamble write newline if " thebibliography " longest.label * " " * write newline " [1] #1 " write newline " url@samestyle " write newline " " write newline " [2] #2 " write newline " =0pt " write newline " " ALTinterwordstretchfactor * " " * write newli...

  2. [2]

    4" FUNCTION default.is.dash.repeated.names #1 FUNCTION default.name.format.string

    11em plus .33em minus .07em 4000 4000 100 4000 4000 500 `\.=1000 = #1 \@IEEEnotcompsoconly \@IEEEcompsoconly #1 * [1] 0pt [0pt][0pt] #1 * [1] 0pt [0pt][0pt] #1 * \| ** #1 \@IEEEauthorblockNstyle \@IEEEcompsocnotconfonly \@IEEEauthorblockAstyle \@IEEEcompsocnotconfonly \@IEEEcompsocconfonly \@IEEEauthordefaulttextstyle \@IEEEcompsocnotconfonly \@IEEEauthor...

    work page 2015

  3. [3]

    write newline

    " write newline "" initialize.prev.this.status FUNCTION begin.bib " write newline preamble empty 'skip preamble write newline if " thebibliography " number.label int.to.str * " " * write newline " [1] #1 " write newline " [1] #1 " write newline " url@samestyle " write newline " " write newline " [2] #2 " write newline " =0pt " write newline " " ALTinterwo...

  4. [4]

    write newline

    " write newline "" initialize.prev.this.status FUNCTION begin.bib " write newline " [1] ^ #1 " write newline preamble empty 'skip preamble write newline if " thebibliography " longest.label * " " * write newline " [1] #1 " write newline " url@samestyle " write newline " " write newline " [2] #2 " write newline " =0pt " write newline " " ALTinterwordstretc...

  5. [5]

    write newline

    " write newline "" initialize.prev.this.status FUNCTION begin.bib " write newline preamble empty 'skip preamble write newline if " thebibliography " longest.label * " " * write newline " [1] #1 " write newline " url@samestyle " write newline " " write newline " [2] #2 " write newline " =0pt " write newline " " ALTinterwordstretchfactor * " " * write newli...

  6. [6]

    write newline

    " write newline "" initialize.prev.this.status FUNCTION begin.bib " write newline preamble empty 'skip preamble write newline if " thebibliography " number.label int.to.str * " " * write newline " [1] #1 " write newline " [1] #1 " write newline " url@samestyle " write newline " " write newline " [2] #2 " write newline " =0pt " write newline " " ALTinterwo...

  7. [7]

    Distributed large-scale natural graph factorization

    Amr Ahmed, Nino Shervashidze, Shravan Narayanamurthy, Vanja Josifovski, and Alexander J Smola. Distributed large-scale natural graph factorization. In Proceedings of World Wide Web, Rio de Janeiro, Brazil, May 2013

    work page 2013

  8. [8]

    Asynchronous saddle point algorithm for stochastic optimization in heterogeneous networks

    Amrit Singh Bedi, Alec Koppel, and Rajawat Ketan. Asynchronous saddle point algorithm for stochastic optimization in heterogeneous networks. IEEE Transactions on Signal Processing, 67 0 (7): 0 1742--1757, 2019. ISSN 1053-587X. doi:10.1109/TSP.2019.2894803

    arXiv 2019

Show all 60 references
  1. [9]

    A convergent incremental gradient method with a constant step size

    Doron Blatt, Alfred O Hero, and Hillel Gauchman. A convergent incremental gradient method with a constant step size. SIAM Journal on Optimization, 18 0 (1): 0 29--51, 2007

    2007

  2. [10]

    The n-city travelling salesman problem: Statistical mechanics and the metropolis algorithm

    Ernesto Bonomi and Jean-Luc Lutton. The n-city travelling salesman problem: Statistical mechanics and the metropolis algorithm. SIAM review, 26 0 (4): 0 551--568, 1984

    1984

  3. [11]

    Distributed optimization and statistical learning via the alternating direction method of multipliers

    Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, Jonathan Eckstein, et al. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends in Machine learning , 3 0 (1): 0 1--122, 2011

    2011

  4. [12]

    Multi-agent distributed optimization via inexact consensus admm

    Tsung-Hui Chang, Mingyi Hong, and Xiangfeng Wang. Multi-agent distributed optimization via inexact consensus admm. IEEE Transactions on Signal Processing, 63 0 (2): 0 482--497, 2014 a

    2014

  5. [13]

    Distributed constrained optimization by consensus-based primal-dual perturbation method

    Tsung-Hui Chang, Angelia Nedi \'c , and Anna Scaglione. Distributed constrained optimization by consensus-based primal-dual perturbation method. IEEE Transactions on Automation and Control, 59 0 (6): 0 1524--1538, 2014 b

    2014

  6. [14]

    The direct extension of admm for multi-block convex minimization problems is not necessarily convergent

    Caihua Chen, Bingsheng He, Yinyu Ye, and Xiaoming Yuan. The direct extension of admm for multi-block convex minimization problems is not necessarily convergent. Mathematical Programming, 155 0 (1-2): 0 57--79, 2016

    2016

  7. [15]

    Lag: Lazily aggregated gradient for communication-efficient distributed learning

    Tianyi Chen, Georgios Giannakis, Tao Sun, and Wotao Yin. Lag: Lazily aggregated gradient for communication-efficient distributed learning. Advances in Neural Information Processing Systems, 31: 0 5055--5065, 2018

    2018

  8. [16]

    Large scale distributed deep networks

    Jeffrey Dean, Greg Corrado, Rajat Monga, Kai Chen, Matthieu Devin, Mark Mao, Andrew Senior, Paul Tucker, Ke Yang, Quoc V Le, et al. Large scale distributed deep networks. Advances in Neural Information Processing Systems, 25: 0 1223--1231, 2012

    2012

  9. [17]

    Parallel multi-block admm with o(1/k) convergence

    Wei Deng, Ming-Jun Lai, Zhimin Peng, and Wotao Yin. Parallel multi-block admm with o(1/k) convergence. Journal of Scientific Computing, 71 0 (2): 0 712--736, 2017

    2017

  10. [18]

    Ant colonies for the travelling salesman problem

    Marco Dorigo and Luca Maria Gambardella. Ant colonies for the travelling salesman problem. biosystems, 43 0 (2): 0 73--81, 1997

    1997

  11. [19]

    UCI machine learning repository, 2017

    Dheeru Dua and Casey Graff. UCI machine learning repository, 2017. URL http://archive.ics.uci.edu/ml

    2017

  12. [20]

    Dual averaging for distributed optimization: Convergence analysis and network scaling

    John C Duchi, Alekh Agarwal, and Martin J Wainwright. Dual averaging for distributed optimization: Convergence analysis and network scaling. IEEE Transactions on Automatic control, 57 0 (3): 0 592--606, 2011

    2011

  13. [21]

    A dual algorithm for the solution of non linear variational problems via finite element approximation

    Daniel Gabay and Bertrand Mercier. A dual algorithm for the solution of non linear variational problems via finite element approximation. Institut de recherche d'informatique et d'automatique, 1975

    1975

  14. [22]

    Sur l'approximation, par \'e l \'e ments finis d'ordre un, et la r \'e solution, par p \'e nalisation-dualit \'e d'une classe de probl \`e mes de dirichlet non lin \'e aires

    Roland Glowinski and A Marroco. Sur l'approximation, par \'e l \'e ments finis d'ordre un, et la r \'e solution, par p \'e nalisation-dualit \'e d'une classe de probl \`e mes de dirichlet non lin \'e aires. ESAIM: Mathematical Modelling and Numerical Analysis-Mod \'e lisation ...

    1975

  15. [23]

    On the convergence rate of incremental aggregated gradient algorithms

    Mert Gurbuzbalaban, Asuman Ozdaglar, and Pablo A Parrilo. On the convergence rate of incremental aggregated gradient algorithms. SIAM Journal on Optimization, 27 0 (2): 0 1035--1048, 2017

    2017

  16. [24]

    A class of projection and contraction methods for monotone variational inequalities

    Bingsheng He. A class of projection and contraction methods for monotone variational inequalities. Applied Mathematics and Optimization, 35 0 (1): 0 69--76, Jan 1997. ISSN 1432-0606. doi:10.1007/BF02683320. URL https://doi.org/10.1007/BF02683320

    1997 doi

  17. [25]

    On the o(1/n) convergence rate of the douglas--rachford alternating direction method

    Bingsheng He and Xiaoming Yuan. On the o(1/n) convergence rate of the douglas--rachford alternating direction method. SIAM Journal on Numerical Analysis, 50 0 (2): 0 700--709, 2012

    2012

  18. [26]

    On non-ergodic convergence rate of douglas--rachford alternating direction method of multipliers

    Bingsheng He and Xiaoming Yuan. On non-ergodic convergence rate of douglas--rachford alternating direction method of multipliers. Numerische Mathematik, 130 0 (3): 0 567--577, 2015

    2015

  19. [27]

    On full jacobian decomposition of the augmented lagrangian method for separable convex programming

    Bingsheng He, Liusheng Hou, and Xiaoming Yuan. On full jacobian decomposition of the augmented lagrangian method for separable convex programming. SIAM Journal on Optimization, 25 0 (4): 0 2274--2312, 2015

    2015

  20. [28]

    Cola: Decentralized linear learning

    Lie He, An Bian, and Martin Jaggi. Cola: Decentralized linear learning. In Advances in Neural Information Processing Systems, pages 4536--4546, 2018

    2018

  21. [29]

    Communication-efficient distributed dual coordinate ascent

    Martin Jaggi, Virginia Smith, Martin Tak \'a c, Jonathan Terhorst, Sanjay Krishnan, Thomas Hofmann, and Michael I Jordan. Communication-efficient distributed dual coordinate ascent. Advances in Neural Information Processing Systems, 27: 0 3068--3076, 2014

    2014

  22. [30]

    Fast distributed gradient methods

    Du s an Jakoveti \'c , Joao Xavier, and Jos \'e MF Moura. Fast distributed gradient methods. IEEE Transactions on Automation and Control Automa. Control, 59 0 (5): 0 1131--1146, 2014

    2014

  23. [31]

    Communication-efficient on-device machine learning: Federated distillation and augmentation under non-iid private data

    Eunjeong Jeong, Seungeun Oh, Hyesung Kim, Jihong Park, Mehdi Bennis, and Seong-Lyun Kim. Communication-efficient on-device machine learning: Federated distillation and augmentation under non-iid private data. presented at Neural Information Processing Systems Workshop on Machi...

    2018 arXiv

  24. [32]

    Jordan, Jason D

    Michael I. Jordan, Jason D. Lee, and Yun Yang. Communication-efficient distributed statistical inference. Journal of the American Statistical Association, 2018

    2018

  25. [33]

    Proximity without consensus in online multiagent optimization

    Alec Koppel, Brian M Sadler, and Alejandro Ribeiro. Proximity without consensus in online multiagent optimization. IEEE Transactions on Signal Processing, 65 0 (12): 0 3062--3077, 2017

    2017

  26. [34]

    Communication-efficient algorithms for decentralized and stochastic optimization

    Guanghui Lan, Soomin Lee, and Yi Zhou. Communication-efficient algorithms for decentralized and stochastic optimization. Mathematical Programming, pages 1--48, 2017

    2017

  27. [35]

    Some simple applications of the travelling salesman problem

    Jan Karel Lenstra and AHG Rinnooy Kan. Some simple applications of the travelling salesman problem. Journal of the Operational Research Society, 26 0 (4): 0 717--733, 1975

    1975

  28. [36]

    Distributed delayed proximal gradient methods

    Mu Li, David G Andersen, and Alexander Smola. Distributed delayed proximal gradient methods. presented at Neural Information Processing Systems Workshop on Optimization for Machine Learning, Lake Tahoe, NV, USA, December 2013

    2013

  29. [37]

    Communication efficient distributed machine learning with the parameter server

    Mu Li, David G Andersen, Alexander J Smola, and Kai Yu. Communication efficient distributed machine learning with the parameter server. Advances in Neural Information Processing Systems, 27: 0 19--27, 2014

    2014

  30. [38]

    Splitting algorithms for the sum of two nonlinear operators

    Pierre-Louis Lions and Bertrand Mercier. Splitting algorithms for the sum of two nonlinear operators. SIAM Journal on Numerical Analysis, 16 0 (6): 0 964--979, 1979

    1979

  31. [39]

    Communication-censored ADMM for decentralized consensus optimization

    Yaohua Liu, Wei Xu, Gang Wu, Zhi Tian, and Qing Ling. Communication-censored ADMM for decentralized consensus optimization. IEEE Transactions on Signal Processing, 67 0 (10): 0 2565--2579, 2019

    2019

  32. [40]

    Distributed optimization with arbitrary local solvers

    Chenxin Ma, Jakub Kone c n \`y , Martin Jaggi, Virginia Smith, Michael I Jordan, Peter Richt \'a rik, and Martin Tak \'a c . Distributed optimization with arbitrary local solvers. Optimization Methods and Software, 32 0 (4): 0 813--848, 2017

    2017

  33. [41]

    On the capacity of channels with gaussian and non-gaussian noise

    Ian W McKeague. On the capacity of channels with gaussian and non-gaussian noise. Information and Control, 51 0 (2): 0 153--173, 1981

    1981

  34. [42]

    Brendan McMahan, Ramage Daniel Moore, Eider, Seth Hampson, and Blaise Ag\" u era yArcas

    H. Brendan McMahan, Ramage Daniel Moore, Eider, Seth Hampson, and Blaise Ag\" u era yArcas. Communication-efficient learning of deep networks from decentralized data. In Proceedings of Artificial Intelligence and Statistics, Fort Lauderdale, FL, USA, April 2017

    2017

  35. [43]

    Distributed optimization over time-varying directed graphs

    Angelia Nedi \'c and Alex Olshevsky. Distributed optimization over time-varying directed graphs. IEEE Trans. Automa. Control, 60 0 (3): 0 601--615, 2014

    2014

  36. [44]

    Distributed subgradient methods for multi-agent optimization

    Angelia Nedi\' c and Asuman Ozdaglar. Distributed subgradient methods for multi-agent optimization. IEEE Transactions on Automation and Control, 54 0 (1): 0 48--61, 2009

    2009

  37. [45]

    Achieving geometric convergence for distributed optimization over time-varying graphs

    Angelia Nedic, Alex Olshevsky, and Wei Shi. Achieving geometric convergence for distributed optimization over time-varying graphs. SIAM Journal on Optimization, 27 0 (4): 0 2597--2633, 2017

    2017

  38. [46]

    Network topology and communication-computation tradeoffs in decentralized optimization

    Angelia Nedi \'c , Alex Olshevsky, and Michael G Rabbat. Network topology and communication-computation tradeoffs in decentralized optimization. Proceedings of the IEEE, 106 0 (5): 0 953--976, 2018

    2018

  39. [47]

    Wireless network intelligence at the edge

    Jihong Park, Sumudu Samarakoon, Mehdi Bennis, and M \'e rouane Debbah. Wireless network intelligence at the edge. to appear in Proceedings of the IEEE [Online]. Early access is available at: https://ieeexplore.ieee.org/document/8865093, November 2019

    2019

  40. [48]

    Parallel distributed approaches to combinatorial optimization: benchmark studies on traveling salesman problem

    Carsten Peterson. Parallel distributed approaches to combinatorial optimization: benchmark studies on traveling salesman problem. Neural computation, 2 0 (3): 0 261--269, 1990

    1990

  41. [49]

    Optimal algorithms for non-smooth distributed optimization in networks

    Kevin Scaman, Francis Bach, S \'e bastien Bubeck, Laurent Massouli \'e , and Yin Tat Lee. Optimal algorithms for non-smooth distributed optimization in networks. In Advances in Neural Information Processing Systems, pages 2740--2749, 2018

    2018

  42. [50]

    Minimizing finite sums with the stochastic average gradient

    Mark Schmidt, Nicolas Le Roux, and Francis Bach. Minimizing finite sums with the stochastic average gradient. Mathematical Programming, 162 0 (1-2): 0 83--112, 2017

    2017

  43. [51]

    A proximal gradient algorithm for decentralized composite optimization

    Wei Shi, Qing Ling, Gang Wu, and Wotao Yin. A proximal gradient algorithm for decentralized composite optimization. IEEE Transactions on Signal Processing, 63 0 (22): 0 6013--6023, 2015

    2015

  44. [52]

    Murthy, and Vaneet Aggarwal

    Nandan Sriranga, Chandra R. Murthy, and Vaneet Aggarwal. A method to improve consensus averaging using quantized admm. In 2019 IEEE International Symposium on Information Theory (ISIT). IEEE, 2019

    2019

  45. [53]

    Distributed mean estimation with limited communication

    Ananda Theertha Suresh, Felix X Yu, Sanjiv Kumar, and H Brendan McMahan. Distributed mean estimation with limited communication. Proceedings of Machine Learning Research, 70: 0 3329--3337, 2017

    2017

  46. [54]

    Distributed consensus over network with noisy links

    Behrouz Touri and Angelia Nedic. Distributed consensus over network with noisy links. In Proceedings of International Conference on Information Fusion, Seattle, WA, USA, July 2009

    2009

  47. [55]

    Tsianos, Sean Lawlor, and Michael G

    Konstantinos I. Tsianos, Sean Lawlor, and Michael G. Rabbat. Consensus-based distributed optimization: Practical issues and applications in large-scale machine learning. In Proceedings of Allerton Conference on Communication, Control, and Computing, Monticello, IL, USA, October 2012

    2012

  48. [56]

    Parallel direction method of multipliers

    Huahua Wang, Arindam Banerjee, and Zhi-Quan Luo. Parallel direction method of multipliers. Advances in Neural Information Processing Systems, 27: 0 181--189

  49. [57]

    Group-based alternating direction method of multipliers for distributed linear classification

    Huihui Wang, Yang Gao, Yinghuan Shi, and Ruili Wang. Group-based alternating direction method of multipliers for distributed linear classification. IEEE Transactions on Cybernetics, 47 0 (11): 0 3568--3582, 2017

    2017

  50. [58]

    Leung, Christian Makaya, Ting He, and Kevin Chan

    Shiqiang Wang, Tiffany Tuor, Theodoros Salonidis, Kin K. Leung, Christian Makaya, Ting He, and Kevin Chan. Adaptive federated learning in resource constrained edge computing systems. ArXiv preprint, abs/1804.05271, 2018

    2018 arXiv

  51. [59]

    Communication-efficient algorithms for statistical optimization

    Yuchen Zhang, Martin J Wainwright, and John C Duchi. Communication-efficient algorithms for statistical optimization. Advances in Neural Information Processing Systems, 25: 0 1502--1510, 2012

    2012

  52. [60]

    Quantized consensus ADMM for multi-agent distributed optimization

    Shengyu Zhu, Mingyi Hong, and Biao Chen. Quantized consensus ADMM for multi-agent distributed optimization. In Proceedings of International Conference on Acoustics, Speech, and Signal Processing, Shanghai, China, March 2016

    2016

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.