Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Adaptive Stepsize Selection in Decentralized Convex Optimization

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper proposes a decentralized optimization algorithm that adaptively selects agent stepsizes using only neighbor-to-neighbor communication, requiring no knowledge of smoothness constants, strong-convexity constants, network…

desk verdict A genuinely clever parameter-free decentralized design with real claims, but v1 cannot be verified because the main theorems are announced without proofs and Theorem 10's boundedness hypotheses are not enforced by Algorithm 1 as written. read the letter →

arxiv 2507.23725 v1 pith:ZKQVK3E4 submitted 2025-07-31 math.OC

Ilya Kuruzov , Xiaokai Chen , Gesualdo Scutari , Alexander Gasnikov This is my paper
classification math.OC MSC 90C2590C3068W15
keywords decentralizedoptimizationadaptivestepsizebacktrackinglinesearchmin-consensusgradienttrackingconvexparameter-freealgorithmnetworkdiameterestimation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes a decentralized optimization algorithm in which every agent selects its own stepsize by local backtracking and one-hop communication, requiring no knowledge of smoothness constants, strong-convexity constants, or network structure, and with agents not needing to know whether the problem is strongly convex. The central claim is that this fully adaptive algorithm still converges linearly when the losses are strongly convex and sublinearly when they are merely convex, matching the best-known rates of nonadaptive methods that require global parameter tuning. If true, it removes the most delicate practical bottleneck in distributed optimization: choosing a stepsize that works across a heterogeneous network. A user could run the method out of the box in settings where manual tuning is infeasible or prohibitively expensive.

What carries the argument

The load-bearing mechanism is the pair of stepsize matrices $\Theta^k$ (primal) and $\Pi^k$ (dual), updated by three neighbor-only routines: a backtracking line search for each local stepsize, a local min-consensus that synchronizes stepsizes within one hop, and a running-diameter estimator $d_i^k$ that doubles whenever local min-consensus has not yet converged. The dual stepsizes $\Pi^k$ track the minimum primal stepsize through scalar auxiliary variables, so the difference $\pi^k_{\max} - \pi^k_{\min}$ stays negligible; Lemma 5 then bounds the deviation of Algorithm 1 from the global-min-consensus benchmark by $O(|\theta^k_{\min}-\pi^k_{\min}| + (\theta^k_{\max}-\theta^k_{\min}) R^k + (\pi^k_{\max}-\pi^k_{\min})\max(\|X^\star\|,R^k))$. The merit functions $V^k$ for the strongly convex case and $M(X)$ for the convex case certify that these discrepancies do not break convergence.

What would settle it

Run the base Algorithm 1 exactly as written, without the bound-enforcing modifications, on a connected graph with smooth convex losses whose backtracking stepsizes keep the local min-consensus value from ever shrinking, and check whether the iterates leave every fixed ball; if they do, the displayed $O(1/\varepsilon)$ guarantee of Theorem 10 as stated would not hold.

Watch

Extended reading notes

Core claim

The paper's discovery is that stepsize adaptivity and provable worst-case rates can coexist in decentralized gradient tracking if one introduces separate primal and dual stepsize variables and continuously re-aligns them. Algorithm 1 couples a local backtracking rule for primal stepsizes with a neighbor-only protocol that tracks the running minimum of those stepsizes through auxiliary variables and an adaptively estimated graph-diameter horizon. The analysis shows the resulting iterates stay close to a benchmark algorithm that uses the global minimum stepsize: Lemma 5 controls the one-step discrepancy between the two, so primal-dual heterogeneity never accumulates. Consequently, strongly convex smooth problems converge linearly with complexity $\tilde O(\kappa/(1-\lambda_2(\tilde W)) \log(1/\varepsilon) + d_G)$, with $\kappa$ evaluated only over the convex hull of the trajectory, and merely convex smooth problems converge at the ergodic $O(1/\varepsilon)$ rate; the paper states this is the first adaptive decentralized method with proven convergence in the nonstrongly convex case.

Load-bearing premise

The sublinear-rate guarantee for merely convex problems assumes the iterates stay inside a fixed bounded region, and the base Algorithm 1 by itself is not shown to enforce that bound; only optional modifications are sketched to make it hold.

Editorial extensions

If this is right

  • For strongly convex problems, no agent needs $L$, $\mu$, the gossip spectrum, or the graph diameter; linear convergence holds automatically.
  • For merely convex problems, the algorithm is the first adaptive decentralized method with a proven $O(1/\varepsilon)$ ergodic convergence rate.
  • Communication remains one-hop and scalar for stepsize coordination: no network-wide flooding is required, unlike the prior adaptive method whose guarantees needed global min-consensus.
  • Because the constants are measured only over the iterates' convex hull, problems whose curvature is benign along the trajectory can converge much faster than worst-case global condition-number bounds suggest.
  • The adaptive diameter estimate can converge to a value smaller than the true graph diameter, avoiding unnecessary synchronization delays on structured graphs such as line graphs with alternating costs.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper's claims, the dual-stepsizes-tracking-a-minimum idea looks transferable: any decentralized gradient-tracking scheme could be retrofitted with the same $\Pi^k$ update to become parameter-free, as long as primal stepsizes are generated by some convergent local rule.
  • The effective-diameter estimator is essentially an online estimate of consensus mixing time, so it could be reused in averaging or flooding protocols on unknown or slowly changing networks, not only for stepsize synchronization.
  • A natural testable extension is stochastic gradients: the current analysis assumes exact gradient evaluations, and it is open whether the same neighbor-only adaptivity survives gradient noise without additional assumptions.
  • One could test whether using the adaptive diameter estimate in a fixed-budget setting yields better wall-clock performance on sparse networks than using the true diameter, since the estimate may shrink synchronization delays.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a fully decentralized, parameter-free algorithm for minimizing the average of smooth (strongly) convex losses over a fixed undirected graph. Each agent runs local backtracking for its primal stepsize, synchronizes it via neighbor-only min-consensus, tracks a dual stepsize via scalar auxiliary variables and an adaptively estimated horizon, and updates primal/dual variables in a primal-dual gradient-tracking form. The main claims are linear convergence for strongly convex losses (Theorem 9) and O(1/epsilon) sublinear convergence for merely convex losses (Theorem 10), with rates matching nonadaptive parameter-dependent methods and requiring no knowledge of smoothness, strong convexity, or network spectral parameters. Numerical experiments on quadratics, ridge regression, and logistic regression compare favorably with the authors' prior global-consensus method [17] and with EXTRA.

Significance. If the stated results are correct, the contribution is significant: it would provide the first adaptive decentralized method with a provable sublinear rate for purely convex problems, and it removes the global flooding/consensus step of the authors' prior work [17] while preserving linear convergence in the strongly convex case. The dual stepsize tracker and the adaptive effective-diameter estimation are genuinely new design elements, and the numerical evidence supports the qualitative claims of stability and competitiveness. However, the central convergence statements are currently unverifiable: Theorems 9 and 10 and the supporting Lemmas 4, 5, 6, and 8 are stated without proofs, and Theorem 10's hypotheses are not enforced by Algorithm 1 as written. The paper also does not provide code or machine-checked artifacts, so the experimental claims are not independently reproducible from the manuscript.

major comments (3)
  1. [Section 4.1, Theorem 9] The definition of the condition number kappa in Theorem 9 is circular. The text states that kappa is the condition number of each f_i restricted to the convex hull of {x*, {x_i^k, x_i^{k+1/2}}_{k=0}^{N_epsilon}}, while N_epsilon is exactly the quantity being bounded in Eq. (11). Since the right-hand side of Eq. (11) depends on N_epsilon through kappa, the displayed complexity bound cannot be interpreted as a standard a priori guarantee. This needs to be redefined, for example with kappa evaluated on a problem-dependent compact set known before the iteration bound, or with an explicit statement that the bound is implicit and solved for N_epsilon.
  2. [Section 4.2, Theorem 10] The sublinear guarantee in Theorem 10 assumes R >= ||X^{k+1} - X*|| and c_theta >= L theta^k_min for all k, but neither condition is enforced by Algorithm 1 as written. The paragraph 'On the compactness of {X^k, Y^k}' concedes this and sketches two modifications: taking gamma_k = 1 for all k beyond some finite index, or introducing binary variables h_i^k defined in Eq. (12). These modifications are not part of the base Algorithm 1, and no proof is provided that either modification yields the uniform bound R and the constant c_theta required by Theorem 10. In particular, the binary-variable rule bounds max(||x_i^k-x_i^0||, theta_i^{k-1} ||y_i^k-y_i^0||), which does not directly imply a uniform bound on ||X^{k+1}-X*||. Thus the stated O(1/epsilon) rate for merely convex problems is not established for the algorithm that is actually proposed and tested.
  3. [Section 4, Theorems 9 and 10 and Lemmas 4-6, 8] The convergence analysis is presented entirely through statements of lemmas and theorems without proofs. Since Lemmas 4, 5, 6, and 8 and Theorems 9 and 10 are the whole basis for the paper's central claims, a reader cannot verify that the merit functions decrease, that the diameter-estimation scheme terminates correctly, or that the rates in Eqs. (11) and (13) follow. This is an omission rather than necessarily a fatal error, but it must be remedied before the claims can be accepted; proofs or a detailed appendix are required.
minor comments (5)
  1. [Assumption 1(ii)] The clause 'each agent i has access only to its own function f_i, but mu and L' appears to be missing 'not'; it should read 'but not mu and L', otherwise it contradicts the stated goal of parameter-free operation.
  2. [Algorithm 1, step (S.3)] The word 'axillary' should be 'auxiliary'.
  3. [Section 5.2] The phrase 'a_ij in R^200 are data problem' should probably be 'are data points'; also, the notation b_ij in {0,1} conflicts with the earlier use of b_i for linear-regression labels.
  4. [Section 4.2, Theorem 10 and Eq. (13)] The definition of the merit function M(X) uses delta > 0, which was introduced as a backtracking line-search parameter; it would help to state explicitly that the same constant is used in both places and that M is well-defined for the ergodic average \hat{X}_k.
  5. [General] The paper does not include a supplementary proof appendix or a link to code for the numerical experiments; adding either would substantially improve verifiability.

Circularity Check

0 steps flagged · score 1.0 of 10

No circular reduction: the adaptive dual-stepsize scheme and convex-case rate are not restatements of [17] or of fitted inputs; the serious caveats are omitted proofs and unenforced compactness assumptions, which are support gaps, not circularity.

full rationale

I found no step in which a claimed prediction or rate reduces by construction to an input or to a self-citation. Algorithm 1 (Sec. 3, S.1–S.6) introduces dual stepsizes Π^k and the effective-diameter tracker d^k_i via update rules (8)–(10); these are new mechanisms, not a renaming of the benchmark Algorithm (1) from [17]. The convergence theorems are conditional: Theorem 10 (Sec. 4.2) assumes R ≥ ‖X^{k+1} − X*‖ and c_θ ≥ Lθ^k_min, and the paper explicitly concedes in “On the compactness of {X^k,Y^k}” that base Algorithm 1 does not enforce these, offering instead two optional modifications (γ_k = 1 eventually, or binary variables h_i^k in Eq. (12)). This is an unproven hypothesis gap, not a fitted parameter renamed as a prediction. Theorem 9's rate is benchmarked against [17] but not derived from it by substitution; the new dual-stepsize update (5) and local min-consensus tracker differ structurally. The only self-citations to [17] are background uses: the fixed-point property “it must be X* = 1(x*)^⊤, with x* being an optimal solution of (P) [17]” (Sec. 3, before Lemma 4) and the rate comparison after Theorem 9; these are not load-bearing uniqueness claims and do not force the paper's conclusions. I also flag that Theorems 9 and 10 are stated without proofs in this v1 and Lemma 8's fixed-point characterization is asserted rather than derived; these are omitted-proof and completeness concerns, not circularity. Overall score 1.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The algorithm introduces no physical or extra-theoretic entities, only internal state variables (Pi, etheta, d_i^k) that are part of the algorithmic construction. The main load-bearing extra assumption beyond the standard smooth-convex model is the boundedness/uniform-stepsize condition in Theorem 10.

free parameters (5)
  • delta (backtracking line-search parameter) = user-set in (0,1]; e.g., 0.5 or 0.9
    Appears in Algorithm 1 (S.2) and in the backtracking oracle; any value in (0,1] works, so it is a benign hyperparameter, not data-fitted.
  • gamma_k (stepsize growth sequence) = user-chosen, e.g., gamma_k = (k+2)/(k+1) in experiments; theory allows gamma_k <= ((k+beta1)/(k+1))^beta2
    Influences how fast stepsizes can grow between backtracking reductions; requires gamma->1 and product gamma_k diverging, but no problem-specific fitting.
  • d0 (initial diameter estimate) = arbitrary positive integer, e.g., 1
    The adaptive diameter tracker (S.4) starts from d0 and doubles on failure; the algorithm is designed to be insensitive to d0.
  • c (gossip mixing weight) = user-set in (0,1/2]
    W = (1-c)I + c fW; the convergence rate depends on c, but any c in (0,1/2] is allowed.
  • initial stepsizes theta^{-1}, pi^{-1}, etheta^{-1} = arbitrary positive numbers
    Initial primal and dual stepsizes; enter V0/log terms in the rate but are not tuned to the problem.
assumptions (3)
  • domain assumption Each f_i is L-smooth and mu-strongly convex on R^d, with mu >= 0 (Assumption 1).
    This is the standard smoothness/strong-convexity model for the decentralized optimization problem (P); the algorithm needs no knowledge of L or mu.
  • domain assumption The communication graph G is undirected, static, and connected, and the gossip matrix W is doubly stochastic with W in W_G and c in (0,1/2] (Assumption 2, Definition 3).
    Standard network model in the decentralized optimization literature.
  • ad hoc to paper Theorem 10 assumes existence of R and c_theta such that R >= ||X^{k+1}-X*|| and c_theta >= L theta^k_min for all k.
    The base Algorithm 1 does not explicitly enforce a uniform R; the paper sketches optional binary-variable (h_i^k) or gamma_k=1 modifications to ensure boundedness, so the convex-rate claim is conditional on this.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Adaptive Stepsize Selection in Decentralized Convex Optimization." pith.science (2026). https://pith.science/paper/ZKQVK3E4

@misc{pith2026250723725,
  author       = {Pith},
  title        = {Pith review of: Adaptive Stepsize Selection in Decentralized Convex Optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZKQVK3E4}},
  note         = {Machine review of arXiv:2507.23725}
}
read the original abstract

We study decentralized optimization where multiple agents minimize the average of their (strongly) convex, smooth losses over a communication graph. Convergence of the existing decentralized methods generally hinges on an apriori, proper selection of the stepsize. Choosing this value is notoriously delicate: (i) it demands global knowledge from all the agents of the graph's connectivity and every local smoothness/strong-convexity constants--information they rarely have; (ii) even with perfect information, the worst-case tuning forces an overly small stepsize, slowing convergence in practice; and (iii) large-scale trial-and-error tuning is prohibitive. This work introduces a decentralized algorithm that is fully adaptive in the choice of the agents' stepsizes, without any global information and using only neighbor-to-neighbor communications--agents need not even know whether the problem is strongly convex. The algorithm retains strong guarantees: it converges at \emph{linear} rate when the losses are strongly convex and at \emph{sublinear} rate otherwise, matching the best-known rates of (nonadaptive) parameter-dependent methods.

Figures

Figures reproduced from arXiv: 2507.23725 by the authors.

Figure 1
Figure 1. Algorithm from [17] using local and global min-consensus, applied to ridge regression. neighbors. While practically more feasible, this local approach results in weaker theoretical guarantees, requiring stronger assumptions like bounded iterates, and yields non-monotonic convergence trajectories. Consequently, decentralized termination criteria become hard to implement, and error trajectories frequently show transie… view at source ↗
Figure 2
Figure 2. Strongly convex quadratic program on different graphs: (left) Line graph; (middle) Erdős-Rényi Graph with edge activation probability p = 0.1; (right) Erdős-Rényi Graph with p = 0.5 5.1.1 Dependence on the condition number Let us consider now fi(x) = ∥Aix − bi∥ 2 + λ∥x∥ 2/2 so that changing λ one can control the condition number of the problem. Graphs and matrices Ai , bi are generated as described in [PITH_FULL_IM… view at source ↗
Figure 3
Figure 3. Ridge regression: Number of iterations N for ∥XN − X⋆ ∥ ≤ 10−5 versus the condition number of agents’ looses on different graphs; (a) Line graph; (b) Erdős-Rényi Graph, with edge activation probability p = 0.1; and (c) Erdős-Rényi Graph, with edge activation probability p = 0.5. Results are reported in [PITH_FULL_IMAGE:figures/full_fig_p012_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Strongly convex quadratic program on graph-line with different number of nodes: number of iterations N for ∥XN − X⋆∥ ≤ 10−5 versus graph diameter [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]
Figure 5
Figure 5. Figure 5: Logistic regression on different graphs: (left) Line graph; (middle) Erdős-Rényi Graph with edge activation probability p = 0.1; (right) Erdős-Rényi Graph with p = 0.5 References [1] R. Aldana-Lopez, A. Macchelli, G. Notarstefano, R. Aragues, and C. Sagues. Towards par…

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. A Line-search-free Method for Adaptive Decentralized Optimization

    math.OC 2026-05 unverdicted novelty 7.0 of 10

    New adaptive decentralized algorithms select stepsizes from local curvature estimates derived from a Lyapunov function, delivering sublinear convergence for convex problems and linear rates for strongly convex ones.

Reference graph

Works this paper leans on

40 extracted references · 33 canonical work pages · cited by 1 Pith paper

  1. [17]

    Kuruzov, G

    I. Kuruzov, G. Scutari, and A. Gasnikov. Achieving linear convergence with parameter-free algorithms in decentralized optimization. InAdvances in Neural Information Processing Systems, 2024

    work page 2024

  2. [1]

    Aldana-Lopez, A

    R. Aldana-Lopez, A. Macchelli, G. Notarstefano, R. Aragues, and C. Sagues. Towards parameter-free distributed optimization: a port-hamiltonian approach.arXiv preprint arXiv:2404.13529, 2024

    arXiv 2024

  3. [2]

    Barzilai and J

    J. Barzilai and J. M. Borwein. Two-point step size gradient methods.IMA Journal of Numerical Analysis, 8(1):141–148, 1988

    1988

  4. [3]

    Chang and C

    C. Chang and C. Lin. Libsvm: A library for support vector machines.ACM Transactions on Intelligent Systems and Technology, 2, 07 2007

    work page 2007

  5. [4]

    Chang, M

    T. Chang, M. Hong, H. Wai, X. Zhang, and S. Lu. Distributed learning in the nonconvex world: From batch data to streaming and beyond.IEEE Signal Processing Magazine, 37(3):26–38, 2020

    work page 2020

  6. [5]

    X. Chen, B. Karimi, W. Zhao, and P. Li. On the convergence of decentralized adaptive gradient methods. In Asian Conference on Machine Learning, pages 217–232. PMLR, 2023

    work page 2023

  7. [6]

    X. Chen, X. Li, and P. Li. Toward communication efficient adaptive gradient method. InProceedings of the 2020 ACM-IMS on Foundations of Data Science Conference, page 119–128, Virtual Event USA, October 2020. ACM

    work page 2020

  8. [7]

    Distributed optimization and learning with automated stepsizes

    Ziqin Chen and Yongqiang Wang. Distributed optimization and learning with automated stepsizes. In2024 IEEE 63rd Conference on Decision and Control (CDC), pages 3121–3126, 2024

    work page 2024

Show all 40 references
  1. [8]

    Cutkosky and H

    A. Cutkosky and H. Mehta. Momentum improves normalized SGD. In Hal Daumé III and Aarti Singh, editors, Proceedings of the 37th International Conference on Machine Learning, volume 119 ofProceedings of Machine Learning Research, pages 2260–2268. PMLR, 13–18 Jul 2020

    2020

  2. [9]

    Duchi, E

    J. Duchi, E. Hazan, and Y. Singer. Adaptive subgradient methods for online learning and stochastic optimization. In Proceedings of the 24th International Conference on Neural Information Processing Systems, pages 257–265, 2011. 13

    2011

  3. [10]

    and Chinwendu E

    Iyanuoluwa E. and Chinwendu E. Q-linear convergence of distributed optimization with barzilai-borwein step sizes. In 58th Annual Allerton Conference on Communication, Control, and Computing (Allerton), pages 1–8, 2022

    2022

  4. [11]

    J. Gao, XW. Liu, YH. Dai, HUang Y., and P. Yang. Achieving geometric convergence for distributed optimization with barzilai-borwein step sizes.Sci. China Inf. Sci., 65:149–204, 2022

    2022

  5. [12]

    Ghaderyan and S

    D. Ghaderyan and S. Werner. Fully adaptive stepsizes: Which system benefit more – centralized or decentralized?

  6. [13]

    J. Hu, X. Chen, L. Zheng, L. Zhang, and H. Li. (rectified version) the barzilai–borwein method for distributed optimization over unbalanced directed networks.arXiv:2305.11469v3, 2024

    2024 arXiv

  7. [14]

    Janssen, N

    T. Janssen, N. BniLam, M. Aernouts, R. Berkvens, and M. Weyn. Lora 2.4 ghz communication link and range. Sensors, 20(16):4366, August 2020

    2020

  8. [15]

    Kim, J.Y

    D.H. Kim, J.Y. Lim, and J.D. Kim. Low-power, long-range, high-data transmission using wi-fi and lora. In2016 6th International Conference on IT Convergence and Security (ICITCS), page 1–3, Prague, Czech Republic, September 2016. IEEE

    2016

  9. [16]

    D. P. Kingma and J. Ba. Adam: A method for stochastic optimization.CoRR, abs/1412.6980, 2014

    2014 arXiv

  10. [18]

    Latafat, A

    P. Latafat, A. Themelis, and P. Patrinos. Adaptive proximal algorithms for convex optimization under local lipschitz continuity of the gradient.arXiv preprint arXiv:2301.04431, 2023

    2023 arXiv

  11. [19]

    J. Li, X. Chen, S. Ma, and M. Hong. Problem-parameter-free decentralized nonconvex stochastic optimization. arXiv preprint arXiv:2402.08821, 2024

    2024 arXiv

  12. [20]

    X. Li, B. Karimi, and P. Li. On distributed adaptive optimization with gradient compression. InInternational Conference on Learning Representations (ICLR), 2022

    2022

  13. [21]

    Li and F

    X. Li and F. Orabona. On the convergence of stochastic gradient descent with adaptive stepsizes. InProceedings of the 22nd International Conference on Artificial Intelligence and Statistics (AISTAT). PMLR, 2019

    2019

  14. [22]

    Z. Li, W. Shi, and M. Yan. A decentralized proximal-gradient method with network independent step-sizes and separated convergence rates.IEEE Transactions on Signal Processing, 67(17):4494–4506, 2019

    2019

  15. [23]

    Di Lorenzo and G

    P. Di Lorenzo and G. Scutari. NEXT: In-network nonconvex optimization.IEEE Transactions on Signal and Information Processing over Networks, 2(2):120–136, 2016

    2016

  16. [24]

    Malitsky and K

    Y. Malitsky and K. Mishchenko. Adaptive gradient descent without descent. InInternational Conference on Machine Learning, 2019

    2019

  17. [25]

    Malitsky and K

    Y. Malitsky and K. Mishchenko. Adaptive proximal gradient method for convex optimization.arXiv preprint arXiv:2308.02261, 2024

    2024 arXiv

  18. [26]

    Nazari, D.A

    P. Nazari, D.A. Tarzanagh, and G. Michailidis. Dadam: A consensus-based distributed adaptive gradient method for online optimization.IEEE Transactions on Signal Processing, 70:6065–6079, 2022

    2022

  19. [27]

    A. Nedic. Distributed gradient methods for convex machine learning problems in networks: Distributed optimization. IEEE Signal Processing Magazine, 37(3):92–101, 2020. 14

    2020

  20. [28]

    Nedić, A

    A. Nedić, A. Olshevsky, and M. Rabbat. Network topology and communication-computation tradeoffs in decentralized optimization. Proceedings of the IEEE, 106:953–976, 2018

    2018

  21. [29]

    Nedić, A

    A. Nedić, A. Olshevsky, and W. Shi. Achieving geometric convergence for distributed optimization over time-varying graphs.SIAM Journal on Optimization, 27:2597–2633, July 2016

    2016

  22. [30]

    Nocedal and S

    J. Nocedal and S. Wright.Numerical Optimization. Springer, 2 edition, 2006

    2006

  23. [31]

    B.T. Polyak. Minimization of unsmooth functionals.USSR Computational Mathematics and Mathematical Physics, 9(3):14–29, 1969

    1969

  24. [32]

    Qu and N

    G. Qu and N. Li. Harnessing smoothness to accelerate distributed optimization.IEEE Transactions on Control of Network Systems, 5(3):1245–1260, Sept 2018

    2018

  25. [33]

    Reddi, Z

    S. Reddi, Z. Burr Charles, M. Zaheer, Z. Garrett, K. Rush, J. Konevcn, S. Kumar, and B. McMahan. Adaptive federated optimization. InInternational Conference on Learning Representations (ICLR), 2021

    2021

  26. [34]

    S. J. Reddi, S. Kale, and S. Kumar. On the convergence of adam and beyond. InInternational Conference on Learning Representations (ICLR), 2018

    2018

  27. [35]

    A. H. Sayed. Adaptation, learning, and optimization over networks.Foundations and Trends in Machine Learning, 7:311–801, January 2014

    2014

  28. [36]

    W. Shi, Q. Ling, G. Wu, and W. Yin. EXTRA: An exact first-order algorithm for decentralized consensus optimization. SIAM J. on Optimization, 25(2):944–966, November 2015

    2015

  29. [37]

    Y. Sun, G. Scutari, and A. Daneshmand. Distributed optimization based on gradient-tracking revisited: Enhancing convergence rate via surrogation.SIAM J. on Optimization, 32:354–385, 2022

    2022

  30. [38]

    R. Ward, X. Wu, and L. Bottou. Adagrad stepsizes: Sharp convergence over nonconvex landscapes.The Journal of Machine Learning Research, 21:1–30, 2020

    2020

  31. [39]

    R. Xin, S. Pu, A. Nedic, and U. A. Khan. A general framework for decentralized optimization with first-order methods. Proceedings of the IEEE, 108(11):1869–1889, November 2020

    2020

  32. [40]

    D. Zhou, S. Ma, and J. Yang. Adabb: Adaptive barzilai-borwein method for convex optimization.arXiv preprint arXiv:2401.08024, 2024. 15

    2024 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.