Pith. sign in

REVIEW 4 minor 62 references

RED-SEGA:Resilient Decentralized Stochastic Proximal Optimization with Gradient Sketching over Time-Varying Networks

T0 review · 0 major / 4 minor · reviewed 2026-07-14 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Gradient sketching gives Byzantine-resilient multi-agent optimization a linear rate even when networks keep changing and full gradients are unavailable.

desk verdict Solid algorithmic paper that cleanly combines SEGA-style sketching with norm-penalized resilient consensus and gives the first linear-rate analysis of that combination on time-varying graphs for non-decomposable objectives. read the letter →

arxiv 2607.10791 v1 pith:CW6OSUY2 submitted 2026-07-12 math.OC

Jinhui Hu , Guo Chen , Huaqing Li , Liang Ran , Hexi Liang , Tao Han , Tingwen Huang This is my paper
classification math.OC MSC 90C1568W1593A14
keywords decentralizedstochasticoptimizationgradientsketchingByzantineresiliencevariancereductiontime-varyingnetworksproximalstructuralriskminimization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Most Byzantine-resilient decentralized optimizers that use variance reduction do so by randomly sampling data points. That works when data are plentiful, but fails when each agent has little data yet the decision vector is high-dimensional, and it requires the local cost to be a sum of terms. This paper shows that variance reduction can instead be obtained by sketching only a few coordinates (or directional derivatives) of each local gradient. The resulting algorithm, RED-SEGA, replaces ordinary averaging with a norm-penalized consensus step so that reliable agents still reach consensus and converge linearly to a neighborhood of the optimum, even when the communication graph is time-varying and some agents are arbitrary adversaries. The same construction also covers black-box settings in which true gradients cannot be evaluated. The result matters because it lets multi-agent systems keep learning under intermittent links and malicious nodes without needing huge local datasets or full back-propagation.

What carries the argument

The sketch-and-project estimator (extended from SEGA) that produces an unbiased stochastic gradient from only a random subset of partial derivatives, combined with a norm-penalized soft-consensus step that replaces ordinary weighted averaging.

What would settle it

Run RED-SEGA on a time-varying network that deliberately permanently disconnects the reliable agents into two or more components while Byzantine agents remain active; if the reliable agents still reach consensus and the measured residual decays linearly, the connectivity claim is false.

Watch

Extended reading notes

Core claim

Under standard strong-convexity and smoothness assumptions, a sufficiently large penalty parameter, and a mild average-connectivity condition on the time-varying graph, RED-SEGA drives all reliable agents to consensus and converges linearly, with constant step-size, to an error ball whose radius is independent of the Byzantine messages and shrinks with the step-size and the number of adversaries.

Load-bearing premise

Reliable agents must keep exchanging information often enough that their long-run average communication graph stays connected; permanent partitions among them break both consensus and the linear rate.

Editorial extensions

If this is right

  • Data-scarce high-dimensional tasks such as multi-agent image deblurring become solvable under Byzantine attacks without full-batch gradients.
  • Zeroth-order (derivative-free) multi-agent learning inherits the same linear-rate Byzantine resilience by using finite-difference sketches.
  • The per-iteration cost scales only with the sketch size times local data size plus neighborhood size, not with full dimension times data size.
  • Decaying step-sizes drive the residual exactly to zero when no Byzantine agents are present, recovering classical rates as a special case.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same sketch-and-penalize pattern should transfer to non-convex multi-agent training once a suitable Lyapunov function replaces strong convexity.
  • Because the method never needs the identity of reliable neighbors, it can be dropped into existing gossip libraries with only a change of the aggregation rule.
  • Communication-delay robustness already observed empirically may be made quantitative by treating delayed packets as additional bounded adversarial inputs inside the same penalty.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

0 major / 4 minor

Summary. The paper formulates a class of structural risk minimization problems over multi-agent systems that may contain Byzantine agents, where local objectives need not be decomposable and true gradients need not be available. It first constructs Gossip-SEGA by embedding a sketch-and-project gradient estimator into decentralized proximal gradient descent with gossip averaging, then obtains the resilient variant RED-SEGA by replacing the weighted average with a norm-penalized soft-consensus term. Under strong convexity/smoothness in weighted norms, mutual independence of local sketching distributions, and sufficient connectivity of the average network among reliable agents, Theorem 1 supplies an explicit lower bound on the penalty parameter that restores exact consensus, while Theorem 2 establishes linear convergence of a Lyapunov function to an error ball whose radius is independent of the Byzantine states. Numerical experiments on constrained least-squares and image deblurring under several attack models illustrate the claims.

Significance. If the claims hold, the work supplies a genuine alternative technical line for Byzantine-resilient decentralized variance reduction that is especially suited to data-scarce, high-dimensional regimes (image deblurring, black-box fine-tuning). The combination of gradient sketching (both first-order multi-coordinate and zeroth-order) with a norm-penalized aggregation that works over time-varying networks is new relative to existing data-sampling VR methods (Byrd-SAGA, Prox-DBRO-VR) and to static-network resilient algorithms. The explicit intermediate lemmas on gradient-estimation and gradient-learning error, the closed-form step-size bounds, and the vanishing of the residual when no Byzantine agents are present are concrete, checkable contributions that practitioners can use.

minor comments (4)
  1. The lower bound φ_min in Theorem 1 depends on the unknown optimum; while the paper correctly treats it as a sufficient condition and later recommends empirical tuning (Remark 6), a short practical discussion of how a user might choose φ without knowledge of x* would strengthen usability.
  2. In Section V the comparison with Prox-DBRO-VR is performed under equal sample size and dimension; a brief remark clarifying that the theoretical guarantees of the baseline hold only for static networks would make the experimental fairness claim more precise.
  3. Notation for the time-varying weight matrix A_k and the sketched matrices S_i,k is dense; a short table or a consistent bold/italic convention would improve readability of Algorithms 1–2 and the subsequent lemmas.
  4. A few typographical slips remain (e.g., “Distraction” label under (17), occasional missing spaces after periods). A careful proof-reading pass is recommended.

Circularity Check

1 steps flagged · score 2.0 of 10

Minor self-citation for static-network consensus baseline; core sketching, time-varying extension, and linear-rate claims are independently derived from Assumptions 1–4 and Lemmas 1–2.

  1. self citation load bearing [Appendix VII-A (Proof of Theorem 1)]
    "Building upon the resilient consensus framework for connected static networks in [31, Appendix D], the following proof details the theoretical adaptations required for intermittently connected time-varying networks. ... The rest of the proof is analogous to [31, Theorem 1], adapted by replacing static network parameters with their average network counterparts."

    The core optimality condition and incidence-matrix argument that establish equivalence of DRC (6) to SRM (1) under a sufficiently large ϕ are imported by direct analogy from the authors’ own prior static-network result rather than re-derived from first principles; only the ergodic average-network substitution is new. This makes the consensus half of the central claim partially dependent on an unverified (within this paper) self-citation, though the subsequent linear-rate analysis does not inherit the dependency.

full rationale

The derivation chain for RED-SEGA is self-contained: unbiasedness of the sketched estimator follows from the sketch-and-project identity (8)–(13) and Assumption 4; gradient estimation and learning errors are bounded in Lemmas 1–2 by direct expansion using the pseudo-inverse property (9) and the unbiasedness condition (11); consensus equivalence (Theorem 1) reduces the time-varying case to an average-network incidence matrix via ergodicity (Assumption 3); and linear convergence (Theorem 2) follows from a standard Lyapunov argument on U_k^W that decouples proximal non-expansiveness, the norm-penalty residual, and the sketching variance terms under an explicit step-size restriction. No quantity is defined in terms of the claimed rate, no parameters are fitted to data and then re-predicted, and no uniqueness theorem is imported to forbid alternatives. The sole minor circularity is partial reliance on the authors’ prior Prox-DBRO-VR framework for the static-network optimality condition that is then adapted; this is not load-bearing for the new claims (time-varying networks, gradient sketching, non-decomposable objectives). Numerical experiments provide independent empirical support. Score 2 reflects only that isolated self-citation.

Assumptions & free parameters 3 free parameters · 5 assumptions · 2 invented entities

The central claims rest on four domain assumptions (strong convexity, general smoothness, average-network connectivity, mutual independence of sketching distributions), standard proximal-operator calculus, and two free algorithmic parameters (step-size α and penalty ϕ) that must be chosen inside theoretically derived ranges. No new physical entities are postulated; the only invented objects are the algorithmic constructs themselves.

free parameters (3)
  • penalty parameter ϕ
    Must satisfy ϕ ≥ ϕ_min (which depends on the unknown optimum gradient) yet cannot be chosen arbitrarily large because it inflates the residual error ball; treated as a tunable hyper-parameter in experiments.
  • constant step-size α
    Upper-bounded by an expression involving the sketching matrices, strong-convexity and smoothness constants; chosen by hand inside that range for the numerical plots.
  • sketching batch size b
    Controls the variance–computation trade-off; set to n, n/2 or 1 in the experiments without a data-driven selection rule.
assumptions (5)
  • domain assumption Local objectives are strongly convex w.r.t. a positive-definite weight matrix W_i (Assumption 1).
    Guarantees uniqueness of the global optimum and supplies the linear-rate constant γ = μ/2.
  • domain assumption Local objectives satisfy a general smoothness inequality w.r.t. a positive-definite matrix Q_i (Assumption 2).
    Used to bound the gradient-learning error and to obtain the admissible step-size range.
  • domain assumption The time-averaged communication graph among reliable agents is bidirectionally connected (Assumption 3).
    Necessary for the resilient-consensus equivalence of Theorem 1; weaker than static connectivity but still required.
  • domain assumption Sketching distributions of distinct reliable agents are mutually independent (Assumption 4).
    Allows the expectation of the sketched gradient estimator to factor across agents.
  • standard math The proximal operator of a closed proper convex function is non-expansive in the W-norm.
    Standard fact used to pass from the intermediate SGD step to the next iterate in the Lyapunov analysis.
invented entities (2)
  • Gossip-SEGA / RED-SEGA algorithms
    purpose: Realize decentralized variance-reduced proximal gradient descent with (resp. without) Byzantine resilience via gradient sketching.
    Algorithmic constructs; their correctness is the claim being proved, not an independent physical entity.
  • Norm-penalized soft consensus (DRC formulation) independent evidence
    purpose: Replace hard equality constraints by an ℓ_a-norm penalty so that Byzantine outliers can be tolerated.
    A modeling device already present in earlier resilient-optimization literature; re-used here with a new variance-reduction engine.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RED-SEGA:Resilient Decentralized Stochastic Proximal Optimization with Gradient Sketching over Time-Varying Networks." pith.science (2026). https://pith.science/paper/CW6OSUY2

@misc{pith2026260710791,
  author       = {Pith},
  title        = {Pith review of: RED-SEGA:Resilient Decentralized Stochastic Proximal Optimization with Gradient Sketching over Time-Varying Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CW6OSUY2}},
  note         = {Machine review of arXiv:2607.10791}
}
read the original abstract

Variance reduction is indispensable in Byzantine-resilient decentralized stochastic optimization over multi-agent systems (MASs) for its ability to mitigate gradient noise and thereby enhance the resilient aggregation process. However, most existing Byzantine-resilient decentralized variance-reduced (VR) stochastic gradient algorithms rely on random data sampling, which proves inefficient in data-scarce yet high-dimensional tasks, for instance, image deblurring. This paper pursues an alternative technical line that achieves variance reduction via gradient sketching. To this end, we first formulate a class of structural risk minimization (SRM) problems, where the local objectives are not necessarily decomposable and their gradients may be unavailable. To solve the SRM problems in a decentralized manner, we integrate a gradient-sketching technique into decentralized stochastic proximal gradient descent with gossip communication to propose a decentralized VR stochastic gradient algorithm, dubbed Gossip-SEGA.Since Gossip-SEGA does not provide any resilience against Byzantine attacks, a resilient extension of Gossip-SEGA,namely RED-SEGA,is developed via replacing the weighted average in Gossip-SEGA by a norm-penalized approximation. Theoretically, we derive sufficient conditions for both consensus (among reliable agents) and linear convergence rate of RED-SEGA over time-varying networks. The effectiveness and resilience of the proposed algorithms are validated through numerical experiments.

Figures

Figures reproduced from arXiv: 2607.10791 by the authors.

Figure 1
Figure 1. Network evolution in a connectivity window of size [PITH_FULL_IMAGE:figures/full_fig_p010_1.png] view at source ↗
Figure 2
Figure 2. Performance comparison under dropout attacks [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗
Figure 3
Figure 3. Network evolution in a window size B = 5 for time-varying networks with 2 Byzantine agents. (a) Consensus error over iterations. (b) Residual over iterations. (c) Consensus error over oracle calls. (d) Residual over oracle calls [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Performance comparison under Gaussian attacks [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: Network evolution in a window size B = 5 for time-varying networks with 3 Byzantine agents. images from the BSDS300 dataset [61], which are resized into 64×64×3 RGB images. We use four metrics: peak signal-to￾noise ratio (PSNR), structural similarity index (SSMI), loss…
Figure 6
Figure 6. Figure 6: Performance comparison under A-Little-Is-Enough attacks [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]
Figure 7
Figure 7. Figure 7: Ablation study of the penalty parameter ϕ for RED-SEGA under sign-flipping attacks over network topologies [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]
Figure 8
Figure 8. Figure 8: A clean image from Set5 dataset [59] and its blurred versions: [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]
Figure 9
Figure 9. Figure 9: Deblurring performance of ZO and FO versions [PITH_FULL_IMAGE:figures/full_fig_p011_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

62 extracted references · 5 linked inside Pith

  1. [1]

    Robust constrained consensus and inequality- constrained distributed optimization with guaranteed differential privacy and accurate convergence,

    Y . Wang and A. Nedi ´c, “Robust constrained consensus and inequality- constrained distributed optimization with guaranteed differential privacy and accurate convergence,”IEEE Transactions on Automatic Control, vol. 69, no. 11, pp. 7463–7478, 2024

    2024

  2. [2]

    Can decentralized algorithms outperform centralized algorithms? A case study for decentralized parallel stochastic gradient descent,

    X. Lian, C. Zhang, H. Zhang, C. J. Hsieh, W. Zhang, and J. Liu, “Can decentralized algorithms outperform centralized algorithms? A case study for decentralized parallel stochastic gradient descent,” in Advances in Neural Information Processing Systems (NeurIPS), 2017, pp. 5331–5341

    2017

  3. [3]

    Byzantine-robust decentralized coordination of LLM agents,

    Y . Jo and C. Park, “Byzantine-robust decentralized coordination of LLM agents,”arXiv:2507.14928, 2025

    arXiv 2025

  4. [4]

    PMGT-VR: A decentralized proximal- gradient algorithmic framework with variance reduction,

    H. Ye, W. Xiong, and T. Zhang, “PMGT-VR: A decentralized proximal- gradient algorithmic framework with variance reduction,”IEEE Trans- actions on Pattern Analysis and Machine Intelligence, vol. 48, no. 1, pp. 408–420, 2025

    2025

  5. [5]

    CoCoL: A communication efficient decentralized collaborative learning method for multi-robot systems,

    J. Huang, Y . Huang, Y . Zhao, W. Meng, and J. Xu, “CoCoL: A communication efficient decentralized collaborative learning method for multi-robot systems,”IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2025

    2025

  6. [6]

    Convergence analysis of distributed generalized Nash equilibria seeking algorithm with asynchrony and delays,

    H. Li, L. Ran, L. Zheng, Z. Li, J. Hu, and J. Li, “Convergence analysis of distributed generalized Nash equilibria seeking algorithm with asynchrony and delays,”IEEE Transactions on Automatic Control, vol. 70, no. 1, pp. 642–648, 2025

    2025

  7. [7]

    Dynamic regret of quantized distributed online bandit optimization in zero-sum games,

    L. Liao, D. W. Ho, D. Yuan, Z. Yu, B. Zhang, and S. Xu, “Dynamic regret of quantized distributed online bandit optimization in zero-sum games,”IEEE Transactions on Cybernetics, vol. 56, no. 1, pp. 192–199, 2026

    2026

  8. [8]

    Decen- tralized dual proximal gradient algorithms for non-smooth constrained composite optimization problems,

    H. Li, J. Hu, L. Ran, Z. Wang, Q. L ¨u, Z. Du, and T. Huang, “Decen- tralized dual proximal gradient algorithms for non-smooth constrained composite optimization problems,”IEEE Transactions on Parallel and Distributed Systems, vol. 32, no. 10, pp. 2594–2605, 2021

    2021

Show all 62 references
  1. [9]

    Decentralized constrained optimization over time-varying directed net- works via subgradient rescaling,

    Q. L ¨u, H. Li, C. Wu, H. Zhou, T. Huang, and P. Nagaratnam Suganthan, “Decentralized constrained optimization over time-varying directed net- works via subgradient rescaling,”IEEE Transactions on Cybernetics, vol. 56, no. 7, pp. 3725–3738, 2026

    2026

  2. [10]

    Balance of communication and convergence: predefined-time distributed optimization based on zero- gradient-sum,

    R. Zhang, G. Guo, and Z. D. Zhou, “Balance of communication and convergence: predefined-time distributed optimization based on zero- gradient-sum,”IEEE Transactions on Cybernetics, vol. 55, no. 2, pp. 661–671, 2025

    2025

  3. [11]

    Distributed optimization with gradient tracking over heterogeneous delay-prone directed networks,

    E. Makridis, G. Oliva, K. R. Narahari, M. Doostmohammadian, U. A. Khan, and T. Charalambous, “Distributed optimization with gradient tracking over heterogeneous delay-prone directed networks,” inEuro- pean Control Conference (ECC), 2024, pp. 2312–2319

    2024

  4. [12]

    Delay-tolerant augmented-consensus-based distributed directed optimization,

    M. Doostmohammadian, N. K. Ramesh, and A. Aghasi, “Delay-tolerant augmented-consensus-based distributed directed optimization,”Systems and Control Letters, vol. 205, no. 106260, 2025

    2025

  5. [13]

    A Byzantine- resilient distributed peer-to-peer energy management approach,

    X. Chang, Y . Xu, Q. Guo, H. Sun, and W. K. Chan, “A Byzantine- resilient distributed peer-to-peer energy management approach,”IEEE Transactions on Smart Grid, vol. 14, no. 1, pp. 623–634, 2023

    2023

  6. [14]

    Decentralized nonconvex robust optimization over unsafe multiagent systems: System modeling, utility, resilience, and privacy analysis,

    J. Hu, G. Chen, H. Li, H. Cheng, X. Guo, and T. Huang, “Decentralized nonconvex robust optimization over unsafe multiagent systems: System modeling, utility, resilience, and privacy analysis,”IEEE Transactions on Cybernetics, vol. 55, no. 8, pp. 3799–3810, 2025

    2025

  7. [15]

    Mean aggregator is more robust than robust aggregators under label poisoning attacks on distributed heterogeneous data,

    J. Peng, W. Li, S. Vlaski, and Q. Ling, “Mean aggregator is more robust than robust aggregators under label poisoning attacks on distributed heterogeneous data,”Journal of Machine Learning Research, vol. 26, no. 27, pp. 1–51, 2025

    2025

  8. [16]

    Learning from history for Byzantine robust optimization,

    S. P. Karimireddy, L. He, and M. Jaggi, “Learning from history for Byzantine robust optimization,” inInternational Conference on Machine Learning (ICML), 2021, pp. 5311–5319

    2021

  9. [17]

    Byzantine-robust decentralized learning via self-centered clipping,

    L. He, S. P. Karimireddy, and M. Jaggi, “Byzantine-robust decentralized learning via self-centered clipping,”arXiv:2202.01545, 2022

    2022 arXiv

  10. [18]

    Distributed statistical machine learning in adversarial settings: Byzantine gradient descent,

    Y . Chen, “Distributed statistical machine learning in adversarial settings: Byzantine gradient descent,” inProceedings of the ACM on Measure- ment and Analysis of Computing Systems, vol. 1, no. 2, 2017, pp. 1–25

    2017

  11. [19]

    Byzantine-robust distributed learning: Towards optimal statistical rates,

    D. Yin, Y . Chen, K. Ramchandran, and P. Bartlett, “Byzantine-robust distributed learning: Towards optimal statistical rates,” inInternational Conference on Machine Learning (ICML), 2018, pp. 5650–5659

    2018

  12. [20]

    Phocas: dimensional Byzantine- resilient stochastic gradient descent,

    C. Xie, O. Koyejo, and I. Gupta, “Phocas: dimensional Byzantine- resilient stochastic gradient descent,”arXiv:1805.09682, 2018

    2018 arXiv

  13. [21]

    Machine learning with adversaries: Byzantine tolerant gradient descent,

    P. Blanchard, E. M. El Mhamdi, R. Guerraoui, and J. Stainer, “Machine learning with adversaries: Byzantine tolerant gradient descent,” inAd- vances in Neural Information Processing Systems (NeurIPS), 2017, pp. 118–128

    2017

  14. [22]

    FABA: An algorithm for fast aggregation against Byzantine attacks in distributed neural networks,

    Q. Xia, Z. Tao, Z. Hao, and Q. Li, “FABA: An algorithm for fast aggregation against Byzantine attacks in distributed neural networks,” in International Joint Conference on Artificial Intelligence (IJCAI), 2019, pp. 4824–4830

    2019

  15. [23]

    RSA: Byzantine- robust stochastic aggregation methods for distributed learning from heterogeneous datasets,

    L. Li, W. Xu, T. Chen, G. B. Giannakis, and Q. Ling, “RSA: Byzantine- robust stochastic aggregation methods for distributed learning from heterogeneous datasets,” inAAAI Conference on Artificial Intelligence (AAAI), 2019, pp. 1544–1551

    2019

  16. [24]

    Byzantine-robust decentralized stochastic optimization over static and time-varying networks,

    J. Peng, W. Li, and Q. Ling, “Byzantine-robust decentralized stochastic optimization over static and time-varying networks,”Signal Processing, vol. 183, p. 108020, 2021

    2021

  17. [25]

    Byzantine-resilient decentralized stochas- tic optimization with robust aggregation rules,

    Z. Wu, T. Chen, and Q. Ling, “Byzantine-resilient decentralized stochas- tic optimization with robust aggregation rules,”IEEE Transactions on Signal Processing, vol. 71, pp. 3179–3195, 2023

    2023

  18. [26]

    Byzantine-robust distributed stochastic non-convex optimization in adversarial environments over unbalanced networks,

    D. Han, K. Liu, Y . Xia, and L. Xie, “Byzantine-robust distributed stochastic non-convex optimization in adversarial environments over unbalanced networks,”IEEE Transactions on Automatic Control, vol. 70, no. 12, pp. 8029–8043, 2025

    2025

  19. [27]

    Order- optimal Byzantine-robust learning under heterogeneity via fair gradient clipping,

    Z.-y. Wang, H. N. Sheng, S. Member, Q. Yang, and H. C. So, “Order- optimal Byzantine-robust learning under heterogeneity via fair gradient clipping,”IEEE Transactions on Cybernetics, vol. 56, no. 3, pp. 1503– 1514, 2026

    2026

  20. [28]

    Byzantine-robust learning on heterogeneous data via gradient splitting,

    Y . Liu, C. Chen, L. Lyu, F. Wu, S. Wu, and G. Chen, “Byzantine-robust learning on heterogeneous data via gradient splitting,” inInternational Conference on Machine Learning (ICML), 2023, pp. 21 404–21 425

    2023

  21. [29]

    Federated variance- reduced stochastic gradient descent with robustness to Byzantine at- tacks,

    Z. Wu, Q. Ling, T. Chen, and G. B. Giannakis, “Federated variance- reduced stochastic gradient descent with robustness to Byzantine at- tacks,”IEEE Transactions on Signal Processing, vol. 68, pp. 4583–4596, 2020

    2020

  22. [30]

    Variance reduction is an antidote to Byzantine workers: Better rates, weaker assumptions and communication compression as a cherry on the Top,

    E. Gorbunov, S. Horv ´ath, P. Richt´arik, and G. Gidel, “Variance reduction is an antidote to Byzantine workers: Better rates, weaker assumptions and communication compression as a cherry on the Top,” inInterna- tional Conference on Learning Representations (ICLR), 2023

    2023

  23. [31]

    Prox-DBRO-VR: A unified analysis on Byzantine-resilient decentralized stochastic compos- ite optimization with variance reduction and nonasymptotic convergence rates,

    J. Hu, G. Chen, H. Li, L. Ran, and T. Huang, “Prox-DBRO-VR: A unified analysis on Byzantine-resilient decentralized stochastic compos- ite optimization with variance reduction and nonasymptotic convergence rates,”IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol...

    2025

  24. [32]

    Robust distributed con- sensus using total variation,

    W. Ben-ameur, P. Bianchi, and J. Jakubowicz, “Robust distributed con- sensus using total variation,”IEEE Transactions on Automatic Control, vol. 61, no. 6, pp. 1550–1564, 2016

    2016

  25. [33]

    Differentially private distributed optimization over diagraphs with application to image de- blurring,

    Z. Yang, W. He, S. Member, and Y . Yuan, “Differentially private distributed optimization over diagraphs with application to image de- blurring,”IEEE Transactions on Control of Network Systems, vol. 12, no. 1, pp. 634–647, 2025

    2025

  26. [34]

    Asynchronous distributed model predictive control for optimal output consensus of high-order multi-agent systems,

    H. Li, L. Zheng, Z. Wang, Y . Li, and L. Ji, “Asynchronous distributed model predictive control for optimal output consensus of high-order multi-agent systems,”IEEE Transactions on Signal and Information Processing over Networks, vol. 7, pp. 689–698, 2021

    2021

  27. [35]

    Deepzero: Scaling up zeroth-order optimization for deep model training,

    A. Chen, Y . Zhang, J. Jia, J. Diffenderfer, J. Liu, K. Parasyris, Y . Zhang, Z. Zhang, B. Kailkhura, S. Liu, and L. Livermore, “Deepzero: Scaling up zeroth-order optimization for deep model training,”International Conference on Learning Representations (ICLR), pp. 50 185–50 206, 2024

    2024

  28. [36]

    Enhancing zeroth-order fine-tuning for language models with low-rank structures,

    Y . Chen, Y . Zhang, L. Cao, K. Yuan, and Z. Wen, “Enhancing zeroth-order fine-tuning for language models with low-rank structures,” arXiv:2410.07698, 2024

    2024 arXiv

  29. [37]

    SEGA: Variance reduction via gradient sketching,

    F. Hanzely, K. Mishchenko, and P. Richt´arik, “SEGA: Variance reduction via gradient sketching,” inAdvances in Neural Information Processing Systems (NeurIPS), 2018, pp. 2082–2093

    2018

  30. [38]

    A unified theory of SGD: Variance reduction, sampling, quantization and coordinate descent,

    E. Gorbunov, F. Hanzely, and P. Richt ´arik, “A unified theory of SGD: Variance reduction, sampling, quantization and coordinate descent,” inInternational Conference on Artificial Intelligence and Statistics (AISTATS), 2020, pp. 680–690

    2020

  31. [39]

    ByRDiE: Byzantine-resilient distributed coordinate descent for decentralized learning,

    Z. Yang and W. U. Bajwa, “ByRDiE: Byzantine-resilient distributed coordinate descent for decentralized learning,”IEEE Transactions on Signal and Information Processing over Networks, vol. 5, no. 4, pp. 611–627, 2019

    2019

  32. [40]

    BRIDGE: Byzantine-resilient decentralized gradient descent,

    C. Fang, Z. Yang, and W. U. Bajwa, “BRIDGE: Byzantine-resilient decentralized gradient descent,”IEEE Transactions on Signal and In- formation Processing over Networks, vol. 8, pp. 610–626, 2022. 17

    2022

  33. [41]

    Fast decentralized nonconvex finite- sum optimization with recursive variance reduction,

    R. Xin, U. A. Khan, and S. Kar, “Fast decentralized nonconvex finite- sum optimization with recursive variance reduction,”SIAM Journal on Optimization, vol. 32, no. 1, pp. 1–28, 2022

    2022

  34. [42]

    On the tradeoff between privacy preser- vation and Byzantine-robustness in decentralized learning,

    H. Ye, H. Zhu, and Q. Ling, “On the tradeoff between privacy preser- vation and Byzantine-robustness in decentralized learning,” inIEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2024, pp. 9336–9340

    2024

  35. [43]

    Walters,An introduction to ergodic theory

    P. Walters,An introduction to ergodic theory. Springer Science & Business Media, 1982

    1982

  36. [44]

    Distributed optimization over time-varying directed graphs,

    A. Nedi ´c and A. Olshevsky, “Distributed optimization over time-varying directed graphs,”IEEE Transactions on Automatic Control, vol. 60, no. 3, pp. 601–615, 2015

    2015

  37. [45]

    Achieving geometric convergence for distributed optimization over time-varying graphs,

    N. Angelia, O. Alex, and W. Shi, “Achieving geometric convergence for distributed optimization over time-varying graphs,”SIAM Journal on Optimization, vol. 27, no. 4, pp. 2597–2633, 2017

    2017

  38. [46]

    Decentralized optimization over time-varying directed graphs with row and column-stochastic matrices,

    F. Saadatniaki, R. Xin, and U. A. Khan, “Decentralized optimization over time-varying directed graphs with row and column-stochastic matrices,”IEEE Transactions on Automatic Control, vol. 65, no. 11, pp. 4769–4780, 2020

    2020

  39. [47]

    Decentralized federated learning with gradient tracking over time-varying directed networks,

    D. Thuy Anh Nguyen, S. Wang, D. Tung Nguyen, A. Nedi ´c, and H. Vin- cent Poor, “Decentralized federated learning with gradient tracking over time-varying directed networks,”arXiv:2409.17189, 2023

    2023 arXiv

  40. [48]

    Accelerated distributed stochastic nonconvex optimization over time-varying directed networks,

    Y . Chen, A. Hashemi, and H. Vikalo, “Accelerated distributed stochastic nonconvex optimization over time-varying directed networks,”IEEE Transactions on Automatic Control, vol. 70, no. 4, pp. 2196–2211, 2025

    2025

  41. [49]

    Distributed Nash equi- librium seeking over time-varying directed communication networks,

    D. T. A. Nguyen, D. T. Nguyen, and A. Nedi ´c, “Distributed Nash equi- librium seeking over time-varying directed communication networks,” IEEE Transactions on Control of Network Systems, pp. 1–12, 2025

    2025

  42. [50]

    Decentralized Byzantine-resilient stochastic optimization over multi-agent systems with arbitrary commu- nication delays,

    J. Hu, X. Guo, W. Li, and S. Zong, “Decentralized Byzantine-resilient stochastic optimization over multi-agent systems with arbitrary commu- nication delays,” in6th International Conference on Electronic Engi- neering and Informatics (EEI), 2024, pp. 958–964

    2024

  43. [51]

    Distributed optimization under adver- sarial nodes,

    S. Sundaram and B. Gharesifard, “Distributed optimization under adver- sarial nodes,”IEEE Transactions on Automatic Control, vol. 64, no. 3, pp. 1063–1076, 2019

    2019

  44. [52]

    SAGA: A fast incremental gradient method with support for non-strongly convex composite objec- tives,

    A. Defazio, F. Bach, and S. Lacoste-Julien, “SAGA: A fast incremental gradient method with support for non-strongly convex composite objec- tives,” inAdvances in Neural Information Processing Systems (NeurIPS), 2014, pp. 1646–1654

    2014

  45. [53]

    Don’t jump through hoops and remove those loops: SVRG and Katyusha are better without the outer loop,

    D. Kovalev, S. Horvath, and P. Richtarik, “Don’t jump through hoops and remove those loops: SVRG and Katyusha are better without the outer loop,” inAlgorithmic Learning Theory, 2020, pp. 451–467

    2020

  46. [54]

    Distributed coordinate descent method for learning with big data,

    P. Richt ´arik and M. Tak ´aˇc, “Distributed coordinate descent method for learning with big data,”Journal of Machine Learning Research, vol. 17, no. 75, pp. 1–25, 2016

    2016

  47. [55]

    Gradient-free optimization of highly smooth functions: improved analysis and a new algorithm,

    A. Akhavan, E. Chzhen, M. Pontil, and A. B. Tsybakov, “Gradient-free optimization of highly smooth functions: improved analysis and a new algorithm,”Journal of Machine Learning Research, vol. 25, no. 370, pp. 1–50, 2024

    2024

  48. [56]

    Distributed zeroth-order optimization: Convergence rates that match centralized counterpart,

    D. Yuan, L. Wang, A. Proutiere, and G. Shi, “Distributed zeroth-order optimization: Convergence rates that match centralized counterpart,” Automatica, vol. 159, p. 111328, 2024

    2024

  49. [57]

    Stochastic reformulations of linear systems: algorithms and convergence theory,

    P. R. Arik, “Stochastic reformulations of linear systems: algorithms and convergence theory,”SIAM Journal on Matrix Analysis and Applica- tions, vol. 41, no. 2, pp. 487–524, 2020

    2020

  50. [58]

    A little is enough: Circumvent- ing defenses for distributed learning,

    M. Baruch, G. Baruch, and Y . Goldberg, “A little is enough: Circumvent- ing defenses for distributed learning,” inAdvances in Neural Information Processing Systems (NeurIPS), 2019, pp. 8635–8645

    2019

  51. [59]

    Low- complexity single-image super-resolution based on nonnegative neighbor embedding,

    M. Bevilacqua, A. Roumy, C. Guillemot, and M. L. A. Morel, “Low- complexity single-image super-resolution based on nonnegative neighbor embedding,” inProceedings of the 23rd British Machine Vision Confer- ence (BMVC), 2012, pp. 1–10

    2012

  52. [60]

    Simultaneous fidelity and regularization learning for image restoration,

    D. Ren, W. Zuo, D. Zhang, L. Zhang, and M.-h. Yang, “Simultaneous fidelity and regularization learning for image restoration,”IEEE Trans- actions on Pattern Analysis and Machine Intelligence, vol. 43, no. 1, pp. 284–299, 2019

    2019

  53. [61]

    A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics,

    D. Martin, C. Fowlkes, D. Tal, and J. Malik, “A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics,” inIEEE International Conference on Computer Vision (ICCV), 2001, pp. 416–423

    2001

  54. [62]

    Golub and Charles F

    Gene H. Golub and Charles F. Van Loan,Matrix computations. Johns Hopkins University Press, 2013

    2013

Pith tools

Reviewed July 14, 2026 · model on record in the stance chip above.