Pith. sign in

REVIEW 4 major objections 5 minor 41 references

Weight for Robustness: A Comprehensive Approach towards Optimal Fault-Tolerant Asynchronous ML

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper proves that Byzantine-robust asynchronous convex optimization can achieve an optimal convergence rate: excess loss shrinking with honest updates and no explicit dimension dependence, by weighting robust aggregators and adding…

desk verdict A plausible first rate for Byzantine-robust async optimization, but the delay definition is loose enough that the main theorem needs repair. read the letter →

arxiv 2501.09621 v2 pith:LMLVNQXE submitted 2025-01-16 cs.LG

Tehila Dahan , Kfir Y. Levy This is my paper · ORCID
classification cs.LG MSC 68W4090C2568T05
keywords asynchronousdistributedlearningByzantinerobustnessweightedrobustaggregationmeta-aggregationmomentumvariancereductionconvexoptimizationfaulttolerancedouble
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether asynchronous distributed training can keep its convergence guarantees when an unknown fraction of the workers are Byzantine, meaning they may send corrupted or malicious gradient information. Earlier asynchronous Byzantine methods could not guarantee that the excess loss decreases as more honest data arrives, and every earlier bound depended explicitly on the problem dimension. The authors claim to close both gaps: their algorithm, asynchronous robust $\mu^2$-SGD with weighted robust aggregation, attains for smooth convex losses an excess loss of order $O((G^*D + LD^2\mu_{\max}\sqrt{1+c_\lambda})/T + D\tilde\sigma\sqrt{1+mc_\lambda}/\sqrt{T})$, with $\tilde\sigma^2 = 2\sigma^2 + 32D^2K^2\sigma_L^2$ and $c_\lambda$ the weighted aggregator's error coefficient. With no Byzantine updates the rate matches Byzantine-free asynchronous SGD, so the method claims robustness at no asymptotic cost in this regime. The key move is to weight each worker by its update count, making the Byzantine fraction $\lambda$ a fraction of updates rather than of workers.

What carries the argument

The load-bearing object is the weighted robust aggregation pair: Definition 3.1's $(c_\lambda,\lambda)$-weighted robustness and the $\omega$-CTMA meta-aggregator (Algorithm 1) built on it. $\omega$-CTMA sorts all submitted vectors by distance from a weighted robust anchor, keeps the smallest total weight $(1-\lambda)$, and reweights the boundary point so the surviving weight is exactly $(1-\lambda)$; this turns a suboptimal aggregator into one with $c_\lambda=O(\lambda)$. The second mechanism is the double momentum of $\mu^2$-SGD: each worker maintains $d_t = g_t + (1-\beta_t)(d_{t-\tau_t}-\tilde g_{t-\tau_t})$ with $\beta_t=1/s_t$, and unrolling this recursion gives a martingale difference sequence whose collective variance is $\tilde\sigma^2/t_G$, where $\tilde\sigma^2 = 2\sigma^2 + 32D^2K^2\sigma_L^2$. The Bounded Delay condition $\tau_{\min}^{(i)}\le\tau_t^{(i)}\le K\tau_{\min}^{(i)}$ bounds the distance between consecutive query points of one worker by $O(KD/s_t)$, which confines delays to a variance term; the Anytime-GD update rule with weights $\alpha_t=t$ then converts the gradient-estimator error into the excess-loss bound.

What would settle it

Run Algorithm 2 on a smooth convex problem with known diameter $D$, smoothness $L$, and noise $\sigma$, using honest workers whose delays alternate between $\tau_{\min}^{(i)}$ and $K\tau_{\min}^{(i)}$ plus a small drift, while keeping $\lambda=0$. If the measured excess loss does not shrink like $O(1/T + \tilde\sigma/\sqrt{T})$ — in particular, if doubling $K$ fails to raise the $1/\sqrt{T}$ level by the factor predicted by $\tilde\sigma^2=2\sigma^2+32D^2K^2\sigma_L^2$, or if the loss stops decaying when the delay ratio exceeds $K$ — then the theorem's rate is not the mechanism actually governing convergence.

Watch

Extended reading notes

Core claim

The paper's central discovery is that the obstacle to Byzantine robustness in asynchronous optimization is the mismatch between equal treatment of workers and unequal arrival rates, not asynchrony itself. Its Definition 3.1 generalizes robust aggregation to weighted inputs: a rule is $(c_\lambda,\lambda)$-weighted robust if its output stays within $c_\lambda$ times the weighted honest deviations from the weighted honest mean, in expectation, whenever honest inputs carry at least a $1-\lambda$ share of total weight. The authors construct $\omega$-CTMA, a weighted centered trimmed meta-aggregator, which anchors on any weighted robust rule, trims away total weight $\lambda$ by distance from that anchor, and averages the remainder; Lemma 3.1 shows the result is $(60\lambda(1+c_\lambda),\lambda)$-robust, which is $O(\lambda)$ for $\lambda\le 1/3$. They then plug weighted aggregation into the $\mu^2$-SGD double-momentum estimator, showing that with momentum correction $\beta_t=1/s_t$ each honest worker's estimator error is bounded by $\tilde\sigma^2/s_t$ and the weighted collective error by $\tilde\sigma^2/t_G$. Theorem 4.2 assembles these pieces into the claimed optimal excess-loss rate for asynchronous Byzantine convex optimization.

Load-bearing premise

The whole proof rests on the Bounded Delay assumption that each worker's delays remain within a fixed multiplicative band $K$ of that worker's own minimum delay; if delays become more irregular than that, the bound on the distance between that worker's consecutive updates breaks and the optimal rate no longer follows.

Editorial extensions

If this is right

  • For any $(c_\lambda,\lambda)$-weighted robust aggregator, Algorithm 2 satisfies the excess-loss bound of Theorem 4.2; with $\omega$-CTMA, $c_\lambda=O(\lambda)$ for $\lambda\le 1/3$, so the bound scales with the Byzantine update fraction.
  • When no Byzantine updates occur, $c_\lambda$ can be driven to zero and the rate matches the optimal Byzantine-free asynchronous SGD rate, so Byzantine resilience does not change the asymptotic rate in this setting.
  • The bound has no explicit dependence on the dimension $d$ and shrinks with the number of honest updates, which earlier asynchronous Byzantine methods did not achieve.
  • The leading stochastic term scales with $\sqrt{1+mc_\lambda}$ in the number of workers and with $K$ through $\tilde\sigma$, so delay-regular scheduling and a smaller Byzantine fraction directly improve the rate.
  • The server must know the Byzantine update fraction $\lambda$ before trimming: $\omega$-CTMA removes total weight $\lambda$, and a wrong value invalidates the aggregator guarantee.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The weighted-robustness definition is not tied to asynchrony: the same machinery should apply to synchronous settings with heterogeneous batch sizes, weighting by batch size and trimming by weight rather than by worker count. This is a direct extension the authors only mention in passing.
  • The bound makes delay regularity, not raw delay size, the main robustness resource: $K$ enters the leading variance term while the mean delay appears only in the lower-order $1/T$ term. Scheduling that balances per-worker delays should therefore improve Byzantine robustness even if average delay is unchanged.
  • The proof relies on convexity through the Anytime-GD regret argument, so extending the rate to non-convex objectives would require new machinery for the bias term in Lemma 4.1; testing the stationarity-gap analogue is an open question.
  • A hidden practical requirement is an online estimate of $\lambda$: since $\omega$-CTMA trims exactly weight $\lambda$, an adversary that hides its update fraction, or a server that overestimates it, will degrade $c_\lambda$; estimating $\lambda$ from the update stream is a natural follow-up.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a weighted robust aggregation framework for Byzantine-robust asynchronous convex optimization. It defines (c_lambda, lambda)-weighted robust aggregators, derives weighted variants of geometric median and coordinate-wise median, constructs a weighted version of the centered trimmed meta-aggregator (omega-CTMA), and integrates these with the mu2-SGD double-momentum method. The main theoretical claim is a convergence bound for the resulting asynchronous Byzantine algorithm, Theorem 4.2, of the form O((G*D + L D^2 mu_max sqrt(1+c_lambda))/T + D sigma_tilde sqrt(1+m c_lambda)/sqrt(T)), where sigma_tilde^2 = 2 sigma^2 + 32 D^2 K^2 sigma_L^2. The paper asserts that this is the first optimal convergence rate in an asynchronous Byzantine environment, with no explicit dimension dependence and with a diminishing excess loss. The authors also report experiments on MNIST and CIFAR-10 comparing weighted vs. non-weighted aggregators, with and without omega-CTMA, and against standard optimizers.

Significance. If the theoretical claims are established, this would be a meaningful step for asynchronous Byzantine ML: prior asynchronous Byzantine methods in the convex setting do not achieve an excess loss that provably vanishes with the number of honest updates, and their bounds depend on the dimension d. The paper's construction of weighted robust aggregators is natural for asynchronous settings where workers contribute unequal numbers of updates, and the use of mu2-SGD to align per-worker momentum variance with the total honest update count is a genuinely useful idea. The appendices contain careful martingale arguments, a bias-variance decomposition, and an Anytime-GD regret analysis, and the paper explicitly compares its rate with the synchronous Byzantine rate of Dahan and Levy. The main reservations are that the 'optimal' claim is not supported by any lower bound for asynchronous Byzantine optimization, and that the proof of the core rate relies on an ambiguous treatment of delay that needs to be resolved before Theorem 4.2 can be accepted as stated.

major comments (4)
  1. [Section 2 (Notation), Eq. (5), Lemma B.3, Lemma 4.1] The proof of Theorem 4.2 depends on a notion of delay that is used inconsistently. In Lemma B.3, the bound ||x^{(i)}_t - x^{(j)}_t|| <= 4D(tau^{(i)}_t + tau^{(j)}_t)/t is asserted by applying Lemma B.1 to each stored momentum. For a worker j that last arrived at time t_j < t, the query point underlying its stored momentum is x_{t_j - tau^{(j)}_{t_j}}, so its true staleness at time t is (t - t_j) + tau^{(j)}_{t_j}. Eq. (5), as written in the asynchronous training description, bounds only the per-update delay tau^{(j)}_{t_j} of an arriving update, and does not bound t - t_j. Under this per-update reading, Lemma B.3 and the bias bounds B1_t and B2_t in Eqs. (14)-(15) do not follow for workers that have not just arrived. Under a total-age reading, Eq. (5) becomes a much stronger assumption that no honest worker's stored momentum is ever older than K tau_min, which is not stated in the paper and is not implied by bounded per-worker computation delays. Since Lemma 4.1 and hence Theorem 4.2 rely directly on these bounds, the proof needs either an explicit bounded-staleness assumption on all stored momentums or a derivation of such a bound from a clearly stated work-conserving model of the asynchronous system.
  2. [Abstract, Section 4, Theorem 4.2, Remarks 4.3-4.5] The paper repeatedly claims an 'optimal convergence rate' in an asynchronous Byzantine environment, but no lower bound for this setting is provided. Remark 4.3 shows that the rate matches known Byzantine-free asynchronous SGD when lambda = 0, and Remark 4.5 compares the upper bound with the synchronous Byzantine upper bound of Dahan and Levy. Neither comparison constitutes a lower bound for the Byzantine asynchronous problem when lambda > 0. The 'optimal' claim should either be supported by a matching lower bound or be softened to claims such as 'first dimension-independent diminishing excess loss' and 'matches the Byzantine-free optimal rate when lambda = 0.' This is a load-bearing aspect of the paper's stated contribution.
  3. [Proof of Theorem 4.1, Eq. (8)] The derivation of Eq. (8) introduces mu_t as the 'average delay' of the worker arriving at time t and uses the identities s_t = t/mu_t and s_t - 1 = (t - tau_t)/mu_{t-tau_t}. If mu_t is defined as t/s_t, these identities are tautological, but then the assumption tau_min <= mu_t <= K tau_min in Eq. (7) needs a proof that average inter-update times of a worker lie within the per-update delay bounds of Eq. (5). If instead mu_t is intended to be the current per-update delay, the identity s_t = t/mu_t is not generally valid. The proof should clarify which definition is used and justify the bounds on mu_t; otherwise the K-dependence in sigma_tilde^2 is not rigorously established.
  4. [Definition 3.1] Definition 3.1 states E||hat{x} - bar{x}_G|| <= c_lambda rho^2, which is dimensionally inconsistent: the left side is a norm expectation and the right side is a squared quantity. The subsequent proofs, including Lemma C.1 and the use of Definition 3.1 in Lemma 4.1, require E||hat{x} - bar{x}_G||^2 <= c_lambda rho^2. This appears to be a typographical error rather than a substantive flaw, but it should be corrected in the formal definition.
minor comments (5)
  1. [Section 2 (Notation) and Figure 1] The notation for tau^{(i)}_t and x^{(i)}_t is used inconsistently: in the Notation paragraph, x^{(i)}_t is defined as x_{t^{(i)}} (the point at the worker's last update), while in the proof of Theorem 4.1, x^{(i)}_t denotes x_{t - tau_t}, the query point underlying the arriving update. These two readings lead to different staleness quantities and should be disambiguated.
  2. [Theorem 4.2 statement] The theorem states E[f(x_T) - f(w*)] but w* is not defined; the optimum is called x* elsewhere in the paper. The symbol should be x* throughout.
  3. [Section 4.1, text after Algorithm 2] The display for d^{(i)}_t = d_{t-tau_t} = g_{t-tau_t} + ... reuses g_t and tilde{g}_t with subscripts that mix the time of computation and the time of arrival. This makes the indexing hard to follow; a consistent convention (e.g., using the time at which the worker computed the update) would improve readability.
  4. [Section 5 and Appendix D] The experimental figures report averaged test accuracy but do not show error bars or standard deviations across the three random seeds, and the number of runs (three) is small. The qualitative claims about weighted vs. non-weighted aggregators would be stronger with a small confidence analysis.
  5. [References] There is a typo in the reference list: 'L eon Bottou' should read 'L'eon Bottou'.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the asynchronous Byzantine rate is derived from explicit assumptions and external lemmas, not from fitted inputs or self-citation.

full rationale

The paper's central claim, Theorem 4.2, is obtained by a chain of explicit derivations: Lemma B.1 (cited from Aviv et al. 2021) bounds the distance between a worker's successive query points; Theorem 4.1 establishes the per-worker and collective variance of the momentum error using a martingale difference sequence property; Lemma 4.1 combines this with the weighted robust aggregation guarantee (Definition 3.1) and explicit bias bounds using smoothness and Lemma B.3; finally Theorem 4.2 applies the AnyTime-GD regret decomposition of Cutkosky 2019. None of these steps fits a parameter to a target rate, renames an existing result, or defines a key quantity in terms of the conclusion. The weighted aggregators' robustness constants c_lambda are proved in Appendix C rather than assumed, and the self-citations to Dahan and Levy 2024 and Levy 2023 are used only as algorithmic starting points (CTMA and mu2-SGD); the asynchronous weighted extension and its analysis are carried out here. The bounded-delay assumption is a stated modeling condition, not a circularly imposed consequence of the theorem. The skeptic's concern about the precise interpretation of tau_t is a correctness/soundness question, not a demonstration that the derivation is equivalent to its inputs by construction. Therefore no significant circularity is present.

Assumptions & free parameters 1 free parameters · 6 assumptions · 0 invented entities

The central claim rests on standard SCO assumptions (bounded domain, bounded gradient variance, L-smoothness), a strong per-worker delay regularity assumption Eq (5), the assumption that the fraction of Byzantine updates is bounded by lambda<1/2 and known, and sample-arrival independence. The analysis also imports the Anytime-GD regret bound (Cutkosky 2019) and standard martingale concentration. No new entities are postulated. The only hand-set input in the algorithm is the Byzantine fraction lambda used for trimming; the learning rate is bounded by theory and set to 0.01 in experiments.

free parameters (1)
  • Byzantine fraction threshold lambda = not fitted; set to 0.1/0.3/0.4 in experiments, assumed known in theory
    Required by omega-CTMA trimming and by Definition 3.1; in practice the server must know or estimate the Byzantine update fraction.
assumptions (6)
  • domain assumption Each worker's delay is bounded within a factor K of its own minimum delay (Eq (5))
    Used to bound the distance between consecutive query points of a worker (Eq (8)) and hence the momentum error variance in Thm 4.1; if delays are bursty this fails.
  • domain assumption Fraction of Byzantine updates lambda is strictly less than 1/2 and known to the server (Eq (6), Def 3.1)
    Required for the weighted robust aggregation guarantee; the aggregator trims with threshold (1-lambda).
  • domain assumption Sample-arrival independence: delays are independent of data samples
    Allows conditioning on delays and treating weights as fixed; standard in async analysis (Arjevani et al. 2020, Aviv et al. 2021).
  • domain assumption Bounded diameter, bounded variance, L-smoothness Eqs (1)-(3)
    Standard SCO assumptions; used throughout the regret and variance analysis.
  • standard math Anytime-GD regret bound (Theorem B.1, rephrased from Cutkosky 2019)
    Core regret decomposition used in Thm 4.2; accepted external result.
  • standard math Martingale difference property and variance summation (Lemma B.2 from Levy 2023)
    Used in Thm 4.1 to bound sums of martingale differences; standard probability result.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Weight for Robustness: A Comprehensive Approach towards Optimal Fault-Tolerant Asynchronous ML." pith.science (2026). https://pith.science/paper/LMLVNQXE

@misc{pith2026250109621,
  author       = {Pith},
  title        = {Pith review of: Weight for Robustness: A Comprehensive Approach towards Optimal Fault-Tolerant Asynchronous ML},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LMLVNQXE}},
  note         = {Machine review of arXiv:2501.09621}
}
read the original abstract

We address the challenges of Byzantine-robust training in asynchronous distributed machine learning systems, aiming to enhance efficiency amid massive parallelization and heterogeneous computing resources. Asynchronous systems, marked by independently operating workers and intermittent updates, uniquely struggle with maintaining integrity against Byzantine failures, which encompass malicious or erroneous actions that disrupt learning. The inherent delays in such settings not only introduce additional bias to the system but also obscure the disruptions caused by Byzantine faults. To tackle these issues, we adapt the Byzantine framework to asynchronous dynamics by introducing a novel weighted robust aggregation framework. This allows for the extension of robust aggregators and a recent meta-aggregator to their weighted versions, mitigating the effects of delayed updates. By further incorporating a recent variance-reduction technique, we achieve an optimal convergence rate for the first time in an asynchronous Byzantine environment. Our methodology is rigorously validated through empirical and theoretical analysis, demonstrating its effectiveness in enhancing fault tolerance and optimizing performance in asynchronous ML systems.

Figures

Figures reproduced from arXiv: 2501.09621 by the authors.

Figure 1
Figure 1. Illustration of the delay interval τ (i) t for worker i at iteration t, marking t (current iteration), t (i) (most recent update from worker i), and t − τ (i) t (previous update from worker i). For a given time (iteration) t, let t (i) be the last iteration when worker i made an update. We denote d (i) t := dt (i) , g (i) t := gt (i) , g˜ (i) t := g˜t (i) , and x (i) t = xt (i) , where the latter are individual vect… view at source ↗
Figure 2
Figure 2. CIFAR-10. Test Accuracy of Weighted vs. Non-Weighted Robust Aggregators. This scenario involves 17 workers, including 8 Byzantine workers, with workers’ arrival probabilities proportional to the square of their IDs. We used the µ 2 -SGD in this scenario. Left: label flipping, λ = 0.3. Right: sign flipping, λ = 0.4. Effectiveness of ω-CTMA. We evaluated the test accuracy of weighted robust aggregators with and withou… view at source ↗
Figure 3
Figure 3. CIFAR-10. Test Accuracy Comparison of Weighted Robust Aggregators With and Without ω-CTMA. This scenario involves 9 workers, including either 1 or 3 Byzantine workers. The arrival probabilities of workers are proportional to their IDs, and we employed µ 2 -SGD. On the left, the label flipping and sign flipping attacks are depicted with λ = 0.3 and λ = 0.4, respectively, using 3 Byzantine workers. On the right, the l… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: CIFAR-10. Test Accuracy Comparison Among Different Optimizers. This scenario involves 9 workers (4 Byzantine) with λ = 0.4, and workers’ arrival probabilities are proportional to their IDs. Left: sign flipping. Right: label flipping. Conclusions and Future Work This pa…
Figure 5
Figure 5. Figure 5: MNIST. Test Accuracy of Weighted vs. Non-Weighted Robust Aggregators. This scenario involves 17 workers, including 8 Byzantine workers. The arrival probabilities of workers are proportional to the square of their IDs, with λ = 0.4. We employed µ 2 -SGD in this setup. L…
Figure 6
Figure 6. Figure 6: MNIST. Test Accuracy Comparison of Weighted Robust Aggregators With and Without ω￾CTMA. This scenario involves 9 workers, with a very fast Byzantine worker, and workers’ arrival probabilities proportional to their IDs. We used the µ 2 -SGD in this scenario. On the left…
Figure 7
Figure 7. Figure 7: MNIST. Test Accuracy Comparison Among Different Optimizers. This scenario involves 9 workers, with a very fast Byzantine worker, λ = 0.4, and workers’ arrival probabilities proportional to their IDs. Left: sign flipping. Right: label flipping. 29 [PITH_FULL_IMAGE:figu…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

41 extracted references · 28 canonical work pages

  1. [1]

    Robust training in high dimensions via block coordinate geometric median descent

    Anish Acharya, Abolfazl Hashemi, Prateek Jain, Sujay Sanghavi, Inderjit S Dhillon, and Ufuk Topcu. Robust training in high dimensions via block coordinate geometric median descent. In International Conference on Artificial Intelligence and Statistics, pages 11145--11168. PMLR, 2022

    work page 2022

  2. [2]

    Byzantine stochastic gradient descent

    Dan Alistarh, Zeyuan Allen-Zhu, and Jerry Li. Byzantine stochastic gradient descent. Advances in Neural Information Processing Systems, 31, 2018

    work page 2018

  3. [3]

    Byzantine-resilient non-convex stochastic gradient descent

    Zeyuan Allen-Zhu, Faeze Ebrahimian, Jerry Li, and Dan Alistarh. Byzantine-resilient non-convex stochastic gradient descent. arXiv preprint arXiv:2012.14368, 2020

    arXiv 2012

  4. [4]

    Fixing by mixing: A recipe for optimal byzantine ml under heterogeneity

    Youssef Allouah, Sadegh Farhadkhani, Rachid Guerraoui, Nirupam Gupta, Rafa \"e l Pinot, and John Stephan. Fixing by mixing: A recipe for optimal byzantine ml under heterogeneity. In International Conference on Artificial Intelligence and Statistics, pages 1232--1300. PMLR, 2023

    work page 2023

  5. [5]

    A tight convergence analysis for stochastic gradient descent with delayed updates

    Yossi Arjevani, Ohad Shamir, and Nathan Srebro. A tight convergence analysis for stochastic gradient descent with delayed updates. In Algorithmic Learning Theory, pages 111--132. PMLR, 2020

    work page 2020

  6. [6]

    Asynchronous distributed learning: Adapting to gradient delays without prior knowledge

    Rotem Zamir Aviv, Ido Hakimi, Assaf Schuster, and Kfir Yehuda Levy. Asynchronous distributed learning: Adapting to gradient delays without prior knowledge. In International Conference on Machine Learning, pages 436--445. PMLR, 2021

    work page 2021

  7. [7]

    A little is enough: Circumventing defenses for distributed learning

    Gilad Baruch, Moran Baruch, and Yoav Goldberg. A little is enough: Circumventing defenses for distributed learning. Advances in Neural Information Processing Systems, 32, 2019

    2019

  8. [8]

    Machine learning with adversaries: Byzantine tolerant gradient descent

    Peva Blanchard, El Mahdi El Mhamdi, Rachid Guerraoui, and Julien Stainer. Machine learning with adversaries: Byzantine tolerant gradient descent. Advances in neural information processing systems, 30, 2017

    2017

Show all 41 references
  1. [9]

    Distributed statistical machine learning in adversarial settings: Byzantine gradient descent

    Yudong Chen, Lili Su, and Jiaming Xu. Distributed statistical machine learning in adversarial settings: Byzantine gradient descent. Proceedings of the ACM on Measurement and Analysis of Computing Systems, 1 0 (2): 0 1--25, 2017

    2017

  2. [10]

    Asynchronous stochastic optimization robust to arbitrary delays

    Alon Cohen, Amit Daniely, Yoel Drori, Tomer Koren, and Mariano Schain. Asynchronous stochastic optimization robust to arbitrary delays. Advances in Neural Information Processing Systems, 34: 0 9024--9035, 2021

    2021

  3. [11]

    Anytime online-to-batch, optimism and acceleration

    Ashok Cutkosky. Anytime online-to-batch, optimism and acceleration. In International conference on machine learning, pages 1446--1454. PMLR, 2019

    2019

  4. [12]

    Momentum-based variance reduction in non-convex sgd

    Ashok Cutkosky and Francesco Orabona. Momentum-based variance reduction in non-convex sgd. Advances in neural information processing systems, 32, 2019

    2019

  5. [13]

    Fault tolerant ml: Efficient meta-aggregation and synchronous training

    Tehila Dahan and Kfir Yehuda Levy. Fault tolerant ml: Efficient meta-aggregation and synchronous training. In Forty-first International Conference on Machine Learning, 2024

    2024

  6. [14]

    Asynchronous byzantine machine learning (the case of sgd)

    Georgios Damaskinos, Rachid Guerraoui, Rhicheek Patra, Mahsa Taziki, et al. Asynchronous byzantine machine learning (the case of sgd). In International Conference on Machine Learning, pages 1145--1154. PMLR, 2018

    2018

  7. [15]

    Optimal distributed online prediction using mini-batches

    Ofer Dekel, Ran Gilad-Bachrach, Ohad Shamir, and Lin Xiao. Optimal distributed online prediction using mini-batches. Journal of Machine Learning Research, 13 0 (1), 2012

    2012

  8. [16]

    Distributed momentum for byzantine-resilient stochastic gradient descent

    El Mahdi El Mhamdi, Rachid Guerraoui, and S \'e bastien Louis Alexandre Rouault. Distributed momentum for byzantine-resilient stochastic gradient descent. In 9th International Conference on Learning Representations (ICLR), number CONF, 2021

    2021

  9. [17]

    Online learning and stochastic approximations

    L eon Bottou. Online learning and stochastic approximations. Online learning in neural networks, 17 0 (9): 0 142, 1998

    1998

  10. [18]

    Aflguard: Byzantine-robust asynchronous federated learning

    Minghong Fang, Jia Liu, Neil Zhenqiang Gong, and Elizabeth S Bentley. Aflguard: Byzantine-robust asynchronous federated learning. In Proceedings of the 38th Annual Computer Security Applications Conference, pages 632--646, 2022

    2022

  11. [19]

    Byzantine machine learning made easy by resilient averaging of momentums

    Sadegh Farhadkhani, Rachid Guerraoui, Nirupam Gupta, Rafael Pinot, and John Stephan. Byzantine machine learning made easy by resilient averaging of momentums. In International Conference on Machine Learning, pages 6246--6283. PMLR, 2022

    2022

  12. [20]

    The hidden vulnerability of distributed learning in byzantium

    Rachid Guerraoui, S \'e bastien Rouault, et al. The hidden vulnerability of distributed learning in byzantium. In International Conference on Machine Learning, pages 3521--3530. PMLR, 2018

    2018

  13. [21]

    Byzantine machine learning: A primer

    Rachid Guerraoui, Nirupam Gupta, and Rafael Pinot. Byzantine machine learning: A primer. ACM Computing Surveys, 2023

    2023

  14. [22]

    Introduction to online convex optimization

    Elad Hazan et al. Introduction to online convex optimization. Foundations and Trends in Optimization , 2 0 (3-4): 0 157--325, 2016

    2016

  15. [23]

    Byzantine-robust learning on heterogeneous datasets via bucketing

    Sai Praneeth Karimireddy, Lie He, and Martin Jaggi. Byzantine-robust learning on heterogeneous datasets via bucketing. arXiv preprint arXiv:2006.09365, 2020

    2006 arXiv

  16. [24]

    Learning from history for byzantine robust optimization

    Sai Praneeth Karimireddy, Lie He, and Martin Jaggi. Learning from history for byzantine robust optimization. In International Conference on Machine Learning, pages 5311--5319. PMLR, 2021

    2021

  17. [25]

    Unixgrad: A universal, adaptive algorithm with optimal guarantees for constrained optimization

    Ali Kavis, Kfir Y Levy, Francis Bach, and Volkan Cevher. Unixgrad: A universal, adaptive algorithm with optimal guarantees for constrained optimization. Advances in neural information processing systems, 32, 2019

    2019

  18. [26]

    The cifar-10 dataset

    Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. The cifar-10 dataset. online: https://www.cs.toronto.edu/ kriz/cifar.html , 55 0 (5), 2014

    2014

  19. [27]

    The byzantine generals problem

    Leslie Lamport, Robert Shostak, and Marshall Pease. The byzantine generals problem. In Concurrency: the works of leslie lamport, pages 203--226. 2019

    2019

  20. [28]

    Slow learners are fast

    John Langford, Alexander Smola, and Martin Zinkevich. Slow learners are fast. arXiv preprint arXiv:0911.0491, 2009

    2009 arXiv

  21. [29]

    Mnist handwritten digit database, 2010

    Yann LeCun, Corinna Cortes, Chris Burges, et al. Mnist handwritten digit database, 2010. URL http://yann.lecun.com/exdb/mnist/. Licensed under CC BY-SA 3.0, available at https://creativecommons.org/licenses/by-sa/3.0/

    2010

  22. [30]

    ^2 -sgd: Stable stochastic optimization via a double momentum mechanism

    Kfir Y Levy. ^2 -sgd: Stable stochastic optimization via a double momentum mechanism. arXiv preprint arXiv:2304.04172, 2023

    2023 arXiv

  23. [31]

    Asynchronous sgd beats minibatch sgd under arbitrary delays

    Konstantin Mishchenko, Francis Bach, Mathieu Even, and Blake E Woodworth. Asynchronous sgd beats minibatch sgd under arbitrary delays. Advances in Neural Information Processing Systems, 35: 0 420--433, 2022

    2022

  24. [32]

    Some methods of speeding up the convergence of iteration methods

    Boris T Polyak. Some methods of speeding up the convergence of iteration methods. Ussr computational mathematics and mathematical physics, 4 0 (5): 0 1--17, 1964

    1964

  25. [33]

    The error-feedback framework: Better rates for sgd with delayed gradients and compressed communication

    Sebastian U Stich and Sai Praneeth Karimireddy. The error-feedback framework: Better rates for sgd with delayed gradients and compressed communication. arXiv preprint arXiv:1909.05350, 2019

    1909 arXiv

  26. [34]

    Fall of empires: Breaking byzantine-tolerant sgd by inner product manipulation

    Cong Xie, Oluwasanmi Koyejo, and Indranil Gupta. Fall of empires: Breaking byzantine-tolerant sgd by inner product manipulation. In Uncertainty in Artificial Intelligence, pages 261--270. PMLR, 2020 a

    2020

  27. [35]

    Zeno++: Robust fully asynchronous sgd

    Cong Xie, Sanmi Koyejo, and Indranil Gupta. Zeno++: Robust fully asynchronous sgd. In International Conference on Machine Learning, pages 10495--10503. PMLR, 2020 b

    2020

  28. [36]

    Basgd: Buffered asynchronous sgd for byzantine learning

    Yi-Rui Yang and Wu-Jun Li. Basgd: Buffered asynchronous sgd for byzantine learning. In International Conference on Machine Learning, pages 11751--11761. PMLR, 2021

    2021

  29. [37]

    Buffered asynchronous sgd for byzantine learning

    Yi-Rui Yang and Wu-Jun Li. Buffered asynchronous sgd for byzantine learning. Journal of Machine Learning Research, 24 0 (204): 0 1--62, 2023

    2023

  30. [38]

    Byzantine-robust distributed learning: Towards optimal statistical rates

    Dong Yin, Yudong Chen, Ramchandran Kannan, and Peter Bartlett. Byzantine-robust distributed learning: Towards optimal statistical rates. In International Conference on Machine Learning, pages 5650--5659. PMLR, 2018

    2018

  31. [39]

    A survey of large language models

    Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. A survey of large language models. arXiv preprint arXiv:2303.18223, 2023

    2023 arXiv

  32. [40]

    Asynchronous byzantine-robust stochastic aggregation with variance reduction for distributed learning

    Zehan Zhu, Yan Huang, Chengcheng Zhao, and Jinming Xu. Asynchronous byzantine-robust stochastic aggregation with variance reduction for distributed learning. In 2023 62nd IEEE Conference on Decision and Control (CDC), pages 151--158. IEEE, 2023

    2023

  33. [41]

    Asynchronous byzantine-robust stochastic aggregation with variance reduction for distributed learning

    Zehan Zhu, Yan Huang, Chengcheng Zhao, and Jinming Xu. Asynchronous byzantine-robust stochastic aggregation with variance reduction for distributed learning. 2024

    2024

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.