Pith. sign in

REVIEW 1 cited by

Local AdaAlter: Communication-Efficient Stochastic Gradient Descent with Adaptive Learning Rates

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 1911.09030 v2 pith:YSCXEP2A submitted 2019-11-20 cs.LG cs.DCstat.ML

Cong Xie , Oluwasanmi Koyejo , Indranil Gupta , Haibin Lin This is my paper
classification cs.LGcs.DCstat.ML
keywords communicationadaptivealgorithmlearningoverheadproposedratesreduces
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

When scaling distributed training, the communication overhead is often the bottleneck. In this paper, we propose a novel SGD variant with reduced communication and adaptive learning rates. We prove the convergence of the proposed algorithm for smooth but non-convex problems. Empirical results show that the proposed algorithm significantly reduces the communication overhead, which, in turn, reduces the training time by up to 30% for the 1B word dataset.

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Adaptive Federated Optimization

    cs.LG 2020-02 unverdicted novelty 6.0 of 10

    Proposes federated adaptive optimizers (FedAdagrad, FedAdam, FedYogi) with convergence analysis for non-convex objectives under data heterogeneity and reports empirical gains over FedAvg.

Pith tools