Semi-Cyclic Stochastic Gradient Descent

H. Brendan McMahan; Hubert Eichner; Kunal Talwar; Nathan Srebro; Tomer Koren

RECORD

Semi-Cyclic Stochastic Gradient Descent

Hubert Eichner, Tomer Koren, H. Brendan McMahan, Nathan Srebro, Kunal Talwar

Machine Learning · Submitted 23 Apr 2019

abstract compiled · 2026-09-23 · no language-model review

The authors' main claim We show that such block-cyclic structure can significantly deteriorate the performance of SGD, but propose a simple approach that allows prediction with the same performance guarantees as for i.i.d., non-cyclic, sampling.

how it is stated · asserted 0.99 read the record →

Comment (0) Read on arXiv ↗

Cite ▾

arXiv ID1904.10120 (version 1)

Pith ID65IFA7G7, Pith's permanent record of this paper and its review

CitationH. Brendan McMahan, Hubert Eichner, Kunal Talwar, Nathan Srebro, Tomer Koren. "Semi-Cyclic Stochastic Gradient Descent." arXiv:1904.10120 (2019). https://pith.science/paper/65IFA7G7

Claim paper

The compiled record

The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.

Open the compiled record

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-09-23: 1 request, 3,001 input tokens.

What the abstract itself states, as probabilities: claims first 0.04 · states limitation 0.38 · human participants 0.11 · controlled comparison 0.51 · randomized assignment 0.03 · states quantitative result 0.09 · compares with prior results 0.63 · mentions code or data availability 0.02.

The abstract, sentence by sentence

arXiv listing abstract v1. Results and the main claim are underlined; show every role.

We consider convex SGD updates with a block-cyclic structure, i.e. where each cycle consists of a small number of blocks, each with many samples from a possibly different, block-specific, distribution. This situation arises, e.g., in Federated Learning where the mobile devices available for updates at different times during the day have different characteristics. We show that such block-cyclic structure can significantly deteriorate the performance of SGD, but propose a simple approach that allows prediction with the same performance guarantees as for i.i.d., non-cyclic, sampling.

background contribution method result interpretation main claim

What the classifier reports for each sentence. The number is the probability it gave the label.
#	sentence	role	the authors' own work	states the main conclusion	strength
s0	We consider convex SGD updates with a block-cyclic structure, i.e. where each cycle consists of a small number of blocks, each with many samples from a possibly different, block-specific, distribution.	background 0.65	0.20	0.07	asserted 0.77
s1	This situation arises, e.g., in Federated Learning where the mobile devices available for updates at different times during the day have different characteristics.	background 0.99	0.06	0.03	hedged possibility 0.52
s2	We show that such block-cyclic structure can significantly deteriorate the performance of SGD, but propose a simple approach that allows prediction with the same performance guarantees as for i.i.d., non-cyclic, sampling.	contribution 0.87	0.94	0.91	asserted 0.99

What kind of paper the classifier reports this is: method or tool 0.62 · evidence from proof or derivation 0.99.

What was examined, and what was not

sources read

arXiv listing abstract v1

not examined

full text
figures and tables
references and citing sentences
relations to other papers

levels run

P abstract

not run

S sections
T body sentences
C citations
R cross-paper relations

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. Every number in the measurements table is the authors', checked by code. Record as JSON

The paper

Original abstract

We consider convex SGD updates with a block-cyclic structure, i.e. where each cycle consists of a small number of blocks, each with many samples from a possibly different, block-specific, distribution. This situation arises, e.g., in Federated Learning where the mobile devices available for updates at different times during the day have different characteristics. We show that such block-cyclic structure can significantly deteriorate the performance of SGD, but propose a simple approach that allows prediction with the same performance guarantees as for i.i.d., non-cyclic, sampling.

Human review

How Pith checks papers → · Full audit of this paper →

References

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learning
cs.LG 2021-01 conditional novelty 6.0 of 10

FedAvg with separate server and worker learning rates achieves a convergence rate that improves with the number of participating workers under non-i.i.d. data and partial participation, without variance-reduction machinery.
Federated Learning With Quantized Global Model Updates
cs.IT 2020-06 conditional novelty 5.0 of 10

Broadcasting the quantized change in the global model, rather than the quantized model, gives devices a more accurate model at equal bit cost and matches lossless federated learning on MNIST and CIFAR-10.
Learning to Detect Malicious Clients for Robust Federated Learning
cs.LG 2020-02 conditional novelty 5.0 of 10

Training a variational autoencoder on unbiased model updates lets a federated learning server flag malicious client updates by their high reconstruction error, detecting Byzantine and backdoor attacks with F1 scores o...
Advances and Open Problems in Federated Learning
cs.LG 2019-12 accept novelty 3.0 of 10

The paper consolidates federated learning into a shared definition, a taxonomy of non-identical client data, a threat-model table, and a section-by-section agenda of open problems across optimization, privacy, robustn...

How the literature uses it

10 arXiv papers cite this in 16 sentences

Each citing sentence is taken from the citing paper's LaTeX source and typed by Pith's classifier. A label appears at probability 0.8 or more (0.95 for support), a contest at 0.7 or more; the quoted words are the citing authors'. How each label was checked · JSON

Papers citing it · 10 sentences without a confident label

Builds on it 1 paper

While in our framework we assume the distribution $$ to be fixed, it can be easily extended to several proper distributions $_j$'s or we can even handle a block-cyclic structure with each block having an arbitrary proper distribution $_j$ over the given block $j$ combining our analysis with the results of [ref] .

A Better Alternative to Error Feedback for Communication-Efficient Distributed Learning · Extensions · 0.92

Compares with it 1 paper

We train all three models with test accuracy comparable to the previous work [ref] .

Learning to Detect Malicious Clients for Robust Federated Learning · Datasets and ML Models · 1.00

Cites it as background 4 papers

[ref] and [ref] proposed star-based and ring-based groups, respectively, without global communication.

TornadoAggregate: Accurate and Scalable Federated Learning via the Ring-Based Architecture · Introduction · 0.82
Ring-based groups, without global communication, while further elaborating on the periodically variational distribution of the data samples, treated by semi-cyclic Stochastic Gradient Descent (SGD) is discussed in [ref] .

Introducing Federated Learning into Internet of Things ecosystems -- preliminary considerations · Work related to Federated Learning topology · 0.90
Different approaches have been studied to mitigate the effect of non-iid data in FL [ref] .

Federated Learning With Quantized Global Model Updates · Introduction · 0.90

Discussion

Know something about this paper? Leave a comment, or highlight any passage above to comment on it.