PyTorch distributed data parallel attains near-linear scalability on 256 GPUs through gradient bucketing, computation-communication overlap, and selective synchronization skipping.
https://pytorch.org/docs/master/rpc.html, 2019
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.DC 1years
2020 1verdicts
ACCEPT 1representative citing papers
citing papers explorer
-
PyTorch Distributed: Experiences on Accelerating Data Parallel Training
PyTorch distributed data parallel attains near-linear scalability on 256 GPUs through gradient bucketing, computation-communication overlap, and selective synchronization skipping.