The paper proposes unbiased gradient renormalization and stale-parameter broadcasts to keep distributed training convergent under random packet loss, with bounded inter-worker drift.
Sdp4bit: Toward 4-bit communication quantization in sharded data parallelism for llm training
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
citation-role summary
background 1
citation-polarity summary
fields
cs.DC 1years
2025 1verdicts
REJECT 1roles
background 1polarities
support 1representative citing papers
citing papers explorer
-
Distributed Training under Packet Loss
The paper proposes unbiased gradient renormalization and stale-parameter broadcasts to keep distributed training convergent under random packet loss, with bounded inter-worker drift.