CoCoDC uses Taylor-expansion delay compensation and adaptive fragment scheduling to reach a target perplexity up to 21% fewer steps than Streaming DiLoCo in simulated cross-region LLM training.
Megatron-LM: Training Multi-Billion Parameter Language Mod- els Using Model Parallelism,
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.DC 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Cross-region Model Training with Communication-Computation Overlapping and Delay Compensation
CoCoDC uses Taylor-expansion delay compensation and adaptive fragment scheduling to reach a target perplexity up to 21% fewer steps than Streaming DiLoCo in simulated cross-region LLM training.