DHelix interleaves two micro-batches at operator granularity and folds the pipeline into a W-shape, hiding communication and improving LLM training throughput by 12-40% on tested clusters.
Chatbots and large language models in radiology: a practical primer for clinical and research applications
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.DC 1years
2024 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Hiding Communication Cost in Distributed LLM Training via Micro-batch Co-execution
DHelix interleaves two micro-batches at operator granularity and folds the pipeline into a W-shape, hiding communication and improving LLM training throughput by 12-40% on tested clusters.