A framework called H2 combines a unified PyTorch interface, device-direct RDMA, and automatically searched pipeline parallelism to train a 100B model on over 1,000 heterogeneous chips, with up to 16.37% higher aggregate throughput than separate homogeneous runs.
Title resolution pending
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.DC 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
H2:Towards Efficient Large-Scale LLM Training on Hyper-Heterogeneous Cluster over 1,000 Chips
A framework called H2 combines a unified PyTorch interface, device-direct RDMA, and automatically searched pipeline parallelism to train a 100B model on over 1,000 heterogeneous chips, with up to 16.37% higher aggregate throughput than separate homogeneous runs.