HelixPipe repartitions transformer layers so attention runs in parallel across pipeline stages, cutting idle time and memory imbalance, and reports a 26% speedup for a 7B model on 128k-length sequences.
Title resolution pending
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
HelixPipe: Efficient Distributed Training of Long Sequence Transformers with Attention Parallel Pipeline Parallelism
HelixPipe repartitions transformer layers so attention runs in parallel across pipeline stages, cutting idle time and memory imbalance, and reports a 26% speedup for a 7B model on 128k-length sequences.