TSP folds tensor parallelism and sequence parallelism onto one device axis, trading extra communication for lower memory use in transformer training and inference.
One Pensando DSC 200 GbE cloud NIC for loading data and checkpoints
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2026 1verdicts
UNVERDICTED 1representative citing papers
citing papers explorer
-
Folding Tensor and Sequence Parallelism for Memory-Efficient Transformer Training & Inference
TSP folds tensor parallelism and sequence parallelism onto one device axis, trading extra communication for lower memory use in transformer training and inference.