An RL agent that co-optimizes parallelism degrees and per-operator sharding dimensions finds distributed inference strategies that beat random search and simulated annealing, and slightly outperform Megatron-LM heuristics in simulator tests.
Uniap: Unifying inter-and intra-layer automatic parallelism by mixed integer quadratic programming
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Learning to Shard: RL for Co-optimizing the Parallelism Degrees and Per-operator Sharding Dimensions in Distributed LLM Inference
An RL agent that co-optimizes parallelism degrees and per-operator sharding dimensions finds distributed inference strategies that beat random search and simulated annealing, and slightly outperform Megatron-LM heuristics in simulator tests.