MatrixFSDP places whole 2D weight matrices on single ZeRO-3 owner ranks so matrix optimizers run locally without optimizer-step collectives, preserving ZeRO-3 memory while achieving up to 54.6x optimizer-step speedup at 8 nodes.
Canzona: A uni- fied, asynchronous, and load-balanced framework for distributed matrix-based optimizers.arXiv preprint arXiv:2602.06079,
2 Pith papers cite this work. Polarity classification is still indexing.
2
Pith papers citing it
fields
cs.DC 2years
2026 2representative citing papers
DMuon delivers 1.48x-3.01x end-to-end and 6.85x-163x optimizer-step speedups for Muon on embodied foundation models and LLMs while matching AdamW per-step latency.
citing papers explorer
-
MatrixFSDP: communication-free matrix optimizers under ZeRO-3 parameter sharding
MatrixFSDP places whole 2D weight matrices on single ZeRO-3 owner ranks so matrix optimizers run locally without optimizer-step collectives, preserving ZeRO-3 memory while achieving up to 54.6x optimizer-step speedup at 8 nodes.
-
DMuon: Efficient Distributed Muon Training with Near-Adam Overhead
DMuon delivers 1.48x-3.01x end-to-end and 6.85x-163x optimizer-step speedups for Muon on embodied foundation models and LLMs while matching AdamW per-step latency.