A reconfigurable optical-plus-mesh architecture could interconnect over 100,000 accelerator chips with less than 10% of the fat-tree cost for All-Reduce bandwidth.
I've Got 99 Problems But FLOPS Ain't One
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
Hyperscalers dominate the landscape of large network deployments, yet they rarely share data or insights about the challenges they face. In light of this supremacy, what problems can we find to solve in this space? We take an unconventional approach to find relevant research directions, starting from public plans to build a $100 billion datacenter for machine learning applications. Leveraging the language models scaling laws, we discover what workloads such a datacenter might carry and explore the challenges one may encounter in doing so, with a focus on networking research. We conclude that building the datacenter and training such models is technically possible, but this requires novel wide-area transports for inter-DC communication, a multipath transport and novel datacenter topologies for intra-datacenter communication, high speed scale-up networks and transports, outlining a rich research agenda for the networking community.
citation-role summary
background 1
citation-polarity summary
fields
cs.AR 1years
2025 1verdicts
CONDITIONAL 1roles
background 1polarities
unclear 1representative citing papers
citing papers explorer
-
RailX: A Flexible, Scalable, and Low-Cost Network Architecture for Hyper-Scale LLM Training Systems
A reconfigurable optical-plus-mesh architecture could interconnect over 100,000 accelerator chips with less than 10% of the fat-tree cost for All-Reduce bandwidth.