Pith. sign in

REVIEW 3 major objections 4 minor 3 cited by

MixNet: A Runtime Reconfigurable Optical-Electrical Fabric for Distributed Mixture-of-Experts Training

T0 review · 3 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read MixNet's reconfigurable fabric matches fat-tree speed while cutting MoE networking cost up to 2.3x.

desk verdict Fresh architectural idea for MoE training interconnects, backed by real measurements and a working prototype, but the central performance claim leans on an unproven NIC re-activation fix. read the letter →

arxiv 2501.03905 v4 pith:R4UX3J7S submitted 2025-01-07 cs.NI cs.LG

Xudong Liao , Yijun Sun , Han Tian , Xinchen Wan , Yilun Jin , Zilong Wang , Zhenghang Ren , Xinyang Huang
show 8 more authors
Wenxue Li Kin Fai Tse Zhizhen Zhong Guyue Liu Ying Zhang Xiaofeng Ye Yiming Zhang Kai Chen
This is my paper · ORCID
classification cs.NIcs.LG
keywords NetworkArchitectureMixture-of-ExpertsOpticalCircuitSwitchingAIInfrastructureDistributedTrainingTopologyReconfigurationGPUInterconnectAll-to-AllCommunication
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MixNet's claim is that Mixture-of-Experts (MoE) training does not need an expensive non-blocking electrical fabric, because the dynamic all-to-all traffic generated by expert routing stays local to each MoE block and can be served by a network that reconfigures itself during training. The paper proposes a mixed fabric in which a regionally reconfigurable optical circuit switch (OCS) sits between the server-local scale-up network and the global electrical packet-switched network, with each server splitting its network connections between the two. A traffic monitor, a greedy circuit-allocation algorithm, and a custom collective communication runtime let the OCS topology track the changing expert traffic across training iterations. On a 32-GPU prototype and in packet-level simulations, MixNet trains four representative MoE models at speeds close to non-blocking fat-tree and rail-optimized fabrics while improving networking cost-efficiency by 1.2x-1.5x at 100 Gbps and 1.9x-2.3x at 400 Gbps link bandwidths. If this holds, a cluster builder could get fat-tree-level training speed with significantly less networking hardware.

What carries the argument

The load-bearing mechanism is the regionally reconfigurable high-bandwidth domain: a commodity OCS with millisecond-scale reconfiguration (a few hundred ports) connected to a subset of each server's network cards, coordinated by a decentralized topology controller per region. The argument is carried by a greedy bottleneck-allocation algorithm (Algorithm 1) that repeatedly finds the server pair whose all-to-all transfer would take longest under the current circuit allocation and adds an optical circuit between them until the available ports are exhausted, producing a network-card-level mapping that the OCS then realizes. Around this sit the traffic monitor, which exploits the partial predictability of the four all-to-all phases per MoE block, and the collective communication runtime, which routes tensor-parallel traffic inside the server, expert-parallel traffic over the OCS fabric, and data- and pipeline-parallel traffic over the electrical packet-switched network.

What would settle it

Run the same 32-GPU prototype training a real MoE model and include the full reconfiguration turnaround, optical switch command, path switching, transceiver re-lock, and network-card re-activation, in every iteration. If the turnaround stays near the measured 5.7 seconds (99th percentile 6.33 s) instead of the 25 ms used in simulation, MixNet's speed parity with fat-tree would fail, since each training iteration contains multiple reconfigurations and the gaps between all-to-all phases are only tens of milliseconds.

Watch

Extended reading notes

Core claim

The central discovery is that MoE training's expert-parallel all-to-all traffic, although non-deterministic and non-uniform, is regionally local: it stays within a single MoE block and its pipeline-parallel stage, so a fabric only needs to reconfigure within a few hundred server ports, not globally. MixNet leverages this by placing a millisecond-reconfigurable OCS at the boundary of the scale-up and scale-out domains and reconfiguring it inside the training iteration, using the four all-to-all phases per MoE layer, which are identical or transposed and partially predictable, as scheduling windows. The paper argues that this regionally reconfigurable design reconciles the OCS trade-off between port count and reconfiguration delay, making in-training topology reconfiguration practical with commodity hardware at 30K+ GPU scales. The paper's stated result is parity in training iteration time with non-blocking fat-tree and rail-optimized fabrics at lower networking cost, which it quantifies as 1.2x-1.5x and 1.9x-2.3x better cost-efficiency at 100 Gbps and 400 Gbps, respectively.

Load-bearing premise

The whole benefit rests on the assumption that after each optical reconfiguration, the network cards and transceivers recover within the computation window, but the prototype measures an average recovery time of about 5.7 seconds, which the paper excludes from training time and attributes to a fixable transceiver limitation.

Editorial extensions

If this is right

  • MoE training can run at fat-tree-level speed while using roughly half the networking hardware, because dense expert-pair traffic is carried by cheap optical circuits instead of over-provisioned electrical switches.
  • In-training reconfiguration becomes a schedulable resource: since the four all-to-all communication phases per MoE layer share the same or transposed traffic matrices, topology changes can be prepared during attention and expert computation, hiding millisecond-scale OCS delays.
  • The fabric scales by partitioning rather than by enlarging OCS port count, so commodity OCS devices (a few hundred ports) suffice for clusters of tens of thousands of GPUs.
  • When co-packaged optical I/O reaches accelerator chips, the same regional OCS concept extends to high-radix scale-up systems, where MixNet's simulations show a 1.3x iteration-time improvement over an NVL72-style cluster.
  • The EPS-OCS split gives natural fault tolerance: network-card and GPU failures can be worked around by forwarding through the other fabric, with simulation overheads of at most a few percent for network-card failures and up to about 13% for a full server failure.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct consequence the paper leaves implicit is that MixNet's cost advantage is partly a trade of transceiver price for switch price; if OCS port or transceiver costs fall faster than electrical switch ports, the optimal optical degree and the breakeven cluster size both shift, changing the reported margins.
  • The regional-locality measurement should be re-tested on models trained with auxiliary load-balancing losses, since the paper's own production data show that the total per-expert volumes converge while the sparse traffic matrix persists; models with explicit expert pruning or biased routing may exhibit stronger or qualitatively different locality.
  • If burst-mode transceivers deliver the millisecond re-activation the paper assumes, the same regional reconfiguration mechanism is a natural fit for MoE inference serving, where token routing is even more dynamic; the paper does not evaluate this application.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. MixNet is a regionally reconfigurable optical-electrical fabric for distributed Mixture-of-Experts (MoE) training. The paper makes three main contributions: (1) a production measurement study of MoE all-to-all traffic showing that, despite temporal and spatial non-uniformity, the traffic has strong locality within MoE blocks; (2) an architecture that augments a static electrical packet-switched fabric with regional optical circuit switches (OCS) and a greedy topology-reconfiguration algorithm, together with a traffic predictor (MixNet-Copilot) and a custom collective communication runtime; and (3) an evaluation using a 32-GPU A100 prototype and large-scale packet-level simulations across four MoE models. The paper claims that MixNet achieves training speed comparable to non-blocking fat-tree and rail-optimized fabrics while improving networking cost-efficiency by 1.2x-1.5x at 100 Gbps and 1.9x-2.3x at 400 Gbps link bandwidths.

Significance. If the central claim holds, MixNet would be a substantial step toward cost-efficient, runtime-reconfigurable interconnects for MoE training, an increasingly important workload for large-scale AI clusters. The paper is commendable for its concrete measurement study, a working testbed, and a transparent cost model based on publicly available component prices. The evaluation is thorough in many respects: it compares against independent baselines (fat-tree, rail-optimized, TopoOpt), reports sensitivity to optical degree and reconfiguration latency, and includes failure-resilience experiments. However, the key hardware assumption that NIC/transceiver re-activation can be made fast enough to hide within MoE computation is not validated in the prototype; the measured 5.67 s average NIC activation time is excluded from the testbed training time. This makes the reported significance conditional on future burst-mode transceiver features. The core algorithmic and architectural ideas are sound, but the performance-cost conclusion is not yet established for current commodity hardware.

major comments (3)
  1. [Appendix C, Figures 22-23 and §6 Figure 10] The prototype evaluation excludes the measured NIC and transceiver activation time from the reported training iteration time. Appendix C explicitly states that the NIC activation time (average 5.67 s, 99th percentile 6.33 s) is excluded from the testbed training time, and the OCS-only reconfiguration is measured at 41.44-46.75 ms on average (99th percentile 60-68 ms). Because every in-training reconfiguration requires the OCS switch and the NIC/transceiver to re-activate before data transfer can resume, the testbed in Figure 10 does not demonstrate that MixNet delivers 'comparable performance' to the EPS baseline under actual current hardware behavior. The paper defers the resolution of this to future burst-mode transceivers, but the central performance claim in §7 is built on the same idealization, so the claim that MixNet is production-ready with commodity hardware is not supported by the presented evidence.
  2. [§7.1 and Appendix D.7, Figure 28] The large-scale simulations set the OCS reconfiguration latency to 25 ms, but the prototype's measured OCS-only reconfiguration time is 41-68 ms even before NIC activation is considered. Figure 28 presents sensitivity to reconfiguration time from 1 µs to 10 s, but it does not mark the 25 ms simulation point or the measured 41-68 ms range, and the y-axis shows only normalized iteration time without the NIC activation component. The manuscript should either repeat the key simulations with the measured OCS reconfiguration delays, or provide a documented engineering justification for why 25 ms is representative of the deployed system. Without this, the cost-efficiency headline (e.g., 1.9x-2.3x at 400 Gbps) is not tied to a validated performance denominator.
  3. [§B.1 and Figure 19] MixNet-Copilot is evaluated only on top-k prediction accuracy for expert load distributions; there is no end-to-end measurement or simulation of how prediction errors affect actual training iteration time. Since the first all-to-all communication in the forward pass relies on this predictor (or on the previous layer's topology), a wrong prediction could yield a suboptimal OCS topology and erase the intended performance benefit. The paper should report the training-time impact under realistic prediction error, even in simulation, to substantiate the claim that proactive reconfiguration for the first all-to-all is beneficial.
minor comments (4)
  1. [§10] The sentence about Shoal reads 'is not unsuitable for large-scale MoE training'; this appears to be a double negative and should probably be 'is not suitable' or 'is unsuitable'.
  2. [§B.1] The text refers to 'Sequential Least Squares Programming (SLAP)', but the standard name for this optimization method is SLSQP (Sequential Least SQuares Programming).
  3. [Appendix C] The exclusion of NIC activation time is a significant limitation that currently appears only in the appendix; it should be prominently disclosed in the main text and in the abstract, since it directly affects the interpretation of the prototype results.
  4. [Figure 28] The figure would be more informative if the x-axis marked the 25 ms simulation assumption and the measured 41-68 ms prototype range, and if the y-axis were annotated to indicate that the NIC activation time is not included.

Circularity Check

0 steps flagged · score 1.0 of 10

No circular derivation found; the central performance claim is evaluated against external baselines and an independent simulator, while the excluded NIC-activation time is an unverified hardware assumption rather than a circular step.

full rationale

The paper's derivation chain is self-contained against external benchmarks. MixNet's training-speed and cost-efficiency results are established by packet-level simulations (FlexFlow plus htsim) that compare against Fat-tree, Rail-optimized, TopoOpt, and oversubscribed Fat-tree, and by a 32-GPU hardware prototype compared with an EPS baseline. The greedy OCS allocation in Algorithm 1 takes traffic demands as input and produces a topology; it is not defined in terms of the resulting iteration time, so there is no self-definitional reduction. The MixNet-Copilot traffic predictor is fit to recent traffic records as a standard time-series estimator and is evaluated on prediction accuracy (Figure 19), not used to define the headline performance-per-dollar result. The only overlapping-author citations are FuseLink [93], used as the RDMA runtime base for the prototype, and TACC [111] in the acknowledgments; neither is load-bearing for the central architectural or performance claim. The most serious limitation asserted in the manuscript is that the prototype excludes the measured 5.67 s average NIC/transceiver activation time, with Appendix C stating 'we currently exclude this NIC activation time to calculate the actual training time in MixNet testbed experiments,' while the scale-out simulations assume a 25 ms reconfiguration. This is a correctness and falsifiability risk about an unproven burst-mode transceiver assumption, not a case where a result reduces by construction to its own inputs. No fitted parameter is renamed as a prediction, and no uniqueness theorem or ansatz is imported from same-author work to force the design. Score 1 reflects the presence of minor non-load-bearing self-citations; the derivation itself is not circular.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

No new physical entities are postulated. MixNet is a system architecture composed of existing OCS, EPS, and NVSwitch components. The 'regional high-bandwidth domain' is a design concept, not a new physical entity.

free parameters (3)
  • Traffic predictor window size k and weights w_i = not reported
    Used in Eq. (1) (Appendix B.1) to predict next-layer all-to-all demand; affects the proactive reconfiguration of the first forward all-to-all communication. No values or sensitivity analysis are provided.
  • Optical degree alpha = 6 per server in simulation; 3 per server in testbed
    Hand-chosen design parameter determining the number of OCS links per server; performance and cost scale with it (Appendix D.6, Figure 27).
  • OCS reconfiguration latency in simulation = 25 ms
    Simulations block the network for 25 ms (Section 7.1), while the prototype measures 41-68 ms (Appendix C, Figure 21). This is a favorable modeling choice.
assumptions (5)
  • domain assumption MoE all-to-all traffic is strongly localized within MoE blocks and pipeline stages, so regional reconfiguration suffices
    Section 3, Figure 5; based on one production measurement of Mixtral 8x7B on 128 H800 GPUs. The entire regional design rests on this locality claim.
  • domain assumption OCS reconfiguration latency can be hidden within the expert and attention computation phases (roughly 100 ms)
    Section 3, Figure 3. If computation phases shrink (higher bandwidth, smaller micro-batch), the hiding window disappears and reconfiguration stalls training.
  • ad hoc to paper NIC re-activation after OCS switching can be reduced to negligible levels via burst-mode transceivers
    Appendix C: actual measured NIC activation is 5.67 s average, excluded from testbed results. The fix is not demonstrated and is treated as a future engineering solution.
  • domain assumption Packet-level simulation with htsim accurately captures RDMA and RoCEv2 dynamics
    Section 7.1; the simulator is calibrated to profiled computation times, but the network model itself is not validated against large-scale measurements.
  • domain assumption Cost model per-port prices in Table 4 reflect production pricing
    Section 7.2; most prices are borrowed from TopoOpt, with 800G NIC estimated at 1.5x the 400G price. Any change in these prices shifts the cost-efficiency advantage.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MixNet: A Runtime Reconfigurable Optical-Electrical Fabric for Distributed Mixture-of-Experts Training." pith.science (2026). https://pith.science/paper/R4UX3J7S

@misc{pith2026250103905,
  author       = {Pith},
  title        = {Pith review of: MixNet: A Runtime Reconfigurable Optical-Electrical Fabric for Distributed Mixture-of-Experts Training},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/R4UX3J7S}},
  note         = {Machine review of arXiv:2501.03905}
}
read the original abstract

Mixture-of-Expert (MoE) models outperform conventional models by selectively activating different subnets, named experts, on a per-token basis. This gated computation generates dynamic communications that cannot be determined beforehand, challenging the existing GPU interconnects that remain static during the distributed training process. In this paper, we advocate for a first-of-its-kind system, called MixNet, that unlocks topology reconfiguration during distributed MoE training. Towards this vision, we first perform a production measurement study and show that the MoE dynamic communication pattern has strong locality, alleviating the requirement of global reconfiguration. Based on this, we design and implement a regionally reconfigurable high-bandwidth domain on top of existing electrical interconnects using optical circuit switching (OCS), achieving scalability while maintaining rapid adaptability. We have built a fully functional MixNet prototype with commodity hardware and a customized collective communication runtime that trains state-of-the-art MoE models with in-training topology reconfiguration across 32 A100 GPUs. Large-scale packet-level simulations show that MixNet delivers comparable performance as the non-blocking fat-tree fabric while boosting the training cost efficiency (e.g., performance per dollar) of four representative MoE models by 1.2x-1.5x and 1.9x-2.3x at 100 Gbps and 400 Gbps link bandwidths, respectively.

Figures

Figures reproduced from arXiv: 2501.03905 by the authors.

Figure 1
Figure 1. Illustration of the MoE’s gated expert architecture and its distributed training strategies. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 3
Figure 3. [Mixtral 8×7B in production] EP’s all-to-all com￾munications occupy 33% to 55% of the total training iteration time in 400 Gbps network. erating the least amount of communication with deterministic vol￾ume. Expert Parallelism (EP). In MoE models, different experts in an MoE block are allocated to different GPUs [59, 74, 77, 82] (Fig￾ure 1b). Since each GPU needs to send its local states to other experts and receive … view at source ↗
Figure 4
Figure 4. [Mixtral 8×7B in production] All-to-all traffic dynamics during MoE training. total volume). The EP generates the second-highest traffic volume (30%), leaving PP and DP contributing less than 6%. For LLaMA-MoE and Qwen-MoE, we find that EP becomes the most communication intensive (more than 80%). This is because the size of the largest layer (i.e., expert) fits into a single GPU memory. 2.2 GPU Interconnects Scale-u… view at source ↗
Figures from the paper (23 more)
Figure 6
Figure 6. Figure 6: The MixNet network architecture. training requires the best fit between the traffic patterns gener￾ated by different parallelization strategies and the interconnect technologies. In [PITH_FULL_IMAGE:figures/full_fig_p005_6.png]
Figure 7
Figure 7. Figure 7: MixNet system implementation. ing iteration are four all-to-all communications with the same or transposed traffic pattern. However, these traffic patterns are non￾deterministic across different training iterations. Hence, we need to develop a mechanism to reconfigure …
Figure 8
Figure 8. Figure 8: Routing of all-to-all communications in MixNet. For presentation simplicity, each server only contains 4 GPUs, and the TX/RX links are merged into a single link. In practice, MixNet supports the standard setup of 8 GPUs or more. synchronize the model parameters. Finall…
Figure 10
Figure 10. Figure 10: [Testbed] End-to-end training iteration time on [PITH_FULL_IMAGE:figures/full_fig_p009_10.png]
Figure 11
Figure 11. Figure 11: [Simulation] Networking cost analysis. 200 400 600 800 Bandwidth (Gbps) 1.0 2.0 Normalized iter. time Fat-tree Rail-optimized TopoOpt OverSub. Fat-tree MixNet (a) Mixtral 8×22B 200 400 600 800 Bandwidth (Gbps) 1.0 2.0 Normalized iter. time Fat-tree Rail-optimized Topo…
Figure 12
Figure 12. Figure 12: [Simulation] Training speed-ups in a cluster of 128 servers with 1024 GPUs. [PITH_FULL_IMAGE:figures/full_fig_p010_12.png]
Figure 13
Figure 13. Figure 13: [Simulation] Performance-cost comparison of different interconnects on four state-of-the-art MoE models. [PITH_FULL_IMAGE:figures/full_fig_p011_13.png]
Figure 14
Figure 14. Figure 14: [Simulation] Failure resiliency of MixNet. 7.4 Cost Efficiency: Pareto Front Analysis To better capture the trade-offs between networking cost and per￾formance, we present the Pareto Front analysis of different inter￾connects in [PITH_FULL_IMAGE:figures/full_fig_p011…
Figure 15
Figure 15. Figure 15: Such an optical switching architecture would enable a [PITH_FULL_IMAGE:figures/full_fig_p011_15.png]
Figure 15
Figure 15. Figure 15: MixNet with co-packaged optical ports directly attached to xPUs. 8 Tbps 16 Tbps Total GPU I/O Bandwidth 0.0 0.5 1.0 1.5 2.0 Norm. Iter. Time NVL72 MixNet (w/ optical I/O) [PITH_FULL_IMAGE:figures/full_fig_p012_15.png]
Figure 16
Figure 16. Figure 16: [Simulation] Performance comparison with a 2048- [PITH_FULL_IMAGE:figures/full_fig_p012_16.png]
Figure 18
Figure 18. Figure 18: [Mixtral 8×7B in production] Non-uniform token distribution across MoE blocks. Random Unmodified MixNet-Copilot Accuracy 0 0.5 1.0 Top K 1 2 3 4 [PITH_FULL_IMAGE:figures/full_fig_p017_18.png]
Figure 19
Figure 19. Figure 19: Average prediction accuracy of MixNet-Copilot. B Implementation Details B.1 Traffic Demand Prediction MixNet aims to handle the first all-to-all communication in the forward pass with a predictive approach. By default, the OCS topol￾ogy for this initial communication …
Figure 17
Figure 17. Figure 17: Timeline of MoE models [PITH_FULL_IMAGE:figures/full_fig_p017_17.png]
Figure 20
Figure 20. Figure 20: Reconfiguration timeline during runtime. [PITH_FULL_IMAGE:figures/full_fig_p018_20.png]
Figure 21
Figure 21. Figure 21: [Testbed] Reconfiguration delay across different [PITH_FULL_IMAGE:figures/full_fig_p018_21.png]
Figure 22
Figure 22. Figure 22: [Testbed] Overall timeline of one OCS control. [PITH_FULL_IMAGE:figures/full_fig_p018_22.png]
Figure 23
Figure 23. Figure 23: [Testbed] CDF of time elapsed from OCS reconfig [PITH_FULL_IMAGE:figures/full_fig_p018_23.png]
Figure 24
Figure 24. Figure 24: [Simulation] Cost comparison of different EPS [PITH_FULL_IMAGE:figures/full_fig_p019_24.png]
Figure 25
Figure 25. Figure 25: [Simulation] Training speed ups of Mixtral models with large batch sizes. [PITH_FULL_IMAGE:figures/full_fig_p020_25.png]
Figure 27
Figure 27. Figure 27: [Simulation] Impact of optical degree 𝛼 in MixNet. cluster size varying from 128 servers to 4,096 servers, covering up to 32768 GPUs. MixNet demonstrates scalability by funda￾mentally relaxing the port limits of OCS through the design of several decentralized regional…
Figure 26
Figure 26. Figure 26: [Simulation] Scalability analysis of MixNet with different cluster sizes. D.4 Training Speed Ups of Mixtral Models with Larger Batch Sizes We further evaluate MixNet’s performance with larger batch sizes using two Mixtral MoE models (Mixtral 8×7B and Mixtral 8×22B). F…
Figure 28
Figure 28. Figure 28: [Simulation] Impact of reconfiguration latency. [PITH_FULL_IMAGE:figures/full_fig_p021_28.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. InfiniteHBD: Building Datacenter-Scale High-Bandwidth Domain for LLM with Optical Circuit Switching Transceivers

    cs.NI 2025-02 conditional novelty 8.0 of 10

    InfiniteHBD embeds optical circuit switching inside each transceiver to build reconfigurable ring networks for GPU clusters, claiming node-level fault isolation at roughly one-third the cost of NVL-72.

  2. Opus: Photonic Rail-Optimized Fabric in ML Datacenters

    cs.NI 2026-02 conditional novelty 6.0 of 10

    Opus time-multiplexes a single photonic rail fabric across parallelism phases in ML training, achieving up to 23x network power reduction and 4x cost savings at under 6.7% training overhead in simulation.

  3. Photonic Rails in ML Datacenters

    cs.NI 2025-07 conditional novelty 6.0 of 10

    A photonic rail design that reconfigures optical circuits between parallelism phases within a training job can emulate electrical rails with about 70% cost and 96% power savings and a few percent iteration-time overhe...

Reference graph

Works this paper leans on

125 extracted references · 53 canonical work pages · cited by 3 Pith papers

  1. [1]

    [n. d.]. 100GBASE-SR4 850nm 100m DOM MPO-12/UPC MMF Optical Transceiver Module. https://www.fs.com/products/48354.html. ([n. d.])

  2. [2]

    [n. d.]. 200GBASE-SR4 850nm 100m DOM MPO-12/UPC MMF Optical Transceiver Module. https://www.fs.com/products/139696.html. ([n. d.])

  3. [3]

    [n. d.]. 400GBASE-SR4 PAM4 850nm 100m DOM MPO-12/APC MMF Optical Transceiver Module. https://www.fs.com/products/226577.html. ([n. d.])

  4. [4]

    [n. d.]. All-to-all traffic demand collection in Megatron-LM. https://github.com /NVIDIA/Megatron-LM/blob/461b06cd6d1fb4a625cebdbca499dac9484087fc/ megatron/core/transformer/moe/token_dispatcher.py#L432. ([n. d.])

  5. [5]

    [n. d.]. Breakout optical cables. https://arubanetworking.hpe.com/techdocs/S witches/xcvrs/xcvr_guide/Content/Chp_overview/spl-opt-cab.htm. ([n. d.])

  6. [6]

    [n. d.]. Calient Optical Circuit Switch. www.calient.net. ([n. d.])

  7. [7]

    [n. d.]. Calix ® 100-04482 Compatible TAA 10GBs XGS-PON OLT XFP Transceiver with Burst Mode (SMF, 1577nmTx/1270nmRx, SC, N1, DOM). https://www.addonnetworks.com/products/transceivers/calix/xfp/10gbase /100-04482-ao. ([n. d.])

  8. [8]

    [n. d.]. Context parallelism overview. https://docs.nvidia.com/megatron- core/developer-guide/latest/api-guide/context_parallel.html. ([n. d.])

Show all 125 references
  1. [9]

    [n. d.]. DeepSeek-V3. https://github.com/deepseek-ai/DeepSeek-V3/blob/mai n/DeepSeek_V3.pdf. ([n. d.])

  2. [10]

    [n. d.]. DeepSpeed. https://openai.com/index/gpt-3-apps/. ([n. d.])

  3. [11]

    [n. d.]. Doubling all2all Performance with NVIDIA Collective Communication Library 2.12. https://developer.nvidia.com/blog/doubling-all2all-performance- with-nvidia-collective-communication-library-2-12/. ([n. d.])

  4. [12]

    [n. d.]. FlexFlow. https://github.com/flexflow/FlexFlow. ([n. d.])

  5. [13]

    [n. d.]. Generic Compatible 800GBASE-SR8 QSFP-DD PAM4 850nm 50m DOM MPO-16/APC MMF Optical Transceiver Module. https://www.fs.com/product s/200921.html?attribute=93760&id=3569240. ([n. d.])

  6. [14]

    [n. d.]. Grok. https://x.ai/blog/grok-1.5v. ([n. d.])

  7. [15]

    [n. d.]. Grok training cluster. https://www.windowscentral.com/software- apps/elon-musk-flaunts-the-most-powerful-training-cluster-in-the-world- that-will-transform-grok-into-the-most-powerful-ai-by-december-to-take- on-microsoft-and-openai. ([n. d.])

  8. [16]

    [n. d.]. htsim simulator. https://github.com/nets-cs-pub-ro/NDP/wiki/NDP- Simulator. ([n. d.])

  9. [17]

    [n. d.]. IPRONICS One. https://ipronics.com/ipronics-optical-networking- engine/. ([n. d.])

  10. [18]

    [n. d.]. LC UPC to LC UPC, Duplex, 2 Fibers. https://www.fs.com/products/ 40191.html. ([n. d.])

  11. [19]

    [n. d.]. LightMatter. https://lightmatter.co/products/passage/. ([n. d.])

  12. [20]

    [n. d.]. LLaMA. https://llama.meta.com. ([n. d.])

  13. [21]

    [n. d.]. Mixtral 8x22B. https://huggingface.co/mistralai/Mixtral-8x22B- Instruct-v0.1. ([n. d.])

  14. [22]

    [n. d.]. Mixtral-8x7B-Instruct-v0.1. https://huggingface.co/mistralai/Mixtral- 8x7B-Instruct-v0.1. ([n. d.])

  15. [23]

    [n. d.]. Mixtral of experts. https://mistral.ai/news/mixtral-of-experts/. ([n. d.])

  16. [24]

    [n. d.]. MSN4700-WS2FC, NVIDIA ® Mellanox Spectrum-3 Based 32-Port Ether- net L3 Data Center Switch, 32 x 400Gb QSFP-DD. https://www.colfaxdirect.c om/store/pc/viewPrd.asp?idproduct=4135. ([n. d.])

  17. [25]

    [n. d.]. Nano-Second Speed PLZT Photonics. http://epiphotonics.com/product s.html. ([n. d.])

  18. [26]

    [n. d.]. NVIDIA Collective Communications Library (NCCL). https://developer. nvidia.com/nccl. ([n. d.])

  19. [27]

    [n. d.]. NVIDIA DGX SuperPOD. https://www.nvidia.com/en-us/data- center/dgx-superpod/. ([n. d.])

  20. [28]

    [n. d.]. NVIDIA GB200 NVL72 Delivers Trillion-Parameter LLM Training and Real-Time Inference. https://developer.nvidia.com/blog/nvidia-gb200-nvl72- delivers-trillion-parameter-llm-training-and-real-time-inference/. ([n. d.])

  21. [29]

    [n. d.]. NVIDIA Mellanox MCX515A-CCAT ConnectX®-5 EN Network Interface Card, 100GbE Single-Port QSFP28. https://www.fs.com/products/119648.html. ([n. d.])

  22. [30]

    [n. d.]. NVIDIA Mellanox MCX653105A-HDAT ConnectX®-6 InfiniBand/VPI Adapter Card 200GbE/HDR, Single-Port QSFP56. https://www.fs.com/product s/168437.html. ([n. d.])

  23. [31]

    [n. d.]. NVIDIA Mellanox MCX75310AAS-NEAT ConnectX®-7 InfiniBand/VPI Adapter Card 400GbE/NDR, Single-Port OSFP. https://www.fs.com/products/ 212161.html. ([n. d.])

  24. [32]

    [n. d.]. NVIDIA Mellanox Spectrum-4 SN5600 800G 64-Port 51.2Tb/s 2U Data Center Switch. https://fireowls.com/products/920-9n42f-00ri-7c0-nvidia- spectrum-sn5600-ethernet-switch?srsltid=AfmBOoqC5-OQefz1MpAi_2Qkz W2tnaCTXA_xGtgoj9b3NphWUXw8dBrl. ([n. d.])

  25. [33]

    [n. d.]. NVIDIA NVLink and NVLink Switch. https://www.nvidia.com/en- us/data-center/nvlink/. ([n. d.])

  26. [34]

    [n. d.]. NVIDIA NVSwitch Technical Overview. https://images.nvidia.com/co ntent/pdf/nvswitch-technical-overview.pdf. ([n. d.])

  27. [35]

    [n. d.]. NVIDIA Spectrum SN3700. https://marketplace.nvidia.com/en- us/enterprise/networking/sn3700/. ([n. d.])

  28. [36]

    [n. d.]. NVLink Switch Specifications. https://www.nvidia.com/en-us/data- center/nvlink/?ncid=no-ncid#nvlink-switch-specifications. ([n. d.])

  29. [37]

    [n. d.]. Polatis Optical Circuit Switch. https://www.polatis.com/series-7000- 384x384-port-software-controlled-optical-circuitswitch-sdn-enabled.asp. ([n. d.])

  30. [38]

    [n. d.]. Polatis Optical Switches. http://www.polatis.com/series-7000-384x384- port-software-controlled-optical-circuit-switch-sdn-enabled.asp. ([n. d.])

  31. [39]

    [n. d.]. PON/Burst-mode Transceivers. https://oesolutions.com/product- type/burst-mode-transceivers/. ([n. d.])

  32. [40]

    [n. d.]. Qwen1.5-MoE-A2.7B. https://huggingface.co/Qwen/Qwen1.5-MoE- A2.7B. ([n. d.])

  33. [41]

    [n. d.]. Rethinking AI Architectures with Optical I/O. https://www.windowsc entral.com/software-apps/elon-musk-flaunts-the-most-powerful-training- cluster-in-the-world-that-will-transform-grok-into-the-most-powerful-ai- by-december-to-take-on-microsoft-and-openai. ([n. d.])

  34. [42]

    [n. d.]. Telescent G4 Network Topology Manager. https://www.telescent.com/ products. ([n. d.])

  35. [43]

    [n. d.]. XVERSE-MoE-A36B MoE base model. https://huggingface.co/xverse/ XVERSE-MoE-A36B. ([n. d.])

  36. [44]

    Mohammad Al-Fares, Alexander Loukissas, and Amin Vahdat. 2008. A scalable, commodity data center network architecture. In ACM SIGCOMM Computer Communication Review, Vol. 38. ACM New York, NY, USA, 63–74

    2008

  37. [45]

    Daniel Amir, Nitika Saran, Tegan Wilson, Robert Kleinberg, Vishal Shrivastav, and Hakim Weatherspoon. 2024. Shale: A Practical, Scalable Oblivious Recon- figurable Network. In Proceedings of the ACM SIGCOMM 2024 Conference (ACM SIGCOMM ’24). https://doi.org/10.1145/3651890.3672248

    2024

  38. [46]

    Daniel Amir, Tegan Wilson, Vishal Shrivastav, Hakim Weatherspoon, and Robert Kleinberg. 2023. Poster: Scalability and congestion control in oblivious re- configurable networks. In Proceedings of the ACM SIGCOMM 2023 Conference . 1138–1140

    2023

  39. [47]

    Daniel Amir, Tegan Wilson, Vishal Shrivastav, Hakim Weatherspoon, Robert Kleinberg, and Rachit Agarwal. 2022. Optimal oblivious reconfigurable net- works. In Proceedings of the 54th Annual ACM SIGACT Symposium on Theory of Computing. 1339–1352

    2022

  40. [48]

    Hitesh Ballani, Paolo Costa, Raphael Behrendt, Daniel Cletheroe, Istvan Haller, Krzysztof Jozwik, Fotini Karinou, Sophie Lange, Kai Shi, Benn Thomsen, et al

  41. [49]

    Darius Bunandar, Shashank Gupta, Jessie Rosenberg, Clifford Chao, Kuang Liu, and Nicholas C Harris. 2024. Optical communication substrate using glass interposer. (Oct. 24 2024). US Patent App. 18/638,820

    2024

  42. [50]

    Kai Chen, Ankit Singla, Atul Singh, Kishore Ramachandran, Lei Xu, Yueping Zhang, Xitao Wen, and Yan Chen. 2012. OSA: An Optical Switching Architec- ture for Data Center Networks with Unprecedented Flexibility. In 9th USENIX Symposium on Networked Systems Design and Implementat...

    2012

  43. [51]

    Li Chen, Kai Chen, Zhonghua Zhu, Minlan Yu, George Porter, Chunming Qiao, and Shan Zhong. 2017. Enabling Wide-Spread Communications on Optical Fabric with MegaSwitch. In 14th USENIX Symposium on Networked Systems Design and Implementation (NSDI 17) . USENIX Association, Boston...

    2017

  44. [52]

    Tianyu Chen, Shaohan Huang, Yuan Xie, Binxing Jiao, Daxin Jiang, Haoyi Zhou, Jianxin Li, and Furu Wei. 2022. Task-specific expert pruning for sparse mixture-of-experts. arXiv preprint arXiv:2206.00277 (2022)

    2022 arXiv

  45. [53]

    Kari Clark, Hitesh Ballani, Polina Bayvel, Daniel Cletheroe, Thomas Gerard, Istvan Haller, Krzysztof Jozwik, Kai Shi, Benn Thomsen, Philip Watts, et al. 2018. Sub-nanosecond clock and data recovery in an optically-switched data centre network. In 2018 European Conference on Op...

    2018

  46. [54]

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei F...

    2025 arXiv

  47. [55]

    Zhang, Hanwei Xu, Hao Yang, Haowei Zhang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Li, Hui Qu, J

    DeepSeek-AI, Aixin Liu, Bei Feng, Bin Wang, Bingxuan Wang, Bo Liu, Cheng- gang Zhao, Chengqi Dengr, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Hanwei Xu, Hao Yang, Haow...

    2024 arXiv

  48. [56]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The Llama 3 Herd of Models. arXiv preprint arXiv:2407.21783 (2024)

    2024 arXiv

  49. [57]

    Nathan Farrington, Alex Forencich, George Porter, P-C Sun, Joseph E Ford, Yeshaiahu Fainman, George C Papen, and Amin Vahdat. 2013. A multiport microsecond optical circuit switch for data center networking. IEEE Photonics Technology Letters 25, 16 (2013), 1589–1592

    2013

  50. [58]

    Nathan Farrington, George Porter, Sivasankar Radhakrishnan, Hamid Ha- jabdolali Bazzaz, Vikram Subramanya, Yeshaiahu Fainman, George Papen, and Amin Vahdat. 2010. Helios: a hybrid electrical/optical switch architec- ture for modular data centers. In Proceedings of the ACM SIGC...

    2010

  51. [59]

    William Fedus, Barret Zoph, and Noam Shazeer. 2022. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. Journal of Machine Learning Research 23, 120 (2022), 1–39

    2022

  52. [60]

    Klaus-Tycho Foerster, Manya Ghobadi, and Stefan Schmid. 2018. Characteriz- ing the algorithmic complexity of reconfigurable data center architectures. In Proceedings of the 2018 Symposium on Architectures for Networking and Commu- nications Systems. 89–96

    2018

  53. [61]

    Adithya Gangidi, Rui Miao, Shengbao Zheng, Sai Jayesh Bondu, Guilherme Goes, Hany Morsy, Rohit Puri, Mohammad Riftadi, Ashmitha Jeevaraj Shetty, Jingyi Yang, et al. 2024. RDMA over Ethernet for Distributed Training at Meta Scale. In Proceedings of the ACM SIGCOMM 2024 Conferen...

    2024

  54. [62]

    Thomas Gerard, Kari Clark, Adam Funnell, Kai Shi, Benn Thomsen, Philip Watts, Krzysztof Jozwik, Istvan Haller, Hugh Williams, Paolo Costa, et al. 2021. Fast and uniform optically-switched data centre networks enabled by amplitude caching. In 2021 Optical Fiber Communications C...

    2021

  55. [63]

    Dan Gibson, Hema Hariharan, Eric Lance, Moray McLaren, Behnam Montazeri, Arjun Singh, Stephen Wang, Hassan MG Wassel, Zhehua Wu, Sunghwan Yoo, et al. 2022. Aquila: A unified, low-latency fabric for datacenter networks. In19th USENIX Symposium on Networked Systems Design and Im...

    2022

  56. [64]

    Albert Greenberg, James R Hamilton, Navendu Jain, Srikanth Kandula, Changhoon Kim, Parantap Lahiri, David A Maltz, Parveen Patel, Sushant Sen- gupta, Jennifer Rexford, et al . 2009. VL2: A scalable and flexible data center network. In ACM SIGCOMM Computer Communication Review ...

    2009

  57. [65]

    Chuanxiong Guo, Hui Wu, Kai Tan, Lei Shiy, Yongguang Zhang, and Songnian Lu. 2009. Bcube: A high performance, server-centric network architecture for modular data centers. In ACM SIGCOMM Computer Communication Review , Vol. 39. ACM New York, NY, USA, 63–74

    2009

  58. [66]

    Chuanxiong Guo, Hui Wu, Kai Tan, Lei Shiy, Yongguang Zhang, and Songnian Lu. 2010. Dcell: A scalable and fault-tolerant network structure for data centers. In ACM SIGCOMM Computer Communication Review , Vol. 40. ACM New York, NY, USA, 63–74

    2010

  59. [67]

    Navid Hamedazimi, Zafar Qazi, Himanshu Gupta, Vyas Sekar, Samir R Das, Jon P Longtin, Himanshu Shah, and Ashish Tanwer. 2014. Firefly: A reconfigurable wireless data center fabric using free-space optics. In Proceedings of the 2014 ACM conference on SIGCOMM. 319–330

    2014

  60. [68]

    Sushant Jain, Alok Kumar, Subhasree Mandal, Joon Ong, Leon Poutievski, Arjun Singh, Subbaiah Venkata, Jim Wanderer, Junlan Zhou, Min Zhu, Jon Zolla, Urs Hölzle, Stephen Stuart, and Amin Vahdat. 2013. B4: experience with a globally- deployed software defined wan. In Proceedings...

    2013

  61. [69]

    Yimin Jiang, Yibo Zhu, Chang Lan, Bairen Yi, Yong Cui, and Chuanxiong Guo

  62. [70]

    Ziheng Jiang, Haibin Lin, Yinmin Zhong, Qi Huang, Yangrui Chen, Zhi Zhang, Yanghua Peng, Xiang Li, Cong Xie, Shibiao Nong, et al . 2024. {MegaScale}: Scaling large language model training to more than 10,000 {GPUs}. In 21st USENIX Symposium on Networked Systems Design and Impl...

    2024

  63. [71]

    Norm Jouppi, George Kurian, Sheng Li, Peter Ma, Rahul Nagarajan, Lifeng Nai, Nishant Patil, Suvinay Subramanian, Andy Swing, Brian Towles, et al . 2023. Tpu v4: An optically reconfigurable supercomputer for machine learning with hardware support for embeddings. InProceedings o...

    2023

  64. [72]

    In 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI 20)

    A unified architecture for accelerating distributed {DNN} training in heterogeneous{GPU/CPU} clusters. In 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI 20) . 463–479

  65. [73]

    Abhishek Vijaya Kumar, Arjun Devraj, Darius Bunandar, and Rachee Singh

  66. [74]

    Dmitry Lepikhin, HyoukJoong Lee, Yuanzhong Xu, Dehao Chen, Orhan Firat, Yanping Huang, Maxim Krikun, Noam Shazeer, and Zhifeng Chen. 2020. Gshard: Scaling giant models with conditional computation and automatic sharding. arXiv preprint arXiv:2006.16668 (2020)

    2020 arXiv

  67. [75]

    Mehrdad Khani, Manya Ghobadi, Mohammad Alizadeh, Ziyi Zhu, Madeleine Glick, Keren Bergman, Amin Vahdat, Benjamin Klenk, and Eiman Ebrahimi

  68. [76]

    Shen Li, Yanli Zhao, Rohan Varma, Omkar Salpekar, Pieter Noordhuis, Teng Li, Adam Paszke, Jeff Smith, Brian Vaughan, Pritam Damania, et al. 2020. Pytorch distributed: Experiences on accelerating data parallel training. arXiv preprint arXiv:2006.15704 (2020)

    2020 arXiv

  69. [77]

    Wenxue Li, Xiangzhou Liu, Yuxuan Li, Yilun Jin, Han Tian, Zhizhen Zhong, Guyue Liu, Ying Zhang, and Kai Chen. 2024. Understanding communication characteristics of distributed training. In Proceedings of the 8th Asia-Pacific Workshop on Networking. 1–8

    2024

  70. [78]

    Bin Lin, Zhenyu Tang, Yang Ye, Jiaxi Cui, Bin Zhu, Peng Jin, Junwu Zhang, Munan Ning, and Li Yuan. 2024. Moe-llava: Mixture of experts for large vision- language models. arXiv preprint arXiv:2401.15947 (2024)

    2024 arXiv

  71. [79]

    Voelker, George Papen, Alex C

    He Liu, Feng Lu, Alex Forencich, Rishi Kapoor, Malveeka Tewari, Geoffrey M. Voelker, George Papen, Alex C. Snoeren, and George Porter. 2014. Circuit Switching Under the Radar with REACToR. In 11th USENIX Symposium on Networked Systems Design and Implementation (NSDI 14) . USEN...

    2014

  72. [80]

    Pingzhi Li, Zhenyu Zhang, Prateek Yadav, Yi-Lin Sung, Yu Cheng, Mohit Bansal, and Tianlong Chen. 2023. Merge, then compress: Demystify efficient SMoe with hints from its routing policy. arXiv preprint arXiv:2310.01334 (2023)

    2023 arXiv

  73. [81]

    Hong Liu, Ryohei Urata, Kevin Yasumura, Xiang Zhou, Roy Bannon, Jill Berger, Pedram Dashti, Norm Jouppi, Cedric Lam, Sheng Li, Erji Mao, Daniel Nelson, George Papen, Mukarram Tariq, and Amin Vahdat. 2023. Lightwave Fabrics: At- Scale Optical Circuit Switching for Datacenter an...

    2023

  74. [82]

    Juncai Liu, Jessie Hui Wang, and Yimin Jiang. 2023. Janus: A unified distributed training framework for sparse mixture-of-experts models. In Proceedings of the ACM SIGCOMM 2023 Conference. 486–498

    2023

  75. [83]

    Xudong Lu, Qi Liu, Yuhui Xu, Aojun Zhou, Siyuan Huang, Bo Zhang, Junchi Yan, and Hongsheng Li. 2024. Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models. arXiv preprint arXiv:2402.14800 (2024)

    2024 arXiv

  76. [84]

    William M Mellette, Rajdeep Das, Yibo Guo, Rob McGuinness, Alex C Snoeren, and George Porter. 2020. Expanding across time to deliver bandwidth efficiency and low latency. In 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI 20). 1–18

    2020

  77. [85]

    He Liu, Matthew K Mukerjee, Conglong Li, Nicolas Feltman, George Papen, Ste- fan Savage, Srinivasan Seshan, Geoffrey M Voelker, David G Andersen, Michael Kaminsky, et al. 2015. Scheduling techniques for hybrid circuit/packet networks. In Proceedings of the 11th ACM Conference ...

    2015

  78. [86]

    William M Mellette, Rob McGuinness, Arjun Roy, Alex Forencich, George Papen, Alex C Snoeren, and George Porter. 2017. Rotornet: A scalable, low-complexity, optical datacenter network. In Proceedings of the Conference of the ACM Special Interest Group on Data Communication . 267–280

    2017

  79. [87]

    William Maxwell Mellette, Glenn M Schuster, George Porter, George Papen, and Joseph E Ford. 2016. A scalable, partially configurable optical switch for data center networks. Journal of Lightwave Technology 35, 2 (2016), 136–144

    2016

  80. [88]

    Deepak Narayanan, Aaron Harlap, Amar Phanishayee, Vivek Seshadri, Nikhil R Devanur, Gregory R Ganger, Phillip B Gibbons, and Matei Zaharia. 2019. PipeDream: Generalized pipeline parallelism for DNN training. In Proceedings of the 27th ACM symposium on operating systems princip...

    2019

  81. [89]

    George Porter, Richard Strong, Nathan Farrington, Alex Forencich, Pang Chen- Sun, Tajana Rosing, Yeshaiahu Fainman, George Papen, and Amin Vahdat. 2013. Integrating microsecond circuit switching into the data center. ACM SIGCOMM Computer Communication Review 43, 4 (2013), 447–458

    2013

  82. [90]

    William M Mellette, Alex Forencich, Rukshani Athapathu, Alex C Snoeren, George Papen, and George Porter. 2024. Realizing RotorNet: Toward Practical Microsecond Scale Optical Networking. In Proceedings of the ACM SIGCOMM 2024 Conference. 392–414

    2024

  83. [91]

    Kun Qian, Yongqing Xi, Jiamin Cao, Jiaqi Gao, Yichi Xu, Yu Guan, Binzhang Fu, Xuemei Shi, Fangbo Zhu, Rui Miao, Chao Wang, Peng Wang, Pengcheng Zhang, Xianlong Zeng, Eddie Ruan, Zhiping Yao, Ennan Zhai, and Dennis Cai. 2024. Alibaba HPN: A Data Center Network for Large Languag...

    2024

  84. [92]

    Arslan Sajid Raja, Sophie Lange, Maxim Karpov, Kai Shi, Xin Fu, Raphael Behrendt, Daniel Cletheroe, Anton Lukashchuk, Istvan Haller, Fotini Karinou, Benn Thomsen, Krzysztof Jozwik, Junqiu Liu, Paolo Costa, Tobias Jan Kippen- berg, and Hitesh Ballani. 2021. Ultrafast optical ci...

    2021 doi

  85. [93]

    Zhenghang Ren, Yuxuan Li, Zilong Wang, Xinyang Huang, Wenxue Li, Kaiqiang Xu, Xudong Liao, Yijun Sun, Bowen Liu, Han Tian, Junxue Zhang, Mingfei Wang, Zhizhen Zhong, Guyue Liu, Ying Zhang, and Kai Chen. 2025. Enabling Effi- cient GPU Communication over Multiple NICs with FuseL...

    2025

  86. [94]

    R Ryf, J Kim, JP Hickey, A Gnauck, D Carr, F Pardo, C Bolle, R Frahm, N Basavan- hally, C Yoh, et al. 2001. 1296-port MEMS transparent optical crossconnect with 2.07 petabit/s switch capacity. In OFC 2001. Optical Fiber Communication Confer- ence and Exhibit. Technical Digest ...

    2001

  87. [95]

    Leon Poutievski, Omid Mashayekhi, Joon Ong, Arjun Singh, Mukarram Tariq, Rui Wang, Jianan Zhang, Virginia Beauregard, Patrick Conner, Steve Gribble, Rishi Kapoor, Stephen Kratzer, Nanfang Li, Hong Liu, Karthik Nagaraj, Jason Ornstein, Samir Sawhney, Ryohei Urata, Lorenzo Vicis...

    2022

  88. [96]

    Tae Joon Seok, Niels Quack, Sangyoon Han, Richard S Muller, and Ming C Wu

  89. [97]

    Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. 2017. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. arXiv preprint arXiv:1701.06538 (2017)

    2017 arXiv

  90. [98]

    Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catanzaro. 2020. Megatron-LM: Training Multi-Billion Param- eter Language Models Using Model Parallelism. (2020). arXiv:cs.CL/1909.08053 https://arxiv.org/abs/1909.08053

    2020 arXiv

  91. [99]

    Chenchen Shou, Guyue Liu, Hao Nie, Huaiyu Meng, Yu Zhou, Yimin Jiang, Wenqing Lv, Yelong Xu, Yuanwei Lu, Zhang Chen, Yanbo Yu, Yichen Shen, Yibo Zhu, and Daxin Jiang. 2025. InfiniteHBD: Building Datacenter-Scale High- Bandwidth Domain for LLM with Optical Circuit Switching Tra...

    2025 arXiv

  92. [100]

    Nitika Saran, Daniel Amir, Tegan Wilson, Robert Kleinberg, Vishal Shrivastav, and Hakim Weatherspoon. 2024. Semi-Oblivious Reconfigurable Datacenter Networks. In Proceedings of the 23rd ACM Workshop on Hot Topics in Networks . 150–158

    2024

  93. [101]

    Arjun Singh, Joon Ong, Amit Agarwal, Glen Anderson, Ashby Armistead, Roy Bannon, Seb Boving, Gaurav Desai, Bob Felderman, Paulie Germano, Anand Kanagala, Jeff Provost, Jason Simmons, Eiichi Tanda, Jim Wanderer, Urs Hölzle, Stephen Stuart, and Amin Vahdat. 2015. Jupiter Rising:...

    2015

  94. [102]

    Sainbayar Sukhbaatar, Olga Golovneva, Vasu Sharma, Hu Xu, Xi Victoria Lin, Baptiste Rozière, Jacob Kahn, Daniel Li, Wen-tau Yih, Jason Weston, et al. 2024. Branch-Train-MiX: Mixing Expert LLMs into a Mixture-of-Experts LLM. arXiv preprint arXiv:2403.07816 (2024)

    2024 arXiv

  95. [103]

    Qwen Team. 2024. Qwen2.5 technical report. arXiv preprint arXiv:2412.15115 (2024)

    2024 arXiv

  96. [104]

    Xinchen Wan, Hong Zhang, Hao Wang, Shuihai Hu, Junxue Zhang, and Kai Chen. 2020. Rat-resilient allreduce tree for distributed machine learning. In Proceedings of the 4th Asia-Pacific Workshop on Networking . 52–57

    2020

  97. [105]

    Guohui Wang, David G Andersen, Michael Kaminsky, Konstantina Papagiannaki, TS Eugene Ng, Michael Kozuch, and Michael Ryan. 2010. c-Through: Part-time optics in data centers. In Proceedings of the ACM SIGCOMM 2010 Conference . 327–338

    2010

  98. [106]

    Vishal Shrivastav, Asaf Valadarsky, Hitesh Ballani, Paolo Costa, Ki Suh Lee, Han Wang, Rachit Agarwal, and Hakim Weatherspoon. 2019. Shoal: A network architecture for disaggregated racks. In 16th USENIX Symposium on Networked Systems Design and Implementation (NSDI 19) . 255–270

    2019

  99. [107]

    Weiyang Wang, Moein Khazraee, Zhizhen Zhong, Manya Ghobadi, Zhihao Jia, Dheevatsa Mudigere, Ying Zhang, and Anthony Kewitsch. 2023. TopoOpt: Co-optimizing Network Topology and Parallelization Strategy for Distributed Training Jobs. In 20th USENIX Symposium on Networked Systems...

    2023

  100. [108]

    Tegan Wilson, Daniel Amir, Nitika Saran, Robert Kleinberg, Vishal Shrivastav, and Hakim Weatherspoon. 2024. Breaking the VLB Barrier for Oblivious Re- configurable Networks. In Proceedings of the 56th Annual ACM Symposium on Theory of Computing. 1865–1876

    2024

  101. [109]

    Tegan Wilson, Daniel Amir, Vishal Shrivastav, Hakim Weatherspoon, and Robert Kleinberg. 2023. Extending optimal oblivious reconfigurable networks to all n. In 2023 Symposium on Algorithmic Principles of Computer Systems (APOCS) . SIAM, 1–16

    2023

  102. [110]

    Yiting Xia, Mike Schlansker, TS Eugene Ng, and Jean Tourrilhes. 2015. Enabling Topological Flexibility for Data Centers Using{OmniSwitch}. In 7th USENIX Workshop on Hot Topics in Cloud Computing (HotCloud 15)

    2015

  103. [111]

    Kaiqiang Xu, Decang Sun, Hao Wang, Zhenghang Ren, Xinchen Wan, Xudong Liao, Zilong Wang, Junxue Zhang, and Kai Chen. 2025. Design and Operation of Shared Machine Learning Clusters on Campus. In Proceedings of the 30th ACM International Conference on Architectural Support for P...

    2025 doi

  104. [112]

    Weiyang Wang, Manya Ghobadi, Kayvon Shakeri, Ying Zhang, and Naader Hasani. 2024. Rail-only: A Low-Cost High-Performance Network for Training LLMs with Trillion Parameters. (2024). arXiv:cs.NI/2307.12169 https://arxiv.or g/abs/2307.12169

    2024 arXiv

  105. [113]

    Yufang Yu, Nan Hua, Zhizhen Zhong, Jialong Li, Ruijie Luo, Zelin Zheng, and Xiaoping Zheng. 2017. Fast-Reconfigurable Optical Interconnect Architecture Based on Time-Synchronized Node Coordination for High Performance Comput- ing, In Asia Communications and Photonics Conferenc...

    2017 doi

  106. [114]

    Mingyang Zhang, Jianan Zhang, Rui Wang, Ramesh Govindan, Jeffrey C Mogul, and Amin Vahdat. 2021. Gemini: Practical reconfigurable datacenter networks with topology and traffic engineering. arXiv preprint arXiv:2110.08374 (2021)

    2021 arXiv

  107. [115]

    Shicheng Zhang, Xuwei Xue, Bingli Guo, Yixuan Li, Wenzhe Li, Shikui Shen, Haoze Qian, Xiaojie Yin, Buzheng Wei, Guojun Yuan, et al. 2024. Fast-tunable Graphene-based AWGR for Deep Learning Training Networks. InProceedings of the 1st SIGCOMM Workshop on Hot Topics in Optical Te...

    2024

  108. [116]

    Xiaosheng Zhang, Ming Chiang A Wu, Andrew S Michaels, and Johannes Henriksson. 2022. Beam-steering system based on a MEMS-actuated vertical- coupler array. (Sept. 13 2022). US Patent 11,441,353. MixNet: A Runtime Reconfigurable Optical-Electrical Fabric SIGCOMM ’25, September ...

    2022

  109. [117]

    Tong Zhu, Xiaoye Qu, Daize Dong, Jiacheng Ruan, Jingqi Tong, Conghui He, and Yu Cheng. 2024. LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-training. arXiv preprint arXiv:2406.16554 (2024). https: //arxiv.org/abs/2406.16554

    2024 arXiv

  110. [118]

    Xiaohui Ye, SJ Ben Yoo, and Venkatesh Akella. 2012. AWGR-based optical topologies for scalable and efficient global communications in large-scale multi- processor systems. Journal of Optical Communications and Networking 4, 9 (2012), 651–662

    2012

  111. [124]

    Yazhou Zu, Alireza Ghaffarkhah, Hoang-Vu Dang, Brian Towles, Steven Hand, Safeen Huda, Adekunle Bello, Alexander Kolbasov, Arash Rezaei, Dayou Du, et al

  112. [125]

    In 21st USENIX Symposium on Networked Systems Design and Implementation (NSDI 24)

    Resiliency at Scale: Managing {Google’s}{ TPUv4} Machine Learning Supercomputer. In 21st USENIX Symposium on Networked Systems Design and Implementation (NSDI 24). 761–774. Appendix Appendices are supporting material that has not been peer-reviewed. A Production Measurement De...

    2025

  113. [593]

    https://www.usenix.org/conference/nsdi17/technical-sessions/presentatio n/chen

  114. [2016]

    Optica 3, 1 (2016), 64–70

    Large-scale broadband digital silicon photonic switches with vertical adiabatic couplers. Optica 3, 1 (2016), 64–70

    2016

  115. [2020]

    Sirius: A flat datacenter network with nanosecond optical switching. In Proceedings of the Annual conference of the ACM Special Interest Group on Data Communication on the applications, technologies, architectures, and protocols for computer communication. 782–797

  116. [2021]

    In Proceedings of the 2021 ACM SIGCOMM 2021 Conference

    SiP-ML: high-bandwidth optical network interconnects for machine learning training. In Proceedings of the 2021 ACM SIGCOMM 2021 Conference . 657–675

    2021

  117. [2024]

    In Proceedings of the 23rd ACM Workshop on Hot Topics in Networks

    A case for server-scale photonic connectivity. In Proceedings of the 23rd ACM Workshop on Hot Topics in Networks . 290–299

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.