Pith. sign in

REVIEW 4 major objections 5 minor 44 references

MoX: Efficient MoE Routing on Direct-Connect Topologies

T0 review · 4 major / 5 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read MoX claims that offline-optimized static routing can make direct-connect networks nearly match switched performance for mixture-of-experts communication.

desk verdict Static, load-oblivious MoE routing backed by solid simulation and an honest proxy validation; the unmeasured per-token forwarding cost is the one assumption that could break the near-switch claim. read the letter →

arxiv 2607.20220 v1 pith:CI3WGCPZ submitted 2026-07-22 cs.NI

Ori Cohen , Jakob Krebs , Daniel Amir , Mark Silberstein This is my paper · ORCID
classification cs.NI
keywords Mixture-of-Expertsdirect-connecttopologymulticastroutingloadbalancingtrafficengineeringopticalnetworksASTRA-simtoken
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that high-performance Mixture-of-Experts (MoE) communication does not require a network that reconfigures itself to each batch's traffic. It presents MoX, which routes tokens over fixed direct-connect topologies using token-aware multicast trees to avoid redundant forwarding, and per-link weights computed offline to balance load without predicting demand. On recorded traffic from large MoE models, MoX accelerates the full MoE block, including dispatch, expert compute, and combine, by up to 1.8x over min-hop routing and comes within 0.5-15% of an ideal packet switch on 16-64 node expanders. It also cuts the bottleneck link load of Google's Boardfly topology by up to 47%. If right, MoE training and inference can run on static optical direct-connect fabrics, avoiding expensive dynamic topology reconfiguration.

What carries the argument

The key machinery is token-aware multicast trees plus per-link weights. MoX builds a directed tree from source to the K expert-hosting accelerators, preferring paths whose intermediate nodes are also destinations to minimize relay 'tax' hops. When several holders can reach an unreached destination, a link weight selects the relay (realized by weighted round robin). Weights are computed offline via Monte Carlo sampling of uniform top-K destination sets and a multiplicative min-max update, serving as a compact surrogate for fractional multicast-tree packing—one weight per directed link, no per-token state. Combine reverses the trees and partially reduces outputs at intermediate nodes.

What would settle it

Measure the per-token forwarding and reduction cost on a real GPU/NIC (e.g., with the destination bitmap of 8 bytes per packet) and replay the DeepSeek-V3 trace end-to-end; if the MoE-block time does not improve by the predicted margin over min-hop routing—or if link utilization is bounded by packet processing rather than bytes—the central claim would be refuted.

Watch

Extended reading notes

Core claim

The central claim is that efficient indirect routing, designed around MoE's per-token multicast semantics and load-balanced offline, lets a static direct-connect network approach switched-network performance. MoX builds a multicast tree for each token's top-K expert destinations, preferring paths whose intermediate nodes are themselves destinations to minimize 'bandwidth tax' relay hops, then resolves ties among equal-tax hops with link weights optimized by a compact surrogate for fractional multicast-tree packing. On recorded DeepSeek-V3 and Qwen-3 MoE traffic simulated in ASTRA-sim, MoX is within 0.5-15% of an ideal switch on 16-64 node degree-8 random expanders for training, and within 2.

Load-bearing premise

The speedups assume per-token multicast forwarding and GPU-side partial reduction add negligible overhead; if per-packet NIC or GPU processing is not cheap relative to the large token transfers, the simulated gains shrink or vanish.

Editorial extensions

If this is right

  • MoE training and inference can run on static direct-connect optical fabrics without per-batch topology reconfiguration.
  • The routing policy is tiny: one weight per directed link (512 weights for a 64-node degree-8 expander), so it can be stored and applied with minimal state.
  • Larger top-K values improve MoX performance because more destinations can act as relays, turning some relay hops into useful deliveries.
  • The maximum-link-load proxy correlates with end-to-end MoE-block time, so offline analytical models can guide topology and degree choices.
  • Higher node degree in expanders yields bigger gains; maintaining d^2/N = 2 keeps performance near-switch, suggesting design guidance for future fabrics.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Since MoX is load-oblivious and assumes a uniform prior over expert combinations, it may be most beneficial when expert load is balanced by auxiliary losses; if real workloads develop persistent per-endpoint hotspots, a small amount of runtime feedback—without full reconfiguration—could extend MoX.
  • The per-token forwarding and GPU-side partial reduction could be implemented in the NIC or GPU; a prototype on real hardware is needed to confirm that per-packet processing does not erode the simulated speedups.
  • The same tree-packing surrogate could apply to other sparse, multicast-like communication patterns (e.g., sparse attention or disaggregated inference), not just MoE.
  • If optical circuit switches remain too slow for per-collective reconfiguration, MoX-type static routing may become the default operating mode, with reconfiguration reserved for coarse traffic phases.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes MoX, an offline, demand-oblivious routing scheme for MoE dispatch and combine on static direct-connect topologies. MoX first builds token-aware multicast trees that use expert-hosting accelerators as relays, then computes one weight per directed link by replaying Monte-Carlo samples of uniform top-K expert combinations with a multiplicative min-max update. At runtime, sources realize the weights via weighted round robin; combine reverses the trees with GPU-side partial reduction. The evaluation uses recorded DeepSeek-V3 and Qwen-3 MoE token-level traces in ASTRA-sim 2 on degree-8 expanders of 16/32/64 nodes and a projected 1,024-TPU Boardfly model. The paper reports up to 1.8x full MoE-block speedup over min-hop routing, near-switch performance (within 0.5-15% depending on size and top-k), and up to 47% reduction of the Boardfly dispatch bottleneck link load. The central claim is that static, offline-computed routing can make direct-connect fabrics competitive with an ideal packet switch for MoE traffic, without dynamic topology reconfiguration.

Significance. If the results hold, MoX is a significant result for large-scale MoE training and inference: it would show that a static direct-connect fabric with multicast-aware, load-balanced routing can approximate switch-level performance, potentially avoiding the cost and complexity of dynamic optical reconfiguration. The paper's strengths are its use of recorded real-model traces, its validation of an analytical min-max-link-load proxy against ASTRA-sim end-to-end times (Fig. 3), its compact routing state (one weight per directed link), and its comparison against a MixNet-style demand-optimized topology. The paper is honest about several limitations, notably that the Boardfly result uses a proxy rather than full simulation and that per-token forwarding overhead is asserted rather than measured. However, the latter assumption is load-bearing for the headline speedups, and the absence of variance reporting makes the fine-grained 'within 0.5%' claims difficult to evaluate.

major comments (4)
  1. [Section 4, 'Implementation'] The central speedup claim is conditional on an unmeasured implementation assumption. The paper states: 'We believe that this can be implemented with low overhead due to the large size of each individual token.' ASTRA-sim models link serialization/transfer times, but not per-packet NIC/GPU processing. MoX introduces per-token multicast forwarding and GPU-side partial reduction on every tree edge, while the min-hop baseline can aggregate sends per source-destination pair. With 5,120 tokens/GPU and top-8, the per-token operations number in the tens of thousands per collective; even a 1 µs per-operation cost could exceed the simulated link-transfer savings. The reported 'up to 1.8x' and 'within 0.5% of switch' are therefore not yet established. Please add a microbenchmark, a per-packet overhead model, or at least a sensitivity analysis over per-operation costs.
  2. [Section 3.3, Boardfly] The Boardfly result is not end-to-end. The text says 'we do not simulate this full topology end to end, we report the validated maximum-link-load proxy for dispatch.' The proxy is validated in Fig. 3 only on degree-8 expanders; Boardfly has asymmetric link capacities (doubled cables), so the proxy's accuracy is not established for that topology. The input trace is also resampled to a projected 4K-expert model rather than measured Boardfly traffic. Thus the abstract's 'up to 47%' bottleneck reduction is a preliminary proxy result, and the claim should be softened or supported by an end-to-end simulation on a Boardfly-like topology.
  3. [Figures 2, 4, and 6] All reported MoE-block times are point estimates. Section 3 mentions 'three late-training DeepSeek-V3 runs,' but no error bars, confidence intervals, or per-run variability are given. The claims include differences as small as 0.5-0.6% from the switch, which are meaningful only if the simulation variance is below the reported gap. Please report variance across random topology instances, trace/seed choices, and weight-init/replay randomness, or restrict the claims accordingly.
  4. [Section 2.4, iterative weight computation] The offline optimization is a heuristic with several free parameters (η=0.05, number of replay/update iterations, sample size of 1,000 tokens/source, best-replay retention), and no sensitivity analysis is provided. The text is internally inconsistent: it says 'replayed across all six iterations—one with uniform weights followed by three weight updates' (1+3≠6). The only validation against full enumeration is at n=16, not at the n=32/64 sizes used in the headline results. Since near-switch performance depends on this heuristic, please show convergence/sensitivity and fix the iteration count.
minor comments (5)
  1. [Section 3 heading] Typo: 'EV ALUATION' should be 'EVALUATION'.
  2. [Figure 5] The three numeric rows (top-k 8/10/12) are not visually separated. The caption should clearly indicate that values are normalized per top-k and identify which bars correspond to BW-Tax vs MoX.
  3. [Section 2.4] The iteration-count inconsistency ('six iterations' vs 'one with uniform weights followed by three weight updates') should be corrected for reproducibility.
  4. [Section 4, 'Implementation'] The statement that 'line rate AllReduce has been demonstrated in GPUs [15,22]' cites an NCCL validation test and an NCCL analysis; neither directly demonstrates GPU-side partial reduction for MoX combine. Consider citing a direct implementation or measurement.
  5. [General] No code, artifacts, or trace release is mentioned, which limits reproducibility for a systems paper. Please state availability or provide a public artifact.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: offline weight optimization and evaluation are decoupled; remaining concerns are implementation-risk, not derivation-circularity.

full rationale

I walked the derivation chain. The central claim is that weights computed offline from a synthetic uniform top-K workload transfer to recorded DeepSeek-V3/Qwen-3 MoE traces. The paper explicitly does not use the recorded traces to set weights: §2.4 samples from 'the assumed uniform workload distribution,' while §3 evaluates on separately recorded Chakra traces. The evaluation metric (end-to-end wall time and max link load) is the same min-max objective as the offline optimization, but the evaluation data are out-of-sample, so there is no fitted-input-called-prediction reduction. The analytical proxy in §3.1 is validated against ASTRA-sim simulations rather than being used to manufacture the headline end-to-end results; the Boardfly analysis uses this proxy after validation, which is an extrapolation but not circular. The multicast-tree building blocks are attributed to prior protocols (e.g., Xcast, BIER), and no load-bearing conclusion rests on a self-citation or on a uniqueness theorem imported from the authors' prior work. Section 4 concedes that per-token forwarding and GPU-side partial reduction are 'believed' to be low overhead without a microbenchmark; that is an unvalidated implementation assumption and a correctness risk, not a derivation-from-inputs circularity. No step in the paper's equations reduces to its own inputs, and no fitted constant is renamed as a prediction. Therefore the circularity score is 0.

Assumptions & free parameters 6 free parameters · 7 assumptions · 0 invented entities

The algorithm itself introduces no new physical entities. The load-bearing assumptions are about MoE traffic statistics, simulator fidelity, and the unmeasured per-token forwarding cost. The free parameters are hyperparameters of the offline optimization, not fitted to the evaluation traces.

free parameters (6)
  • eta (weight update rate) = 0.05
    Used in Eq. (6); chosen by hand; affects convergence of the multiplicative min-max controller.
  • replay/update iterations
    Text says six iterations (one uniform weights + three updates); ambiguous; chosen by hand and affects weight convergence.
  • Monte Carlo sample size = 1,000 tokens/source
    Sample size for offline weight computation (§2.4); chosen by hand; larger sample improves estimate at higher cost.
  • best-replay retention
    MoX keeps the weight vector from the replay with the lowest maximum link load (§2.4); a model-selection rule on the synthetic workload.
  • expander degree = 8 (and 16 in Fig 6b)
    Main evaluation uses degree-8 expanders; §4 recommends d^2/N ≥ 2. This is an experimental topology parameter, not fitted to data.
  • Boardfly projected config = 4K experts / 1K TPUs
    DeepSeek trace resampled to a projected 4K-expert model with four experts per TPU (§3.3); a modeling choice for the Boardfly extrapolation.
assumptions (7)
  • domain assumption Uniform prior over top-K expert combinations is an adequate surrogate for real aggregated MoE traffic
    Invoked in §1 and §2.4 to generate Monte Carlo workloads for weight computation; if real layer-aggregated traffic deviates strongly from uniform, the precomputed weights may balance poorly. The paper tests skews (Zipf, real traces) to support it.
  • domain assumption Experts are placed randomly across accelerators and independent across layers
    Used in §1 and §3 to justify that hot experts do not create persistent physical hotspots; supports the uniform-prior assumption.
  • domain assumption ASTRA-sim2 with per-link transfers faithfully models MoE-block completion time
    All results in §3 come from ASTRA-sim; no hardware validation is provided.
  • domain assumption Maximum physical-link load is a valid proxy for end-to-end MoE-block time
    Validated against ASTRA-sim on expanders in §3.1 (Fig. 3), then applied to Boardfly in §3.3 without end-to-end validation on that topology.
  • domain assumption Modern MoE training maintains balanced expert utilization via auxiliary losses or routing biases
    Stated in §1; needed so the uniform prior is plausible. Cites [18,16,36].
  • ad hoc to paper Per-token multicast forwarding and GPU-side partial reduction add negligible overhead
    Stated in §4 as a belief ('We believe that this can be implemented with low overhead'); not implemented or measured, yet central to the claimed end-to-end speedup.
  • domain assumption Boardfly topology parameters from Google's public description are accurate
    Used in §3.3 to construct the 1,024-TPU model; the paper does not verify against vendor hardware.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MoX: Efficient MoE Routing on Direct-Connect Topologies." pith.science (2026). https://pith.science/paper/CI3WGCPZ

@misc{pith2026260720220,
  author       = {Pith},
  title        = {Pith review of: MoX: Efficient MoE Routing on Direct-Connect Topologies},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CI3WGCPZ}},
  note         = {Machine review of arXiv:2607.20220}
}
read the original abstract

Optically switched networks suit the regular communication of dense ML models, but MoE introduces sparse, runtime-dependent traffic. We show that efficient offline-optimized routing enables efficient MoE training and inference on direct-connect topologies without the need for MoE traffic matrix or dynamic topology reconfiguration. MoX constructs token-aware multicast trees to reduce bandwidth tax, then uses static, precomputed link weights to balance traffic by solving a restricted multicast tree-packing problem. Using recorded traffic from large MoE models, token-level traces, and ASTRA-sim, we find that MoX accelerates the full MoE block -- dispatch, expert computation, and combine -- by up to 1.8x over min-hop routing. Moreover, it attains nearly ideal packet-switched network performance in random expander topologies. On a 1,024-TPU model of Google's Boardfly topology, MoX reduces the dispatch bottleneck link load by up to 47%. These results show that high-performance MoE on static direct-connect fabrics can be achieved via optimized load-oblivious routing without demand-driven reconfiguration.

Figures

Figures reproduced from arXiv: 2607.20220 by the authors.

Figure 1
Figure 1. Routing an embedding whose expert desti [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. MoE-block wall time normalized to an ideal switch, for the MoE load distribution of DeepSeek-V3. MoX [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Analytical maximum-link-load overhead and ASTRA-sim end-to-end slowdown relative to the switch; the two correlate closely, overlapping in many points. in [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (2 more)
Figure 5
Figure 5. Figure 5: Boardfly dispatch bottleneck: maximum physical-cable load, normalized per top-𝑘 to shortest￾path routing, for training and inference. The bottle￾neck is an inter-group cable in every case. 16 32 64 Expander Size (GPUs) 0 50 100 MoE Block Wall Time Overhead [%] MoX Min-…
Figure 6
Figure 6. Figure 6: Overhead analysis for different system pa [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

44 extracted references · 2 canonical work pages

  1. [1]

    Dennis Abts, Garrin Kimmell, Andrew Ling, John Kim, Matt Boyd, Andrew Bitar, Sahil Parmar, Ibrahim Ahmed, Roberto DiCecco, David Han, John Thompson, Michael Bye, Jennifer Hwang, Jeremy Fow- ers, Peter Lillian, Ashwin Murthy, Elyas Mehtabuddin, Chetan Tekur, Thomas Sohmers, Kris Kang, Stephen Maresh, and Jonathan Ross

  2. [2]

    Aaron Adcock, Aayushi Srivastava, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pande, Abhinav Pandey, Abhinav Sharma, Abhishek Kadian, Abhishek Kumawat, Adam Kelsey, et al . 2026. The Llama 4 Herd: Architecture, Training, Evaluation, and Deployment Notes. arXiv preprint arXiv:2601.11659(2026)

    arXiv 2026

  3. [3]

    George Almási, Philip Heidelberger, Charles J Archer, Xavier Martorell, C Chris Erway, José E Moreira, Burkhard Steinmacher-Burow, and Yili Zheng. 2005. Optimization of MPI collective communication on BlueGene/L systems. InProceedings of the 19th annual international conference on Supercomputing. 253–262

    2005

  4. [4]

    Amazon Web Services. 2023. Amazon EC2 Inf2 Architec- ture. https://awsdocs-neuron.readthedocs-hosted.com/en/latest/ about-neuron/arch/neuron-hardware/inf2-arch.html

    2023

  5. [5]

    Anantharamu, Abhishek Dosi, Mohit Garg, Mahdieh Ghazi, Torsten Hoefler, Deepal Jayasinghe, Jithin Jose, Abdul Kabbani, Guohan Lu, Yang Wang, K

    Joao Araujo, Alex Chow, Mark Handley, Ryder Lewis, Christoph Paasch, Jitendra Padhye, Michael Papamichael, Greg Steinbrecher, Amin Tootoonchian, Lihua Yuan, S. Anantharamu, Abhishek Dosi, Mohit Garg, Mahdieh Ghazi, Torsten Hoefler, Deepal Jayasinghe, Jithin Jose, Abdul Kabbani, Guohan Lu, Yang Wang, K. Doddapaneni, Mu- rali Garimella, Vipin Jain, Yanfang ...

    arXiv 2026

  6. [6]

    Andreas Baltz and Anand Srivastav. 2004. Fast Approximation of Min- imum Multicast Congestion—Implementation versus Theory.RAIRO Operations Research38, 4 (2004), 319–344. https://doi.org/10.1051/ro: 2004028

    work page doi:10.1051/ro: 2004

  7. [7]

    Prithwish Basu, Liangyu Zhao, Jason Fantl, Siddharth Pal, Arvind Krishnamurthy, and Joud Khoury. 2024. Efficient all-to-all Collective Communication Schedules for Direct-connect Topologies. InProceed- ings of the 33rd International Symposium on High-Performance Parallel and Distributed Computing(Pisa, Italy)(HPDC ’24). Association for Computing Machinery,...

    2024

  8. [8]

    Theophilus Benson, Ashok Anand, Aditya Akella, and Ming Zhang

Show all 44 references
  1. [9]

    Giacomo Bernardi, Ratul Mahajan, C Seshadhri, Enrico Carlesso, Chinchu Merine Joseph, Saurabh Kumar, Pavan Manikonda, Luiza Popa, Randy Ram, Steven Robinson, et al. 2026. RNG: Flat Datacenter Networks at Scale.arXiv preprint arXiv:2604.15261(2026)

    2026 arXiv

  2. [10]

    Rick Boivie, Nancy Feldman, Yuji Imai, Wim Livens, and Dirk Ooms

  3. [11]

    Zixian Cai, Zhengyang Liu, Saeed Maleki, Madanlal Musuvathi, Todd Mytkowicz, Jacob Nelson, and Olli Saarikivi. 2021. Synthesizing opti- mal collective algorithms. InProceedings of the 26th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming. 62–75

    2021

  4. [12]

    Shiwen Chen, Oktay Gunluk, and Bulent Yener. 2000. The Multicast Packing Problem.IEEE/ACM Transactions on Networking8, 3 (2000), 311–318. https://doi.org/10.1109/90.851977

    2000 doi

  5. [13]

    Sheng-Hao Chiang, Jian-Jhih Kuo, Shan-Hsiang Shen, De-Nian Yang, and Wen-Tsuen Chen. 2018. Online multicast traffic engineering for software-defined networks. InIEEE INFOCOM 2018-IEEE Conference on Computer Communications. IEEE, 414–422

    2018

  6. [14]

    Peizhuang Cong, Aomufei Yuan, Shimao Chen, Yuxuan Tian, Bowen Ye, and Tong Yang. 2024. Prediction Is All MoE Needs: Expert Load Distribution Goes from Fluctuating to Stabilizing.arXiv preprint arXiv:2404.16914(2024)

    2024 arXiv

  7. [15]

    NVIDIA Corporation. 2025. Validate the cluster level NCCL test with 4 nodes and 32 GPUs. https://docs.nvidia.com/dgx- basepod/deployment-guide-dgx-basepod/latest/mn-nccl.html. https://docs.nvidia.com/dgx-basepod/deployment-guide-dgx- basepod/latest/mn-nccl.html

    2025

  8. [16]

    DeepSeek-AI. 2024. DeepSeek-V3 Technical Report.arXiv preprint arXiv:2412.19437(2024)

    2024 arXiv

  9. [17]

    Eric Ding, Chuhan Ouyang, and Rachee Singh. 2025. Photonic rails in ML datacenters. InProceedings of the 24th ACM Workshop on Hot Topics in Networks. 149–159

    2025

  10. [18]

    William Fedus, Barret Zoph, and Noam Shazeer. 2022. Switch Trans- formers: Scaling to Trillion Parameter Models with Simple and Effi- cient Sparsity.Journal of Machine Learning Research23, 120 (2022), 1–39. https://www.jmlr.org/papers/v23/21-0998.html

    2022

  11. [19]

    Handley, Hugh Holbrook, Isidor Kouvelas, Rishabh Parekh, Zhaohui (Jeffrey) Zhang, and Lianshu Zheng

    Bill Fenner, Mark J. Handley, Hugh Holbrook, Isidor Kouvelas, Rishabh Parekh, Zhaohui (Jeffrey) Zhang, and Lianshu Zheng. 2016. Protocol Independent Multicast - Sparse Mode (PIM-SM): Protocol Specification (Revised). RFC 7761

    2016

  12. [20]

    Google Cloud. 2026. Google TPU 8i (Boardfly) architecture overview. https://cloud.google.com/blog/products/compute/tpu-8t- and-tpu-8i-technical-deep-dive

    2026

  13. [21]

    Chi-Yao Hong, Srikanth Kandula, Ratul Mahajan, Ming Zhang, Vijay Gill, Mohan Nanduri, and Roger Wattenhofer. 2013. Achieving high utilization with software-driven WAN. InProceedings of the ACM SIGCOMM 2013. 15–26

    2013

  14. [22]

    Zhiyi Hu, Siyuan Shen, Tommaso Bonato, Sylvain Jeaugey, Cedell Alexander, Eric Spada, James Dinan, Jeff Hammond, and Torsten Hoe- fler. 2025. Demystifying NCCL: An in-depth analysis of GPU com- munication protocols and algorithms. In2025 IEEE Symposium on High-Performance Inte...

    2025

  15. [23]

    Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al . 2024. Mixtral of experts.arXiv preprint arXiv:2401.04088(2024)

    2024 arXiv

  16. [24]

    Towles, Clifford Young, Xiang Zhou, Zongwei Zhou, and David A Patterson

    Norm Jouppi, George Kurian, Sheng Li, Peter Ma, Rahul Nagarajan, Lifeng Nai, Nishant Patil, Suvinay Subramanian, Andy Swing, Brian 7 Cohen et al. Towles, Clifford Young, Xiang Zhou, Zongwei Zhou, and David A Patterson. 2023. TPU v4: An Optically Reconfigurable Supercomputer fo...

    2023

  17. [25]

    Dan Li, Yuanjie Li, Jianping Wu, Sen Su, and Jiangwei Yu. 2011. ESM: Efficient and scalable data center multicast routing.IEEE/ACM Trans- actions on Networking20, 3 (2011), 944–955

    2011

  18. [26]

    Jiamin Li, Yimin Jiang, Yibo Zhu, Cong Wang, and Hong Xu. 2023. Accelerating Distributed MoE Training and Inference with Lina. In 2023 USENIX Annual Technical Conference (USENIX ATC 23). USENIX Association, 945–959

    2023

  19. [27]

    Xudong Liao, Yijun Sun, Han Tian, Xinchen Wan, Yilun Jin, Zilong Wang, Zhenghang Ren, Xinyang Huang, Wenxue Li, Kin Fai Tse, et al

  20. [28]

    Moonshot AI. 2026. Kimi K3: Open Frontier Intelligence. https://www. kimi.com/blog/kimi-k3

    2026

  21. [29]

    Reiner Pope, Sholto Douglas, Aakanksha Chowdhery, Jacob Devlin, James Bradbury, Anselm Levskaya, Jonathan Heek, Kefan Xiao, Shiv- ani Agrawal, and Jeff Dean. 2023. Efficiently Scaling Transformer Inference. InProceedings of Machine Learning and Systems (MLSys)

    2023

  22. [30]

    Qwen Team. 2025. Qwen3 Technical Report.arXiv preprint arXiv:2505.09388(2025)

    2025 arXiv

  23. [31]

    Qwen Team. 2026. Qwen3.5-397B-A17B model card. https:// huggingface.co/Qwen/Qwen3.5-397B-A17B

    2026

  24. [32]

    Srinivas Sridharan, Taekyung Heo, Louis Feng, Zhaodong Wang, Matt Bergeron, Wenyin Fu, Shengbao Zheng, Brian Coutinho, Saeed Rashidi, Changhai Man, and Tushar Krishna. 2023. Chakra: Advancing Perfor- mance Benchmarking and Co-design using Standardized Execution Traces.arXiv pr...

    2023 arXiv

  25. [33]

    Young-Joo Suh and S Valamanchili. 1998. All-to-all communication with minimum start-up costs in 2D/3D tori and meshes.IEEE Trans- actions on Parallel and Distributed Systems9, 5 (1998), 442–458

    1998

  26. [34]

    Yehui Tang, Yichun Yin, Yaoyuan Wang, Hang Zhou, Yu Pan, Wei Guo, Ziyang Zhang, Miao Rang, Fangcheng Liu, Naifu Zhang, Binghan Li, Yonghan Dong, Xiaojun Meng, Yasheng Wang, Dong Li, Yin Li, Dandan Tu, Can Chen, Youliang Yan, Fisher Yu, Ruiming Tang, Yunhe Wang, Botian Huang, B...

    2025 arXiv

  27. [35]

    Amin Vahdat. 2026. Our eighth generation TPUs: two chips for the agentic era. https://blog.google/innovation-and-ai/infrastructure- and-cloud/google-cloud/eighth-generation-tpu-agentic-era/. https://blog.google/innovation-and-ai/infrastructure-and- cloud/google-cloud/eighth-ge...

    2026

  28. [36]

    Lean Wang, Huazuo Gao, Chenggang Zhao, Xu Sun, and Damai Dai

  29. [37]

    2023.{TopoOpt}: Co-optimizing network topology and parallelization strategy for distributed training jobs

    Weiyang Wang, Moein Khazraee, Zhizhen Zhong, Manya Ghobadi, Zhihao Jia, Dheevatsa Mudigere, Ying Zhang, and Anthony Kewitsch. 2023.{TopoOpt}: Co-optimizing network topology and parallelization strategy for distributed training jobs. In20th USENIX Symposium on Networked Systems...

    2023

  30. [38]

    Rosen, Andrew Dolganow, Tony Przy- gienda, and Sam Aldrin

    IJsbrand Wijnands, Eric C. Rosen, Andrew Dolganow, Tony Przy- gienda, and Sam Aldrin. 2017. Multicast Using Bit Index Explicit Replication (BIER). IETF RFC 8279

    2017

  31. [39]

    William Won, Taekyung Heo, Saeed Rashidi, Srinivas Sridharan, Sudar- shan Srinivasan, and Tushar Krishna. 2023. ASTRA-sim2.0: Modeling Hierarchical Networks and Disaggregated Systems for Large-model Training at Scale. InIEEE International Symposium on Performance Analysis of S...

    2023

  32. [2007]

    IETF RFC 5058

    Explicit Multicast (Xcast) Concepts and Options. IETF RFC 5058

  33. [2011]

    InPro- ceedings of the seventh conference on emerging networking experiments and technologies

    MicroTE: Fine grained traffic engineering for data centers. InPro- ceedings of the seventh conference on emerging networking experiments and technologies. 1–12

  34. [2022]

    InProceedings of the 49th Annual International Symposium on Computer Architecture(New York, New York)(ISCA ’22)

    A software-defined tensor streaming multiprocessor for large- scale machine learning. InProceedings of the 49th Annual International Symposium on Computer Architecture(New York, New York)(ISCA ’22). Association for Computing Machinery, New York, NY, USA, 567–580

  35. [2024]

    Auxiliary-loss-free load balancing strategy for mixture-of- experts.arXiv preprint arXiv:2408.15664(2024)

    2024 arXiv

  36. [2025]

    InProceedings of the ACM SIGCOMM 2025 Conference

    Mixnet: A runtime reconfigurable optical-electrical fabric for distributed mixture-of-experts training. InProceedings of the ACM SIGCOMM 2025 Conference. 554–574

    2025

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.