Pith. sign in

REVIEW 5 major objections 5 minor 52 references

Congestion-Aware Path Selection for Load Balancing in AI Clusters

T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Hopper, a host-only load balancer that probes alternative paths when RTT rises, cuts average and tail flow completion times by up to 20% and 14% versus random-switching host-based schemes.

desk verdict Hopper is a credible host-based RDMA load balancer; its FCT gains are plausible in the tested scenarios but the missing threshold sensitivity analysis keeps the headline from being load-bearing. read the letter →

arxiv 2506.08132 v1 pith:5SZWLW4O submitted 2025-06-09 cs.NI

Erfan Nosrati , Majid Ghaderi This is my paper
classification cs.NI
keywords RDMAloadbalancingAIclusternetworkshost-basedpathswitchingRTT-basedcongestiondetectionflowcompletiontimeout-of-orderpacketcontrolpower-of-two-choicesprobingECMPhashing
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Hopper is a host-only load-balancing technique for RDMA traffic in AI clusters that claims to fix the two failure modes of random path switching: landing on another congested path and generating excessive out-of-order packets. It measures per-packet RTT on the active path, probes two randomly chosen alternative paths when RTT crosses a threshold, and switches the flow only when one probed path is clearly better, delaying the switch in proportion to the delay difference. The paper argues that modern RDMA NICs already have the RTT measurement and limited reordering support Hopper needs, so the technique requires no switch modifications and only standard ECMP hashing. In simulations and a testbed, Hopper improves average and 99-percentile flow completion time by up to 20% and 14% compared to the random-switching host baseline, reduces training time roughly by half in the testbed, and comes close to switch-based schemes at moderate load. If the claim holds, AI clusters can get substantially better load balancing from a host-side software change alone.

What carries the argument

The load-bearing object is Hopper's RTT-gated control loop, which combines four pieces: congestion detection via a moving-average RTT with two thresholds; power-of-two-choices probing that examines two randomly chosen unexplored paths through queue pairs bound to distinct UDP source ports, with a TTL cache to avoid revisiting recently probed paths; condition-gated switching that moves the flow only when a probed path's RTT is lower by a margin (δ_rtt) than the current path; and delay-based out-of-order control, in which a linear RTT-trend estimator computes a switch delay proportional to the RTT difference so packets on the new path do not overtake in-flight packets on the old path. Together these let a host redirect standard ECMP hashing by changing the UDP source port, requiring no switch or NIC hardware changes.

What would settle it

In the paper's own simulation setup, change only the base link latency from 1 µs to 0.5 µs (halving base RTT to 4 µs) while keeping all Table 1 thresholds fixed; if Hopper's average-FCT improvement over the random-switching baseline drops below the reported 20% or reverses, the central claim is conditional on tuned thresholds rather than on the mechanism.

Watch

Extended reading notes

Core claim

The paper's central claim is that informed path selection, not just congestion-triggered switching, is what makes host-based load balancing work for RDMA. Hopper keeps a moving average of the active path's RTT; when that average exceeds a threshold, it creates two queue pairs bound to distinct UDP source ports and probes two randomly chosen unexplored paths. If the active path's RTT then exceeds a second threshold and one probed path beats it by a configurable margin, Hopper switches the flow to that path, and it delays the switch by an amount derived from a linear estimate of the RTT difference so that in-flight packets on the old path arrive before the first packets on the new path, keeping reordering within the RNIC's limited buffer. The paper shows this mechanism outperforms the random-switching host baseline on datacenter and ML training workloads, and that at moderate load the remaining gap to switch-based in-network reordering is small.

Load-bearing premise

The claimed gains rest on the hand-tuned thresholds in Table 1—probe at 1.5x base RTT, switch at 2.5x, probe TTL 4x, improvement margin 80%—transferring to other AI cluster topologies, workloads, and link speeds; the paper only says they 'found to work well in our setup.'

Editorial extensions

If this is right

  • AI clusters can obtain most of the benefit of in-network load balancing through a host-side software change, without programmable switches or new NIC hardware.
  • Hopper's gains concentrate exactly where ECMP fails hardest: the large, synchronized flows of collective communication (AllReduce, AllGather, ReduceScatter) that dominate ML training.
  • Because Hopper switches only when a clearly better path exists, it avoids the tail-latency penalty of random switching onto an equally congested path.
  • The delay-based switching keeps out-of-order packets within the RNIC's reordering threshold, so the technique scales to the limited on-chip memory of commercial RDMA NICs.
  • On asymmetric topologies, Hopper's informed probing steers traffic to high-bandwidth paths, cutting testbed training time by about half compared with the random-switching baseline.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The fixed thresholds in Table 1 (1.5x, 2.5x, 4x base RTT, 80% margin) are likely load-, topology-, and speed-dependent; making them adaptive to measured base RTT would be a natural next step the paper does not explore.
  • Hopper's probe-to-path mapping relies on the assumption that a UDP source port maps stably to one ECMP path; deployments with dynamic hashing or re-routing would need periodic re-profiling, which the paper mentions only as a testbed practice.
  • The power-of-two-choices limit could be raised under sustained asymmetry, and ECN could be fused with RTT as a proactive probe trigger; the paper notes ECN could signal congestion earlier but leaves that for future work.
  • If Hopper's control logic were embedded in a collective communication library rather than the user-space testbed prototype, it could coordinate path choices across the many QPs of a single collective, potentially improving global balance beyond per-flow decisions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper presents Hopper, a host-based load-balancing mechanism for RDMA traffic in AI clusters. Hopper uses per-flow RTT measurements to detect congestion on the current path, probes two randomly chosen alternative paths when RTT exceeds a threshold, and switches the flow to a better path found via power-of-two-choices probing, while delaying the switch based on RTT differences to limit out-of-order packets. The authors evaluate Hopper through ns-3 simulations under datacenter and ML-training workloads and through a testbed with asymmetric path bandwidths. They report that Hopper reduces average and 99th-percentile FCT slowdown by up to 20% and 14%, respectively, compared to the host-based FlowBender baseline, and that it approaches switch-based ConWeave at moderate load.

Significance. If Hopper's benefits are robust, the paper would make a useful practical contribution: it proposes a load balancer that requires no switch modifications, only RNIC capabilities already present in modern hardware, and it includes both simulation and testbed evidence. The paper is honest about several limitations: it explicitly cautions that 10MB flow results in the Hadoop workload are unstable, it acknowledges that switch-based ConWeave outperforms Hopper at high load, and it provides a parameter table rather than hiding its configuration. These transparency measures are commendable. However, the central empirical claim rests on a small set of hand-tuned thresholds and a single simulation/testbed configuration, with no sensitivity analysis or confidence intervals, which currently limits the strength of the conclusions that can be drawn.

major comments (5)
  1. [§4.1.1, Table 1] The four central thresholds (th_probe=1.5x, th_cong=2.5x, ttl_probe=4.0x, delta_rtt=80%) are stated to 'work well in our setup' with no sensitivity analysis. Hopper's decision logic is a cascade of threshold comparisons on RTT measurements, so the reported 20%/14% improvements could be specific to these values and the particular 100G leaf-spine, DCQCN parameters, and load levels. The authors should provide a parameter sweep (e.g., varying each threshold around the chosen value at 50% and 80% load) or a principled calibration rule tied to base RTT and network scale. Without this, the reader cannot distinguish a robust mechanism from a configuration that was fitted to the evaluated scenarios.
  2. [§4.2.1, Figs. 6] The testbed experiments are repeated only five times and reported as averages, with no error bars, confidence intervals, or per-run data. Given the large FCT slowdown values (p95 and p99 in the range 10-30 in some conditions), the differences between Hopper and FlowBender could plausibly be within run-to-run variability. The authors should report variance or use a statistical test to support the claimed improvements (e.g., 45% average and 77.2% p99 improvements with 1MB chunks).
  3. [§4.2.1, §3.2] Hopper's testbed relies on pre-profiling to map specific source ports to distinct paths. This mapping is assumed stable, but in a production AI cluster, ECMP rehashing, link failures, or topology changes could invalidate the mapping, causing the probing QPs to collide on the same path or to miss available paths. The paper should discuss how Hopper detects and recovers from stale port-to-path mappings, or why this assumption is safe for its intended deployment.
  4. [§4.1.2, Fig. 3] The paper cautions that the 10MB-flow results for the Hadoop workload are 'not stable,' yet the same section states that Hopper outperforms FlowBender by up to 19.6% in p99 tail slowdown. If the maximum improvement is taken from the unstable 10MB data point, the headline claim is not supported. The authors should identify exactly which flow-size bin and load level produce each maximum (7.8%, 19.6%, and the ML-workload 20%/14%) and exclude or clearly flag unstable bins from those maxima.
  5. [§4.2.1] The testbed compares Hopper against a 'simplified software version' of FlowBender in which congestion is inferred from RTT increases instead of the original ECN-based signal. This modifies the baseline's detection mechanism to match Hopper's own signal, potentially disadvantaging FlowBender if ECN would have reacted earlier under DCQCN. The authors should justify this modification or present a version with ECN-based FlowBender, especially since the simulations use the original FlowBender.
minor comments (5)
  1. [§1] The word 'scalbility' in the first paragraph is a typo; it should be 'scalability.'
  2. [§4.1.1] The phrase 'we also implement and extended version of IRN' should read 'an extended version.'
  3. [§4.1.2] The text says Hopper improves average FCT by up to 7.8% for Hadoop workload, but Fig. 3 appears to show larger differences at some flow sizes; please clarify how the percentage is computed (e.g., relative reduction in slowdown, and whether it is per-bin or overall).
  4. [§2] The motivation section would benefit from a citation for the claim that hardware timestamping is supported by commodity RNICs; references are given later in §5 but the connection is not explicit.
  5. [Appendix C] The caption '99 th' should be '99th' for consistency with the other figures.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Hopper's FCT improvements are empirical measurements from ns-3 and testbed comparisons, not derived by construction from its own definitions, fitted parameters, or self-citations.

full rationale

The paper is an empirical systems paper rather than a mathematical derivation, and I find no circular step that reduces a claimed result to its own inputs. The core claim is that Hopper improves average and p99 FCT slowdown compared to FlowBender, CONGA, and ConWeave; this is supported by ns-3 simulations and a hardware testbed, with the comparisons measured after running the mechanisms. Table 1 lists parameters such as th_probe=1.5x base RTT and th_cong=2.5x base RTT, and the text notes they 'are found to work well in our setup.' That is parameter tuning without sensitivity analysis, which is a robustness concern rather than circularity: the FCT improvements are not algebraic consequences of these thresholds, and the paper does not rename a fitted value as a prediction. The authors do not rely on their own prior work for any load-bearing premise; the reference list contains no self-citations by Nosrati or Ghaderi, and no uniqueness theorem or ansatz is imported from the authors' earlier papers. The design reuses well-known components (RTT measurement, power-of-two-choices probing, ECMP switching) with external citations, but the novelty claim is the host-based combination, and the evaluation is self-contained against external baselines and workloads. Accordingly, the appropriate finding is no significant circularity.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

Hopper is a software system, not a new physical entity, so no invented entities are listed. The central claim rests on six hand-tuned parameters and five domain assumptions, most notably the transferability of the thresholds and the reliability of RTT as a congestion signal. The linear RTT extrapolation model is an ad hoc modeling assumption introduced by the paper.

free parameters (6)
  • th_probe = 1.5 x base RTT (12 us in the workflow example)
    Threshold to start probing for alternative paths; Table 1 says the value is 'found to work well in our setup', not derived from theory.
  • th_cong = 2.5 x base RTT
    Threshold at which the current path is considered congested and switching is triggered; tuned to the setup.
  • ttl_probe = 4.0 x base RTT
    Time interval to avoid re-probing a recently explored path; tuned.
  • delta_rtt = 80%
    Required RTT improvement margin before switching to an alternative path; no sensitivity analysis is given.
  • alpha (EWMA smoothing) = not clearly stated (Table 1 shows a garbled value)
    Moving average parameter in Algorithm 1; the printed value appears corrupted in the table.
  • testbed chunk size = 1 MB chosen as good trade-off (also tested at 10 MB)
    Chunk size determines path-switching granularity in the testbed; the paper reports 1 MB as a good trade-off and notes chunks below 100 KB degrade performance.
assumptions (5)
  • domain assumption Host-measured RTT is a reliable per-path congestion signal for RDMA flows
    The entire congestion detection and path probing design rests on RTT as a meaningful signal; the paper cites Timely but does not validate the signal on the tested RDMA workloads beyond the experiments (§3.1).
  • domain assumption Modern RNICs can support the QP creation and limited reordering required by probing without significant performance loss
    Hopper creates two probing QPs per flow when switching; the paper notes RNIC SRAM and QP scalability concerns but does not measure the overhead of QP creation on the testbed (§2, §3.2).
  • domain assumption ECMP hashing based on UDP source port maps flows to distinct stable paths
    Hopper changes only the source port to steer traffic; the testbed needs a profiling step to map source ports to paths, which assumes the mapping is stable (§3.1, §4.2.1).
  • domain assumption The ConWeave ns-3 simulation framework, topology, and DCQCN parameters are representative of production AI clusters
    Simulations reuse ConWeave's setup with the same topology, transport parameters, and workload generator; only the Meta collective message size distribution is added (§4.1.1).
  • ad hoc to paper The RTT of the last in-flight packet can be estimated by linear extrapolation of the current RTT trend
    The switching delay is computed from a linear regression model that extrapolates the RTT increase rate; no validation of the linear model's accuracy is provided (§3.3, Fig. 1).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Congestion-Aware Path Selection for Load Balancing in AI Clusters." pith.science (2026). https://pith.science/paper/5SZWLW4O

@misc{pith2026250608132,
  author       = {Pith},
  title        = {Pith review of: Congestion-Aware Path Selection for Load Balancing in AI Clusters},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5SZWLW4O}},
  note         = {Machine review of arXiv:2506.08132}
}
read the original abstract

Fast training of large machine learning models requires distributed training on AI clusters consisting of thousands of GPUs. The efficiency of distributed training crucially depends on the efficiency of the network interconnecting GPUs in the cluster. These networks are commonly built using RDMA following a Clos-like datacenter topology. To efficiently utilize the network bandwidth, load balancing is employed to distribute traffic across multiple redundant paths. While there exists numerous techniques for load-balancing in traditional datacenters, these are often either optimized for TCP traffic or require specialized network hardware, thus limiting their utility in AI clusters. This paper presents the design and evaluation of Hopper, a new load-balancing technique optimized for RDMA traffic in AI clusters. Operating entirely at the host level, Hopper requires no specialized hardware or modifications to network switches. It continuously monitors the current path for congestion and dynamically switches traffic to a less congested path when congestion is detected. Furthermore, it incorporates a lightweight mechanism to identify alternative paths and carefully controls the timing of path switching to prevent excessive out-of-order packets. We evaluated Hopper using ns-3 simulations and a testbed implementation. Our evaluations show that Hopper reduces the average and 99-percentile tail flow completion time by up to 20% and 14%, respectively, compared to state-of-the-art host-based load balancing techniques.

Figures

Figures reproduced from arXiv: 2506.08132 by the authors.

Figure 1
Figure 1. Hopper employs a linear estimator to predict path RTT. and delay information for a few RTTs to avoid repeatedly probing the same congested paths. Although this approach may introduce some delay in switching away from congested paths, it effectively prevents excessive path switching when all links are persistently congested. Additionally, it avoids the issue where multiple flows from congested paths are rerouted to t… view at source ↗
Figure 2
Figure 2. Traffic characteristics of representative workloads. The [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Average and tail FCT slowdown for Hadoop workload at [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Average and tail FCT slowdown for ML Training workload at [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: The leaf-spine topology of the testbed with asymmetric [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Performance of different load balancing approaches in the testbed with [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Hoppers workflow. 0 1.7K 3.5K 4.6K 5.5K 6.3K 7.2K 8.5K 16K 31K 2.0M Flow Size (Bytes) 1.2 1.3 2 3 Avg FCT Slowdown ECMP Hopper ConWeave FlowBender CONGA (a) AliCloud 50% Load (Avg) 0 1.7K 3.5K 4.6K 5.5K 6.3K 7.2K 8.5K 16K 31K 2.0M Flow Size (Bytes) 3 4 6 10 17 p99 FCT …
Figure 8
Figure 8. Figure 8: Average and tail FCT slowdown for AliCloud workload at [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]
Figure 9
Figure 9. Figure 9: Average and tail FCT slowdown for Hadoop workload at [PITH_FULL_IMAGE:figures/full_fig_p011_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

52 extracted references · 45 canonical work pages

  1. [1]

    Mohammad Alizadeh, Tom Edsall, Sarang Dharmapurikar, Ramanan Vaidyanathan, Kevin Chu, Andy Fingerhut, Vinh The Lam, Francis Ma- tus, Rong Pan, Navindra Yadav, and George Varghese. 2014. CONGA: distributed congestion-aware load balancing for datacenters. InProceedings of the 2014 ACM Conference on SIGCOMM(Chicago, Illinois, USA)(SIGCOMM ’14). Association f...

    work page 2014

  2. [2]

    InfiniBand Trade Association. 2007. Infiniband Architecture Specifications. https: //www.infinibandta.org/ibta-specification/

    work page 2007

  3. [3]

    Jiamin Cao, Yu Guan, Kun Qian, Jiaqi Gao, Wencong Xiao, Jianbo Dong, Binzhang Fu, Dennis Cai, and Ennan Zhai. 2024. Crux: GPU-Efficient Communication Scheduling for Deep Learning Training. InProceedings of the Conference of the ACM Special Interest Group on Data Communication(Sydney, NSW, Australia) (ACM SIGCOMM ’24). Association for Computing Machinery, ...

    work page 2024

  4. [4]

    Chuhao Chen, Jiarui Ye, Yongbo Gao, Sen Liu, and Yang Xu. 2024. HFˆ2T: Host- Based Flowlet Fine-Tuning for RDMA Load Balancing. InProceedings of the 8th Asia-Pacific Workshop on Networking(Sydney, Australia)(APNet ’24). Association for Computing Machinery, New York, NY, USA, 9–15

    work page 2024

  5. [5]

    Xiaoqi Chen, Shay Vargaftik, and Ran Ben Basat. 2024. When ML Training Cuts Through Congestion: Just-in-Time Gradient Compression via Packet Trimming. InProceedings of the 23rd ACM Workshop on Hot Topics in Networks(Irvine, CA, USA)(HotNets ’24). Association for Computing Machinery, New York, NY, USA, 177–185

    work page 2024

  6. [6]

    NVIDIA Corporation. 2025. NVIDIA Collective Communications Library (NCCL). https://developer.nvidia.com/nccl. Accessed: 2025-05-29

    work page 2025

  7. [7]

    Charlie Hu, and Ramana Rao Kompella

    Advait Dixit, Pawan Prakash, Y. Charlie Hu, and Ramana Rao Kompella. 2013. On the impact of packet spraying in data center networks. In2013 Proceedings IEEE INFOCOM. 2130–2138

    work page 2013

  8. [8]

    Adithya Gangidi, Rui Miao, Shengbao Zheng, Sai Jayesh Bondu, Guilherme Goes, Hany Morsy, Rohit Puri, Mohammad Riftadi, Ashmitha Jeevaraj Shetty, Jingyi Yang, Shuqiang Zhang, Mikel Jimenez Fernandez, Shashidhar Gandham, and Hongyi Zeng. 2024. RDMA over Ethernet for Distributed Training at Meta Scale. InProceedings of the ACM SIGCOMM 2024 Conference(Sydney,...

    work page 2024

Show all 52 references
  1. [9]

    Brighten Godfrey, Yashar Ganjali, and Amin Firoozshahian

    Soudeh Ghorbani, Zibin Yang, P. Brighten Godfrey, Yashar Ganjali, and Amin Firoozshahian. 2017. DRILL: Micro Load Balancing for Low-latency Data Center Networks. InProceedings of the Conference of the ACM Special Interest Group on Data Communication(Los Angeles, CA, USA)(SIGCO...

    2017

  2. [10]

    Junzhi Gong, Tong Yang, Haowei Zhang, Hao Li, Steve Uhlig, Shigang Chen, Lorna Uden, and Xiaoming Li. 2018. HeavyKeeper: An Accurate Algorithm for Finding Top-k Elephant Flows. In2018 USENIX Annual Technical Conference (USENIX ATC 18). USENIX Association, Boston, MA, 909–921

    2018

  3. [11]

    Hamilton, Navendu Jain, Srikanth Kandula, Changhoon Kim, Parantap Lahiri, David A

    Albert Greenberg, James R. Hamilton, Navendu Jain, Srikanth Kandula, Changhoon Kim, Parantap Lahiri, David A. Maltz, Parveen Patel, and Sudipta Sengupta. 2009. VL2: a scalable and flexible data center network. InProceedings of the ACM SIGCOMM 2009 Conference on Data Communicat...

    2009

  4. [12]

    Christian Hopps. 2000. Analysis of an Equal-Cost Multi-Path Algorithm. RFC 2992

    2000

  5. [13]

    IEEE. 2008. 802.1Qbb – Priority-based Flow Control. https://1.ieee802.org/dcb/ 802-1qbb/

    2008

  6. [14]

    Intel Corporation. 2025. Intel®Ethernet Network Adapter E810-2CQDA2 Specifi- cations. https://www.intel.com/content/www/us/en/products/sku/210969/intel- ethernet-network-adapter-e8102cqda2/specifications.html. Accessed: 2025-06- 02

    2025

  7. [15]

    Abdul Kabbani, Balajee Vamanan, Jahangir Hasan, and Fabien Duchene. 2014. FlowBender: Flow-level Adaptive Routing for Improved Latency and Through- put in Datacenter Networks. InProceedings of the 10th ACM International on Conference on Emerging Networking Experiments and Tech...

    2014

  8. [16]

    Anuj Kalia, Michael Kaminsky, and David Andersen. 2019. Datacenter RPCs can be General and Fast. In16th USENIX Symposium on Networked Systems Design and Implementation (NSDI 19). USENIX Association, Boston, MA, 1–16

    2019

  9. [17]

    Naga Katta, Aditi Ghag, Mukesh Hira, Isaac Keslassy, Aran Bergman, Changhoon Kim, and Jennifer Rexford. 2017. Clove: Congestion-Aware Load Balancing at the Virtual Edge. InProceedings of the 13th International Conference on Emerging Networking EXperiments and Technologies(Inch...

    2017

  10. [18]

    Gautam Kumar, Nandita Dukkipati, Keon Jang, Hassan M. G. Wassel, Xian Wu, Behnam Montazeri, Yaogong Wang, Kevin Springborn, Christopher Alfeld, Michael Ryan, David Wetherall, and Amin Vahdat. 2020. Swift: Delay is Simple and Effective for Congestion Control in the Datacenter. ...

    2020

  11. [19]

    Yanfang Le, Rong Pan, Peter Newman, Jeremias Blendin, Abdul Kabbani, Vipin Jain, Raghava Sivaramu, and Francis Matus. 2024. STrack: A Reliable Multipath Transport for AI/ML Clusters. arXiv:2407.15266

    2024 arXiv

  12. [20]

    Shen Li, Yanli Zhao, Rohan Varma, Omkar Salpekar, Pieter Noordhuis, Teng Li, Adam Paszke, Jeff Smith, Brian Vaughan, Pritam Damania, and Soumith Chintala

  13. [21]

    Yuliang Li, Rui Miao, Hongqiang Harry Liu, Yan Zhuang, Fei Feng, Lingbo Tang, Zheng Cao, Ming Zhang, Frank Kelly, Mohammad Alizadeh, and Minlan Yu

  14. [22]

    Linux RDMA. 2025. RDMA core userspace libraries and daemons. https://github. com/linux-rdma/rdma-core. Accessed: 2025-05-16

    2025

  15. [23]

    Minfei Long, Jiangping Han, Wentao Wang, Jiayu Yang, and Kaiping Xue. 2024. LSCC: Link-Segmented Congestion Control for RDMA in Cross-Datacenter Net- works. In2024 IEEE/ACM 32nd International Symposium on Quality of Service (IWQoS). 1–10. https://doi.org/10.1109/IWQoS61813.202...

    2024

  16. [24]

    Yuanwei Lu, Guo Chen, Bojie Li, Kun Tan, Yongqiang Xiong, Peng Cheng, Jian- song Zhang, Enhong Chen, and Thomas Moscibroda. 2018. Multi-Path Transport for RDMA in Datacenters. In15th USENIX Symposium on Networked Systems Design and Implementation (NSDI 18). USENIX Association,...

    2018

  17. [25]

    Radhika Mittal, Vinh The Lam, Nandita Dukkipati, Emily Blem, Hassan Wassel, Monia Ghobadi, Amin Vahdat, Yaogong Wang, David Wetherall, and David Zats

  18. [26]

    Radhika Mittal, Alexander Shpiner, Aurojit Panda, Eitan Zahavi, Arvind Krishna- murthy, Sylvia Ratnasamy, and Scott Shenker. 2018. Revisiting network support for RDMA. InProceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication(Budapest, Hungar...

    2018

  19. [27]

    ns-3 Project. 2025. ns-3: A Discrete-Event Network Simulator. https://www. nsnam.org/. Accessed: 2025-05-02

    2025

  20. [28]

    NVIDIA. 2024. ZTR-RTT Congestion Control Algorithm Overview v1.0. https://docs.nvidia.com/networking/display/ ztrrttcongestioncontrolalgorithmoverviewv10 Accessed: 2025-05-16

    2024

  21. [29]

    NVIDIA. [n.d.]. ConnectX-5 SmartNIC Adapter. https://www.nvidia.com/en- sg/networking/ethernet/connectx-5/. Accessed: 2025-05-28

    2025

  22. [30]

    NVIDIA. [n.d.]. ConnectX-6 SmartNIC Adapter. https://www.nvidia.com/en- sg/networking/ethernet/connectx-6/. Accessed: 2025-05-28

    2025

  23. [31]

    DPDK Project. 2025. Data Plane Development Kit (DPDK). https://www.dpdk. org/. Accessed: 2025-05-02

    2025

  24. [32]

    SONiC Project. 2025. Software for Open Networking in the Cloud. https:// sonicfoundation.dev/. Accessed: 2025-06-28

    2025

  25. [33]

    Kun Qian, Yongqing Xi, Jiamin Cao, Jiaqi Gao, Yichi Xu, Yu Guan, Binzhang Fu, Xuemei Shi, Fangbo Zhu, Rui Miao, Chao Wang, Peng Wang, Pengcheng Zhang, Xianlong Zeng, Eddie Ruan, Zhiping Yao, Ennan Zhai, and Dennis Cai

  26. [34]

    Mubashir Adnan Qureshi, Yuchung Cheng, Qianwen Yin, Qiaobin Fu, Gautam Kumar, Masoud Moshref, Junhua Yan, Van Jacobson, David Wetherall, and Abdul Kabbani. 2022. PLB: congestion signals are simple and effective for network load balancing. InProceedings of the Conference of the...

    2022

  27. [35]

    2024.Conges- tion Signaling (CSIG)

    Abhiram Ravi, Nandita Dukkipati, Naoshad Mehta, and Jai Kumar. 2024.Conges- tion Signaling (CSIG). Internet-Draft draft-ravi-ippm-csig-01. Internet Engineer- ing Task Force. https://datatracker.ietf.org/doc/draft-ravi-ippm-csig/01/ Work in Progress

    2024

  28. [36]

    Arjun Roy, Hongyi Zeng, Jasmeet Bagga, George Porter, and Alex C. Snoeren

  29. [37]

    Cha Hwan Song, Xin Zhe Khooi, Raj Joshi, Inho Choi, Jialin Li, and Mun Choon Chan. 2023. Network Load Balancing with In-network Reordering Support for RDMA. InProceedings of the Conference of the ACM Special Interest Group on Data Communication(New York, NY, USA)(ACM SIGCOMM ’...

    2023

  30. [38]

    Erico Vanini, Rong Pan, Mohammad Alizadeh, Parvin Taheri, and Tom Edsall

  31. [39]

    Zirui Wan, Jiao Zhang, Mingxuan Yu, Junwei Liu, Jun Yao, Xinghua Zhao, and Tao Huang. 2024. BiCC: Bilateral Congestion Control in Cross-datacenter RDMA Networks. InIEEE INFOCOM 2024 - IEEE Conference on Computer Communications. 1381–1390. https://doi.org/10.1109/INFOCOM52122.2...

    2024

  32. [40]

    Zilong Wang, Layong Luo, Qingsong Ning, Chaoliang Zeng, Wenxue Li, Xinchen Wan, Peng Xie, Tao Feng, Ke Cheng, Xiongfei Geng, Tianhao Wang, Weicheng Ling, Kejia Huo, Pingbo An, Kui Ji, Shideng Zhang, Bin Xu, Ruiqing Feng, Tao Ding, Kai Chen, and Chuanxiong Guo. 2023. SRNIC: A S...

    2023

  33. [41]

    Inside the Social Network’s (Datacenter) Network.SIGCOMM Comput. Commun. Rev.45, 4 (Aug. 2015), 123–137

    2015

  34. [42]

    Yunhong Xu, Keqiang He, Rui Wang, Minlan Yu, Nick Duffield, Hassan Wassel, Shidong Zhang, Leon Poutievski, Junlan Zhou, and Amin Vahdat. 2022. Hashing Design in Modern Networks: Challenges and Mitigation Techniques. In2022 USENIX Annual Technical Conference (USENIX ATC 22). US...

    2022

  35. [43]

    Hong Zhang, Junxue Zhang, Wei Bai, Kai Chen, and Mosharaf Chowdhury. 2017. Resilient Datacenter Load Balancing in the Wild. InProceedings of the Conference of the ACM Special Interest Group on Data Communication(Los Angeles, CA, USA)(SIGCOMM ’17). Association for Computing Mac...

    2017

  36. [44]

    Jiao Zhang, Yuqing Wang, Xiaolong Zhong, Mingxuan Yu, Haoyu Pan, Yali Zhang, Zixuan Guan, Biyao Che, Zirui Wan, Tian Pan, and Tao Huang. 2024. PACC: A Proactive CNP Generation Scheme for Datacenter Networks.IEEE/ACM Trans. Netw.32, 3 (Feb. 2024), 2586–2599. https://doi.org/10....

    2024

  37. [45]

    Yanli Zhao, Andrew Gu, Rohan Varma, Liang Luo, Chien-Chin Huang, Min Xu, Less Wright, Hamid Shojanazeri, Myle Ott, Sam Shleifer, Alban Desmaison, Can Balioglu, Pritam Damania, Bernard Nguyen, Geeta Chauhan, Yuchen Hao, Ajit Mathews, and Shen Li. 2023. PyTorch FSDP: Experiences...

    2023 arXiv

  38. [46]

    Yibo Zhu, Haggai Eran, Daniel Firestone, Chuanxiong Guo, Marina Lipshteyn, Yehonatan Liron, Jitendra Padhye, Shachar Raindel, Mohamad Haj Yahia, and Ming Zhang. 2015. Congestion Control for Large-Scale RDMA Deployments. In Proceedings of the 2015 ACM Conference on Special Inte...

    2015

  39. [47]

    William Won, Taekyung Heo, Saeed Rashidi, Srinivas Sridharan, Sudarshan Srinivasan, and Tushar Krishna. 2023. ASTRA-sim2.0: Modeling Hierarchical Networks and Disaggregated Systems for Large-model Training at Scale. In2023 IEEE International Symposium on Performance Analysis o...

    2023

  40. [2015]

    InProceedings of the 2015 ACM Conference on Special Interest Group on Data Communication(Lon- don, United Kingdom)(SIGCOMM ’15)

    TIMELY: RTT-based Congestion Control for the Datacenter. InProceedings of the 2015 ACM Conference on Special Interest Group on Data Communication(Lon- don, United Kingdom)(SIGCOMM ’15). Association for Computing Machinery, New York, NY, USA, 537–550

    2015

  41. [2017]

    In14th USENIX Symposium on Networked Systems Design and Implementation (NSDI 17)

    Let It Flow: Resilient Asymmetric Load Balancing with Flowlet Switching. In14th USENIX Symposium on Networked Systems Design and Implementation (NSDI 17). USENIX Association, Boston, MA, 407–420

  42. [2019]

    InProceedings of the ACM Special Interest Group on Data Communication(Beijing, China)(SIGCOMM ’19)

    HPCC: high precision congestion control. InProceedings of the ACM Special Interest Group on Data Communication(Beijing, China)(SIGCOMM ’19). Association for Computing Machinery, New York, NY, USA, 44–58

  43. [2020]

    arXiv:2006.15704 [cs.DC] https://arxiv.org/abs/2006.15704

    PyTorch Distributed: Experiences on Accelerating Data Parallel Training. arXiv:2006.15704 [cs.DC] https://arxiv.org/abs/2006.15704

    2006 arXiv

  44. [2024]

    InProceedings of the ACM SIGCOMM 2024 Conference(Sydney, NSW, Australia) (ACM SIGCOMM ’24)

    Alibaba HPN: A Data Center Network for Large Language Model Training. InProceedings of the ACM SIGCOMM 2024 Conference(Sydney, NSW, Australia) (ACM SIGCOMM ’24). Association for Computing Machinery, New York, NY, USA, 691–706

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.