NetSenseML: Network-Adaptive Compression for Efficient Distributed Machine Learning

Dirk Kutscher; Huangxun Chen; Ruilong Wu; Xinjiao Li; Yisu Wang

REVIEW 4 major objections 6 minor 24 references

NetSenseML: Network-Adaptive Compression for Efficient Distributed Machine Learning

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read NetSenseML claims that holding compressed gradient size near the network's bandwidth-delay product keeps distributed training fast and accurate, yielding 1.55–9.84× throughput gains over static compression under bandwidth limits.

desk verdict Plausible network-adaptive compression design, but the evaluation's asymmetric stopping rule breaks the accuracy/TTA claim; throughput gains are measured and likely real. read the letter →

arxiv 2506.16235 v1 pith:PCLXMKV4 submitted 2025-06-19 cs.DC

Yisu Wang , Xinjiao Li , Ruilong Wu , Huangxun Chen , Dirk Kutscher This is my paper · ORCID

classification cs.DC

keywords network-adaptivecompressiongradientdistributeddeeplearningbandwidth-delayproducttime-to-accuracyquantizationsparsificationcongestioncontrol

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

NetSenseML aims to settle the central trade-off of distributed training over wide-area networks: gradient compression reduces communication but normally costs model accuracy. The paper's claim is that compression need not be static — by continuously estimating the network's bandwidth-delay product (BDP) and compressing gradients just enough to keep transmitted data below $0.9 \times BDP$, a training system can avoid congestion while sending the maximum possible gradient information. The compression ratio rises when the network is idle and falls when it is congested, and quantization, pruning, and sparsification are applied in proportion to that ratio. On an eight-worker testbed training ResNet18 and VGG16 over 200 Mbps to 10 Gbps links, the paper reports 1.55–9.84× training-throughput gains and faster time-to-accuracy than AllReduce and fixed-rate TopK compression. If the claim holds, adaptive network sensing turns bandwidth constraints from a training blocker into a manageable, automatic condition.

What carries the argument

The carrying mechanism is a bandwidth-delay-product feedback loop: estimate $\mathit{BtlBw}$ as the maximum of $\mathrm{data\_size}/RTT$ over recent intervals, $\mathit{RTprop}$ as the minimum RTT, set $BDP = \mathit{BtlBw} \times \mathit{RTprop}$, and adjust the compression ratio so that the compressed gradient size stays near $0.9 \times BDP$ rather than exceeding it (Eqs. 1–3). This single ratio then governs a three-stage reduction pipeline — adaptive quantization gated by the gradient L2 norm, weight pruning at rate $0.5 \times (1 - \mathrm{ratio})$, and TopK sparsification at the ratio itself — so that the fidelity of transmitted gradients scales with the network headroom detected by the sensing phase.

What would settle it

Run NetSenseML on the paper's testbed while instrumenting the bottleneck switch's queue depth: if compressed gradient bursts at the $0.9 \times BDP$ target still produce growing queues, rising RTT, or packet loss during synchronized all-reduce — or if the same workloads congest on high-latency links where BDP is large — then the capacity model is miscalibrated and the ratio rule fails.

Watch

Extended reading notes

Core claim

The paper's discovery is that congestion, not raw bandwidth, is the right target for gradient compression. NetSenseML measures each gradient transmission interval, estimates bottleneck bandwidth as $EBB = \mathrm{data\_size}/RTT$, tracks its maximum as $\mathit{BtlBw}$ and the minimum RTT as $\mathit{RTprop}$, and derives the bandwidth-delay product $BDP = \mathit{BtlBw} \times \mathit{RTprop}$. The compression ratio is reduced multiplicatively ($\times 0.5$) whenever the transmitted data size exceeds $0.9 \times BDP$ and increased additively ($+0.01$) otherwise, so gradient payloads track network capacity in real time. The same ratio drives the compression pipeline: adaptive 16-bit quantization kicks in below a ratio threshold when the gradient L2 norm is high, pruning zeros out small-weight gradients at rate $0.5 \times (1 - \mathrm{ratio})$, and TopK sparsification retains only the largest-magnitude gradients. This, the paper argues, preserves essential gradient information under extreme constraints and avoids both the throughput collapse of un-compressed AllReduce and the accuracy drift of fixed-rate compression.

Load-bearing premise

The load-bearing premise is that the bandwidth-delay product, estimated as the maximum of $\mathrm{data\_size}/RTT$ times the minimum RTT, accurately predicts the capacity available for gradient transfers, and that keeping compressed data near $0.9 \times BDP$ prevents congestion for the bursty, synchronized all-reduce traffic of distributed training.

Editorial extensions

If this is right

Bandwidth-constrained and fluctuating WAN links no longer require manual tuning of compression rates; the training job adapts on its own.
Time-to-accuracy, not raw throughput, becomes the quantity the system optimizes, and compression is applied only when the network is the bottleneck.
When bandwidth is plentiful, the feedback loop raises the compression ratio toward 1, so unnecessary accuracy loss is avoided.
Fixed-volume schemes like AllReduce and static TopK degrade as capacity shrinks because they cannot reduce their data footprint, which is precisely the failure NetSenseML targets.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The $0.9 \times BDP$ target is inherited from single-flow congestion control; an unstated extension would be to test whether synchronized all-reduce, where many workers' gradients arrive at once, needs a lower target to avoid transient queue buildup.
The evaluation covers two vision models on one dataset; extending the controller to transformer workloads with much larger gradient tensors would test whether the L2-norm and density thresholds still hold.
Because the ratio is corrected using the previous transmission interval, the controller could lag sudden bandwidth drops; coupling it with explicit congestion signals such as ECN would give faster reaction.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

NetSenseML is a network-adaptive gradient compression framework for distributed training that uses BBR-style bandwidth and RTT estimates to set the compression ratio. The combination of BtlBw/RTprop-based BDP tracking with quantization, pruning, and TopK sparsification is not present in DC2, Crux, or Espresso, so the integration is new. The design is clean and the intent—keep compressed gradient size near 0.9*BDP to avoid queue buildup—is a sensible adaptation of congestion control thinking to the all-reduce pattern. The dynamic bandwidth scenarios show behavior that looks responsive and stable.

What the paper does well: the algorithms are easy to follow, the problem is real, and the throughput measurements are direct. I expect the 1.55–9.84x throughput advantage under low bandwidth is directionally correct, because the system is actively reducing payload while baselines are not.

The soft spots are mostly in the evaluation. The biggest problem is Section 5.3: the baselines are terminated at the moment NetSenseML reaches its best accuracy. That makes every accuracy and time-to-accuracy number for AllReduce and TopK uninterpretable. The N/A convergence times in Tables 1 and 2 confirm the baselines were not allowed to converge, so the paper's central claim that NetSenseML preserves accuracy while improving TTA is not supported. This is a load-bearing flaw, not a cosmetic one.

Secondary issues: there are no error bars or multiple seeds; the comparison does not include DC2 or Crux, which are cited as state-of-the-art adaptive compression systems; and the hand-tuned thresholds (trq, trd, alpha, beta2) have no sensitivity analysis. The BDP-based model is adopted from BBR without validation for bursty, synchronized all-reduce traffic, which is a reasonable concern but not fatal—the throughput results depend less on that model than the accuracy claim does.

Who this is for: researchers working on network-aware distributed training or gradient compression. The paper has a useful idea and a readable write-up, but the evaluation would need major revision before the accuracy claims are credible. I would not reject the idea; I would send it to peer review with the expectation that the authors redo the comparison fairly (let baselines converge, add seeds and error bars, and ideally compare against DC2 or Crux). The throughput result alone is worth reporting, but the headline claim needs better support.

Referee Report

4 major / 6 minor

Summary. The paper presents NetSenseML, a network-adaptive gradient compression framework for distributed deep learning. The system estimates bottleneck bandwidth and round-trip propagation time from past gradient transmissions (Eqs. 1–2), computes a bandwidth-delay product, and adjusts a compression ratio to keep the compressed gradient size near 0.9×BDP (Eq. 3). The compression pipeline adaptively applies quantization, pruning, and TopK sparsification based on the ratio and gradient L2 norm (Algorithm 2). The authors evaluate on an eight-worker PyTorch DDP testbed with ResNet18 and VGG16 on CIFAR-100 under static, degrading, and fluctuating bandwidth conditions, reporting time-to-accuracy, convergence time, training throughput, and best test accuracy. They claim 1.55–9.84× throughput improvement over TopK and AllReduce in bandwidth-constrained conditions.

Significance. If fully supported, NetSenseML would be a practical contribution to bandwidth-constrained distributed training. The paper has real strengths: a working prototype on PyTorch DDP/NCCL, a real testbed, and direct throughput measurements that are partly convincing. The adaptive design is plausibly motivated by BBR's BDP concept. However, the central evaluation claim—that NetSenseML preserves accuracy while improving time-to-accuracy—is not supported as written. The stopping rule for baselines invalidates the accuracy and TTA comparisons; there are no error bars or repeated seeds; and the system is not compared with the adaptive compression baselines (DC2, Crux) it cites. The throughput advantage is likely directionally correct, but the paper as submitted does not demonstrate the advertised balance between throughput and accuracy, and the performance of the adaptive rule is not separated from the effect of a favorable stopping point.

major comments (4)

[§5.3, Tables 1–2] The accuracy comparison is invalid. The text states: 'We use the point at which NetSenseML achieves its best test accuracy as the benchmark and terminate the training of AllReduce and TopK at that point.' Therefore the reported 'Best Test Accuracy' for AllReduce (e.g., 67.34% at 200 Mbps in Table 1) and TopK (66.52%) are accuracies at an early stop chosen by NetSenseML's trajectory, not converged accuracies. The N/A convergence-time entries confirm that the baselines were not allowed to finish. This stopping rule makes the time-to-accuracy and accuracy-preservation claims uninterpretable, since a favorable TTA could arise simply from truncating the baselines. The experiments must be rerun with each method trained to full convergence or to an independently defined target accuracy.
[§5, Tables 1–2 and Figs. 5–8] No variance or repeated-seed information is reported. All throughput, accuracy, and convergence-time numbers appear to come from single runs, with no error bars or statistical tests. This is a load-bearing problem because the central claims include stability and robustness (e.g., the fluctuating-bandwidth scenario), and because the adaptive controller depends on stochastic network measurements. Without multiple seeds and variance estimates, the claimed 1.55–9.84× throughput range and the claimed accuracy preservation cannot be assessed.
[§4.1, Eqs. (1)–(3)] The adaptive control rule is not validated for the bursty, synchronized all-reduce traffic of distributed training. Equations (1)–(3) directly transplant BBR's BDP logic, but BBR paces packet transmissions over a connection, whereas DDP gradient transfers are bursts triggered by collective communication. The paper offers no evidence that data_size > 0.9×BDP during a gradient transfer is the correct congestion signal, no direct measurement of queue buildup or RTT inflation, and no sensitivity study for the thresholds (0.9, α=0.5, β2=0.01). If this model fails, the compression ratio is set incorrectly and the claimed balance between throughput and accuracy collapses.
[§3, §5.2] The baseline selection undermines the 'state-of-the-art' claim. Related Work introduces DC2 and Crux as adaptive compression systems, but neither appears in the evaluation; the only baselines are AllReduce and a fixed TopK-0.1. Additionally, key free parameters—initial ratio 0.01, trq, trd, pruning factor 0.5, α, and β2—are fixed without a sensitivity analysis. Consequently, the results do not establish superiority over state-of-the-art adaptive methods, and it is unclear how much of the reported gain depends on hand-tuned parameter choices.

minor comments (6)

[Abstract] The abstract contains a formatting artifact: 'Ourapproachensuresefficientresourceusagebyadaptingreductiontechniques...' should be separated into words.
[Fig. 3] Figure 3 appears garbled (e.g., 'qtrratio<'), which makes the adaptive quantization flow in Algorithm 2 difficult to follow.
[§4.1, Eq. (1)] Algorithm 1 defines EBB using data_size_i−1 / RTT_i−1, but it is not stated whether data_size refers to the uncompressed gradient size or the size after compression; this distinction is essential for interpreting the BDP comparison in Eq. (3).
[§4.2, Algorithm 2] The interaction between the quantization step and the sparsification step is unclear: after quantization, ratio is doubled, and then ratio is used both for pruning_rate = 0.5 × (1 − ratio) and for the TopK sparsification ratio. The semantics of 'ratio' as a communication budget versus a sparsification fraction should be clarified.
[§5.1] The definition of convergence time as 'the time required for the model's accuracy to stabilize at a target threshold' is vague; no target threshold value is given, which makes the N/A entries and the convergence times in Tables 1–2 hard to interpret.
[§5.3] The text says the ResNet18 model size is 46.2MB, making AllReduce impractical without at least 500 Mbps, yet AllReduce is still evaluated at 200 Mbps in Fig. 5 and Table 1; this inconsistency should be explained or corrected.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: throughput and convergence results are direct measurements against external baselines, and the adaptive ratio is a feedback controller, not a quantity defined by the claimed outcome.

full rationale

This paper does not derive its headline throughput or accuracy results from a first-principles model whose output equals its input by construction. The adaptive compression ratio is a feedback loop: it estimates BtlBw and RTprop from observed transmission intervals, computes BDP (Eqs. 1-2), and adjusts the ratio toward 0.9 BDP (Eq. 3). The ratio influences compressed gradient size, which then feeds the next measurement; this is closed-loop control, not a circular derivation. No load-bearing argument relies on author self-citations, a uniqueness theorem, or an imported ansatz that is itself the conclusion. The Sec. 5.3 practice of stopping AllReduce and TopK at the point where NetSenseML reaches its best test accuracy is a methodological concern for the accuracy and time-to-accuracy comparisons, but it is not an equation-level circularity: the reported throughput values are measured against external baselines rather than being forced by the algorithm's definition. Therefore no significant circularity is found, and the score is 0.

Assumptions & free parameters 6 free parameters · 3 assumptions · 0 invented entities

The system has no invented physical entities. The load-bearing assumptions are the BBR-style network model, the convergence-preservation property of TopK, and the L2-norm-based quantization decision. The hand-tuned hyperparameters alpha, beta2, initial ratio, trq, trd, and the pruning factor are free parameters that are not justified experimentally.

free parameters (6)

alpha (multiplicative decrease factor) = 0.5
Used in Eq. (3) to reduce compression ratio when data_size exceeds 0.9*BDP. Hand-chosen with no sensitivity analysis.
beta2 (additive increase step) = 0.01
Used in Eq. (3) to increase compression ratio when network is underutilized. Hand-chosen.
initial compression ratio = 0.01
Algorithm 1 starts with ratio 0.01 and increases it during startup. This aggressive starting point affects early training behavior.
trq (quantization check threshold) = not specified
Algorithm 2 only triggers quantization when ratio < trq. The value is not reported, making replication impossible.
trd (gradient density threshold) = not specified
Algorithm 2 uses L2 > trd to decide whether to quantize. The value is not reported.
pruning rate factor 0.5 = 0.5
pruning_rate = 0.5 * (1 - ratio) in Algorithm 2. The 0.5 factor is hand-chosen.

assumptions (3)

domain assumption Bottleneck bandwidth and RTprop estimation from BBR (Eq. 1-2) apply to gradient traffic.
The paper adopts BBR's model of network capacity without validating it for the synchronized, bursty all-reduce traffic of distributed training.
domain assumption TopK sparsification preserves convergence.
The sparsification step relies on prior work (Aji and Heafield) and on the assumption that keeping the largest-magnitude gradients is sufficient for convergence.
domain assumption The L2 norm of a gradient is a valid indicator of its informativeness for quantization.
Algorithm 2 uses L2 > trd to decide when to quantize, assuming that high-L2 gradients are worth preserving at reduced precision.

how reviews work

0 comments

Cite this review

Pith. "Pith review of NetSenseML: Network-Adaptive Compression for Efficient Distributed Machine Learning." pith.science (2026). https://pith.science/paper/PCLXMKV4

@misc{pith2026250616235,
  author       = {Pith},
  title        = {Pith review of: NetSenseML: Network-Adaptive Compression for Efficient Distributed Machine Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PCLXMKV4}},
  note         = {Machine review of arXiv:2506.16235}
}

read the original abstract

Training large-scale distributed machine learning models imposes considerable demands on network infrastructure, often resulting in sudden traffic spikes that lead to congestion, increased latency, and reduced throughput, which would ultimately affect convergence times and overall training performance. While gradient compression techniques are commonly employed to alleviate network load, they frequently compromise model accuracy due to the loss of gradient information. This paper introduces NetSenseML, a novel network adaptive distributed deep learning framework that dynamically adjusts quantization, pruning, and compression strategies in response to real-time network conditions. By actively monitoring network conditions, NetSenseML applies gradient compression only when network congestion negatively impacts convergence speed, thus effectively balancing data payload reduction and model accuracy preservation. Our approach ensures efficient resource usage by adapting reduction techniques based on current network conditions, leading to shorter convergence times and improved training efficiency. We present the design of the NetSenseML adaptive data reduction function and experimental evaluations show that NetSenseML can improve training throughput by a factor of 1.55 to 9.84 times compared to state-of-the-art compression-enabled systems for representative DDL training jobs in bandwidth-constrained conditions.

Figures

Figures reproduced from arXiv: 2506.16235 by the authors.

**Figure 1.** NetSenseML Overview 3 Related Work Modern distributed machine learning frameworks, such as PyTorch DDP [4], generate a substantial volume of data packets that accumulate in communication queues, slowing down iteration speed. To address this challenge, DC2 [1] leverages a delay monitoring component to track communication latency for each training iteration, enabling dynamic adjustment of gradient compression. Crux [… view at source ↗

**Figure 2.** Network status sensing based on BtlBw and RTprop. In this phase, the network bandwidth is underutilized. However, as the size of the transmitted data exceeds the BDP, the RTT increases significantly, indicating a state of network congestion where the data sent surpass the network’s transmission capacity, i.e., BtlBw. Our goal is to adjust the compression ratio so that the size of the compressed gradients appr… view at source ↗

**Figure 3.** Adaptive quantization based on L2 norm. To enhance the precision and accelerate the convergence of the model, we have developed an adaptive compression method that incorporates quantization, pruning, and sparsification, as shown in Algorithm . 2. This method optimizes gradient transmission under challenging network conditions through quantization, boosts model generalizability via pruning, and maintains criti… view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: Evaluation topology. Metrics: We define time-to-accuracy (TTA) as the training time required to reach a target validation accuracy. Additionally, we present the training throughput (measured as images per second, referred to as samples per second) across all training t…

**Figure 5.** Figure 5: Time to accuracy (TTA) comparison under different bottleneck band [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]

**Figure 6.** Figure 6: Time to accuracy (TTA) comparison under different bottleneck band [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]

**Figure 8.** Figure 8: Impact of Bandwidth Fluctuation on Training Throughput of ResNet18 ent levels of bandwidth. Using the ResNet18 model, we gradually decreased the bottleneck bandwidth - from 2000 to 200 Mbps in steps of 200 Mbps - and measured training performance at each level. This …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

24 extracted references · 20 canonical work pages

[1]

In: IEEE INFOCOM 2021 - IEEE Conference on Com- puter Communications

Abdelmoniem, A.M., Canini, M.: Dc2: Delay-aware compression control for dis- tributed machine learning. In: IEEE INFOCOM 2021 - IEEE Conference on Com- puter Communications. pp. 1–10 (2021) 14 Y. Wang et al

work page 2021
[2]

In: Palmer, M., Hwa, R., Riedel, S

Aji, A.F., Heafield, K.: Sparse communication for distributed gradient descent. In: Palmer, M., Hwa, R., Riedel, S. (eds.) Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. pp. 440–445. Association for Computational Linguistics, Copenhagen, Denmark (Sep 2017)

work page 2017
[3]

SIGCOMM Comput

Alizadeh, M., Greenberg, A., Maltz, D.A., Padhye, J., Patel, P., Prabhakar, B., Sengupta, S., Sridharan, M.: Data center tcp (dctcp). SIGCOMM Comput. Com- mun. Rev.40(4), 63–74 (Aug 2010)

work page 2010
[4]

Ansel, J., Yang, E., , et al.: Pytorch 2: Faster machine learning through dynamic python bytecode transformation and graph compilation. p. 929–947. ASPLOS ’24, Association for Computing Machinery, New York, NY, USA (2024)

work page 2024
[5]

CoRR abs/2005.14165 (2020)

Brown, T.B., Mann, B., Ryder, N., et al.: Language models are few-shot learners. CoRR abs/2005.14165 (2020)

arXiv 2020
[6]

Cao, J., Guan, Y., Qian, K., Gao, J., Xiao, W., Dong, J., Fu, B., Cai, D., Zhai, E.: Crux: Gpu-efficient communication scheduling for deep learning training. p. 1–15. ACM SIGCOMM ’24, Association for Computing Machinery, New York, NY, USA (2024)

work page 2024
[7]

Queue14(5), 20–53 (Oct 2016)

Cardwell, N., Cheng, Y., Gunn, C.S., Yeganeh, S.H., Jacobson, V.: Bbr: Congestion-based congestion control: Measuring bottleneck bandwidth and round- trip propagation time. Queue14(5), 20–53 (Oct 2016)

work page 2016
[8]

In: 2019 USENIX annual technical conference (USENIX ATC 19)

Duplyakin, D., Ricci, R., Maricq, A., Wong, G., Duerig, J., Eide, E., Stoller, L., Hibler, M., Johnson, D., Webb, K., et al.: The design and operation of{CloudLab}. In: 2019 USENIX annual technical conference (USENIX ATC 19). pp. 1–14 (2019)

work page 2019

Show all 24 references

[9]

ACM SIGOPS operating systems review42(5), 64–74 (2008)

Ha, S., Rhee, I., Xu, L.: Cubic: a new tcp-friendly high-speed tcp variant. ACM SIGOPS operating systems review42(5), 64–74 (2008)

2008
[10]

In: Advances in Neural Information Processing Systems (NeurIPS)

Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. In: Advances in Neural Information Processing Systems (NeurIPS). pp. 1135–1143 (2015)

2015
[11]

Han, S., Pool, J., Tran, J., Dally, W.J.: Learning both weights and connections for efficient neural networks (2015)

2015
[12]

Han, W., Vargaftik, S., Mitzenmacher, M., Karp, B., Basat, R.B.: Beyond through- put and compression ratios: Towards high end-to-end utility of gradient compres- sion (2024)

2024
[13]

He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition (2015), https://arxiv.org/abs/1512.03385

2015 arXiv
[14]

In: Proceedings of the Conference of the ACM Special Interest Group on Data Communication

Langley, A., Riddoch, A., Wilk, A., Vicente, A., Krasic, C., Zhang, D.e.a.: The quic transport protocol: Design and internet-scale deployment. In: Proceedings of the Conference of the ACM Special Interest Group on Data Communication. p. 183–196. SIGCOMM ’17, Association for Co...

2017
[15]

In: International Conference on Learning Representations (ICLR) (2016)

Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. In: International Conference on Learning Representations (ICLR) (2016)

2016
[16]

Li, M., Basat, R.B., Vargaftik, S., Lao, C., Xu, K., Mitzenmacher, M., Yu, M.: Thc:Acceleratingdistributeddeeplearningusingtensorhomomorphiccompression (2024)

2024
[17]

In: 11th USENIX Symposium on Operating Systems Design and Implemen- tation (OSDI 14)

Li, M., Andersen, D.G., Park, J.W., Smola, A.J., Ahmed, A., Josifovski, V., Long, J., Shekita, E.J., Su, B.Y.: Scaling distributed machine learning with the parameter server. In: 11th USENIX Symposium on Operating Systems Design and Implemen- tation (OSDI 14). pp. 583–598. USE...

2014
[18]

CoRR (2017) NetSenseML 15

Lin, Y., Han, S., Mao, H., Wang, Y., Dally, W.J.: Deep gradient compression: Reducing the communication bandwidth for distributed training. CoRR (2017) NetSenseML 15

2017
[19]

https://developer

NVIDIA: Nvidia collective communications library (nccl). https://developer. nvidia.com/nccl. (2024)

2024
[20]

Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition (2015),https://arxiv.org/abs/1409.1556

2015 arXiv
[21]

https: //github.com/esnet/iperf (2024)

iPerf Team: iperf - the ultimate speed test tool for tcp, udp and sctp. https: //github.com/esnet/iperf (2024)

2024
[22]

Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., Rodriguez, A., Joulin, A., Grave, E., Lample, G.: Llama: Open and efficient foundation language models (2023)

2023
[23]

In: 21st USENIX Symposium on Networked Systems Design and Implementation (NSDI 24)

Wang, H., Tian, H., Chen, J., Wan, X., Xia, J., et al.: Towards Domain-Specific network transport for distributed DNN training. In: 21st USENIX Symposium on Networked Systems Design and Implementation (NSDI 24). pp. 1421–1443. USENIX Association, Santa Clara, CA (Apr 2024)

2024
[24]

In: Proceedings of the Eighteenth European Conference on Computer Systems

Wang, Z., Lin, H., Zhu, Y., Ng, T.E.: Hi-speed dnn training with espresso: Unleash- ing the full potential of gradient compression with near-optimal usage strategies. In: Proceedings of the Eighteenth European Conference on Computer Systems. pp. 867–882 (2023)

2023

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

In: IEEE INFOCOM 2021 - IEEE Conference on Com- puter Communications

Abdelmoniem, A.M., Canini, M.: Dc2: Delay-aware compression control for dis- tributed machine learning. In: IEEE INFOCOM 2021 - IEEE Conference on Com- puter Communications. pp. 1–10 (2021) 14 Y. Wang et al

work page 2021

[2] [2]

In: Palmer, M., Hwa, R., Riedel, S

Aji, A.F., Heafield, K.: Sparse communication for distributed gradient descent. In: Palmer, M., Hwa, R., Riedel, S. (eds.) Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. pp. 440–445. Association for Computational Linguistics, Copenhagen, Denmark (Sep 2017)

work page 2017

[3] [3]

SIGCOMM Comput

Alizadeh, M., Greenberg, A., Maltz, D.A., Padhye, J., Patel, P., Prabhakar, B., Sengupta, S., Sridharan, M.: Data center tcp (dctcp). SIGCOMM Comput. Com- mun. Rev.40(4), 63–74 (Aug 2010)

work page 2010

[4] [4]

Ansel, J., Yang, E., , et al.: Pytorch 2: Faster machine learning through dynamic python bytecode transformation and graph compilation. p. 929–947. ASPLOS ’24, Association for Computing Machinery, New York, NY, USA (2024)

work page 2024

[5] [5]

CoRR abs/2005.14165 (2020)

Brown, T.B., Mann, B., Ryder, N., et al.: Language models are few-shot learners. CoRR abs/2005.14165 (2020)

arXiv 2020

[6] [6]

Cao, J., Guan, Y., Qian, K., Gao, J., Xiao, W., Dong, J., Fu, B., Cai, D., Zhai, E.: Crux: Gpu-efficient communication scheduling for deep learning training. p. 1–15. ACM SIGCOMM ’24, Association for Computing Machinery, New York, NY, USA (2024)

work page 2024

[7] [7]

Queue14(5), 20–53 (Oct 2016)

Cardwell, N., Cheng, Y., Gunn, C.S., Yeganeh, S.H., Jacobson, V.: Bbr: Congestion-based congestion control: Measuring bottleneck bandwidth and round- trip propagation time. Queue14(5), 20–53 (Oct 2016)

work page 2016

[8] [8]

In: 2019 USENIX annual technical conference (USENIX ATC 19)

Duplyakin, D., Ricci, R., Maricq, A., Wong, G., Duerig, J., Eide, E., Stoller, L., Hibler, M., Johnson, D., Webb, K., et al.: The design and operation of{CloudLab}. In: 2019 USENIX annual technical conference (USENIX ATC 19). pp. 1–14 (2019)

work page 2019