pith. sign in

arxiv: 2605.00755 · v2 · submitted 2026-05-01 · 💻 cs.NI

AdvNet: Revealing Performance Issues in Network Protocols by Generating Adversarial Environments

Shehab Sarar Ahmed , William Sentosa , Yinjie Zhang , Yoav Lebendiker , Michael Shnaiderman , Tomer Gilad , Nathan H. Jay , Brighten Godfrey
show 1 more author
Michael Schapira
This is my paper

Pith reviewed 2026-05-09 18:21 UTC · model grok-4.3

classification 💻 cs.NI
keywords adversarial testingcongestion controlnetwork protocolsmachine learning optimizationprotocol robustnessLinux kernel bugstransport protocols
0
0 comments X

The pith

AdvNet generates adversarial network environments to expose performance issues and bugs in congestion control protocols.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

AdvNet is a system that uses machine learning optimization to automatically create network conditions where congestion control protocols perform poorly. Traditional testing relies on hand-designed cases or real data captures, which can miss unexpected environments. By focusing on transport protocols and their implementations, the approach identifies problematic conditions that expose Linux kernel bugs and limitations in the algorithms. This matters because reliable protocol performance across diverse Internet environments is essential, and missed cases can lead to real-world failures. The results indicate that automated adversarial testing offers a new way to improve protocol robustness.

Core claim

AdvNet employs machine learning-based optimization to generate adversarial network environments, incorporating a robust noise-handling mechanism to mitigate performance variability, and applies this to 27 kernel-space implementations of single-path and multi-path congestion control protocols across several use cases to identify problematic network conditions, expose previously unnoticed Linux kernel bugs, uncover hidden limitations in the implementations, and provide insights about robustness.

What carries the argument

AdvNet, a system that employs machine learning-based optimization with noise-handling to generate adversarial network environments that cause target protocol implementations to perform poorly.

If this is right

  • Identifies problematic network conditions that expose previously unnoticed Linux kernel bugs in congestion control implementations.
  • Uncovers hidden limitations in CC implementations for both single-path and multi-path variants.
  • Provides concrete insights about the robustness of these protocols under stress.
  • Positions automated adversarial testing as a valuable addition to protocol development processes.
  • Establishes robustness as a useful new dimension for benchmarking CC protocols.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same generation approach could extend to infrastructure protocols outside of transport and congestion control.
  • Protocol designers could incorporate this style of testing into iterative development to catch issues earlier.
  • Focusing on robustness metrics might shift how protocols are evaluated beyond average-case performance.

Load-bearing premise

The adversarial environments produced by the optimization process reflect meaningful real-world protocol behaviors rather than artifacts specific to the simulation or setup.

What would settle it

Reproducing the generated adversarial conditions in a physical network testbed or live deployment and verifying whether the same performance degradations and bugs appear.

Figures

Figures reproduced from arXiv: 2605.00755 by Brighten Godfrey, Michael Schapira, Michael Shnaiderman, Nathan H. Jay, Shehab Sarar Ahmed, Tomer Gilad, William Sentosa, Yinjie Zhang, Yoav Lebendiker.

Figure 1
Figure 1. Figure 1: High-level architecture of AdvNet. Intuitively, an adversarial environment is one where target performs poorly, despite having the potential to improve its performance. To quantify this, AdvNet allows the user to spec￾ify a reference measure of performance (hereby referred as reference) that can be either a con￾crete protocol execution, or a calculation. Ad￾vNet’s adversary then attempts to generate en￾vir… view at source ↗
Figure 2
Figure 2. Figure 2: Mean score achieved by GA+MRE under different time allocations for the PLS phase. In view at source ↗
Figure 3
Figure 3. Figure 3: Performance comparison of optimization algorithms and the impact of the PLS phase. view at source ↗
Figure 4
Figure 4. Figure 4: Pairwise robustness comparison of 17 TCP protocols. bbr as target to yield scores above 0.8. Similarly, vegas, which relies on delay as a congestion signal, also becomes an easy target under this setting, achieving scores above 0.9 regardless of the choice of reference view at source ↗
Figure 5
Figure 5. Figure 5: Mean score of each protocol as target for 𝑡coeff = 0.5 and 𝑡coeff = 1. 4.4 Looking Inside the Box In this section, we conduct a detailed analysis of the adversarial environments generated by AdvNet across different experiments. We aim to understand the under￾lying causes of adversarial behavior and, where possible, propose potential fixes to mitigate these issues. 4.4.1 Case 1: bbr. AdvNet successfully ide… view at source ↗
Figure 6
Figure 6. Figure 6: Adversarial environment for bbr v3 view at source ↗
Figure 7
Figure 7. Figure 7: Bitrates across time for DChan￾nel with different values of 𝛼 and with only eMBB. We compared the throughput over time of DChannel (with different values of its internal 𝛼 parameter) and a single HBP (labeled “only eMBB”) in view at source ↗
Figure 8
Figure 8. Figure 8: Number of bytes transmitted over time by balia1 and balia2 view at source ↗
Figure 9
Figure 9. Figure 9: The effect of number of parallel executions on experienced throughput and delay. view at source ↗
Figure 10
Figure 10. Figure 10: Effect of parallelism on achieved score by GA view at source ↗
Figure 11
Figure 11. Figure 11: Mean score achieved by different selection view at source ↗
read the original abstract

Infrastructure protocols like Congestion Control (CC) seek to provide reliable performance across a wide range of Internet environments. Currently, protocol designers assess performance through hand-designed test cases or data sets captured from real environments. However, such approaches may inadvertently overlook critical facets of the algorithm's behavior when they encounter an unanticipated environment or workload. We seek to understand the unanticipated with AdvNet, a system that automatically generates adversarial network environments that cause a target protocol implementation to perform poorly. AdvNet employs machine learning-based optimization to generate environments, and incorporates a robust noise-handling mechanism to mitigate the variability inherent in real-world protocol performance. Although our approach is more general, this paper focuses specifically on transport protocols and their CC implementations. We showcase AdvNet's capability to create adversarial scenarios for 27 kernel-space implementations of both single-path and multi-path CC protocols, for several use cases with different performance goals. AdvNet identifies problematic network conditions that expose previously unnoticed Linux kernel bugs and uncovers hidden limitations in CC implementations, and provides insights about robustness. These results suggest that automated adversarial testing can be a valuable tool in protocol development, and that robustness is a useful new dimension for benchmarking CC protocols.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper introduces AdvNet, an ML-based optimization system that automatically generates adversarial network environments (varying bandwidth, delay, loss, etc.) to expose poor performance in congestion control (CC) protocol implementations. Focused on transport-layer protocols, it evaluates the approach on 27 kernel-space single-path and multi-path CC implementations across multiple use cases, claims to uncover previously unnoticed Linux kernel bugs and hidden CC limitations, and argues that automated adversarial testing plus robustness benchmarking can improve protocol development.

Significance. If the central claims hold, the work would be significant for the networking community by shifting protocol testing from hand-crafted cases to automated search over environment spaces, potentially revealing robustness issues missed by conventional methods. The evaluation across 27 real kernel implementations is a concrete strength, as is the framing of robustness as an explicit benchmarking dimension. The noise-handling mechanism, if effective, addresses a practical challenge in protocol performance measurement.

major comments (2)
  1. [Evaluation] Evaluation section: The claim that AdvNet 'exposes previously unnoticed Linux kernel bugs' is load-bearing for the central contribution, yet the manuscript provides no evidence of reproduction on real hardware, a different simulator, or the Linux netem stack outside the training environment. Without such validation, it remains possible that the optimizer converged on simulator-specific artifacts (e.g., queueing or timing models) rather than general protocol or kernel issues.
  2. [Design] Design and noise-handling description: The abstract and method sections assert a 'robust noise-handling mechanism' that mitigates variability in protocol performance, but no quantitative evaluation (e.g., variance reduction metrics, ablation with/without the mechanism, or comparison to standard statistical tests) is supplied to show that the discovered adversarial environments remain stable and meaningful under repeated runs or different random seeds.
minor comments (2)
  1. [Abstract] The abstract states results for 'several use cases with different performance goals' but does not enumerate the exact goals or metrics in the summary; a short table or explicit list would improve clarity.
  2. [Design] Notation for the optimization objective and environment parameters (bandwidth, delay, loss, etc.) should be introduced once with consistent symbols rather than re-defined inline in multiple places.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments, which help clarify the scope and presentation of our contributions. We address each major comment below and indicate the revisions we will make to strengthen the manuscript.

read point-by-point responses

  1. Referee: [Evaluation] Evaluation section: The claim that AdvNet 'exposes previously unnoticed Linux kernel bugs' is load-bearing for the central contribution, yet the manuscript provides no evidence of reproduction on real hardware, a different simulator, or the Linux netem stack outside the training environment. Without such validation, it remains possible that the optimizer converged on simulator-specific artifacts (e.g., queueing or timing models) rather than general protocol or kernel issues.

    Authors: We agree that external validation is necessary to rule out simulator-specific artifacts and to support the claim of previously unnoticed kernel bugs. In the revised manuscript we will add reproduction experiments using real hardware testbeds and the Linux netem stack, confirming that the same adversarial conditions trigger the reported performance degradations and kernel behaviors outside the original training environment. revision: yes

  2. Referee: [Design] Design and noise-handling description: The abstract and method sections assert a 'robust noise-handling mechanism' that mitigates variability in protocol performance, but no quantitative evaluation (e.g., variance reduction metrics, ablation with/without the mechanism, or comparison to standard statistical tests) is supplied to show that the discovered adversarial environments remain stable and meaningful under repeated runs or different random seeds.

    Authors: We acknowledge that the current manuscript lacks quantitative evidence for the noise-handling mechanism. We will incorporate an ablation study and supporting metrics (variance reduction, stability across random seeds, and comparison against standard statistical aggregation) into the revised evaluation section to demonstrate that the identified adversarial environments remain consistent and meaningful under repeated measurements. revision: yes

Circularity Check

0 steps flagged

No circularity: AdvNet uses external ML optimization on simulator parameters without reducing claims to self-defined inputs or self-citations

full rationale

The abstract and provided text frame AdvNet as an ML-driven search over network parameters (bandwidth, delay, loss) to expose protocol weaknesses, with a noise-handling mechanism. No equations, derivations, or predictions are shown that equate outputs to fitted inputs by construction. No self-citation chains, uniqueness theorems, or ansatzes are invoked to justify core claims. The evaluation on 27 implementations and reported kernel bugs are presented as empirical outcomes of the independent optimization process rather than tautological renamings or load-bearing self-references. This matches the default expectation of a non-circular paper.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 1 invented entities

The central claim depends on the domain assumption that ML optimization over network parameters can locate real performance issues and that the noise-handling mechanism accurately mitigates variability without introducing bias.

axioms (1)
  • domain assumption Machine learning optimization can effectively search network environment spaces to identify conditions causing poor protocol performance
    This is the core mechanism described in the abstract for generating adversarial cases.
invented entities (1)
  • AdvNet no independent evidence
    purpose: Automated generation of adversarial network environments for protocol testing
    New system introduced by the paper to address limitations of hand-designed tests.

pith-pipeline@v0.9.0 · 5539 in / 1192 out tokens · 53556 ms · 2026-05-09T18:21:33.155170+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. CCLab: Adversarial Testing of Learning- and Non-Learning-Based Congestion Controllers

    cs.CR 2026-05 unverdicted novelty 7.0

    CCLab is an adversarial testing framework showing learning-based congestion controllers are generally more robust than traditional human-designed ones under feature- and environment-level attacks, with adversarial tra...

Reference graph

Works this paper leans on

37 extracted references · 4 canonical work pages · cited by 1 Pith paper

  1. [1]

    Venkat Arun, Mohammad Alizadeh, and Hari Balakrishnan. 2022. Starvation in end-to-end congestion control. In Proceedings of the ACM SIGCOMM 2022 Conference. 177–192

    2022

  2. [2]

    Venkat Arun, Mina Tahmasbi Arashloo, Ahmed Saeed, Mohammad Alizadeh, and Hari Balakrishnan. 2021. Toward formally verifying congestion control behavior. InProceedings of the 2021 ACM SIGCOMM 2021 Conference. 1–16

    2021

  3. [3]

    Lionel C Briand. 2008. Novel applications of machine learning in software testing. In2008 The Eighth International Conference on Quality Software. IEEE, 3–10

    2008

  4. [4]

    Cristian Cadar, Daniel Dunbar, Dawson R Engler, et al. 2008. Klee: unassisted and automatic generation of high-coverage tests for complex systems programs.. InOSDI, Vol. 8. 209–224

    2008

  5. [5]

    Chun-Hung Chen. 1995. An effective approach to smartly allocate computing budget for discrete event simulation. In Proceedings of 1995 34th IEEE Conference on Decision and Control, Vol. 3. 2598–2603 vol.3. doi:10.1109/CDC.1995.478499

    work page doi:10.1109/cdc.1995.478499 1995

  6. [6]

    Mike Chow, Yang Wang, William Wang, Ayichew Hailu, Rohan Bopardikar, Bin Zhang, Jialiang Qu, David Meisner, Santosh Sonawane, Yunqi Zhang, Rodrigo Paim, Mack Ward, Ivor Huang, Matt McNally, Daniel Hodges, Zoltan Farkas, Caner Gocmen, Elvis Huang, and Chunqiang Tang. 2024. ServiceLab: Preventing Tiny Performance Regressions at Hy- perscale through Pre-Prod...

    2024

  7. [7]

    Shuo Deng, Ravi Netravali, Anirudh Sivaraman, and Hari Balakrishnan. 2014. WiFi, LTE, or both? Measuring multi- homed wireless internet performance. InProceedings of the 2014 Conference on Internet Measurement Conference. 181–194

    2014

  8. [8]

    Siyang Gao, Weiwei Chen, and Leyuan Shi. 2017. A new budget allocation framework for the expected opportunity cost.Operations Research65, 3 (2017), 787–803

    2017

  9. [9]

    Tomer Gilad, Nathan H Jay, Michael Shnaiderman, Brighten Godfrey, and Michael Schapira. 2019. Robustifying network protocols with adversarial examples. InProceedings of the 18th ACM Workshop on Hot Topics in Networks. 85–92

    2019

  10. [10]

    Patrice Godefroid, Nils Klarlund, and Koushik Sen. 2005. DART: Directed automated random testing. InProceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation. 213–223

    2005

  11. [11]

    Patrice Godefroid, Hila Peleg, and Rishabh Singh. 2017. Learn&fuzz: Machine learning for input fuzzing. In2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 50–59

    2017

  12. [12]

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets.Advances in neural information processing systems27 (2014)

    2014

  13. [13]

    Ramesh Govindan, Ina Minei, Mahesh Kallahalla, Bikash Koley, and Amin Vahdat. 2016. Evolve or Die: High-Availability Design Principles Drawn from Google’s Network Infrastructure. https://dl.acm.org/doi/10.1145/2934872.2934891

    work page doi:10.1145/2934872.2934891 2016

  14. [14]

    Holzmann

    Gerard J. Holzmann. 1997. The model checker SPIN.IEEE Transactions on software engineering23, 5 (1997), 279–295

    1997

  15. [15]

    Syed Hussain, Omar Chowdhury, Shagufta Mehnaz, and Elisa Bertino. 2018. LTEInspector: A systematic approach for adversarial testing of 4G LTE. InNetwork and Distributed Systems Security (NDSS) Symposium 2018

    2018

  16. [16]

    2009.Introduction to network simulator 2 (NS2)

    Teerawat Issariyakul, Ekram Hossain, Teerawat Issariyakul, and Ekram Hossain. 2009.Introduction to network simulator 2 (NS2). Springer

    2009

  17. [17]

    Jana Iyengar, Martin Thomson, et al. 2021. QUIC: A UDP-based multiplexed and secure transport. InRFC 9000. Internet Engineering Task Force (IETF) Fremont, CA, USA

    2021

  18. [18]

    Toshihiko Kato, Adhikari Diwakar, Ryo Yamamoto, Satoshi Ohzahata, and Nobuo Suzuki. 2019. Experimental analysis of MPTCP congestion control algorithms; LIA, OLIA and BALIA. In8th International Conference on Theory and Practice in Modern Computing (TPMC 2019). 135–142

    2019

  19. [19]

    TV Lakshman, Upamanyu Madhow, and Bernhard Suter. 2000. TCP/IP performance with random loss and bidirectional congestion.IEEE/ACM transactions on networking8, 5 (2000), 541–555

    2000

  20. [20]

    Ravi Netravali, Anirudh Sivaraman, Somak Das, Ameesh Goyal, Keith Winstein, James Mickens, and Hari Balakrishnan

  21. [21]

    In2015 USENIX Annual Technical Conference (USENIX ATC 15)

    Mahimahi: accurate {Record-and-Replay} for {HTTP }. In2015 USENIX Annual Technical Conference (USENIX ATC 15). 417–429

  22. [22]

    Roy P Pargas, Mary Jean Harrold, and Robert R Peck. 1999. Test-data generation using genetic algorithms.Software testing, verification and reliability9, 4 (1999), 263–282

    1999

  23. [23]

    Devdeep Ray and Srinivasan Seshan. 2022. CC-fuzz: genetic algorithm-based fuzzing for stress testing congestion control algorithms. InProceedings of the 21st ACM Workshop on Hot Topics in Networks. 31–37

    2022

  24. [24]

    George F Riley and Thomas R Henderson. 2010. The ns-3 network simulator. InModeling and tools for network simulation. Springer, 15–34

    2010

  25. [25]

    Vern Paxson, and Mark Allman

    Matt Sargent, Jerry Chu, Dr. Vern Paxson, and Mark Allman. 2011. Computing TCP’s Retransmission Timer. RFC 6298. doi:10.17487/RFC6298 Proc. ACM Netw., Vol. 4, No. CoNEXT2, Article 12. Publication date: June 2026. 12:20 Shehab et al

    work page doi:10.17487/rfc6298 2011

  26. [26]

    William Sentosa, Balakrishnan Chandrasekaran, P Brighten Godfrey, Haitham Hassanieh, and Bruce Maggs. 2023. {DChannel}: Accelerating Mobile Applications With Parallel High-bandwidth and Low-latency Channels. In20th USENIX Symposium on Networked Systems Design and Implementation (NSDI 23). 419–436

    2023

  27. [27]

    Guoqiang Shu and David Lee. 2007. Testing security properties of protocol implementations-a machine learning based approach. In27th International Conference on Distributed Computing Systems (ICDCS’07). IEEE, 25–25

    2007

  28. [28]

    William M Spears and Kenneth A De Jong. 1991. An analysis of multi-point crossover. InFoundations of genetic algorithms. Vol. 1. Elsevier, 301–315

    1991

  29. [29]

    Talal Touseef, William Sentosa, Milind Kumar Vaddiraju, Debopam Bhattacherjee, Balakrishnan Chandrasekaran, Brighten Godfrey, and Shubham Tiwari. 2023. Boosting Application Performance using Heterogeneous Virtual Channels: Challenges and Opportunities. InProceedings of the 22nd ACM Workshop on Hot Topics in Networks. 139–146

    2023

  30. [30]

    Ranysha Ware, Matthew K Mukerjee, Srinivasan Seshan, and Justine Sherry. 2019. Modeling BBR’s interactions with loss-based congestion control. InProceedings of the internet measurement conference. 137–143

    2019

  31. [31]

    Brian White, Jay Lepreau, Leigh Stoller, Robert Ricci, Shashi Guruprasad, Mac Newbold, Mike Hibler, Chad Barb, and Abhijeet Joglekar. 2002. An integrated experimental environment for distributed systems and networks.ACM SIGOPS Operating Systems Review36, SI (2002), 255–270

    2002

  32. [32]

    Damon Wischik, Costin Raiciu, Adam Greenhalgh, and Mark Handley. 2011. Design, implementation and evaluation of congestion control for multipath {TCP}. In8th USENIX Symposium on Networked Systems Design and Implementation (NSDI 11)

    2011

  33. [33]

    Zhengxu Xia, Yajie Zhou, Francis Y Yan, and Junchen Jiang. 2022. Genet: automatic curriculum generation for learning adaptation in networking. InProceedings of the ACM SIGCOMM 2022 Conference. 397–413

    2022

  34. [34]

    Francis Y Yan, Hudson Ayers, Chenzhi Zhu, Sadjad Fouladi, James Hong, Keyi Zhang, Philip Levis, and Keith Winstein

  35. [35]

    In17th USENIX Symposium on Networked Systems Design and Implementation (NSDI 20)

    Learning in situ: a randomized experiment in video streaming. In17th USENIX Symposium on Networked Systems Design and Implementation (NSDI 20). 495–511

  36. [36]

    Francis Y Yan, Jestin Ma, Greg D Hill, Deepti Raghavan, Riad S Wahby, Philip Levis, and Keith Winstein. 2018. Pantheon: the training ground for Internet congestion-control research. In2018 USENIX Annual Technical Conference (USENIX ATC 18). 731–743

    2018

  37. [37]

    Songyang Zhang. 2019. An evaluation of BBR and its variants.arXiv preprint arXiv:1909.03673(2019). A Level of Parallelism Before determining the optimal level of parallelism, we first investigate the maximum degree of parallelism that the underlying machine can reliably support. To quantify the overhead introduced by evaluating environments in parallel ra...

    work page arXiv 2019