Pith. sign in

REVIEW 4 major objections 5 minor 40 references

The paper claims that a multi-agent, retrieval-augmented fuzzer can outperform single-LLM and coverage-guided protocol fuzzers on RTSP, with higher branch coverage and deeper state exploration.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

MultiFuzz combines retrieval-augmented generation and multiple LLM agents within the ChatAFL protocol fuzzer, reporting marginal and statistically unsupported gains in branch coverage and state exploration for RTSP.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection The architecture is worth a look, but the evaluation is too thin to support the paper's claims—this is a promising system description, not a demonstrated result. the 4 major comments →

arxiv 2508.14300 v1 pith:VXBUNFZY submitted 2025-08-19 cs.CR cs.CLcs.MAcs.NI

MultiFuzz: A Dense Retrieval-based Multi-Agent System for Network Protocol Fuzzing

Youssef Maklad , Fares Wael , Ali Hamdi , Wael Elsersy , Khaled Shaban This is my paper
classification cs.CR cs.CLcs.MAcs.NI
keywords protocol fuzzingretrieval-augmented generationmulti-agent LLM systemsstateful protocol testingRTSPbranch coveragefinite-state machineLLM-guided fuzzing
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to show that protocol fuzzing can be made substantially more effective by giving an LLM-driven fuzzer structured access to the protocol's own specification. MultiFuzz builds on the ChatAFL pipeline and replaces its single LLM prompt with three specialized crews of agents—grammar extraction, seed enrichment, and coverage-plateau surpassing—that share a dense retrieval agent. That agent queries a vector store built from an RFC text that has been split into atomic propositions and grouped into semantic chunks. On the RTSP/Live555 target, the paper reports higher average branch coverage and more protocol states and transitions than NSFuzz, AFLNet, and ChatAFL over three 24-hour runs. A careful reader would care because stateful protocol bugs are hard to reach, and any repeatable gain in state exploration is a security-relevant result.

Core claim

The central claim is that a fuzzer whose LLM components are organized as collaborating agents and grounded in retrieved protocol documentation will explore a stateful protocol's state machine more deeply than a single-LLM fuzzer or a coverage-guided fuzzer. The paper reports average branch coverage of 2940 branches for MultiFuzz versus 2912.67 for ChatAFL, 2860 for AFLNet, and 2807 for NSFuzz; 14.67 explored states versus 14.33, 10.0, and 11.7; and 163.33 state transitions versus 159.67, 84.0, and 90.33. The authors attribute the gains to the retrieval-augmented, multi-agent design, which keeps generated requests syntactically valid and semantically targeted at uncovered transitions.

What carries the argument

The load-bearing mechanism is a retrieval-augmented multi-agent pipeline: RFC text is filtered, decomposed into atomic propositions, grouped into semantic chunks, embedded, and indexed in a dense vector store; a shared retrieval agent then supplies relevant chunks to three crews of LLM agents that generate grammars, enrich seeds, and break coverage plateaus. The retrieval context is what lets agents adhere to RTSP syntax and target states not yet covered.

Load-bearing premise

The load-bearing premise is that the measured difference comes from the retrieval and multi-agent design, rather than from run-to-run variability or from picking the most effective LLM for each role after seeing the results.

What would settle it

Run the same 24-hour RTSP/Live555 benchmark with the retrieval and multi-agent layers disabled, keeping the same models and prompts collapsed into a single chat-style LLM call. If branch coverage and state transitions do not fall back to ChatAFL levels, the reported gain is not caused by the new machinery. A second check: rerun with models assigned randomly to agent roles; if the advantage vanishes, model selection is the cause.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If the gains are causal, fuzzing a stateful protocol no longer depends on hand-written grammars: indexing the protocol's RFC gives agents the syntax and transition rules they need.
  • The same pipeline can be pointed at a new protocol by swapping the document store, making the approach scalable to protocols whose specifications are public but whose implementations are closed.
  • Coverage plateaus become addressable at runtime: an agent that reads fuzzing history and retrieves transition rules can generate a targeted packet to cross into an unexplored state.
  • Structured, explained outputs mean each generated request carries a record of the state it was meant to reach, which should make failures easier to triage.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper does not isolate which component is responsible; an ablation turning off retrieval and an ablation collapsing the three crews into one LLM call would test whether the multi-agent coordination or the document grounding matters more.
  • The per-role model search was performed on the same benchmark that produced the headline numbers; on a new protocol, the chosen models may not be optimal, so the reported margins may be optimistic.
  • RTSP is a favorable case for documentation retrieval because its RFC defines a small, explicit method set; the approach may not help as much for protocols with vague or incomplete specifications.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes MultiFuzz, a dense-retrieval-based multi-agent system built on top of ChatAFL for network protocol fuzzing. It preprocesses RFC documents into propositions, groups them via agentic chunking, indexes them in a vector store, and uses three crews of LLM agents (grammar extraction, seed enrichment, coverage plateau) with a shared dense retrieval agent. The system is evaluated on RTSP/Live555 against NSFuzz, AFLNet, and ChatAFL, reporting branch coverage, number of states, and state transitions across three 24-hour runs. The paper claims that MultiFuzz significantly improves all three metrics over the baselines.

Significance. If the empirical claims held, the paper would make a useful contribution to LLM-guided protocol fuzzing: it combines RAG over protocol RFCs with a multi-agent decomposition of the fuzzing pipeline, and it provides concrete prompt designs and a system architecture. However, the current evidence is not sufficient to support the central claim. There are no ablations, no significance tests, only three runs on a single protocol, and the LLM model assignment was tuned on the same benchmark used for the reported comparison. The claimed 'dramatic improvements' are contradicted by the tabulated variance and by one run in which ChatAFL outperforms MultiFuzz. The contribution is therefore not yet established.

major comments (4)
  1. [V.C and Table I] The claim of 'significant improvement' over ChatAFL is unsupported by the reported data. With n=3 runs and no significance tests, the average gains are small: branch coverage +0.9% (2940.0 vs 2912.67), state transitions +2.3% (163.33 vs 159.67), and states +2.4% (14.67 vs 14.33). In Experiment 2 of Table I, ChatAFL reaches 2998 branches while MultiFuzz reaches 2910, i.e., the baseline wins by a larger margin than the average advantage. The text in Section V.C also says MultiFuzz ranges from 2970 to 2940 branches, but the table includes 2910. At minimum, the paper needs confidence intervals or significance testing and more repetitions; the current numbers are within run-to-run noise.
  2. [V (Model selection)] The model assignment per agent role was chosen iteratively on the same RTSP benchmark used for the final evaluation: 'we explored different combinations of these models across the various agent groups... until the most effective model was identified for each specific subtask.' This is post-hoc selection on the test set, which inflates the reported numbers. The paper does not use a held-out protocol or a validation set, so the comparison to baselines is not a fair evaluation of the proposed system.
  3. [V, RQ2] RQ2 asks whether multi-agent collaboration improves over single-LLM approaches, but the experiment does not isolate the architecture. MultiFuzz differs from ChatAFL not only in multi-agent coordination but also in the LLM provider and models (Groq-Cloud Llama-family vs ChatAFL's GPT), the retrieval-augmented context, and additional tools (CVE retrieval, packet parsing). Without an ablation that varies only the architecture (e.g., ChatAFL with the same LLM and RAG, or MultiFuzz with a single agent), the observed differences cannot be attributed to the multi-agent design.
  4. [Abstract and V.A] The general claim that MultiFuzz 'significantly improves' coverage and explores deeper states is based on a single protocol (RTSP) and a single server implementation (Live555). The paper evaluates no second protocol from ProFuzzBench, so the improvements could be specific to the characteristics of RTSP. The conclusions in Section VII overstate the generality. Either additional protocol targets or substantially qualified claims are needed.
minor comments (5)
  1. [IV.H] Typo: 'it’s features' should be 'its features'.
  2. [Table I] The arrow notation is ambiguous in Experiment 2: the '↓ -2.9%' in the ChatAFL column actually indicates that ChatAFL outperforms MultiFuzz, since 2998 > 2910. Use explicit win/loss indicators or state the comparison direction.
  3. [V.A] The text says 'All experiments were repeated multiple times' but only three runs are reported. Specify the exact number of runs and whether the means/std are over independent sessions.
  4. [IV.C] The compatibility threshold θ is introduced but no value or sensitivity analysis is given. Since it controls chunk creation, its choice may affect retrieval quality.
  5. [Figure 1] Figure 1 is referenced but the figure is not described; readers cannot verify the system architecture from the text alone. Also, no code or artifact availability statement is provided.

Circularity Check

1 steps flagged

Central comparison is partly a fitted result: per-agent LLM assignments were tuned on the same RTSP benchmark used for the reported gains, and no ablation or significance testing isolates the RAG/multi-agent contribution.

specific steps
  1. fitted input called prediction [Section V, 'Experimental Design and Evaluation', model-selection paragraph before 'A. Experiments Setup']
    "Throughout the experimentation process, we explored different combinations of these models across the various agent groups in the framework. Tasks such as grammar extraction, seed enrichment, and plateau surpassing were assigned to different models iteratively until the most effective model was identified for each specific subtask, optimizing the overall performance of MultiFuzz."

    The paper treats MultiFuzz's performance as evidence that its dense-retrieval multi-agent architecture improves over SOTA fuzzers, but the configuration being evaluated was fit to the same RTSP/Live555 benchmark that is later reported as the comparison. The per-agent model choices are free parameters optimized on the test set ('iteratively until the most effective model was identified'), so the headline gains over ChatAFL (0.9% branch coverage, 2.3% transitions, 2.4% states) are in part a tuned result rather than an out-of-sample prediction. With n=3 and one run where ChatAFL outperformed MultiFuzz (Table I: 2998 vs 2910), the claimed causal advantage of the architecture is not independently established; the reported 'prediction' of superiority is statistically inflated by benchmark-specif

full rationale

This is an empirical systems paper rather than a formal derivation, so most circularity patterns do not apply. The evaluation uses external baselines (NSFuzz, AFLNet, ChatAFL) and ProFuzzBench, and the system is built on published, independently available frameworks, so the central claim is not circular by construction. The one significant circularity-adjacent issue is that the authors selected the LLM model for each agent role by iterative experimentation on the same RTSP benchmark used for the reported comparison. That is test-set selection: the configuration is a fitted hyperparameter, and the reported gains are partly a product of that fitting rather than an independent test of the proposed architecture. Additionally, RQ2 asks whether multi-agent collaboration improves over single-LLM ChatAFL, but the comparison confounds architecture with model family (Llama models via Groq vs ChatAFL's original GPT-based setup) and no ablation isolates the dense-retrieval pipeline or agent crews. These are correctness and statistical-validity concerns more than definitional circularity; the paper's derivations, such as they are, do not reduce to their inputs. The self-citation to the authors' prior RAG/CoT work (ref [17]) is background support, not load-bearing for the main result. Score 4 reflects partial circularity through fitted-input-called-prediction, while acknowledging that the system is still an independent empirical construction with external benchmarks.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

The paper introduces no physical or ontological entities. Its contributions are architectural: three crews of LLM agents, a dense retrieval agent, and an agentic chunking method. These are software components, not entities requiring independent falsifiable handles. The main free parameters are the chunking threshold, the per-role LLM choices (selected by searching on the benchmark), and the number of inserted seed requests.

free parameters (3)
  • Chunk compatibility threshold theta = not reported
    Used in agentic chunking to decide whether a proposition joins an existing chunk or starts a new one (Section IV-C); value not specified.
  • LLM model assignment per agent role = final model names reported (e.g. llama3.3-70b-versatile)
    The authors iteratively selected the model per subtask during experimentation on the same benchmark (Section V), making model identity a fitted parameter.
  • Number of inserted seed requests = 2
    Seed Enrichment Crew inserts two client requests into seed sequences; this choice is not justified.
axioms (4)
  • domain assumption RFC-2326 correctly describes the RTSP semantics implemented by Live555
    The whole RAG pipeline uses RFC-2326 as ground truth for grammar and state transitions (Section IV-A).
  • domain assumption The retrieved vector-database chunks provide semantically relevant context for the LLM agents
    No evaluation of retrieval quality is provided; the system assumes dense retrieval over agentic chunks suffices (Section IV-D).
  • domain assumption Branch coverage as measured by ProFuzzBench is a comparable and unbiased metric across fuzzers
    The paper compares raw branch counts across fuzzers with no discussion of instrumentation or path conditions (Section V-B).
  • ad hoc to paper Three 24-hour runs are representative of fuzzer performance
    Only three runs per fuzzer are used to compute averages; no statistical tests are reported (Section V-A).

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of MultiFuzz: A Dense Retrieval-based Multi-Agent System for Network Protocol Fuzzing." pith.science (2026). https://pith.science/paper/VXBUNFZY

@misc{pith2026250814300,
  author       = {Pith},
  title        = {Pith review of: MultiFuzz: A Dense Retrieval-based Multi-Agent System for Network Protocol Fuzzing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VXBUNFZY}},
  note         = {Machine review of arXiv:2508.14300}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Traditional protocol fuzzing techniques, such as those employed by AFL-based systems, often lack effectiveness due to a limited semantic understanding of complex protocol grammars and rigid seed mutation strategies. Recent works, such as ChatAFL, have integrated Large Language Models (LLMs) to guide protocol fuzzing and address these limitations, pushing protocol fuzzers to wider exploration of the protocol state space. But ChatAFL still faces issues like unreliable output, LLM hallucinations, and assumptions of LLM knowledge about protocol specifications. This paper introduces MultiFuzz, a novel dense retrieval-based multi-agent system designed to overcome these limitations by integrating semantic-aware context retrieval, specialized agents, and structured tool-assisted reasoning. MultiFuzz utilizes agentic chunks of protocol documentation (RFC Documents) to build embeddings in a vector database for a retrieval-augmented generation (RAG) pipeline, enabling agents to generate more reliable and structured outputs, enhancing the fuzzer in mutating protocol messages with enhanced state coverage and adherence to syntactic constraints. The framework decomposes the fuzzing process into modular groups of agents that collaborate through chain-of-thought reasoning to dynamically adapt fuzzing strategies based on the retrieved contextual knowledge. Experimental evaluations on the Real-Time Streaming Protocol (RTSP) demonstrate that MultiFuzz significantly improves branch coverage and explores deeper protocol states and transitions over state-of-the-art (SOTA) fuzzers such as NSFuzz, AFLNet, and ChatAFL. By combining dense retrieval, agentic coordination, and language model reasoning, MultiFuzz establishes a new paradigm in autonomous protocol fuzzing, offering a scalable and extensible foundation for future research in intelligent agentic-based fuzzing systems.

Figures

Figures reproduced from arXiv: 2508.14300 by Ali Hamdi, Fares Wael, Khaled Shaban, Wael Elsersy, Youssef Maklad.

Figure 1
Figure 1. Figure 1: High-Level System Architecture of the MultiFuzz Framework, based on AFLNet and ChatAFL formats and stateful interactions. Existing techniques outlined earlier in section II, which are blackbox, whitebox, and graybox fuzzing approaches, offer different trade-offs between scalability, precision, and required prior knowledge of the target protocol. 1) Blackbox Fuzzing: Blackbox fuzzers operate without any int… view at source ↗
Figure 2
Figure 2. Figure 2: System prompt used for propositional transformation of [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Sample document chunk after the agentic chunking phase of [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Summary of the MultiFuzz’s crews, showing each crew’s internal agents and integrated tools. All three crews share a Dense Retrieval Agent for semantic context fetching. • A list of constituent propositions {pk} K k=1, • A concise chunk summary sj generated by the LLM. • A descriptive, technically precise chunk title tj . The internal logic can be modeled as a two-stage LLM pipeline: 1) Chunk Selection: Giv… view at source ↗
Figure 5
Figure 5. Figure 5: Sample prompt asking the final agent to generate a coverage [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

40 extracted references · 12 linked inside Pith

  1. [1]

    Sutton, A

    M. Sutton, A. Greene, and P. Amini, Fuzzing: brute force vulnerability discovery. Pearson Education, 2007

    2007

  2. [2]

    A survey of network protocol fuzzing: Model, techniques and directions,

    S. Jiang, Y . Zhang, J. Li, H. Yu, L. Luo, and G. Sun, “A survey of network protocol fuzzing: Model, techniques and directions,” arXiv preprint arXiv:2402.17394, 2024

    Pith/arXiv arXiv 2024

  3. [3]

    A survey of automatic protocol reverse engineering tools,

    J. Narayan, S. K. Shukla, and T. C. Clancy, “A survey of automatic protocol reverse engineering tools,” ACM Computing Surveys (CSUR) , vol. 48, no. 3, pp. 1–26, 2015

    2015

  4. [4]

    State selection algorithms and their impact on the performance of stateful network protocol fuzzing,

    D. Liu, V .-T. Pham, G. Ernst, T. Murray, and B. I. Rubinstein, “State selection algorithms and their impact on the performance of stateful network protocol fuzzing,” in 2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER). IEEE, 2022, pp. 720–730

    2022

  5. [5]

    Evaluating large language models trained on code,

    M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. D. O. Pinto, J. Kaplan, H. Edwards, Y . Burda, N. Joseph, G. Brockman et al., “Evaluating large language models trained on code,” arXiv preprint arXiv:2107.03374 , 2021

    Pith/arXiv arXiv 2021

  6. [6]

    Language models can solve computer tasks,

    G. Kim, P. Baldi, and S. McAleer, “Language models can solve computer tasks,” Advances in Neural Information Processing Systems , vol. 36, pp. 39 648–39 677, 2023

    2023

  7. [7]

    Large language models based fuzzing techniques: A survey,

    L. Huang, P. Zhao, H. Chen, and L. Ma, “Large language models based fuzzing techniques: A survey,” arXiv preprint arXiv:2402.00350 , 2024

    Pith/arXiv arXiv 2024

  8. [8]

    Generative ai and large language models for cyber security: All insights you need,

    M. A. Ferrag, F. Alwahedi, A. Battah, B. Cherif, A. Mechri, and N. Tihanyi, “Generative ai and large language models for cyber security: All insights you need,” Available at SSRN 4853709 , 2024

    2024

  9. [9]

    Large language model guided protocol fuzzing,

    R. Meng, M. Mirchev, M. B ¨ohme, and A. Roychoudhury, “Large language model guided protocol fuzzing,” in Proceedings of the 31st Annual Network and Distributed System Security Symposium (NDSS) , 2024

    2024

  10. [10]

    Dense x retrieval: What retrieval granularity should we use?

    T. Chen, H. Wang, S. Chen, W. Yu, K. Ma, X. Zhao, H. Zhang, and D. Yu, “Dense x retrieval: What retrieval granularity should we use?” in Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 2024, pp. 15 159–15 177

    2024

  11. [11]

    Retrieval- augmented generation for knowledge-intensive nlp tasks,

    P. Lewis, E. Perez, A. Piktus, F. Petroni, V . Karpukhin, N. Goyal, H. K ¨uttler, M. Lewis, W.-t. Yih, T. Rockt ¨aschel et al. , “Retrieval- augmented generation for knowledge-intensive nlp tasks,” Advances in Neural Information Processing Systems , vol. 33, pp. 9459–9474, 2020

    2020

  12. [12]

    React: Synergizing reasoning and acting in language models,

    S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y . Cao, “React: Synergizing reasoning and acting in language models,” arXiv preprint arXiv:2210.03629, 2022

    Pith/arXiv arXiv 2022

  13. [13]

    The art, science, and engineering of fuzzing: A survey,

    V . J. Man `es, H. Han, C. Han, S. K. Cha, M. Egele, E. J. Schwartz, and M. Woo, “The art, science, and engineering of fuzzing: A survey,” IEEE Transactions on Software Engineering , vol. 47, no. 11, pp. 2312–2331, 2019

    2019

  14. [14]

    A survey on the development of network protocol fuzzing techniques,

    Z. Zhang, H. Zhang, J. Zhao, and Y . Yin, “A survey on the development of network protocol fuzzing techniques,” Electronics, vol. 12, no. 13, p. 2904, 2023

    2023

  15. [15]

    Attention is all you need,

    A. Vaswani, “Attention is all you need,” Advances in Neural Information Processing Systems, 2017

    2017

  16. [16]

    Chatphishdetector: Detecting phishing sites using large language models,

    T. Koide, H. Nakano, and D. Chiba, “Chatphishdetector: Detecting phishing sites using large language models,” IEEE Access, 2024

    2024

  17. [17]

    Retrieval augmented generation based llm evaluation for protocol state machine inference with chain-of-thought reasoning,

    Y . Maklad, F. Wael, W. Elsersy, and A. Hamdi, “Retrieval augmented generation based llm evaluation for protocol state machine inference with chain-of-thought reasoning,” arXiv preprint arXiv:2502.15727 , 2025

    Pith/arXiv arXiv 2025

  18. [18]

    Harnessing large lan- guage models for seed generation in greybox fuzzing,

    W. Shi, Y . Zhang, X. Xing, and J. Xu, “Harnessing large lan- guage models for seed generation in greybox fuzzing,” arXiv preprint arXiv:2411.18143, 2024

    Pith/arXiv arXiv 2024

  19. [19]

    Codamosa: Escaping coverage plateaus in test generation with pre-trained large language models,

    C. Lemieux, J. P. Inala, S. K. Lahiri, and S. Sen, “Codamosa: Escaping coverage plateaus in test generation with pre-trained large language models,” in 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE). IEEE, 2023, pp. 919–931

    2023

  20. [20]

    Erpa: Efficient rpa model integrating ocr and llms for intelligent document processing,

    O. H. Abdellaif, A. N. Hassan, and A. Hamdi, “Erpa: Efficient rpa model integrating ocr and llms for intelligent document processing,” in 2024 International Mobile, Intelligent, and Ubiquitous Computing Conference (MIUCC). IEEE, 2024, pp. 295–300

    2024

  21. [21]

    Lmrpa: Large lan- guage model-driven efficient robotic process automation for ocr,

    O. H. Abdellaif, A. Nader, and A. Hamdi, “Lmrpa: Large lan- guage model-driven efficient robotic process automation for ocr,” arXiv preprint arXiv:2412.18063, 2024

    Pith/arXiv arXiv 2024

  22. [22]

    Lmv-rpa: Large model voting- based robotic process automation,

    O. Abdellatif, A. Ayman, and A. Hamdi, “Lmv-rpa: Large model voting- based robotic process automation,” arXiv preprint arXiv:2412.17965 , 2024

    Pith/arXiv arXiv 2024

  23. [23]

    Llm multi-agent systems: Challenges and open problems,

    S. Han, Q. Zhang, Y . Yao, W. Jin, and Z. Xu, “Llm multi-agent systems: Challenges and open problems,” arXiv preprint arXiv:2402.03578, 2024

    Pith/arXiv arXiv 2024

  24. [24]

    Pentestagent: Incorporating llm agents to automated penetration testing,

    X. Shen, L. Wang, Z. Li, Y . Chen, W. Zhao, D. Sun, J. Wang, and W. Ruan, “Pentestagent: Incorporating llm agents to automated penetration testing,” arXiv preprint arXiv:2411.05185 , 2024

    Pith/arXiv arXiv 2024

  25. [25]

    Ics protocol fuzzing: Coverage guided packet crack and generation,

    Z. Luo, F. Zuo, Y . Shen, X. Jiao, W. Chang, and Y . Jiang, “Ics protocol fuzzing: Coverage guided packet crack and generation,” in 2020 57th ACM/IEEE Design Automation Conference (DAC) . IEEE, 2020, pp. 1–6

    2020

  26. [26]

    Bbuzz: A bit-aware fuzzing framework for network protocol systematic reverse engineering and analysis,

    B. Blumbergs and R. Vaarandi, “Bbuzz: A bit-aware fuzzing framework for network protocol systematic reverse engineering and analysis,” in MILCOM 2017-2017 IEEE Military Communications Conference (MILCOM). IEEE, 2017, pp. 707–712

    2017

  27. [27]

    Pulsar: Stateful black-box fuzzing of proprietary network protocols,

    H. Gascon, C. Wressnegger, F. Yamaguchi, D. Arp, and K. Rieck, “Pulsar: Stateful black-box fuzzing of proprietary network protocols,” in Security and Privacy in Communication Networks: 11th EAI Interna- tional Conference, SecureComm 2015, Dallas, TX, USA, October 26-29, 2015, Proceedings 11 . Springer, 2015, pp. 330–347

    2015

  28. [28]

    American fuzzy lop,

    M. Zalewski, “American fuzzy lop,” 2014, http://lcamtuf.coredump.cx/ afl/

    2014

  29. [29]

    {AFL++}: Combin- ing incremental steps of fuzzing research,

    A. Fioraldi, D. Maier, H. Eißfeldt, and M. Heuse, “ {AFL++}: Combin- ing incremental steps of fuzzing research,” in 14th USENIX Workshop on Offensive Technologies (WOOT 20) , 2020

    2020

  30. [30]

    Aflnet: a greybox fuzzer for network protocols,

    V .-T. Pham, M. B¨ohme, and A. Roychoudhury, “Aflnet: a greybox fuzzer for network protocols,” in 2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST) . IEEE, 2020, pp. 460–465

    2020

  31. [31]

    Nsfuzz: Towards efficient and state-aware network service fuzzing,

    S. Qin, F. Hu, Z. Ma, B. Zhao, T. Yin, and C. Zhang, “Nsfuzz: Towards efficient and state-aware network service fuzzing,” ACM Transactions on Software Engineering and Methodology , vol. 32, no. 6, pp. 1–26, 2023

    2023

  32. [32]

    Augmenting greybox fuzzing with generative ai,

    J. Hu, Q. Zhang, and H. Yin, “Augmenting greybox fuzzing with generative ai,” arXiv preprint arXiv:2306.06782 , 2023

    Pith/arXiv arXiv 2023

  33. [33]

    Msfuzz: Augmenting protocol fuzzing with message syntax comprehension via large language models

    M. Cheng, K. Zhu, Y . Chen, G. Yang, Y . Lu, and C. Lu, “Msfuzz: Augmenting protocol fuzzing with message syntax comprehension via large language models.” Electronics (2079-9292), vol. 13, no. 13, 2024

    2079

  34. [34]

    Large language models are zero-shot fuzzers: Fuzzing deep-learning libraries via large language models,

    Y . Deng, C. S. Xia, H. Peng, C. Yang, and L. Zhang, “Large language models are zero-shot fuzzers: Fuzzing deep-learning libraries via large language models,” in Proceedings of the 32nd ACM SIGSOFT interna- tional symposium on software testing and analysis , 2023, pp. 423–435

    2023

  35. [35]

    Large language models are edge-case fuzzers: Testing deep learning libraries via fuzzgpt,

    Y . Deng, C. S. Xia, C. Yang, S. D. Zhang, S. Yang, and L. Zhang, “Large language models are edge-case fuzzers: Testing deep learning libraries via fuzzgpt,” arXiv preprint arXiv:2304.02014 , 2023

    Pith/arXiv arXiv 2023

  36. [36]

    NVD - Home,

    “NVD - Home,” https://nvd.nist.gov/, [Accessed 19-06-2025]

    2025

  37. [37]

    LangChain,

    “LangChain,” https://www.langchain.com/, [Accessed 27-05-2025]

    2025

  38. [38]

    “CrewAI,” https://www.crewai.com/, [Accessed 27-05-2025]

    2025

  39. [39]

    Groq is Fast AI Inference,

    “Groq is Fast AI Inference,” https://groq.com/, [Accessed 19-06-2025]

    2025

  40. [40]

    Profuzzbench: A benchmark for stateful protocol fuzzing,

    R. Natella and V .-T. Pham, “Profuzzbench: A benchmark for stateful protocol fuzzing,” in Proceedings of the 30th ACM SIGSOFT interna- tional symposium on software testing and analysis , 2021, pp. 662–665

    2021

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.