Pith. sign in

REVIEW 4 major objections 6 minor 91 references

This paper claims that an LLM security agent that writes its own property-based input generators can produce proof-of-vulnerability inputs faster and more reliably than existing directed fuzzers.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 18:33 UTC pith:W7A6CGRW

load-bearing objection PBFuzz is a well-engineered agentic fuzzing system with real novelty, but its headline 57-CVE claim rests on a thin margin and some non-comparable successes; it deserves serious review after a fair first revision. the 4 major comments →

arxiv 2512.04611 v2 pith:W7A6CGRW submitted 2025-12-04 cs.CR cs.SE

PBFuzz: Agentic Directed Fuzzing for PoV Generation

Haochen Zeng , Andrew Bao , Jiajun Cheng , Chengyu Song This is my paper
classification cs.CR cs.SE
keywords proof-of-vulnerabilityagentic fuzzingLLM agentsproperty-based testingdirected fuzzingconstraint extractionvulnerability reproductionfuzz harness synthesis
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that proof-of-vulnerability generation can be automated by an LLM-driven agent that reasons about code the way a human analyst would: extract the conditions that reach a bug and the conditions that trigger it, encode those conditions as a parameterized input generator, search the parameter space with property-based testing, and refine the model using debugger feedback. On a standard ground-truth vulnerability suite, the agent triggered 57 vulnerabilities in a single 30-minute run per target, more than any baseline fuzzer achieved in ten 24-hour runs, and it was the only tool to trigger 17 of them. Median time to a triggering input was roughly 339 seconds versus 8680 seconds for the strongest conventional fuzzer, at an API cost near $1.83 per vulnerability. If true, the work changes the economics of vulnerability verification: PoV generation could shift from resource-intensive overnight fuzzing campaigns to short, cheap agentic sessions, including cases where no fuzz harness exists yet.

Core claim

The central claim is that the barrier to PoV generation is not raw search power but the mapping from source-level semantics to input-level structure. The agent's PLAN phase infers semantic reachability and triggering constraints from code; the IMPLEMENT phase turns those constraints into typed parameter spaces and a generator function; the EXECUTE phase searches with property-based testing; the REFLECT phase uses fine-grained execution feedback to correct false hypotheses. The paper's core discovery is that this loop—rather than distance-guided mutation or one-shot LLM prompting—is what lets a tool trigger vulnerabilities that require nested structural invariants, representation changes, or

What carries the argument

The load-bearing object is the four-phase workflow state machine (PLAN, IMPLEMENT, EXECUTE, REFLECT), enforced by a gatekeeper and backed by persistent structured memory. PLAN records bug predicates, preconditions, root causes, and trigger plans; IMPLEMENT materializes those into a typed parameter space, a Python generator implementing generate(**params)->bytes, concrete parameter sets, and debugger breakpoints; EXECUTE runs a two-stage property-based search that first tries the agent's concrete guesses, then samples the parameter space; REFLECT uses deviation detection and interactive debugging to diagnose failures. Property-based testing is the bridge: it replaces slow per-input LLM genera

Load-bearing premise

The headline count treats six vulnerabilities as successes even though reaching them required abandoning the benchmark's provided harnesses in favor of different or newly synthesized ones; if the benchmark's protocol is taken as fixed, those six must be discounted.

What would settle it

Pick the six harness-switched cases in the paper's numbering (PHP001, PHP003, PHP010, PHP013, PHP014, SSL006) and rerun PBFuzz with a hard rule that only the benchmark's original harness may be used; if none trigger, the claimed 57-vulnerability coverage and the 17-unique set shrink by six, changing the comparison to fixed-harness fuzzers.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • PoV generation for a known bug drops from a 24-hour, multi-trial fuzzing campaign to a single 30-minute agent run in the paper's experiments.
  • Vulnerabilities whose triggers require coordinated multi-field structural changes—such as updating a length field whenever a component count grows—become automatable, whereas byte-level mutation fails.
  • If the agent can pick or build a harness when the provided one cannot reach the bug, the same system extends PoV generation to code paths that current harnesses do not exercise.
  • At roughly $1.83 per vulnerability, LLM-driven verification becomes cheap enough to run per patch rather than per release.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: six of the paper's 'unique' successes were scored after the agent switched to non-benchmark harnesses or synthesized a custom harness. Under a strict fixed-harness protocol those six would not count on equal terms, and the real contribution may be harness-and-PoV co-generation rather than PoV generation in isolation.
  • The paper's own failure analysis points toward a hybrid system: agentic generators for structured, invariant-heavy triggers plus mutation-based search for encoding variants and spec-violating forms. Combining both would likely raise the total beyond either approach alone.
  • If the largely language-agnostic tools generalize beyond file parsers, the same plan-implement-execute-reflect loop could be aimed at protocol state machines, compiler inputs, or smart contracts, where semantic constraint extraction is likewise the bottleneck.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper presents PBFuzz, an agentic fuzzing framework for proof-of-vulnerability (PoV) input generation. PBFuzz uses an LLM agent in a four-phase PLAN–IMPLEMENT–EXECUTE–REFLECT workflow, supported by MCP tools, persistent structured memory, and property-based testing, to extract reachability and triggering constraints and encode them as parameterized input generators. The evaluation on the Magma benchmark reports that PBFuzz triggered 57 of 129 CVEs in a single 30-minute run per target, outperforming all baselines, uniquely triggering 17 CVEs, and reducing median time-to-exposure to 339 s from 8680 s for AFL++-CmpLog, at an API cost of $1.83 per vulnerability. The paper also reports real-world reproduction of three FFmpeg 1-day CVEs in the abstract. The central claim is that an agentic, constraint-extraction-plus-property-based-testing approach is substantially more effective and efficient than directed and coverage-guided fuzzers for PoV generation.

Significance. If the reported results hold, PBFuzz represents a meaningful step beyond both traditional directed fuzzing and LLM-assisted mutation-based fuzzing: it replaces blind byte mutation with structured, semantically informed input generation and shows evidence that LLM agents can perform the required constraint reasoning. The evaluation uses an external ground-truth benchmark (Magma) with no fitted constants, and the paper includes a detailed ablation study, per-project tool-call analysis, model-variant comparisons, and honest qualitative analysis of failure modes. These are strengths. However, the headline quantitative claims are clouded by a non-uniform benchmark protocol and by internal inconsistencies in the reported counts, so the contribution is currently stronger as an architecture and proof-of-concept than as the "decisive superiority" stated in the abstract.

major comments (4)
  1. [§5.2.1, §5.3.1, Figure 5] The comparison is not protocol-fair. Six of the 57 triggered CVEs (PHP001, PHP003, PHP010, PHP013, PHP014, SSL006) and six of the 17 "unique" CVEs were obtained by switching to non-Magma harnesses or synthesizing a custom harness, while the baselines were restricted to Magma's stock harnesses. The paper acknowledges this in §5.3.1 and argues it shows a real capability, but it is a different task from PoV generation over the fixed benchmark interface. Excluding those six leaves 51 vs 49 over AFL++-CmpLog and 11 unique CVEs, and the 19/18 unique-over-baseline counts shrink accordingly. Since PBFuzz was also run once while baseline counts are the union of 10 trials, the "decisive superiority" claim is not established under a common protocol. The harness-adaptation result should be reported separately, and the headline claims should be re-framed accordingly, or the baselines should be allowe
  2. [Abstract, §1, §5.2, §8] The paper reports inconsistent vulnerability counts. The abstract and §5.2/§8 state PBFuzz triggered 57 vulnerabilities, while §1 states it "successfully triggered 59 out of the 129 CVEs, including 17 CVEs that none of the previous approaches triggered." Figure 5 and the body use 57. This is a load-bearing numerical inconsistency because the central effectiveness claim depends on the exact count. The authors must reconcile the numbers and ensure the abstract, introduction, and evaluation tables use the same, verifiable set.
  3. [Abstract; no corresponding body section] The abstract claims PBFuzz "reproduced three FFmpeg 1-day CVEs that had no public PoVs," but the full text contains no FFmpeg evaluation: no CVE IDs, no reproduction setup, no TTE/cost data, and no mention of FFmpeg in §5 or §6. This is a falsifiable empirical claim that is currently unsupported. Either add the missing experimental data in a dedicated evaluation section or remove the claim from the abstract. As written, it overstates the paper's validated scope.
  4. [§5.2.3, Table 2, §5.1.4] The time-to-exposure analysis needs to be more precise about censoring. Table 2 includes many vulnerabilities for which PBFuzz timed out (e.g., PDF006, SQL013, SSL001, TIF001, XML012), yet the reported median TTE of 339 s appears to be computed over successful cases only. If so, the 25.6x efficiency claim is conditional on success and is subject to selection bias: it ignores the cases where PBFuzz failed within its 30-minute budget. The authors should state explicitly which subset the TTE medians cover, whether timeouts are treated as censored observations, and how the efficiency comparison changes if timeouts are included via survival-style analysis.
minor comments (6)
  1. [§4.6, §9.2] Terminology is inconsistent: the paper's main text uses "PoV" everywhere, but §4.6 and the workflow template use "PoC Found" and "PoC". Unify to a single term.
  2. [§5.5.1, Figure 7] The ablation labels "cursor†" and "cursor‡" in the text appear as "cursorFB" and "cursorTools" in Figure 7. Use the same names in text and figure.
  3. [§5.2.2, §5.3] The introductory sentence of §5.3 says the authors analyze "17 vulnerabilities that only PBFuzz can trigger, and 2 vulnerabilities that only PBFuzz can reach," but the two reach-only vulnerabilities are not identified in that section. Name them or omit the count.
  4. [§5.1.3] The description of PBFuzz's time budget is ambiguous: "30-minute timeout per target for constraint refinement" could mean 30 minutes total per target or 30 minutes per workflow phase. Clarify because the reported maximum TTE of 24 minutes and the cost analysis depend on this.
  5. [§6.2.1, §5.1.3] The claim that PBFuzz's MCP tools are "largely language-agnostic" is overstated: the static analysis tools (LLVM bitcode, type-based call graph, 599 lines of C++) are specifically tied to C/C++ and the LLVM toolchain. The language-general claim should be softened or conditioned on the LLM's code-reading tools.
  6. [Table 2, Table 3, Figure 5] Several table and figure labels are hard to parse (e.g., the ordering of bars in Figure 5 and the abbreviated "CmpLog"). Add a clear legend and spell out what each row/bar represents, especially for readers comparing the unique-CVE rows.

Axiom & Free-Parameter Ledger

3 free parameters · 6 axioms · 0 invented entities

This is a systems-engineering paper rather than a derivation paper: the central claim imports LLM capabilities, Magma labels, and existing fuzzing infrastructure as external grounding. The most important ledger entries are the assumption that bug predicates are given and the ad hoc acceptance of alternative harnesses as benchmark successes. No new physical or mathematical entity is postulated.

free parameters (3)
  • Per-vulnerability ParameterSpace bounds and concrete parameter sets = e.g., CVE-2017-9047: element_prefix_length 1-200, element_name_length 1-200, nesting_depth 1-50, num_elements 1-100, plu
    Chosen by the LLM agent from code reading; no derivation or fitting procedure. If the chosen ranges omit a necessary dimension (e.g., nesting depth), PBFuzz fails, as shown by the RQ3 failures.
  • Static-analysis reachability exclusion set = 83 of 361 Magma targets excluded as unreachable
    Protocol decision changes the evaluation denominator to 278 targets; the paper does not state whether baseline counts were recomputed on the same subset.
  • Time, iteration, and budget thresholds = 30-minute per-target timeout; Stage 2 iteration threshold; MAX 3 manual-test REFLECT budget
    Protocol parameters that define 'success within 30 minutes' while baselines received 24 hours across 10 trials.
axioms (6)
  • domain assumption Local vulnerability triggering conditions (bug predicates) are known and provided (Magma LOG)
    Section 4.1.1 lists this as a prerequisite and Section 4.3 formalizes BugPredicates from MAGMA_LOG rather than discovering them from scratch.
  • domain assumption The program under test is instrumented with reaching/triggering signals
    Section 4.1.1; the EXECUTE and REFLECT phases depend on binary reached/triggered feedback to validate or reject hypotheses.
  • domain assumption LLM agent code reasoning and tool orchestration are reliable enough to infer constraints and synthesize input generators
    Whole method; Section 5.5's ablation shows no non-LLM mechanism performs the constraint inference or generator synthesis.
  • domain assumption Type-based call graph analysis is sufficient; LLM can compensate for imprecision
    Section 4.7 says the static analysis is 'imprecise' but asserts LLM agents can overcome the imprecision through source-code reasoning.
  • ad hoc to paper Counting vulnerabilities triggered via alternative or synthesized harnesses is a valid benchmark result
    Section 5.3.1 counts PHP001, PHP003, PHP010, PHP013, PHP014, and SSL006 as PBFuzz successes even though baselines were restricted to Magma-provided harnesses.
  • domain assumption Property-based testing with LLM-synthesized generators can solve the constraints that LLMs identify
    This is H2, and the RQ3 construct-validity bias (TIF001, TIF002, SQL002, SQL014) shows the assumption fails when the required input is malformed or violates structural invariants.

pith-pipeline@v1.3.0-alltime-deepseek · 36796 in / 13000 out tokens · 118691 ms · 2026-08-03T18:33:16.660715+00:00 · methodology

0 comments
read the original abstract

Proof-of-Vulnerability (PoV) input generation is a critical task in software security and supports downstream applications such as path generation and validation. Generating a PoV input requires solving two sets of constraints: (1) reachability constraints for reaching vulnerable code locations, and (2) triggering constraints for activating the target vulnerability. Existing approaches, including directed greybox fuzzing and LLM-assisted fuzzing, struggle to efficiently satisfy these constraints. This work presents an agentic method that mimics human experts. Human analysts iteratively study code to extract semantic reachability and triggering constraints, form hypotheses about PoV triggering strategies, encode them as test inputs, and refine their understanding using debugging feedback. We automate this process with an agentic directed fuzzing framework called PBFuzz. PBFuzz tackles four challenges in agentic PoV generation: autonomous code reasoning for semantic constraint extraction, custom program-analysis tools for targeted inference, persistent memory to avoid hypothesis drift, and property-based testing for efficient constraint solving while preserving input structure. Experiments on the Magma benchmark show strong results. PBFuzz triggered 57 vulnerabilities, surpassing all baselines, and uniquely triggered 17 vulnerabilities not exposed by existing fuzzers. PBFuzz achieved this within a 30-minute budget per target, while conventional approaches use 24 hours. Median time-to-exposure was 339 seconds for PBFuzz versus 8680 seconds for AFL++ with CmpLog, giving a 25.6x efficiency improvement with an API cost of 1.83 USD per vulnerability. In real-world application, PBFuzz reproduced three FFmpeg 1-day CVEs that had no public PoVs.

Figures

Figures reproduced from arXiv: 2512.04611 by Andrew Bao, Chengyu Song, Haochen Zeng, Jiajun Cheng.

Figure 1
Figure 1. Figure 1: CVE-2017-9047 buffer overflow in libxml2’s [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: At a high level, the workflow layer defines a state ma￾chine with four phases: PLAN and REFLECT phases for inferring and refining semantic constraints for triggering the vulnerability; IMPLEMENT and EXECUTE phases for encoding the constraints into input-level parameter spaces and efficient constraint solving. Beneath it, the LLM agent acts as the “brain” to perform semantic reasoning to (1) generate hypoth… view at source ↗
Figure 3
Figure 3. Figure 3: Property-based fuzzing workflow with two-stage [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Static analysis workflow for call graph analysis and [PITH_FULL_IMAGE:figures/full_fig_p010_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Effectiveness across Magma benchmarks ordered [PITH_FULL_IMAGE:figures/full_fig_p011_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Reproducibility heatmap: triggering frequency [PITH_FULL_IMAGE:figures/full_fig_p012_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Vulnerability trigger counts across system variants. [PITH_FULL_IMAGE:figures/full_fig_p017_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Average tool invocations per vulnerability, by [PITH_FULL_IMAGE:figures/full_fig_p018_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Initial system prompt defining task objectives, [PITH_FULL_IMAGE:figures/full_fig_p022_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Initial system prompt template for cursor agent [PITH_FULL_IMAGE:figures/full_fig_p022_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: project_config.md template [PITH_FULL_IMAGE:figures/full_fig_p025_11.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

91 extracted references · 1 canonical work pages

  1. [1]

    proptest: Property testing framework for Rust

    2017. proptest: Property testing framework for Rust. https://github.com/proptest- rs/proptest

    2017

  2. [2]

    Useful Properties to Check with Fuzz Testing

    2021. Useful Properties to Check with Fuzz Testing. https://www.mayhem. security/blog/useful-properties-to-check-with-fuzz-testing. Accessed: Novem- ber 4, 2025. PBFuzz: Agentic Directed Fuzzing for PoV Generation , ,

    2021

  3. [3]

    Cursor: The AI Code Editor

    2024. Cursor: The AI Code Editor. https://www.cursor.com/. Accessed: December 3, 2025

    2024

  4. [4]

    AI Cyber Challenge (AIxCC)

    n.d.. AI Cyber Challenge (AIxCC). https://aicyberchallenge.com/. Accessed: November 4, 2025

    2025

  5. [5]

    Pranjal Aggarwal, Seungone Kim, Jack Lanchantin, Sean Welleck, Jason Weston, Ilia Kulikov, and Swarnadeep Saha. 2025. OptimalThinkingBench: Evaluating Over and Underthinking in LLMs. arXiv:2508.13141 [cs.CL] https://arxiv.org/ abs/2508.13141

    arXiv 2025

  6. [6]

    Vivi Andersson, Sofia Bobadilla, Harald Hobbelhagen, and Martin Monperrus

  7. [7]

    Anthropic. 2024. Claude: Anthropic’s AI Assistant. https://www.anthropic.com/ claude. Accessed: December 3, 2025

    2024

  8. [8]

    Cornelius Aschermann, Sergej Schumilo, Ali Abbasi, and Thorsten Holz. 2020. IJON: Exploring Deep State Spaces via Fuzzing. InIEEE Symposium on Security and Privacy (Oakland). IEEE

    2020

  9. [9]

    Domagoj Babić, Stefan Bucur, Yaohui Chen, Franjo Ivančić, Tim King, Markus Kusano, Caroline Lemieux, László Szekeres, and Wei Wang. 2019. Fudge: fuzz driver generation at scale. InACM SIGSOFT Symposium on the Foundations of Software Engineering (FSE)

    2019

  10. [10]

    Andrew Bao, Wenjia Zhao, Yanhao Wang, Yueqiang Cheng, Stephen McCamant, and Pen-Chung Yew. 2025. From Alarms to Real Bugs: Multi-target Multi-step Directed Greybox Fuzzing for Static Analysis Result Verification. InUSENIX Security Symposium (Security)

    2025

  11. [11]

    Marcel Böhme and Soumya Paul. 2014. On the efficiency of automated testing. InProceedings of the 22nd ACM SIGSOFT international symposium on foundations of software engineering. 632–642

    2014

  12. [12]

    Marcel Böhme and Soumya Paul. 2015. A probabilistic analysis of the efficiency of automated software testing.IEEE Transactions on Software Engineering42, 4 (2015), 345–360

    2015

  13. [13]

    Marcel Böhme, Van-Thuan Pham, Manh-Dung Nguyen, and Abhik Roychoud- hury. 2017. Directed greybox fuzzing. InACM Conference on Computer and Communications Security (CCS)

    2017

  14. [14]

    Sicong Cao, Biao He, Xiaobing Sun, Yu Ouyang, Chao Zhang, Xiaoxue Wu, Ting Su, Lili Bo, Bin Li, Chuanlei Ma, et al. 2023. ODDFUZZ: Discovering Java Deserialization Vulnerabilities via Structure-Aware Directed Greybox Fuzzing. InIEEE Symposium on Security and Privacy (Oakland)

    2023

  15. [15]

    Chuyang Chen, Brendan Dolan-Gavitt, and Zhiqiang Lin. 2025. ELFuzz: Effi- cient Input Generation via LLM-driven Synthesis Over Fuzzer Space. InUSENIX Security Symposium (Security)

    2025

  16. [16]

    Hongxu Chen, Yinxing Xue, Yuekang Li, Bihuan Chen, Xiaofei Xie, Xiuheng Wu, and Yang Liu. 2018. Hawkeye: Towards a desired directed grey-box fuzzer. In ACM Conference on Computer and Communications Security (CCS)

    2018

  17. [17]

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021. Evaluating Large Language Models Trained on Code.arXiv preprint arXiv:2107.03374(2021). https://arxiv.org/abs/2107.03374

    Pith/arXiv arXiv 2021

  18. [18]

    Peng Chen and Hao Chen. 2018. Angora: Efficient Fuzzing by Principled Search. InIEEE Symposium on Security and Privacy (Oakland)

    2018

  19. [19]

    Xu Chen, Ningning Cui, Zhe Pan, Liwei Chen, Gang Shi, and Dan Meng. 2025. Critical Variable State-Aware Directed Greybox Fuzzing. InInternational Confer- ence on Software Engineering (ICSE)

    2025

  20. [20]

    Yaohui Chen, Peng Li, Jun Xu, Shengjian Guo, Rundong Zhou, Yulong Zhang, Tao Wei, and Long Lu. 2020. Savior: Towards bug-driven hybrid testing. InIEEE Symposium on Security and Privacy (Oakland)

    2020

  21. [21]

    Koen Claessen and John Hughes. 2000. QuickCheck: a lightweight tool for random testing of Haskell programs. InACM SIGPLAN International Conference on Functional Programming (ICFP). 268–279

    2000

  22. [22]

    Koen Claessen and John Hughes. 2011. QuickCheck: a lightweight tool for random testing of Haskell programs.Acm sigplan notices46, 4 (2011), 53–64

    2011

  23. [23]

    Yinlin Deng, Chunqiu Steven Xia, Haoran Peng, Chenyuan Yang, and Lingming Zhang. 2023. Large language models are zero-shot fuzzers: Fuzzing deep-learning libraries via large language models. InInternational Symposium on Software Testing and Analysis (ISSTA)

    2023

  24. [24]

    Xiaotao Feng, Xiaogang Zhu, Kun Hu, Jincheng Wang, Yingjie Cao, Guang Gong, and Jianfeng Pan. 2025. Fuzzing: Randomness? Reasoning! Efficient Directed Fuzzing via Large Language Models.arXiv preprint arXiv:2507.22065(2025)

    Pith/arXiv arXiv 2025

  25. [25]

    2020.{AFL++}: Combining incremental steps of fuzzing research

    Andrea Fioraldi, Dominik Maier, Heiko Eißfeldt, and Marc Heuse. 2020.{AFL++}: Combining incremental steps of fuzzing research. In14th USENIX workshop on offensive technologies (WOOT 20)

    2020

  26. [26]

    Andrea Fioraldi, Dominik Christian Maier, Dongjia Zhang, and Davide Balzarotti

  27. [27]

    Elia Geretto, Andrea Jemmett, Cristiano Giuffrida, and Herbert Bos. 2025. LibAFLGo: Evaluating and Advancing Directed Greybox Fuzzing

    2025

  28. [28]

    Harrison Goldstein, Joseph W Cutler, Daniel Dickstein, Benjamin C Pierce, and Andrew Head. 2024. Property-based testing in practice. InInternational Confer- ence on Software Engineering (ICSE)

    2024

  29. [29]

    Google. 2010. honggfuzz. https://github.com/google/honggfuzz

    2010

  30. [30]

    Google. 2016. OSS-Fuzz - continuous fuzzing of open source software. https: //github.com/google/oss-fuzz

    2016

  31. [31]

    Google. 2020. FuzzTest: A C++ testing framework for property-based fuzzing. https://github.com/google/fuzztest

    2020

  32. [32]

    2025.Agentic Design Patterns: A Hands-On Guide to Building Intelligent Systems

    Antonio Gullí. 2025.Agentic Design Patterns: A Hands-On Guide to Building Intelligent Systems. Springer Cham. https://doi.org/10.1007/978-3-032-01402-3 Edition 1; Pages XLIII, 427

    work page doi:10.1007/978-3-032-01402-3 2025

  33. [33]

    Ahmad Hazimeh, Adrian Herrera, and Mathias Payer. 2020. Magma: A ground- truth fuzzing benchmark.Proceedings of the ACM on Measurement and Analysis of Computing Systems4, 3 (2020), 1–29

    2020

  34. [34]

    Heqing Huang, Yiyuan Guo, Qingkai Shi, Peisen Yao, Rongxin Wu, and Charles Zhang. 2022. Beacon: Directed grey-box fuzzing with provable path pruning. In IEEE Symposium on Security and Privacy (Oakland)

    2022

  35. [35]

    Heqing Huang, Peisen Yao, CHIU Hung-Chun, Yiyuan Guo, and Charles Zhang

  36. [36]

    Kyriakos Ispoglou, Daniel Austin, Vishwath Mohan, and Mathias Payer. 2020. FuzzGen: Automatic fuzzer generation. InUSENIX Security Symposium (Security)

    2020

  37. [37]

    Zongze Jiang, Ming Wen, Jialun Cao, Xuanhua Shi, and Hai Jin. 2024. Towards understanding the effectiveness of large language models on directed test in- put generation. InIEEE/ACM International Conference on Automated Software Engineering (ASE)

    2024

  38. [38]

    Taesoo Kim, HyungSeok Han, Soyeon Park, Dae R Jeong, Dohyeok Kim, Dongk- wan Kim, Eunsoo Kim, Jiho Kim, Joshua Wang, Kangsu Kim, et al . 2025. AT- LANTIS: AI-driven Threat Localization, Analysis, and Triage Intelligence System. arXiv preprint arXiv:2509.14589(2025)

    arXiv 2025

  39. [39]

    Tae Eun Kim, Jaeseung Choi, Kihong Heo, and Sang Kil Cha. 2023. DAFL: Directed Grey-box Fuzzing guided by Data Dependency. InUSENIX Security Symposium (Security)

    2023

  40. [40]

    Yihe Li, Ruijie Meng, and Gregory J Duck. 2025. Large language model pow- ered symbolic execution.Proceedings of the ACM on Programming Languages9, OOPSLA2 (2025), 3148–3176

    2025

  41. [41]

    Hongliang Liang, Lin Jiang, Lu Ai, and Jinyi Wei. 2020. Sequence directed hybrid fuzzing. InIEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)

    2020

  42. [42]

    Peihong Lin, Pengfei Wang, Xu Zhou, Wei Xie, Kai Lu, and Gen Zhang

  43. [43]

    Dongge Liu, Jonathan Metzman, Oliver Chang, and Google Open Source Se- curity Team. 2023. AI-Powered Fuzzing: Breaking the Bug Hunting Bar- rier. https://security.googleblog.com/2023/08/ai-powered-fuzzing-breaking-bug- hunting.html

    2023

  44. [44]

    Kangjie Lu, Chengyu Song, Taesoo Kim, and Wenke Lee. 2016. Unisan: Proactive kernel memory initialization to eliminate data leakages. InACM Conference on Computer and Communications Security (CCS)

    2016

  45. [45]

    Changhua Luo, Wei Meng, and Penghui Li. 2023. Selectfuzz: Efficient directed fuzzing with selective path exploration. InIEEE Symposium on Security and Privacy (Oakland)

    2023

  46. [46]

    HyperGo: Probability-based Directed Hybrid Fuzzing.arXiv preprint arXiv:2307.07815(2023)

    Pith/arXiv arXiv 2023

  47. [47]

    Yunlong Lyu, Yuxuan Xie, Peng Chen, and Hao Chen. 2024. Prompt fuzzing for fuzz driver generation. InACM Conference on Computer and Communications Security (CCS)

    2024

  48. [48]

    Muhammad Maaz, Liam DeVoe, Zac Hatfield-Dodds, and Nicholas Carlini. 2025. Agentic Property-Based Testing: Finding Bugs Across the Python Ecosystem. arXiv preprint arXiv:2510.09907(2025)

    arXiv 2025

  49. [49]

    David R MacIver, Zac Hatfield-Dodds, et al. 2019. Hypothesis: A new approach to property-based testing.Journal of Open Source Software4, 43 (2019), 1891

    2019

  50. [50]

    Zhengxiong Luo, Huan Zhao, Dylan Wolff, Cristian Cadar, and Abhik Roychoud- hury. 2026. Agentic Concolic Execution. InIEEE Symposium on Security and Privacy (Oakland)

    2026

  51. [51]

    Jiaqi Peng, Feng Li, Bingchang Liu, Lili Xu, Binghong Liu, Kai Chen, and Wei Huo. 2019. 1dvul: Discovering 1-day vulnerabilities through binary patches. In International Conference on Dependable Systems and Networks (DSN)

    2019

  52. [52]

    Alex Rebert. 2021. What is Property-based Testing? https://www.mayhem. security/blog/what-is-property-based-testing. Accessed: November 4, 2025

    2021

  53. [53]

    Huanyao Rong, Wei You, Xiaofeng Wang, and Tianhao Mao. 2024. Toward Unbi- ased Multiple-Target Fuzzing with Path Diversity. InUSENIX Security Symposium (Security)

    2024

  54. [54]

    Ruijie Meng, Martin Mirchev, Marcel Böhme, and Abhik Roychoudhury. 2024. Large language model guided protocol fuzzing. InAnnual Network and Distributed System Security Symposium (NDSS)

    2024

  55. [55]

    Erik Schluntz and Barry Zhang. 2024. Building effective agents. https://www. anthropic.com/engineering/building-effective-agents. Accessed: November 4, 2025

    2024

  56. [56]

    Kosta Serebryany. 2016. Continuous fuzzing with libfuzzer and addresssanitizer. InIEEE Cybersecurity Development (SecDev). IEEE. , , Haochen Zeng, Andrew Bao, Jiajun Cheng, and Chengyu Song

    2016

  57. [57]

    Abhishek Shah, Dongdong She, Samanway Sadhu, Krish Singal, Peter Coffman, and Suman Jana. 2022. MC2: Rigorous and Efficient Directed Greybox Fuzzing. InACM Conference on Computer and Communications Security (CCS)

    2022

  58. [58]

    Gaetano Sapia and Marcel Böhme. 2026. Scaling Security Testing by Addressing the Reachability Gap. InInternational Conference on Software Engineering (ICSE)

    2026

  59. [59]

    Dongdong She, Adam Storek, Yuchong Xie, Seoyoung Kweon, Prashast Srivastava, and Suman Jana. 2024. Fox: Coverage-guided fuzzing as online stochastic control. InACM Conference on Computer and Communications Security (CCS)

    2024

  60. [60]

    Ze Sheng, Qingxiao Xu, Jianwei Huang, Matthew Woodcock, Heqing Huang, Alastair F Donaldson, Guofei Gu, and Jeff Huang. 2025. All You Need Is A Fuzzing Brain: An LLM-Powered System for Automated Vulnerability Detection and Patching.arXiv preprint arXiv:2509.07225(2025)

    Pith/arXiv arXiv 2025

  61. [61]

    Wenxuan Shi, Yunhang Zhang, Xinyu Xing, and Jun Xu. 2024. Harnessing large language models for seed generation in greybox fuzzing.arXiv preprint arXiv:2411.18143(2024)

    Pith/arXiv arXiv 2024

  62. [62]

    Asif Shahriar, Md Nafiu Rahman, Sadif Ahmed, Farig Sadeque, and Md Rizwan Parvez. 2025. A Survey on Agentic Security: Applications, Threats and Defenses. arXiv preprint arXiv:2510.06445(2025)

    arXiv 2025

  63. [63]

    Haoxin Tu, Seongmin Lee, Yuxian Li, Peng Chen, Lingxiao Jiang, and Marcel Böhme. 2025. Large Language Model-Driven Concolic Execution for Highly Structured Test Input Generation.arXiv preprint arXiv:2504.17542(2025)

    arXiv 2025

  64. [64]

    Vasudev Vikram, Caroline Lemieux, Joshua Sunshine, and Rohan Padhye. 2023. Can large language models write good property-based tests?arXiv preprint arXiv:2307.04346(2023)

    Pith/arXiv arXiv 2023

  65. [65]

    Jinghan Wang, Yue Duan, Wei Song, Heng Yin, and Chengyu Song. 2019. Be Sensitive and Collaborative: Analyzing Impact of Coverage Metrics in Grey- box Fuzzing. InInternational Symposium on Research in Attacks, Intrusions and Defenses (RAID)

    2019

  66. [66]

    Noah Shinn, Federico Cassano, Beck Labash, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: Language agents with verbal reinforcement learning, 2023.URL https://arxiv. org/abs/2303.113661 (2023)

    Pith/arXiv arXiv 2023

  67. [67]

    Felix Weissberg, Jonas Möller, Tom Ganz, Erik Imgrund, Lukas Pirch, Lukas Seidel, Moritz Schloegel, Thorsten Eisenhofer, and Konrad Rieck. 2024. Sok: Where to fuzz? assessing target selection methods in directed fuzzing. InProceedings of the 19th ACM Asia Conference on Computer and Communications Security. 1539–1553

    2024

  68. [68]

    Chunqiu Steven Xia, Matteo Paltenghi, Jia Le Tian, Michael Pradel, and Ling- ming Zhang. 2024. Fuzz4all: Universal fuzzing with large language models. In International Conference on Software Engineering (ICSE)

    2024

  69. [69]

    Hanxiang Xu, Wei Ma, Ting Zhou, Yanjie Zhao, Kai Chen, Qiang Hu, Yang Liu, and Haoyu Wang. 2025. CKGFuzzer: LLM-Based Fuzz Driver Generation Enhanced By Code Knowledge Graph. InIEEE/ACM International Conference on Software Engineering: Companion Proceedings (ICSE-Companion)

    2025

  70. [70]

    Pengfei Wang, Xu Zhou, Kai Lu, Tai Yue, and Yingying Liu. 2020. Sok: The progress, challenges, and perspectives of directed greybox fuzzing.Challenges, and perspectives of directed greybox fuzzing(2020)

    2020

  71. [71]

    Chenyuan Yang, Zijie Zhao, Zichen Xie, Haoyu Li, and Lingming Zhang. 2025. Knighter: Transforming static analysis with llm-synthesized checkers. InACM Symposium on Operating Systems Principles (SOSP)

    2025

  72. [72]

    Yupeng Yang, Shenglong Yao, Jizhou Chen, and Wenke Lee. 2025. Hybrid Lan- guage Processor Fuzzing via{LLM-Based} Constraint Solving. InUSENIX Secu- rity Symposium (Security)

    2025

  73. [73]

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. 2022. React: Synergizing reasoning and acting in language models. InInternational Conference on Learning Representations (ICLR)

    2022

  74. [74]

    Hanxiang Xu, Yanjie Zhao, and Haoyu Wang. 2025. Directed Greybox Fuzzing via Large Language Model.arXiv preprint arXiv:2505.03425(2025)

    Pith/arXiv arXiv 2025

  75. [75]

    Michal Zalewski. 2014. American Fuzzy Lop.(2014). http://lcamtuf.coredump.cx/ afl

    2014

  76. [76]

    Hongxiang Zhang, Yuyang Rong, Yifeng He, and Hao Chen. 2024. Llamafuzz: Large language model enhanced greybox fuzzing.arXiv preprint arXiv:2406.07714 (2024)

    arXiv 2024

  77. [77]

    Kunpeng Zhang, Zongjie Li, Daoyuan Wu, Shuai Wang, and Xin Xia. 2025. Low- Cost and Comprehensive Non-textual Input Fuzzing with LLM-Synthesized Input Generators. InUSENIX Security Symposium (Security)

    2025

  78. [78]

    Zijun Yao, Yantao Liu, Yanxu Chen, Jianhui Chen, Junfeng Fang, Lei Hou, Juanzi Li, and Tat-Seng Chua. 2025. Are Reasoning Models More Prone to Hallucination? arXiv:2505.23646 [cs.CL] https://arxiv.org/abs/2505.23646

    Pith/arXiv arXiv 2025

  79. [79]

    Kunlun Zhu, Zijia Liu, Bingxuan Li, Muxin Tian, Yingxuan Yang, Jiaxun Zhang, Pengrui Han, Qipeng Xie, Fuyang Cui, Weijia Zhang, et al . 2025. Where LLM Agents Fail and How They can Learn From Failures.arXiv preprint arXiv:2509.25370(2025)

    arXiv 2025

  80. [80]

    Peiyuan Zong, Tao Lv, Dawei Wang, Zizhuang Deng, Ruigang Liang, and Kai Chen. 2020. FuzzGuard: Filtering out unreachable inputs in directed grey-box fuzzing through deep learning. InUSENIX Security Symposium (Security). 9 Appendix 9.1 Prompts # Security Research Autonomous Agent ## Mission You are a security expert specializing in C/C++ vulnerabilities. Y...

    2020

Showing first 80 references.