Pith. sign in

REVIEW 3 major objections 4 minor 74 references

Direct fuzzing misses crashes because its starting seeds are blind; this paper argues that an agentic LLM pipeline can write crash-triggering seeds up front, cutting crash time by 11.5x to 14.7x and unlocking 16 previously unreachable bugs.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 07:43 UTC pith:6JOJMVS3

load-bearing objection SeedSmith is a solid engineering contribution, but the headline speedups are not yet trustworthy because the LLM could be recalling public PoCs rather than reasoning. the 3 major comments →

arxiv 2607.08949 v2 pith:6JOJMVS3 submitted 2026-07-09 cs.CR cs.SE

SeedSmith: LLM-Driven Seed Synthesis for Directed Fuzzing

Junmin Zhu , Siyu Liu , Jie Hu , Fabio Gritti , Ati Priya Bajaj , Hulin Wang , Wenbo Guo , Tiffany Bao
show 2 more authors
Christopher Kruegel Giovanni Vigna
This is my paper
classification cs.CR cs.SE
keywords directed fuzzingseed generationlarge language modelsagentic LLMindirect callscrash preconditionsvulnerability discoveryseed corpus
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper claims that directed fuzzers fail for two reasons: static analysis misses indirect calls, leaving the target path invisible to distance guidance, and crash preconditions are too precise for blind mutation to satisfy. The proposed fix is not a new fuzzer but a better initial seed corpus: seeds that already encode the correct control-flow path and key crash conditions turn the fuzzer's task into local refinement. To build such seeds, the paper presents a two-stage LLM pipeline that mimics a security analyst: one agent iteratively searches the codebase to resolve indirect calls and extract crash preconditions, and a second agent converts that analysis into concrete seed files, refined by execution feedback. In evaluation, seeds from the pipeline cut crash time by over an order of magnitude in geometric mean across four fuzzers on a widely used benchmark, and triggered 16 real-world bugs that default seeds never reached across 10 projects.

Core claim

The central claim is that the bottlenecks of directed fuzzing—missing indirect call edges and semantically demanding crash preconditions—can be bypassed at the seed layer rather than in the fuzzer. The system's analysis agent starts from the sink function and, using a context-aware code-search tool, iteratively reconstructs the execution path from the harness, resolves function-pointer and virtual-dispatch calls that static call graphs omit, and identifies the input-level conditions that must hold for the crash to fire. A second agent consumes the resulting report and writes Python scripts that construct structurally valid seeds; those seeds are run against a sanitizer-instrumented binary an

What carries the argument

The load-bearing mechanism is the two-stage agentic pipeline. The analysis agent searches a project index with a context-aware retrieval tool that returns whole function bodies, type definitions, or configuration files rather than fixed line windows; this lets it resolve indirect calls by reading type information and function bodies, and discover crash preconditions by tracing data flow. A path-optimization step compresses the over-approximated static call graph into a single linearized path with a connector node for divergent middles, keeping the search within context limits. The seed generation agent then converts the analysis report into Python scripts that build inputs with format librar

Load-bearing premise

The load-bearing premise is that a pretrained LLM genuinely reconstructs reachability and crash preconditions from the code through iterative search, rather than recalling benchmark-specific code or proof-of-concept inputs from its training data; the paper tests the pipeline end-to-end but never isolates this factor.

What would settle it

Run the pipeline on a set of sink functions whose crashing inputs were created after the LLM's training cutoff (e.g., freshly injected vulnerabilities or private targets) and compare the one-shot crash rate and crash-time speedups against the reported numbers; a large drop would indicate that memorization, not reconstruction, is the mechanism. A second check is to strip the analysis agent's code-search tool and feed only the static call-graph hint: if one-shot crashes persist without any tool access, the LLM alone is doing the work.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Seeds are fuzzer-agnostic: any mutation-based fuzzer, directed or coverage-guided, can drop them into its initial corpus without modification and inherit the crash-time speedups.
  • The effect is front-loaded: several of the 23 benchmark bugs crashed on the first generated seed, before any mutation, and cumulative crash curves plateau within the first hour.
  • The approach gives general-purpose fuzzers directed-crash capability: a coverage-guided fuzzer using the seeds outperformed a dedicated directed fuzzer at exposing targeted crashes.
  • On real-world vulnerabilities, the primary benefit is expanding reachable bugs (16 previously unreachable crashes across 10 projects) rather than merely accelerating already-reachable ones.
  • The upfront cost is small relative to a 24-hour fuzzing campaign—roughly $5 and 11 minutes per target on average—so the pipeline is practical as a pre-fuzzing step.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Because the evaluation does not control for training-data memorization, the strongest non-obvious inference is that the pipeline's real-world ceiling depends on whether the LLM is reconstructing reachability from code or recalling benchmark-specific exploits; a test on freshly created, non-public vulnerabilities would separate these.
  • The analysis/seed-generation split suggests the design can ride the trend of improving code-reasoning models: a stronger analysis model should improve seed quality with no pipeline change—and equally, the system is hostage to a single model's weaknesses.
  • Pairing the seed generator with an automatic sink locator is the natural route from known-vulnerability reproduction to zero-day discovery; the paper reports doing this in a competition setting but evaluates only the N-day case.
  • The pattern of results implies an editing or triage workflow: run the seed generator first, then fuzz, and focus remaining compute on the few targets that even good seeds do not crack.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. SeedSmith is an agentic LLM-based seed-generation front-end for directed fuzzing of C/C++ programs. Starting from a harness and a user-provided sink function, an Analysis Agent uses a context-aware code-search tool to reconstruct a path from the harness to the sink, resolve indirect calls that static call-graph analysis misses, and identify crash preconditions; a Seed Generation Agent then emits Python scripts that construct concrete seeds, which are validated against a sanitizer-instrumented binary in a refinement loop of up to ten rounds. The resulting seeds are added to the initial corpus of a downstream fuzzer without modifying the fuzzer. The paper evaluates SeedSmith on 23 Magma bugs with AFL++, FairFuzz, AFLGo, and AFLRun, and on 115 ARVO targets with AFL++ and AFLRun, comparing Default, SeedSmith-only, and SeedSmith-combined seed configurations. It reports geometric-mean crash-time speedups of 11.51x (AFL++) to 14.66x (AFLGo) on Magma, 16 previously un-triggered ARVO bugs across 10 projects, a comparison against Locus on Magma, and ablations of the scan strategy and CodeQL-based control-flow support.

Significance. If the quantitative claims hold, SeedSmith addresses a real and well-motivated problem: directed fuzzers frequently fail to trigger crashes because static call graphs miss indirect edges and because crash preconditions are invisible to coverage feedback. The intervention at the seed-corpus level is clean, fuzzer-agnostic, and supported by a large evaluation that is more extensive than most work in this area: 23 Magma bugs, 115 ARVO targets, four fuzzers, ten trials per configuration, per-bug Mann-Whitney p-values, and validation of seeds against sanitizer-instrumented binaries. The cost analysis (Table 4) and the appendices with full prompts and a sample analysis report also strengthen reproducibility. The principal risk is that the central generalization claim is not separated from the possibility that the LLM recalls public PoCs and vulnerable code from training data, since Magma and ARVO are public benchmarks and the pipeline feeds exact sink code into the model. That risk is material and needs to be resolved before the headline claims can be accepted at face value.

major comments (3)
  1. [§4.2, §6.2.2, Table 3, and §7] The central claim—that agentic code exploration recovers reachability and crash preconditions that static analysis misses—is not separated from the possibility that Claude Sonnet 4 recalls public PoCs and vulnerable code from its training data. Magma bugs are injected from public CVEs and ARVO is a public dataset with PoCs; the pipeline gives the model the exact sink function code and the harness. The five Magma one-shot crashes and the 16 ARVO unlocks could be produced by memorization rather than by the agent's iterative search and reasoning. The evaluation contains no contamination control: no held-out bugs introduced after the model training cutoff, no refactored/variable-renamed code variants, no comparison against a one-shot LLM prompt (or non-agentic retrieval baseline), and no recall probe. Since the abstract claims the seeds improve any downstream mutation-based fuzzer on novel t
  2. [§6.1 Metrics; Table 4] The speedup computation treats '1-shot' crashes as 1 s of fuzzer time while ignoring the 678 s average seed-generation cost per target reported in Table 4. For the five Magma one-shot cases, the per-bug ratio over a 24h default timeout is 86400 under the current convention; including the 678s generation cost reduces each such ratio to about 127. Because these entries are the largest terms in the geometric mean, the headline 11.51x–14.66x speedups are materially dependent on this convention. The paper discloses the convention, and the cost may be small relative to a full 24h campaign, but the metric is labeled 'crash time' and the abstract presents the speedups without qualification. Please recompute all geomeans with end-to-end time (seed generation + fuzzing) or clearly separate fuzzer-only and end-to-end numbers in every table and in the abstract.
  3. [§6.2.2, Table 3, and Conclusion] The abstract's '16 previously unreachable bugs' is stronger than what the experiments show: the comparison is against AFL++ and AFLRun with default seeds within a 24h budget. These bugs may be reachable by other fuzzers or with more time, so the claim should be 'not triggered by the evaluated baseline configurations within 24h.' The statistical support for the ARVO speedup is also uneven: the 3.09x/3.02x geomeans are dominated by the 16 unlock cases, while on the 21/31 bugs that both configurations trigger the speedup is 1.46x/1.71x and not significant (p=0.50 and p=0.18, respectively). This distinction should be reflected in the abstract, not only buried in §6.2.2. Additionally, the conclusion reports 'p=0.58' for the shared-bug comparison, which does not match the p-values given in §6.2.2.
minor comments (4)
  1. [§5 Implementation] The context-window thresholds (X=10, Y=2, 50K-token cap, 30-tool-call halt, 10 refinement rounds) are described as empirically chosen defaults, but no sensitivity analysis is reported. Since these parameters affect both cost and effectiveness, reporting one or two variations (e.g., X ∈ {5, 20} or rounds ∈ {5, 15}) would make the robustness of the pipeline easier to assess.
  2. [§6.2.3 / Table 4] The cost analysis is reported only for Magma. The ARVO evaluation uses 115 targets across 26 projects, each requiring an analysis report and seed generation; reporting total or average ARVO cost would help readers judge end-to-end deployability.
  3. [§6.4, Tables 5 and 6] The control-flow-support ablation is computed on only 8 Magma bugs where CodeQL succeeds, and the geomeans in Table 6 are dominated by a few timeout-to-one-shot ratios. For FairFuzz, the 540.9x figure is based on two non-timeout ratios and is explicitly non-comparable. The raw per-bug ratios should be shown so the reader can see how fragile these marginal speedups are.
  4. [Throughout] Minor text issues: 'Geneartion' in Figure 3; 'the the vulnerability' in §4.2; 'they they' in §1; 'less ineffective' in §7; and the inconsistency in the p-value for the shared ARVO subset between §6.2.2 (p=0.50/0.18) and the Conclusion (p=0.58). An artifact/reproducibility statement with code links and exact model versions would also help.

Circularity Check

0 steps flagged

No derivation-level circularity: SeedSmith's claims are validated by external sanitizer-instrumented executions and external fuzz benchmarks.

full rationale

The paper's evidence chain is empirical, not definitional. SeedSmith takes a sink and harness, uses an LLM agent to explore code, emits Python scripts, executes the generated seeds against sanitizer-instrumented builds in a refinement loop, and then passes the resulting corpus to unmodified fuzzers. Every headline quantity—11.51x/14.66x geomean speedups, 22/23 Magma bugs, 16 ARVO unlocks—is measured against external ground truth (Magma canaries; ARVO PoC dedup tokens) and compared with Default/No-seed corpora under identical fuzzer configurations. No equation or 'prediction' is defined in terms of the fitted inputs: X=10, Y=2, the 50K token cap, 30-tool halt, and 10 refinement rounds are openly described as empirically chosen defaults, and the paper does not relabel them as predictions. The ARVO benchmark includes a co-author of this paper (Tiffany Bao), which is a benchmark-independence caveat, but the dataset is public and the crash labels are external ground truth, so the citation is not load-bearing. The LLM-memorization/contamination concern is a real external-validity threat, but it is a hypothesis about the model's training data and cannot be exhibited as a reduction in the paper's own derivation; per the rules, it is correctness risk, not circularity. The limitations section (Section 7, 'Precision of the Sink Function' and 'Zero-Day Discovery') candidly concedes that sink precision is assumed and that zero-day use requires an external sink-discovery component, so the paper does not silently rewrite its assumptions as conclusions. Thus no circular step reaches the evidence bar.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 1 invented entities

The paper's central contribution is an empirical LLM pipeline, so the load-bearing free parameters are the empirically tuned retrieval/window/halt knobs and the 1s 1-shot metric convention. The axioms are domain assumptions about static-analysis incompleteness, LLM capability, and seed-to-fuzzer transfer. No new physical entities are introduced.

free parameters (6)
  • X (context lines for type/structure matches) = 10
    Section 5: 'X=10 suffices to capture most struct/typedef definitions... chosen empirically.'
  • Y (context lines for non-source matches) = 2
    Section 5: 'Y=2 provides enough context for key-value entries in configuration files... chosen empirically.'
  • Tool output token cap = 50K tokens
    Section 5: if combined results exceed 50K tokens the output is discarded; a default that worked well in experiments.
  • Analysis early-termination threshold = 30 consecutive no-match tool calls
    Section 5: 'early termination heuristic... defaults that worked well in our experiments.'
  • Seed refinement rounds = 10
    Section 4.3/5: 'iterative refinement loop of up to ten rounds.'
  • 1-shot crash time convention = 1 s
    Section 6.1: '1-shot denotes a crash from the LLM-generated seed before any fuzzer mutation (counted as 1 s when computing speedups)'; this ad hoc convention inflates reported geomean speedups.
axioms (5)
  • domain assumption C1: directed fuzzers fail because static call-graph analysis misses indirect calls; C2: blind mutation cannot satisfy crash preconditions.
    Section 3 motivates with nginx and openjpeg examples; the two-challenge framing is a premise for the whole design, not proven generally.
  • domain assumption Pretrained LLMs can resolve indirect calls, infer crash preconditions, and reconstruct input formats by reading code, without fine-tuning on the target.
    Central capability assumption invoked in Sections 2 and 4; end-to-end results support it but do not separate genuine reasoning from memorized public PoCs.
  • domain assumption A seed that encodes the correct path and preconditions reduces the fuzzer's task to local refinement, and this transfers to any mutation-based fuzzer.
    Design hypothesis from Section 3.3; tested only on AFL++, AFLGo, AFLRun, and FairFuzz, with some per-bug regressions.
  • ad hoc to paper The path-optimized linearized call graph preserves the information the agent needs while fitting in the context budget.
    Section 4.1 path optimization collapses divergent middles into connector nodes; a heuristic with no completeness guarantee.
  • domain assumption RMST with 24h substitution for one-sided timeouts yields valid crash-time comparisons.
    Section 6.1 metric choice; substituting 24h when only one side times out biases speedups upward and affects significance.
invented entities (1)
  • Connector node in path optimization no independent evidence
    purpose: Internal representation collapsing divergent middle segments of candidate harness-to-sink paths into a single expandable node (Section 4.1).
    A software abstraction of the described system, not a new physical/mathematical entity with a falsifiable handle; listed for completeness only.

pith-pipeline@v1.3.0-alltime-deepseek · 28546 in / 17476 out tokens · 178188 ms · 2026-08-02T07:43:41.354153+00:00 · methodology

0 comments
read the original abstract

Directed fuzzing steers fuzzers toward user-defined sink functions to identify vulnerabilities, but it frequently fails to trigger crashes even after long campaigns. We identify two challenges that prevent directed fuzzers from exposing crashes: incomplete static analysis of indirect calls, which leaves reachable paths invisible to distance-based guidance, and lack of semantic guidance for crash preconditions, which blind mutation cannot satisfy within practical time budgets. A natural intervention point is the initial seed corpus: seeds that encode the right control-flow path and satisfy key crash preconditions shift fuzzing from blind exploration to local refinement. Existing seed generation approaches address neither: grammar-based and format-driven methods produce structurally valid inputs with no sink awareness, while LLM-based methods either lack sink targeting or inherit static analysis limitations through one-shot prompting. We present SeedSmith, an agentic LLM pipeline that replicates a security analyst's workflow: starting from a sink, it iteratively explores the codebase, resolves indirect calls, identifies crash preconditions, and synthesizes concrete inputs that satisfy them. Because SeedSmith operates as a seed generation front-end, its seeds are fuzzer-agnostic and improve any downstream mutation-based fuzzer without modification. On Magma, fuzzers using SeedSmith seeds achieve geometric mean crash-time speedups of 11.51 times (AFL++) to 14.66 times (AFLGo) over default seeds. On ARVO, SeedSmith enables fuzzers to trigger 16 previously unreachable bugs spanning 10 projects with diverse input formats.

Figures

Figures reproduced from arXiv: 2607.08949 by Ati Priya Bajaj, Christopher Kruegel, Fabio Gritti, Giovanni Vigna, Hulin Wang, Jie Hu, Junmin Zhu, Siyu Liu, Tiffany Bao, Wenbo Guo.

Figure 1
Figure 1. Figure 1: Motivating example for C1: an nginx bug invisible to static call-graph guidance. Consider the bug shown in Figure 1a, injected into nginx [41] dur￾ing the DARPA AIxCC competition [1]. The vulnerability is a heap￾buffer overflow in ngx_sendfile_r, which allocates a fixed-size buffer using the hard-coded constant NGX_SENDFILE_R_MAXSIZE (Line 3) instead of the actual resource size. Triggering it requires the … view at source ↗
Figure 2
Figure 2. Figure 2: Motivating example for C2: an openjpeg crash whose six tile-geometry predicates must all hold. AFLRun correctly prioritizes inputs that reach the enclosing function, but it never triggers the crash. The fuzzer’s distance com￾putation provides no signal for satisfying the structural constraints: A seed that reaches the function through the correct path looks identical to one that satisfies all six predicate… view at source ↗
Figure 3
Figure 3. Figure 3: SeedSmith overview. The system processes a C/C++ project through three main stages: 1 Target preparation indexes functions, builds call graphs, and instruments the code with sanitizers. 2 The Analysis Agent explores the codebase using tool-assisted retrieval to understand how to reach and crash the sink function. 3 The Seed Generation Agent creates concrete test inputs based on the analysis, which serve as… view at source ↗
Figure 4
Figure 4. Figure 4: Cumulative unique crashes triggered over 24 h on [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 6
Figure 6. Figure 6: Cumulative unique crashes triggered over 24 h on [PITH_FULL_IMAGE:figures/full_fig_p009_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Simplified PDF018 trigger chain: from Page::displaySlice through AnnotInk::parseInkList to the null-pointer dereference in AnnotInk::draw. observes that AnnotInk::draw iterates over inkList and deref￾erences path->getCoordsLength() with no null check, and asks when inkList[i] can be null (Q1–Q3). Q2’s retrieval of parseInkList exposes the critical pattern: memset(inkList, 0, ...) zero-initializes the array… view at source ↗
Figure 8
Figure 8. Figure 8: Reasoning trace for PDF018: starting from raw har￾ness and sink, the agent’s six tool-driven queries fan out into sink-side and harness-side sub-goals and converge on a single crash-triggering seed. 6.3.2 Analysis of Accelerated Magma Targets. The pattern observed in PDF018 generalizes across the other nine targets highlighted in green in [PITH_FULL_IMAGE:figures/full_fig_p011_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: System Prompt of Code Exploration Agent [PITH_FULL_IMAGE:figures/full_fig_p017_9.png] view at source ↗
Figure 9
Figure 9. Figure 9: System Prompt of Code Exploration Agent [PITH_FULL_IMAGE:figures/full_fig_p016_9.png] view at source ↗

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

74 extracted references · 8 linked inside Pith

  1. [1]

    Artificial Intelligence Cyber Challenge

    2025. Artificial Intelligence Cyber Challenge. https://aicyberchallenge.com/

    2025

  2. [2]

    Neo4j Graph Database & Analytics – The Leader in Graph Databases

    2025. Neo4j Graph Database & Analytics – The Leader in Graph Databases. https://neo4j.com/

    2025

  3. [3]

    2026. CodeQL. https://codeql.github.com/

    2026

  4. [4]

    Langchain

    2026. Langchain. https://www.langchain.com/

    2026

  5. [5]

    tree-sitter

    2026. tree-sitter. https://tree-sitter.github.io/tree-sitter/

    2026

  6. [6]

    Asmita, Yaroslav Oliinyk, Michael Scott, Ryan Tsang, Chongzhou Fang, and Houman Homayoun. 2024. Fuzzing BusyBox: Leveraging LLM and Crash Reuse for Embedded Bug Unearthing. 883–900. https://www.usenix.org/conference/ usenixsecurity24/presentation/asmita

    2024

  7. [7]

    Roberto Baldoni, Emilio Coppa, Daniele Cono D’elia, Camil Demetrescu, and Irene Finocchi. 2018. A survey of symbolic execution techniques.ACM Computing Surveys (CSUR)51, 3 (2018), 1–39

    2018

  8. [8]

    Andrew Bao, Wenjia Zhao, Yanhao Wang, Yueqiang Cheng, Stephen McCamant, and Pen-Chung Yew. 2025. From Alarms to Real Bugs: Multi-target Multi-step Di- rected Greybox Fuzzing for Static Analysis Result Verification. 6977–6997. https: //www.usenix.org/conference/usenixsecurity25/presentation/bao-andrew

    2025

  9. [9]

    Tim Blazytko, Cornelius Aschermann, Moritz Schlögel, Ali Abbasi, Sergej Schu- milo, Simon Wörner, and Thorsten Holz. 2019. GRIMOIRE: Synthesizing Structure while Fuzzing. In28th USENIX Security Symposium (USENIX Security 19). USENIX Association, Santa Clara, CA, 1985–2002. https://www.usenix.org/conference/ usenixsecurity19/presentation/blazytko

    2019

  10. [10]

    Marcel Böhme, Van-Thuan Pham, Manh-Dung Nguyen, and Abhik Roychoudhury

  11. [11]

    Cristian Cadar, Daniel Dunbar, Dawson R Engler, et al. 2008. Klee: unassisted and automatic generation of high-coverage tests for complex systems programs.. InProceedings of the USENIX Symposium on Operating Systems Design and Imple- mentation, Vol. 8. 209–224

    2008

  12. [12]

    Hongxu Chen, Yinxing Xue, Yuekang Li, Bihuan Chen, Xiaofei Xie, Xiuheng Wu, and Yang Liu. 2018. Hawkeye: Towards a desired directed grey-box fuzzer. In Proceedings of the ACM Conference on Computer and Communications Security (CCS). 2095–2108

    2018

  13. [13]

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Pondé de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian...

    Pith/arXiv arXiv 2021

  14. [14]

    Peng Chen and Hao Chen. 2018. Angora: Efficient Fuzzing by Principled Search. doi:10.1109/SP.2018.00046

    arXiv 2018

  15. [15]

    Yinlin Deng, Chunqiu Steven Xia, Haoran Peng, Chenyuan Yang, and Lingming Zhang. 2023. Large Language Models Are Zero-Shot Fuzzers: Fuzzing Deep- Learning Libraries via Large Language Models. InProceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2023). Association for Computing Machinery, New York, NY, USA, ...

    arXiv 2023

  16. [16]

    Yinlin Deng, Chunqiu Steven Xia, Chenyuan Yang, Shizhuo Dylan Zhang, Shu- jing Yang, and Lingming Zhang. 2024. Large Language Models are Edge-Case Generators: Crafting Unusual Programs for Fuzzing Deep Learning Libraries. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineer- ing (ICSE ’24). Association for Computing Machinery,...

    arXiv 2024

  17. [17]

    Zhengjie Du, Yuekang Li, Yang Liu, and Bing Mao. 2022. Windranger: A directed greybox fuzzer driven by deviation basic blocks. InProceedings of the International Conference on Software Engineering. 2440–2451

    2022

  18. [18]

    Rafael Dutra, Rahul Gopinath, and Andreas Zeller. 2023. FormatFuzzer: Effective Fuzzing of Binary File Formats.ACM Trans. Softw. Eng. Methodol.33, 2, Article 53 (Dec. 2023), 29 pages. doi:10.1145/3628157

    doi:10.1145/3628157 2023

  19. [20]

    Elia Geretto, Andrea Jemmett, Cristiano Giuffrida, and Herbert Bos. 2025. LibAFLGo: Evaluating and Advancing Directed Greybox Fuzzing. In2025 IEEE 10th European Symposium on Security and Privacy (EuroS&P). 355–373. doi:10. 1109/EuroSP63326.2025.00029 ISSN: 2995-1356

    arXiv 2025

  20. [21]

    Patrice Godefroid, Nils Klarlund, and Koushik Sen. 2005. DART: Directed auto- mated random testing. InProceedings of the ACM SIGPLAN conference on Pro- gramming Language Design and Implementation. 213–223

    2005

  21. [22]

    Istvan Haller, Asia Slowinska, Matthias Neugschwandtner, and Herbert Bos. 2013. Dowsing for{Overflows}: A Guided Fuzzer to Find Buffer Boundary Violations. InProceedings of the USENIX Security Symposium. 49–64

    2013

  22. [23]

    Ahmad Hazimeh, Adrian Herrera, and Mathias Payer. 2020. Magma: A ground- truth fuzzing benchmark.Proceedings of the ACM on Measurement and Analysis of Computing Systems4, 3 (2020), 1–29. Conference acronym ’XX, June 03–05, 2018, Woodstock, NY Anonymous

    2020

  23. [24]

    Heqing Huang, Yiyuan Guo, Qingkai Shi, Peisen Yao, Rongxin Wu, and Charles Zhang. 2022. Beacon: Directed grey-box fuzzing with provable path pruning. In Proceedings of the IEEE Symposium on Security and Privacy (S&P). IEEE, 36–50

    2022

  24. [25]

    Heqing Huang, Peisen Yao, Hung-Chun Chiu, Yiyuan Guo, and Charles Zhang

  25. [26]

    Tae Eun Kim, Jaeseung Choi, Kihong Heo, and Sang Kil Cha. 2023. {DAFL}: Directed Grey-box Fuzzing guided by Data Dependency. InProceedings of the USENIX Security Symposium. 4931–4948

    2023

  26. [27]

    Gwangmu Lee, Woochul Shim, and Byoungyoung Lee. 2021. Constraint-guided directed greybox fuzzing. InProceedings of the USENIX Security Symposium. 3559– 3576

    2021

  27. [28]

    Caroline Lemieux and Koushik Sen. 2018. Fairfuzz: A targeted mutation strategy for increasing greybox fuzz testing coverage. InProceedings of the 33rd ACM/IEEE international conference on automated software engineering. 475–485

    2018

  28. [29]

    Ziyang Li, Saikat Dutta, and Mayur Naik. 2025. IRIS: LLM-Assisted Static Analysis for Detecting Security Vulnerabilities. doi:10.48550/arXiv.2405.17238 arXiv:2405.17238 [cs]

    Pith/arXiv arXiv doi:10.48550/arxiv.2405.17238 2025

  29. [30]

    Hongliang Liang, Xinglin Yu, Xianglin Cheng, Jie Liu, and Jin Li. 2023. Multiple targets directed greybox fuzzing.IEEE Transactions on Dependable and Secure Computing21, 1 (2023), 325–339

    2023

  30. [31]

    Dongge Liu, Oliver Chang, Jonathan metzman, Martin Sablotny, and Mihai Maruseac. 2024. OSS-Fuzz-Gen: Automated Fuzz Target Generation. https: //github.com/google/oss-fuzz-gen original-date: 2024-01-25T00:51:49Z

    2024

  31. [32]

    Danushka Liyanage, Marcel Böhme, Chakkrit Tantithamthavorn, and Stephan Lipp. 2023. Reachable Coverage: Estimating Saturation in Fuzzing. In2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE). 371–

    2023

  32. [33]

    Yunlong Lyu, Yuxuan Xie, Peng Chen, and Hao Chen. 2024. Prompt Fuzzing for Fuzz Driver Generation. InProceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security (CCS ’24). Association for Computing Machinery, New York, NY, USA, 3793–3807. doi:10.1145/3658644.3670396

    arXiv 2024

  33. [34]

    Kin-Keung Ma, Khoo Yit Phang, Jeffrey S Foster, and Michael Hicks. 2011. Directed symbolic execution. InInternational Static Analysis Symposium. Springer, 95–111

    2011

  34. [35]

    Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. 2023. Self-Refine: Iterative Refinement with Self-Feedback. arXiv:2303.17651 [cs.CL] https://arxiv.or...

    Pith/arXiv arXiv 2023

  35. [36]

    Paul Dan Marinescu and Cristian Cadar. 2013. KATCH: High-coverage testing of software patches. InProceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering. 235–245

    2013

  36. [37]

    Xiang Mei, Pulkit Singh Singaria, Jordi Del Castillo, Haoran Xi, Abdeloua- hab, Benchikh, Tiffany Bao, Ruoyu Wang, Yan Shoshitaishvili, Adam Doupé, Hammond Pearce, and Brendan Dolan-Gavitt. 2024. ARVO: Atlas of Repro- ducible Vulnerabilities for Open Source Software. doi:10.48550/arXiv.2408.02153 arXiv:2408.02153 [cs]

    Pith/arXiv arXiv doi:10.48550/arxiv.2408.02153 2024

  37. [38]

    Dongyu Meng, Michele Guerriero, Aravind Machiry, Hojjat Aghakhani, Priyanka Bose, Andrea Continella, Christopher Kruegel, and Giovanni Vigna. 2021. Bran: Reduce Vulnerability Search Space in Large Open Source Repositories by Learning Bug Symptoms. InProceedings of the 2021 ACM Asia Conference on Computer and Communications Security(Virtual Event, Hong Kon...

    doi:10.1145/3433210 2021

  38. [39]

    Ruijie Meng, Martin Mirchev, Marcel Böhme, and Abhik Roychoudhury. 2024. Large Language Model guided Protocol Fuzzing. InNetwork and Distributed System Security (NDSS) Symposium 2024. San Diego, CA, USA. https://www.ndss- symposium.org/ndss-paper/large-language-model-guided-protocol-fuzzing/

    2024

  39. [40]

    Daye Nam, Andrew Macvean, Vincent Hellendoorn, Bogdan Vasilescu, and Brad Myers. 2024. Using an LLM to Help With Code Understanding. InProceedings of the IEEE/ACM 46th International Conference on Software Engineering (ICSE ’24). Association for Computing Machinery, New York, NY, USA, 1–13. doi:10.1145/ 3597503.3639187

    arXiv 2024

  40. [41]

    nginx. [n. d.]. nginx is the world’s most popular Web Server. https://github.com/ nginx/nginx

  41. [42]

    Sebastian Österlund, Kaveh Razavi, Herbert Bos, and Cristiano Giuffrida. 2020. {ParmeSan}: Sanitizer-guided greybox fuzzing. InProceedings of the USENIX Security Symposium. 2289–2306

    2020

  42. [43]

    Sebastian Poeplau and Aurélien Francillon. 2020. Symbolic execution with {SymCC}: Don’t interpret, compile!. InProceedings of the USENIX Security Sym- posium. 181–198

    2020

  43. [44]

    Huanyao Rong, Wei You, XiaoFeng Wang, and Tianhao Mao. 2024. Toward Unbiased Multiple-Target Fuzzing with Path Diversity. 2475–2492. https://www. usenix.org/conference/usenixsecurity24/presentation/rong

    2024

  44. [45]

    Yulei Sui and Jingling Xue. 2016. SVF: interprocedural static value-flow analysis in LLVM. InProceedings of the 25th International Conference on Compiler Construction (CC ’16). Association for Computing Machinery, New York, NY, USA, 265–266. doi:10.1145/2892208.2892235

    arXiv 2016

  45. [46]

    Chenlin Wang, Wei Meng, Changhua Luo, and Penghui Li. 2025. Predator: Directed Web Application Fuzzing for Efficient Vulnerability Validation. In2025 IEEE Symposium on Security and Privacy (SP). 886–902. doi:10.1109/SP61157.2025. 00066 ISSN: 2375-1207

    arXiv 2025

  46. [47]

    Chenlin Wang, Wei Meng, Changhua Luo, and Penghui Li. 2025. Predator: Directed Web Application Fuzzing for Efficient Vulnerability Validation. InPro- ceedings of the IEEE Symposium on Security and Privacy (S&P). IEEE, 886–902

    2025

  47. [48]

    Dawei Wang, Geng Zhou, Li Chen, Dan Li, and Yukai Miao. 2024. ProphetFuzz: Fully Automated Prediction and Fuzzing of High-Risk Option Combinations with Only Documentation via Large Language Model. InProceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security (CCS ’24). Association for Computing Machinery, New York, NY, USA, 73...

    arXiv 2024

  48. [49]

    Junjie Wang, Bihuan Chen, Lei Wei, and Yang Liu. 2017. Skyfire: Data-Driven Seed Generation for Fuzzing. In2017 IEEE Symposium on Security and Privacy (SP). 579–594. doi:10.1109/SP.2017.23

    doi:10.1109/sp.2017.23 2017

  49. [50]

    Chunqiu Steven Xia, Matteo Paltenghi, Jia Le Tian, Michael Pradel, and Ling- ming Zhang. 2024. Fuzz4all: Universal fuzzing with large language models. In Proceedings of the International Conference on Software Engineering. 1–13

    2024

  50. [51]

    Yi Xiang, Xuhong Zhang, Peiyu Liu, Shouling Ji, Hong Liang, Jiacheng Xu, and Wenhai Wang. 2024. Critical code guided directed greybox fuzzing for commits. InProceedings of the USENIX Security Symposium. 2459–2474

    2024

  51. [52]

    Yijiang Xu, Hongrui Jia, Liguo Chen, Xin Wang, Zhengran Zeng, Yidong Wang, Qing Gao, Jindong Wang, Wei Ye, Shikun Zhang, et al. 2024. ISC4DGF: Enhancing Directed Grey-Box Fuzzing with LLM-Driven Initial Seed Corpus Generation. arXiv preprint arXiv:2409.14329(2024)

    Pith/arXiv arXiv 2024

  52. [53]

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. ReAct: Synergizing Reasoning and Acting in Language Models. arXiv:2210.03629 [cs.CL] https://arxiv.org/abs/2210.03629

    Pith/arXiv arXiv 2023

  53. [54]

    Jiaxi Ye, Ruilin Li, and Bin Zhang. 2020. RDFuzz: Accelerating directed fuzzing with intertwined schedule and optimized mutation.Mathematical Problems in Engineering2020, 1 (2020), 7698916

    2020

  54. [55]

    2018.{QSYM}: A practical concolic execution engine tailored for hybrid fuzzing

    Insu Yun, Sangho Lee, Meng Xu, Yeongjin Jang, and Taesoo Kim. 2018.{QSYM}: A practical concolic execution engine tailored for hybrid fuzzing. InProceedings of the USENIX Security Symposium. 745–761

    2018

  55. [56]

    Lee, Joshua Wang, Michael Pelican, David J

    Cen Zhang, Younggi Park, Fabian Fleischer, Yu-Fu Fu, Jiho Kim, Dongkwan Kim, Youngjoon Kim, Qingxiao Xu, Andrew Chin, Ze Sheng, Hanqing Zhao, Brian J. Lee, Joshua Wang, Michael Pelican, David J. Musliner, Jeff Huang, Jon Silliman, Mikel Mcdaniel, Jefferson Casavant, Isaac Goldthwaite, Nicholas Vi- dovich, Matthew Lehman, and Taesoo Kim. 2026. SoK: DARPA’s...

    Pith/arXiv arXiv doi:10.48550/arxiv.2602.07666 2026

  56. [57]

    Cen Zhang, Yaowen Zheng, Mingqiang Bai, Yeting Li, Wei Ma, Xiaofei Xie, Yuekang Li, Limin Sun, and Yang Liu. 2024. How Effective Are They? Exploring Large Language Model Based Fuzz Driver Generation. InProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2024). Association for Computing Machinery, New York, N...

    arXiv 2024

  57. [58]

    Kunpeng Zhang, Zongjie Li, Daoyuan Wu, Shuai Wang, and Xin Xia. 2025. Low- Cost and Comprehensive Non-textual Input Fuzzing with LLM-Synthesized Input Generators. 6999–7018. https://www.usenix.org/conference/usenixsecurity25/ presentation/zhang-kunpeng

    2025

  58. [59]

    Yujian Zhang, Yaokun Liu, Jinyu Xu, and Yanhao Wang. 2024. Predecessor-aware Directed Greybox Fuzzing. IEEE Computer Society, 1884–1900. doi:10.1109/ SP54263.2024.00040

    arXiv 2024

  59. [60]

    Han Zheng, Jiayuan Zhang, Yuhang Huang, Zezhong Ren, He Wang, Chunjie Cao, Yuqing Zhang, Flavio Toffalini, and Mathias Payer. 2023. {FISHFUZZ}: Catch deeper bugs by throwing larger nets. InProceedings of the USENIX Security Symposium. 1343–1360

    2023

  60. [61]

    Zhuotong Zhou, Yongzhuo Yang, Susheng Wu, Yiheng Huang, Bihuan Chen, and Xin Peng. 2024. Magneto: A Step-Wise Approach to Exploit Vulnerabilities in Dependent Libraries via LLM-Empowered Directed Fuzzing. InProceedings of the ACM/IEEE International Conference on Automated Software Engineering. 1633–1644

    2024

  61. [62]

    Jie Zhu, Chihao Shen, Ziyang Li, Jiahao Yu, Yizheng Chen, and Kexin Pei. 2025. Locus: Agentic Predicate Synthesis for Directed Fuzzing. doi:10.1145/3744916. 3773102 arXiv:2508.21302 [cs]

    Pith/arXiv arXiv 2025

  62. [63]

    2020.{FuzzGuard}: Filtering out unreachable inputs in directed grey-box fuzzing through deep learning

    Peiyuan Zong, Tao Lv, Dawei Wang, Zizhuang Deng, Ruigang Liang, and Kai Chen. 2020.{FuzzGuard}: Filtering out unreachable inputs in directed grey-box fuzzing through deep learning. InProceedings of the USENIX Security Symposium. 2255–2269. A Ethics Considerations In this work, we present SeedSmith, a tool that can be used for vulnerability discovery. As o...

    2020

  63. [67]

    Step-by-step explanations

  64. [68]

    Analysis of control-flow paths, relevant conditions, and seed generation strategies

  65. [69]

    + Employ generalized, yet precise patterns

    Clear instructions for reproducing the crash ## Essential Guidelines: Tool Usage: + Use the provided grep tool (grep -rnE <expression>) to thoroughly analyze the entire source code directory. + Employ generalized, yet precise patterns. For instance, use patterns like ->param instead of specific references like obj->param to ensure broader and more inclusi...

  66. [70]

    Sink function details: location, signature, purpose

  67. [71]

    Harnesses: file names, line numbers, entry points

  68. [72]

    Call trace: from harness to sink (if provided), or discovered via analysis

  69. [73]

    on the path

    Relevant conditions: if, switch, etc. on the path

  70. [74]

    The size of the seed MUST be under 2MB

    Seed generation strategy: Use libraries (base64, zlib, etc.) where applicable, and Python code (or pseudocode) for long/complex seeds. The size of the seed MUST be under 2MB

  71. [75]

    IF YOU HAVE TO SHOW SUCH EXAMPLES, PLEASE PROVIDE A SHORTER SUMMARY

    IMPORTANT: THIS GENERATED REPORT MUST NOT CONTAIN REPETITIVE PATTERNS TO DEMONSTRATE A SEED THAT COULD EXPLOIT THE VULNERABILITY. IF YOU HAVE TO SHOW SUCH EXAMPLES, PLEASE PROVIDE A SHORTER SUMMARY. After completing the analysis, you MUST output a report in the specified format. Your final report must be meticulously structured, explicit, and thorough to ...

  72. [383]

    doi:10.1109/ICSE48619.2023.00042 ISSN: 1558-1225

    arXiv 2023

  73. [2017]

    InProceedings of the ACM Conference on Computer and Communications Security (CCS)

    Directed greybox fuzzing. InProceedings of the ACM Conference on Computer and Communications Security (CCS). 2329–2344

  74. [2024]

    InProceedings of the IEEE Symposium on Security and Privacy (S&P)

    Titan: Efficient multi-target directed greybox fuzzing. InProceedings of the IEEE Symposium on Security and Privacy (S&P). IEEE, 1849–1864