Pith. sign in

REVIEW 3 major objections 6 minor 91 references

Reactive Bottom-Up Testing

T0 review · 3 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper introduces Reactive Bottom-Up Testing: fuzz a function in isolation, capture each crash as a symbolic constraint, then stitch that constraint backward through the call graph so only reachable, triggerable vulnerabilities are repor

desk verdict A credible reactive twist on bottom-up fuzzing with a real precision gap around global variables; worth refereeing, but the headline 100% precision claim needs fixing. read the letter →

arxiv 2509.03711 v1 pith:H2YFEUNW submitted 2025-09-03 cs.CR cs.SE

Siddharth Muralee , Sourag Cherupattamoolayil , James C. Davis , Antonio Bianchi , Aravind Machiry This is my paper · ORCID
classification cs.CRcs.SE
keywords reactivebottom-uptestingconstraintstitchingfunction-levelfuzzingsymbolicexecutioncrashreachabilityfalsepositivefilteringcallgraphbacktrackingvulnerabilitydetection
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Most fuzzing starts at the program's entry point, so vulnerabilities hidden deep in the call graph are hard to reach. The paper argues that testing a function in isolation finds bugs fast but is not enough: a crash in isolation may be unreachable from the real program's inputs. Its answer, Reactive Bottom-Up Testing, fuzzes a target function with an automatically generated harness, turns each crashing input into a symbolic constraint, and then walks backward up the call graph, stitching that constraint with each caller's path condition and argument values to see whether the crash is feasible from main. The prototype Griller reproduced 28 of 48 known vulnerabilities, found 6 previously unknown ones, and every completed backtrack to main was a confirmed true positive. The point is to spend reachability analysis only on states that are already known to trigger bugs, not on all feasible states up front.

What carries the argument

The load-bearing mechanism is constraint stitching. A crash in a target function is first represented as a crash constraint: the path conditions that led to it conjoined with a root-cause assertion such as 'this pointer is NULL' or 'this buffer is too small.' For every call edge above the target function, Griller records an edge constraint pair: the path condition needed to reach the callsite and the symbolic values of the arguments passed at that callsite. Stitching substitutes the callee's crash constraint into the caller's argument values and conjoins the caller's path condition; if the result is unsatisfiable, the crash is infeasible from that callsite and is dropped. The same stitched c

What would settle it

For any crashing input, log the sequence of taken branches or a coverage-trace hash under both the fuzzing harness and the pre-constrained symbolic run. If the two traces differ, the path-consistency premise is violated; if Griller still reports such a crash as feasible from main, the false-positive filter is unreliable for that case.

Watch

Extended reading notes

Core claim

Griller's central claim is that the way to make bottom-up testing practical is to be reactive: do not spend effort enumerating feasible program states before testing a function; instead, test the function with an automatically generated, context-aware harness, collect crashing inputs, and only then ask whether any crash is feasible in the whole program. Feasibility is decided by symbolic constraint stitching. Each crashing input is replayed under pre-constrained symbolic execution to produce a crash constraint: the path constraints that led to the crash conjoined with a root-cause assertion. Starting at the target function's caller, Griller conjoins that constraint with the path condition fo

Load-bearing premise

The load-bearing premise is that the crashing input forces the same execution path under native fuzzing and under pre-constrained symbolic execution; if the paths diverge, the stitched feasibility check can return a wrong satisfiability answer.

Editorial extensions

If this is right

  • A bottom-up tester no longer needs to generate feasible program states up front: fuzzing the target in isolation finds crashes quickly, and reachability is decided only for the small set of crash-triggering inputs.
  • Every crash whose constraint stitches successfully all the way to main is reported with a validated path to the entry point, so a developer sees both the bug and a concrete route to it.
  • The staged design stores symbolic state per call edge, so multiple crashes and multiple target functions can reuse the same backtracking information instead of re-exploring the program from scratch.
  • In the evaluation, complete backtracking to main achieved 100% precision, and Griller triggered 6 crashes that no baseline fuzzer triggered, making the approach complementary to top-down and directed fuzzing.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial: The stitching machinery only needs a crash constraint, not a particular crash type; injecting root-cause assertions for other observable properties such as assertion failures, leaks, or races would let the same reactive paradigm verify those properties too.
  • Editorial: The 100% precision figure depends on concrete and symbolic executions taking the same path; adding a runtime branch-trace comparison that fails loudly on divergence would convert a silent accuracy risk into an explicit diagnostic.
  • Editorial: With per-call-edge symbolic state persisted in the program database, incremental testing after a code change could re-analyze only the affected edges, which would make the approach more practical in continuous integration.
  • Editorial: The current implementation is source-based; a binary-level port would need equivalent type inference and instrumentation, but the conceptual claim about reactive feasibility checking does not depend on source access.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces Reactive Bottom-Up Testing (Reactive BOUT), an approach that tests individual functions in isolation and then validates discovered crashes against the surrounding program context. The proposed system, Griller, operates in three stages: (1) selecting target functions and generating type-aware harnesses; (2) fuzzing the harnesses and, for each crash, using pre-constrained KLEE execution to extract a crashing constraint; (3) backtracking along the program call graph and stitching the crash constraint with symbolic argument values and path conditions at each call edge, reporting only crashes whose stitched constraints are satisfiable up to main. The evaluation uses 48 confirmed CVEs across five projects, reporting 28 detected vulnerabilities with 6 false positives, and six previously unknown vulnerabilities in mblaze, pacman, and pspg, four of which were patched. The paper claims 100% precision for the 15 vulnerabilities whose backtracking reached main.

Significance. If the central precision claim holds, the reactive strategy is a meaningful improvement over proactive BOUT and whole-program fuzzing, because it decouples deep function analysis from the expensive reachability problem and filters only the bug-triggering states. The paper is honest about limitations: it reports false positives separately for partial backtracking, states that global values are not extracted (A.6), and acknowledges path divergence in KLEE (A.10.4) and concrete input generation difficulties (§5.4). The 28/48 detection result and the six zero-day disclosures (four patched) are concrete, falsifiable outputs. However, no machine-checked proof or released artifact is provided, and the load-bearing validation step—constraint stitching—has a gap concerning global variables that directly affects the claimed 100% precision for complete backtracking. The empirical comparison to AFGen is weakened by reliance on published numbers on a refined dataset. Overall, the contribution is potentially significant but requires a fix and a more guarded evaluation before the precision claim can be accepted.

major comments (3)
  1. [A.6 / §4.5 / §5.1] The claim in §5.1 that complete backtracking achieves 100% precision is not supported for crashes involving global variables. Edge constraints are defined as path condition plus symbolic argument values at the callsite (§4.5). A.6 explicitly says global variable values are not extracted because they are 'shared across the program and automatically included in the symbolic state.' This reasoning is incorrect under Griller's own execution model: the target driver and each caller driver are separately generated and separately executed, so the global variable in the target run is a different symbolic variable (or concrete value) from the global in the caller run. No equality constraint is generated at stitching. Consequently, a stitched constraint can leave a global-dependent crash condition as an unconstrained symbolic variable, and the solver may report satisfiable even though the actual c
  2. [§5.1 / A.8] The comparison to AFGen is not direct. Griller is not run against baselines; it relies on published AFGen results. The dataset is refined post hoc using criteria (active maintenance, verification, reproducibility) that remove targets from AFGen's original list. Target functions are manually supplied from vulnerability reports, so the target identification component is not tested end-to-end; RQ2's ranking result does not demonstrate the full pipeline. This weakens the claims of being 'comparable with AFGen' and 'outperforming AFL++ and Beacon.' Please either run the baselines on the same versions and inputs, or present the comparison only as a reference point and clearly label the dataset differences and manual target selection.
  3. [Challenge 2 / A.10.4] The false-positive filter assumes identical execution paths in native fuzzing and pre-constrained KLEE. The paper itself lists floating-point operations and pointer-arithmetic based control flow as causes of KLEE path divergence (A.10.4). If divergence occurs, path constraints are not the path of the crashing input, and stitching can return incorrect satisfiability. The paper does not quantify divergence for the 15 complete-backtracking cases. Please provide a per-case check (e.g., compare path constraints with the observed path, or count cases where seed patching was required) and discuss the impact on the 100% precision claim.
minor comments (6)
  1. [§5.1] The sentence beginning 'For each of the 20' is incomplete; it should refer to the 20 undetected CVEs and state the failure analysis criteria.
  2. [Table 3] The column header 'Ver / Prog' is ambiguous. Please clarify the version/program layout, e.g., with separate columns or an explicit explanation.
  3. [Listing 1] Multiple code listings contain stray 'ὁB' characters that appear to be rendering artifacts. These should be removed or replaced with the intended markers.
  4. [Algorithm 1] The loop indentation in Algorithm 1 obscures that the score is summed inside the loop. Please restructure the pseudocode to make the loop body explicit.
  5. [General] The paper does not state an artifact availability policy. Please include a link or explicitly state that the code will not be released.
  6. [A.3.3 / A.10.4] Typos: 'Leapord' should be 'LEOPARD', 'Constriant' should be 'Constraint', and 'experimetn' should be 'experiment'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Griller's constraint-stitching pipeline is self-contained and validated against external CVE ground truth.

full rationale

The paper's central derivation is an empirical pipeline: target functions/harnesses are generated, fuzzing finds crashes, KLEE replays those crashes to extract path/root-cause constraints, and the Stitcher combines caller edge constraints and checks satisfiability with Z3. Each stage consumes the previous stage's concrete artifacts rather than the evaluation labels. The crash constraints and edge constraints are defined operationally in Sec. 2.3 and Sec. 4.4/4.5, and the feasibility check is a genuine SMT satisfiability query over stitched constraints; it is not, by construction, equal to the CVE label. Evaluation against AFGen's 48-CVE benchmark is an external, empirically falsifiable comparison. Providing the list of vulnerable target functions by reading CVE reports is a benchmark convenience and does not force the tool's crashes, constraints, or satisfiability results. The threshold heuristics (0.5 priority threshold, null-byte value 13, default array size 128) are fixed engineering choices, not parameters fitted to the benchmark results. The self-citations to 3C and LEOPARD are used as implementation building blocks or ranking heuristics, not as a uniqueness theorem or as the proof of the central feasibility claim. The paper's own admitted limitations (KLEE path divergence from floating point/pointer arithmetic, global-value handling, partial-backtracking false positives) are soundness/completeness concerns rather than circular reductions. No step in the derivation reduces the reported predictions to the inputs by definition, so no circular step is identified.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central evaluation rests on assumptions about path consistency between concrete and symbolic execution, completeness of the generated harnesses, and the correctness of constraint stitching. The paper acknowledges the path-consistency limitation (Challenge 2) and reports KLEE divergence for floating-point and pointer arithmetic. The target identification heuristics are ad hoc and one citation (LEOPARD) is mis-referenced. These do not make the empirical claims circular, but they bound the confidence in the end-to-end results.

free parameters (6)
  • low_priority_threshold = 0.5
    Functions with vulnerability*fuzzability score below 0.5 are ranked LOW priority; threshold chosen by hand (Appendix A.3.3).
  • null_byte_threshold = 13
    Pointer null option byte: byte < 13 yields NULL (~5% probability). Chosen in Appendix A.4.
  • unknown_array_size_default = 128
    Arrays of unknown size are scored as size 128 in fuzzability (Appendix A.3.2, Algorithm 1).
  • pointer_complexity_multiplier = 2
    Pointer types double their element complexity score in Algorithm 1.
  • fuzzing_time_budget = 0.5 to 3 hours per function/call edge
    Per-target fuzzing time selected 'depending upon the complexity of the function' (Sec 5.1).
  • stitching_timeout = 1 minute
    Per-constraint stitching timeout (Sec 5.1).
assumptions (5)
  • domain assumption The same crashing input follows identical execution paths in native fuzzing and KLEE pre-constrained symbolic execution.
    Required for constraint extraction; stated as Challenge 2 in Sec 3.2; later admitted to fail for floating point and pointer arithmetic (Sec A.10.4).
  • domain assumption The harness-generated input space completely covers the target function's input space, including nested pointer types and opaque types.
    Needed for meaningful function-level testing; addressed in Challenge 1 (Sec 3.1) and Sec 4.2 with type inference, but opaque types require user-written harnesses (Sec A.4).
  • ad hoc to paper The vulnerability and fuzzability metrics correlate with actual vulnerability likelihood and ease of fuzzing.
    Target selection relies on these heuristics (Sec 4.1); the LEOPARD citation is incorrect (ref [42] is the Checked C repo).
  • domain assumption Indirect call targets are correctly resolved by address-taken and signature matching.
    Sec 4.1.4; needed for call graph completeness.
  • standard math Z3 solves the stitched constraints correctly and the unsat-cores are accurate.
    The satisfiability checks rely on Z3; the paper validates 100 sampled unsat cases (Sec 5.2.4).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Reactive Bottom-Up Testing." pith.science (2026). https://pith.science/paper/H2YFEUNW

@misc{pith2026250903711,
  author       = {Pith},
  title        = {Pith review of: Reactive Bottom-Up Testing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/H2YFEUNW}},
  note         = {Machine review of arXiv:2509.03711}
}
read the original abstract

Modern computing systems remain rife with software vulnerabilities. Engineers apply many means to detect them, of which dynamic testing is one of the most common and effective. However, most dynamic testing techniques follow a top-down paradigm, and struggle to reach and exercise functions deep within the call graph. While recent works have proposed Bottom-Up approaches to address these limitations, they face challenges with false positives and generating valid inputs that adhere to the context of the entire program. In this work, we introduce a new paradigm that we call Reactive Bottom-Up Testing. Our insight is that function-level testing is necessary but not sufficient for the validation of vulnerabilities in functions. What we need is a systematic approach that not only tests functions in isolation but also validates their behavior within the broader program context, ensuring that detected vulnerabilities are both reachable and triggerable. We develop a three-stage bottom-up testing scheme: (1) identify likely-vulnerable functions and generate type- and context-aware harnesses; (2) fuzz to find crashes and extract input constraints via symbolic execution; (3) verify crashes by combining constraints to remove false positives. We implemented an automated prototype, which we call Griller. We evaluated Griller in a controlled setting using a benchmark of 48 known vulnerabilities across 5 open-source projects, where we successfully detected 28 known vulnerabilities. Additionally, we evaluated Griller on several real-world applications such as Pacman, and it discovered 6 previously unknown vulnerabilities. Our findings suggest that Reactive Bottom-Up Testing can significantly enhance the detection of vulnerabilities in complex systems, paving the way for more robust security practices.

Figures

Figures reproduced from arXiv: 2509.03711 by the authors.

Figure 1
Figure 1. Difference between TDT and BOUT testing ap￾proaches. files, sockets, etc. The PUT sets up the states along the exe￾cution flow needed for subsequent functions. The PUT might constrain the state space at arbitrary program points (e.g., through if statements), i.e., consider certain states as invalid and terminate execution (as illustrated by [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Design Overview of Griller metric proposed by LEOPARD [42] as the baseline for our heuristics. 4.1.2 Fuzzability Metric The Fuzzability metric is meant to assess the ease of fuzzing a function. Functions with complex/larger parameter types can be relatively harder to fuzz, since the input space is larger and decreases fuzzing efficiency. We compute a Type Score for each function that estimates the complexity of gene… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

91 extracted references · 78 canonical work pages

  1. [1]

    https://github.com/ rockcarry/ffjpeg

    A simple jpeg codec. https://github.com/ rockcarry/ffjpeg

  2. [2]

    Introduction to software testing

    Paul Ammann and Jeff Offutt. Introduction to software testing. Cambridge University Press, 2016

    2016

  3. [3]

    Restler: Stateful rest api fuzzing

    Vaggelis Atlidakis, Patrice Godefroid, and Marina Pol- ishchuk. Restler: Stateful rest api fuzzing. In 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE), pages 748–758. IEEE, 2019

    2019

  4. [4]

    The smt-lib standard: Version 2.0

    Clark Barrett, Aaron Stump, Cesare Tinelli, et al. The smt-lib standard: Version 2.0. In Proceedings of the 8th international workshop on satisfiability modulo theories (Edinburgh, UK), volume 13, page 14, 2010

    2010

  5. [5]

    Fuzzing: Challenges and reflections

    Marcel Böhme, Cristian Cadar, and Abhik Roychoud- hury. Fuzzing: Challenges and reflections. IEEE Soft- ware, 38(3):79–86, 2020

    2020

  6. [6]

    Directed greybox fuzzing

    Marcel Böhme, Van-Thuan Pham, Manh-Dung Nguyen, and Abhik Roychoudhury. Directed greybox fuzzing. In Proceedings of the 2017 ACM SIGSAC conference on computer and communications security, pages 2329– 2344, 2017

    2017

  7. [7]

    Klee: unassisted and automatic generation of high- coverage tests for complex systems programs

    Cristian Cadar, Daniel Dunbar, and Dawson Engler. Klee: unassisted and automatic generation of high- coverage tests for complex systems programs. In Pro- ceedings of the 8th USENIX Conference on Operating Systems Design and Implementation , OSDI’08, page 209–224, USA, 2008. USENIX Association

    2008

  8. [8]

    Binary analysis and symbolic execution with angr

    Eric Cheng. Binary analysis and symbolic execution with angr. PhD thesis, PhD thesis, 2016

    2016

Show all 91 references
  1. [9]

    A guide to stakeholder analysis for cybersecurity researchers, 2025

    James C Davis, Sophie Chen, Huiyun Peng, Paschal C Amusuo, and Kelechi G Kalu. A guide to stakeholder analysis for cybersecurity researchers, 2025

    2025

  2. [10]

    Mock objects in software testing: An analysis of usage in open-source projects

    Raphael De Almeida, Romulo Martins Da Silva, Lu- cas Silveira Serrano, Heleno De Souza Campos Junior, and Vânia de Oliveira Neves. Mock objects in software testing: An analysis of usage in open-source projects. In Proceedings of the XXII Brazilian Symposium on Software Quality...

    2023

  3. [11]

    Z3: An ef- ficient smt solver

    Leonardo de Moura and Nikolaj Bjørner. Z3: An ef- ficient smt solver. In C. R. Ramakrishnan and Jakob Rehof, editors, Tools and Algorithms for the Construc- tion and Analysis of Systems , pages 337–340, Berlin, Heidelberg, 2008. Springer Berlin Heidelberg

    2008

  4. [12]

    Package manager: The core of a gnu/linux distribution

    Andrey Falko. Package manager: The core of a gnu/linux distribution. Simon’s Rock College, 2007

    2007

  5. [13]

    AFL++: Combining incremental steps of fuzzing research

    Andrea Fioraldi, Dominik Maier, Heiko Eißfeldt, and Marc Heuse. AFL++: Combining incremental steps of fuzzing research. In 14th USENIX Workshop on Offen- sive Technologies (WOOT 20). USENIX Association, August 2020

    2020

  6. [14]

    Be- yond the coverage plateau: A comprehensive study of fuzz blockers (registered report)

    Wentao Gao, Van-Thuan Pham, Dongge Liu, Oliver Chang, Toby Murray, and Benjamin IP Rubinstein. Be- yond the coverage plateau: A comprehensive study of fuzz blockers (registered report). In Proceedings of the 2nd International Fuzzing Workshop, pages 47–55, 2023

    2023

  7. [15]

    Intelligent rest api data fuzzing

    Patrice Godefroid, Bo-Yuan Huang, and Marina Pol- ishchuk. Intelligent rest api data fuzzing. InProceedings of the 28th ACM joint meeting on European software en- gineering conference and symposium on the foundations of software engineering, pages 725–736, 2020. 15

    2020

  8. [16]

    Levin, and David Molnar

    Patrice Godefroid, Michael Y . Levin, and David Molnar. Sage: whitebox fuzzing for security testing. Commun. ACM, 55(3):40–44, March 2012

    2012

  9. [17]

    Autofuzz: Automated network protocol fuzzing framework

    Serge Gorbunov and Arnold Rosenbloom. Autofuzz: Automated network protocol fuzzing framework. Ijcsns, 10(8):239, 2010

    2010

  10. [18]

    Deriving semantics-aware fuzzers from web api schemas

    Zac Hatfield-Dodds and Dmitry Dygalo. Deriving semantics-aware fuzzers from web api schemas. In Proceedings of the ACM/IEEE 44th International Con- ference on Software Engineering: Companion Proceed- ings, pages 345–346, 2022

    2022

  11. [19]

    Seed selection for successful fuzzing

    Adrian Herrera, Hendra Gunadi, Shane Magrath, Michael Norrish, Mathias Payer, and Antony L Hosking. Seed selection for successful fuzzing. In Proceedings of the 30th ACM SIGSOFT international symposium on software testing and analysis, pages 230–243, 2021

    2021

  12. [20]

    Beacon: Directed grey-box fuzzing with provable path pruning

    Heqing Huang, Yiyuan Guo, Qingkai Shi, Peisen Yao, Rongxin Wu, and Charles Zhang. Beacon: Directed grey-box fuzzing with provable path pruning. In 2022 IEEE Symposium on Security and Privacy (SP), pages 36–50, 2022

    2022

  13. [21]

    ISO/IEC/IEEE 24765:2010 - Systems and soft- ware engineering – V ocabulary

    International Organization for Standardization. ISO/IEC/IEEE 24765:2010 - Systems and soft- ware engineering – V ocabulary. Standard 24765, ISO/IEC/IEEE, Dec 2010

    2010

  14. [22]

    Unit testing principles, practices, and patterns

    Vladimir Khorikov. Unit testing principles, practices, and patterns. Simon and Schuster, 2020

    2020

  15. [23]

    Drivefuzz: Discovering autonomous driving bugs through driving quality-guided fuzzing

    Seulbae Kim, Major Liu, Junghwan" John" Rhee, Yuseok Jeon, Yonghwi Kwon, and Chung Hwan Kim. Drivefuzz: Discovering autonomous driving bugs through driving quality-guided fuzzing. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security, pages 1...

    2022

  16. [24]

    appneta/tcpreplay, Feb 2024

    Fred Klassen. appneta/tcpreplay, Feb 2024

    2024

  17. [25]

    Lattner and V

    C. Lattner and V . Adve. Llvm: a compilation frame- work for lifelong program analysis & transformation. In International Symposium on Code Generation and Optimization, 2004. CGO 2004., pages 75–86, 2004

    2004

  18. [26]

    Github - leahneukirchen/mblaze: Unix utilities to deal with maildir, Jul 2017

    leahneukirchen. Github - leahneukirchen/mblaze: Unix utilities to deal with maildir, Jul 2017

    2017

  19. [27]

    Fairfuzz: a tar- geted mutation strategy for increasing greybox fuzz test- ing coverage

    Caroline Lemieux and Koushik Sen. Fairfuzz: a tar- geted mutation strategy for increasing greybox fuzz test- ing coverage. In Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engi- neering, ASE ’18, page 475–485, New York, NY , USA,

  20. [28]

    Redefin- ing indirect call analysis with kallgraph

    Guoren Li, Manu Sridharan, and Zhiyun Qian. Redefin- ing indirect call analysis with kallgraph. In 2025 IEEE Symposium on Security and Privacy (SP), pages 2957–

    2025

  21. [29]

    Github - libming/libming: Swf output library, 2020

    libming. Github - libming/libming: Swf output library, 2020

    2020

  22. [30]

    Improving Indirect-Call analysis in LLVM with type and Data-Flow Co-Analysis

    Dinghao Liu, Shouling Ji, Kangjie Lu, and Qinming He. Improving Indirect-Call analysis in LLVM with type and Data-Flow Co-Analysis. In 33rd USENIX Security Symposium (USENIX Security 24), pages 5895–5912, Philadelphia, PA, August 2024. USENIX Association

    2024

  23. [31]

    Afgen: Whole-function fuzzing for appli- cations and libraries

    Yuwei Liu, Yanhao Wang, Xiangkun Jia, Zheng Zhang, and Purui Su. Afgen: Whole-function fuzzing for appli- cations and libraries. In 2024 IEEE Symposium on Se- curity and Privacy (SP), pages 1901–1919. IEEE, 2024

    2024

  24. [32]

    Afgen: Whole-function fuzzing for ap- plications and libraries

    Yuwei Liu, Yanhao Wang, Xiangkun Jia, Zheng Zhang, and Purui Su. Afgen: Whole-function fuzzing for ap- plications and libraries. In 2024 IEEE Symposium on Security and Privacy (SP), pages 1901–1919, 2024

    2024

  25. [33]

    Where Does It Go? Refin- ing Indirect-Call Targets with Multi-Layer Type Anal- ysis

    Kangjie Lu and Hong Hu. Where Does It Go? Refin- ing Indirect-Call Targets with Multi-Layer Type Anal- ysis. In Proceedings of the 26th ACM Conference on Computer and Communications Security (CCS) , Lon- don, UK, 2019

    2019

  26. [34]

    Smartseed: Smart seed generation for efficient fuzzing

    Chenyang Lyu, Shouling Ji, Yuwei Li, Junfeng Zhou, Jianhai Chen, and Jing Chen. Smartseed: Smart seed generation for efficient fuzzing. arXiv preprint arXiv:1807.02606, 2018

    2018 arXiv

  27. [35]

    Directed symbolic execution

    Kin-Keung Ma, Khoo Yit Phang, Jeffrey S Foster, and Michael Hicks. Directed symbolic execution. In Static Analysis: 18th International Symposium, SAS 2011, Venice, Italy, September 14-16, 2011. Proceed- ings 18, pages 95–111. Springer, 2011

    2011

  28. [36]

    C to checked c by 3c

    Aravind Machiry, John Kastner, Matt McCutchen, Aaron Eline, Kyle Headley, and Michael Hicks. C to checked c by 3c. Proc. ACM Program. Lang., 6(OOP- SLA1), April 2022

    2022

  29. [37]

    Hybrid concolic testing

    Rupak Majumdar and Koushik Sen. Hybrid concolic testing. In 29th International Conference on Software Engineering (ICSE’07), pages 416–426. IEEE, 2007

    2007

  30. [38]

    Demystify the fuzzing methods: A comprehensive survey

    Sanoop Mallissery and Yu-Sung Wu. Demystify the fuzzing methods: A comprehensive survey. ACM Com- puting Surveys, 56(3):1–38, 2023

    2023

  31. [39]

    The art, science, and engineering of fuzzing: A survey

    Valentin Jean Marie Manès, HyungSeok Han, Choong- woo Han, Sang Kil Cha, Manuel Egele, Edward J Schwartz, and Maverick Woo. The art, science, and engineering of fuzzing: A survey. IEEE Transactions on Software Engineering, 2019. 16

    2019

  32. [40]

    Github - marc-q/libwav: A simple wav library written in pure c., 2016

    marc q. Github - marc-q/libwav: A simple wav library written in pure c., 2016

    2016

  33. [41]

    Algorithms for maximum satisfiability using unsatisfiable cores

    Joao Marques-Silva and Jordi Planes. Algorithms for maximum satisfiability using unsatisfiable cores. In Proceedings of the conference on Design, automation and test in Europe, pages 408–413, 2008

    2008

  34. [42]

    Checked c clang repository

    Microsoft. Checked c clang repository. GitHub reposi- tory

  35. [43]

    Github - miniupnp/ngiflib: Gif picture format decoding library written in c

    Miniupnp. Github - miniupnp/ngiflib: Gif picture format decoding library written in c

  36. [44]

    Vulnerability-oriented fuzz testing for con- nected autonomous vehicle systems

    Lama J Moukahal, Mohammad Zulkernine, and Martin Soukup. Vulnerability-oriented fuzz testing for con- nected autonomous vehicle systems. IEEE transactions on reliability, 70(4):1422–1437, 2021

    2021

  37. [45]

    John Wiley & Sons, 2011

    Glenford J Myers, Corey Sandler, and Tom Badgett.The art of software testing. John Wiley & Sons, 2011

    2011

  38. [46]

    Stateafl: Greybox fuzzing for state- ful network servers

    Roberto Natella. Stateafl: Greybox fuzzing for state- ful network servers. Empirical Software Engineering, 27(7):191, 2022

    2022

  39. [47]

    Ccured: Type- safe retrofitting of legacy software

    George C Necula, Jeremy Condit, Matthew Harren, Scott McPeak, and Westley Weimer. Ccured: Type- safe retrofitting of legacy software. ACM Transactions on Programming Languages and Systems (TOPLAS) , 27(3):477–526, 2005

    2005

  40. [48]

    The hu- man side of fuzzing: Challenges faced by developers during fuzzing activities

    Olivier Nourry, Yutaro Kashiwa, Bin Lin, Gabriele Bavota, Michele Lanza, and Yasutaka Kamei. The hu- man side of fuzzing: Challenges faced by developers during fuzzing activities. ACM transactions on software engineering and methodology, 33(1):1–26, 2023

    2023

  41. [49]

    Combining static analysis and targeted symbolic execution for scalable bug-finding in application binaries

    Muhammad Riyad Parvez. Combining static analysis and targeted symbolic execution for scalable bug-finding in application binaries. Master’s thesis, University of Waterloo, 2016

    2016

  42. [50]

    Locksmith: Practical static race detection for c

    Polyvios Pratikakis, Jeffrey S Foster, and Michael Hicks. Locksmith: Practical static race detection for c. ACM Transactions on Programming Languages and Systems (TOPLAS), 33(1):1–55, 2011

    2011

  43. [51]

    Nsfuzz: Towards efficient and state-aware network service fuzzing

    Shisong Qin, Fan Hu, Zheyu Ma, Bodong Zhao, Tingting Yin, and Chao Zhang. Nsfuzz: Towards efficient and state-aware network service fuzzing. ACM Transactions on Software Engineering and Methodology, 32(6):1–26, 2023

    2023

  44. [52]

    Optimizing seed selection for fuzzing

    Alexandre Rebert, Sang Kil Cha, Thanassis Avgerinos, Jonathan Foote, David Warren, Gustavo Grieco, and David Brumley. Optimizing seed selection for fuzzing. In 23rd USENIX Security Symposium (USENIX Security 14), pages 861–875, 2014

    2014

  45. [53]

    Nyx-net: network fuzzing with incremental snapshots

    Sergej Schumilo, Cornelius Aschermann, Andrea Jem- mett, Ali Abbasi, and Thorsten Holz. Nyx-net: network fuzzing with incremental snapshots. In Proceedings of the Seventeenth European Conference on Computer Systems, pages 166–180, 2022

    2022

  46. [54]

    Static program analysis as a fuzzing aid

    Bhargava Shastry, Markus Leutner, Tobias Fiebig, Kashyap Thimmaraju, Fabian Yamaguchi, Konrad Rieck, Stefan Schmid, Jean-Pierre Seifert, and Anja Feldmann. Static program analysis as a fuzzing aid. In Research in Attacks, Intrusions, and Defenses: 20th International Symposium,...

    2017

  47. [55]

    Neuzz: Efficient fuzzing with neural program smoothing

    Dongdong She, Kexin Pei, Dave Epstein, Junfeng Yang, Baishakhi Ray, and Suman Jana. Neuzz: Efficient fuzzing with neural program smoothing. In 2019 IEEE Symposium on Security and Privacy (SP) , pages 803–

    2019

  48. [56]

    No harness, no prob- lem: Oracle-guided harnessing for auto-generating c api fuzzing harnesses

    Gabriel Sherman and Stefan Nagy. No harness, no prob- lem: Oracle-guided harnessing for auto-generating c api fuzzing harnesses. In 2025 IEEE/ACM 47th Inter- national Conference on Software Engineering (ICSE), pages 775–775. IEEE Computer Society, 2025

    2025

  49. [57]

    To mock or not to mock? an empirical study on mocking practices

    Davide Spadini, Maurício Aniche, Magiel Bruntink, and Alberto Bacchelli. To mock or not to mock? an empirical study on mocking practices. In 2017 IEEE/ACM 14th International Conference on Mining Software Reposito- ries (MSR), pages 402–412. IEEE, 2017

    2017

  50. [58]

    Mock objects for testing java systems: Why and how developers use them, and how they evolve

    Davide Spadini, Maurício Aniche, Magiel Bruntink, and Alberto Bacchelli. Mock objects for testing java systems: Why and how developers use them, and how they evolve. Empirical Software Engineering, 24:1461–1498, 2019

    2019

  51. [59]

    One fuzz doesn’t fit all: Optimizing directed fuzzing via target-tailored program state restriction

    Prashast Srivastava, Stefan Nagy, Matthew Hicks, Anto- nio Bianchi, and Mathias Payer. One fuzz doesn’t fit all: Optimizing directed fuzzing via target-tailored program state restriction. In Proceedings of the 38th Annual Com- puter Security Applications Conference , ACSAC ’22...

    2022

  52. [60]

    Driller: Augmenting fuzzing through selective symbolic execution

    Nick Stephens, John Grosen, Christopher Salls, Andrew Dutcher, Ruoyu Wang, Jacopo Corbetta, Yan Shoshi- taishvili, Christopher Kruegel, and Giovanni Vigna. Driller: Augmenting fuzzing through selective symbolic execution. In NDSS, volume 16, pages 1–16, 2016

    2016

  53. [61]

    Klee symbolic virtual machine, 2015

    KLEE Team. Klee symbolic virtual machine, 2015. https://github.com/klee/klee

    2015

  54. [62]

    Arbiter: Bridging the static and dynamic divide in vulnerability discovery on binary programs

    Jayakrishna Vadayath, Moritz Eckert, Kyle Zeng, Nico- laas Weideman, Gokulkrishna Praveen Menon, Yanick 17 Fratantonio, Davide Balzarotti, Adam Doupé, Tiffany Bao, Ruoyu Wang, Christophe Hauser, and Yan Shoshi- taishvili. Arbiter: Bridging the static and dynamic divide in vuln...

    2022

  55. [63]

    The progress, challenges, and per- spectives of directed greybox fuzzing

    Pengfei Wang, Xu Zhou, Tai Yue, Peihong Lin, Yingy- ing Liu, and Kai Lu. The progress, challenges, and per- spectives of directed greybox fuzzing. Software Testing, Verification and Reliability, 34(2):e1869, 2024

    2024

  56. [64]

    Not all cov- erage measurements are equal: Fuzzing by coverage accounting for input prioritization

    Yanhao Wang, Xiangkun Jia, Yuwei Liu, Kyle Zeng, Tiffany Bao, Dinghao Wu, and Purui Su. Not all cov- erage measurements are equal: Fuzzing by coverage accounting for input prioritization. In NDSS, 2020

    2020

  57. [65]

    Neufuzz: Efficient fuzzing with deep neural network

    Yunchao Wang, Zehui Wu, Qiang Wei, and Qingxian Wang. Neufuzz: Efficient fuzzing with deep neural network. IEEE Access, 7:36340–36352, 2019

    2019

  58. [66]

    Directed incremental symbolic execu- tion

    Guowei Yang, Suzette Person, Neha Rungta, and Sar- fraz Khurshid. Directed incremental symbolic execu- tion. ACM Transactions on Software Engineering and Methodology (TOSEM), 24(1):1–42, 2014

    2014

  59. [67]

    QSYM : A practical concolic execution en- gine tailored for hybrid fuzzing

    Insu Yun, Sangho Lee, Meng Xu, Yeongjin Jang, and Taesoo Kim. QSYM : A practical concolic execution en- gine tailored for hybrid fuzzing. In 27th USENIX Secu- rity Symposium (USENIX Security 18), pages 745–761, Baltimore, MD, August 2018. USENIX Association

    2018

  60. [68]

    Neural network guided evolutionary fuzzing for finding traffic violations of autonomous vehicles

    Ziyuan Zhong, Gail Kaiser, and Baishakhi Ray. Neural network guided evolutionary fuzzing for finding traffic violations of autonomous vehicles. IEEE Transactions on Software Engineering, 49(4):1860–1875, 2022. A Appendix A.1 Outline of Appendices The appendix contains the foll...

    2022

  61. [70]

    Number of parameter variables

  62. [71]

    Number of variables as parameters for callee function

  63. [72]

    Number of pointer arithmetic

  64. [73]

    Number of variables involved in pointer arithmetic

  65. [74]

    Maximum pointer arithmetic a variable is involved In

  66. [75]

    Number of Nested Control Structures

  67. [76]

    Maximum nesting level of control structures

  68. [77]

    Maximum of control-dependent control structures

  69. [78]

    Maximum of data-dependent control structures

  70. [79]

    Number of if structures without else

  71. [80]

    /seeds/test.pcap

    Number of variables involved in control predicates We calculate each score and then normalize each individual metric across the entire target. We then calculate the sum of all the normalized scores. A.3.2 Fuzzability Metrics Algorithm 1 shows the pseudocode of our Type Scoring...

  72. [81]

    For example, the libwav [40] project, was last updated in 2017, and all the CVEs reported were not verified, ac- knowledged or fixed by the developers

    Active maintenance: We excluded targets that have not had active commits in the last five years, ensuring our evaluation focuses on currently relevant software. For example, the libwav [40] project, was last updated in 2017, and all the CVEs reported were not verified, ac- kno...

    2017

  73. [82]

    Vulnerability verification: We only included vulnera- bilities that have been verified and fixed, enhancing the credibility of our results

  74. [83]

    invalid input params ! \n

    Reproducibility: We ensured that all included vulnera- bilities are reproducible, facilitating future research and validation of our findings. Side Effect Bug During our analysis of ngiflib [43], we ob- served a discrepancy between the execution paths taken dur- ing concrete (...

  75. [84]

    Complex input format: similar to the example dis- cussed in the previous section, the fuzzer requires a spe- cific input format

  76. [85]

    Unmodeled Environment: functions from third-party libraries such as libpcap, which check for availability of network devices prohibit fuzzer from proceeding further

  77. [86]

    This is mainly due to the fact that our strict runtime santization does’t allow the fuzzer to proceed from these crashes

    Crashes: In certain cases, there are crashes in the parent function that prevent it from reaching the child function. This is mainly due to the fact that our strict runtime santization does’t allow the fuzzer to proceed from these crashes. In some of the cases we patched these...

  78. [87]

    Complex Input Formats: As demonstrated in the pre- vious section, functions requiring strictly structured in- put formats (e.g., valid image files, network protocols) present significant challenges for constraint-based input generation

  79. [88]

    If these are unmodeled, execution terminates before reaching target functions

    Unmodeled Environmental Dependencies: In certain cases, functions utilizing third-party libraries (e.g., libp- cap) often include runtime checks for system resources such as network device availability or file system per- missions. If these are unmodeled, execution terminates ...

  80. [89]

    A.10.4 Constraint Generation Failures There were no Constriant Generation failures that affected all the inputs in an call-edge or a target functions

    Intermediate Crashes: Parent functions containing un- related vulnerabilities prevent execution from reaching child functions due to our strict runtime sanitization poli- cies. A.10.4 Constraint Generation Failures There were no Constriant Generation failures that affected all...

  81. [90]

    } if (strcmp(argv[1], "-d") == 0) { jfif = jfif_load(argv[2]); jfif_decode(jfif, &bmp); ○

    KLEE memory errors: KLEE may encounter memory errors (due to invalid access) during execution, this is due to limitation of ASan, which cannot track off-by-one 23 int main(int argc, char *argv[]) { if (argc < 3) { ... } if (strcmp(argv[1], "-d") == 0) { jfif = jfif_load(argv[2...

  82. [91]

    Path Disparity: While GrillSan is able to handle most of the cases where the path diverges, there are cases where the path diverges and KLEE is unable to reach the callsite of the callee function. The two most common reasons that we observed for this are: (1) Floating point op...

  83. [2018]

    Association for Computing Machinery

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.