Pith. sign in

REVIEW 4 major objections 4 minor 66 references

Directed Grammar-Based Test Generation

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that grammar-based test generation can be steered to a chosen goal by learning production probabilities from test feedback, and reports that this steering outperforms all five baselines in 86% of settings.

desk verdict A plausible goal-directed grammar fuzzer with a strong headline number, but the unreadable full text makes the central claim unverifiable from my copy; deserves a referee who can see the real paper and per-goal breakdown. read the letter →

arxiv 2508.01472 v1 pith:IGOQYR5K submitted 2025-08-02 cs.SE

Lukas Kirschner (Saarland University , University of Luxembourg) , Ezekiel Soremekun (Singapore University of Technology , Design) This is my paper · ORCID
classification cs.SE
keywords grammar-basedtestgenerationfeedback-directedfuzzingprobabilisticgrammargoal-specificinputscodecoverageguidancefailureinductionevolutionarysearchsoftwaretesting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that grammar-based test generation can be made goal-directed without hand-tuning by repeatedly learning which input shapes help achieve a chosen testing goal. Its method, FdLoop, takes a probabilistic grammar, generates candidates, executes them, and re-weights the grammar's production rules from feedback so later inputs land closer to the goal. The authors instantiate this for four goals—unique coverage, input-to-code complexity, program exceptions, and long execution time—and evaluate it on three input formats and 20 open-source programs. In 86% of settings FdLoop beats all five baselines and is up to twice as effective as the strongest prior grammar-based method at provoking erroneous behavior. If that holds, testers can push fuzzers toward specific objectives by stating the goal, not by writing grammar mutations by hand.

What carries the argument

The mechanism that carries the argument is a probabilistic context-free grammar whose expansion probabilities are updated from execution feedback. Given a testing goal, FdLoop maintains a corpus of inputs; an input mutator varies existing seeds, a grammar mutator re-weights production rules, and test feedback scores the generated inputs against the goal. The grammar mutator is the central control knob: by shifting probability mass toward syntactic structures that produced goal-relevant behavior, the sampler becomes a directed search rather than a random enumeration of valid inputs. The feedback components define how the signal from each executed input is converted into the weight update.

What would settle it

Run FdLoop and its five baselines on a held-out set of parsers and interpreters for JSON, CSS, and JavaScript, counting unique exceptions and sanitizer-detected crashes per CPU-hour instead of the guided metrics; if the roughly 2x failure-induction advantage over the best baseline does not reproduce, the central effectiveness claim is falsified.

Watch

Extended reading notes

Core claim

The authors propose FdLoop, a generator that turns the selection of goal-specific test inputs into an iterative learning problem over a probabilistic grammar. Starting from a grammar and a stated goal, FdLoop generates inputs, runs them on the target program, scores each input by goal-specific feedback (unique coverage, input-to-code complexity, exceptions, or execution time), and re-weights the grammar's production probabilities so that future generation emphasizes the structures that scored well. The paper's central claim is that this feedback loop makes grammar-based testing goal-directed without hand-tuning: in 86% of the experimental settings it exceeds all five baselines, and it is up to twice as effective as the best baseline at inducing erroneous behavior. The authors also report ablation results in which the input mutator, grammar mutator, and feedback components each contribute positively, and they show the approach handles single goals and combinations of goals across varied parameter settings.

Load-bearing premise

The finding rests on the assumption that the feedback signals used to guide FdLoop are faithful proxies for the testing goals that matter, and that the 20 programs and three input formats are representative enough to generalize.

Editorial extensions

If this is right

  • Grammar-based fuzzers can be pointed at a stated objective—coverage, crashes, complexity, or runtime—without rewriting the grammar.
  • If the 86% win rate holds, FdLoop-style feedback loops become a default choice for structured-format testing where a grammar is already available.
  • The 2x failure-induction result implies that spending the same test budget yields more exception-triggering inputs when generation is feedback-guided.
  • The ablation result implies that each of the three components earns its place; removing the mutator or feedback degrades goal-directedness.
  • Multiple goals can be pursued in one run by combining feedback signals, so testers do not need a separate campaign per objective.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: the loop should transfer to other generative representations, such as abstract syntax trees or byte-level mutation, because the load-bearing part is the score-then-reweight cycle, not the grammar formalism itself.
  • Inference: the aggregate 86% figure would gain force from per-goal and per-program breakdowns with confidence intervals; an independent rerun on a held-out program set with a crash oracle is the natural test of the failure-induction advantage.
  • Inference: the same guidance mechanism could be pointed at security goals like sanitizer-detected memory errors, provided a cheap feedback signal exists; designing that signal is likely the main barrier.
  • Inference: because the guiding signals double as scoring signals, an external evaluation using different metrics than the ones guiding the search would show whether the method generalizes or overfits to its feedback functions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes FdLoop, a feedback-driven grammar-based test generator that iteratively selects, evolves, and learns the input distribution of goal-specific test inputs from test feedback and a probabilistic grammar. It concretizes the approach for four testing goals (unique code coverage, input-to-code complexity, program failures/exceptions, and long execution time) and reports an evaluation over three input formats (JSON, CSS, JavaScript), 20 open-source programs, and five baselines (random, probabilistic and inverse-probabilistic grammar-based generators, EvoGFuzz, and DynaMosa). The abstract's central claims are an 86% win rate against all five baselines and up to 2x effectiveness of EvoGFuzz at inducing erroneous behaviors, plus an ablation showing positive contributions from the main components.

Significance. If the empirical claims hold, FdLoop would address a real limitation of grammar-based test generation: random generation has low probability of achieving arbitrary, goal-specific testing objectives. The evaluation design described in the abstract is sensible in principle, using external baselines, real-world subject programs, multiple input grammars, and four distinct goals. The reported 86% aggregate win rate and 2x improvement over EvoGFuzz on failure induction would be practically meaningful for software testing. However, the full text supplied for review is unreadable mojibake, so none of the method, evaluation protocol, tables, or statistics can be verified. The strength of the claim is therefore entirely unassessed at this point; the paper needs a legible version and disaggregated evidence.

major comments (4)
  1. [Full Text] The entire body of the provided manuscript is unreadable due to mojibake corruption. No algorithm description, evaluation setup, result table, ablation result, or statistical test can be inspected. Because every load-bearing claim depends on this content, the paper cannot currently be evaluated for soundness. Please provide a clean, legible version of the full text.
  2. [Abstract] The headline result, "In most (86%) settings, FdLoop outperforms all five tested baselines," is an aggregate across four goals, five baselines, and 20 programs. The abstract provides no per-goal breakdown, no confidence intervals, and no effect sizes, and the unreadable full text cannot supply them. To support the claim of goal-specific effectiveness, the authors must report disaggregated results for each goal, especially separating the failure-exception and unique-coverage goals from the input-to-code complexity and long-execution-time goals, since the latter can be gamed by syntactically inflating inputs or nesting depth.
  3. [Abstract] The claim that FdLoop is "(up to) twice (2X) as effective as the best baseline (EvoGFuzz) in inducing erroneous behaviors" lacks the measurement details needed to interpret it: what counts as an erroneous behavior, what the execution budget is, how many seeds or runs were used, and what the subject-level effect sizes and dispersion are. The paper should report these numbers, ideally with per-subject and per-seed variance.
  4. [Abstract] The ablation statement ("the main components of FdLoop ... contribute positively to its effectiveness") is asserted without any visible ablation results. The paper should provide a complete ablation table that isolates each component, preferably with a statistical comparison across subjects, so the contribution of the input mutator, grammar mutator, and feedback signals can be checked independently.
minor comments (4)
  1. [Abstract] The abstract contains grammatical errors, e.g., "FdLoop iteratively selects, evolves and learn the input distribution" and "In most (86%) settings"; the paper should be proofread before resubmission.
  2. [Abstract] The term "settings" in "86% settings" should be defined precisely (e.g., goal-subject pairs) so the denominator of the win-rate claim is unambiguous.
  3. [Abstract] The closing statement that FdLoop "scales to multiple testing goals across varying parameter settings" is vague; specify which parameters were varied and what "scales" means quantitatively.
  4. [Full Text / References] No references, related work discussion, or threat-to-validity section is visible in the supplied text. These should be present and legible in the revised version.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation found; the evaluation is benchmark-based and externally anchored.

full rationale

The available abstract describes FdLoop as an iterative approach that 'selects, evolves and learn the input distribution of goal-specific test inputs via test feedback and a probabilistic grammar.' The central quantitative claims — an 86% win rate and up to 2x effectiveness over EvoGFuzz — are comparative results against five external baselines and 20 open-source programs across JSON, CSS, and JavaScript. The goal metrics used for guidance (unique code coverage, input-to-code complexity, program failures, long execution time) are also used for evaluation, but this is a standard optimization-and-benchmark setup, not a self-definitional reduction: the paper does not define any goal metric in terms of FdLoop's own learned distribution, nor does it fit a parameter to a subset and then report that same subset as a prediction. No uniqueness theorem, load-bearing self-citation, or ansatz-smuggling citation is visible in the available text. The supplied full text is mojibake, so equations and per-goal tables could not be inspected; however, unreadable text is an evidence limitation, not a circularity. The skeptical concern that size- and latency-oriented goals may be easier to satisfy is a benchmark-validity question, not a circularity question. Under the required standard, circularity must be exhibited with a specific reduction from the paper's own equations or citations, and none can be exhibited here. Therefore the appropriate finding is no significant circularity.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

Abstract-level review only. The axioms listed are the domain assumptions behind the empirical claim; no free parameters or invented entities are visible without the method section.

assumptions (3)
  • domain assumption Grammar-based input generation is an appropriate framework for the target testing goals.
    The method is built on the premise that syntactically valid inputs from a grammar can be steered toward coverage, complexity, failures, and runtime goals.
  • domain assumption The chosen feedback signals (coverage, exceptions, execution time) accurately reflect the user-specified testing goals.
    FdLoop's learning loop uses test feedback to update the probabilistic grammar, so the feedback must be a valid proxy for the goal.
  • domain assumption The 20 open-source subjects and JSON, CSS, and JavaScript grammars are representative of real-world grammar-based testing workloads.
    The headline 86% win rate is only meaningful if the benchmark selection is not biased; this cannot be checked from the abstract.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Directed Grammar-Based Test Generation." pith.science (2026). https://pith.science/paper/IGOQYR5K

@misc{pith2026250801472,
  author       = {Pith},
  title        = {Pith review of: Directed Grammar-Based Test Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IGOQYR5K}},
  note         = {Machine review of arXiv:2508.01472}
}
read the original abstract

To effectively test complex software, it is important to generate goal-specific inputs, i.e., inputs that achieve a specific testing goal. However, most state-of-the-art test generators are not designed to target specific goals. Notably, grammar-based test generators, which (randomly) produce syntactically valid inputs via an input specification (i.e., grammar) have a low probability of achieving an arbitrary testing goal. This work addresses this challenge by proposing an automated test generation approach (called FdLoop) which iteratively learns relevant input properties from existing inputs to drive the generation of goal-specific inputs. Given a testing goal, FdLoop iteratively selects, evolves and learn the input distribution of goal-specific test inputs via test feedback and a probabilistic grammar. We concretize FdLoop for four testing goals, namely unique code coverage, input-to-code complexity, program failures (exceptions) and long execution time. We evaluate FdLoop using three (3) well-known input formats (JSON, CSS and JavaScript) and 20 open-source software. In most (86%) settings, FdLoop outperforms all five tested baselines namely the baseline grammar-based test generators (random, probabilistic and inverse-probabilistic methods), EvoGFuzz and DynaMosa. FdLoop is (up to) twice (2X) as effective as the best baseline (EvoGFuzz) in inducing erroneous behaviors. In addition, we show that the main components of FdLoop (i.e., input mutator, grammar mutator and test feedbacks) contribute positively to its effectiveness. Finally, our evaluation demonstrates that FdLoop effectively achieves single testing goals (revealing erroneous behaviors, generating complex inputs, or inducing long execution time) and scales to multiple testing goals across varying parameter settings.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

66 extracted references · 59 canonical work pages

  1. [1]

    write newline

    " write newline "" initialize.prev.this.status FUNCTION begin.bib " write newline preamble empty 'skip preamble write newline if " thebibliography " longest.label * " " * write newline " [1] #1 " write newline " url@samestyle " write newline " " write newline " [2] #2 " write newline " =0pt " write newline " " ALTinterwordstretchfactor * " " * write newli...

  2. [2]

    4" FUNCTION default.is.dash.repeated.names #1 FUNCTION default.name.format.string

    11em plus .33em minus .07em 4000 4000 100 4000 4000 500 `\.=1000 = #1 \@IEEEnotcompsoconly \@IEEEcompsoconly #1 * [1] 0pt [0pt][0pt] #1 * [1] 0pt [0pt][0pt] #1 * \| ** #1 \@IEEEauthorblockNstyle \@IEEEcompsocnotconfonly \@IEEEauthorblockAstyle \@IEEEcompsocnotconfonly \@IEEEcompsocconfonly \@IEEEauthordefaulttextstyle \@IEEEcompsocnotconfonly \@IEEEauthor...

    work page 2007

  3. [3]

    write newline

    " write newline "" initialize.prev.this.status FUNCTION begin.bib " write newline preamble empty 'skip preamble write newline if " thebibliography " longest.label * " " * write newline " [1] #1 " write newline " url@samestyle " write newline " " write newline " [2] #2 " write newline " =0pt " write newline " " ALTinterwordstretchfactor * " " * write newli...

  4. [4]

    Hodov \'a n, \'A

    R. Hodov \'a n, \'A . Kiss, and T. Gyim \'o thy, ``Grammarinator: a grammar-based open source fuzzer,'' in Proceedings of the 9th ACM SIGSOFT international workshop on automating TEST case design, selection, and evaluation, 2018, pp. 45--48

    work page 2018

  5. [5]

    Havrikov and A

    N. Havrikov and A. Zeller , ``Systematically covering input structure,'' in 2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE), Nov 2019, pp. 189--199

    work page 2019

  6. [6]

    Holler, K

    C. Holler, K. Herzig, and A. Zeller, ``Fuzzing with code fragments,'' in Presented as part of the 21st USENIX Security Symposium ( USENIX Security 12) . 1em plus 0.5em minus 0.4em Bellevue, WA: USENIX , 2012, pp. 445--458. [Online]. Available: https://www.usenix.org/conference/usenixsecurity12/technical-sessions/presentation/holler

    work page 2012

  7. [7]

    [Online]

    w3resource.com, ``Java exercises: Prove that euclid’s algorithm computes the greatest common divisor of two positive given integers (cc by-nc 4.0),'' 2022. [Online]. Available: https://www.w3resource.com/java-exercises/basic/java-basic-exercise-157.php

    work page 2022

  8. [8]

    Soremekun , E

    E. Soremekun , E. Pavese , N. Havrikov , L. Grunske , and A. Zeller , ``Inputs from hell learning input distributions for grammar-based test generation,'' IEEE Transactions on Software Engineering, pp. 1--1, 2020

    work page 2020

Show all 66 references
  1. [9]

    Eberlein, Y

    M. Eberlein, Y. Noller, T. Vogel, and L. Grunske, ``Evolutionary grammar-based fuzzing,'' in Search-Based Software Engineering: 12th International Symposium, SSBSE 2020, Bari, Italy, October 7--8, 2020, Proceedings 12. 1em plus 0.5em minus 0.4em Springer, 2020, pp. 105--120

    2020

  2. [10]

    T. Parr. (2014) Antlr. [Online]. Available: http://www.antlr.org

    2014

  3. [11]

    V. J. M. Man \` e s, H. Han, C. Han, S. K. Cha, M. Egele, E. J. Schwartz, and M. Woo, ``Fuzzing: Art, science, and engineering,'' CoRR, vol. abs/1812.00140, 2018. [Online]. Available: http://arxiv.org/abs/1812.00140

    2018 arXiv

  4. [12]

    Bendrissou, C

    B. Bendrissou, C. Cadar, and A. F. Donaldson, ``Grammar mutation for testing input parsers,'' ACM Trans. Softw. Eng. Methodol., vol. 34, no. 4, Apr. 2025. [Online]. Available: https://doi.org/10.1145/3708517

    2025 doi

  5. [13]

    Zeller, R

    A. Zeller, R. Gopinath, M. B \"o hme, G. Fraser, and C. Holler, The Fuzzing Book. 1em plus 0.5em minus 0.4em CISPA Helmholtz Center for Information Security, 2024, retrieved 2024-07-01 16:50:18+02:00. [Online]. Available: https://www.fuzzingbook.org/

    2024

  6. [14]

    Luke, ``Two fast tree-creation algorithms for genetic programming,'' Evolutionary Computation, IEEE Transactions on, vol

    S. Luke, ``Two fast tree-creation algorithms for genetic programming,'' Evolutionary Computation, IEEE Transactions on, vol. 4, pp. 274 -- 283, 10 2000

    2000

  7. [15]

    Parr and K

    T. Parr and K. Fisher, ``Ll(*): The foundation of the antlr parser generator,'' SIGPLAN Not., vol. 46, no. 6, p. 425–436, jun 2011. [Online]. Available: https://doi.org/10.1145/1993316.1993548

    2011

  8. [16]

    Panichella, F

    A. Panichella, F. M. Kifetew, and P. Tonella, ``Automated test case generation as a many-objective optimisation problem with dynamic selection of the targets,'' IEEE Transactions on Software Engineering, vol. 44, no. 2, pp. 122--158, 2017

    2017

  9. [17]

    Fraser and A

    G. Fraser and A. Arcuri, ``Evosuite: automatic test suite generation for object-oriented software,'' in Proceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on Foundations of software engineering, 2011, pp. 416--419

    2011

  10. [18]

    H. B. Mann and D. R. Whitney, `` On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other ,'' The Annals of Mathematical Statistics, vol. 18, no. 1, pp. 50 -- 60, 1947. [Online]. Available: https://doi.org/10.1214/aoms/1177730491

    1947

  11. [19]

    Arcuri and L

    A. Arcuri and L. Briand, ``A hitchhiker's guide to statistical tests for assessing randomized algorithms in software engineering,'' Software Testing, Verification and Reliability, vol. 24, no. 3, pp. 219--250, 2014

    2014

  12. [20]

    Kirschner and E

    L. Kirschner and E. SOREMEKUN, `` Directed Grammar-Based Test Generation - Replication Package ,'' 12 2023. [Online]. Available: https://figshare.com/articles/journal_contribution/Directed_Grammar-Based_Test_Generation_-_Replication_Package/24549064

    2023

  13. [21]

    S. Morin. (2021) Lz4 3.1.3. [Online]. Available: https://pypi.org/project/lz4

    2021

  14. [22]

    J. D. Hunter, ``Matplotlib: A 2d graphics environment,'' Computing in science & engineering, vol. 9, no. 3, pp. 90--95, 2007

    2007

  15. [23]

    C. R. Harris, K. J. Millman, S. J. van der Walt, R. Gommers, P. Virtanen, D. Cournapeau, E. Wieser, J. Taylor, S. Berg, N. J. Smith, R. Kern, M. Picus, S. Hoyer, M. H. van Kerkwijk, M. Brett, A. Haldane, J. Fernández del Río, M. Wiebe, P. Peterson, P. Gérard-Marchant, K. Shepp...

    2020

  16. [24]

    Hagberg, D

    A. Hagberg, D. Schult, and M. Renieris. (2021) pygraphviz 1.7. [Online]. Available: pygraphviz.github.io

    2021

  17. [25]

    Tretyakov

    K. Tretyakov. (2020) matplotlib-venn 0.11.6. [Online]. Available: https://pypi.org/project/matplotlib-venn

    2020

  18. [26]

    [Online]

    (2021) tqdm 4.62.3. [Online]. Available: https://pypi.org/project/tqdm

    2021

  19. [27]

    Jenks, ``Python sorted containers,'' Journal of Open Source Software, vol

    G. Jenks, ``Python sorted containers,'' Journal of Open Source Software, vol. 4, no. 38, p. 1330, 2019. [Online]. Available: https://doi.org/10.21105/joss.01330

    2019 doi

  20. [28]

    Krekel, B

    H. Krekel, B. Oliveira, R. Pfannschmidt, F. Bruynooghe, B. Laugher, and F. Bruhin, ``pytest 6.2.5,'' 2004. [Online]. Available: https://github.com/pytest-dev/pytest

    2004

  21. [29]

    Merkel, ``Docker: lightweight linux containers for consistent development and deployment,'' Linux journal, vol

    D. Merkel, ``Docker: lightweight linux containers for consistent development and deployment,'' Linux journal, vol. 2014, no. 239, p. 2, 2014

    2014

  22. [30]

    G. Inc. (2018) Rest api v3. [Online]. Available: https://developer.github.com/v3/

    2018

  23. [31]

    [Online]

    Havrikov, ``Bug report: Confusing exceptions,'' 2018. [Online]. Available: https://github.com/google/gson/issues/1247

    2018

  24. [32]

    Havrikov, ``Bug report: Stringindexoutofboundsexception,'' 2018

    N. Havrikov, ``Bug report: Stringindexoutofboundsexception,'' 2018. [Online]. Available: https://github.com/joelittlejohn/jsonschema2pojo/issues/830

    2018

  25. [33]

    [Online]

    owlike, ``Genson issues,'' 2023. [Online]. Available: https://github.com/owlike/genson/issues

    2023

  26. [34]

    W. W. T. Surveys, ``Usage statistics of json-ld for websites,'' 2023. [Online]. Available: https://w3techs.com/technologies/details/da-jsonld

    2023

  27. [35]

    [Online]

    Statista, ``Most used programming languages among developers worldwide as of 2023,'' 2023. [Online]. Available: https://www.statista.com/statistics/793628/worldwide-developer-survey-most-used-languages/

    2023

  28. [36]

    2.0, ``Githut 2.0:a small place to discover languages in github,'' 2023

    G. 2.0, ``Githut 2.0:a small place to discover languages in github,'' 2023. [Online]. Available: https://madnight.github.io/githut/#/pull_requests/2023/3

    2023

  29. [37]

    Srivastava and M

    P. Srivastava and M. Payer, ``Gramatron: Effective grammar-aware fuzzing,'' in Proceedings of the 30th acm sigsoft international symposium on software testing and analysis, 2021, pp. 244--256

    2021

  30. [38]

    J. Wang, B. Chen, L. Wei, and Y. Liu, ``Superion: Grammar-aware greybox fuzzing,'' in 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE). 1em plus 0.5em minus 0.4em IEEE, 2019, pp. 724--735

    2019

  31. [39]

    H. L. Nguyen and L. Grunske, ``Bedivfuzz: integrating behavioral diversity into generator-based fuzzing,'' in Proceedings of the 44th International Conference on Software Engineering, 2022, pp. 249--261

    2022

  32. [40]

    Wang , B

    J. Wang , B. Chen , L. Wei , and Y. Liu , ``Skyfire: Data-driven seed generation for fuzzing,'' in 2017 IEEE Symposium on Security and Privacy (SP), May 2017, pp. 579--594

    2017

  33. [41]

    Aschermann, T

    C. Aschermann, T. Frassetto, T. Holz, P. Jauernig, A. Sadeghi, and D. Teuchert, ``Nautilus: Fishing for deep bugs with grammars,'' in NDSS, 2019

    2019

  34. [42]

    Poulding, R

    S. Poulding, R. Alexander, J. A. Clark, and M. J. Hadley, ``The optimisation of stochastic grammars to enable cost-effective probabilistic structural testing,'' in Proceedings of the 15th annual conference on Genetic and evolutionary computation, 2013, pp. 1477--1484

    2013

  35. [43]

    Olsthoorn, A

    M. Olsthoorn, A. van Deursen, and A. Panichella, ``Generating highly-structured input data by combining search-based testing and grammar-based fuzzing,'' in Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering, 2020, pp. 1224--1228

    2020

  36. [44]

    F. M. Kifetew, R. Tiella, and P. Tonella, ``Combining stochastic grammars and genetic programming for coverage testing at the system level,'' in Search-Based Software Engineering: 6th International Symposium, SSBSE 2014, Fortaleza, Brazil, August 26-29, 2014. Proceedings 6. 1e...

    2014

  37. [45]

    ------, ``Generating valid grammar-based test inputs by means of genetic programming and annotated grammars,'' Empirical Software Engineering, vol. 22, no. 2, pp. 928--961, 2017

    2017

  38. [46]

    Lemieux, R

    C. Lemieux, R. Padhye, K. Sen, and D. Song, ``Perffuzz: Automatically generating pathological inputs,'' in Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis, ser. ISSTA 2018. 1em plus 0.5em minus 0.4em New York, NY, USA: Association f...

    2018

  39. [47]

    S. Kim, M. Xu, S. Kashyap, J. Yoon, W. Xu, and T. Kim, ``Finding semantic bugs in file systems with an extensible fuzzing framework,'' in SOSP '19, 2019

    2019

  40. [48]

    Padhye, C

    R. Padhye, C. Lemieux, and K. Sen, ``Jqf: Coverage-guided property-based testing in java,'' in Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis, ser. ISSTA 2019. 1em plus 0.5em minus 0.4em New York, NY, USA: Association for Computing...

    2019

  41. [49]

    Padhye, C

    R. Padhye, C. Lemieux, K. Sen, M. Papadakis, and Y. L. Traon, ``Zest: Validity fuzzing and parametric generators for effective random testing,'' CoRR, vol. abs/1812.00078, 2018. [Online]. Available: http://arxiv.org/abs/1812.00078

    2018 arXiv

  42. [50]

    Lemieux and K

    C. Lemieux and K. Sen, ``Fairfuzz: A targeted mutation strategy for increasing greybox fuzz testing coverage,'' in Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering, ser. ASE 2018. 1em plus 0.5em minus 0.4em New York, NY, USA: Associat...

    2018

  43. [51]

    Huang, Y

    H. Huang, Y. Guo, Q. Shi, P. Yao, R. Wu, and C. Zhang, ``Beacon: Directed grey-box fuzzing with provable path pruning,'' in 2022 IEEE Symposium on Security and Privacy (SP). 1em plus 0.5em minus 0.4em IEEE, 2022, pp. 36--50

    2022

  44. [52]

    Majumdar and R.-G

    R. Majumdar and R.-G. Xu, ``Directed test generation using symbolic grammars,'' in Proceedings of the 22nd IEEE/ACM International Conference on Automated Software Engineering, ser. ASE '07. 1em plus 0.5em minus 0.4em New York, NY, USA: Association for Computing Machinery, 2007...

    2007

  45. [53]

    T. Ji, Z. Wang, Z. Tian, B. Fang, Q. Ruan, H. Wang, and W. Shi, ``Aflpro: Direction sensitive fuzzing,'' Journal of Information Security and Applications, vol. 54, p. 102497, 2020. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S2214212619305733

    2020

  46. [54]

    R. K. Medicherla, R. Komondoor, and A. Roychoudhury, ``Fitness guided vulnerability detection with greybox fuzzing,'' in Proceedings of the IEEE/ACM 42nd International Conference on Software Engineering Workshops, ser. ICSEW'20. 1em plus 0.5em minus 0.4em New York, NY, USA: As...

    2020

  47. [55]

    Liang, X

    H. Liang, X. Yu, X. Cheng, J. Liu, and J. Li, ``Multiple targets directed greybox fuzzing,'' IEEE Transactions on Dependable and Secure Computing, 2023

    2023

  48. [56]

    Nguyen, S

    M.-D. Nguyen, S. Bardin, R. Bonichon, R. Groz, and M. Lemerre, ``Binary-level directed fuzzing for \ Use-After-Free \ vulnerabilities,'' in 23rd International Symposium on Research in Attacks, Intrusions and Defenses (RAID 2020), 2020, pp. 47--62

    2020

  49. [57]

    G. Lee, W. Shim, and B. Lee, ``Constraint-guided directed greybox fuzzing,'' in 30th USENIX Security Symposium (USENIX Security 21), 2021, pp. 3559--3576

    2021

  50. [58]

    M. E. Garbelini, C. Wang, and S. Chattopadhyay, ``Greyhound: Directed greybox wi-fi fuzzing,'' IEEE Transactions on Dependable and Secure Computing, vol. 19, no. 2, pp. 817--834, 2022

    2022

  51. [59]

    B \"o hme, V.-T

    M. B \"o hme, V.-T. Pham, M.-D. Nguyen, and A. Roychoudhury, ``Directed greybox fuzzing,'' in Proceedings of the 2017 ACM SIGSAC conference on computer and communications security, 2017, pp. 2329--2344

    2017

  52. [60]

    H. Chen, Y. Xue, Y. Li, B. Chen, X. Xie, X. Wu, and Y. Liu, ``Hawkeye: Towards a desired directed grey-box fuzzer,'' in Proceedings of the 2018 ACM SIGSAC conference on computer and communications security, 2018, pp. 2095--2108

    2018

  53. [61]

    Misherghi and Z

    G. Misherghi and Z. Su, ``Hdd: hierarchical delta debugging,'' in Proceedings of the 28th international conference on Software engineering, 2006, pp. 142--151

    2006

  54. [62]

    Zeller and R

    A. Zeller and R. Hildebrandt, ``Simplifying and isolating failure-inducing input,'' IEEE Transactions on software engineering, vol. 28, no. 2, pp. 183--200, 2002

    2002

  55. [63]

    Kampmann, N

    A. Kampmann, N. Havrikov, E. O. Soremekun, and A. Zeller, ``When does my program do this? learning circumstances of software behavior,'' in Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engine...

    2020

  56. [64]

    Gopinath, A

    R. Gopinath, A. Kampmann, N. Havrikov, E. O. Soremekun, and A. Zeller, ``Abstracting failure-inducing inputs,'' in Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis, ser. ISSTA 2020. 1em plus 0.5em minus 0.4em New York, NY, USA: Assoc...

    2020

  57. [65]

    Kirschner, E

    L. Kirschner, E. Soremekun, and A. Zeller, ``Debugging inputs,'' in Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering, 2020, pp. 75--86

    2020

  58. [66]

    Kirschner, E

    L. Kirschner, E. Soremekun, R. Gopinath, and A. Zeller, ``Input repair via synthesis and lightweight error feedback,'' 2022. [Online]. Available: https://arxiv.org/abs/2208.08235

    2022 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.