Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

An Empirical Study of Fuzz Harness Degradation

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Fuzz harnesses in OSS-Fuzz keep finding bugs for years without updates, with coverage staying stable on average, as long as they still build.

desk verdict First large-scale longitudinal study of harness decay in OSS-Fuzz; valuable taxonomy and open artifacts, but the 'no explicit updates' longevity claim depends on an unvalidated update-detection heuristic. read the letter →

arxiv 2505.06177 v2 pith:VDBZOPGD submitted 2025-05-09 cs.SE cs.CR

Philipp Görz , Joschua Schilling , Nicolai Bissantz , Thorsten Holz This is my paper · ORCID
classification cs.SEcs.CR
keywords fuzzharnessdegradationOSS-Fuzzcodecoveragebug-findingcapabilitycontinuousfuzzingempiricalstudymeasurement
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether the fuzz harnesses that feed inputs into software projects decay as the projects themselves evolve, and answers with a longitudinal study of the OSS-Fuzz continuous-fuzzing platform: 29,019 harness versions across 433 C/C++ projects, with daily coverage reports and bug records spanning about eight years. The central finding is that coverage stays stable on average and that harnesses keep finding bugs for long periods even without explicit updates, as long as they still build. Only about 5% of harness versions lose at least 5 percentage points of coverage within half a year, and manual examination of 308 coverage-drop cases attributes these losses to partial build failures, newly added project code, external libraries entering the coverage report, and corpus-size artifacts rather than to gradual harness rot. The paper proposes new monitoring metrics, implemented in OSS-Fuzz and Fuzz Introspector, that would alert maintainers when a harness quietly degrades.

What carries the argument

The central object is the harness version, defined by a heuristic that detects harness-changing commits: any change to a C/C++ file in the project whose path contains 'fuzz' (excluding 'fuzzy'), any change to a file defining the standard entry point LLVMFuzzerTestOneInput or LLVMFuzzerInitialize, and any OSS-Fuzz repository change outside project metadata and Docker/Jenkins files. Each daily Clang line-coverage report from OSS-Fuzz is then attached to the currently active harness version, and coverage is measured as relative line-coverage percentage, using the maximum of the first three days after an update as the baseline to compensate for slow saturation. The argument that degradation is rare rests on the resulting coverage-change curves over time and over changed lines, while the causal taxonomy is built from 308 manually examined coverage drops filtered for a 5-percentage-point sustained decline.

What would settle it

Recompute the coverage-change curves using only coverage days whose Fuzz Introspector report shows no external-library files in the denominator and whose run completed without timeout or out-of-memory; if the median coverage since the last harness update then declines by several percentage points within a year, the surprising-longevity conclusion collapses to a measurement artifact.

Watch

Extended reading notes

Core claim

The authors establish a counter-intuitive result: fuzz harnesses in OSS-Fuzz have a surprisingly long effective lifetime. Coverage measured from the days after each harness update shows only a slight downward trend in the mean and a neutral median, and bug-finding per changed line of code stays roughly flat over at least the first half year of a harness version's life. Harness updates themselves have only a modest average effect — a mean coverage increase of 0.26% and a median of exactly 0% — yet the minority of updates that raise coverage by more than 5 percentage points produce a clear burst of newly found bugs. When coverage does drop sharply, the categorized root causes are partial build failures (such as a dependency that stops compiling), project code added or changed without corresponding harness maintenance, external library code that inflates the coverage denominator, and corpus-size collapses caused by out-of-memory or timeout runs being recorded as successful coverage measurements. The paper concludes that the real risk is not slow rot but discrete, silent events that break or distort coverage, and it contributes metrics to detect those events.

Load-bearing premise

The longevity result is computed from OSS-Fuzz's daily line-coverage reports, and the paper shows those reports can be distorted — third-party libraries can enter the coverage denominator and out-of-memory or timeout runs are recorded as successful measurements; if those artifacts are widespread, the apparent stability is partly a measurement artifact rather than genuine harness longevity.

Editorial extensions

If this is right

  • A harness that still builds can be treated as a continuing asset: its bug-finding rate per changed line stays roughly flat for at least half a year, so maintainers can prioritize other work over routine harness refresh.
  • The benefits of harness maintenance are concentrated in discrete events: updates that lift coverage by more than 5 percentage points are followed by a burst of newly found bugs, while the average update changes coverage by almost nothing.
  • Roughly one harness version in 20 loses at least 5 percentage points of coverage within half a year, and the cause is usually a build failure, newly added code, a distorted coverage report, or a shrunken corpus — so monitoring for those specific events is the practical response.
  • Projects whose coverage stays below about 35% find roughly one bug per 14,474 changed lines, versus one per 3,985 lines for higher-coverage projects, making coverage level a first-order driver of fuzzing return.
  • The proposed metrics — tracking removed or renamed harnesses, coverage stability over time, harness statefulness, and corpus size relative to covered complexity — would in principle detect all of the categorized causes of coverage drops and are being contributed to OSS-Fuzz and Fuzz Introspector.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the stability result generalizes beyond OSS-Fuzz's mature, security-critical projects, automatic harness-generation and maintenance tools (including LLM-based ones) should target discrete breakage events rather than continuous re-synthesis; keeping builds green and the coverage denominator clean may matter more than frequent regeneration.
  • A testable extension follows from the taxonomy: projects that pin their dependencies or exclude third-party libraries from coverage reports should show fewer spurious 'degradation' alarms, which the proposed metrics could verify by comparing alarmed versus non-alarmed projects across such policy changes.
  • The harness-change heuristic likely undercounts updates made in separate harness repositories or in code paths that do not match the pattern; cross-checking the heuristic against per-harness Fuzz Introspector reports could raise or lower the longevity estimate.
  • Because the paper itself shows that out-of-memory and timeout runs are recorded as successful coverage and that external code can inflate the denominator, a re-analysis that excludes those artifacts could shift the stable mean and median downward; the magnitude of the shift would quantify how much of the 'surprising longevity' is real.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper presents a large-scale longitudinal study of fuzz harness degradation in Google's OSS-Fuzz platform. Using coverage reports, commit histories, Monorail bug data, and Fuzz Introspector reports for 433 C/C++ projects and 29,019 harness versions over 2016–2024, the authors address five research questions on the immediate effects of harness updates, the rate of coverage degradation over time, the degradation of bug-finding capability, the causes of coverage drops, and practical detection metrics. The central findings are that overall coverage in OSS-Fuzz remains stable, that harness updates have only a small mean effect on coverage (0.26%), that bug-finding capability does not automatically decline for harnesses that still build, that approximately 5% of harness versions lose at least 5 percentage points of coverage within half a year, and that manual analysis of 308 coverage-drop cases yields four main causes: partial build failures, added project code or churn, third-party code polluting coverage reports, and corpus-size decreases caused by OOM/timeout measurement artifacts. The authors also report contributions to OSS-Fuzz and Fuzz Introspector in the form of new detection metrics.

Significance. This is, to my knowledge, the first systematic longitudinal study of fuzz harness degradation at the scale of a major continuous fuzzing platform. If the findings hold, they have practical value: they suggest that harnesses in mature OSS-Fuzz projects are more resilient than often assumed, while also identifying concrete, actionable failure modes. The paper is unusually transparent: the dataset, scraping code, analysis notebook, and case-study notes are public, the data-cleaning steps are documented in detail, and the 308 degradation cases were manually reviewed with category counts provided. The proposed metrics, if integrated, could help maintainers detect silent degradation. However, the significance is conditional on two methodological pillars: the heuristic that identifies harness-update events, and the integrity of the coverage reports used to measure stability. Both are explicitly acknowledged as imperfect in the manuscript, but the magnitude of their impact on the central claims is not quantified.

major comments (3)
  1. [§3.1.4, Figs. 3–5] The harness-change heuristic is load-bearing for the 'no explicit updates' longevity claim, and its false-negative rate is unmeasured. The heuristic counts a commit as a harness change only if it touches an OSS-Fuzz file other than project.yaml/Dockerfile/Jenkins, or a C/C++ file whose path contains 'fuzz' (excluding 'fuzzy') or that defines LLVMFuzzerTestOneInput/LLVMFuzzerInitialize. This misses several plausible update modes: changes to project build files (CMakeLists.txt, Makefile) that add or alter fuzz targets; dependency version bumps or vendored-code updates that change what the harness links against; header or signature changes in files whose names do not contain 'fuzz'; harnesses moved to files outside the matching paths; and commits to separate fuzzing repositories beyond the two projects manually detected. The manual 'suspicious gaps' check described in §3.1.4 can only catch long silent periods in relatively active projects; it does not provide a quantitative false-negative rate for projects with frequent commits. Because RQ2 and RQ3 define 'time since last harness update' based entirely on this heuristic (Figures 3–5), a non-trivial false-negative rate would mean that many periods labeled 'unmaintained' actually contain maintenance, in which case the observed flat coverage and sustained bug rates would be expected even if unmaintained harnesses do degrade. I request a validation of the heuristic against a statistically sampled subset of manually labeled commits, or an alternative confirmation using OSS-Fuzz build logs to establish when harness files were actually built or changed.
  2. [§3.5.3, §3.5.4, §3.3] Coverage measurement artifacts are not excluded from the RQ2 stability analysis, even though the paper's own evidence indicates they affect the coverage metric on a large scale. The paper shows that third-party code entering the coverage report can cause a 93-percentage-point relative coverage drop (grpc-httpjson-transcoding, §3.5.3) and that OOM/timeout runs are recorded as successful coverage measurements (§3.5.4), an issue stated in §3.6 to affect 'at least 10% of harnesses'. These artifacts are included in the coverage data used to compute the 'surprisingly stable' coverage result and the roughly 5% degradation rate in Figure 3. The manuscript does not provide a sensitivity analysis that excludes known artifact patterns (e.g., harnesses with corpus-size drops or repeated OOM/timeout behavior) or corrects the denominator for external third-party code. Without such an analysis, the central claim that coverage is stable over time could be partly a property of the measurement infrastructure rather than of the harnesses. I ask the authors to rerun Figure 3 and the associated degradation statistics after removing or adjusting for these artifact-prone observations, and to report whether the qualitative conclusions change.
  3. [§3.3, §3.4] The exclusion of harnesses that no longer build is a scope limitation that should be stated as part of the central claim, not only in the methodology. The abstract and §3.3 conclude 'surprising longevity' of bug-finding capability, qualified by 'as long as they still build'. This is a legitimate conditional claim, but the practical implication is weakened because a harness that fails to build finds zero bugs and is the clearest form of degradation. The paper cites Nourry et al. [41] for the prevalence of broken builds, but does not combine that information with the new data to give an overall picture of how often unmaintained harnesses actually fail. I would like the authors to state explicitly, in the abstract or conclusion, the proportion of harness versions that stop building entirely during the study period, so that the conditional 'as long as they still build' is placed in context.
minor comments (6)
  1. [§3] The text says 'In total, we investigate four research questions' immediately before listing RQ1 through RQ5; this should be 'five'.
  2. [§3.1.4] The function name is misspelled as 'LLVMFuzzerInititalize'; it should be 'LLVMFuzzerInitialize'.
  3. [§3.5] There is a missing space in 'This results in267 instances'.
  4. [Fig. 4 caption] The caption describes coverage changes as '>5% coverage increase, >5% coverage decrease and everything in between', but the legend in the figure shows '< -5%', '-5% to +5%', and '< +5%'. Please reconcile the notation.
  5. [§3.3] The sentence 'we cut off the tail end where#Harnesses falls below 100' contains a rendering issue with the '#' character; please fix the typography.
  6. [§3.4] The phrase 'after which our data starts to be too sparse' is ambiguous; it is unclear whether 'after which' refers to half a year or to the cutoff point in the figure.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the study is a direct empirical analysis of independently collected OSS-Fuzz coverage, bug, and commit data.

full rationale

The paper contains no derivation chain that reduces to its inputs. Its central claims—stable average coverage and sustained bug-finding capability of harness versions that are not explicitly updated—are direct observations over OSS-Fuzz coverage reports, Monorail bug reports, and git commits. The harness-change heuristic in Section 3.1.4 is an independent input definition, not a fitted parameter: it classifies commits by file path and function-name patterns, while the outcomes (coverage change, bugs per modified line, time since last harness update) are measured from separate data sources. The paper explicitly reports results that contradict its own initial hypothesis, such as the near-zero mean coverage change (0.26%) after harness updates, which would not occur if the conclusion were built into the definition. The cited coauthored prior results, e.g., Zhu and Böhme on regression bugs and Böhme et al. on coverage-based benchmarking, are used for contextual comparison and are not load-bearing inputs that force the findings. The acknowledged measurement artifacts in Sections 3.5.3 and 3.5.4 are validity threats, not evidence of circularity. Therefore, the analysis is self-contained and no circular step can be identified.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The study rests on observational data and heuristics rather than on fitted parameters or invented entities. The main axioms are the validity of OSS-Fuzz coverage and bug reports for measuring harness effectiveness, the accuracy of the harness-change heuristic, and the representativeness of mature OSS-Fuzz projects; each is acknowledged as a limitation in the paper.

assumptions (4)
  • domain assumption OSS-Fuzz daily coverage reports are a valid measure of fuzzer effectiveness over time.
    The RQ2 longevity analysis is computed entirely from Clang source-based coverage reports; the paper itself documents distortions from external code (§3.5.3) and from OOM/timeout runs reported as successful (§3.5.4).
  • domain assumption Monorail bug and crash-revision data accurately reflect the bugs found by fuzzing.
    RQ3 uses Monorail reports; security bugs are hidden for 90 days and 25 pre-integration reports were removed (§3.1.5), and bugs without crash revisions are excluded (§3.4).
  • domain assumption The harness-change heuristic (path contains 'fuzz', functions LLVMFuzzerTestOneInput or LLVMFuzzerInitialize) correctly identifies harness updates.
    This heuristic defines 'time since last harness update' in RQ2 and RQ3; false negatives would shift the time axis. The authors manually filtered only a few anomalous projects (§3.1.4).
  • domain assumption OSS-Fuzz projects are representative of fuzzing practice in general.
    The external validity section (§4) notes OSS-Fuzz projects are mature and security-motivated, and the C/C++ focus may not generalize to memory-safe languages.

how reviews work

0 comments
Cite this review

Pith. "Pith review of An Empirical Study of Fuzz Harness Degradation." pith.science (2026). https://pith.science/paper/VDBZOPGD

@misc{pith2026250506177,
  author       = {Pith},
  title        = {Pith review of: An Empirical Study of Fuzz Harness Degradation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VDBZOPGD}},
  note         = {Machine review of arXiv:2505.06177}
}
read the original abstract

The purpose of continuous fuzzing platforms is to enable fuzzing for software projects via fuzz harnesses -- but as the projects continue to evolve, are these harnesses updated in lockstep, or do they run out of date? If these harnesses remain unmaintained, will they degrade over time in terms of coverage achieved or number of bugs found? This is the subject of our study. We study Google's OSS-Fuzz continuous fuzzing platform containing harnesses for 510 open-source C/C++ projects, many of which are security-critical. A harness is the glue code between the fuzzer and the project, so it needs to adapt to changes in the project. It is often added by a project maintainer or as part of a, sometimes short-lived, testing effort. Our analysis shows a consistent overall fuzzer coverage percentage for projects in OSS-Fuzz and a surprising longevity of the bug-finding capability of harnesses even without explicit updates, as long as they still build. However, we also identify and manually examine individual cases of harness coverage degradation and categorize their root causes. Furthermore, we contribute to OSS-Fuzz and Fuzz Introspector to support metrics to detect harness degradation in OSS-Fuzz projects guided by this research.

Figures

Figures reproduced from arXiv: 2505.06177 by the authors.

Figure 1
Figure 1. Fuzz coverage across C/C++ projects in OSS-Fuzz. 50% of the project’s source code [21]. The importance of well-fitting harnesses is also highlighted, especially in long￾lasting fuzzing campaigns such as OSS-Fuzz, as a coverage plateau during fuzzing is often caused by an inadequate har￾ness design [17]. However, how does the effectiveness of the provided har￾ness develop after the initial onboarding over time? Moder… view at source ↗
Figure 2
Figure 2. Bug finding relative to harness updates. 3.2 RQ1: What are the immediate effects of harness updates? We hypothesize that harnesses slowly degrade over their life￾time and that this can be counteracted by maintenance in the form of harness updates. Which cause an immediate increase in coverage and a burst of new bugs after the introduction of the new harness version. To verify our hypothesis, we study the specific ev… view at source ↗
Figure 3
Figure 3. Relative coverage change since a harness update, aggregated across all projects. To filter out minor changes, we divide harness updates into three categories. We consider a harness improved, if the project coverage increases at least 5%. Similarly, we consider a harness degraded if coverage decreases by 5% or more. Everything in between is considered as a maintained harness, which is the predominant category with 92… view at source ↗
Figures from the paper (12 more)
Figure 4
Figure 4. Figure 4: Found bugs per changed lines since the last harness update This result likely stems from the maturity of the projects in OSS-Fuzz, as only relatively mature projects with a sig￾nificant user base are added to OSS-Fuzz. For less mature projects with more active developm…
Figure 5
Figure 5. Figure 5: Bugs by crash revision type over time since the last harness update across all projects. is why the number of harnesses reduces over time. Note that we again cut off the plot when reaching 500 harnesses to avoid misleading results. In Figure 4a we can see the ratio of …
Figure 6
Figure 6. Figure 6: Reports and coverage for Curl (a, b, c) and Serenity (d). Note that we are excluding harnesses that no longer built at all. While this can be seen as an extreme form of harness degradation, which results in zero coverage, it is also easily detected, and thus not intere…
Figure 7
Figure 7. Figure 7: Sudoers coverage over time. We noticed several cases where the corpus created by the fuzzer drops to a small fraction of its previous size. For example, one harness for Sudo [48] decreases from ~12k corpus entries13 to 2114 at the start of 2023, as shown in [PITH_FULL…
Figure 8
Figure 8. Figure 8: The coverage performance and report of grpc-httpjson-transcoding. to covered code or covered complexity. While coverage is a useful metric, it does not allow assessing how well the covered code is fuzzed. For example, a single seed corpus entry may cover 80% of the cod…
Figure 9
Figure 9. Figure 9: Cumulative number of projects, respectively, by number of commits, harness updates, and ratio of commits to harness updates. Dotted lines in red mark the cutoff points. As the projects are quite diverse in their activity level and number of harness updates. We separate…
Figure 10
Figure 10. Figure 10: Coverage since the last harness update for projects with a high ratio of harness updates [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]
Figure 11
Figure 11. Figure 11: Coverage since the last harness update for projects with a low ratio of harness updates. Coverage change since last harness update #H arnesses 0 5,000 10,000 (a) Time 1 day 2 days 1 week 2 weeks 4 weeks 8 weeks 16 weeks 32 weeks 64 weeks 128 weeks Lin e Covera ge % −1…
Figure 12
Figure 12. Figure 12: Coverage since the last harness update for projects with a high number of commits. Coverage change since last harness update #H arnesses 250 500 750 1,000 (a) Time 1 day 2 days 1 week 2 weeks 4 weeks 8 weeks 16 weeks 32 weeks 64 weeks 128 weeks Lin e Covera ge % −10 −…
Figure 13
Figure 13. Figure 13: Coverage since the last harness update for projects with a low number of commits [PITH_FULL_IMAGE:figures/full_fig_p015_13.png]
Figure 14
Figure 14. Figure 14: Coverage since the last harness update for projects with a high number of commits and a high ratio of harness updates. Coverage change since last harness update #H arnesses 0 1,000 2,000 3,000 4,000 5,000 6,000 (a) Time 1 day 2 days 1 week 2 weeks 4 weeks 8 weeks 16 w…
Figure 15
Figure 15. Figure 15: Coverage since the last harness update for projects with a high number of commits and a low ratio of harness updates [PITH_FULL_IMAGE:figures/full_fig_p016_15.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Thinking More, Harnessing Better: State Machine Guided Harness Automatic Generation with Project Digestion and Workflow Decomposition

    cs.CR 2026-07 accept novelty 6.0 of 10

    SynapseFlow uses dataflow-based function grouping and a staged rollback LLM workflow to auto-generate fuzz harnesses, achieving higher coverage and bug detection than prior tools on 25 C projects.

Reference graph

Works this paper leans on

59 extracted references · 56 canonical work pages · cited by 1 Pith paper

  1. [41]

    My fuzzers won’t build: An empirical study of fuzzing build failures

    Olivier Nourry, Yutaro Kashiwa, Weiyi Shang, Honglin Shu, and Yasu- taka Kamei. My fuzzers won’t build: An empirical study of fuzzing build failures. ACM Transactions on Software Engineering and Method- ology

    work page

  2. [1]

    A Parser Generator Tool

    Tree-sitter. A Parser Generator Tool. Documentation, 2024. URL https: //tree-sitter.github.io/

    work page 2024

  3. [2]

    An empirical study of design degradation: How software projects get worse over time

    Iftekhar Ahmed, Umme Ayda Mannan, Rahul Gopinath, and Carlos Jensen. An empirical study of design degradation: How software projects get worse over time. In 2015 ACM/IEEE International Sym- posium on Empirical Software Engineering and Measurement (ESEM) , pages 1–10. IEEE, 2015

    work page 2015

  4. [3]

    Maintenance of automated test suites in industry: An empirical study on visual gui testing

    Emil Alégroth, Robert Feldt, and Pirjo Kolström. Maintenance of automated test suites in industry: An empirical study on visual gui testing. Information and Software Technology, 73:66–80, 2016

    work page 2016

  5. [4]

    The analysis of current state of agile software devel- opment

    Samer Atawneh. The analysis of current state of agile software devel- opment. Journal of Theoretical and Applied Information Technology , 97 (22):3197–3028, 2019

    work page 2019

  6. [5]

    Soft- ware architecture degradation in open source software: A systematic literature review

    Ahmed Baabad, Hazura Binti Zulzalil, Salmi Binti Baharom, et al. Soft- ware architecture degradation in open source software: A systematic literature review. IEEE Access, 8:173681–173709, 2020

    work page 2020

  7. [6]

    Fudge: fuzz driver generation at scale

    Domagoj Babić, Stefan Bucur, Yaohui Chen, Franjo Ivančić, Tim King, Markus Kusano, Caroline Lemieux, László Szekeres, and Wei Wang. Fudge: fuzz driver generation at scale. In Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering , pages 975–985, 2019

    work page 2019

  8. [7]

    Observations and lessons learned from automated testing

    Stefan Berner, Roland Weber, and Rudolf K Keller. Observations and lessons learned from automated testing. In Proceedings of the 27th international conference on Software engineering , pages 571–579, 2005

    work page 2005

Show all 59 references
  1. [8]

    Evaluating software degradation through entropy

    Alessandro Bianchi, Danilo Caivano, Filippo Lanubile, and Giuseppe Visaggio. Evaluating software degradation through entropy. In Pro- ceedings Seventh International Software Metrics Symposium, pages 210–

  2. [9]

    Test Harness Definition

    International Software Testing Qualifications Board. Test Harness Definition. International Software Testing Qualifications Board, 2024. URL https://glossary.istqb.org/en_US/term/test-harness

    2024

  3. [10]

    Fine grained software degradation models for optimal rejuvenation policies

    Andrea Bobbio, Matteo Sereno, and Cosimo Anglano. Fine grained software degradation models for optimal rejuvenation policies. Per- formance Evaluation, 46(1):45–62, 2001

    2001

  4. [11]

    Fuzzing: On the exponential cost of vulnerability discovery

    Marcel Böhme and Brandon Falk. Fuzzing: On the exponential cost of vulnerability discovery. In Proceedings of the 28th ACM joint meeting on European software engineering conference and symposium on the foundations of software engineering , pages 713–724, 2020

    2020

  5. [12]

    Fuzzing: Challenges and reflections

    Marcel Böhme, Cristian Cadar, and Abhik Roychoudhury. Fuzzing: Challenges and reflections. IEEE Software, 38(3):79–86, 2020

    2020

  6. [13]

    On the reliability of coverage-based fuzzer benchmarking

    Marcel Böhme, László Szekeres, and Jonathan Metzman. On the reliability of coverage-based fuzzer benchmarking. In Proceedings of the 44th International Conference on Software Engineering , pages 1621–1633, 2022

    2022

  7. [14]

    Oss- fuzzgen: Automated fuzzing of open source java projects

    Sheung Chi Chan, Adam Korczynski, and David Korczynski. Oss- fuzzgen: Automated fuzzing of open source java projects. 2023

    2023

  8. [15]

    curl: command line tool and library for transferring data with URL syntax

    curl Project. curl: command line tool and library for transferring data with URL syntax. https://github.com/curl/curl. Accessed: 2024-10-19

    2024

  9. [16]

    An empirical study of oss-fuzz bugs

    Zhen Yu Ding and Claire Le Goues. An empirical study of oss-fuzz bugs. In 2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR), pages 131–142, 2021. doi: 10.1109/MSR52588.2021. 00026

    2021

  10. [17]

    Beyond the coverage plateau: A comprehensive study of fuzz blockers (registered report)

    Wentao Gao, Van-Thuan Pham, Dongge Liu, Oliver Chang, Toby Mur- ray, and Benjamin IP Rubinstein. Beyond the coverage plateau: A comprehensive study of fuzz blockers (registered report). In Proceed- ings of the 2nd International Fuzzing Workshop , pages 47–55, 2023

    2023

  11. [18]

    googletest: GoogleTest - Google Testing and Mocking Frame- work

    Google. googletest: GoogleTest - Google Testing and Mocking Frame- work. https://github.com/google/googletest, . Accessed: 2024-10-27

    2024

  12. [19]

    OSS-Fuzz: Setting up a new project

    Google. OSS-Fuzz: Setting up a new project . Google, . URL https: //google.github.io/oss-fuzz/

  13. [20]

    OSS-Fuzz: Accepting New Projects

    Google. OSS-Fuzz: Accepting New Projects . Google, . URL https:// google.github.io/oss-fuzz/getting-started/accepting-new-projects/

  14. [21]

    OSS-Fuzz: Reward Program

    Google. OSS-Fuzz: Reward Program . Google, . URL https://bughunters.google.com/about/rules/open-source/ 5097259337383936/oss-fuzz-reward-program-rules

  15. [22]

    OSS-Fuzz: Continuous Fuzzing for Open Source Software

    Google. OSS-Fuzz: Continuous Fuzzing for Open Source Software . Google, 2017. URL https://github.com/google/oss-fuzz

    2017

  16. [23]

    Fuzz Introspector Repository

    Google. Fuzz Introspector Repository. Google, 2024. URL https://github. com/ossf/fuzz-introspector

    2024

  17. [24]

    Fuzz Introspector: Fuzzing Introspection of OSS-Fuzz projects

    Google. Fuzz Introspector: Fuzzing Introspection of OSS-Fuzz projects . Google, 2024. URL https://introspector.oss-fuzz.com/

    2024

  18. [25]

    OSS-Fuzz-Gen: AI-Powered Fuzzing: Breaking the Bug Hunting Barrier

    Google. OSS-Fuzz-Gen: AI-Powered Fuzzing: Breaking the Bug Hunting Barrier. Google, 2024. URL https://security.googleblog.com/2023/08/ai- powered-fuzzing-breaking-bug-hunting.html

    2024

  19. [26]

    grpc-httpjson-transcoding

    grpc-httpjson-transcoding Project. grpc-httpjson-transcoding. https: //github.com/grpc-ecosystem/grpc-httpjson-transcoding . Accessed: 2024-10-19

    2024

  20. [27]

    The rise and evolution of agile software development

    Rashina Hoda, Norsaremah Salleh, and John Grundy. The rise and evolution of agile software development. IEEE software, 35(5):58–63, 2018

    2018

  21. [28]

    {FuzzGen}: Automatic fuzzer generation

    Kyriakos Ispoglou, Daniel Austin, Vishwath Mohan, and Mathias Payer. {FuzzGen}: Automatic fuzzer generation. In 29th USENIX Security Symposium (USENIX Security 20) , pages 2271–2287, 2020

    2020

  22. [29]

    Sp-fuzz: Fuzzing soft plc with semi-automated harness synthesis

    Seungho Jeon and Jung Taek Seo. Sp-fuzz: Fuzzing soft plc with semi-automated harness synthesis. In International Conference on Information Security Applications, pages 282–293. Springer, 2023

    2023

  23. [30]

    Utopia: Automatic generation of fuzz driver using unit tests

    Bokdeuk Jeong, Joonun Jang, Hayoon Yi, Jiin Moon, Junsik Kim, Intae Jeon, Taesoo Kim, WooChul Shim, and Yong Ho Hwang. Utopia: Automatic generation of fuzz driver using unit tests. In 2023 IEEE Symposium on Security and Privacy (SP) , pages 2676–2692. IEEE, 2023

    2023

  24. [31]

    Winnie: Fuzzing windows applications with harness synthesis and fast cloning

    Jinho Jung, Stephen Tong, Hong Hu, Jungwon Lim, Yonghwi Jin, and Taesoo Kim. Winnie: Fuzzing windows applications with harness synthesis and fast cloning. In Proceedings of the 2021 Network and Distributed System Security Symposium (NDSS 2021) , 2021

    2021

  25. [32]

    Empirical observations on software testing automation

    Katja Karhu, Tiina Repo, Ossi Taipale, and Kari Smolander. Empirical observations on software testing automation. In 2009 International Conference on Software Testing Verification and Validation, pages 201–

    2009

  26. [33]

    Evaluating fuzz testing

    George Klees, Andrew Ruef, Benji Cooper, Shiyi Wei, and Michael Hicks. Evaluating fuzz testing. In Proceedings of the 2018 ACM SIGSAC conference on computer and communications security , pages 2123–2138, 2018. An Empirical Study of Fuzz Harness Degradation

    2018

  27. [34]

    Effectiveness and scalability of fuzzing techniques in ci/cd pipelines

    Thijs Klooster, Fatih Turkmen, Gerben Broenink, Ruben ten Hove, and Marcel Böhme. Effectiveness and scalability of fuzzing techniques in ci/cd pipelines. arXiv preprint arXiv:2205.14964, 2022

    2022 arXiv

  28. [35]

    Fuzzing: a survey

    Jun Li, Bodong Zhao, and Chao Zhang. Fuzzing: a survey. Cybersecu- rity, 1:1–13, 2018

    2018

  29. [36]

    Fuzz testing in practice: Obstacles and solutions

    Jie Liang, Mingzhe Wang, Yuanliang Chen, Yu Jiang, and Renwei Zhang. Fuzz testing in practice: Obstacles and solutions. In 2018 IEEE 25th International Conference on Software Analysis, Evolution and Reengineering (SANER), pages 562–566. IEEE, 2018

    2018

  30. [37]

    Source-based Code Coverage

    LLVM. Source-based Code Coverage. Clang Documentation. LLVM, 2024. URL https://clang.llvm.org/docs/SourceBasedCodeCoverage.html

    2024

  31. [38]

    libFuzzer – A library for coverage-guided fuzz testing

    LLVM. libFuzzer – A library for coverage-guided fuzz testing. LLVM,

  32. [39]

    Prompt fuzzing for fuzz driver generation

    Yunlong Lyu, Yuxuan Xie, Peng Chen, and Hao Chen. Prompt fuzzing for fuzz driver generation. arXiv preprint arXiv:2312.17677, 2023

    2023 arXiv

  33. [40]

    On the relevance of code anomalies for identifying architecture degradation symptoms

    Isela Macia, Roberta Arcoverde, Alessandro Garcia, Christina Chavez, and Arndt Von Staa. On the relevance of code anomalies for identifying architecture degradation symptoms. In 2012 16Th european conference on software maintenance and reengineering , pages 277–286. IEEE, 2012

    2012

  34. [42]

    The human side of fuzzing: Challenges faced by developers during fuzzing activities

    Olivier Nourry, Yutaro Kashiwa, Bin Lin, Gabriele Bavota, Michele Lanza, and Yasutaka Kamei. The human side of fuzzing: Challenges faced by developers during fuzzing activities. ACM Transactions on Software Engineering and Methodology , 33(1):1–26, 2023

    2023

  35. [43]

    Openssl: Tls/ssl and crypto library

    OpenSSL Project. Openssl: Tls/ssl and crypto library. https://github. com/openssl/openssl. Accessed: 2024-10-19

    2024

  36. [44]

    protobuf

    protobuf Project. protobuf. https://github.com/protocolbuffers/ protobuf. Accessed: 2024-10-19

    2024

  37. [45]

    Sok: Prudent evaluation practices for fuzzing

    Moritz Schloegel, Nils Bars, Nico Schiller, Lukas Bernhard, Tobias Scharnowski, Addison Crump, Arash Ale-Ebrahim, Nicolai Bissantz, Marius Muench, and Thorsten Holz. Sok: Prudent evaluation practices for fuzzing. In 2024 IEEE Symposium on Security and Privacy (SP), pages 1974–...

    2024

  38. [46]

    OSS-Fuzz-Google’s Continuous Fuzzing Service for Open Source Software, 2017

    Kostya Serebryany. OSS-Fuzz-Google’s Continuous Fuzzing Service for Open Source Software, 2017

    2017

  39. [47]

    serenity: The Serenity Operating System

    serenity Project. serenity: The Serenity Operating System. https: //github.com/SerenityOS/serenity. Accessed: 2024-10-27

    2024

  40. [48]

    sudo: Utility to execute a command as another user

    sudo Project. sudo: Utility to execute a command as another user. https://github.com/sudo-project/sudo. Accessed: 2024-10-19

    2024

  41. [49]

    Fuzzing: brute force vulnerability discovery

    Michael Sutton, Adam Greene, and Pedram Amini. Fuzzing: brute force vulnerability discovery. Pearson Education, 2007

    2007

  42. [50]

    An empir- ical study of security culture in open source software communities

    Shao-Fang Wen, Mazaher Kianpour, and Stewart Kowalski. An empir- ical study of security culture in open source software communities. In Proceedings of the 2019 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining , pages 863–870, 2019

    2019

  43. [51]

    Agile devel- opment: Mainstream adoption has changed agility

    Dave West, Tom Grant, Mary Gerush, and David D’Silva. Agile devel- opment: Mainstream adoption has changed agility. Forrester Research, 2(1):41, 2010

    2010

  44. [52]

    A survey of human- machine collaboration in fuzzing

    Qian Yan, Minhuan Huang, and Huayang Cao. A survey of human- machine collaboration in fuzzing. In 2022 7th IEEE International Con- ference on Data Science in Cyberspace (DSC) , pages 375–382. IEEE, 2022

    2022

  45. [53]

    The Fuzzing Book

    Andreas Zeller, Rahul Gopinath, Marcel Böhme, Gordon Fraser, and Christian Holler. The Fuzzing Book. CISPA Helmholtz Center for Infor- mation Security, 2024. URL https://www.fuzzingbook.org/. Retrieved 2024-07-01 16:50:18+02:00

    2024

  46. [54]

    In 30th USENIX Security Symposium (USENIX Security 21) , pages 2811–2828, 2021

    Cen Zhang, Xingwei Lin, Yuekang Li, Yinxing Xue, Jundong Xie, Hongxu Chen, Xinlei Ying, Jiashui Wang, and Yang Liu.{APICraft}: Fuzz driver generation for closed-source {SDK} libraries. In 30th USENIX Security Symposium (USENIX Security 21) , pages 2811–2828, 2021

    2021

  47. [55]

    Automata-guided control-flow-sensitive fuzz driver generation

    Cen Zhang, Yuekang Li, Hao Zhou, Xiaohan Zhang, Yaowen Zheng, Xian Zhan, Xiaofei Xie, Xiapu Luo, Xinghua Li, Yang Liu, et al. Automata-guided control-flow-sensitive fuzz driver generation. In USENIX Security Symposium, pages 2867–2884, 2023

    2023

  48. [56]

    Intelligen: Automatic driver synthesis for fuzz testing

    Mingrui Zhang, Jianzhong Liu, Fuchen Ma, Huafeng Zhang, and Yu Jiang. Intelligen: Automatic driver synthesis for fuzz testing. In 2021 IEEE/ACM 43rd International Conference on Software Engineer- ing: Software Engineering in Practice (ICSE-SEIP) , pages 318–327. IEEE, 2021

    2021

  49. [57]

    Daisy: Effective fuzz driver synthesis with object usage sequence analysis

    Mingrui Zhang, Chijin Zhou, Jianzhong Liu, Mingzhe Wang, Jie Liang, Juan Zhu, and Yu Jiang. Daisy: Effective fuzz driver synthesis with object usage sequence analysis. In 2023 IEEE/ACM 45th International Conference on Software Engineering: Software Engineering in Practice (ICS...

    2023

  50. [58]

    Regression greybox fuzzing

    Xiaogang Zhu and Marcel Böhme. Regression greybox fuzzing. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, pages 2169–2182, 2021. Philipp Görz, Joschua Schilling, Thorsten Holz, and Marcel Böhme A Supplementary Materials A.1 Coverage S...

    2021

  51. [2024]

    URL https://llvm.org/docs/LibFuzzer.html

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.