Pith. sign in

REVIEW 4 major objections 7 minor 1 cited by

BandFuzz: An ML-powered Collaborative Fuzzing Framework

T0 review · 4 major / 7 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A Thompson-sampling bandit that treats each fuzzer as an arm lets one CPU core match or beat a team of fuzzers.

desk verdict A solid system paper with a real bandit-based scheduling idea; the headline outperformance over AFL++ is inside its own noise threshold, but the autofz comparison and ablation make it worth a serious referee. read the letter →

arxiv 2507.10845 v2 pith:H6SN6SOI submitted 2025-07-14 cs.CR cs.SE

Wenxuan Shi , Hongwei Li , Jiahao Yu , Xinqian Sun , Wenbo Guo , Xinyu Xing This is my paper · ORCID
classification cs.CRcs.SE
keywords collaborativefuzzingmulti-armedbanditsThompsonsamplingresourceallocationfuzzerevaluationcoverageintervalseedsynchronizationmutationscore
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to establish that collaborative fuzzing — running several fuzzers as a team — can beat both individual fuzzers and prior ensembles without consuming extra CPU cores. The mechanism is a multi-armed bandit: each fuzzer is an arm, and a Thompson-sampling rule decides, round by round, which fuzzer gets to run on the single core. The reward that drives learning is a new difficulty-aware coverage metric, the coverage interval $c = t - t_p$ for each newly covered branch. The paper argues that this long-horizon, exploration-aware allocation is globally better than the greedy allocation used by autofz, and reports that BandFuzz beats ten individual fuzzers on FuzzBench, autofz-8 by 16.0% average branch coverage on FTS, and all competitors in the SBFT 2024 mutation-score ranking. A sympathetic reader would care because fuzzing progress has stagnated on standard benchmarks, and this is a concrete proposal for getting ensemble robustness from a single core.

What carries the argument

The load-bearing object is the coverage interval $c = t - t_p$: the number of fuzzing rounds between the discovery of a branch's predecessor basic block and the discovery of the branch itself, used as a proxy for branch difficulty and as the per-branch reward. Around it, the machinery is a customized Thompson-sampling bandit in which each fuzzer is an arm with a $\mathrm{Beta}(\alpha_F, \beta_F)$ weight distribution; each round the algorithm samples a success probability per arm, runs the fuzzer with the highest sample, converts the normalized coverage-interval reward into a Bernoulli observation, updates that arm's Beta parameters, and periodically resets all parameters so the bandit can track non-stationary reward distributions. The auto-cycle mechanism and real-time global seed synchronization keep the reward comparisons fair by giving each selected fuzzer roughly equal wall-clock time and by preventing fuzzers from re-exploring branches already in the global pool.

What would settle it

Run a controlled comparison in which the same target, same fuzzers, and same seed pool are scheduled by BandFuzz but the coverage-interval reward in Algorithm 1 is computed with a scheduler-independent round counter (for example, actual execution time or cycles spent since the predecessor was found) instead of the bandit's round number; if total branch coverage or the chosen fuzzer sequence changes materially, the reward is not measuring branch difficulty alone and the global-optimality claim is scheduling-dependent. A simpler check: count rounds where a selected fuzzer produces no new branches — those skipped rounds inflate $c$ for any later discovery — and ask whether an easy branch discovered after a long drought earns the same inflated reward as a genuinely hard branch.

Watch

Extended reading notes

Core claim

On its own terms, the paper's claim is that a single-core collaborative fuzzer can learn, online, which fuzzer deserves the next slice of time, and that this learned schedule outperforms both fixed single-fuzzer runs and the greedy exploration-exploitation schedule of autofz. BandFuzz integrates ten fuzzers, maintains a global seed pool synchronized in real time with the selected fuzzer, and evaluates the selected fuzzer not by raw coverage but by the summed coverage intervals $c = t - t_p$ of the new branches its seeds cover, where $t_p$ is the round in which the branch's predecessor basic block was first discovered. These rewards are normalized, discretized into Bernoulli samples, and fed to Thompson sampling with periodic parameter resets; an auto-cycle mechanism converts a fixed time budget into a fair number of fuzzing cycles per fuzzer. The reported evidence: highest median and average scores among all integrated fuzzers on 21 FuzzBench targets; 16.0% higher average branch coverage than autofz-8 over 25 FTS targets and 4.8% over full autofz; and an average mutation score of 98, first place, at SBFT 2024.

Load-bearing premise

The load-bearing premise is that the number of rounds between the first discovery of a branch's predecessor block and the discovery of the branch itself measures how difficult and how valuable that branch is, independently of the bandit's own scheduling decisions.

Editorial extensions

If this is right

  • Collaborative fuzzing no longer needs parallel cores: the same single CPU that would run one fuzzer can run a team, so ensemble robustness is available on resource-constrained systems.
  • Greedy resource allocation is not enough: the reported 16.0% average coverage gain over autofz-8 is attributed to the bandit's ability to favor fuzzers whose payoff is delayed.
  • The framework can be extended to new fuzzers with only minor modifications to their underlying harnesses, since the bandit treats each fuzzer as an opaque arm.
  • A fuzzer that crashes or stalls does not derail the campaign: management mechanisms skip or restart it, and the bandit shifts weight to working fuzzers.
  • Difficulty-aware evaluation can be reused independent of the bandit, as a real-time seed-quality or fuzzer-quality score.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the coverage-interval reward is confounded by the scheduler — because $t_p$ was set by whichever fuzzer discovered the predecessor and skipped rounds inflate $c$ even for easy branches — then the bandit's updates may partly reward scheduling artifacts rather than fuzzer skill; a testable extension would recompute rewards under a scheduler-independent round counter and compare coverage.
  • The same reward signal could be lifted out of the collaborative setting and used inside a single fuzzer to prioritize mutation effort toward inputs that unlock branches whose predecessors have been known for many rounds.
  • Adversarial bandit algorithms designed for non-stationary rewards, such as the EXP3 variant the paper names as future work, are a natural next comparison since the reset mechanism is a heuristic for non-stationarity rather than a formal treatment.
  • If branch difficulty is the real quantity being measured, the coverage interval could be replaced by or combined with structural difficulty measures such as basic-block centrality or condition complexity; the paper lists these as future seed-quality metrics.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper introduces BandFuzz, a collaborative fuzzing framework that selects one of ten individual fuzzers to run on a single CPU core at each round, using a Thompson-sampling multi-armed bandit to allocate resources. A global seed pool synchronizes discoveries among fuzzers, and a novel reward signal—based on the round interval between a branch's discovery and its predecessor block's first discovery—is used to update each fuzzer's Beta distribution. The paper evaluates BandFuzz on 21 FuzzBench targets and 25 Fuzzer Test Suite targets with 10 runs of 24 hours each, compares it to individual fuzzers and autofz, performs ablations for seed synchronization, reward design, and bandit allocation, tests hyperparameter sensitivity, and reports a first-place finish in the SBFT 2024 fuzzing competition with an average mutation score of 98.

Significance. If the empirical claims are statistically sound, BandFuzz would be a valuable contribution: it is a collaborative fuzzer that requires no additional CPU resources over a single fuzzer, reports strong results on public benchmarks, and has a competition victory as external validation. The paper's experimental scale is substantial (approximately 7.5 CPU-years of computation), it uses public benchmarks (FuzzBench, FTS), each experiment is repeated 10 times, and it includes a systematic ablation study and hyperparameter sensitivity analysis. The proposed coverage-interval reward is a creative departure from simple coverage or uniqueness metrics. However, the headline FuzzBench advantage over AFL++ is within the 1% threshold that the paper's own benchmark note says should be treated as a random effect, and no significance tests are provided for the central comparisons. In addition, the reward definition in Algorithm 1 is entangled with the bandit's own scheduling history, which threatens the validity of the learning signal and the claimed global optimality. These issues are load-bearing and require revision before the central claims can be accepted.

major comments (4)
  1. [§5.3, Table 1] The paper's own benchmark note states that any difference less than 1% is considered a random effect and disregarded. The average FuzzBench score for BandFuzz is 95.26 versus 94.46 for AFL++, a difference of 0.80 percentage points, which is below that threshold. No paired significance test, confidence interval, or effect size is reported for the per-target scores. The claim that BandFuzz 'outperforms widely used individual fuzzers' is therefore not statistically established by the main FuzzBench experiment. Please report a paired test (e.g., Wilcoxon signed-rank on per-target scores) or a bootstrap confidence interval on the average score, and explicitly address how the result stands relative to the 1% random-effect threshold.
  2. [§5.3, Table 2] The reported 16.0% average coverage enhancement of BandFuzz-8 over autofz-8 is not accompanied by any significance testing. Several rows show very large standard deviations (e.g., proj4-2017-08-14: autofz-8 1015.1±595.9; sqlite-2016-11-14: BandFuzz-8 1981.0±112.8), so the average enhancement may be driven by a few targets. Please provide a paired test across the 25 targets, per-target confidence intervals, and clarify whether the 16.0% is the mean of per-target relative improvements or a mean of absolute coverage differences. Without this, the superiority of BandFuzz's collaboration strategy over the greedy autofz baseline is not statistically supported.
  3. [§3.2, Algorithm 1 (lines 4-6)] The reward for a new branch is computed as c = t − t_p, where t_p is the round in which the predecessor basic block was first discovered. This t_p is determined by which fuzzer happened to discover that predecessor and by the bandit's own scheduling; when rounds are skipped or seeds are synchronized from the global pool, a fuzzer can traverse pending branches and receive a large c without having worked harder than a fuzzer that follows immediately after a discovery. The reward is therefore confounded with the scheduler's history, which corrupts the bandit's updates and undermines the claim that the algorithm learns 'globally optimal' strategies. Please either replace the reward with a scheduler-independent measure (e.g., actual CPU time or number of fuzzer cycles since the predecessor was discovered) or provide an analysis demonstrating that the bias does not affect the learning outcome, along with a sensitivity test using an alternative reward definition.
  4. [Abstract, §1, §3.3, §9] The paper repeatedly claims that BandFuzz 'enables discovery of globally optimal collaborative strategies', but no theoretical guarantee is provided. The customized Thompson sampling uses a Bernoulli discretization of a continuous reward, periodic parameter resets, and an auto-cycle mechanism; none of these is covered by standard Thompson sampling regret bounds. The term 'globally optimal' is not supported by any theorem or convergence result. Please either supply a formal optimality or regret analysis under a clearly stated non-stationary reward model, or soften the claim to describe the method as an effective heuristic strategy with empirical support.
minor comments (7)
  1. [§3.2, worked example] The arithmetic in the worked reward example is inconsistent: the text first states that branch b3 has coverage interval c = IZ − IX, but the final reward sum uses (IZ − IY) for that branch. The total should be 2(IZ − IX) if both b3 and b8 use IZ − IX, or the text should justify why b3 uses IZ − IY. Please correct the example.
  2. [§3.3, Equation (1)] Equation (1) updates β with subscript Fk: 'β_Fk ← β_Fk + 1 − r_t'. This should be β_Ft, consistent with Algorithm 2 line 16. The current subscript makes the update ambiguous.
  3. [Figures 4 and 5 captions] Both captions say 'Comparison among BAND FUZZ and two variants', but Figure 4 shows four curves (BandFuzz, random, no-sync, naive-reward) and Figure 5 appears to show three. Please adjust the captions to match the number of configurations displayed.
  4. [§6, Table 3] The text states that the second-ranked fuzzer is TuneFuzz [66], but Table 3 lists FishFuzz and reference [66] is FishFuzz. Please align the name used in the text with the table and reference.
  5. [§5.2, Experiment I] The text says 'we run each method 10 times and report the mean', but FuzzBench scores are based on median branch coverage relative to the maximum per target. Please clarify which per-target statistic is used (median over runs) and how the average score is computed.
  6. [§5.3, Experiment V] The Nemenyi post-hoc test is mentioned, but no p-values or critical difference values are reported in the text; only a CD diagram is referenced. Please report the actual statistical results or specify where they can be found, so that 'marginal changes' is supported by the data.
  7. [General] The paper does not state an artifact availability plan or provide a link to source code and benchmark configuration. For a systems paper with such large experiments, releasing the code and exact build scripts would substantially aid reproducibility and is recommended.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: BandFuzz's resource-allocation reward is a design input, not a fitted/predicted quantity, and the central claims are evaluated against external benchmarks and an independent competition.

full rationale

I walked the paper's derivation chain from the proposed fuzzer-evaluation reward (Algorithm 1), through the Thompson-sampling bandit allocation (Algorithms 2 and 3), to the experimental comparisons. The reward r_t = sum of coverage intervals c = t - t_p is a heuristic definition of branch difficulty and fuzzer contribution; it is not a parameter fitted to the final coverage metric and then re-predicted. The bandit's Beta updates use this reward as an input, and the paper's success claims are measured by external metrics: FuzzBench average score, FTS branch coverage against autofz, and SBFT 2024 mutation scores. None of these outputs is equivalent by construction to the reward formula or to a self-citation. The cited prior results, including Thompson sampling [57] and the discretization method [61], are external and machine-checkable/standard; the only self-citation, GPTFuzzer [74], appears in related work and is not load-bearing. The reader-identified concern that c = t - t_p is influenced by the bandit's own scheduling is a correctness/threat-to-validity issue about reward quality, not a circularity: the reward could be biased or suboptimal, but the paper does not claim that reward itself as an externally validated prediction. Similarly, the statistical weakness of the FuzzBench comparison (0.80-point gap below the paper's own 1% random-effect threshold) is an evidence-quality concern, not a circular-derivation concern. The paper's 'globally optimal' language is over-claimed relative to the provided evidence, but no load-bearing step reduces to its own input by definition. Accordingly, the appropriate circularity score is 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The claimed result is an engineering system, not a first-principles derivation. The bandit's behavior is governed by three hand-set parameters (time budget, reset interval, Beta priors), none fitted to a target prediction. The axioms are mostly domain assumptions about fuzzer reward structure; the paper's own ablation tests support some (sync, evaluation, allocation) but do not isolate the reward-feedback bias. No new physical entities are introduced.

free parameters (3)
  • time budget per round T_I = 120 seconds
    Controls how long a selected fuzzer runs per round. Authors chose the default and test 90-180s sensitivity (Experiment V), showing insensitivity.
  • reset interval I_R = 120 minutes
    Periodically resets all Beta parameters to handle non-stationary rewards. Default chosen; sensitivity tested from 90-180 minutes.
  • initial Beta parameters alpha_F, beta_F = 1, 1
    Uniform prior for Thompson sampling; standard choice, not fitted to data.
assumptions (4)
  • standard math Thompson sampling achieves near-optimal regret in stationary stochastic bandits
    Invoked to justify 'globally optimal' resource allocation, but the paper's own reward distribution is non-stationary (hence the reset mechanism), so the cited guarantees (Refs 56,60) do not directly apply.
  • domain assumption Coverage interval c=t-t_p measures branch difficulty and fuzzer long-term value
    Algorithm 1 lines 4-6 define reward as time since predecessor block discovery; this conflates intrinsic difficulty with the bandit's own selection schedule, an unanalyzed feedback.
  • domain assumption Reward distributions are approximately stationary within each reset interval I_R
    The reset mechanism (Algorithm 2 lines 21-24) assumes performance changes are slow enough that 120-minute windows are meaningful; no analysis of window size versus fuzzer dynamics.
  • ad hoc to paper Bernoulli discretization of a continuous reward preserves enough information for Thompson sampling
    Algorithm 3 samples a Bernoulli with probability r_t; this throws away magnitude information, and the paper does not analyze the bias this introduces.

how reviews work

0 comments
Cite this review

Pith. "Pith review of BandFuzz: An ML-powered Collaborative Fuzzing Framework." pith.science (2026). https://pith.science/paper/H6SN6SOI

@misc{pith2026250710845,
  author       = {Pith},
  title        = {Pith review of: BandFuzz: An ML-powered Collaborative Fuzzing Framework},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/H6SN6SOI}},
  note         = {Machine review of arXiv:2507.10845}
}
read the original abstract

Collaborative fuzzing combines multiple individual fuzzers and dynamically chooses appropriate combinations for different programs. Unlike individual fuzzers that rely on specific assumptions, collaborative fuzzing relaxes assumptions on target programs, providing robust performance across various programs. However, existing collaborative fuzzing frameworks face challenges including additional computational resource requirements and inefficient resource allocation among fuzzers. To tackle these challenges, we present BANDFUZZ, an ML-powered collaborative fuzzing framework that outperforms individual fuzzers without requiring additional computational resources. The key contribution of BANDFUZZ lies in its novel resource allocation algorithm driven by our proposed multi-armed bandits model. Different from greedy methods in existing frameworks, BANDFUZZ models the long-term impact of individual fuzzers, enabling discovery of globally optimal collaborative strategies. We propose a novel fuzzer evaluation method that assesses not only code coverage but also the fuzzer's capability of solving difficult branches. Finally, we integrate a real-time seed synchronization mechanism and implementation-wise optimizations to improve fuzzing efficiency and stability. Through extensive experiments on Fuzzbench and Fuzzer Test Suite, we show that BANDFUZZ outperforms state-of-the-art collaborative fuzzing framework autofz and widely used individual fuzzers. We verify BANDFUZZ's key designs through comprehensive ablation study. Notably, we demonstrate BANDFUZZ's effectiveness in real-world bug detection by analyzing results of a worldwide fuzzing competition, where BANDFUZZ won first place.

Figures

Figures reproduced from arXiv: 2507.10845 by the authors.

Figure 1
Figure 1. An example of two individual fuzzer’s code coverage on [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. The overview of BANDFUZZ. In the figure, BANDFUZZ manages four fuzzers. In the current round t, the fuzzer F2 is chosen. Initially, F2’s seed pool contains seeds S1 ∼ S3, while the global seed pool contains S1 ∼ S6. After seed synchronization, F2 extends its seed pool with the missing seeds and then proceeds to execute the fuzzing task, discovering new seeds S7 and S8. The new seeds are added to the global seed pool… view at source ↗
Figure 3
Figure 3. Motivating example for demonstrating reward computing. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Comparison among BANDFUZZ and two variants. Each line plot represents the arithmetic mean along with a 95% con￾fidence interval, derived from 10 independent trials. The label random refers to BANDFUZZ that employs random resource allo￾cation in each round, and no-sync …
Figure 5
Figure 5. Figure 5: Comparison among BANDFUZZ and two variants. Each line plot represents the arithmetic mean along with a 95% confi￾dence interval, derived from 10 independent trials. The label no￾reset refers to BANDFUZZ without reset mechanism, and naive￾reward represents BANDFUZZ adop…
Figure 6
Figure 6. Figure 6: Critical Difference (CD) among various configurations [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: Bar graph representation of fuzzers achieving the highest mathematical expectation for reward distribution over consecutive [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]
Figure 8
Figure 8. Figure 8: Evaluation of BANDFUZZ and individual fuzzers on FuzzBench: Each line plot in the graph represents the arithmetic mean along with a 95% confidence interval, derived from 10 independent trials. “Branch coverage” denotes the count of branches that each fuzzer has explore…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. SoK: DARPA's AI Cyber Challenge (AIxCC): Competition Design, Architectures, and Lessons Learned

    cs.CR 2026-02 conditional novelty 7.0 of 10

    Reliability and accuracy, not novel techniques, decided DARPA's 2025 AI cyber contest; LLM-based reasoning found 22 vulnerabilities that dedicated fuzzing missed, while auto-generated patches still fail semantic corre...

Reference graph

Works this paper leans on

96 extracted references · 79 canonical work pages · cited by 1 Pith paper

  1. [1]

    Neuzz: Efficient fuzzing with neural program smoothing,

    D. She, K. Pei, D. Epstein, J. Yang, B. Ray, and S. Jana, “Neuzz: Efficient fuzzing with neural program smoothing,” in Proc. of S&P , 2019

    2019

  2. [2]

    Mtfuzz: fuzzing with a multi-task neural network,

    D. She, R. Krishna, L. Yan, S. Jana, and B. Ray, “Mtfuzz: fuzzing with a multi-task neural network,” in Proc. of ESEC/FSE , 2020

    2020

  3. [3]

    Cerebro: context-aware adaptive fuzzing for effective vulnerability detection,

    Y . Li, Y . Xue, H. Chen, X. Wu, C. Zhang, X. Xie, H. Wang, and Y . Liu, “Cerebro: context-aware adaptive fuzzing for effective vulnerability detection,” in Proc. of ESEC/FSE , 2019

    2019

  4. [4]

    Not all coverage measurements are equal: Fuzzing by coverage accounting for input prioritization

    Y . Wang, X. Jia, Y . Liu, K. Zeng, T. Bao, D. Wu, and P. Su, “Not all coverage measurements are equal: Fuzzing by coverage accounting for input prioritization.” in Proc. of NDSS , 2020

    2020

  5. [5]

    Alphuzz: Monte carlo search on seed-mutation tree for coverage-guided fuzzing,

    Y . Zhao, X. Wang, L. Zhao, Y . Cheng, and H. Yin, “Alphuzz: Monte carlo search on seed-mutation tree for coverage-guided fuzzing,” in Proc. of ACSAC, 2022

    2022

  6. [6]

    Reinforcement learning-based hierar- chical seed scheduling for greybox fuzzing,

    J. Wang, C. Song, and H. Yin, “Reinforcement learning-based hierar- chical seed scheduling for greybox fuzzing,” Journal of Systems and Software, 2021

    2021

  7. [7]

    Optimizing seed selection for fuzzing,

    A. Rebert, S. K. Cha, T. Avgerinos, J. Foote, D. Warren, G. Grieco, and D. Brumley, “Optimizing seed selection for fuzzing,” in Proc. of USENIX Security, 2014

    2014

  8. [8]

    Deepfuzzer: Accelerated deep greybox fuzzing,

    J. Liang, Y . Jiang, M. Wang, X. Jiao, Y . Chen, H. Song, and K.- K. R. Choo, “Deepfuzzer: Accelerated deep greybox fuzzing,” IEEE Transactions on Dependable and Secure Computing , 2019

    2019

Show all 96 references
  1. [9]

    Seed selection for successful fuzzing,

    A. Herrera, H. Gunadi, S. Magrath, M. Norrish, M. Payer, and A. L. Hosking, “Seed selection for successful fuzzing,” in Proc. of ISSTA, 2021

    2021

  2. [10]

    Collafl: Path sensitive fuzzing,

    S. Gan, C. Zhang, X. Qin, X. Tu, K. Li, Z. Pei, and Z. Chen, “Collafl: Path sensitive fuzzing,” in Proc. of S&P , 2018

    2018

  3. [11]

    Better pay attention whilst fuzzing,

    S. Zhu, J. Wang, J. Sun, J. Yang, X. Lin, T. Wang, L. Zhang, and P. Cheng, “Better pay attention whilst fuzzing,” IEEE Transactions on Software Engineering , 2023

    2023

  4. [12]

    Coverage-based greybox fuzzing as markov chain,

    M. B ¨ohme, V .-T. Pham, and A. Roychoudhury, “Coverage-based greybox fuzzing as markov chain,” in Proc. of CCS , 2016

    2016

  5. [13]

    Ecofuzz: Adaptive energy-saving greybox fuzzing as a variant of the adversarial multi-armed bandit,

    T. Yue, P. Wang, Y . Tang, E. Wang, B. Yu, K. Lu, and X. Zhou, “Ecofuzz: Adaptive energy-saving greybox fuzzing as a variant of the adversarial multi-armed bandit,” in Proc. of USENIX Security , 2020

    2020

  6. [14]

    Vuzzer: Application-aware evolutionary fuzzing

    S. Rawat, V . Jain, A. Kumar, L. Cojocar, C. Giuffrida, and H. Bos, “Vuzzer: Application-aware evolutionary fuzzing.” in Proc. of NDSS, 2017

    2017

  7. [15]

    Fairfuzz: A targeted mutation strategy for increasing greybox fuzz testing coverage,

    C. Lemieux and K. Sen, “Fairfuzz: A targeted mutation strategy for increasing greybox fuzz testing coverage,” in Proc. of ASE , 2018

    2018

  8. [16]

    Darwin: Survival of the fittest fuzzing mutators,

    P. Jauernig, D. Jakobovic, S. Picek, E. Stapf, and A.-R. Sadeghi, “Darwin: Survival of the fittest fuzzing mutators,” in Proc. of NDSS, 2023

    2023

  9. [17]

    Mopt: Optimized mutation scheduling for fuzzers,

    C. Lyu, S. Ji, C. Zhang, Y . Li, W.-H. Lee, and Y . Song, “Mopt: Optimized mutation scheduling for fuzzers,” in Proc. of USENIX Security, 2022

    2022

  10. [18]

    Redqueen: Fuzzing with input-to-state correspondence,

    C. Aschermann, S. Schumilo, T. Blazytko, R. Gawlik, and T. Holz, “Redqueen: Fuzzing with input-to-state correspondence,” in Proc. of NDSS, 2019

    2019

  11. [19]

    Aki helin / radamsa · gitlab,

    “Aki helin / radamsa · gitlab,” https://gitlab.com/akihe/radamsa, 2023

    2023

  12. [20]

    Profuzzer: On-the-fly input type probing for better zero- day vulnerability discovery,

    W. You, X. Wang, S. Ma, J. Huang, X. Zhang, X. Wang, and B. Liang, “Profuzzer: On-the-fly input type probing for better zero- day vulnerability discovery,” in Proc. of S&P , 2019

    2019

  13. [21]

    Cmfuzz: context- aware adaptive mutation for fuzzers,

    X. Wang, C. Hu, R. Ma, D. Tian, and J. He, “Cmfuzz: context- aware adaptive mutation for fuzzers,” Empirical Software Engineer- ing, 2021

    2021

  14. [22]

    Learning seed-adaptive mutation strate- gies for greybox fuzzing,

    M. Lee, S. Cha, and H. Oh, “Learning seed-adaptive mutation strate- gies for greybox fuzzing,” in Proc. of ICSE , 2023. 14

    2023

  15. [23]

    Sok: Prudent evaluation practices for fuzzing,

    M. Schloegel, N. Bars, N. Schiller, L. Bernhard, T. Scharnowski, A. Crump, A. Ale-Ebrahim, N. Bissantz, M. Muench, and T. Holz, “Sok: Prudent evaluation practices for fuzzing,” in Proc. of S&P , 2024

    2024

  16. [24]

    Fuzzbench: 2023-12-15-aflpp report,

    “Fuzzbench: 2023-12-15-aflpp report,” https://fuzzbench.com/reports/ experimental/2023-12-15-aflpp/index.html, 2023

    2023

  17. [25]

    Sbft tool competition 2023–fuzzing track,

    D. Liu, J. Metzman, M. B ¨ohme, O. Chang, and A. Arya, “Sbft tool competition 2023–fuzzing track,” ArXiv Preprint, 2023

    2023

  18. [26]

    Afl++: Combining incremental steps of fuzzing research,

    A. Fioraldi, D. Maier, H. Eißfeldt, and M. Heuse, “Afl++: Combining incremental steps of fuzzing research,” in Proc. of WOOT, 2020

    2020

  19. [27]

    Fuzzbench reports,

    “Fuzzbench reports,” https://fuzzbench.com/reports/experimental/, 2024

    2024

  20. [28]

    Sbft 2023 fuzzing competition,

    “Sbft 2023 fuzzing competition,” https://sbft23.github.io/tools/ fuzzing, 2023

    2023

  21. [29]

    Collabfuzz: A framework for collaborative fuzzing,

    S. ¨Osterlund, E. Geretto, A. Jemmett, E. G ¨uler, P. G ¨orz, T. Holz, C. Giuffrida, and H. Bos, “Collabfuzz: A framework for collaborative fuzzing,” in Proc. of EuroSec, 2021

    2021

  22. [30]

    Cupid: Automatic fuzzer selection for collaborative fuzzing,

    E. G ¨uler, P. G ¨orz, E. Geretto, A. Jemmett, S. ¨Osterlund, H. Bos, C. Giuffrida, and T. Holz, “Cupid: Automatic fuzzer selection for collaborative fuzzing,” in Proc. of ACSAC, 2020

    2020

  23. [31]

    Enfuzz: Ensemble fuzzing with seed synchronization among diverse fuzzers,

    Y . Chen, Y . Jiang, F. Ma, J. Liang, M. Wang, C. Zhou, X. Jiao, and Z. Su, “Enfuzz: Ensemble fuzzing with seed synchronization among diverse fuzzers,” in Proc. of USENIX Security , 2019

    2019

  24. [32]

    autofz: Automated fuzzer composition at runtime,

    Y .-F. Fu, J. Lee, and T. Kim, “autofz: Automated fuzzer composition at runtime,” in Proc. of USENIX Security , 2023

    2023

  25. [33]

    Fuzzbench: an open fuzzer benchmarking platform and service,

    J. Metzman, L. Szekeres, L. Simon, R. Sprabery, and A. Arya, “Fuzzbench: an open fuzzer benchmarking platform and service,” in Proc. of ESEC/FSE , 2021

    2021

  26. [34]

    Fuzzer test suite (fts),

    Google, “Fuzzer test suite (fts),” https://opensource.google/projects/ fuzzer-test-suite, 2021

    2021

  27. [35]

    American fuzzy lop,

    M. Zalewski, “American fuzzy lop,” http://lcamtuf.coredump.cx/afl/, 2013

    2013

  28. [36]

    Angora: Efficient fuzzing by principled search,

    P. Chen and H. Chen, “Angora: Efficient fuzzing by principled search,” in Proc. of S&P , 2018

    2018

  29. [37]

    Send hardest problems my way: Probabilistic path prioritization for hybrid fuzzing

    L. Zhao, Y . Duan, H. Yin, and J. Xuan, “Send hardest problems my way: Probabilistic path prioritization for hybrid fuzzing.” in Proc. of NDSS, 2019

    2019

  30. [38]

    Smart greybox fuzzing,

    V .-T. Pham, M. B¨ohme, A. E. Santosa, A. R. C ˘aciulescu, and A. Roy- choudhury, “Smart greybox fuzzing,” IEEE Transactions on Software Engineering, 2019

    2019

  31. [39]

    Path tran- sitions tell more: Optimizing fuzzing schedules via runtime program states,

    K. Zhang, X. Xiao, X. Zhu, R. Sun, M. Xue, and S. Wen, “Path tran- sitions tell more: Optimizing fuzzing schedules via runtime program states,” in Proc. of ICSE , 2022

    2022

  32. [40]

    A novel coverage-guided greybox fuzzing based on power schedule optimization with time complexity,

    J. Chen, S. Wang, S. Cai, C. Zhang, H. Chen, J. Chen, and J. Zhang, “A novel coverage-guided greybox fuzzing based on power schedule optimization with time complexity,” in Proc. of ASE , 2022

    2022

  33. [41]

    Driller: Augmenting fuzzing through selective symbolic execution

    N. Stephens, J. Grosen, C. Salls, A. Dutcher, R. Wang, J. Corbetta, Y . Shoshitaishvili, C. Kruegel, and G. Vigna, “Driller: Augmenting fuzzing through selective symbolic execution.” in Proc. of NDSS , 2016

    2016

  34. [42]

    Symsan: Time and space efficient concolic execution via dynamic data-flow analysis,

    J. Chen, W. Han, M. Yin, H. Zeng, C. Song, B. Lee, H. Yin, and I. Shin, “Symsan: Time and space efficient concolic execution via dynamic data-flow analysis,” in Proc. of USENIX Security , 2022

    2022

  35. [43]

    Symbolic execution with symcc: Don’t interpret, compile!

    S. Poeplau and A. Francillon, “Symbolic execution with symcc: Don’t interpret, compile!” in Proc. of USENIX Security , 2020

    2020

  36. [44]

    Qsym: A practical concolic execution engine tailored for hybrid fuzzing,

    I. Yun, S. Lee, M. Xu, Y . Jang, and T. Kim, “Qsym: A practical concolic execution engine tailored for hybrid fuzzing,” in Proc. of USENIX Security, 2018

    2018

  37. [45]

    Symqemu: Compilation-based sym- bolic execution for binaries,

    S. Poeplau and A. Francillon, “Symqemu: Compilation-based sym- bolic execution for binaries,” in Proc. of NDSS , 2021

    2021

  38. [46]

    Pangolin: Incre- mental hybrid fuzzing with polyhedral path abstraction,

    H. Huang, P. Yao, R. Wu, Q. Shi, and C. Zhang, “Pangolin: Incre- mental hybrid fuzzing with polyhedral path abstraction,” in Proc. of S&P, 2020

    2020

  39. [47]

    Fuzzolic: mixing fuzzing and concolic execution,

    L. Borzacchiello, E. Coppa, and C. Demetrescu, “Fuzzolic: mixing fuzzing and concolic execution,” Computers & Security , 2021

    2021

  40. [48]

    Drifuzz: Harvesting bugs in device drivers from golden seeds,

    Z. Shen, R. Roongta, and B. Dolan-Gavitt, “Drifuzz: Harvesting bugs in device drivers from golden seeds,” in Proc. of USENIX Security , 2022

    2022

  41. [49]

    Pata: Fuzzing with path aware taint analysis,

    J. Liang, M. Wang, C. Zhou, Z. Wu, Y . Jiang, J. Liu, Z. Liu, and J. Sun, “Pata: Fuzzing with path aware taint analysis,” in Proc. of S&P, 2022

    2022

  42. [50]

    Greyone: Data flow sensitive fuzzing,

    S. Gan, C. Zhang, P. Chen, B. Zhao, X. Qin, D. Wu, and Z. Chen, “Greyone: Data flow sensitive fuzzing,” in Proc. of USENIX Security, 2020

    2020

  43. [51]

    Hfl: Hybrid fuzzing on the linux kernel

    K. Kim, D. R. Jeong, C. H. Kim, Y . Jang, I. Shin, and B. Lee, “Hfl: Hybrid fuzzing on the linux kernel.” in Proc. of NDSS , 2020

    2020

  44. [52]

    Di- rected greybox fuzzing,

    M. B ¨ohme, V .-T. Pham, M.-D. Nguyen, and A. Roychoudhury, “Di- rected greybox fuzzing,” in Proc. of CCS , 2017

    2017

  45. [53]

    Perffuzz: Automati- cally generating pathological inputs,

    C. Lemieux, R. Padhye, K. Sen, and D. Song, “Perffuzz: Automati- cally generating pathological inputs,” in Proc. of ISSTA, 2018

    2018

  46. [54]

    Maxafl: Maximizing code coverage with a gradient-based optimization technique,

    Y . Kim and J. Yoon, “Maxafl: Maximizing code coverage with a gradient-based optimization technique,” Electronics, 2020

    2020

  47. [55]

    No free lunch theorems for op- timization,

    D. H. Wolpert and W. G. Macready, “No free lunch theorems for op- timization,” IEEE Transactions on Evolutionary Computation , 1997

    1997

  48. [56]

    A tutorial on thompson sampling,

    D. J. Russo, B. Van Roy, A. Kazerouni, I. Osband, Z. Wen et al., “A tutorial on thompson sampling,” Foundations and Trends in Machine Learning, 2018

    2018

  49. [57]

    On the likelihood that one unknown probability exceeds another in view of the evidence of two samples,

    W. R. Thompson, “On the likelihood that one unknown probability exceeds another in view of the evidence of two samples,” Biometrika, 1933

    1933

  50. [58]

    Matching in multi-arm bandit with collision,

    Y . Zhang, S. Wang, and Z. Fang, “Matching in multi-arm bandit with collision,” in Proc. of NeurIPS , 2022

    2022

  51. [59]

    Finite-time analysis of the multiarmed bandit problem,

    P. Auer, N. Cesa-Bianchi, and P. Fischer, “Finite-time analysis of the multiarmed bandit problem,” Machine Learning, 2002

    2002

  52. [60]

    An empirical evaluation of thompson sam- pling,

    O. Chapelle and L. Li, “An empirical evaluation of thompson sam- pling,” in Proc. of NeurIPS , 2011

    2011

  53. [61]

    Analysis of thompson sampling for the multi-armed bandit problem,

    S. Agrawal and N. Goyal, “Analysis of thompson sampling for the multi-armed bandit problem,” in Proc. of COLT , 2012

    2012

  54. [62]

    Honggfuzz: Security oriented software fuzzer,

    Google, “Honggfuzz: Security oriented software fuzzer,” https:// github.com/google/honggfuzz, 2020

    2020

  55. [63]

    Laf-intel,

    “Laf-intel,” https://lafintel.wordpress.com/, 2016

    2016

  56. [64]

    Slime: program-sensitive energy allocation for fuzzing,

    C. Lyu, H. Liang, S. Ji, X. Zhang, B. Zhao, M. Han, Y . Li, Z. Wang, W. Wang, and R. Beyah, “Slime: program-sensitive energy allocation for fuzzing,” in Proc. of ISSTA, 2022

    2022

  57. [65]

    Mobfuzz: Adaptive multi-objective optimization in gray-box fuzzing,

    G. Zhang, P. Wang, T. Yue, X. Kong, S. Huang, X. Zhou, and K. Lu, “Mobfuzz: Adaptive multi-objective optimization in gray-box fuzzing,” in Proc. of NDSS , 2022

    2022

  58. [66]

    Fishfuzz: Catch deeper bugs by throwing larger nets,

    H. Zheng, J. Zhang, Y . Huang, Z. Ren, H. Wang, C. Cao, Y . Zhang, F. Toffalini, and M. Payer, “Fishfuzz: Catch deeper bugs by throwing larger nets,” in Proc. of USENIX Security , 2023

    2023

  59. [67]

    Hollander, D

    M. Hollander, D. A. Wolfe, and E. Chicken, Nonparametric statistical methods. John Wiley & Sons, 2013

    2013

  60. [68]

    Sbft 2024 fuzzing competition,

    “Sbft 2024 fuzzing competition,” https://sbft24.github.io/tools/ fuzzing, 2024

    2024

  61. [69]

    Systematic assessment of fuzzers using mutation anal- ysis,

    P. G ¨orz, B. Mathis, K. Hassler, E. G ¨uler, T. Holz, A. Zeller, and R. Gopinath, “Systematic assessment of fuzzers using mutation anal- ysis,” in Proc. of USENIX Security , 2023

    2023

  62. [70]

    Libafl: A framework to build modular and reusable fuzzers,

    A. Fioraldi, D. C. Maier, D. Zhang, and D. Balzarotti, “Libafl: A framework to build modular and reusable fuzzers,” in Proc. of CCS , 2022. 15

    2022

  63. [71]

    libfuzzer – a library for coverage-guided fuzz testing,

    “libfuzzer – a library for coverage-guided fuzz testing,” https://llvm. org/docs/LibFuzzer.html, 2018

    2018

  64. [72]

    Pastis: A collaborative approach to combine heterogeneous software testing techniques,

    R. David, R. Abou Chaaya, and C. Heitman, “Pastis: A collaborative approach to combine heterogeneous software testing techniques,” in Proc. of SBFT , 2023

    2023

  65. [73]

    Slopt: Bandit optimization framework for mutation-based fuzzing,

    Y . Koike, H. Katsura, H. Yakura, and Y . Kurogome, “Slopt: Bandit optimization framework for mutation-based fuzzing,” in Proc. of ACSAC, 2022

    2022

  66. [74]

    Gptfuzzer: Red teaming large language models with auto-generated jailbreak prompts,

    J. Yu, X. Lin, and X. Xing, “Gptfuzzer: Red teaming large language models with auto-generated jailbreak prompts,” ArXiv Preprint, 2023

    2023

  67. [75]

    Large language models are zero-shot fuzzers: Fuzzing deep-learning libraries via large language models,

    Y . Deng, C. S. Xia, H. Peng, C. Yang, and L. Zhang, “Large language models are zero-shot fuzzers: Fuzzing deep-learning libraries via large language models,” in Proc. of ISSTA, 2023

    2023

  68. [76]

    Fuzz4all: Universal fuzzing with large language models,

    C. S. Xia, M. Paltenghi, J. Le Tian, M. Pradel, and L. Zhang, “Fuzz4all: Universal fuzzing with large language models,” ArXiv Preprint, 2024

    2024

  69. [77]

    Syzvegas: Beating kernel fuzzing odds with re- inforcement learning,

    D. Wang, Z. Zhang, H. Zhang, Z. Qian, S. V . Krishnamurthy, and N. Abu-Ghazaleh, “Syzvegas: Beating kernel fuzzing odds with re- inforcement learning,” in Proc. of USENIX Security , 2021

    2021

  70. [78]

    Fuzz- guard: Filtering out unreachable inputs in directed grey-box fuzzing through deep learning,

    P. Zong, T. Lv, D. Wang, Z. Deng, R. Liang, and K. Chen, “Fuzz- guard: Filtering out unreachable inputs in directed grey-box fuzzing through deep learning,” in Proc. of USENIX Security , 2020

    2020

  71. [79]

    Compiler fuzzing through deep learning,

    C. Cummins, P. Petoumenos, A. Murray, and H. Leather, “Compiler fuzzing through deep learning,” in Proc. of ISSTA, 2018

    2018

  72. [80]

    Evaluating and improving neural program- smoothing-based fuzzing,

    M. Wu, L. Jiang, J. Xiang, Y . Zhang, G. Yang, H. Ma, S. Nie, S. Wu, H. Cui, and L. Zhang, “Evaluating and improving neural program- smoothing-based fuzzing,” in Proc. of ICSE , 2022

    2022

  73. [81]

    Revisiting neural program smoothing for fuzzing,

    M.-I. Nicolae, M. Eisele, and A. Zeller, “Revisiting neural program smoothing for fuzzing,” in Proc. of ESEC/FSE , 2023

    2023

  74. [82]

    Deep reinforcement fuzzing,

    K. B ¨ottinger, P. Godefroid, and R. Singh, “Deep reinforcement fuzzing,” in Proc. of S&PW , 2018

    2018

  75. [83]

    Deepfuzz: Automatic generation of syntax valid c programs for fuzz testing,

    X. Liu, X. Li, R. Prajapati, and D. Wu, “Deepfuzz: Automatic generation of syntax valid c programs for fuzz testing,” in Proc. of AAAI, 2019

    2019

  76. [84]

    Rltrace: Synthesizing high- quality system call traces for os fuzz testing,

    W. Chen, H. Wang, W. Gu, and S. Wang, “Rltrace: Synthesizing high- quality system call traces for os fuzz testing,” in Proc. of InfoSec , 2023

    2023

  77. [85]

    Alphaprog: reinforcement generation of valid programs for compiler fuzzing,

    X. Li, X. Liu, L. Chen, R. Prajapati, and D. Wu, “Alphaprog: reinforcement generation of valid programs for compiler fuzzing,” in Proc. of AAAI , 2022

    2022

  78. [86]

    Evolutionary mutation-based fuzzing as monte carlo tree search,

    Y . Zhao, X. Wang, L. Zhao, Y . Cheng, and H. Yin, “Evolutionary mutation-based fuzzing as monte carlo tree search,” ArXiv Preprint, 2021

    2021

  79. [87]

    One fuzzing strategy to rule them all,

    M. Wu, L. Jiang, J. Xiang, Y . Huang, H. Cui, L. Zhang, and Y . Zhang, “One fuzzing strategy to rule them all,” in Proc. of ICSE , 2022

    2022

  80. [88]

    Banditfuzz: fuzzing smt solvers with multi-agent reinforcement learning,

    J. Scott, T. Sudula, H. Rehman, F. Mora, and V . Ganesh, “Banditfuzz: fuzzing smt solvers with multi-agent reinforcement learning,” in Proc. of ISFM, 2021

    2021

  81. [89]

    Adaptive grey-box fuzz- testing with thompson sampling,

    S. Karamcheti, G. Mann, and D. Rosenberg, “Adaptive grey-box fuzz- testing with thompson sampling,” in Proc. of AI & Security , 2018

    2018

  82. [90]

    Effective seed scheduling for fuzzing with graph centrality analysis,

    D. She, A. Shah, and S. Jana, “Effective seed scheduling for fuzzing with graph centrality analysis,” in Proc. of S&P , 2022

    2022

  83. [91]

    The non- stochastic multiarmed bandit problem,

    P. Auer, N. Cesa-Bianchi, Y . Freund, and R. E. Schapire, “The non- stochastic multiarmed bandit problem,” SIAM Journal on Computing, 2002

    2002

  84. [92]

    syzkaller,

    Google, “syzkaller,” https://github.com/google/syzkaller, 2016

    2016

  85. [93]

    A survey of compiler testing,

    J. Chen, J. Patra, M. Pradel, Y . Xiong, H. Zhang, D. Hao, and L. Zhang, “A survey of compiler testing,” Acm Computing Surveys (Csur), 2020

    2020

  86. [94]

    Dsmith: Compiler fuzzing through generative deep learning model with attention,

    H. Xu, Y . Wang, S. Fan, P. Xie, and A. Liu, “Dsmith: Compiler fuzzing through generative deep learning model with attention,” in Proc. of IJCNN , 2020. Appendix

    2020

  87. [95]

    Let’s assume that we use the method in §3.1 to sched- ule three fuzzers throughout the fuzzing campaign: AFL, AFLFast and AFL++

    Example of customized multi-arm bandit algo- rithm and resource allocation We present a comprehensive explanation of our cus- tomized multi-arm bandits, accompanied by a straightfor- ward example that demonstrates the shift in weight distri- bution. Let’s assume that we use th...

  88. [96]

    Branch coverage

    More FuzzBench Results In Figure 8, we show the branch coverage of B AND - FUZZ vs individual fuzzers across the time. The results are consistent with Table 1. 16 ++ Figure 7: Bar graph representation of fuzzers achieving the highest mathematical expectation for reward distrib...

    1956

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.