{"id":"218e1d95-24b6-45e9-a971-d5636c5455c2","arxiv_id":"2506.23063","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"HF-DGF's hybrid feedback scheduling reproduces 41 benchmark crashes faster than AFL, AFLGo, WindRanger, DAFL, and Beacon, with average speedups from 2.56x to 73.75x.","lead":"HF-DGF is a directed fuzzer that combines control-flow distance, value-flow influence, and slice coverage to guide inputs toward a target crash location. On a 41-vulnerability benchmark it reports reproducing crashes up to 73.75 times faster than five existing fuzzers, which could speed up crash triage and patch verification.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline crash-reproduction speedups are computed on a favorable subset with arithmetic-mean ratios; a reanalysis including N.A. rows and rank-based tests could overturn the average.","rationale":"I read the paper in good faith. The static-analysis precision issue identified by the reader is real and is conceded in §VII-B, but I do not think it is the single most load-bearing weakness because an acknowledged limitation in a subset of dispatch-heavy programs can coexist with a true average advantage. The more decisive issue is that the headline speedup itself is computed in a way that can manufacture a large average from the same Table II numbers: it averages only over cases where all tools have median TTEs, and it uses arithmetic means of ratios, which are highly sensitive to a few extreme wins and to the exclusion of failures. The supplied table already contains five cases where HF-DGF is more than an order of magnitude slower than the best baseline, so the choice of summary statistic is not a cosmetic detail; it determines whether the central '5x faster' claim survives. A reanalysis of the existing table with all 41 rows, a fixed failure penalty, and paired rank tests would settle this without any new experiments. I therefore keep the reader's CONDITIONAL verdict: the paper should not be accepted as reporting robust state-of-the-art speedups until this reanalysis is provided. If the reanalysis confirms the advantage, the remaining static-analysis and coverage-measurement concerns become the main conditions; if it does not, the central claim fails.","tokens_in":20453,"tokens_out":22610,"duration_ms":236948,"concrete_test":"Recompute the headline comparison from Table II over all 41 vulnerabilities, assigning a fixed penalty (e.g., 86,400 s, the 24 h budget) to tools whose result is N.A. Report (a) the median of per-case speedup ratios and the ratio of median TTEs for HF-DGF vs each baseline, and (b) a paired Wilcoxon signed-rank test on the 20-run medians including the N.A.-as-86400s rows. If the median ratio is not >1 or the rank test is not significant against AFLGo and DAFL, the average-speedup claim in the Abstract should be retracted or heavily qualified.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing number is the Abstract's average speedup claim. In §VI-B it is computed only over the subset of Table II where every tool produced a median TTE, and the summary is the arithmetic mean of per-case speedup ratios. This protocol can change the conclusion. First, N.A. rows are excluded: the 41 rows are reduced to the 20 reported benchmarks, so systematic failures of HF-DGF do not count against the average. Table II shows multiple order-of-magnitude regressions on dispatch-heavy binutils targets: CVE-2017-8392 (3270s vs DAFL 15s), CVE-2017-8393 (17601s vs AFL 707s), CVE-2017-8395 (3879s vs DAFL 8s), CVE-2017-8398 (16384s vs AFL 709s), and CVE-2017-7303 (4934s vs AFL 91s). Second, averaging ratios lets a few large wins (e.g., 33x on CVE-2017-5969) dominate; the median of the same per-case ratios is far lower than the reported mean. Without a defined failure penalty and rank-based significance testing, the claimed '5.05x faster than AFL, 5.79x faster than AFLGo' is not a robust summary of the data, and the central empirical claim is unverified.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes HF-DGF, a directed grey-box fuzzer that combines three feedback signals for seed scheduling: basic-block-level control-flow distance computed by a backward-stepping algorithm over a virtual inter-procedural CFG, a value-flow influence score derived from SVF value-flow analysis, and slice coverage. To reduce overhead, it instruments only sliced basic blocks, boundary blocks, and value-flow-influencing blocks. The authors evaluate HF-DGF on DAFL's 41 CVE benchmark against AFL, AFLGo, WindRanger, DAFL, and Beacon, reporting average crash-reproduction speedups of 5.05x vs. AFL, 5.79x vs. AFLGo, 73.75x vs. WindRanger, 2.56x vs. DAFL, and 8.45x vs. Beacon, plus claims of more vulnerabilities triggered, lower coverage indicating better directionality, and faster static analysis. An ablation study isolates the contribution of each feedback component.","tokens_in":20780,"tokens_out":13602,"duration_ms":121446,"significance":"If the reported results hold, HF-DGF advances directed grey-box fuzzing by integrating data-flow-guided state-space exploration with control-flow distance, and its selective instrumentation is a promising approach to controlling runtime overhead. The manuscript has concrete strengths: it uses 20 repetitions per benchmark, reports a full 41-CVE table, includes an ablation study, and makes the source code available. The Discussion's candid acknowledgment of the over-tainting limitation on CVE-2017-8395 is also a strength. However, the headline claims are not yet convincingly supported because of the aggregation protocol for time-to-exposure and the non-comparable coverage metric, both detailed below.","major_comments":[{"comment":"The headline speedup figures (5.05x vs. AFL, 5.79x vs. AFLGo, 73.75x vs. WindRanger, 2.56x vs. DAFL, 8.45x vs. Beacon) are arithmetic means of per-benchmark ratios computed only over the 20 benchmarks where every tool produced a median TTE. This protocol excludes all N.A. rows, including the three cases where HF-DGF itself has N.A. while a baseline succeeds (CVE-2019-9114, CVE-2017-9048, CVE-2018-14498) and several cases where baselines fail; it also leaves the mean dominated by the 33x win on CVE-2017-5969. The paper's own Discussion VII-B concedes over-tainting caused underperformance on CVE-2017-8395, where Table II reports HF-DGF at 3879 s against DAFL's 8 s, a regression not captured by the average. The authors should report a robust summary over all 41 cases (e.g., treating a 24-hour failure as the TTE), median speedup ratios, and a paired significance test (e.g., Wilcoxon signed-rank). Without such a reanalysis, the central empirical claim is not established.","section":"§VI-B, Table II, Abstract"},{"comment":"The RQ2 claim that HF-DGF exhibits the lowest code coverage and therefore superior directionality is confounded by the selective instrumentation strategy. HF-DGF instruments only about 24% of basic blocks for coverage (#Cov-Ins/#BB = 23.96% in Table IV), so its coverage bitmap is defined over a different block set from that of the full-instrumentation baselines; lower measured coverage can be an artifact of a smaller instrumentation domain rather than evidence of focused exploration. Additionally, the 18 targets analyzed in Fig. 4 are chosen as the 'top-performing targets according to the TTE HF-DGF achieved,' which is a selection bias. The coverage comparison should either use an identical instrumentation for all fuzzers or report coverage as a fraction of instrumented blocks, and the target selection should be justified as representative.","section":"§VI-C, Fig. 4, Table IV"},{"comment":"The energy schedule PHF-DGF in Eq. (10) contains five free constants (10, 1, 0.5, -5, and the exponent base 2) and the product form of Eq. (11) is asserted without sensitivity analysis. Because the evaluation benchmark (DAFL's 41 CVEs) is the same one on which the design appears to have been developed, it is unclear whether these constants are overfit to this particular dataset. A parameter-sensitivity study or at least a discussion of how the constants were chosen is needed to support the external validity of the reported speedups.","section":"§IV-C, Eqs. (10)-(11)"}],"minor_comments":[{"comment":"The title of Algorithm 1 contains a typo: 'Backward-Steping' should be 'Backward-Stepping.'","section":"Algorithm 1"},{"comment":"In the case bb ∈ Entries, the formula uses Depth(bb), which is 0 for any entry block; based on Algorithm 1 (line 13), this should likely be Depth(cs), the depth of the call-site block. Please clarify or correct.","section":"Eq. (2)"},{"comment":"The text states 'After excluding the 9 cases where no fuzzing tool could obtain a median value,' but Table II appears to mark only 8 rows with an asterisk; the count should be verified.","section":"§VI-B"},{"comment":"The paper refers to 'the 20 benchmarks where all the tools successfully reported a median TTE,' yet by my count of Table II, 19 rows have numeric TTEs for all five ASAN-instrumented tools; please reconcile this discrepancy.","section":"§VI-B"},{"comment":"The term 'virtual ICFG' is used repeatedly but never formally defined; please explain what distinguishes it from a standard interprocedural control-flow graph.","section":"§IV-A"}],"recommendation":"major_revision","confidential_remarks":"The central issue is whether the speedup claims survive a more rigorous statistical treatment. I would encourage the editor to ask for a reanalysis that treats N.A. as a 24-hour TTE and reports median ratios and effect sizes. The selective-instrumentation confound in RQ2 is also important. The paper is not a reject; it is a solid systems paper with a fixable evaluation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the short version: HF-DGF is a synthesis of known DGF pieces with two genuinely new metrics—a backward-stepping basic-block distance on a virtual ICFG and a value-flow influence score—plus a selective instrumentation strategy that lowers overhead. The paper is honest about its main weakness (over-tainting, Section VII-B) and the code is public. The core empirical claim, though, is shakier than the abstract's averages suggest.\n\nWhat's new: the backward-stepping distance computation (Algorithm 1, Eq. 2) and the value-flow influence score (Eqs. 6–8) are not in the cited AFLGo, DAFL, WindRanger, or SelectFuzz papers. Combining control-flow distance, value-flow influence, and slice coverage into one scheduler is a reasonable integration, and the ablation study (RQ5) gives each component some credit. The static analyzer is faster than the baselines by a wide margin, which matters in practice.\n\nThe soft spots are real. The headline speedups (5.05x vs AFL, 5.79x vs AFLGo, etc.) are computed only over the 20 of 41 benchmarks where every tool produced a median TTE, and they are arithmetic means of per-case ratios. That protocol excludes N.A. rows—so systematic failures do not count—and lets a few large wins dominate. Table II shows multiple order-of-magnitude regressions on binutils targets (CVE-2017-8393, 8395, 8398, 7303). A median or rank-based comparison over all 41 cases would give a fairer picture. The coverage-direction claim (RQ2) is also confounded: HF-DGF intentionally instruments about 24% of blocks, so reporting lower coverage while saying it triggered the bugs is evidence of directionality only if the comparison uses the same instrumentation scope. The energy schedule (Eq. 10) has hand-chosen constants with no sensitivity analysis, and there is no statistical testing.\n\nThat said, the paper does not overclaim the limitations. Section VII-B explicitly concedes over-tainting and names a case where it underperforms. The benchmark is external (DAFL's 41 CVEs), so the central idea is not defined into existence. The work is a credible incremental step in directed fuzzing, not a breakthrough and not a dud.\n\nWho this is for: anyone working on directed-greybox or data-flow-guided fuzzing. The proposed metrics are worth reading carefully, and the evaluation protocol is a good case study for why averaging ratios over a convenient subset can mislead.\n\nRecommendation: engage it in peer review—yes. Require a reanalysis of the TTE data with all 41 cases, a defined failure penalty, and rank-based significance tests, plus a clarified coverage comparison. With those fixes, the empirical claims could be made solid.","headline":"A genuinely new pair of DGF metrics and an honest ablation, but the headline speedup averages hide a favorable-subset problem and need reanalysis before the strong claims are accepted.","tokens_in":21276,"tokens_out":4054,"would_cite":true,"duration_ms":29656,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"HF-DGF claims that a directed fuzzer guided by control-flow distance, value-flow influence, and slice coverage reproduces real-world crashes 2.56 to 73.75 times faster than five baseline fuzzers.","keywords":["directed grey-box fuzzing","hybrid feedback","value-flow analysis","control-flow distance","selective instrumentation","crash reproduction","seed scheduling","static analysis"],"falsifier":"Give HF-DGF a program whose value-flow graph is missing one true edge to the target's data, then compare its time-to-exposure against a distance-only fuzzer; if the missing edge does not erase the speedup, the claimed value-flow guidance is not what drives the result.","tokens_in":20287,"feed_emoji":"🎯","tokens_out":10820,"duration_ms":95606,"temperature":0.7,"pith_summary":"HF-DGF tries to establish that a directed grey-box fuzzer steered by three complementary runtime signals can reach a targeted bug site faster and then explore the data states around it more thoroughly than fuzzers guided by control-flow distance alone. The three signals are a basic-block-level control-flow distance computed by a backward-stepping algorithm on a virtual interprocedural control-flow graph, a value-flow influence score that measures how strongly executed code modifies target data, and coverage of the statically sliced target-relevant region. The framework couples these signals in seed prioritization and energy allocation to implement what the authors call a reach-then-explore paradigm, and it uses selective instrumentation so that only boundary blocks, sliced blocks, and value-flow-relevant blocks are instrumented. If the claims hold, crash reproduction and patch testing on large C programs would get measurably faster; the reported median time-to-exposure speedups over five baselines range from 2.56 to 73.75 times.","feed_headline":"Hybrid feedback fuzzer reproduces crashes up to 73 times faster","feed_subtitle":"Combining control-flow distance, value-flow influence, and slice coverage steers fuzzing to the right code and right data states.","key_machinery":"The load-bearing mechanism is a hybrid feedback scheduler operating on three static-analysis-derived metrics. Control-flow distance is computed per basic block by a backward-stepping algorithm over a virtual interprocedural control-flow graph: call sites are treated as edges, reachable functions are unwrapped to basic-block level, and a seed's distance is the average distance of the boundary blocks in its execution trace. The value-flow influence score is computed from the value-flow graph with field-sensitive Andersen-style pointer analysis: each instruction gets a distance to the target data, each basic block takes the minimum value-flow influence over its instructions, and a seed's score is the sum over executed value-flow-relevant blocks. Slice coverage is the set of target-relevant basic blocks reached, where the slice is the union of a control-flow backward slice and a value-flow backward slice from the target. These three signals are combined in a product energy formula with simulated annealing applied separately to control-flow distance and to value-flow influence, while selective instrumentation limits feedback collection to boundary blocks, sliced blocks, and value-flow-relevant blocks.","core_discovery":"The central claim is that directed fuzzing splits naturally into reaching the target and then exploring the target's state space, and that each phase needs its own feedback signal. HF-DGF computes control-flow distance at basic-block granularity with a backward-stepping algorithm on a virtual interprocedural control-flow graph, so seeds are ranked by structural proximity rather than by function-level call-graph distance. For state-space exploration it defines value-flow influence: instructions that modify or influence target data receive a score based on their value-flow distance to the target, and a seed's score is the accumulated influence of the basic blocks on its execution path; slice coverage then measures how much of the target-relevant sliced region a seed has touched. Selective instrumentation restricts feedback collection to boundary blocks, sliced blocks, and value-flow-relevant blocks, cutting instrumented coverage by roughly 76 percent while preserving vulnerability discovery. On 41 real-world vulnerabilities the prototype reports median time-to-exposure speedups of 5.05 times, 5.79 times, 73.75 times, 2.56 times, and 8.45 times over the five baseline fuzzers, more vulnerabilities triggered within 24 hours, and the lowest code coverage among the fuzzers that did trigger the crash.","pith_inferences":["The same three-signal decomposition could be carried over to patch testing or to verifying static-analysis warnings, where the target set is a list of sinks rather than a single crash site.","The value-flow influence score is a static proxy for data relevance; coupling it with dynamic taint or concolic solving could let the fuzzer satisfy tight value constraints instead of relying on random mutation.","The boundary-block distance idea could be ported to binary-only fuzzing by recovering an analogous boundary set from disassembly, accepting coarser pointer reasoning.","The paper's own over-tainting discussion points to a concrete refinement: make slicing region-aware so dispatch functions that parse different input regions are not all included in the target-relevant set."],"forward_implications":["If the reported speedups hold, crash reproduction and patch validation on large C programs could be run with a fraction of the current fuzzing budget: median time-to-exposure drops by factors of 2.56 to 73.75 on the benchmark suite.","The reach-then-explore split offers a reusable template: use one signal to get to the target, a second to decide which data matters, and a third to track how much of the relevant region has been explored.","Selective instrumentation at boundary, slice, and value-flow blocks means that rich multi-signal feedback does not require whole-program instrumentation, which keeps the approach affordable on large binaries.","The observation that the fuzzer succeeds with the lowest code coverage suggests target-relevant coverage, not global coverage, is the better measure of directedness for future fuzzing campaigns."],"supporting_citations":[{"why":"Supplies the distance-guided baseline and the exponential cooling schedule that the hybrid energy formula adapts; most of the comparative speedups are against this design.","marker":"[21]"},{"why":"Provides the 41-vulnerability benchmark dataset and the data-dependency-guided baseline DAFL; the evaluation dataset and one of the strongest baselines come from here.","marker":"[22]"},{"why":"Introduces deviation basic blocks and taint-guided mutation, which motivates the boundary-block distance feedback and serves as a baseline.","marker":"[23]"},{"why":"Andersen-style pointer analysis is used in the static analyzer to resolve indirect calls and field-sensitive value-flow edges.","marker":"[26]"},{"why":"SVF constructs the value-flow graph used to compute the value-flow influence scores and supports the implementation.","marker":"[34]"},{"why":"Serves as the path-pruning baseline in the non-ASAN comparison experiments.","marker":"[36]"},{"why":"Supplies the core grey-box fuzzing engine and circular seed queue that the scheduling layer is built on.","marker":"[32]"}],"fun_headline_variants":["Hybrid feedback fuzzer speeds crash reproduction by up to 73x","Fuzzer triggers crashes 73x faster using data-flow guidance","Guiding fuzzing with value flow cuts crash search time 73-fold","Hybrid feedback fuzzer reproduces crashes 73x faster, with less code exploration"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole speedup rests on the static analysis correctly deciding which code can reach the target and which code shapes the target's data; if that map is wrong, the hybrid feedback steers effort in the wrong direction.","fun_headline_variants_meta":{"raw":{"variants":["Hybrid feedback fuzzer speeds crash reproduction by up to 73x","Fuzzer triggers crashes 73x faster using data-flow guidance","Guiding fuzzing with value flow cuts crash search time 73-fold","Hybrid feedback fuzzer reproduces crashes 73x faster, with less code exploration"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000811,"raw_usage":{"total_tokens":3621,"prompt_tokens":1075,"completion_tokens":2546,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":691,"completion_tokens_details":{"reasoning_tokens":2462}},"tokens_in":691,"tokens_out":2546,"duration_ms":27988,"temperature":1.0,"reasoning_tokens":2462,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T21:51:10.825953+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Give HF-DGF a program whose value-flow graph is missing one true edge to the target's data, then compare its time-to-exposure against a distance-only fuzzer; if the missing edge does not erase the speedup, the claimed value-flow guidance is not what drives the result.","supporting_citations":[{"cited_title":"Directed greybox fuzzing,","cited_arxiv_id":null,"evidence_quote":"Supplies the distance-guided baseline and the exponential cooling schedule that the hybrid energy formula adapts; most of the comparative speedups are against this design."},{"cited_title":"{DAFL}: Directed Grey- box Fuzzing guided by Data Dependency,","cited_arxiv_id":null,"evidence_quote":"Provides the 41-vulnerability benchmark dataset and the data-dependency-guided baseline DAFL; the evaluation dataset and one of the strongest baselines come from here."},{"cited_title":"WindRanger: a directed greybox fuzzer driven by deviation basic blocks,","cited_arxiv_id":null,"evidence_quote":"Introduces deviation basic blocks and taint-guided mutation, which motivates the boundary-block distance feedback and serves as a baseline."},{"cited_title":"Program analysis and specialization for the C pro- gramming language,","cited_arxiv_id":null,"evidence_quote":"Andersen-style pointer analysis is used in the static analyzer to resolve indirect calls and field-sensitive value-flow edges."},{"cited_title":"Svf: interprocedural static value-flow analysis in llvm,","cited_arxiv_id":null,"evidence_quote":"SVF constructs the value-flow graph used to compute the value-flow influence scores and supports the implementation."},{"cited_title":"Beacon: Directed grey-box fuzzing with provable path pruning,","cited_arxiv_id":null,"evidence_quote":"Serves as the path-pruning baseline in the non-ASAN comparison experiments."},{"cited_title":"American fuzzy lop,","cited_arxiv_id":null,"evidence_quote":"Supplies the core grey-box fuzzing engine and circular seed queue that the scheduling layer is built on."}],"review_version":1}