Pith. sign in

REVIEW 3 major objections 6 minor 39 references

Empirical Notes on the Interaction Between Continuous Kernel Fuzzing and Development

T0 review · 3 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read For syzbot-reported Linux kernel crashes, bug type and review status do not explain fix times, while commit-level code churn gives only a weak signal.

desk verdict A useful descriptive snapshot of syzbot's software-engineering side, but the one regression result that matters for RQ.3 is not credible due to unaddressed pseudo-replication. read the letter →

arxiv 1909.02441 v1 pith:AW646JSW submitted 2019-09-05 cs.SE cs.CR

Jukka Ruohonen , Kalle Rindell This is my paper · ORCID
classification cs.SEcs.CR
keywords continuouskernelfuzzingsyzkallersyzbotbugfixingtimecodechurnLinuxBSDkernelscrashtriage
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper examines the practical lifecycle of crashes reported by the syzkaller/syzbot continuous kernel-fuzzing infrastructure, using a 30 June 2019 snapshot of the online dashboard for Linux, FreeBSD, NetBSD, and OpenBSD. It tries to establish that fuzzing-induced kernel bugs are numerous and often long-lived, with over 800 unresolved crashes across the four kernels, Linux medians of 136 days open and 38 days to fix, and faster reported fixes in the BSD kernels. Bug type and whether the fix commit was reviewed or tested show no reliable association with time-to-fix in a proportional-hazards regression; the only significant churn metric, files modified, gives a weak positive signal (coefficient 0.040, p = 0.042). If this holds, triaging fuzzer output by crash category or review status alone will not predict repair effort, and the practical bottleneck is the volume and age of unresolved reports.

What carries the argument

The carrying mechanisms are the syzbot dashboard as a data source and three measurement instruments: string matching on sanitizer output to assign bug types; searching fix-commit messages for reviewed-by, tested-by, and reviewed-and-tested-by tags; and proportional-hazards regression for time-to-fix, checked with ANOVA and Welch tests for bug-type effects. The dashboard supplies crash counts, open durations, and fix commits; sanitizer summary strings produce the nine bug-type categories; and the regression tests whether bug type, review/test status, or churn metrics (files modified, lines added, lines deleted) shift the hazard of a fix. The churn metrics come from git's shortstat output at the commit level, with the fixed-bug analysis run at bug level while acknowledging replicated commit-based values.

What would settle it

Take the syzbot dashboard, match each closed crash to its fixing commit using the commit hashes the dashboard exposes, and count how many crash reports share one fixing commit; if many reports map to one commit, or if the same underlying bug appears under multiple sanitizer labels, then the bug-level result (files modified coefficient 0.040, p = 0.042) is not about distinct bugs and the claim would need revision.

Watch

Extended reading notes

Core claim

The central discovery, stated on the paper's own terms, is that for syzbot-reported Linux kernel crashes, neither the bug's type nor whether its fix commit was reviewed or tested explains how long it takes to fix, while one code-churn metric does, weakly. The analysis separates open crashes (days-since-reported) from closed bugs (time-to-fix): 708 open Linux crashes and 1,344 closed Linux bugs were examined, with medians of 136 days open and 38 days to fix, while the BSD kernels resolved their reported fuzzing bugs faster (median below 20 days). Bug-type composition is dominated by debug warnings (WARNING), assertions (BUG), use-after-free issues, and general-protection faults. Only about 23% of fixed Linux bugs had reviewed-by, tested-by, or both tags in their fix commits. After merging fault and deadlock classes and unifying review/test indicators, the proportional-hazards model finds no statistically significant coefficient for any bug type or for review/test status, and the only significant churn metric is files modified (coefficient 0.040, p = 0.042). The conclusion the author draws is that simple, static explanations for fuzzing-induced bug repair times are not plausible.

Load-bearing premise

The paper's counts and regressions treat each syzbot dashboard crash entry as an independent bug with reliable report and fix dates, but the paper itself notes that a single bug can produce many crash reports, so if that one-to-many relation is large the central numbers and regression sample would be distorted.

Editorial extensions

If this is right

  • More than 800 unresolved syzbot crashes across the four kernels imply a substantial unresolved triage and debugging backlog, including 80 Linux crashes older than a year.
  • The shorter reported fix times for the BSD kernels suggest development-process differences, reporting differences, or both, which the paper does not fully separate.
  • The null results for bug type and review/testing mean that prioritization schemes based on these static categories will not predict repair times for syzbot-reported kernel bugs.
  • The weak files-modified signal, if it is real, points to fix complexity rather than bug nature as the main measurable correlate of repair delay.
  • Future work needs other predictors, such as stack traces, severity, or subsystem context, because the tested categories do not explain the fix times.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension is to link syzbot crash reports to their actual fixing commits and count how many reports share one commit; if the one-to-many relation between crashes and bugs is large, the backlog counts and regression results would need rescaling.
  • The same dashboard data could be used to test whether policy interventions, such as mass-CVE filing or mandatory review for fuzzer-reported bugs, actually change fix times; the paper's near-zero review signal suggests these interventions may not move the median.
  • Re-running the same proportional-hazards specification on a later dashboard snapshot would show whether the weak files-modified signal is stable as syzbot matures and the backlog ages.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper reports an empirical study of a June 30, 2019 snapshot of the syzbot continuous-fuzzing dashboard for Linux, FreeBSD, NetBSD, and OpenBSD. It poses three research questions: how long fuzzing-induced bugs take to be fixed and whether the distributions differ across kernels; what bug types occur in Linux and how often fixes were reviewed or tested; and whether bug type, review/testing, or code churn explain Linux time-to-fix. The main descriptive findings are 828 open crash reports, a median time-to-fix of 38 days for Linux versus below 20 for the BSD kernels, a dominance of warning/debug, use-after-free, and general-protection faults, and about 23% of Linux fixes carrying review/test markers. The regression analysis using Cox proportional hazards finds that only the 'files modified' churn metric is statistically significant among the tested variables.

Significance. The paper would provide a useful descriptive baseline for the software-engineering side of continuous kernel fuzzing, an area that is mostly studied from the fuzzing-technique perspective. Its strengths are the use of publicly available dashboard data, explicit acknowledgment of key limitations such as crash-to-bug ambiguity and replicated commit covariates, and a simple methodology that can be applied to later snapshots. The inferential part is modest, and the central RQ.3 claim rests on one marginal p-value that needs robustness checks; if the finding survives adjustment, it is a genuinely interesting negative result for bug-type and review-based explanations of fuzzing fix times.

major comments (3)
  1. [II-B, Table VI] The RQ.3 result in Table VI is not supported by the standard errors as reported. Section II-B acknowledges that 'some of the commit-based values are replicated across multiple observations,' and the same fixing commit can be repeated across bug-level observations. The Cox partial likelihood in Table VI treats all 1344 closed-bug observations as independent even though 'files modified,' 'lines added,' and 'lines deleted' are constant for all bugs fixed by one commit. This pseudo-replication can understate standard errors, and the only significant coefficient (files modified, coefficient 0.040, p=0.042) is exactly the kind of marginal result that can disappear when clustering on the fixing commit is accounted for. The authors should report the number of unique fixing commits, use cluster-robust standard errors or a frailty model, or analyze at the commit level. Without such an analysis, the claim that 'only code churn provides a weak statistical signal' is not yet established.
  2. [III-C, Table V] The proportional-hazards check in Table V is borderline for the TTF model: the global test is p=0.052 and the 'faults and deadlocks' variable is p=0.038. The authors acknowledge this but still interpret the coefficients directly. Because the RQ.3 conclusion is largely a claim about which coefficients are non-significant, the robustness of the Cox model to non-proportional hazards should be shown, for example by a stratified model on the offending variable or by including time-dependent effects. This is a checkable revision rather than a fatal flaw.
  3. [II-B, IV-B, Fig. 1] The paper states in Section IV-B that 'the crashes reported by syzbot may or may not equate to unique bugs.' Since a single bug can produce many crash reports, the counts and empirical distributions in Fig. 1 and Table I describe crash reports, not necessarily distinct bugs. This is not a reason to reject, but the authors should either phrase every RQ.1 conclusion at the crash-report level or provide a sensitivity analysis that groups reports into syzbot bugs; currently the abstract and conclusion sometimes say 'bugs' for quantities that are crash-level.
minor comments (6)
  1. [Throughout] The paper uses 'ANOV A' and 'Leneve's test' in several places; these should be 'ANOVA' and 'Levene's test.'
  2. [Abstract, IV-A] The phrase 'have went through code review' should be 'have gone through code review.'
  3. [Data/Methods] No replication data or analysis scripts are provided. Given the modest dataset size and the centrality of the Cox result, making the scraped snapshot and R code available would substantially strengthen reproducibility.
  4. [III-A] The definition of time-to-fix (TTF) is understandable but informal; the authors should state the exact formula with the snapshot date and any time-zone assumptions used when subtracting dates from the dashboard.
  5. [III-C] The sensitivity analysis for the large commit ab8085c130edd65be0d95cc95c28b51c4c6faf9d is only described in one sentence; the paper should state which variable is affected and report the coefficient and p-value after exclusion.
  6. [II-C, III-B] The 'reviewed/tested' variables are based only on exact commit-message string matches; other legitimate forms of code review or testing that do not generate those tags are not captured, and this should be acknowledged more explicitly in the interpretation of the 23% figure.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: all results are direct measurements from external syzbot and git data; the Cox regression estimates are not fitted inputs renamed as predictions.

full rationale

The paper is an observational empirical study of syzbot dashboard data and git commit logs. All dependent and independent variables are measured externally: time-to-fix (TTF) is computed from syzbot report and close dates, bug types from sanitizer summary strings, reviewed/tested flags from commit-message string matching, and code churn from git shortstat output. None of these quantities is defined in terms of the paper's conclusions, and no parameter is fitted to a target result and then reported as a prediction. The Cox proportional hazards regression in Table VI estimates associations; the 'files modified' coefficient (0.040, p = 0.042) is a data-derived estimate, not an input embedded in the model by construction. The only author self-citations ([15], [16], [39]) are background references to fix-time literature and text-mining methods; they are not used to justify the central empirical claims or to forbid alternative models. Section IV-B explicitly acknowledges the one-to-many crash-to-bug relation and replicated commit-based values; this is a stated methodological limitation about possible pseudo-replication in the regression, which would be a correctness or robustness concern, not circularity. Because every result is a direct measurement or standard statistical analysis of external data, no derivation reduces to its own input.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No free parameters or invented entities are introduced. All assumptions are data-provenance and measurement assumptions, listed above; the regression coefficients are estimated from external data, not tuned to a target result.

assumptions (4)
  • domain assumption Syzbot dashboard crash entries and status or date fields are accurate and comparable across the four kernels.
    Section II-B: data come from a syzbot dashboard snapshot with no independent validation against kernel bug trackers or mailing lists; BSD integration with syzbot is acknowledged to be incomplete.
  • domain assumption Crash-to-bug grouping at bug level is meaningful.
    Section II-B handles one-to-many commit relations at bug level, while Section IV-B notes crashes may not equal unique bugs; counts and regressions still treat bug-level entries as units.
  • domain assumption String matching of commit messages (reviewed-by, tested-by, reviewed-and-tested-by) accurately measures code review and additional testing.
    Section II-C: RQ.2b and RQ.3b rely solely on searching lower-cased commit messages; trailers can be missing or added without actual review, so measurement is approximate.
  • domain assumption The proportional-hazards assumption for the Cox regression is sufficiently satisfied after merging deadlock or stall with general protection faults.
    Section III-C: the global PH test is p = 0.052 and the merged variable still shows p = 0.038; treating the model as interpretable relies on this assumption being close enough.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Empirical Notes on the Interaction Between Continuous Kernel Fuzzing and Development." pith.science (2026). https://pith.science/paper/AW646JSW

@misc{pith2026190902441,
  author       = {Pith},
  title        = {Pith review of: Empirical Notes on the Interaction Between Continuous Kernel Fuzzing and Development},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AW646JSW}},
  note         = {Machine review of arXiv:1909.02441}
}
read the original abstract

Fuzzing has been studied and applied ever since the 1990s. Automated and continuous fuzzing has recently been applied also to open source software projects, including the Linux and BSD kernels. This paper concentrates on the practical aspects of continuous kernel fuzzing in four open source kernels. According to the results, there are over 800 unresolved crashes reported for the four kernels by the syzkaller/syzbot framework. Many of these have been reported relatively long ago. Interestingly, fuzzing-induced bugs have been resolved in the BSD kernels more rapidly. Furthermore, assertions and debug checks, use-after-frees, and general protection faults account for the majority of bug types in the Linux kernel. About 23% of the fixed bugs in the Linux kernel have either went through code review or additional testing. Finally, only code churn provides a weak statistical signal for explaining the associated bug fixing times in the Linux kernel.

Figures

Figures reproduced from arXiv: 1909.02441 by the authors.

Figure 1
Figure 1. Pending Bugs (Crashes) Across Four Kernels [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Bug Fixing Delays Across Four Kernels TABLE I TWO-SAMPLE KOLMOGOROV-SMIRNOV TESTSa . Days-since-reported (see [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

39 extracted references · 38 canonical work pages

  1. [1]

    A Systematic Review of Fuzzing Techniques,

    C. Chen, B. Cui, J. Ma, R. Wu, J. Guo, and W. Liu, “A Systematic Review of Fuzzing Techniques,” Computers & Security , vol. 75, pp. 118–137, 2018

    work page 2018

  2. [2]

    Fuzzing the Linux Kernel,

    C. Carabas and M. Carabas, “Fuzzing the Linux Kernel,” in Proceedings of the Computing Conference . London: IEEE, 2017, pp. 839–843

    work page 2017

  3. [3]

    DIFUZE: Interface Aware Fuzzing for Kernel Drivers,

    J. Corina, A. Machiry, C. Salls, Y . Shoshitaishvili, S. Hao, C. Kruegel, and G. Vigna, “DIFUZE: Interface Aware Fuzzing for Kernel Drivers,” in Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (CCS 2017) . Dallas: ACM, 2017, pp. 2123–2138

    work page 2017

  4. [4]

    IMF: Inferred Model-Based Fuzzer,

    H. Han and S. K. Cha, “IMF: Inferred Model-Based Fuzzer,” in Proceedings of the Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (CCS 2017). Dallas: ACM, 2017, pp. 2345–2358

    work page 2017

  5. [5]

    kAFL: Hardware-Assisted Feedback Fuzzing for OS Kernels,

    S. Schumilo, C. Aschermann, R. Gawlik, S. Schinzel, and T. Holz, “kAFL: Hardware-Assisted Feedback Fuzzing for OS Kernels,” in Proceedings of the 26th USENIX Security Symposium . Vancouver: USENIX, 2017, pp. 167–182

    work page 2017

  6. [6]

    Empirical Analysis and Modeling of Black-Box Mutational Fuzzing,

    M. Zhao and P. Liu, “Empirical Analysis and Modeling of Black-Box Mutational Fuzzing,” in Proceedings of the 8th International Symposium on Engineering Secure Software and Systems (ESSoS 2016), Lecture Notes in Computer Science (Volume 9639) , J. Caballero, E. Bodden, and E. Athanasopoulos, Eds. London: Springer, 2016, pp. 173–189

    work page 2016

  7. [7]

    Proactive Security Testing and Fuzzing,

    A. Takanen, “Proactive Security Testing and Fuzzing,” in Highlights of the Information Security Solutions Europe 2009 Conference (ISSE 2009), N. Pohlmann, H. Reimer, and W. Schneider, Eds. The Hague: Springer, pp. 312–319

    work page 2009

  8. [8]

    Fuzzing for the Masses,

    ——, “Fuzzing for the Masses,” Network Security, no. 8, pp. 4–6, 2008

    work page 2008

Show all 39 references
  1. [9]

    CVE-less Vulnerabilities,

    J. Edge, “CVE-less Vulnerabilities,” 2019, Linux Weekly News (LWN), available online in June 2019: https://lwn.net/Articles/791855/

    2019

  2. [10]

    Handling Kernel Security Problems,

    J. Corbet, “Handling Kernel Security Problems,” 2008, Linux Weekly News (LWN), available online in June 2019: https://lwn.net/Articles/ 290227/

    2008

  3. [11]

    Mining Bug Databases for Unidentified Software Vulnerabilities,

    D. Wijayasekara, M. Manic, J. L. Wright, and M. McQueen, “Mining Bug Databases for Unidentified Software Vulnerabilities,” in Proceed- ings of the 5th International Conference on Human System Interactions (HSI 2012). Perth: IEEE, 2012, pp. 89–96

    2012

  4. [12]

    syzbot and the Tale of Thousand Kernel Bugs,

    D. Vyukov, “syzbot and the Tale of Thousand Kernel Bugs,” in Linux Security Summit , Vancouver, 2018, Available online in July 2019: https://events.linuxfoundation.org/wp-content/uploads/2017/11/ Syzbot-and-the-Tale-of-Thousand-Kernel-Bugs-Dmitry-Vyukov-Google. pdf

    2018

  5. [13]

    syzkaller,

    Google, Inc. et al., “syzkaller,” 2019, Available online in July: https: //github.com/google/syzkaller

    2019

  6. [14]

    ——, “syzbot,” 2019, Data scraped in 30th of June 2019 from the online dashboard at: https://syzkaller.appspot.com/

    2019

  7. [15]

    A Demand-Side Viewpoint to Software Vulnerabilities in WordPress Plugins,

    J. Ruohonen, “A Demand-Side Viewpoint to Software Vulnerabilities in WordPress Plugins,” in Proceedings of the 23rd Conference on the Evaluation and Assessment in Software Engineering (EASE 2019) . Copenhagen: ACM, 2019, pp. 222–228

    2019

  8. [16]

    A Case Study on Software Vulnerability Coordination,

    J. Ruohonen, S. Rauti, S. Hyrynsalmi, and V . Lepp ¨anen, “A Case Study on Software Vulnerability Coordination,” Information and Software Technology, vol. 103, pp. 239–257, 2018

    2018

  9. [17]

    Experience Report: Study of Vulnerabilities of Enterprise Operating Systems,

    A. Gorbenko, A. Romanovsky, O. Tarasyuk, and O. Biloborodov, “Experience Report: Study of Vulnerabilities of Enterprise Operating Systems,” in Proceedings of the IEEE 28th International Symposium on Software Reliability Engineering (ISSRE 2017). Toulouse: IEEE, 2017, pp. 205–215

    2017

  10. [18]

    A Market-Based Bug Allocation Mechanism Using Predictive Bug Lifetimes,

    H. Hosseini, R. Nguyen, and M. W. Godfrey, “A Market-Based Bug Allocation Mechanism Using Predictive Bug Lifetimes,” in Proceedings of the 16th European Conference on Software Maintenance and Reengi- neering (CSMR 2012) . Szeged: IEEE, 2012, pp. 149–158

    2012

  11. [19]

    On the Lag of Library Vulnerability Updates: An Investigation into the Repackage and Delivery of Security Fixes Within The npm JavaScript Ecosystem,

    B. Chinthanet, R. G. Kula, T. Ishio, A. Ihara, and K. Matsumoto, “On the Lag of Library Vulnerability Updates: An Investigation into the Repackage and Delivery of Security Fixes Within The npm JavaScript Ecosystem,” 2019, Archived manuscript, available online in July: https: /...

    2019 arXiv

  12. [20]

    Filtering Bug Reports for Fix-Time Analysis,

    A. Lamkanfi and S. Demeyer, “Filtering Bug Reports for Fix-Time Analysis,” in Proceedings of the 16th European Conference on Software Maintenance and Reengineering (CSMR 2012) . Szeged: IEEE, 2012, pp. 379–384

    2012

  13. [21]

    Time to Discover and Fix Software Vulnerabilities in Open Source Software Projects: Notes on Measurement and Data Availability,

    S. M. Muegge and S. M. M. Murshed, “Time to Discover and Fix Software Vulnerabilities in Open Source Software Projects: Notes on Measurement and Data Availability,” in Proceedings of the Portland In- ternational Conference on Management of Engineering and Technology (PICMET 20...

    2018

  14. [22]

    A Tale of Four Kernels,

    D. Spinellis, “A Tale of Four Kernels,” in Proceedings of the Inter- national Conference on Software Engineering (ICSE 2008) . Leipzig: ACM, 2008, pp. 381–390

    2008

  15. [23]

    Faults in Linux 2.6,

    N. Palix, G. Thomas, S. Saha, C. Calv `es, G. Muller, and J. Lawall, “Faults in Linux 2.6,” ACM Transactions on Computer Systems, vol. 32, no. 2, pp. 4:1–4:40, 2014

    2014

  16. [24]

    Peer Review on Open-Source Software Projects: Parameters, Statistical Models, and Theory,

    P. C. Rigby, D. M. German, L. Cowen, and M.-A. Storey, “Peer Review on Open-Source Software Projects: Parameters, Statistical Models, and Theory,”ACM Transactions on Software Engineering and Methodology, vol. 23, no. 4, pp. 35:1–35:33, 2014

    2014

  17. [25]

    When a Patch Goes Bad: Exploring the Properties of Vulnerability-Contributing Commits,

    A. Meneely, H. Srinivasan, A. Musa, A. R. Tejeda, M. Mokary, and B. Spates, “When a Patch Goes Bad: Exploring the Properties of Vulnerability-Contributing Commits,” in Proceedings of the ACM/IEEE International Symposium on Empirical Software Engineering and Mea- surement (ESEM...

    2013

  18. [26]

    Use of Relative Code Churn Measures to Predict System Defect Density,

    N. Nagappan and T. Ball, “Use of Relative Code Churn Measures to Predict System Defect Density,” in Proceedings. 27th International Conference on Software Engineering (ICSE 2005) . Saint Louis: IEEE, 2005, pp. 284–292

    2005

  19. [27]

    Evaluating Complexity, Code Churn, and Developer Activity Metrics as Indicators of Software Vul- nerabilities,

    Y . Shin, A. Meneely, and L. Williams, “Evaluating Complexity, Code Churn, and Developer Activity Metrics as Indicators of Software Vul- nerabilities,” IEEE Transactions on Software Engineering, vol. 37, no. 6, pp. 772–787, 2011

    2011

  20. [28]

    syzbot: Automated Kernel Testing,

    D. Vyukov, “syzbot: Automated Kernel Testing,” in Linux Plumbers Conference , Vancouver, 2018, Available online in July 2019: https://www.linuxplumbersconf.org/event/2/contributions/237/ attachments/61/71/syzbot automated kernel testing.pdf

    2018

  21. [29]

    Coverage-Guided Kernel Fuzzing with syzkaller,

    D. Drysdale, “Coverage-Guided Kernel Fuzzing with syzkaller,” 2016, Linux Weekly News (LWN), available online in June 2019: https://lwn. net/Articles/677764/

    2016

  22. [30]

    Regression Models and Life-Tables,

    D. R. Cox, “Regression Models and Life-Tables,” Journal of the Royal Statistical Society. Series B (Methodological) , vol. 34, no. 2, pp. 187–220, 1972

    1972

  23. [31]

    Empirical Comparisons of Proportional Hazards, Poisson, and Logistic Regression Modeling of Occupational Cohort Data,

    P. W. Callas, H. Pastides, and D. W. Hosmer, “Empirical Comparisons of Proportional Hazards, Poisson, and Logistic Regression Modeling of Occupational Cohort Data,” American Journal of Industrial Medicine , vol. 33, no. 1, pp. 33–47, 1998

    1998

  24. [32]

    Fox and S

    J. Fox and S. Weisberg, An R Companion to Applied Regression . Thousand Oaks: Sage, 2011

    2011

  25. [33]

    An Empirical Study of Long Lived Bugs,

    R. K. Saha, S. Khurshid, and D. E. Perry, “An Empirical Study of Long Lived Bugs,” in Proceedings of the IEEE Conference on Software Maintenance, Reengineering, and Reverse Engineering (CSMR-WCRE) . Antwerp: IEEE, 2014, pp. 144–153

    2014

  26. [34]

    The Generalization of ‘Student’s’ Problem when Several Different Population Variances are Involved,

    B. L. Welch, “The Generalization of ‘Student’s’ Problem when Several Different Population Variances are Involved,” Biometrika, vol. 34, no. 1/2, pp. 28–35, 1947

    1947

  27. [35]

    Cross Platform Bug Correlation Using Stack Traces,

    M. A. Ghafoor and J. H. Siddiqui, “Cross Platform Bug Correlation Using Stack Traces,” in Proceedings of the International Conference on Frontiers of Information Technology (FIT 2016) . Islamabad: IEEE, 2016, pp. 199–204

    2016

  28. [36]

    A Review of Machine Learning Applications in Fuzzing,

    G. J. Saavedra, K. N. Rodhouse, D. M. Dunlavy, and P. W. Kegelmeyer, “A Review of Machine Learning Applications in Fuzzing,” 2019, Archived manuscript, available online in June 2019: https://arxiv.org/ abs/1906.11133

    2019 arXiv

  29. [37]

    Do Stack Traces Help Developers Fix Bugs,

    A. Schr ¨oter, N. Bettenburg, and R. Premraj, “Do Stack Traces Help Developers Fix Bugs,” in Proceedings of the 7th IEEE Working Confer- ence on Mining Software Repositories (MSR 2010) . Cape Town: IEEE, 2010, pp. 118–121

    2010

  30. [38]

    ExploitMeter: Combining Fuzzing with Machine Learning for Automated Evaluation of Software Exploitability,

    G. Yan, J. Lu, Z. Shu, and Y . Kucuk, “ExploitMeter: Combining Fuzzing with Machine Learning for Automated Evaluation of Software Exploitability,” in Proceedings of the IEEE Symposium on Privacy- Aware Computing (PAC 2017). Washington: IEEE, 2017, pp. 164–175

    2017

  31. [39]

    Toward Validation of Textual Infor- mation Retrieval Techniques for Software Weaknesses,

    J. Ruohonen and V . Lepp ¨anen, “Toward Validation of Textual Infor- mation Retrieval Techniques for Software Weaknesses,” in Proceedings of the 29th International Workshop on Database and Expert Systems Applications (DEXA 2018) . Regensburg: Springer, 2018, pp. 265–277

    2018

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.