Pith. sign in

REVIEW 3 major objections 4 minor 42 references

ShadowProbe finds hidden algorithmic complexity bugs by screening for library APIs that hide superlinear costs, then reconstructing executable tests and measuring runtime growth.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-11 12:35 UTC pith:MGTINDXZ

load-bearing objection Solid systems result: shadow-complexity screening plus auto context recovery and measured growth finds many maintainer-confirmed ACVs across languages; the security-relevance leap from minimal Candidates is the soft spot, not a collapse. the 3 major comments →

arxiv 2607.05474 v1 pith:MGTINDXZ submitted 2026-07-06 cs.CR cs.SE

ShadowProbe: Language-Extensible Detection of Hidden Algorithmic Complexity Vulnerabilities

Yuanmin Xie , Xiangfan Wu , Wenhao Wu , Lingyun Ying , Puzhuo Liu , Haipeng Qu , Zhongyuan Chen , Min Zhou
show 1 more author
Chengnian Sun
This is my paper
classification cs.CR cs.SE
keywords algorithmic complexity vulnerabilitiesshadow complexitystatic analysisLLM-assisted testingworst-case complexitydenial of servicelanguage-extensible analysisruntime growth inference
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that many algorithmic complexity vulnerabilities (ACVs) come from shadow complexity: non-trivial work buried inside ordinary library calls (string slices, joins, parses, scans) that look constant-time at the call site. When those calls sit inside input-dependent loops or recursion, total time can grow superlinearly and enable denial of service. Existing detectors are usually language-specific, need heavy harnesses or instrumentation, and do not scale to large runtimes. ShadowProbe answers this with a staged pipeline: static screening that prioritizes loop/recursion structure plus shadow-complexity sinks; automatic reconstruction of a minimal executable context from a project symbol index; LLM synthesis of size-parameterized input generators under a fixed harness; and measurement-based classification of growth that filters GC/JIT noise. On the WISE suite it recovers ground-truth complexity classes with far fewer average seconds than SPF-WCA or Badger; on CPython, the JDK, Zig, Rustc, and vLLM it reports 162 previously unknown ACVs, of which 84 maintainers have confirmed and 41 have already fixed. The practical claim is that language-extensible static screening plus reconstructed candidates and wall-clock growth inference can surface real, fixable ACVs in mature ecosystems without per-language symbolic engines or hand-written drivers.

Core claim

ShadowProbe can discover previously unknown ACVs at scale across languages by treating shadow complexity as a first-class screening signal, automatically recovering compact executable contexts, using an LLM only to propose size-controlled generators, and confirming candidates solely when measured execution time shows non-linear growth after noise-aware sampling. On WISE it matches ground-truth complexity classes and is substantially faster than prior tools; on five large systems it found 162 ACVs with 84 maintainer confirmations.

What carries the argument

The four-stage ShadowProbe pipeline: shadow-complexity-guided static screening (loops/recursion plus sink families such as slicing, join, regex, parse, and hidden iteration), project-level symbol-index context recovery into a compilable fragment, multi-strategy LLM synthesis of Θ(n) input generators under a fixed harness, and adaptive logarithmic sampling with curve-fitting growth inference that classifies Low-order vs Polynomial/Exponential while discarding GC/JIT noise.

Load-bearing premise

That superlinear wall-clock growth of a reconstructed minimal test under LLM-chosen size-controlled inputs, after coarse filtering of runtime noise, is enough evidence of a real algorithmic blowup that also matters in the full project.

What would settle it

Run the same pipeline on a large corpus of functions whose true worst-case complexity is independently known (or exhaustively measured under full project contexts); if many Polynomial/Exponential reports collapse to linear once the full context or different generators are used, or if most maintainer-confirmed cases fail to reproduce outside the reconstructed Candidate, the validation claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. ShadowProbe is a multi-stage, language-extensible pipeline for detecting algorithmic complexity vulnerabilities (ACVs) that arise from shadow complexity—non-trivial costs hidden inside library APIs that appear trivial at call sites. It performs lightweight static screening for loops/recursion plus shadow-complexity sinks, recovers compact executable contexts via a project-level symbol index and dependency closure, uses an LLM to synthesize multiple size-parameterized input generators under a fixed harness contract, and validates candidates by adaptive logarithmic wall-clock sampling followed by coarse growth classification (Low-order / Polynomial / Exponential). On the WISE benchmark it recovers ground-truth complexity classes for all programs while running substantially faster than SPF-WCA and Badger; on CPython, JDK, Zig, Rustc and vLLM it reports 162 previously unknown cases of which 84 have been maintainer-confirmed (41 fixed).

Significance. If the validation pipeline is sound, the work supplies a practical, low-instrumentation method for scaling ACV discovery across language runtimes and large codebases—something prior fuzzing/symbolic tools have struggled with because of harness construction and language specificity. The concrete engineering strengths are real: modest language-specific front-ends (a few hundred to ~1.7 kLOC), systematic ablations on PyEmailBench that move FPR/FNR in the expected directions, multi-LLM robustness checks, and a non-trivial number of maintainer-confirmed and already-fixed issues in production systems (including long-lived CPython email and functools cases). These results are useful both for security triage and for library maintainers even if some findings ultimately classify as performance rather than pure DoS bugs.

major comments (3)
  1. [Section III-D, Algorithm 1] Section III-D and Algorithm 1 treat super-linear wall-clock growth of a reconstructed Candidate (minimal context + LLM generator) under adaptive log sampling as sufficient evidence of an algorithmic blow-up. The paper never demonstrates that the recovered fragment preserves the same asymptotic path that would be exercised inside the full project (missing callers, global state, library versions or JIT/GC interactions can alter complexity). Three runs per size plus a four-class fit also leave residual noise unquantified; without reported fit diagnostics (e.g., log-log slope confidence intervals, R² thresholds, or sensitivity to the [nmin,nmax] bounds) it is hard to judge how often GC/JIT/allocator effects are mis-classified as Polynomial.
  2. [Table IV, Section VI-B] Table IV’s headline numbers (162 Found / 84 Confirmed) rest on an unreported manual-triage step that converts raw growth reports into “ACV” cases. Section VI-B correctly notes that security relevance is context-dependent, yet the abstract and introduction present the counts as previously unknown ACVs. Without a breakdown of how many confirmed cases are reachable under adversarial inputs versus merely slow under non-adversarial or artificially constructed generators, the security-scale claim is stronger than the evidence supplied.
  3. [Section III-B] Context recovery (Section III-B) expands a bounded dependency closure from a symbol index. The paper gives no empirical measurement of how often the recovered fragment is incomplete (unresolved symbols, missing enclosing state) or how often incompleteness changes the observed complexity class. The ablation “w/o Ctx.” shows a large FNR increase, but that only demonstrates that context helps; it does not quantify residual incompleteness on the real-world projects that drive the main claim.
minor comments (4)
  1. [Abstract, throughout] The abstract opens with the duplicated token “Abstract—Abstract—”. Several places omit spaces after “ShadowProbe” (“ShadowProbeuses”, “ShadowProbeon”). These are easy to clean.
  2. [Table I, Section III-A] Table I lists representative sink families but never states how the static scanner actually matches them (exact API names, over-approximation, language-specific rules). A short paragraph or appendix would make Stage 1 reproducible.
  3. [Figure 5, Section III-C] Figure 5’s prompt structure is helpful, yet the paper never releases the concrete prompts or the repair-feedback loop. Releasing them (or a redacted version) would strengthen reproducibility claims.
  4. [Table II, Section IV-B] The WISE comparison (Table II) is limited to Java; the multi-language claim would be stronger if at least one non-Java baseline (even a simple cost-guided fuzzer) were run on the Python or Zig subjects.

Circularity Check

0 steps flagged

No circular derivation: complexity classes and real-world ACV counts rest on measured wall-clock growth and external maintainer confirmation, not on self-defined or fitted targets.

full rationale

ShadowProbe’s load-bearing chain is empirical, not definitional. Stage 1 screens with syntactic/shadow-sink patterns; Stage 2 recovers project symbols; Stage 3 has an LLM synthesize size-parameterized generators; Stage 4 (Algorithm 1) measures wall-clock times under adaptive log sampling and classifies growth into Low-order / Polynomial / Exponential. That classification is grounded in observed (n, t) traces, not in a parameter fitted to the same quantity being “predicted.” On WISE, reported classes are checked against independent ground-truth complexity labels and against SPF-WCA/Badger; agreement is an external benchmark result, not a tautology. Real-world counts (Table IV: 162 Found / 84 Confirmed) are filtered by manual triage and then by maintainer acknowledgment—external falsification, not self-citation. PyEmailBench labels used only for ablations are expert-built evaluation data, not inputs that force the headline claims. No uniqueness theorem, ansatz, or prior result by the same authors is invoked to make the complexity class or the ACV count true by construction. Residual concerns (minimal Candidate vs full project, LLM path coverage, security vs performance) are validity/scope issues, not circular reductions. Score 0 is therefore the correct finding.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 2 invented entities

The central claim rests on engineering and measurement assumptions rather than free physical constants: that a fixed catalog of library sinks plus loop/recursion structure yields a high-density candidate set; that a bounded symbol-index closure yields an executable context faithful enough for worst-case probing; that LLM generators under a size contract can hit expensive paths; and that wall-clock growth after adaptive sampling separates algorithmic blowups from GC/JIT noise. Free parameters are operational budgets and classification cutoffs. No new physical entities are postulated; “shadow complexity” is a named software phenomenon with independent examples in the paper.

free parameters (5)
  • adaptive sampling budget B and max samples k
    Algorithm 1’s exploration budget and sample cap control which (n,t) points enter growth inference and can change Low-order vs Polynomial calls.
  • number of LLM input-generation strategies (default three)
    Multi-strategy selection (score(Gi)) is a design choice; ablation shows recall depends on it.
  • complexity classification cutoffs (Low-order ≤ O(n log n) vs Polynomial vs Exponential)
    Coarse taxonomy decides whether a Candidate is reported as an ACV; thresholds are methodological, not derived.
  • per-run timeout and feasible input-size bounds [nmin, nmax]
    Boundary probing discards failures/timeouts; chosen bounds affect which growth regimes are observed.
  • context-recovery budget / dependency-closure depth
    Bounded expansion of the symbol index can omit state needed for true worst-case paths.
axioms (5)
  • domain assumption A fixed family of library APIs (text processing, collections, regex, parsing, implicit iteration) are reliable shadow-complexity sinks when nested under input-dependent control flow.
    Table I and Stage 1 screening treat these families as prior knowledge guiding candidate selection.
  • domain assumption Wall-clock execution time of a reconstructed Candidate, aggregated over a few runs and fitted in log-log space, is a valid proxy for asymptotic algorithmic complexity after discarding GC/JIT noise.
    Section III-D and Algorithm 1 make measurement-based growth the sole confirmation criterion.
  • domain assumption An LLM given signature, recovered context, and harness contract can synthesize generators whose dominant size measure grows as Θ(n) and that exercise high-cost paths often enough to be useful.
    Stage 3 depends on multi-strategy LLM synthesis plus compile/size-check repair loops.
  • domain assumption A bounded project-level symbol-index dependency closure yields a compilable fragment sufficient to expose the target’s worst-case behavior without the full project.
    Stage 2 Context Recovery is required for automated harness construction (Challenge 2).
  • standard math Standard asymptotic classes and curve-fitting in log-log space under T(n)=C n^k are appropriate for labeling Low-order / Polynomial / Exponential.
    Used in Dynamic Validation to map measured traces to complexity categories.
invented entities (2)
  • shadow complexity independent evidence
    purpose: Name and organize hidden non-trivial costs inside library/built-in APIs that are invisible at call sites and can amplify into ACVs.
    Central framing device of the paper; illustrated with CPython expandvars and slicing examples. Independent evidence exists via concrete quadratic cases and maintainer-confirmed bugs, so it is not a free-floating postulate.
  • ShadowProbe Candidate (size-parameterized generator + harness) no independent evidence
    purpose: Executable unit that makes worst-case claims measurable without instrumenting the full program under test.
    Pipeline artifact defined in Stage 3; usefulness is empirical via WISE and real-world runs rather than external prior definition.

pith-pipeline@v1.1.0-grok45 · 22420 in / 3670 out tokens · 36067 ms · 2026-07-11T12:35:37.709861+00:00 · methodology

0 comments
read the original abstract

Algorithmic Complexity Vulnerabilities (ACVs) arise when adversarial inputs trigger worst-case execution behavior, causing severe performance degradation or Denial-of-Service conditions. A key but underexplored source is shadow complexity: non-trivial computational costs hidden inside seemingly benign standard library APIs. Because these costs are invisible at call sites, attackers can exploit them to induce unexpected superlinear runtime behavior. Existing ACV detectors often rely on fuzzing, symbolic execution, or hybrid analysis, but they are usually language-specific, require substantial manual effort to construct harnesses, and depend on heavy runtime instrumentation. We present ShadowProbe, a scalable and language-extensible framework for discovering ACVs through lightweight static analysis, automated reconstruction of execution contexts, and Large Language Model (LLM) assisted test generation. ShadowProbe uses a structured multi-stage pipeline: it statically screens for candidate functions guided by shadow-complexity signals, reconstructs minimal executable contexts from project-level symbols, and synthesizes size-controlled inputs to probe worst-case behavior. It then validates candidates using execution-time measurements and robust statistical growth inference, separating true algorithmic blowups from runtime noise such as garbage collection and JIT compilation effects. We evaluate ShadowProbe on the WISE benchmark, where it consistently improves analysis efficiency over existing approaches. We further apply it to large-scale systems including CPython, the JDK, Zig, Rustc, and vLLM, uncovering many previously unknown ACVs, many of which have been confirmed and partially remediated by maintainers. These results show that ShadowProbe can identify hidden algorithmic risks across diverse real-world codebases.

Figures

Figures reproduced from arXiv: 2607.05474 by Chengnian Sun, Haipeng Qu, Lingyun Ying, Min Zhou, Puzhuo Liu, Wenhao Wu, Xiangfan Wu, Yuanmin Xie, Zhongyuan Chen.

Figure 1
Figure 1. Figure 1: A vulnerability from hidden library operations in [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the ShadowProbe workflow. are common syntactic indicators of potentially superlinear behavior. However, relying solely on these structures remains too coarse at large-project scale. We therefore incorporate shadow complexity sinks to refine the static patterns and improve the density of selected target functions. Specifically, we define two categories of code patterns that are prone to triggeri… view at source ↗
Figure 3
Figure 3. Figure 3: shows a representative Python example where N = |methodname| and the worst-case input is (<) N . The guard slice methodname[:1] copies at most one character. By contrast, because CPython strings are immutable, methodname[1:] creates a new string by copying the remaining suffix. When the current length is k, this slice costs Θ(k), so the total work is PN k=1 Θ(k) = Θ(N2 ). This copying cost is hidden in sli… view at source ↗
Figure 5
Figure 5. Figure 5: Prompt structure for synthesizing candidate worst-case [PITH_FULL_IMAGE:figures/full_fig_p005_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: A Candidate for _parseparam. the LLM to propose multiple input-generation strategies under the same harness. Each strategy describes a different way to construct inputs whose size is controlled by n and whose structure is intended to exercise a high-cost execution path. The strategies are instantiated as candidate generators, and ShadowProbe keeps the valid generator with the strongest measured growth on p… view at source ↗
Figure 7
Figure 7. Figure 7: Top-5 packages/modules with the largest shares of [PITH_FULL_IMAGE:figures/full_fig_p009_7.png] view at source ↗
Figure 9
Figure 9. Figure 9: Execution time: vulnerable vs. fixed _c3_mro. set_payload set_type set_charset set_param del_param get_params get_filename get_boundary set_boundary get_param get_content_charset _get_params_preserve _parseparam × n × n × n × n : Repeated invocations API Methods Amplifiers Direct Entry Root Cause [PITH_FULL_IMAGE:figures/full_fig_p010_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Structural view of exposure, trigger, and root cause [PITH_FULL_IMAGE:figures/full_fig_p010_10.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

42 extracted references · 7 canonical work pages

  1. [1]

    Denial of service via algorithmic complexity attacks,

    S. A. Crosby and D. S. Wallach, “Denial of service via algorithmic complexity attacks,” inProceedings of the 12th USENIX Security Symposium, Washington, D.C., USA, August 4-8, 2003. USENIX Association, 2003. [Online]. Available: https://www.usenix.org/confere nce/12th-usenix-security-symposium/denial-service-algorithmic-compl exity-attacks

    2003

  2. [2]

    Exploiting unix file-system races via algorithmic complexity attacks,

    X. Cai, Y . Gui, and R. Johnson, “Exploiting unix file-system races via algorithmic complexity attacks,” in30th IEEE Symposium on Security and Privacy (SP 2009), 17-20 May 2009, Oakland, California, USA. IEEE Computer Society, 2009, pp. 27–41. [Online]. Available: https://doi.org/10.1109/SP.2009.10

    work page doi:10.1109/sp.2009.10 2009

  3. [3]

    Slowfuzz: Automated domain-independent detection of algorithmic complexity vulnerabilities,

    T. Petsios, J. Zhao, A. D. Keromytis, and S. Jana, “Slowfuzz: Automated domain-independent detection of algorithmic complexity vulnerabilities,” inProceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, CCS 2017, Dallas, TX, USA, October 30 - November 03, 2017, B. Thuraisingham, D. Evans, T. Malkin, and D. Xu, Eds. ACM, 2017...

    doi:10.1145/3133956.3134073 2017

  4. [4]

    Acquirer: A hybrid approach to detecting algorithmic complexity vulnerabilities,

    Y . Liu and W. Meng, “Acquirer: A hybrid approach to detecting algorithmic complexity vulnerabilities,” inProceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security, CCS 2022, Los Angeles, CA, USA, November 7-11, 2022, H. Yin, A. Stavrou, C. Cremers, and E. Shi, Eds. ACM, 2022, pp. 2071–2084. [Online]. Available: https://doi.org...

    work page doi:10.1145/3548606.3559337 2022

  5. [5]

    DISCOVER: detecting algorithmic complexity vulnerabilities,

    P. Awadhutkar, G. R. Santhanam, B. Holland, and S. C. Kothari, “DISCOVER: detecting algorithmic complexity vulnerabilities,” in Proceedings of the ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/SIGSOFT FSE 2019, Tallinn, Estonia, August 26-30, 2019, M. Dumas, D. Pfahl, S. Apel, ...

    doi:10.1145/3338906.3341177 2019

  6. [6]

    Hotfuzz: Discovering algorithmic denial-of- service vulnerabilities through guided micro-fuzzing,

    W. Blair, A. Mambretti, S. Arshad, M. Weissbacher, W. Robertson, E. Kirda, and M. Egele, “Hotfuzz: Discovering algorithmic denial-of- service vulnerabilities through guided micro-fuzzing,” in27th Annual Network and Distributed System Security Symposium, NDSS 2020, San Diego, California, USA, February 23-26, 2020. The Internet Society,

    2020

  7. [7]

    Available: https://www.ndss-symposium.org/ndss-paper /hotfuzz-discovering-algorithmic-denial-of-service-vulnerabilities-thr ough-guided-micro-fuzzing/

    [Online]. Available: https://www.ndss-symposium.org/ndss-paper /hotfuzz-discovering-algorithmic-denial-of-service-vulnerabilities-thr ough-guided-micro-fuzzing/

  8. [8]

    CWE-407: Inefficient algorithmic complexity,

    MITRE Corporation, “CWE-407: Inefficient algorithmic complexity,” https://cwe.mitre.org/data/definitions/407.html, accessed: 2025-08-10

    2025

  9. [9]

    CVE-2020-10735,

    NIST, “CVE-2020-10735,” https://nvd.nist.gov/vuln/detail/CVE-2020-1 0735, accessed: 2025-08-10

    2020

  10. [10]

    CVE-2014-1474,

    ——, “CVE-2014-1474,” https://nvd.nist.gov/vuln/detail/CVE-2014-1 474, accessed: 2025-08-10

    2014

  11. [11]

    Singularity: pattern fuzzing for worst case complexity,

    J. Wei, J. Chen, Y . Feng, K. Ferles, and I. Dillig, “Singularity: pattern fuzzing for worst case complexity,” inProceedings of the 2018 ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/SIGSOFT FSE 2018, Lake Buena Vista, FL, USA, November 04-09, 2018, G. T. Leavens, A. Garcia, an...

    arXiv 2018

  12. [12]

    WISE: automated test generation for worst-case complexity,

    J. Burnim, S. Juvekar, and K. Sen, “WISE: automated test generation for worst-case complexity,” in31st International Conference on Software Engineering, ICSE 2009, May 16-24, 2009, Vancouver, Canada, Proceedings. IEEE, 2009, pp. 463–473. [Online]. Available: https://doi.org/10.1109/ICSE.2009.5070545

    work page doi:10.1109/icse.2009.5070545 2009

  13. [13]

    Symbolic complexity analysis using context-preserving histories,

    K. S. Luckow, R. Kersten, and C. S. Pasareanu, “Symbolic complexity analysis using context-preserving histories,” in2017 IEEE International Conference on Software Testing, Verification and Validation, ICST 2017, Tokyo, Japan, March 13-17, 2017. IEEE Computer Society, 2017, pp. 58–68. [Online]. Available: https://doi.org/10.1109/ICST.2017.13

    work page doi:10.1109/icst.2017.13 2017

  14. [14]

    Badger: complexity analysis with fuzzing and symbolic execution,

    Y . Noller, R. Kersten, and C. S. Pasareanu, “Badger: complexity analysis with fuzzing and symbolic execution,” inProceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA 2018, Amsterdam, The Netherlands, July 16-21, 2018, F. Tip and E. Bodden, Eds. ACM, 2018, pp. 322–332. [Online]. Available: https://doi.org/10....

    doi:10.1145/3213846.3213868 2018

  15. [15]

    vllm: A high-throughput and memory- efficient inference and serving engine for large language models,

    vLLM Project Contributors, “vllm: A high-throughput and memory- efficient inference and serving engine for large language models,” https://github.com/vllm-project/vllm, accessed: 2026-01-21

    2026

  16. [16]

    Evaluating large language models trained on code,

    M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. de Oliveira Pinto, J. Kaplan, H. Edwards, Y . Burda, N. Joseph, G. Brockman, A. Ray, R. Puri, G. Krueger, M. Petrov, H. Khlaaf, G. Sastry, P. Mishkin, B. Chan, S. Gray, N. Ryder, M. Pavlov, A. Power, L. Kaiser, M. Bavarian, C. Winter, P. Tillet, F. P. Such, D. Cummings, M. Plappert, F. Chantzis, E. Barnes, A. Her...

    Pith/arXiv arXiv 2021

  17. [17]

    Assisting static analysis with large language models: A chatgpt experiment,

    H. Li, Y . Hao, Y . Zhai, and Z. Qian, “Assisting static analysis with large language models: A chatgpt experiment,” inProceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2023, San Francisco, CA, USA, December 3-9, 2023, S. Chandra, K. Blincoe, and P. Tonella, Eds. A...

    doi:10.1145/3611643.3613078 2023

  18. [18]

    An empirical evaluation of using large language models for automated unit test generation,

    M. Schäfer, S. Nadi, A. Eghbali, and F. Tip, “An empirical evaluation of using large language models for automated unit test generation,” IEEE Trans. Software Eng., vol. 50, no. 1, pp. 85–105, 2024. [Online]. Available: https://doi.org/10.1109/TSE.2023.3334955

    doi:10.1109/tse.2023.3334955 2024

  19. [19]

    Tree-sitter: An incremental parsing system,

    Tree-sitter Project, “Tree-sitter: An incremental parsing system,” https: //tree-sitter.github.io/, accessed: 2026-02-01

    2026

  20. [20]

    Introducing GPT-4.1 in the API,

    OpenAI, “Introducing GPT-4.1 in the API,” https://openai.com/index/g pt-4-1/, published: 2025-04-14; Accessed: 2026-02-02

    2025

  21. [21]

    DeepSeek-V3.2: Pushing the frontier of open large language models,

    DeepSeek-AI, “DeepSeek-V3.2: Pushing the frontier of open large language models,” https://huggingface.co/deepseek-ai/DeepSeek-V3.2, published: 2025-12-01; Accessed: 2026-02-02

    2025

  22. [22]

    DeepSeek-V4: Towards highly efficient million-token context intelligence,

    ——, “DeepSeek-V4: Towards highly efficient million-token context intelligence,” https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash, 2026, accessed: 2026-06-28

    2026

  23. [23]

    HypoFuzz,

    Z. Hatfield-Dodds, “HypoFuzz,” https://github.com/Zac-HD/hypofuzz, accessed: 2026-06-19

    2026

  24. [24]

    PEP 443 – Single-dispatch Generic Functions,

    Ł. Langa, “PEP 443 – Single-dispatch Generic Functions,” https://peps .python.org/pep-0443/, 2013, accessed: 2026-01-05

    2013

  25. [25]

    A monotonic superclass linearization for dylan,

    K. Barrett, B. Cassels, P. Haahr, D. A. Moon, K. Playford, and P. T. Withington, “A monotonic superclass linearization for dylan,” inProceedings of the 1996 ACM SIGPLAN Conference on Object- Oriented Programming Systems, Languages & Applications, OOPSLA 1996, San Jose, California, USA, October 6-10, 1996, L. Anderson and J. Coplien, Eds. ACM, 1996, pp. 69...

    work page doi:10.1145/236337.236343 1996

  26. [26]

    The python 2.3 method resolution order,

    M. Simionato, “The python 2.3 method resolution order,” https://www. python.org/download/releases/2.3/mro/, 2003, accessed: 2026-01-05

    2003

  27. [27]

    Django web framework,

    Django Software Foundation, “Django web framework,” https://github .com/django/django, accessed: 2026-02-01

    2026

  28. [28]

    Flask-mail,

    Pallets Team, “Flask-mail,” https://github.com/pallets-eco/flask-mail, accessed: 2026-02-01

    2026

  29. [29]

    Gnu mailman,

    GNU Project, “Gnu mailman,” https://gitlab.com/mailman, accessed: 2026-02-01

    2026

  30. [30]

    What’s new in python 2.4: Optimizations,

    A. M. Kuchling, “What’s new in python 2.4: Optimizations,” https: //docs.python.org/2/whatsnew/2.4.html#optimizations, 2005, accessed: 2026-01-25

    2005

  31. [31]

    Repeated string concatenation is quadratic in PyPy (and CPython),

    C. Bolz-Tereick, “Repeated string concatenation is quadratic in PyPy (and CPython),” https://pypy.org/posts/2023/01/string-concatenation-q uadratic.html, 2023, accessed: 2026-01-25

    2023

  32. [32]

    XSTRESSOR : Automatic generation of large-scale worst-case test inputs by inferring path conditions,

    C. Saumya, J. Koo, M. Kulkarni, and S. Bagchi, “XSTRESSOR : Automatic generation of large-scale worst-case test inputs by inferring path conditions,” in12th IEEE Conference on Software Testing, Validation and Verification, ICST 2019, Xi’an, China, April 22-27, 2019. IEEE, 2019, pp. 1–12. [Online]. Available: https://doi.org/10.1109/ICST.2019.00011

    doi:10.1109/icst.2019.00011 2019

  33. [33]

    Large language model for vulnerability detection and repair: Literature review and the road ahead,

    X. Zhou, S. Cao, X. Sun, and D. Lo, “Large language model for vulnerability detection and repair: Literature review and the road ahead,” ACM Trans. Softw. Eng. Methodol., vol. 34, no. 5, pp. 145:1–145:31,

  34. [34]

    Available: https://doi.org/10.1145/3708522

    [Online]. Available: https://doi.org/10.1145/3708522

    doi:10.1145/3708522

  35. [35]

    Understanding the effectiveness of large language models in detecting security vulnerabilities,

    A. Khare, S. Dutta, Z. Li, A. Solko-Breslin, R. Alur, and M. Naik, “Understanding the effectiveness of large language models in detecting security vulnerabilities,” inIEEE Conference on Software Testing, Verification and Validation, ICST 2025, Napoli, Italy, March 31 - April 4, 2025. IEEE, 2025, pp. 103–114. [Online]. Available: https://doi.org/10.1109/IC...

    doi:10.1109/icst62969.2025.10988968 2025

  36. [36]

    Llm4vuln: A unified evaluation framework for decoupling and enhancing llms’ vulnerability reasoning,

    Y . Sun, D. Wu, Y . Xue, H. Liu, W. Ma, L. Zhang, M. Shi, and Y . Liu, “Llm4vuln: A unified evaluation framework for decoupling and enhancing llms’ vulnerability reasoning,”CoRR, vol. abs/2401.16185,

    Pith/arXiv arXiv

  37. [37]

    Available: https://doi.org/10.48550/arXiv.2401.16185

    [Online]. Available: https://doi.org/10.48550/arXiv.2401.16185

    Pith/arXiv arXiv doi:10.48550/arxiv.2401.16185

  38. [38]

    Exploring chatgpt’s capabilities on vulnerability management,

    P. Liu, J. Liu, L. Fu, K. Lu, Y . Xia, X. Zhang, W. Chen, H. Weng, S. Ji, and W. Wang, “Exploring chatgpt’s capabilities on vulnerability management,” in33rd USENIX Security Symposium, USENIX Security 2024, Philadelphia, PA, USA, August 14-16, 2024, D. Balzarotti and W. Xu, Eds. USENIX Association, 2024. [Online]. Available: https: //www.usenix.org/confer...

    2024

  39. [39]

    Static detection of dos vulnerabilities in programs that use regular expressions,

    V . Wüstholz, O. Olivo, M. J. H. Heule, and I. Dillig, “Static detection of dos vulnerabilities in programs that use regular expressions,” in Tools and Algorithms for the Construction and Analysis of Systems - 23rd International Conference, TACAS 2017, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2017, Uppsala, ...

    work page doi:10.1007/978-3-662-54580-5_1 2017

  40. [40]

    Detecting DNS amplification attacks,

    G. Kambourakis, T. Moschos, D. Geneiatakis, and S. Gritzalis, “Detecting DNS amplification attacks,” inCritical Information Infrastructures Security, Second International Workshop, CRITIS 2007, Málaga, Spain, October 3-5, 2007. Revised Papers, ser. Lecture Notes in Computer Science, J. López and B. M. Hämmerli, Eds., vol. 5141. Springer, 2007, pp. 185–196...

    work page doi:10.1007/978-3-540-89173-4_16 2007

  41. [41]

    Nxnsattack: Recursive DNS inefficiencies and vulnerabilities,

    Y . Afek, A. Bremler-Barr, and L. Shafir, “Nxnsattack: Recursive DNS inefficiencies and vulnerabilities,” in29th USENIX Security Symposium, USENIX Security 2020, August 12-14, 2020, S. Capkun and F. Roesner, Eds. USENIX Association, 2020, pp. 631–648. [Online]. Available: https://www.usenix.org/conference/usenixsecurity20/presentation/afek

    2020

  42. [42]

    Revealer: Detecting and exploiting regular expression denial-of-service vulnerabilities,

    Y . Liu, M. Zhang, and W. Meng, “Revealer: Detecting and exploiting regular expression denial-of-service vulnerabilities,” in42nd IEEE Symposium on Security and Privacy, SP 2021, San Francisco, CA, USA, 24-27 May 2021. IEEE, 2021, pp. 1468–1484. [Online]. Available: https://doi.org/10.1109/SP40001.2021.00062

    doi:10.1109/sp40001.2021.00062 2021