Pith. sign in

REVIEW 3 major objections 5 minor 82 references

SAVANT: Vulnerability Detection in Application Dependencies through Semantic-Guided Reachability Analysis

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read SAVANT claims that LLM reflection over proof-of-vulnerability tests can decide whether a known vulnerable library API is actually exploitable in each application, reporting 83.8% precision and 73.8% recall on 55 Java projects.

desk verdict SAVANT is a plausible LLM-based SCA pipeline with a real circularity problem: the benchmark labels and the method's seeds come from the same proof-of-vulnerability artifacts, so the headline numbers likely measure retrieval of PoV-like code, not true exploitability. read the letter →

arxiv 2506.17798 v2 pith:MAOG6JWS submitted 2025-06-21 cs.SE cs.CR

Wang Lingxiang , Quanzhi Fu , Wenjia Song , Gelei Deng , Yi Liu , Dan Williams , Ying Zhang This is my paper · ORCID
classification cs.SEcs.CR
keywords softwarecompositionanalysisvulnerableAPIdetectionproof-of-vulnerabilitytestLLM-basedcodesemanticreachabilityASTsegmentationJavadependencysecuritysupplychain
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that software composition analysis can move from 'which library version is vulnerable' to 'is this application actually exploitable through a vulnerable library API.' It proposes SAVANT, a two-phase pipeline that first segments an application's Java source into AST-aligned code blocks and embeds them, then uses the vulnerable API's signature and its proof-of-vulnerability test as seeds to retrieve relevant code, iteratively asks an LLM to notice missing context, and finally asks the LLM to judge whether the vulnerable API is reachable and exploitable. On a third-party benchmark of 55 real-world Java applications, SAVANT reports 83.8% precision, 73.8% recall, 69.0% accuracy, and 78.5% F1, outperforming two existing SCA tools, which fail to analyze 17 and 35 projects respectively. The practical stake is that a correct SAVANT would let development teams ignore harmless version matches and focus fixes on genuinely reachable vulnerabilities.

What carries the argument

The load-bearing mechanism is the reflection loop over semantic retrieval. SAVANT maintains a database of AST-segmented code blocks paired with embeddings; for each candidate call site it loops between Q, an LLM reflection query that reports whether the current code context is sufficient and why not; P, an LLM code-inference query that names the missing code snippet and structural scope; and Search, embedding similarity with scope filtering. The proof-of-vulnerability test functions as the semantic anchor that tells the loop what 'triggered' looks like, replacing the work a call-graph reachability analysis would do.

What would settle it

Have an independent team produce ground-truth labels for the same 55 applications without seeing SAVANT's outputs or the PoV test functions, then rerun SAVANT with PoV tests withheld from the retrieval stage; if recall drops sharply, the PoV seeds were leaking the answer rather than providing generalizable semantics.

Watch

Extended reading notes

Core claim

SAVANT's central claim is that proof-of-vulnerability test cases, which security patches ship to show how a vulnerability is triggered, encode the semantic conditions of exploitability, and that an LLM can use those conditions to perform reachability analysis without a complete call graph. The pipeline starts from the vulnerable API signature and the PoV test, retrieves semantically similar source blocks by embedding cosine similarity, gates them with an LLM grader to confirm real API invocations, then iteratively reflects: the LLM says whether its context is sufficient, generates a query for missing code and scope constraints, retrieves those blocks, and repeats until context is complete or no new blocks are found. The final decision is disjunctive: the application is labeled vulnerable if any context-complete candidate makes the LLM answer 'vulnerable', and secure only if every candidate is judged safe. On the evaluation benchmark SAVANT identifies 31 of 42 vulnerable projects correctly plus 6 false positives, and the authors report F1 between 0.72 and 0.87 across six LLMs.

Load-bearing premise

The load-bearing premise is that the 55-project ground-truth labels, taken from the third-party benchmark plus the authors' manual audit, are correct and independent of the proof-of-vulnerability tests SAVANT is seeded with; if a label was derived from the same PoV test, the reported precision and recall would not transfer to fresh projects.

Editorial extensions

If this is right

  • Version-based scanners can be triaged: projects whose vulnerable dependency versions are flagged can be filtered by whether the specific API is invoked and whether context shows exploitable conditions, removing false positives like the safe BCryptPasswordEncoder use in Apache Kylin.
  • Call-graph-free reachability becomes feasible for large codebases; SAVANT analyzed all 55 projects while the baseline tools each failed on a large subset due to memory and classpath limitations.
  • The method's accuracy is coupled to LLM semantic ability and context completeness: across six models F1 spans 0.64 to 0.87, and segment sizes of 2,000 to 2,500 tokens give the best results.
  • The authors state the approach is theoretically language-agnostic, so the same two-phase pipeline could be applied to other ecosystems, though the paper only evaluates Java.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If SAVANT's advantage comes from PoV tests, then a corollary not tested in the paper is that libraries without PoV tests will see degraded recall; a natural extension is to synthesize PoV-like triggers from CVE descriptions or patch diffs.
  • The disjunctive final rule (any vulnerable candidate wins) explains the precision/recall balance and implies that a single hallucinated 'vulnerable' answer overrides many secure judgments; measuring per-candidate agreement across repeated LLM runs would quantify this fragility.
  • The method could be turned into a differential analyzer: run SAVANT on two consecutive commits of a project to isolate which dependency upgrade actually removes exploitable reachability, giving developers an upgrade-priority signal.
  • Because ground truth is the shaky plank, a strong extension is to build a new benchmark of projects whose exploitability is confirmed by executable tests rather than manual audit, then rerun the comparison.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes SAVANT, a two-phase LLM-based system for detecting whether Java applications are actually impacted by known vulnerabilities in their third-party library dependencies. Phase I segments application source code with an AST-based heuristic and embeds the segments; Phase II retrieves candidate code blocks using vulnerable API signatures and proof-of-vulnerability (PoV) test cases as seeds, iteratively expands context via LLM reflection, and asks an LLM to decide whether each candidate is exploitable. The evaluation on 55 Java projects from the Test Mimicry benchmark reports that SAVANT achieves 83.8% precision, 73.8% recall, 69.0% accuracy, and 78.5% F1, outperforming Eclipse Steady and VAScanner, with an additional ablation study across six LLMs, two embedding models, and several code-segment size thresholds.

Significance. If the empirical claims held, SAVANT would be a practically useful advance for software composition analysis, since it targets the well-known false-positive problem of version-based scanners and the scalability limitations of full call-graph reachability analysis. The motivating example (Kylin's safe use of BCryptPasswordEncoder) is well chosen, and the paper's architecture is clearly described. The per-project results in Table 2 are detailed enough to allow independent recomputation of SAVANT's own metrics: I confirmed precision 0.838, recall 0.738, accuracy 0.691, and F1 0.785 from the table, which is a strength. The ablation across LLMs and embedding models is also useful and gives a sense of robustness across model choices. However, the evaluation's ground-truth construction overlaps with the method's own seed signal, and the baseline comparison table is not internally consistent with the per-project results; these issues make the central 'outperforms state-of-the-art SCA tools' claim unverified as presented.

major comments (3)
  1. [§4.3.1, §4.3.3, §5.1] The evaluation is vulnerable to circularity. SAVANT's candidate identification in Eq. (5) is seeded by the proof-of-vulnerability test ftest, and the final reachability query in Section 4.3.3 also includes ftest. The ground-truth labels, however, come from Kang et al.'s Test Mimicry benchmark [34], whose labels are constructed by adapting library PoV tests to downstream projects. Thus the same type of artifact that underlies the benchmark labels is given to SAVANT as input. High agreement may therefore show that SAVANT retrieves code resembling a PoV test rather than that the 31 flagged projects are truly exploitable. The manual audit in Section 5.1, which traces from known vulnerable APIs and is performed by authors who know the benchmark labels, does not break this circularity. Please provide an evaluation on ground truth obtained without using PoV tests (for example, independent exploitability validation or a reachability-based labeling procedure), and include an ablation that removes the PoV seed while keeping only vulnerable API signatures as input.
  2. [Table 2 and Table 3] The headline comparison cannot be reproduced from the published per-project results. Using the stated convention of Table 2 (✓ secure, ✗ insecure, - failed) and excluding failed projects from the denominator, I recompute Eclipse Steady's precision as 15/19 = 0.789 and recall as 15/42 = 0.357, whereas Table 3 reports 0.700 precision and 0.241 recall. For VAScanner, the same exercise gives precision 13/15 = 0.867 and recall 13/42 = 0.310, not the reported 0.73 and 0.262. Other treatments of the '-' entries yield other values, but none match Table 3. The exact denominators, the treatment of failed runs, and the counting rule must be specified and the two tables reconciled before the claimed advantage over existing SCA tools can be assessed.
  3. [§4.3.1, Eq. (5), §6.1] The core similarity threshold τ is never disclosed. Eq. (5) defines candidate selection by the condition Sc > τ, and Section 6.1 admits that parameter sensitivity has not been analyzed and is left to future work. Because τ directly controls which code blocks enter the pipeline, the reported 83.8% precision and 78.5% F1 are not reproducible without this value. In addition, no repeated LLM runs or confidence intervals are reported; given the known non-determinism of LLM-based analysis (e.g., SecLLMHolmes [64]), please report the value of τ, provide a sensitivity sweep over τ, and give repeated-run statistics for the headline metrics.
minor comments (5)
  1. [Figure 1] The figure contains typos: 'Candidata Identification', 'Relection-based Context Retrival', and 'Semantic-guilded Reachability Analysis' should be corrected.
  2. [§4.3.2, §4.3.3, Abstract] There are several language issues: the abstract says 'we proposed SAVANT' where 'we propose' is intended; Section 4.3.2 says 'SAVANT querys' instead of 'queries'; and the Section 4.3.3 heading reads 'Sematic-Guided Reachability Analysis'.
  3. [§4.3.1, Eq. (5)] The notation in Eq. (5) is inconsistent with the database definition in Section 4.2.2: D stores tuples di = (bi, vi), but Ccandidate is written as ci = (vi, bi, vapi) ∈ D. Please align the tuple definitions.
  4. [Table 2] The caption does not state how '-' (tool failed to run) entries were counted when computing the metrics in Table 3; even after the numbers are corrected, the counting rule should be explicit in the caption.
  5. [Figure 3] The legend distinguishes embedding models only by bar color ('green bars' vs 'purple bars'), which is not legible in grayscale; adding direct labels or hatching would improve readability.

Circularity Check

1 steps flagged · score 4.0 of 10

PoV-test seed signal overlaps the Test Mimicry benchmark labels, so the headline metrics partly reward reproducing the label-generation signal; the inference pipeline itself is not fitted to the labels.

  1. other [Section 4.3.1 (Equation 5) and Section 5.1 (Benchmark Dataset)]
    "SAVANT begins by leveraging a dual-seed approach: (1) vulnerable APIs and (2) PoV test cases. ... Ccandidate ={ci = (vi,bi,vapi )| ci∈ D ∧ (Sc(vapi ,vi) > τ∨ Sc(vtest ,vi) > τ) ∧ LLMgrade(bi,bapi ) = "yes"} (5) ... we utilized a third-party benchmark [34] comprising 25 distinct vulnerabilities across 55 open-source applications."

    The cited benchmark [34] is Kang et al.'s 'Test mimicry' dataset: by its title, exploitability labels are produced by mimicking the library's proof-of-vulnerability tests in downstream projects. SAVANT is given exactly those PoV tests as retrieval seeds (Eq. 5 includes Sc(vtest, vi) > tau) and again in the final decision query Qi=(ci, API, ftest, R, T). The label-generation signal and the method's input signal are therefore the same PoV test, so a high F1 can partly mean SAVANT is good at reproducing Test Mimicry's PoV-based labels rather than independently confirming that projects are 'actually impacted.' The manual audit mitigates this only partially because it traces from the same known vulnerable APIs and is not reported as blind to SAVANT's outputs.

full rationale

There is no fitted-parameter circularity: SAVANT does not train on the 55-project labels, and no equation in the paper reuses a label as a prediction input. The inference chain (embedding retrieval, LLM verification, reflection-based context expansion, binary decision) is self-contained. The circularity risk is concentrated in the evaluation construct: the benchmark [34] is a Test Mimicry dataset whose labels are generated from the same PoV tests that SAVANT consumes as seeds. Because the paper's central empirical claim (83.8% precision, 73.8% recall) is measured against those labels, the PoV overlap is load-bearing for the headline result. This is not a self-citation chain or a renamed known result, and the ablation study remains independent evidence of configuration sensitivity. The severity is moderate: the method is a genuine semantic-guided reachability approach, but the benchmark cannot fully separate 'matches the PoV-based label' from 'actually exploitable.'

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The paper introduces no physical or abstract entity beyond the software pipeline itself. The load-bearing assumptions are that PoV tests are faithful, that benchmark labels are correct, and that LLM reasoning is a reliable oracle. Two numerical parameters, tau and theta, are tuned or undisclosed and directly affect the reported results.

free parameters (2)
  • tau (embedding similarity threshold)
    Used in filter F1 (Eq. 5) to select candidate code blocks. No numerical value or selection procedure is reported anywhere, even though Section 6.1 says parameter sensitivity will be studied later.
  • theta (maximum code segment size) = not stated for headline run; optimal claimed at 2000-2500 tokens
    Used in the AST segmentation rule in Section 4.2.1. Section 5.4 sweeps values from 500 to 3000 and reports the best F1 around 2000-2500 tokens, but the default value used for the Table 3 results is not specified, so the headline numbers may depend on a tuned choice.
assumptions (3)
  • domain assumption Proof-of-vulnerability tests shipped with security patches precisely capture the vulnerable API usage patterns for each CVE.
    This is the core seed signal for SAVANT (Section 4.3.1) and is asserted in Section 1. If PoV tests are missing, stale, or not representative, candidate identification and final judgment degrade.
  • domain assumption Ground-truth labels for the 55 projects (third-party benchmark plus authors' manual audit) are correct.
    Section 5.1 defines labels from [34] and a manual audit; all reported metrics are computed against these labels.
  • domain assumption LLM judgments about code semantics and exploitability are sufficiently reliable to act as a vulnerability oracle.
    Sections 4.3.2 and 4.3.3 delegate context-completeness and final reachability decisions to LLMs, with no independent verification of the LLM's reasoning.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SAVANT: Vulnerability Detection in Application Dependencies through Semantic-Guided Reachability Analysis." pith.science (2026). https://pith.science/paper/MAOG6JWS

@misc{pith2026250617798,
  author       = {Pith},
  title        = {Pith review of: SAVANT: Vulnerability Detection in Application Dependencies through Semantic-Guided Reachability Analysis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MAOG6JWS}},
  note         = {Machine review of arXiv:2506.17798}
}
read the original abstract

The integration of open-source third-party library dependencies in Java development introduces significant security risks when these libraries contain known vulnerabilities. Existing Software Composition Analysis (SCA) tools struggle to effectively detect vulnerable API usage from these libraries due to limitations in understanding API usage semantics and computational challenges in analyzing complex codebases, leading to inaccurate vulnerability alerts that burden development teams and delay critical security fixes. To address these challenges, we proposed SAVANT by leveraging two insights: proof-of-vulnerability test cases demonstrate how vulnerabilities can be triggered in specific contexts, and Large Language Models (LLMs) can understand code semantics. SAVANT combines semantic preprocessing with LLM-powered context analysis for accurate vulnerability detection. SAVANT first segments source code into meaningful blocks while preserving semantic relationships, then leverages LLM-based reflection to analyze API usage context and determine actual vulnerability impacts. Our evaluation on 55 real-world applications shows that SAVANT achieves 83.8% precision, 73.8% recall, 69.0% accuracy, and 78.5% F1-score, outperforming state-of-the-art SCA tools.

Figures

Figures reproduced from arXiv: 2506.17798 by the authors.

Figure 1
Figure 1. The Overview of SAVANT (1) efficiently analyzing large codebases without relying on complete call graph construction, and (2) understanding the semantic context of API invocations to determine their true security implications. SAVANT leverages LLMs for seman￾tic code analysis, enabling accurate vulnerability assessment even in a complex codebase. 4 Design 4.1 Overview To address the limitations of existing approache… view at source ↗
Figure 2
Figure 2. An example of context-complete code retrieval for BCryptPasswordEncoder.encode() method. [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Performance comparison of different LLMs and embedding models across varying maximum code lengths. The legend [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

82 extracted references · 78 canonical work pages

  1. [34]

    Large language model-powered smart contract vulnerability detection: New perspectives

    Sihao Hu, Tiansheng Huang, Fatih ˙Ilhan, Selim Furkan Tekin, and Ling Liu. Large language model-powered smart contract vulnerability detection: New perspectives. In 2023 5th IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applica- tions (TPS-ISA), pages 297–306. IEEE, 2023. 13

    work page 2023

  2. [64]

    SonarQube

    SonarSource. SonarQube. https://www.sonarqube. org/, 2021. Accessed: 2024-01-22

    work page 2021

  3. [1]

    https://nvd.nist.gov/ vuln/detail/CVE-2020-5408, May 2020

    CVE-2020-5408 Detail. https://nvd.nist.gov/ vuln/detail/CVE-2020-5408, May 2020. Accessed: 2025-01-19

    2020

  4. [2]

    https://owasp.org/ www-project-dependency-check/ , 2020

    OWASP Dependency-Check. https://owasp.org/ www-project-dependency-check/ , 2020

    2020

  5. [3]

    https: //www.csoonline.com/article/3191947/ supply-chain-attacks-show-why-you-should-be-wary-of-third-party-providers

    Supply chain attacks show why you should be wary of third-party providers. https: //www.csoonline.com/article/3191947/ supply-chain-attacks-show-why-you-should-be-wary-of-third-party-providers. html, 2021

    work page arXiv 2021

  6. [4]

    https://docs.npmjs.com/cli/v9/ commands/npm-audit, 2023

    npm-audit. https://docs.npmjs.com/cli/v9/ commands/npm-audit, 2023

    work page 2023

  7. [5]

    https://nvd.nist.gov, 2023

    NVD. https://nvd.nist.gov, 2023

    work page 2023

  8. [6]

    https://docs.snyk.io/ snyk-cli/commands/test, 2023

    Test - Snyk User Docs. https://docs.snyk.io/ snyk-cli/commands/test, 2023

    work page 2023

Show all 82 references
  1. [7]

    Meta llama 3.1 announcement

    Meta AI. Meta llama 3.1 announcement. https://ai. meta.com/blog/meta-llama-3-1/ , 2024. Accessed: 2024-10-30

    2024

  2. [8]

    V oyage code 3.https://blog.voyageai

    V oyage AI. V oyage code 3.https://blog.voyageai. com/2024/12/04/voyage-code-3/. Published: 2024- 12-04, Accessed: 2025-01-20

    2024

  3. [9]

    Claude sonnet

    Anthropic. Claude sonnet. https://www.anthropic. com/claude/sonnet, 2024. Accessed: 2024-10-30

    2024

  4. [10]

    MySqlJdbcUrl.java

    Apache Software Foundation. MySqlJdbcUrl.java. https://github.com/apache/gobblin/blob/ 44a7e1a27cc73387cf309487f45895801984059d/ gobblin-metastore/src/main/java/org/apache/ gobblin/metastore/util/MySqlJdbcUrl.java# L51, 2023. Accessed: 2024-01-22

    2023

  5. [11]

    Apache Log4j ™

    Apache Software Foundation. Apache Log4j ™

  6. [12]

    https://logging.apache.org/log4j/2.x/,

  7. [13]

    User- Controller.java - Apache Kylin

    Apache Software Foundation. User- Controller.java - Apache Kylin. https: //github.com/apache/kylin/blob/ 443c2523e27e86ed397c526f741db62a805b95c4/ server-base/src/main/java/org/apache/kylin/ rest/controller/UserController.java, 2025. Accessed: 2025-01-19

    2025

  8. [14]

    Kyli- nUserService.java - Apache Kylin

    Apache Software Foundation. Kyli- nUserService.java - Apache Kylin. https: //github.com/apache/kylin/blob/ 443c2523e27e86ed397c526f741db62a805b95c4/ server-base/src/main/java/org/apache/kylin/ 12 rest/service/KylinUserService.java, 2025. Accessed: 2025-01-19

    2025

  9. [15]

    Cryptollm: Harnessing the power of llms to detect cryp- tographic api misuse

    Heewon Baek, Minwook Lee, and Hyoungshick Kim. Cryptollm: Harnessing the power of llms to detect cryp- tographic api misuse. In European Symposium on Re- search in Computer Security, pages 353–373. Springer, 2024

    2024

  10. [16]

    Self-rag: Self-reflective retrieval augmented generation

    Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. Self-rag: Self-reflective retrieval augmented generation. arXiv preprint arXiv:2310.11511, 2023

    2023 arXiv

  11. [17]

    When Large Language Models Confront Repository-Level Automatic Program Repair: How Well They Done?

    Yuxiao Chen, Jingzheng Wu, Xiang Ling, Changjiang Li, Zhiqing Rui, Tianyue Luo, and Yanjun Wu. When Large Language Models Confront Repository-Level Automatic Program Repair: How Well They Done? . In 2024 IEEE/ACM 46th International Conference on Software Engineering: Companion...

    2024

  12. [18]

    Coca: Improving and explain- ing graph neural network-based vulnerability detection systems

    Sicong Cao, Xiaobing Sun, Xiaoxue Wu, David Lo, Lili Bo, Bin Li, and Wei Liu. Coca: Improving and explain- ing graph neural network-based vulnerability detection systems. In Proceedings of the IEEE/ACM 46th Interna- tional Conference on Software Engineering, ICSE ’24, 2024

    2024

  13. [19]

    Bodin Chinthanet, Serena Elisa Ponta, Henrik Plate, An- tonino Sabetta, Raula Gaikovina Kula, Takashi Ishio, and Kenichi Matsumoto. Code-based vulnerability de- tection in node.js applications: How far are we? In Proceedings of the 35th IEEE/ACM International Con- ference on A...

    2021

  14. [20]

    How about bug-triggering paths?- understanding and characterizing learning-based vulner- ability detectors

    Xiao Cheng, Xu Nie, Ningke Li, Haoyu Wang, Zheng Zheng, and Yulei Sui. How about bug-triggering paths?- understanding and characterizing learning-based vulner- ability detectors. IEEE Transactions on Dependable and Secure Computing, 21(2):542–558, 2022

    2022

  15. [21]

    {PentestGPT}: Evaluating and harnessing large language models for automated penetration testing

    Gelei Deng, Yi Liu, Víctor Mayoral-Vilches, Peng Liu, Yuekang Li, Yuan Xu, Tianwei Zhang, Yang Liu, Martin Pinzger, and Stefan Rass. {PentestGPT}: Evaluating and harnessing large language models for automated penetration testing. In 33rd USENIX Security Sympo- sium (USENIX Sec...

    2024

  16. [22]

    Bugtreemodel.java

    FindBugs Project Contributors. Bugtreemodel.java. https://github. com/tonydamage/findbugs/blob/ fd7ec8b5cc0b1b143589674cdcdb901fa5dc0dda/ findbugs/src/gui/edu/umd/cs/findbugs/gui2/ BugTreeModel.java, 2023. Accessed: 2024-10-30

    2023

  17. [23]

    Llmeffichecker: Understanding and testing ef- ficiency degradation of large language models

    Xiaoning Feng, Xiaohong Han, Simin Chen, and Wei Yang. Llmeffichecker: Understanding and testing ef- ficiency degradation of large language models. ACM Transactions on Software Engineering and Methodol- ogy, 2024

    2024

  18. [24]

    Eclipse Steady

    Eclipse Foundation. Eclipse Steady. https: //projects.eclipse.org/projects/technology. steady, 2024. Accessed: 2024-01-22

    2024

  19. [25]

    Stack overflow considered harmful? the impact of copy&paste on android application security

    Felix Fischer, Konstantin Böttinger, Huang Xiao, Chris- tian Stransky, Yasemin Acar, Michael Backes, and Sascha Fahl. Stack overflow considered harmful? the impact of copy&paste on android application security. In 2017 IEEE Symposium on Security and Privacy (SP), pages 121–136...

    2017

  20. [26]

    Find Security Bugs

    Find Security Bugs Project. Find Security Bugs. https://find-sec-bugs.github.io/, 2021. Ac- cessed: 2024-01-22

    2021

  21. [27]

    Linevul: A transformer-based line-level vulnerability prediction

    Michael Fu and Chakkrit Tantithamthavorn. Linevul: A transformer-based line-level vulnerability prediction. In Proceedings of the 19th International Conference on Mining Software Repositories, pages 608–620, 2022

    2022

  22. [28]

    Jira issue: Httpclient-

    Apache Software Foundation. Jira issue: Httpclient-

  23. [29]

    Gemini ai

    Google. Gemini ai. https://gemini.google.com/. Accessed: 2025-01-20

    2025

  24. [30]

    Linevd: statement-level vulnerability detection using graph neural networks

    David Hin, Andrey Kan, Huaming Chen, and M Ali Babar. Linevd: statement-level vulnerability detection using graph neural networks. In Proceedings of the 19th international conference on mining software reposito- ries, pages 596–607, 2022

    2022

  25. [31]

    Detecting suspicious pack- age updates

    Kalil Garrett, Gabriel Ferreira, Limin Jia, Joshua Sun- shine, and Christian Kästner. Detecting suspicious pack- age updates. In 2019 IEEE/ACM 41st International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER) , pages 13–16. IEEE, 2019

    2019

  26. [32]

    An Empiri- cal Study on Fine-Tuning Large Language Models of Code for Automated Program Repair

    Kai Huang, Xiangxin Meng, Jian Zhang, Yang Liu, Wen- jie Wang, Shuhao Li, and Yuqing Zhang. An Empiri- cal Study on Fine-Tuning Large Language Models of Code for Automated Program Repair . In 2023 38th IEEE/ACM International Conference on Automated Soft- ware Engineering (ASE)...

    2023

  27. [33]

    Md Mahir Asef Kabir, Ying Wang, Danfeng Yao, and Na Meng. How do developers follow security-relevant best practices when using npm packages? In 2022 IEEE Secure Development Conference (SecDev), pages 77– 83, Los Alamitos, CA, USA, oct 2022. IEEE Computer Society

    2022

  28. [35]

    Cog- nicrypt: supporting developers in using cryptography

    Stefan Krüger, Sarah Nadi, Michael Reif, Karim Ali, Mira Mezini, Eric Bodden, Florian Göpfert, Felix Gün- ther, Christian Weinert, Daniel Demmler, et al. Cog- nicrypt: supporting developers in using cryptography. In 2017 32nd IEEE/ACM International Conference on Automated Soft...

    2017

  29. [36]

    Towards the detection of malicious java packages

    Piergiorgio Ladisa, Henrik Plate, Matias Martinez, Olivier Barais, and Serena Elisa Ponta. Towards the detection of malicious java packages. In Proceedings of the 2022 ACM Workshop on Software Supply Chain Of- fensive Research and Ecosystem Defenses, pages 63–72, 2022

    2022

  30. [37]

    Test mimicry to assess the exploitability of library vulnerabilities

    Hong Jin Kang, Truong Giang Nguyen, Bach Le, Co- rina S P˘as˘areanu, and David Lo. Test mimicry to assess the exploitability of library vulnerabilities. In Proceed- ings of the 31st ACM SIGSOFT International Sympo- sium on Software Testing and Analysis, pages 276–288, 2022

    2022

  31. [38]

    Yi Li, Shaohua Wang, and Tien N. Nguyen. Vulner- ability detection with fine-grained interpretations. In Proceedings of the 29th ACM Joint Meeting on Euro- pean Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2021, page 292–303, 2021

    2021

  32. [39]

    Sysevr: A framework for using deep learning to detect software vulnerabilities

    Zhen Li, Deqing Zou, Shouhuai Xu, Hai Jin, Yawei Zhu, and Zhaoxuan Chen. Sysevr: A framework for using deep learning to detect software vulnerabilities. IEEE Transactions on Dependable and Secure Computing , 19(4):2244–2258, 2021

    2021

  33. [40]

    A lightweight assisted vulnerability discovery method using deep neural networks

    Runhao Li, Chao Feng, Xing Zhang, and Chaojing Tang. A lightweight assisted vulnerability discovery method using deep neural networks. IEEE Access, 7:80079– 80092, 2019

    2019

  34. [41]

    Vuldeep- ecker: A deep learning-based system for vulnerability detection

    Zhen Li, Deqing Zou, Shouhuai Xu, Xinyu Ou, Hai Jin, Sujuan Wang, Zhijun Deng, and Yuyi Zhong. Vuldeep- ecker: A deep learning-based system for vulnerability detection. In Network and Distributed Systems Security (NDSS) Symposium, 2018

    2018

  35. [42]

    Exploring{ChatGPT’s} capa- bilities on vulnerability management

    Peiyu Liu, Junming Liu, Lirong Fu, Kangjie Lu, Yifan Xia, Xuhong Zhang, Wenzhi Chen, Haiqin Weng, Shoul- ing Ji, and Wenhai Wang. Exploring{ChatGPT’s} capa- bilities on vulnerability management. In 33rd USENIX Security Symposium (USENIX Security 24), pages 811– 828, 2024

    2024

  36. [43]

    SySeVR: A Framework for Us- ing Deep Learning to Detect Software Vulnerabilities

    Zhen Li, Deqing Zou, Shouhuai Xu, Hai Jin, Yawei Zhu, and Zhaoxuan Chen. SySeVR: A Framework for Us- ing Deep Learning to Detect Software Vulnerabilities . IEEE Transactions on Dependable and Secure Comput- ing, 19(04):2244–2258, July 2022

    2022

  37. [44]

    Dependency- check issue #4629: False positive detection of cve-2020-5408

    Jeremy Long and contributors. Dependency- check issue #4629: False positive detection of cve-2020-5408. https://github.com/jeremylong/ DependencyCheck/issues/4629, September 2023. Accessed: 2025-01-19

    2020

  38. [45]

    Dependency- check issue #6685: False positive detection of cve- 2023-4759

    Jeremy Long and contributors. Dependency- check issue #6685: False positive detection of cve- 2023-4759. https://github.com/jeremylong/ DependencyCheck/issues/6685, January 2025. Ac- cessed: 2025-01-19

    2023

  39. [46]

    Prompt Fix: Vulnerability Automatic Repair Technology Based on Prompt Engineering

    Peng Liu, He Wang, Chen Zheng, and Yuqing Zhang. Prompt Fix: Vulnerability Automatic Repair Technology Based on Prompt Engineering . In 2024 International Conference on Computing, Networking and Communi- cations (ICNC), pages 116–120, February 2024

    2024

  40. [47]

    VulChecker: Graph- based vulnerability localization in source code

    Yisroel Mirsky, George Macon, Michael Brown, Carter Yagemann, Matthew Pruett, Evan Downing, Sukarno Mertoguno, and Wenke Lee. VulChecker: Graph- based vulnerability localization in source code. In 32nd USENIX Security Symposium (USENIX Security 23), pages 6557–6574, Anaheim, C...

    2023

  41. [48]

    To- wards detection of software supply chain attacks by forensic artifacts

    Marc Ohm, Arnold Sykosch, and Michael Meier. To- wards detection of software supply chain attacks by forensic artifacts. In Proceedings of the 15th interna- tional conference on availability, reliability and security, pages 1–6, 2020

    2020

  42. [49]

    Grace: Empowering llm-based soft- ware vulnerability detection with graph structure and in-context learning

    Guilong Lu, Xiaolin Ju, Xiang Chen, Wenlong Pei, and Zhilong Cai. Grace: Empowering llm-based soft- ware vulnerability detection with graph structure and in-context learning. Journal of Systems and Software, 212:112031, 2024

    2024

  43. [50]

    Embedding models: Openai docu- mentation

    OpenAI. Embedding models: Openai docu- mentation. https://platform.openai.com/docs/ guides/embeddings/embedding-models, 2024. Ac- cessed: 2024-10-30

    2024

  44. [51]

    Model overview

    OpenAI. Model overview. https://platform. openai.com/docs/models, 2024. Accessed: 2024-10- 30

    2024

  45. [52]

    Embeddings guide

    OpenAI. Embeddings guide. https://platform. openai.com/docs/guides/embeddings. Accessed: 2025-01-20. 14

    2025

  46. [53]

    Examining Zero-Shot Vulnerability Repair with Large Language Models

    Hammond Pearce, Benjamin Tan, Baleegh Ahmad, Ramesh Karri, and Brendan Dolan-Gavitt. Examining Zero-Shot Vulnerability Repair with Large Language Models . In 2023 IEEE Symposium on Security and Privacy (SP), pages 2339–2356, May 2023

    2023

  47. [54]

    Detection, assessment and mitigation of vulnerabilities in open source dependencies

    Serena Elisa Ponta, Henrik Plate, and Antonino Sabetta. Detection, assessment and mitigation of vulnerabilities in open source dependencies. Empirical Software Engi- neering, 25(5):3175–3215, 2020

    2020

  48. [55]

    OWASP Dependency-Check

    OWASP Foundation. OWASP Dependency-Check. https://owasp.org/ www-project-dependency-check/ , 2024. Ac- cessed: 2024-01-22

    2024

  49. [56]

    Cryptoguard: High precision detection of cryptographic vulnerabilities in massive- sized java projects

    Sazzadur Rahaman, Ya Xiao, Sharmin Afrose, Fahad Shaon, Ke Tian, Miles Frantz, Murat Kantarcioglu, and Danfeng (Daphne) Yao. Cryptoguard: High precision detection of cryptographic vulnerabilities in massive- sized java projects. In Proceedings of the 2019 ACM SIGSAC Conference...

    2019

  50. [57]

    Xanitizer

    RIGS IT. Xanitizer. https://www.rigs-it.com/ xanitizer/, 2021. Accessed: 2024-01-22

    2021

  51. [58]

    Cryptoguard: High precision detection of cryptographic vulnerabilities in massive-sized java projects

    Sazzadur Rahaman, Ya Xiao, Sharmin Afrose, Fahad Shaon, Ke Tian, Miles Frantz, Murat Kantarcioglu, and Danfeng Yao. Cryptoguard: High precision detection of cryptographic vulnerabilities in massive-sized java projects. In Proceedings of the 2019 ACM SIGSAC Conference on Comput...

    2019

  52. [59]

    VulSim: Leveraging similarity of Multi-Dimensional neighbor embeddings for vulnerability detection

    Samiha Shimmi, Ashiqur Rahman, Mohan Gadde, Hamed Okhravi, and Mona Rahimi. VulSim: Leveraging similarity of Multi-Dimensional neighbor embeddings for vulnerability detection. In 33rd USENIX Security Symposium (USENIX Security 24), pages 1777–1794, Philadelphia, PA, August 202...

    2024

  53. [60]

    Cryptotutor: Teaching secure coding practices through misuse pattern detection

    Larry Singleton, Rui Zhao, Myoungkyu Song, and Har- vey Siy. Cryptotutor: Teaching secure coding practices through misuse pattern detection. In Proceedings of the 21st Annual Conference on Information Technology Education, pages 403–408, 2020

    2020

  54. [61]

    FVD-DPM: Fine- grained vulnerability detection via conditional diffusion probabilistic models

    Miaomiao Shao and Yuxin Ding. FVD-DPM: Fine- grained vulnerability detection via conditional diffusion probabilistic models. In 33rd USENIX Security Sympo- sium (USENIX Security 24), pages 7375–7392, Philadel- phia, PA, August 2024. USENIX Association

    2024

  55. [62]

    State of the software supply chain report 2024: 10-year review

    Sonatype. State of the software supply chain report 2024: 10-year review. https://www.sonatype.com/ state-of-the-software-supply-chain/2024/ 10-year-look, 2024. Accessed: 2024-01-13

    2024

  56. [63]

    Dataflow analysis-inspired deep learning for efficient vulnerability detection

    Benjamin Steenhoek, Hongyang Gao, and Wei Le. Dataflow analysis-inspired deep learning for efficient vulnerability detection. In Proceedings of the 46th IEEE/ACM International Conference on Software Engi- neering, pages 1–13, 2024

    2024

  57. [65]

    Llms cannot reliably identify and reason about security vulner- abilities (yet?): A comprehensive evaluation, framework, and benchmarks

    Saad Ullah, Mingji Han, Saurabh Pujar, Hammond Pearce, Ayse Coskun, and Gianluca Stringhini. Llms cannot reliably identify and reason about security vulner- abilities (yet?): A comprehensive evaluation, framework, and benchmarks. In IEEE Symposium on Security and Privacy, 2024

    2024

  58. [66]

    V AScanner.https: //github.com/VAScanner/VAScanner, 2024

    V AScanner Project Contributors. V AScanner.https: //github.com/VAScanner/VAScanner, 2024. Ac- cessed: 2024-01-22

    2024

  59. [67]

    LLMs Cannot Reliably Identify and Reason About Security Vulnerabilities (Yet?): A Comprehensive Evaluation, Framework, and Benchmarks

    Saad Ullah, Mingji Han, Saurabh Pujar, Hammond Pearce, Ayse Coskun, and Gianluca Stringhini. LLMs Cannot Reliably Identify and Reason About Security Vulnerabilities (Yet?): A Comprehensive Evaluation, Framework, and Benchmarks . In 2024 IEEE Sympo- sium on Security and Privacy...

    2024

  60. [68]

    How effective are neural networks for fixing security vulnerabilities

    Yi Wu, Nan Jiang, Hung Viet Pham, Thibaud Lutellier, Jordan Davis, Lin Tan, Petr Babkin, and Sameena Shah. How effective are neural networks for fixing security vulnerabilities. In Proceedings of the 32nd ACM SIG- SOFT International Symposium on Software Testing and Analysis, ...

    2023

  61. [69]

    Understanding the threats of upstream vulnerabilities to downstream projects in the maven 15 ecosystem

    Yulun Wu, Zeliang Yu, Ming Wen, Qiang Li, Deqing Zou, and Hai Jin. Understanding the threats of upstream vulnerabilities to downstream projects in the maven 15 ecosystem. In 2023 IEEE/ACM 45th International Con- ference on Software Engineering (ICSE), pages 1046–

    2023

  62. [70]

    Copiloting the copilots: Fusing large language models with completion engines for automated program repair

    Yuxiang Wei, Chunqiu Steven Xia, and Lingming Zhang. Copiloting the copilots: Fusing large language models with completion engines for automated program repair. In Proceedings of the 31st ACM Joint European Soft- ware Engineering Conference and Symposium on the Foundations of ...

    2023

  63. [71]

    Tracer: Finding patches for open source software vulnerabilities

    Congying Xu, Bihuan Chen, Chenhao Lu, Kaifeng Huang, Xin Peng, and Yang Liu. Tracer: Finding patches for open source software vulnerabilities. arXiv preprint arXiv:2112.02240, 2021

    2021 arXiv

  64. [72]

    Analyzing cryptographic api usages for android appli- cations using hmm and n-gram

    Zhiwu Xu, Xiongya Hu, Yida Tao, and Shengchao Qin. Analyzing cryptographic api usages for android appli- cations using hmm and n-gram. In 2020 International Symposium on Theoretical Aspects of Software Engi- neering (TASE), pages 153–160. IEEE, 2020

    2020

  65. [73]

    Automated Program Repair in the Era of Large Pre- trained Language Models

    Chunqiu Steven Xia, Yuxiang Wei, and Lingming Zhang. Automated Program Repair in the Era of Large Pre- trained Language Models . In 2023 IEEE/ACM 45th In- ternational Conference on Software Engineering (ICSE), pages 1482–1494, May 2023

    2023

  66. [74]

    Mitigating persistence of open-source vulnerabilities in maven ecosystem

    Lyuye Zhang, Chengwei Liu, Sen Chen, Zhengzi Xu, Lingling Fan, Lida Zhao, Yiran Zhang, and Yang Liu. Mitigating persistence of open-source vulnerabilities in maven ecosystem. In 2023 38th IEEE/ACM Interna- tional Conference on Automated Software Engineering (ASE), pages 191–20...

    2023

  67. [75]

    Automatic detection of java crypto- graphic api misuses: Are we there yet? IEEE Transac- tions on Software Engineering, 49(1):288–303, 2022

    Ying Zhang, Md Mahir Asef Kabir, Ya Xiao, Danfeng Yao, and Na Meng. Automatic detection of java crypto- graphic api misuses: Are we there yet? IEEE Transac- tions on Software Engineering, 49(1):288–303, 2022

    2022

  68. [76]

    Does the vulnerability threaten our projects? automated vulnerable api detec- tion for third-party libraries

    Fangyuan Zhang, Lingling Fan, Sen Chen, Miaoying Cai, Sihan Xu, and Lida Zhao. Does the vulnerability threaten our projects? automated vulnerable api detec- tion for third-party libraries. IEEE Transactions on Software Engineering, 2024

    2024

  69. [77]

    Software composition analysis for vulnerability detection: An em- pirical study on java projects

    Lida Zhao, Sen Chen, Zhengzi Xu, Chengwei Liu, Lyuye Zhang, Jiahui Wu, Jun Sun, and Yang Liu. Software composition analysis for vulnerability detection: An em- pirical study on java projects. In Proceedings of the 31st ACM Joint European Software Engineering Con- ference and S...

    2023

  70. [78]

    Large language model for vulnerability detection: Emerging results and future directions

    Xin Zhou, Ting Zhang, and David Lo. Large language model for vulnerability detection: Emerging results and future directions. In Proceedings of the 2024 ACM/IEEE 44th International Conference on Software Engineering: New Ideas and Emerging Results, pages 47–51, 2024

    2024

  71. [79]

    Example-based vul- nerability detection and repair in java code

    Ying Zhang, Ya Xiao, Md Mahir Asef Kabir, Dan- feng (Daphne) Yao, and Na Meng. Example-based vul- nerability detection and repair in java code. In Proceed- ings of the 30th IEEE/ACM International Conference on Program Comprehension, ICPC ’22, pages 190–201, New York, NY , USA,...

    2022

  72. [82]

    muVulDeePecker: A Deep Learning-Based System for Multiclass Vulnerability Detection

    Deqing Zou, Sujuan Wang, Shouhuai Xu, Zhen Li, and Hai Jin. muVulDeePecker: A Deep Learning-Based System for Multiclass Vulnerability Detection . IEEE Transactions on Dependable and Secure Computing , 18(05):2224–2236, September 2021. 16

    2021

  73. [1803]

    Accessed: 2024-10-24

    https://issues.apache.org/jira/browse/ HTTPCLIENT-1803, 2017. Accessed: 2024-10-24

    2017

  74. [2024]

    Accessed: 2024-01-22

    2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.