Pith. sign in

REVIEW 3 major objections 6 minor 65 references

SVA-ICL: Improving LLM-based Software Vulnerability Assessment via In-Context Learning and Information Fusion

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Fused code-description retrieval lifts LLM severity scoring to 77%

desk verdict Honest, well-run benchmark for ICL-based vulnerability severity assessment, but every number sits on recrawled CVSS v3 labels that were never validated; referee it, but make label validation a hard requirement. read the letter →

arxiv 2505.10008 v2 pith:WPDSYIIJ submitted 2025-05-15 cs.SE

Chaoyang Gao , Xiang Chen , Guangbei Zhang This is my paper · ORCID
classification cs.SE
keywords softwarevulnerabilityassessmentin-contextlearninglargelanguagemodelinformationfusioncodesimilaritydescriptionCVSSv3demonstrationselection
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a large language model can assess software vulnerability severity well using in-context learning, provided the demonstration examples are selected by fusing source-code similarity with vulnerability-description textual similarity. On a self-built dataset of 12,071 C/C++ vulnerabilities labeled with CVSS v3 severity, the proposed SVA-ICL approach reports 77.07% accuracy, 67.89% F1-score, and 63.01% MCC, outperforming nine baselines including bimodal models. The key claim is that demonstration quality, not model capacity, drives the gain, so no fine-tuning is needed. A reader should care because this offers a practical, low-cost path to automated vulnerability triage using off-the-shelf LLM APIs.

What carries the argument

The load-bearing mechanism is a two-level fused similarity score used for demonstration retrieval: Sim(Va, Vb) = φ × CodeSim(A, B) + (1 − φ) × TextSim(Da, Db), where TextSim comes from a CoSENT text-embedding model, and CodeSim itself blends syntactic similarity (Levenshtein distance over tree-sitter AST sequences) with lexical similarity (Jaccard overlap of code tokens). Semantic retrieval is pre-computed via CodeBERT embeddings transformed by BERT-whitening, so online cost per query is low. This module selects the top-k examples that are then inserted into a structured prompt, which is fed to DeepSeek-V2 with temperature set to zero.

What would settle it

Take a random sample of 1,000 entries from the released dataset and compare the paper's recrawled severity labels against the official NVD CVSS v3 base scores; if agreement is below 95%, recompute SVA-ICL's accuracy on the corrected subset to see how much of the reported 77.07% persists. A second check is to run SVA-ICL on vulnerabilities disclosed after DeepSeek-V2's training cutoff with the vulnerability description removed, so only code remains; if accuracy drops sharply, part of the gain comes from the LLM's memorized CVE knowledge rather than retrieval quality.

Watch

Extended reading notes

Core claim

The central claim is that combining code similarity (semantic, lexical, and syntactic) with vulnerability-description textual similarity, fused at a 70% code to 30% description ratio, yields higher-quality in-context demonstrations than any single modality, and that these demonstrations are what make the LLM accurate at severity assessment. The paper shows the optimal configuration to be four demonstrations arranged in ascending order of similarity to the test input, with code similarity itself blended as 40% syntactic and 60% lexical. Under this configuration, SVA-ICL with DeepSeek-V2 scores 77.07% accuracy, 67.89% F1, and 63.01% MCC, improving on the best baseline (MTLM) by 2.53, 3.04, and 3.49 percentage points respectively. The authors further show that random demonstration selection drops accuracy to 61.75%, while a zero-shot baseline achieves only 44.21% accuracy.

Load-bearing premise

The recrawled CVSS v3 severity labels used as ground truth are assumed correct, but no independent validation against NVD's official scores or a second annotator is reported, so any label noise propagates directly into every reported accuracy, F1, and MCC figure.

Editorial extensions

If this is right

  • Organizations can implement LLM-based severity triage for C/C++ code without fine-tuning, as long as they maintain a historical corpus with precomputed embeddings.
  • The 70/30 code-to-description fusion ratio indicates that source code carries more severity-relevant signal than natural-language descriptions, a guideline for future multimodal retrieval designs.
  • Demonstration count (four) and ascending-similarity ordering emerge as tunable transferable hyperparameters for ICL in other security tasks.
  • The released 12,071-entry CVSS v3 dataset built from MegaVul provides a common benchmark aligned with the newer scoring standard.
  • If the retrieval mechanism is the true driver, then performance should scale with the quality and coverage of the historical vulnerability repository, making corpus curation a practical lever.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the recrawled severity labels were validated against NVD's official CVSS v3 scores, the gap between SVA-ICL and baselines might shrink; the paper does not provide such validation, so part of the reported gain could be an artifact of label noise inherited by all methods.
  • A testable extension would strip CVE identifiers and vendor names from the vulnerability descriptions in the test prompts; if accuracy falls substantially, part of the gain comes from the LLM's memorized CVE knowledge rather than from retrieval quality alone.
  • The same fused-retrieval ICL recipe likely transfers to other structured software tasks such as bug triage, patch prioritization, or risk scoring of configuration files, where multi-modal similarity selection could matter as much as the choice of LLM.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper presents SVA-ICL, an in-context learning (ICL) approach to software vulnerability assessment (SVA) that classifies C/C++ vulnerabilities into CVSS v3 severity levels (Critical, High, Medium, Low). It selects demonstrations by fusing source-code similarity (semantic similarity from CodeBERT with BERT-whitening, lexical Jaccard similarity, and syntactic AST-sequence Levenshtein similarity) with vulnerability-description similarity from CoSENT, then prompts DeepSeek-V2 with up to four demonstrations. The authors construct a 12,071-entry dataset from MegaVul, compare with nine baselines, and report higher Accuracy, macro F1, and MCC, together with ablations on fusion ratios, demonstration count, ordering, and choice of LLM.

Significance. If the empirical claims hold, the paper offers a useful contribution: a reproducible few-shot LLM pipeline for severity assessment, an openly shared dataset and code, baseline reruns with hyperparameter tuning, and unusually thorough ablations including multiple random splits and a post-cutoff leakage check. The margins over the strongest baselines are modest (e.g., 2.53 pp Accuracy, 3.04 pp F1, 3.49 pp MCC over MTLM in Table 3), so the credibility of the comparison depends heavily on the validity of the recrawled severity labels and on the consistency of the reported experimental configuration. The paper should be made stronger by directly validating the labels and by clarifying which configuration produced the headline numbers.

major comments (3)
  1. [Section 4.2, Table 3] The recrawled CVSS v3 severity labels are not validated. The authors state that MegaVul did not fully capture CVSS v3 ratings and that they recrawled severity ratings and scores, but they provide no comparison with NVD official CVSS v3 base scores, no second-annotator agreement, and no error-rate estimate. Section 6.7 does not list label noise as a threat. Since every Accuracy, F1, and MCC value in Table 3 and all ablation tables is computed against these labels, and the advantage over the second-best baseline is small (2.53 pp, 3.04 pp, 3.49 pp), even a small systematic labeling error could change the ranking. As the dataset is a claimed contribution, label validity needs direct evidence, such as a random-sample audit against NVD or another authoritative source, with per-class agreement rates.
  2. [Section 4.6 vs. Section 5.5 / Table 7] The main experimental configuration is reported inconsistently. Section 4.6 states that the ratio of syntactic to lexical similarity is set to 60% to 40%, but RQ5 concludes that the best setting is 40% syntactic and 60% lexical, and Table 7 reports the headline numbers of 77.07% Accuracy, 67.89% F1, and 63.01% MCC in the 40%/60% row. This discrepancy must be resolved: either Section 4.6 has a typo, or the RQ1/RQ2/RQ3/RQ4 results were generated with a different configuration than stated. Without clarification, the main result is not reproducible.
  3. [Section 6.5 / Section 6.7] The data-leakage analysis is under-specified. The post-cutoff subset size is not reported, the choice of July 2023 as DeepSeek-V2's cutoff is not justified, and the vulnerability descriptions in the prompts are CVE/NVD-style text that the model may have seen even if the code was introduced after the cutoff. The argument in Section 6.7 that poor zero-shot performance indicates low leakage probability is not compelling, because ICL demonstrations can still expose memorized label patterns. Please report the size and severity distribution of the post-cutoff subset, and state explicitly which aspects of leakage this check can and cannot rule out.
minor comments (6)
  1. [Introduction, Section 1] The Introduction states that SVA-ICL improves Accuracy, F1, and MCC by at least 7.68 pp, 5.34 pp, and 13.59 pp, but Table 3 shows the improvements over the second-best baseline (MTLM) are 2.53 pp, 3.04 pp, and 3.49 pp. These numbers should be reconciled; the current 'at least' phrasing is inconsistent with the reported table.
  2. [Table 7 / Section 3.1.2] The notation is inconsistent: Table 7 uses 'LexiSim' while Section 3.1.2 and Eq. (4) use 'LexSim'. Please use one term throughout.
  3. [Section 6.1, Table 8] The text says GPT-4o performs slightly better than DeepSeek-V2, but Table 8 shows GPT-4o has higher F1 and MCC while lower Accuracy. The comparison should be described as mixed, or a statistical test should be added if the claim is intended to be substantive.
  4. [Section 6.5] Please provide the number of test samples collected after July 2023 and the exact cutoff date for the DeepSeek-V2 API version used. Without the sample size, the leakage check cannot be interpreted.
  5. [Section 4.2] The paper refers to 'the CVSS v3 standard' without specifying whether CVSS v3.0 or v3.1 is used. Since the severity buckets are the same but vector-string handling may differ, this detail matters for reproducing the label recrawling.
  6. [Figure 5] The caption lists subplots (a) Accuracy, (b) F1-score, and (c) MCC, but the displayed panels appear to contain all three measures in each panel. Please verify the figure and caption correspondence.

Circularity Check

1 steps flagged · score 6.0 of 10

Ablation hyperparameters (fusion ratio, demo count, ordering, code-similarity weights) are selected on the same test split whose numbers are then reported as the validated SVA-ICL result; self-citations are secondary.

  1. fitted input called prediction [Section 4.6 and Section 5.2 / Table 4 vs Table 3; same pattern in Tables 5-7]
    "For information fusion, we set the ratio of source code similarity to vulnerability description similarity at 70% to 30%. The rationale for these experimental settings is detailed in our ablation study analysis. ... when the code similarity and text similarity ratios are set to 70% and 30%, respectively, our proposed approach SVA-ICL achieves the best performance in terms of the MCC measure."

    The 70/30 fusion ratio is selected because it maximizes MCC in Table 4, and the Table 4 row (77.07/67.90/63.01) is numerically the same split as the headline Table 3 SVA-ICL result (77.07/67.89/63.01). The paper describes a train/validation/test split but never states that the ablations were run on the validation split; the identical numbers show that the ablation tuned the ratio on the same test set whose SVA-ICL performance is then reported as the validated result. The 'optimal ratio' finding is therefore a selection criterion re-reported as evidence, not an out-of-sample prediction. The same test-set selection appears for demonstration count (Table 5), ordering strategy (Table 6), and syntactic/lexical ratio (Table 7).

full rationale

The paper's equations (1)-(5) are definitions (L2 distance, Levenshtein/Jaccard similarities, linear fusions), so there is no derivation chain that reduces to its inputs. The similarity-based demonstration selection is an empirical pipeline, and the self-citations to previous work [25,26] are not load-bearing: the current paper runs its own ablation (RQ5), and the cited retrieval method is an external published artifact. The circularity that does exist is in the hyperparameter selection protocol. Section 4.6 fixes the fusion ratio at 70%:30% 'detailed in our ablation study analysis'; Section 5.2 then reports that 70%:30% 'achieves the best performance' from Table 4. The numbers in that row are the same as the headline Table 3 SVA-ICL result, indicating the ablation table is computed on the same test split used for the final comparison. The paper describes a held-out validation set but never states that ablations used it. Thus the 'optimal ratio' finding is a restatement of a test-set selection, and the final test metrics are not an independent out-of-sample evaluation of the chosen configuration. The same pattern appears for demonstration count, ordering strategy, and code-similarity weights. This is partial circularity: the headline comparison against baselines is not forced by construction, but the reported optimal settings and the associated margins are statistically inflated by test-set tuning. The unvalidated recrawled CVSS v3 labels (Section 4.2) are a separate external-validity risk, not a circularity step, and Section 6.7 does not list this label-validation gap as a threat.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The approach introduces no new entities, particles, forces, or data structures beyond a weighted similarity formula. The main axiomatic burden is the reliability of the recrawled CVSS v3 labels and the validity of the similarity-as-quality proxy. The free parameters are fitted on validation/ablations, then frozen for the main test, which is legitimate but means the headline numbers partly reflect tuning.

free parameters (5)
  • phi, fusion weight for code vs description similarity = 0.7 (code 70%, description 30%)
    Selected from 11 candidate ratios 0.0 to 1.0 in RQ2, and the selected value is then used in the main RQ1 result.
  • lambda, fusion weight for syntactic vs lexical similarity = 0.4 syntax / 0.6 lexical
    Selected from 11 candidate ratios in RQ5, then used in the main RQ1 result.
  • k, number of demonstrations = 4
    Chosen by comparing 0, 1, 4, and 5 demonstrations in RQ3, then used in the main result.
  • n, number of semantic-retrieval candidates = top-10
    The paper states top-10 candidates are retrieved before lexical/syntactic and text scoring; no ablation is reported for n.
  • demonstration ordering = ascending similarity
    Selected from three strategies in RQ4; this ordering is then used in the main result.
assumptions (3)
  • domain assumption The MegaVul-derived severity labels recrawled by the authors are correct CVSS v3 base scores.
    Section 4.2 states they recrawled all severity ratings after finding MegaVul did not fully capture CVSS v3, but no validation of the recrawled labels is provided.
  • domain assumption The similarity of demonstration examples to the target vulnerability is well captured by the weighted combination in Eq. 5.
    The whole retrieval pipeline assumes that CodeSim (itself a weighted blend of SynSim and LexSim) plus TextSim is a good proxy for usefulness of a demonstration for severity assessment.
  • domain assumption DeepSeek-V2's behavior on the test prompts reflects the version named in the paper.
    Section 6.7 acknowledges API versions vary and recommends using the DeepSeek-V2 version; reproduction depends on API availability and versioning.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SVA-ICL: Improving LLM-based Software Vulnerability Assessment via In-Context Learning and Information Fusion." pith.science (2026). https://pith.science/paper/WPDSYIIJ

@misc{pith2026250510008,
  author       = {Pith},
  title        = {Pith review of: SVA-ICL: Improving LLM-based Software Vulnerability Assessment via In-Context Learning and Information Fusion},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WPDSYIIJ}},
  note         = {Machine review of arXiv:2505.10008}
}
read the original abstract

Context: Software vulnerability assessment (SVA) is critical for identifying, evaluating, and prioritizing security weaknesses in software applications. Objective: Despite the increasing application of large language models (LLMs) in various software engineering tasks, their effectiveness in SVA remains underexplored. Method: To address this gap, we introduce a novel approach SVA-ICL, which leverages in-context learning (ICL) to enhance LLM performance. Our approach involves the selection of high-quality demonstrations for ICL through information fusion, incorporating both source code and vulnerability descriptions. For source code, we consider semantic, lexical, and syntactic similarities, while for vulnerability descriptions, we focus on textual similarity. Based on the selected demonstrations, we construct context prompts and consider DeepSeek-V2 as the LLM for SVA-ICL. Results: We evaluate the effectiveness of SVA-ICL using a large-scale dataset comprising 12,071 C/C++ vulnerabilities. Experimental results demonstrate that SVA-ICL outperforms state-of-the-art SVA baselines in terms of Accuracy, F1-score, and MCC measures. Furthermore, ablation studies highlight the significance of component customization in SVA-ICL, such as the number of demonstrations, the demonstration ordering strategy, and the optimal fusion ratio of different modalities. Conclusion: Our findings suggest that leveraging ICL with information fusion can effectively improve the effectiveness of LLM-based SVA, warranting further research in this direction.

Figures

Figures reproduced from arXiv: 2505.10008 by the authors.

Figure 1
Figure 1. An example of using in-context learning for software vulnerability [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Framework of our proposed approach SVA-ICL [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The prompt template designed by our proposed approach SVA-ICL. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Two cases of the base severity returned by our proposed approach [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]
Figure 5
Figure 5. Figure 5: Comparison results between SVA-ICL and baselines in terms of di [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

65 extracted references · 47 canonical work pages

  1. [1]

    T. H. Le, H. Chen, M. A. Babar, A survey on data-driven software vul- nerability assessment and prioritization, ACM Computing Surveys 55 (5) (2022) 1–39

    work page 2022

  2. [2]

    Dissanayake, A

    N. Dissanayake, A. Jayatilaka, M. Zahedi, M. A. Babar, Software secu- rity patch management-a systematic literature review of challenges, ap- proaches, tools and practices, Information and Software Technology 144 (2022) 106771

    work page 2022

  3. [3]

    K. Liu, Y . Zhou, Q. Wang, X. Zhu, Vulnerability severity prediction with deep neural network, in: 2019 5th international conference on big data and information analytics (BigDIA), IEEE, 2019, pp. 114–119

    work page 2019

  4. [4]

    T. H. M. Le, M. A. Babar, On the use of fine-grained vulnerable code statements for software vulnerability assessment models, in: Proceedings of the 19th International Conference on Mining Software Repositories, 2022, pp. 621–633

    2022

  5. [5]

    Vaithilingam, T

    P. Vaithilingam, T. Zhang, E. L. Glassman, Expectation vs. experience: Evaluating the usability of code generation tools powered by large lan- guage models, in: Chi conference on human factors in computing systems extended abstracts, 2022, pp. 1–7

    work page 2022

  6. [6]

    J. Liu, C. S. Xia, Y . Wang, L. Zhang, Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code gen- eration, Advances in Neural Information Processing Systems 36 (2024)

    work page 2024

  7. [7]

    M. D. Purba, A. Ghosh, B. J. Radford, B. Chu, Software vulnerability detection using large language models, in: 2023 IEEE 34th International Symposium on Software Reliability Engineering Workshops (ISSREW), IEEE, 2023, pp. 112–119

    work page 2023

  8. [8]

    X. Zhou, T. Zhang, D. Lo, Large language model for vulnerability de- tection: Emerging results and future directions, in: Proceedings of the 2024 ACM/IEEE 44th International Conference on Software Engineer- ing: New Ideas and Emerging Results, 2024, pp. 47–51

    work page 2024

Show all 65 references
  1. [9]

    M. Geng, S. Wang, D. Dong, H. Wang, G. Li, Z. Jin, X. Mao, X. Liao, Large language models are few-shot summarizers: Multi-intent com- ment generation via in-context learning, in: Proceedings of the 46th IEEE/ACM International Conference on Software Engineering, 2024, pp. 1–13

    2024

  2. [10]

    Q. Dong, L. Li, D. Dai, C. Zheng, Z. Wu, B. Chang, X. Sun, J. Xu, Z. Sui, A survey on in-context learning, arXiv preprint arXiv:2301.00234 (2022)

    2022 arXiv

  3. [11]

    Babalau, D

    I. Babalau, D. Corlatescu, O. Grigorescu, C. Sandescu, M. Dascalu, Severity prediction of software vulnerabilities based on their text descrip- tion, in: 2021 23rd International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC), IEEE, 2021, pp. 171– 177

    2021

  4. [12]

    J. Hao, S. Luo, L. Pan, A novel vulnerability severity assessment method for source code based on a graph neural network, Information and Soft- ware Technology 161 (2023) 107247

    2023

  5. [13]

    arXiv:2405.04434

    DeepSeek-AI, Deepseek-v2: A strong, economical, and e fficient mixture- of-experts language model (2024). arXiv:2405.04434

    2024 arXiv

  6. [14]

    Humayun, N

    M. Humayun, N. Jhanjhi, M. F. Almufareh, M. I. Khalil, Security threat and vulnerability assessment and measurement in secure software devel- opment, Comput. Mater. Contin 71 (2022) 5039–5059

    2022

  7. [15]

    Common Vulnerability Scoring System

    2024. Common Vulnerability Scoring System. https://www.first. org/cvss/

    2024

  8. [16]

    T. H. M. Le, B. Sabir, M. A. Babar, Automated software vulnerability assessment with concept drift, in: 2019 IEEE /ACM 16th International Conference on Mining Software Repositories (MSR), IEEE, 2019, pp. 371–382

    2019

  9. [17]

    T. H. M. Le, D. Hin, R. Croft, M. A. Babar, Deepcva: Automated commit- level vulnerability assessment with deep multi-task learning, in: 2021 36th IEEE/ACM International Conference on Automated Software Engi- neering (ASE), IEEE, 2021, pp. 717–729

    2021

  10. [18]

    National Vulnerability Database

    2024. National Vulnerability Database. https://nvd.nist.gov/

    2024

  11. [19]

    Kasneci, K

    E. Kasneci, K. Seßler, S. K ¨uchemann, M. Bannert, D. Dementieva, F. Fis- cher, U. Gasser, G. Groh, S. G ¨unnemann, E. H ¨ullermeier, et al., Chatgpt for good? on opportunities and challenges of large language models for education, Learning and individual differences 103 (2023) 102274

    2023

  12. [20]

    Chang, X

    Y . Chang, X. Wang, J. Wang, Y . Wu, L. Yang, K. Zhu, H. Chen, X. Yi, C. Wang, Y . Wang, et al., A survey on evaluation of large language mod- els, ACM Transactions on Intelligent Systems and Technology 15 (3) (2024) 1–45

    2024

  13. [21]

    H. Tang, K. Hu, J. P. Zhou, S. Zhong, W.-L. Zheng, X. Si, K. Ellis, Code repair with llms gives an exploration-exploitation tradeoff, arXiv preprint arXiv:2405.17503 (2024)

    2024 arXiv

  14. [22]

    S. Min, X. Lyu, A. Holtzman, M. Artetxe, M. Lewis, H. Hajishirzi, L. Zettlemoyer, Rethinking the role of demonstrations: What makes in- context learning work?, arXiv preprint arXiv:2202.12837 (2022)

    2022 arXiv

  15. [23]

    Gao, X.-C

    S. Gao, X.-C. Wen, C. Gao, W. Wang, H. Zhang, M. R. Lyu, What makes good in-context demonstrations for code intelligence tasks with llms?, in: 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE), IEEE, 2023, pp. 761–773

    2023

  16. [24]

    Y . Wang, W. Wang, S. Joty, S. C. Hoi, Codet5: Identifier-aware unified pre-trained encoder-decoder models for code understanding and genera- tion, arXiv preprint arXiv:2109.00859 (2021)

    2021 arXiv

  17. [25]

    G. Yang, K. Liu, X. Chen, Y . Zhou, C. Yu, H. Lin, Ccgir: Informa- tion retrieval-based code comment generation method for smart contracts, Knowledge-Based Systems 237 (2022) 107858

    2022

  18. [26]

    J. Zhao, X. Chen, G. Yang, Y . Shen, Automatic smart contract comment generation via large language models and in-context learning, Informa- tion and Software Technology 168 (2024) 107405

    2024

  19. [27]

    Z. Feng, D. Guo, D. Tang, N. Duan, X. Feng, M. Gong, L. Shou, B. Qin, T. Liu, D. Jiang, et al., Codebert: A pre-trained model for programming and natural languages, arXiv preprint arXiv:2002.08155 (2020)

    2020 arXiv

  20. [28]

    J. Su, J. Cao, W. Liu, Y . Ou, Whitening sentence representations for better semantics and faster retrieval, arXiv preprint arXiv:2103.15316 (2021)

    2021 arXiv

  21. [29]

    Ming, Similarities: similarity calculation and semantic search toolkit, https://github.com/shibing624/similarities (2022)

    X. Ming, Similarities: similarity calculation and semantic search toolkit, https://github.com/shibing624/similarities (2022)

    2022

  22. [30]

    K. Liu, X. Chen, C. Chen, X. Xie, Z. Cui, Automated question title refor- mulation by mining modification logs from stack overflow, IEEE Trans- actions on Software Engineering (2023)

    2023

  23. [31]

    W. Yin, L. Shang, E fficient nearest neighbor emotion classification with bert-whitening, in: Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 2022, pp. 4738–4745

    2022

  24. [32]

    Jiang, J

    T. Jiang, J. Jiao, S. Huang, Z. Zhang, D. Wang, F. Zhuang, F. Wei, H. Huang, D. Deng, Q. Zhang, Promptbert: Improving bert sentence em- beddings with prompts, arXiv preprint arXiv:2201.04337 (2022)

    2022 arXiv

  25. [33]

    G. Yang, X. Chen, J. Cao, S. Xu, Z. Cui, C. Yu, K. Liu, Comformer: Code comment generation via transformer and fusion method-based hy- brid code representation, in: 2021 8th International Conference on De- pendable Systems and Their Applications (DSA), IEEE, 2021, pp. 30–41

    2021

  26. [34]

    J. K. Siow, S. Liu, X. Xie, G. Meng, Y . Liu, Learning program semantics with code representations: An empirical study, in: 2022 IEEE Interna- tional Conference on Software Analysis, Evolution and Reengineering (SANER), IEEE, 2022, pp. 554–565

    2022

  27. [35]

    Yujian, L

    L. Yujian, L. Bo, A normalized levenshtein distance metric, IEEE trans- 17 actions on pattern analysis and machine intelligence 29 (6) (2007) 1091– 1095

    2007

  28. [36]

    S. Bag, S. K. Kumar, M. K. Tiwari, An e fficient recommendation gener- ation using relevant jaccard similarity, Information Sciences 483 (2019) 53–64

    2019

  29. [37]

    M. R. Shahid, H. Debar, Cvss-bert: Explainable natural language process- ing to determine the severity of a computer security vulnerability from its description, in: 2021 20th IEEE International Conference on Machine Learning and Applications (ICMLA), IEEE, 2021, pp. 1600–1607

    2021

  30. [38]

    J. Xue, X. Chen, J. Wang, Z. Cui, Towards prompt tuning-based software vulnerability assessment with continual learning, Computers & Security 150 (2025) 104184

    2025

  31. [39]

    X. Du, S. Zhang, Y . Zhou, H. Du, A vulnerability severity prediction method based on bimodal data and multi-task learning, Journal of Sys- tems and Software 213 (2024) 112039

    2024

  32. [40]

    F. Xue, Y . Fu, W. Zhou, Z. Zheng, Y . You, To repeat or not to repeat: Insights from scaling llm under token-crisis, Advances in Neural Infor- mation Processing Systems 36 (2024)

    2024

  33. [41]

    J. He, W. Jiang, G. Hou, W. Fan, R. Zhang, H. Li, Talk too much: Poisoning large language models under token limit, arXiv preprint arXiv:2404.14795 (2024)

    2024 arXiv

  34. [42]

    C. Ni, L. Shen, X. Yang, Y . Zhu, S. Wang, Megavul: Ac/c++ vulnerability dataset with comprehensive code representations, in: 2024 IEEE /ACM 21st International Conference on Mining Software Repositories (MSR), IEEE, 2024, pp. 738–742

    2024

  35. [43]

    Common Vulnerabilities and Exposures

    2024. Common Vulnerabilities and Exposures. https://cve.mitre. org/

    2024

  36. [44]

    T. K. Ho, Random decision forests, in: Proceedings of 3rd international conference on document analysis and recognition, V ol. 1, IEEE, 1995, pp. 278–282

    1995

  37. [45]

    G. Ke, Q. Meng, T. Finley, T. Wang, W. Chen, W. Ma, Q. Ye, T.-Y . Liu, Lightgbm: A highly e fficient gradient boosting decision tree, Advances in neural information processing systems 30 (2017)

    2017

  38. [46]

    S. J. Russell, P. Norvig, Artificial intelligence: a modern approach, Pear- son, 2016

    2016

  39. [47]

    Cortes, V

    C. Cortes, V . Vapnik, Support-vector networks, Machine learning 20 (1995) 273–297

    1995

  40. [48]

    S. H. Walker, D. B. Duncan, Estimation of the probability of an event as a function of several independent variables, Biometrika 54 (1-2) (1967) 167–179

    1967

  41. [49]

    Luque, A

    A. Luque, A. Carrasco, A. Mart ´ın, A. de Las Heras, The impact of class imbalance in classification performance metrics based on the binary con- fusion matrix, Pattern Recognition 91 (2019) 216–231

    2019

  42. [50]

    Spanos, L

    G. Spanos, L. Angelis, A multi-target approach to estimate software vul- nerability characteristics and severity scores, Journal of Systems and Soft- ware 146 (2018) 152–166

    2018

  43. [51]

    Gorodkin, Comparing two k-category assignments by a k-category cor- relation coefficient, Computational biology and chemistry 28 (5-6) (2004) 367–374

    J. Gorodkin, Comparing two k-category assignments by a k-category cor- relation coefficient, Computational biology and chemistry 28 (5-6) (2004) 367–374

    2004

  44. [52]

    J. Ye, X. Chen, N. Xu, C. Zu, Z. Shao, S. Liu, Y . Cui, Z. Zhou, C. Gong, Y . Shen, et al., A comprehensive capability analysis of gpt-3 and gpt-3.5 series models, arXiv preprint arXiv:2303.10420 (2023)

    2023 arXiv

  45. [53]

    Anand, Z

    Y . Anand, Z. Nussbaum, B. Duderstadt, B. Schmidt, A. Mulyar, Gpt4all: Training an assistant-style chatbot with large scale data distillation from gpt-3.5-turbo, GitHub (2023)

    2023

  46. [54]

    Z. Dai, Z. Yang, Y . Yang, J. Carbonell, Q. V . Le, R. Salakhutdinov, Transformer-xl: Attentive language models beyond a fixed-length con- text, arXiv preprint arXiv:1901.02860 (2019)

    2019 arXiv

  47. [55]

    Nashid, M

    N. Nashid, M. Sintaha, A. Mesbah, Retrieval-based prompt selection for code-related few-shot learning, in: 2023 IEEE /ACM 45th International Conference on Software Engineering (ICSE), IEEE, 2023, pp. 2450– 2462

    2023

  48. [56]

    Cheng, T

    Z. Cheng, T. Xie, P. Shi, C. Li, R. Nadkarni, Y . Hu, C. Xiong, D. Radev, M. Ostendorf, L. Zettlemoyer, et al., Binding language models in sym- bolic languages, arXiv preprint arXiv:2210.02875 (2022)

    2022 arXiv

  49. [57]

    Bulatov, Y

    A. Bulatov, Y . Kuratov, M. Burtsev, Recurrent memory transformer, Advances in Neural Information Processing Systems 35 (2022) 11079– 11091

    2022

  50. [58]

    Ahmed, K

    T. Ahmed, K. S. Pai, P. Devanbu, E. Barr, Automatic semantic augmen- tation of language model prompts (for code summarization), in: Proceed- ings of the IEEE/ACM 46th International Conference on Software Engi- neering, 2024, pp. 1–13

    2024

  51. [59]

    Ishizue, K

    R. Ishizue, K. Sakamoto, H. Washizaki, Y . Fukazawa, Improved program repair methods using refactoring with gpt models, in: Proceedings of the 55th ACM Technical Symposium on Computer Science Education V . 1, 2024, pp. 569–575

    2024

  52. [60]

    Do Viet, K

    T. Do Viet, K. Markov, Using large language models for bug localization and fixing, in: 2023 12th International Conference on Awareness Science and Technology (iCAST), IEEE, 2023, pp. 192–197

    2023

  53. [61]

    T. Y . Zhuo, Large language models are state-of-the-art evaluators of code generation, arXiv preprint arXiv:2304.14317 (2023)

    2023 arXiv

  54. [62]

    J. Li, G. Li, Y . Zhao, Y . Li, Z. Jin, H. Zhu, H. Liu, K. Liu, L. Wang, Z. Fang, et al., Deveval: Evaluating code generation in practical software projects, arXiv preprint arXiv:2401.06401 (2024)

    2024 arXiv

  55. [63]

    Z. Han, X. Li, Z. Xing, H. Liu, Z. Feng, Learning to predict severity of software vulnerability using only vulnerability description, in: 2017 IEEE International conference on software maintenance and evolution (ICSME), IEEE, 2017, pp. 125–136

    2017

  56. [64]

    Ganesh, T

    S. Ganesh, T. Ohlsson, F. Palma, Predicting security vulnerabilities us- ing source code metrics, in: 2021 Swedish workshop on data science (SweDS), IEEE, 2021, pp. 1–7. Chaoyang Gao is currently pursuing the Master degree at the School of Artificial Intelligence and Computer ...

    2021

  57. [2002]

    degrees in com- puter software and theory from Nanjing University in 2008 and 2011, respectively

    Then he received his M.Sc., and Ph.D. degrees in com- puter software and theory from Nanjing University in 2008 and 2011, respectively. He is an Associate Professor at the School of Artificial Intelligence and Computer Science, Nantong Uni- versity. He has authored or co-autho...

    2008

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.