Pith. sign in

REVIEW 3 major objections 6 minor 44 references

Vulnerability Detection in Popular Programming Languages with Language Models

T0 review · 3 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that language models detect vulnerabilities far better in JavaScript and Java than in C/C++, and that code complexity does not explain the gap.

desk verdict Useful per-language benchmark, but the headline JavaScript-vs-C/C++ comparison is not controlled enough to support the paper's conclusion. read the letter →

arxiv 2412.15905 v2 pith:NMKRIYLM submitted 2024-12-20 cs.CR cs.SE

Syafiq Al Atiiq , Christian Gehrmann , Kevin Dahlén This is my paper · ORCID
classification cs.CRcs.SE
keywords vulnerabilitydetectionlanguagemodelsJavaScriptC/C++codecomplexityCVEFixesfine-tuningcross-language
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether large language models can detect security vulnerabilities in languages other than C/C++, which has been the near-exclusive focus of prior work. By cleaning the CVEFixes database into language-specific subsets for JavaScript, Java, Python, PHP, Go and C/C++, and fine-tuning four language models under identical conditions, the authors find that detection performance varies strongly by language. JavaScript comes out best, with F1 scores near 70 percent, considerably ahead of C/C++, and both JavaScript and Java outperform the C/C++ results reported in earlier studies. The authors also find that standard code-complexity metrics show only weak and statistically insignificant correlations with F1 scores, so complexity does not explain the differences.

What carries the argument

A controlled cross-language comparison built on a single dataset source: the paper filters CVEFixes into language-specific sets, applies identical preprocessing (duplicate removal, stratified temporal train/test split), fine-tunes the same four models on each language under the same hyperparameters, and then measures F1 and false-positive rate. The language-dependent results are validated on independent datasets for Java, Python and PHP. A second analysis computes Pearson correlations between six code-complexity metrics (token length, Halstead volume, difficulty and effort, cyclomatic complexity, and lines of code) and each model's F1 score across the six languages to test whether complexity explains performance.

What would settle it

Rebuild the JavaScript test set so that every non-vulnerable function is taken from unrelated code instead of from the same fix history as the vulnerable function, then re-fine-tune and re-measure F1; if the JavaScript advantage over C/C++ disappears or drops sharply, the result was an artifact of distinguishing buggy from fixed code rather than detecting vulnerabilities. Alternatively, cap every language at the same training-sample count (for example, the Go count of 6,283) and check whether the ranking survives.

Watch

Extended reading notes

Core claim

On a cleaned subset of CVEFixes covering six programming languages and four fine-tuned language models (CodeT5, CodeBERT, UniXcoder and DeepSeek-Coder), the paper's central finding is that vulnerability detection performance is strongly language-dependent. JavaScript achieves the best F1 scores (about 70 percent with CodeBERT and UniXcoder), followed by Java, while C/C++ lags with F1 scores in the low 30s on the same data — and the non-C/C++ results are considerably better than the C/C++ figures reported by the DiverseVul and PrimeVul studies. The paper further claims that code complexity, measured by token length, Halstead metrics, cyclomatic complexity and lines of code, correlates only weakly with F1 scores, and that none of the correlations are statistically significant. This is presented as evidence that language characteristics beyond raw complexity drive how well LMs can learn vulnerabilities, and that JavaScript and Java are more practical targets for LM-based detection than the traditionally studied C/C++.

Load-bearing premise

The central comparison assumes that the language-specific datasets differ only in language, not in how vulnerable and non-vulnerable samples were chosen, so a higher F1 score reflects easier detectability rather than easier data.

Editorial extensions

If this is right

  • Practitioners should target JavaScript and Java codebases first if they plan to deploy LM-based vulnerability detection today, since those languages give the best detection quality.
  • The poor C/C++ results across CVEFixes, PrimeVul, DiverseVul and Gen2Spec indicate the problem is not a single dataset artifact but something about the language or its vulnerability patterns.
  • Because complexity metrics do not predict F1, model developers should not use complexity as a proxy for expected detection difficulty.
  • The mixed external validation results for Python and PHP mean that language-level conclusions should not be transferred blindly to new data sources.
  • The strongest JavaScript F1 scores come from the 125M-parameter encoder models, not the larger 1.3B Decoder model, so architecture matters more than raw size for this task.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural explanation the authors do not fully explore is that dataset size drives part of the JavaScript advantage: with 100,000 samples, JavaScript has roughly 12 times the data of Go and Java, so a matched-size experiment would separate language effects from data abundance.
  • If the non-vulnerable JavaScript samples are the fixed counterparts of the vulnerable functions, the model may learn to spot the diff itself; one testable fix is to build a JavaScript evaluation set where non-vulnerable code is unrelated to the vulnerable function.
  • The weak complexity correlation could be an artifact of the narrow complexity range in this dataset; applying the same correlation analysis to a more varied, multi-language corpus would tell whether complexity ever matters.
  • The authors' claim that JS and Java are more practical targets than C/C++ would be strengthened if independent JavaScript and Go datasets existed; building such datasets is a concrete next step for the community.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper presents a cross-language empirical study of language-model-based vulnerability detection using the CVEfixes dataset. The authors construct language-specific subsets for JavaScript, PHP, Java, Python, Go, and C/C++, fine-tune four LMs (CodeT5, CodeBERT, UnixCoder, DeepSeek-Coder), and report accuracy, F1, precision, recall, and FPR. Their central finding is that JavaScript shows the best detection performance, with 'considerably better and more practical detection capabilities compared to C/C++', while PHP and Go show lower F1 despite high accuracy. As a secondary analysis, they compute Pearson correlations between six code-complexity metrics and per-language F1 scores across all four models, concluding that the correlations are weak and statistically insignificant. The paper also validates some models on external datasets and releases the curated data, scripts, and results.

Significance. If the main finding were robust, this would fill a real gap: prior vulnerability-detection benchmarks for LMs concentrate heavily on C/C++, and a controlled cross-language comparison would be valuable for practitioners and benchmark designers. The authors are transparent about raw numbers, provide model-level metrics, and open-source their dataset and scripts, which supports reproducibility. However, the current evidence does not yet sustain the headline claim because the cross-language comparison is threatened by uncontrolled dataset construction differences and by a likely copy error in the complexity analysis. The significance is therefore contingent on the comparability concerns being resolved; with additional controls, this could become a useful reference study.

major comments (3)
  1. [Section 4.1 (Data Preprocessing), Table 1] The manuscript never states whether the non-vulnerable samples are the fixed counterparts of the vulnerable functions drawn from the same CVEfixes vulnerability-fixing commits. Because CVEfixes is constructed from fixing commits, the non-vulnerable entries are typically post-fix versions of the same functions that appear as vulnerable entries; if such paired before/after versions survive preprocessing, the binary classifier can succeed by recognizing which side of a diff it is looking at rather than by learning vulnerability semantics, and the JS-versus-C/C++ ranking would be invalidated. Please clarify whether such pairs were identified and separated across the train/test split, and report a control experiment with paired samples removed or with train and test sets constrained to disjoint commits.
  2. [Table 1 and Section 5.2] The cross-language F1 comparison is confounded by dataset size and class balance: JavaScript has 100,000 samples (46,802 vulnerable) and C/C++ has 20,060 (8,299 vulnerable), while PHP has only 4,758 vulnerable samples out of 28,257 total. The manuscript states that these sizes reflect the NVD distribution, but that does not remove the confound for the claim that language-dependent detectability differs. Without matched-size subsamples, propensity weighting, or a direct demonstration that the larger and more balanced JavaScript data alone does not explain the gap, the conclusion that JavaScript yields 'considerably better and more practical detection' is not supported by the reported experiments.
  3. [Table 5 and Section 5.3] The Java and C/C++ rows of Table 5 are identical for all six complexity metrics (Token Length 274.79, Halstead Volume 879.30, Halstead Difficulty 10.81, Halstead Effort 16462.39, Cyclomatic Complexity 3.27, NLOC 20.50), which indicates a copy error. Since Section 5.3 uses these per-language means to compute the Pearson correlations reported in Table 6, the complexity analysis and its conclusion of weak, statistically insignificant correlation are not reliable until the table is corrected and the correlations are recomputed.
minor comments (6)
  1. [Section 4.1] The description 'divide the data into training and testing sets based on the commit timestamps using stratified sampling' is ambiguous: stratified sampling preserves class proportions, while a temporal split orders by commit date; please clarify how the two procedures are combined and whether the stratification is applied within time intervals.
  2. [Table 1 footnote] The cap of 100,000 JavaScript entries is reported only in a footnote without justification; please explain why the first 100,000 entries were chosen and discuss the potential impact on representativeness and comparability with other language subsets.
  3. [Section 2.2] The sentence 'The major issue with the detection results reported from the PrimeVul [11] and DiverseVul [7] is is the low F1 scores' contains a duplicated 'is' and a subject-verb agreement error; it should be rephrased.
  4. [Section 2.2] The sentence beginning 'Furthermore, the vulnerability types diversity, each with its own unique characteristics...' is grammatically incomplete, and the citation [2] is to the authors' own work only; please rephrase and provide broader citations for the diversity of vulnerability types.
  5. [Section 5.2] The sentence 'This indicates a better ability to detect vulnerabilities with LLM for JavaScript compared to C/C++' overstates the evidence given the dataset-size and pairing confounds discussed above; please condition this conclusion on the comparability analysis or temper the wording.
  6. [Section 4.1, Figure 1] Figure 1 is referenced in the text but does not appear in the provided manuscript; please ensure the figure is included and legible, since it is the sole support for the claim about language distribution in CVEfixes.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the paper's claims are direct empirical benchmark results, not derivations from fitted inputs or self-citations.

full rationale

This paper performs a supervised train/test benchmark on language-specific subsets of CVEfixes and reports measured metrics. The central claim that JavaScript models achieve higher F1 than C/C++ models comes directly from Table 3, where models were fine-tuned on timestamp-split training folds and evaluated on held-out test folds. There is no fitted equation whose output is renamed as a prediction, no parameter fitted to a subset and then used to 'predict' a closely related quantity, and no derivation chain where a conclusion is equivalent to its assumptions by construction. The potential confounds identified by the skeptic—dataset-size differences, class imbalance, and CVEfixes' paired before/after construction—are data-validity and comparability threats, not circularity: the test labels are independent of the model outputs and are not generated from the paper's claims. The only self-citation, [2], is used for background discussion of vulnerability-type diversity and to report Gen2Spec C/C++ results in Table 4; it is not load-bearing for the paper's new JavaScript/Java findings, and no uniqueness theorem or fitting ansatz is imported from the authors' prior work. The complexity correlation analysis (Section 5.3) computes Pearson correlations between measured complexity metrics and measured F1 scores; the conclusion of weak, statistically insignificant correlation is a descriptive statistical result, not a prediction implied by those metrics. The paper is therefore self-contained as an empirical evaluation, and no circular step can be exhibited.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The central comparison rests on assumptions about label quality and comparability across language subsets, not on mathematical axioms. No new entities are postulated.

free parameters (1)
  • JavaScript subset cap = 100,000 entries (57.17% of total)
    The cap is chosen by hand for JavaScript, the language with the best reported F1; the order and rule for taking first entries are not specified, so they can bias the central comparison.
assumptions (3)
  • domain assumption CVEFixes labels are equally reliable across languages and the non-vulnerable class is not the fixed counterpart of the vulnerable class.
    The language comparison depends on this; the paper does not state how non-vulnerable samples are defined beyond citing CVEFixes.
  • ad hoc to paper F1 scores can be compared across subsets of very different sizes and class balances as indicators of language difficulty.
    JavaScript has 100,000 entries and Go has 6,283, yet the authors interpret F1 differences as language properties.
  • ad hoc to paper Six language-level data points are sufficient to draw a conclusion about weak correlation.
    The paper reports non-significant p-values but treats that as evidence for a weak relationship, which is not the same as a precisely estimated null effect.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Vulnerability Detection in Popular Programming Languages with Language Models." pith.science (2026). https://pith.science/paper/NMKRIYLM

@misc{pith2026241215905,
  author       = {Pith},
  title        = {Pith review of: Vulnerability Detection in Popular Programming Languages with Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NMKRIYLM}},
  note         = {Machine review of arXiv:2412.15905}
}
read the original abstract

Vulnerability detection is crucial for maintaining software security, and recent research has explored the use of Language Models (LMs) for this task. While LMs have shown promising results, their performance has been inconsistent across datasets, particularly when generalizing to unseen code. Moreover, most studies have focused on the C/C++ programming language, with limited attention given to other popular languages. This paper addresses this gap by investigating the effectiveness of LMs for vulnerability detection in JavaScript, Java, Python, PHP, and Go, in addition to C/C++ for comparison. We utilize the CVEFixes dataset to create a diverse collection of language-specific vulnerabilities and preprocess the data to ensure quality and integrity. We fine-tune and evaluate state-of-the-art LMs across the selected languages and find that the performance of vulnerability detection varies significantly. JavaScript exhibits the best performance, with considerably better and more practical detection capabilities compared to C/C++. We also examine the relationship between code complexity and detection performance across the six languages and find only a weak correlation between code complexity metrics and the models' F1 scores.

Figures

Figures reproduced from arXiv: 2412.15905 by the authors.

Figure 1
Figure 1. Distribution of programming languages in the CVEfixes dataset. Data Preprocessing and Filtering We first split CVEfixes based on the programming language. For each subset, we perform the following preprocessing steps: 1. Duplication removal: We identify and remove any duplicate entries in the dataset to avoid data leakage [29]. 2. Train-test splitting: To maintain temporal integrity and simulate a realistic evaluati… view at source ↗
Figure 2
Figure 2. Mean Halstead effort vs. F1 score for each programming language [PITH_FULL_IMAGE:figures/full_fig_p015_2.png] view at source ↗
Figure 3
Figure 3. Mean cyclomatic complexity vs. F1 score for each programming language [PITH_FULL_IMAGE:figures/full_fig_p015_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

44 extracted references · 16 canonical work pages

  1. [1]

    Alon, U., Zilberstein, M., Levy, O., Yahav, E.: code2vec: learning distributed rep- resentations of code. Proc. ACM Program. Lang. 3(POPL) (Jan 2019). https: //doi.org/10.1145/3290353, https://doi.org/10.1145/3290353

    doi:10.1145/3290353 2019

  2. [2]

    Atiiq,S.A.,Gehrmann,C.,Dahlén,K.,Khalil,K.:Fromgeneralisttospecialist:Ex- ploring cwe-specific vulnerability detection (2024),https://arxiv.org/abs/2408. 02329

    work page 2024

  3. [3]

    SIGSOFT Softw

    Ball, T.: The concept of dynamic analysis. SIGSOFT Softw. Eng. Notes24(6), 216–234 (Oct 1999). https://doi.org/10.1145/318774.318944, https://doi. org/10.1145/318774.318944

    work page arXiv 1999

  4. [4]

    In: Proceedings of the 17th International Conference on Predictive Models and Data Analytics in Soft- ware Engineering

    Bhandari, G., Naseer, A., Moonen, L.: Cvefixes: automated collection of vul- nerabilities and their fixes from open-source software. In: Proceedings of the 17th International Conference on Predictive Models and Data Analytics in Soft- ware Engineering. p. 30–39. PROMISE 2021, Association for Computing Machin- ery, New York, NY, USA (2021). https://doi.org...

    arXiv 2021

  5. [5]

    In: Proceedings of the 30th ACM Joint European Software En- gineering Conference and Symposium on the Foundations of Software Engineering

    Braz, L., Bacchelli, A.: Software security during modern code review: the devel- oper’s perspective. In: Proceedings of the 30th ACM Joint European Software En- gineering Conference and Symposium on the Foundations of Software Engineering. p. 810–821. ESEC/FSE 2022, Association for Computing Machinery, New York, NY, USA (2022),https://doi.org/10.1145/3540...

    arXiv 2022

  6. [6]

    Chakraborty, S., Krishna, R., Ding, Y., Ray, B.: Deep learning based vulnerability detection: Are we there yet? IEEE Transactions on Software Engineering48(09), 3280–3296 (sep 2022).https://doi.org/10.1109/TSE.2021.3087402

    arXiv 2022

  7. [7]

    In: Proceedings of the 26th International Symposium on Research in Attacks, Intru- sions and Defenses

    Chen, Y., Ding, Z., Alowain, L., Chen, X., Wagner, D.: Diversevul: A new vul- nerable source code dataset for deep learning based vulnerability detection. In: Proceedings of the 26th International Symposium on Research in Attacks, Intru- sions and Defenses. p. 654–668. RAID ’23, Association for Computing Machinery, New York, NY, USA (2023),https://doi.org...

    arXiv 2023

  8. [8]

    IEEE Security & Privacy2(6), 76–79 (2004)

    Chess, B., McGraw, G.: Static analysis for security. IEEE Security & Privacy2(6), 76–79 (2004). https://doi.org/10.1109/MSP.2004.111

    doi:10.1109/msp.2004.111 2004

Show all 44 references
  1. [9]

    In: Ku, L.W., Martins, A., Srikumar, V

    Dai, D., Deng, C., Zhao, C., Xu, R., Gao, H., Chen, D., Li, J., Zeng, W., Yu, X., Wu, Y., Xie, Z., Li, Y., Huang, P., Luo, F., Ruan, C., Sui, Z., Liang, W.: DeepSeekMoE: Towards ultimate expert specialization in mixture-of-experts lan- guage models. In: Ku, L.W., Martins, A., ...

    2024

  2. [10]

    In: Burstein, J., Doran, C., Solorio, T

    Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: Burstein, J., Doran, C., Solorio, T. (eds.) Proceedings of the 2019 Conference of the North Amer- ican Chapter of the Association for Computat...

    2019 doi

  3. [11]

    Ding, Y., Fu, Y., Ibrahim, O., Sitawarin, C., Chen, X., Alomair, B., Wagner, D., Ray, B., Chen, Y.: Vulnerability detection with code language models: How far are we? In: Proceedings of the 2025 IEEE/ACM 47th International Conference on Software Engineering (ICSE). pp. 469–481...

    2025

  4. [12]

    In: Cohn, T., He, Y., Liu, Y

    Feng, Z., Guo, D., Tang, D., Duan, N., Feng, X., Gong, M., Shou, L., Qin, B., Liu, T., Jiang, D., Zhou, M.: CodeBERT: A pre-trained model for programming and natural languages. In: Cohn, T., He, Y., Liu, Y. (eds.) Findings of the Association for Computational Linguistics: EMNL...

    2020 doi

  5. [13]

    In: Muresan, S., Nakov, P., Villavi- cencio, A

    Guo, D., Lu, S., Duan, N., Wang, Y., Zhou, M., Yin, J.: UniXcoder: Unified cross- modal pre-training for code representation. In: Muresan, S., Nakov, P., Villavi- cencio, A. (eds.) Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume ...

    2022

  6. [14]

    In: International Conference on Learning Representations (2021), https://openreview.net/forum?id=jLoC4ez43PZ

    Guo, D., Ren, S., Lu, S., Feng, Z., Tang, D., LIU, S., Zhou, L., Duan, N., Svy- atkovskiy, A., Fu, S., Tufano, M., Deng, S.K., Clement, C., Drain, D., Sundaresan, N., Yin, J., Jiang, D., Zhou, M.: Graphcode{bert}: Pre-training code represen- tations with data flow. In: Interna...

    2021

  7. [15]

    Guo, D., Zhu, Q., Yang, D., Xie, Z., Dong, K., Zhang, W., Chen, G., Bi, X., Wu, Y., Li, Y.K., Luo, F., Xiong, Y., Liang, W.: Deepseek-coder: When the large Title Suppressed Due to Excessive Length 19 language model meets programming – the rise of code intelligence (2024),https...

    2024 arXiv

  8. [16]

    Li,H.,Hao,Y.,Zhai,Y.,Qian,Z.:Enhancingstaticanalysisforpracticalbugdetec- tion: An llm-integrated approach. Proc. ACM Program. Lang.8(OOPSLA1) (Apr 2024). https://doi.org/10.1145/3649828, https://doi.org/10.1145/3649828

    2024 doi

  9. [17]

    In: Proceedings 2018 Network and Distributed System Security Symposium

    Li, Z., Zou, D., Xu, S., Ou, X., Jin, H., Wang, S., Deng, Z., Zhong, Y.: Vuldeep- ecker: A deep learning-based system for vulnerability detection. In: Proceedings 2018 Network and Distributed System Security Symposium. NDSS 2018, Internet Society (2018), http://dx.doi.org/10.1...

    2018

  10. [18]

    Computers & Security 148, 104098 (2025)

    Liang, C., Wei, Q., Du, J., Wang, Y., Jiang, Z.: Survey of source code vul- nerability analysis based on deep learning. Computers & Security 148, 104098 (2025). https://doi.org/https://doi.org/10.1016/j.cose.2024.104098, https://www.sciencedirect.com/science/article/pii/S01674...

    2025

  11. [19]

    Proceedings of the IEEE108(10), 1825–1848 (2020)

    Lin, G., Wen, S., Han, Q.L., Zhang, J., Xiang, Y.: Software vulnerability detection using deep neural networks: A survey. Proceedings of the IEEE108(10), 1825–1848 (2020). https://doi.org/10.1109/JPROC.2020.2993293

    2020

  12. [20]

    ACM Comput

    Lin, R., Fu, Y., Yi, W., Yang, J., Cao, J., Dong, Z., Xie, F., Li, H.: Vulnerabil- ities and security patches detection in oss: A survey. ACM Comput. Surv. (Sep 2024). https://doi.org/10.1145/3694782, https://doi.org/10.1145/3694782, just Accepted

    2024 doi

  13. [21]

    Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., Stoyanov, V.: Ro{bert}a: A robustly optimized {bert} pretraining approach (2020), https://openreview.net/forum?id=SyxS0T4tvS

    2020

  14. [22]

    In: Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1) (2021),https: //openreview.net/forum?id=6lE4dQXaUcb

    Lu, S., Guo, D., Ren, S., Huang, J., Svyatkovskiy, A., Blanco, A., Clement, C., Drain, D., Jiang, D., Tang, D., Li, G., Zhou, L., Shou, L., Zhou, L., Tu- fano, M., GONG, M., Zhou, M., Duan, N., Sundaresan, N., Deng, S.K., Fu, S., LIU, S.: CodeXGLUE: A machine learning benchmar...

    2021

  15. [23]

    In: Proceed- ings of the 51st ACM Technical Symposium on Computer Science Education

    Lumbroso, J., Evans, J.: Making manual code review scale. In: Proceed- ings of the 51st ACM Technical Symposium on Computer Science Education. p. 1390. SIGCSE ’20, Association for Computing Machinery, New York, NY, USA(2020). https://doi.org/10.1145/3328778.3367026, https://do...

    2020

  16. [24]

    IEEE Security & Privacy 2(2), 80–83 (2004)

    McGraw, G.: Software security. IEEE Security & Privacy 2(2), 80–83 (2004). https://doi.org/10.1109/MSECP.2004.1281254

    2004 arXiv

  17. [25]

    In: Proceedings of the IEEE/ACM 46th Interna- tional Conference on Software Engineering

    Nam, D., Macvean, A., Hellendoorn, V., Vasilescu, B., Myers, B.: Using an llm to help with code understanding. In: Proceedings of the IEEE/ACM 46th Interna- tional Conference on Software Engineering. ICSE ’24, Association for Computing Machinery, New York, USA (2024),https://d...

    2024

  18. [26]

    In: 2024 IEEE/ACM 21st Inter- national Conference on Mining Software Repositories (MSR)

    Ni, C., Shen, L., Yang, X., Zhu, Y., Wang, S.: Megavul: A c/c++ vulnerability dataset with comprehensive code representations. In: 2024 IEEE/ACM 21st Inter- national Conference on Mining Software Repositories (MSR). pp. 738–742 (2024)

    2024

  19. [27]

    In: Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Sym- posium on the Foundations of Software Engineering

    Nikitopoulos, G., Dritsa, K., Louridas, P., Mitropoulos, D.: Crossvul: a cross- language vulnerability dataset with commit data. In: Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Sym- posium on the Foundations of Software Engineering...

    2021

  20. [28]

    OpenAI blog1(8), 9 (2019)

    Radford,A.,Wu,J.,Child,R.,Luan,D.,Amodei,D.,Sutskever,I.,etal.:Language models are unsupervised multitask learners. OpenAI blog1(8), 9 (2019)

    2019

  21. [29]

    Nature Communications 15(1), 1829 (2024)

    Rosenblatt, M., Tejavibulya, L., Jiang, R., Noble, S., Scheinost, D.: Data leak- age inflates prediction performance in connectome-based machine learning models. Nature Communications 15(1), 1829 (2024)

    2024

  22. [30]

    In: Proceedings of the 40th Interna- tional Conference on Software Engineering: Software Engineering in Practice

    Sadowski, C., Söderberg, E., Church, L., Sipko, M., Bacchelli, A.: Modern code review: a case study at google. In: Proceedings of the 40th Interna- tional Conference on Software Engineering: Software Engineering in Practice. p. 181–190. ICSE-SEIP ’18, Association for Computing...

    2018

  23. [31]

    https://doi.org/10.1109/TNN.2008.2005605

    Scarselli, F., Gori, M., Tsoi, A.C., Hagenbuchner, M., Monfardini, G.: The graph neuralnetworkmodel.IEEETransactionsonNeuralNetworks 20(1),61–80(2009). https://doi.org/10.1109/TNN.2008.2005605

    2009

  24. [32]

    In: 2010 IEEE Symposium on Security and Privacy

    Schwartz, E.J., Avgerinos, T., Brumley, D.: All you ever wanted to know about dynamic taint analysis and forward symbolic execution (but might have been afraid to ask). In: 2010 IEEE Symposium on Security and Privacy. pp. 317–331 (2010). https://doi.org/10.1109/SP.2010.26

    2010 doi

  25. [33]

    Shi, K., Altınbüken, D., Anand, S., Christodorescu, M., Grünwedel, K., Koenings, A., Naidu, S., Pathak, A., Rasi, M., Ribeiro, F., Ruffin, B., Sanyam, S., Tabachnyk, M., Toth, S., Tu, R., Welp, T., Yin, P., Zaheer, M., Chandra, S., Sutton, C.: Natural language outlines for cod...

    2024 arXiv

  26. [34]

    In: 2016 IEEE International Conference on Software Testing, Verification and Val- idation (ICST)

    Stivalet, B., Fong, E.: Large scale generation of complex and faulty php test cases. In: 2016 IEEE International Conference on Software Testing, Verification and Val- idation (ICST). pp. 409–415 (2016).https://doi.org/10.1109/ICST.2016.43

    2016 doi

  27. [35]

    In: Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering

    Svyatkovskiy, A., Deng, S.K., Fu, S., Sundaresan, N.: Intellicode compose: code generation using transformer. In: Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. p. 1433–1443. ESEC/...

    2020

  28. [36]

    Team, G.: Gemini: A family of highly capable multimodal models (2024),https: //arxiv.org/abs/2312.11805

    2024 arXiv

  29. [37]

    Templeton, A., Conerly, T., Marcus, J., Lindsey, J., Bricken, T., Chen, B., Pearce, A., Citro, C., Ameisen, E., Jones, A., et al.: Scaling monosemanticity: extracting interpretable features from claude 3 sonnet, transformer circuits thread (2024)

    2024

  30. [38]

    Queue 19(4), 29–41 (Sep 2021)

    Thomson, P.: Static analysis: An introduction: The fundamental challenge of software engineering is one of complexity. Queue 19(4), 29–41 (Sep 2021). https://doi.org/10.1145/3487019.3487021, https://doi.org/10.1145/ 3487019.3487021

    2021

  31. [39]

    Tymchuk, Y.: The false false positives of static analysis (2017)

    2017

  32. [40]

    In: 2024 IEEE Symposium on Security and Privacy (SP)

    Ullah, S., Han, M., Pujar, S., Pearce, H., Coskun, A., Stringhini, G.: Llms can- not reliably identify and reason about security vulnerabilities (yet?): A compre- hensive evaluation, framework, and benchmarks. In: 2024 IEEE Symposium on Security and Privacy (SP). pp. 862–880. ...

    2024

  33. [41]

    In: Guyon, I., Luxburg, Title Suppressed Due to Excessive Length 21 U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R

    Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L.u., Polosukhin, I.: Attention is all you need. In: Guyon, I., Luxburg, Title Suppressed Due to Excessive Length 21 U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. (e...

    2017

  34. [42]

    In: Moens, M.F., Huang, X., Specia, L., Yih, S.W.t

    Wang, Y., Wang, W., Joty, S., Hoi, S.C.: CodeT5: Identifier-aware unified pre- trained encoder-decoder models for code understanding and generation. In: Moens, M.F., Huang, X., Specia, L., Yih, S.W.t. (eds.) Proceedings of the 2021 Con- ference on Empirical Methods in Natural ...

    2021 doi

  35. [43]

    ACM Comput

    Yang, Y., Xia, X., Lo, D., Grundy, J.: A survey on deep learning for software engineering. ACM Comput. Surv.54(10s) (Sep 2022).https://doi.org/10.1145/ 3505243, https://doi.org/10.1145/3505243

    2022 doi

  36. [44]

    In: Wallach, H., Larochelle, H., Beygelzimer, A., d'Alché-Buc, F., Fox, E., Garnett, R

    Zhou, Y., Liu, S., Siow, J., Du, X., Liu, Y.: Devign: Effective vulnerability identifi- cation by learning comprehensive program semantics via graph neural networks. In: Wallach, H., Larochelle, H., Beygelzimer, A., d'Alché-Buc, F., Fox, E., Garnett, R. (eds.) Advances in Neur...

    2019

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.