Pith. sign in

REVIEW 4 major objections 7 minor 52 references

SIExVulTS: Sensitive Information Exposure Vulnerability Detection System using Transformer Models and Static Analysis

T0 review · 4 major / 7 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that a three-stage pipeline combining transformer classifiers, CodeQL data-flow analysis, and a GraphCodeBERT-based verifier detects CWE-200 sensitive-information-exposure vulnerabilities in Java with enough precision to…

desk verdict A real gap and six real CVEs, but the headline 87.2% precision figure for the flow verifier is likely in-sample and needs a held-out evaluation before it can be believed. read the letter →

arxiv 2508.19472 v1 pith:OEWE7ZYW submitted 2025-08-26 cs.CR cs.AI

Kyler Katz , Sara Moshtari , Ibrahim Mujhid , Mehdi Mirakhorli , Derek Garcia This is my paper · ORCID
classification cs.CRcs.AI
keywords CWE-200sensitiveinformationexposurevulnerabilitydetectionstaticanalysisCodeQLtransformermodelsGraphBERTJavasecurity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that sensitive-information-exposure vulnerabilities (CWE-200) in Java, a category most detectors ignore, can be found reliably by chaining transformer-based classification, CodeQL static data-flow analysis, and a semantic flow-verification step. The three-stage system first labels variables, strings, comments, and API sinks as sensitive or not, then traces source-to-sink paths with CodeQL queries built around an eight-category model of sensitive data, and finally uses a GraphCodeBERT-based classifier to discard flow paths that do not correspond to real exposures. On a benchmark of 300 synthetic samples the full pipeline reports an F1 score of 85.71%, and on 2,555 labeled flows from 31 open-source projects the verification step raises precision from 22.61% to 87.23% while keeping recall at 93.8%. The authors also report six previously unknown CVEs found in Apache IoTDB, Apache Pulsar, a Jenkins plugin, and the Para platform, which is what a practical detector must do to earn trust.

What carries the argument

The load-bearing mechanism is the pairing of the CWE-200 hierarchy with a flow-verification classifier. The hierarchy is re-expressed as eight concrete categories of sensitive data (credentials, PII, financial information, sensitive file paths, configuration information, encryption material, application-specific data, and query parameters); these categories define what the attack-surface classifiers label and which CodeQL source/sink rules are instantiated. Attack surface detection pairs name and method/line context embeddings from Sentence-BERT, concatenates them, and feeds a residual network classifier. The Flow Verification Engine is the second critical mechanism: it parses source code around each CodeQL flow node, serializes deduplicated flows with their CWE identifier and propagation path, embeds them with GraphCodeBERT (a transformer pretrained for code semantics), and runs a transformer-based aggregator plus residual classifier to label each flow as true or false positive. That mechanism is what converts high-recall static analysis into high-precision findings.

What would settle it

Re-run the pipeline while holding out one or more entire projects from the 31 used to train the Flow Verification Engine, and measure precision on those held-out projects. If precision stays near 22.6% instead of 87.2%, the verifier is memorizing labels rather than generalizing.

Watch

Extended reading notes

Core claim

The central claim is that CWE-200 can be treated as a structured detection problem: an eight-category taxonomy of sensitive data derived from the CWE hierarchy, plus transformer classifiers for attack surface, plus customizable CodeQL rules, plus a learned verifier on enriched data-flow paths. The paper claims this combination detects a broad set of CWE-200 subtypes and, crucially, that the verifier removes most static-analysis false positives: global precision on labeled flows from 31 projects rises from 0.226 to 0.872, with recall dropping only from 1.000 to 0.938, and F1 rising from 0.369 to 0.904. The discovery of six confirmed CVEs is presented as evidence that the pipeline finds real, previously unknown exposures in mature projects rather than only synthetic patterns.

Load-bearing premise

The headline precision gain assumes the Flow Verification Engine was evaluated on flows not seen during training; the paper reports the 87.23% figure on the same 31 projects that supplied the 2,555 labeled flows and does not state a train/test split.

Editorial extensions

If this is right

  • On the 31-project flow dataset, every project either improved or stayed the same in precision after verification (28 improved, 3 unchanged), so the precision gain is not driven by a single outlier.
  • Real-world recall is bounded by sink coverage: missed Jenkins CVEs such as CVE-2017-2606 are attributed to unrecognized REST-API sinks, so improving sink detection is the clearest path to finding more vulnerabilities.
  • For CWE subtypes without data flow, such as CWE-615 (sensitive comments), the pipeline reduces to attack-surface detection alone and the benchmark F1 drops to 66.67%, meaning the verification stage cannot help where no flow exists.
  • The six new CVEs show the pipeline can be deployed as a proactive audit tool; the authors report twelve additional reports under review, so the count of confirmed finds may grow.
  • The design instantiates queries only for detected sources and sinks, so it avoids scanning all possible paths and stays efficient on large codebases, an explicit design goal for practical use.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the flow-verification numbers were measured on training data, a held-out project evaluation would likely show lower precision; testing on one or more projects excluded from the 31 would settle this directly.
  • The eight-category sensitive-data taxonomy is language-neutral in principle; porting the same pipeline to Python or JavaScript is plausible, but the sink catalog and CodeQL rules would need rework, and REST/API sinks are the known weak point.
  • The six findings cluster around logging of credentials and configuration values, suggesting that logging-related exposure patterns are a high-yield target and that a specialized logging-sink detector could raise recall quickly.
  • A natural extension is to use the flow verifier's rejected paths as active-learning examples for the attack-surface classifier, since the paper shows false positives concentrate in flows through complex objects such as maps.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper introduces SIExVulTS, a three-stage system for detecting CWE-200 sensitive-information-exposure vulnerabilities in Java: an Attack Surface Detection Engine using sentence embeddings and classifiers; an Exposure Analysis Engine that instantiates CodeQL queries aligned with a CWE-200 hierarchy; and a Flow Verification Engine that uses GraphCodeBERT embeddings and a Transformer aggregator to filter false-positive data flows. The authors evaluate the system on three self-constructed datasets: a CVE-derived dataset with 40 real vulnerabilities, a synthetic benchmark of 300 labeled code samples, and a set of 2,555 manually labeled data flows from 31 open-source Java projects. Headline results are F1 scores above 93% for attack-surface classification, an F1 of 85.71% on the benchmark, a precision improvement from 22.61% to 87.23% attributable to the Flow Verification Engine, and the discovery of six previously unknown CVEs in Apache IoTDB, Apache Pulsar, Jenkins, and Para. The central claim is that SIExVulTS is both accurate and practically useful for CWE-200 detection.

Significance. If the reported quantitative results hold under out-of-sample evaluation, the paper addresses a genuinely under-served vulnerability class (CWE-200) with a thoughtfully designed multi-stage architecture, and it contributes three labeled datasets plus a public implementation. The six assigned CVEs are an unusually strong form of real-world validation; they demonstrate that the system can surface true positives that matter to maintainers. However, the paper's quantitative accuracy claims are currently not substantiated because the evaluation protocols described in Sections V.A and V.C do not establish that the Flow Verification Engine and the Attack Surface Detection classifiers are evaluated on data they did not see during training. The benchmark and flow-level metrics may therefore be in-sample fits rather than predictions. The practical CVE discoveries cannot substitute for a held-out precision estimate, because they measure only true positives and not the false-positive rate.

major comments (4)
  1. [Section V.C.1, V.A.3, IV.C] Table V reports a precision improvement from 22.61% to 87.23% across 31 projects, but the Flow Verification Engine was trained on the 2,555 labeled flows extracted from those same 31 projects (Section V.A.3). Section IV.C states only that training used 'stratified splits' and gives no ratio or project-disjoint split, in contrast to the explicit 70:15:15 split for the Attack Surface Detection Engine in Section IV.A.2. If the flows used for Table V overlap with the training flows, the 87.23% precision is an in-sample fit and does not support the claim that the engine reduces false positives on unseen projects. Please specify the exact train/validation/test protocol for the Flow Verification Engine, hold out entire projects for testing, and report Table V on the held-out test set.
  2. [Section V.A.4 and Table III] The benchmark dataset used for RQ2 appears to have been part of the training data for the Attack Surface Detection classifiers. Table I aggregates samples from the 'CVE' and 'Bench' columns into a single total, and Section V.A.4 refers to the combined 'collected data' when describing training for the Attack Surface Detection models. Unless the benchmark samples were exclusively placed in the 15% test split mentioned in Section IV.A.2, the 85.71% F1 reported in Table III is contaminated by training data. Please clarify whether the benchmark was used to train the classifiers, and if so, re-evaluate the benchmark using a model trained only on non-benchmark data.
  3. [Section V.C.1, Table V] No confidence intervals or per-project variability measures are reported for the precision, recall, F1, or accuracy figures in Table V. The text states that 28 of 31 projects improved, 3 stayed the same, and none decreased, but the table omits this per-project breakdown. Given the relatively small number of projects and the likelihood of project-specific effects, please provide a per-project analysis with confidence intervals (e.g., bootstrap) and clearly state the number of flows per project.
  4. [Section VI.D, V.A.3] The Flow Verification Dataset labels (Yes/No for true and false positive flows) are manually assigned, but the paper reports inter-rater agreement only for variables, strings, and comments (Table VI), not for the flow-level labels. Since Table V's ground truth consists entirely of these flow labels, the reliability of the flow labeling is load-bearing for the precision measurement. Please report Cohen's Kappa (or equivalent) for the flow-level labeling process and provide the labeling instructions in an appendix or repository.
minor comments (7)
  1. [Section IV.C] The Flow Verification Engine training details are underspecified; please state the split ratio, number of hyperparameter search trials, dropout, learning rate, and early-stopping criteria, as was done for the Attack Surface Detection Engine.
  2. [Table III] The CWE-538 row contains only 5 total samples; an F1 of 100% on this row is unstable. Consider reporting bootstrapped confidence intervals or excluding such small categories from the aggregate F1 calculation.
  3. [Section V.C.1, Table V] The text says 22.6% and 87.2% while Table V reports 0.226 and 0.872; please keep the notation consistent throughout.
  4. [Table II] The heading 'Weighed performance' should read 'Weighted performance'.
  5. [Abstract] The abstract reports 'an average F1 score greater than 93%' without the exact value; please report the exact average and, if possible, the standard deviation across the four attack-surface categories.
  6. [Section I] The paper mentions 'Section VIII concludes' but the conclusion section is numbered VIII after an ethics section VII; the cross-reference is correct, but the prose in Section I says 'Finally, Section VIII concludes the paper' while the actual numbering may confuse readers; please verify the section numbering.
  7. [Section II.B] The hierarchical model in Fig. 1 is described as 'adapted' from the CWE repository; please state explicitly which of the shown CWE subtypes are actually covered by the implemented CodeQL queries and which are aspirational.

Circularity Check

1 steps flagged · score 6.0 of 10

The 87.23% Flow-Verification precision in Table V is reported across the same 31 projects whose 2,555 labeled flows trained the engine, with no held-out split stated.

  1. fitted input called prediction [Section V.A.3 (Flow Verification Dataset) and Section V.C.1 / Table V]
    "Flow Verification Dataset: This dataset includes labeled dataflows from 31 real-world open-source Java projects, extracted using our predefined CodeQL queries. Each flow is manually labeled as Yes (true positive) or No (false positive) based on whether it results in actual sensitive data exposure. The dataset is used to train and evaluate the Flow Verification Engine, helping it learn to distinguish between valid and invalid flows."

    Table V, which reports the headline precision gain from 0.226 to 0.872 'across 31 projects tested,' does not state any train/test split. Section IV.C for the Flow Verification Engine says only 'Training is conducted using stratified splits, SMOTE for balancing, and randomized hyperparameter search,' unlike the Attack Surface engine's explicit 70:15:15 split. Since the 2,555 labeled flows from those same 31 projects are the FVE training data, Table V is, as written, an evaluation on the training population: the 87.23% precision figure is a fit to the manual Yes/No labels rather than an out-of-sample prediction.

full rationale

The main circularity is concentrated in the Flow Verification Engine evaluation. Section V.A.3 says the 2,555 flows from 31 projects are used both to train and to evaluate the engine, and Section V.C.1 reports the headline precision lift 'across 31 projects tested' without giving a train/test split or an external test set. As written, Table V therefore measures performance on the same distribution (and likely the same flows) used to fit the GraphCodeBERT classifier, so the 22.61% to 87.23% improvement is an in-sample fit to the manual labels, not an out-of-sample prediction. This is the core accuracy claim in the abstract and in Section V.C.1. However, other parts of the paper are independent: the Attack Surface engine uses an explicit 70:15:15 split; the Jenkins known-CVE detection and the six newly assigned CVEs provide external evidence that the pipeline finds real true positives, even though they do not quantify false-positive rates; and self-citations such as [27] are definitional rather than load-bearing. The benchmark evaluation of the Exposure Analysis engine is a self-consistent coverage test on author-created synthetic examples, not a circular derivation. Overall, the central precision figure is partially circular as reported, but the system's discovery claims are anchored externally, so the paper deserves a 6 rather than a higher score.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central evaluation rests on the authors' own datasets and labels. No formula-driven derivation is present; the system is assembled from fitted transformer classifiers and hand-written CodeQL queries. The key additional assumptions are the completeness of the CWE-200 taxonomy used and the correctness of manual flow labels.

free parameters (2)
  • Attack Surface Detection classifier weights (SentBERT-based) = Trained on authors' CVE + benchmark datasets (2,683 variables, 2,349 strings, 1,003 comments, 3,443 API calls plus…
    The claimed F1 > 93% depends on these fitted weights; no closed-form constant is derived.
  • Flow Verification classifier weights (GraphCodeBERT + Transformer aggregator) = Trained on 2,555 labeled flows from 31 projects
    The precision improvement from 22.61% to 87.23% is generated by this fitted model; its evaluation separation is not specified.
assumptions (3)
  • domain assumption The adapted CWE-200 hierarchy (Fig. 1) accurately enumerates the relevant sensitive-information exposure categories.
    The taxonomy drives the eight sensitivity classes, CodeQL query instantiation, and benchmark design; any omitted exposure class is invisible to the tool.
  • domain assumption Manual labels of sensitive/non-sensitive elements and true/false flows are correct.
    These labels are the ground truth for all reported F1 and precision values; Cohen's Kappa of 0.77 shows agreement but not ground truth.
  • domain assumption CodeQL's global taint tracking detects every source-to-sink path that exists in the code.
    The Exposure Analysis Engine's recall is capped by CodeQL reachability; if it misses flows, the whole pipeline cannot detect them.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SIExVulTS: Sensitive Information Exposure Vulnerability Detection System using Transformer Models and Static Analysis." pith.science (2026). https://pith.science/paper/OEWE7ZYW

@misc{pith2026250819472,
  author       = {Pith},
  title        = {Pith review of: SIExVulTS: Sensitive Information Exposure Vulnerability Detection System using Transformer Models and Static Analysis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OEWE7ZYW}},
  note         = {Machine review of arXiv:2508.19472}
}
read the original abstract

Sensitive Information Exposure (SIEx) vulnerabilities (CWE-200) remain a persistent and under-addressed threat across software systems, often leading to serious security breaches. Existing detection tools rarely target the diverse subcategories of CWE-200 or provide context-aware analysis of code-level data flows. Aims: This paper aims to present SIExVulTS, a novel vulnerability detection system that integrates transformer-based models with static analysis to identify and verify sensitive information exposure in Java applications. Method: SIExVulTS employs a three-stage architecture: (1) an Attack Surface Detection Engine that uses sentence embeddings to identify sensitive variables, strings, comments, and sinks; (2) an Exposure Analysis Engine that instantiates CodeQL queries aligned with the CWE-200 hierarchy; and (3) a Flow Verification Engine that leverages GraphCodeBERT to semantically validate source-to-sink flows. We evaluate SIExVulTS using three curated datasets, including real-world CVEs, a benchmark set of synthetic CWE-200 examples, and labeled flows from 31 open-source projects. Results: The Attack Surface Detection Engine achieved an average F1 score greater than 93\%, the Exposure Analysis Engine achieved an F1 score of 85.71\%, and the Flow Verification Engine increased precision from 22.61\% to 87.23\%. Moreover, SIExVulTS successfully uncovered six previously unknown CVEs in major Apache projects. Conclusions: The results demonstrate that SIExVulTS is effective and practical for improving software security against sensitive data exposure, addressing limitations of existing tools in detecting and verifying CWE-200 vulnerabilities.

Figures

Figures reproduced from arXiv: 2508.19472 by the authors.

Figure 1
Figure 1. CWE-200 Hierarchical Model 1 public class BAD_MedicalRecordSystemLookup{ 2 public void retrieveRecord(String patientId){ 3 Connection connection = null; 4 PreparedStatement statement = null; 5 ResultSet resultSet = null; 6 7 try{ 8 // Establish connection (Simulated) 9 connection = getConnection("jdbc:mysql://localhost:3306/medicalrecordsystem, 10 "user", "password"); 11 12 // Prepare statement to medical record 13 … view at source ↗
Figure 2
Figure 2. Example illustrating the data flow from source to sink, demonstrating how sensitive information propagates through [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. An overview of the SIExVulTS tool. following space: • Learning rate: {1e-5, 1e-4, 1e-3, 1e-2} • Dropout rate: {0.2, 0.3} • Activation functions: ReLU, ELU, Sigmoid • Batch size: {32, 64} • Epochs: {50, 60} A stratified 70:15:15 train/validation/test split was used to ensure representation across classes. During training, early stopping monitored validation loss with a patience of 10 epochs, and learning rate reducti… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Prompt asks ChatGPT to identify sensitive variables in source code [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]
Figure 5
Figure 5. Figure 5: CodeQL query to detect sensitive data in exception messages (CWE-537) [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

52 extracted references · 47 canonical work pages

  1. [1]

    https://github.com/ MSUSEL/hawaii-pique-cwe200/releases/tag/1.0.0, 2025

    hawaii-pique-cwe200: Official release v1.0.0. https://github.com/ MSUSEL/hawaii-pique-cwe200/releases/tag/1.0.0, 2025. GitHub re- lease v1.0.0

    work page 2025

  2. [2]

    S., AND MIKKONEN , T

    A HMAD , A., W ASEEM , M., L IANG , P., F AHMIDEH , M., A KTAR, M. S., AND MIKKONEN , T. Towards human-bot collaborative software architecting with chatgpt. In Proceedings of the 27th international con- ference on evaluation and assessment in software engineering (2023), pp. 279–285

    work page 2023

  3. [3]

    Sinkfinder: harvesting hundreds of unknown interesting function pairs with just one seed

    B IAN , P., LIANG , B., H UANG , J., S HI, W., WANG , X., AND ZHANG , J. Sinkfinder: harvesting hundreds of unknown interesting function pairs with just one seed. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (2020), pp. 1101–1113

    work page 2020

  4. [4]

    D., D HARIWAL , P., N EELAKANTAN , A., S HYAM, P., S ASTRY, G., ASKELL , A., ET AL

    B ROWN , T., M ANN , B., R YDER , N., S UBBIAH , M., K APLAN , J. D., D HARIWAL , P., N EELAKANTAN , A., S HYAM, P., S ASTRY, G., ASKELL , A., ET AL . Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877–1901

    work page 2020

  5. [5]

    A study on prompt design, advantages and limitations of chatgpt for deep learning program repair

    C AO, J., L I, M., W EN, M., AND CHEUNG , S.- C. A study on prompt design, advantages and limitations of chatgpt for deep learning program repair. arXiv preprint arXiv:2304.08191 (2023)

    arXiv 2023

  6. [6]

    Learning to detect memory-related vulnerabilities

    C AO, S., S UN, X., B O, L., W U, R., L I, B., W U, X., T AO, C., Z HANG , T., AND LIU, W. Learning to detect memory-related vulnerabilities. ACM Transactions on Software Engineering and Methodology 33 , 2 (2023), 1–35

    work page 2023

  7. [7]

    Deep learn- ing based vulnerability detection: Are we there yet? IEEE Transactions on Software Engineering 48 , 9 (2021), 3280–3296

    C HAKRABORTY , S., K RISHNA , R., D ING , Y., AND RAY, B. Deep learn- ing based vulnerability detection: Are we there yet? IEEE Transactions on Software Engineering 48 , 9 (2021), 3280–3296

    work page 2021

  8. [8]

    V., B OWYER , K

    C HAWLA , N. V., B OWYER , K. W., H ALL , L. O., AND KEGELMEYER , W. P. Smote: Synthetic minority over-sampling technique. Journal of Artificial Intelligence Research 16 (June 2002), 321–357

    work page 2002

Show all 52 references
  1. [9]

    Deepwukong: Statically detecting software vulnerabilities using deep graph neural network

    C HENG , X., W ANG , H., H UA, J., X U, G., AND SUI, Y. Deepwukong: Statically detecting software vulnerabilities using deep graph neural network. ACM Transactions on Software Engineering and Methodology (TOSEM) 30, 3 (2021), 1–33

    2021

  2. [10]

    Deepxss: Cross site scripting detection based on deep learning

    F ANG , Y., L I, Y., L IU, L., AND HUANG , C. Deepxss: Cross site scripting detection based on deep learning. In Proceedings of the 2018 international conference on computing and artificial intelligence (2018), pp. 47–51

    2018

  3. [11]

    Codebert: A pre-trained model for programming and natural languages, 2020

    F ENG , Z., G UO, D., T ANG , D., D UAN, N., F ENG , X., G ONG , M., SHOU , L., Q IN, B., L IU, T., J IANG , D., AND ZHOU , M. Codebert: A pre-trained model for programming and natural languages, 2020

    2020

  4. [12]

    LineVul: A Transformer-based Line-level Vulnerability Prediction

    F U, M., AND TANTITHAMTHAVORN , C. LineVul: A Transformer-based Line-level Vulnerability Prediction. In Proceedings of the 19th Inter- national Conference on Mining Software Repositories (MSR) (2022), pp. 608–620

    2022

  5. [13]

    GITHUB. Codeql. https://codeql.github.com, -. [Online; accessed 10- July-2024]

    2024

  6. [14]

    Raisin: Identifying rare sensitive functions for bug detection

    H UANG , J., N IE, J., G ONG , Y., Y OU, W., L IANG , B., AND BIAN , P. Raisin: Identifying rare sensitive functions for bug detection. In Pro- ceedings of the IEEE/ACM 46th International Conference on Software Engineering (2024), pp. 1–12

    2024

  7. [15]

    D., M ORAN , K., AND LAM, W

    J ALIL , S., R AFI, S., L ATOZA , T. D., M ORAN , K., AND LAM, W. Chatgpt and software testing education: Promises & perils. In 2023 IEEE international conference on software testing, verification and validation workshops (ICSTW) (2023), IEEE, pp. 4130–4137

    2023

  8. [16]

    Large language models are few- shot testers: Exploring llm-based general bug reproduction

    K ANG , S., Y OON , J., AND YOO, S. Large language models are few- shot testers: Exploring llm-based general bug reproduction. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) (2023), IEEE, pp. 2312–2323

    2023

  9. [17]

    Sentimentgpt: Exploiting gpt for advanced sentiment analysis and its departure from current machine learning

    K HEIRI , K., AND KARIMI , H. Sentimentgpt: Exploiting gpt for advanced sentiment analysis and its departure from current machine learning. arXiv preprint arXiv:2307.10234 (2023)

    2023 arXiv

  10. [18]

    Llm-based and retrieval-augmented control code generation

    K OZIOLEK , H., G R ¨UNER , S., H ARK , R., A SHIWAL , V., L INSBAUER , S., AND ESKANDANI , N. Llm-based and retrieval-augmented control code generation. In Proc. 1st Int. Workshop on Large Language Models for Coffice (LLM4Code) at ICSE (2024), vol. 2024

    2024

  11. [19]

    Vuldeepecker: A deep learning-based system for vulnerability detection

    L I, Z., Z OU, D., X U, S., O U, X., J IN, H., W ANG , S., D ENG , Z., AND ZHONG , Y. Vuldeepecker: A deep learning-based system for vulnerability detection. arXiv preprint arXiv:1801.01681 (2018)

    2018 arXiv

  12. [20]

    Cross-project transfer representation learning for vulnerable function discovery

    L IN, G., Z HANG , J., L UO, W., P AN, L., X IANG , Y., D E VEL, O., AND MONTAGUE , P. Cross-project transfer representation learning for vulnerable function discovery. IEEE Transactions on Industrial Informatics 14, 7 (2018), 3289–3297

    2018

  13. [21]

    L IU, Z., T ANG , Y., L UO, X., Z HOU , Y., AND ZHANG , L. F. No need to lift a finger anymore? assessing the quality of code generation by chatgpt. IEEE Transactions on Software Engineering (2024)

    2024

  14. [22]

    Improving re- quirements completeness: Automated assistance through large language models

    L UITEL , D., H ASSANI , S., AND SABETZADEH , M. Improving re- quirements completeness: Automated assistance through large language models. Requirements Engineering 29 , 1 (2024), 73–95

    2024

  15. [23]

    Common weakness enumeration (cwe)

    MITRE. Common weakness enumeration (cwe). https://cwe.mitre.org, -. [Online; accessed 30-April-2024]

    2024

  16. [24]

    Cwe-200: Exposure of sensitive information to an unautho- rized actor

    MITRE. Cwe-200: Exposure of sensitive information to an unautho- rized actor. https://cwe.mitre.org/data/definitions/200.html, -. [Online; accessed 30-April-2024]

    2024

  17. [25]

    Cwe-359: Exposure of private personal information to an unauthorized actor

    MITRE. Cwe-359: Exposure of private personal information to an unauthorized actor. https://cwe.mitre.org/data/definitions/359.html, -. [Online; accessed 30-April-2024]

    2024

  18. [26]

    Cwe-537: Cwe-537: Java runtime error message containing sensitive information

    MITRE. Cwe-537: Cwe-537: Java runtime error message containing sensitive information. https://cwe.mitre.org/data/definitions/537.html, -. [Online; accessed 30-April-2024]

    2024

  19. [27]

    A grounded theory based approach to characterize software attack surfaces

    M OSHTARI , S., O KUTAN , A., AND MIRAKHORLI , M. A grounded theory based approach to characterize software attack surfaces. In Pro- ceedings of the 44th International Conference on Software Engineering (2022), pp. 13–24

    2022

  20. [28]

    Evaluating and comparing complexity, coupling and a new proposed set of coupling metrics in cross-project vulnerability prediction

    M OSHTARI , S., AND SAMI , A. Evaluating and comparing complexity, coupling and a new proposed set of coupling metrics in cross-project vulnerability prediction. In Proceedings of the 31st annual ACM symposium on applied computing (2016), pp. 1415–1421

    2016

  21. [29]

    O PENAI. Chatgpt. https://openai.com/chatgpt/, -. [Online; accessed 10-July-2024]

    2024

  22. [30]

    Owasp top ten

    OWASP. Owasp top ten. https://owasp.org/www-project-top-ten/, -. [Online; accessed 10-July-2024]

    2024

  23. [31]

    Fine-grained commit- level vulnerability type prediction by cwe tree structure

    P AN, S., B AO, L., X IA, X., L O, D., AND LI, S. Fine-grained commit- level vulnerability type prediction by cwe tree structure. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) (2023), IEEE, pp. 957–969

    2023

  24. [32]

    R AFFEL , C., S HAZEER , N., R OBERTS , A., L EE, K., N ARANG , S., MATENA , M., Z HOU , Y., L I, W., AND LIU, P. J. Exploring the limits of transfer learning with a unified text-to-text transformer, 2023

    2023

  25. [33]

    Sentence-bert: Sentence embeddings using siamese bert-networks

    R EIMERS , N., AND GUREVYCH , I. Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084 (2019)

    2019 arXiv

  26. [34]

    SonarQube: Continuous Code Quality and Security

    S ONAR SOURCE . SonarQube: Continuous Code Quality and Security. https://www.sonarqube.org. Accessed: 2025-01-13

    2025

  27. [35]

    Automatic code summa- rization via chatgpt: How far are we? arXiv preprint arXiv:2305.12865 (2023)

    S UN, W., F ANG , C., Y OU, Y., M IAO, Y., L IU, Y., L I, Y., D ENG , G., HUANG , S., C HEN , Y., Z HANG , Q., ET AL . Automatic code summa- rization via chatgpt: How far are we? arXiv preprint arXiv:2305.12865 (2023)

    2023 arXiv

  28. [36]

    Detection of sql injection based on artificial neural network

    T ANG , P., Q IU, W., H UANG , Z., L IAN , H., AND LIU, G. Detection of sql injection based on artificial neural network. Knowledge-Based Systems 190 (2020), 105528

    2020

  29. [37]

    P., N ATARA- JAN, N., K ANADE , A., P ARTHASARATHY , S., AND RAJAMANI , S

    W ADHWA , N., P RADHAN , J., S ONWANE , A., S AHU , S. P., N ATARA- JAN, N., K ANADE , A., P ARTHASARATHY , S., AND RAJAMANI , S. Core: Resolving code quality issues using llms. Proceedings of the ACM on Software Engineering 1 , FSE (2024), 789–811

    2024

  30. [38]

    Efficiently detecting reentrancy vulnerabilities in complex smart contracts

    W ANG , Z., C HEN , J., W ANG , Y., Z HANG , Y., Z HANG , W., AND ZHENG , Z. Efficiently detecting reentrancy vulnerabilities in complex smart contracts. Proceedings of the ACM on Software Engineering 1 , FSE (2024), 161–181

    2024

  31. [39]

    Spotting silent buffer overflows in execution trace through graph neural network assisted data flow analysis

    W ANG , Z., Y U, L., W ANG , S., AND LIU, P. Spotting silent buffer overflows in execution trace through graph neural network assisted data flow analysis. arXiv preprint arXiv:2102.10452 (2021)

    2021 arXiv

  32. [40]

    S., W EI, Y., AND ZHANG , L

    X IA, C. S., W EI, Y., AND ZHANG , L. Automated program repair in the era of large pre-trained language models. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) (2023), IEEE, pp. 1482–1494

    2023

  33. [41]

    Automated identification of sensitive data from implicit user specification

    Y ANG , Z., AND LIANG , Z. Automated identification of sensitive data from implicit user specification. Cybersecurity 1, 13 (2018). IX. A PPENDIX ChatGPT Prompt for Identifying Sensitive Variables ### Task I want you to detect sensitive variables in the Java Source Code. Sensi...

    2018

  34. [42]

    Authentication and Authorization and Credentials Information: Variables holding passwords, API keys, keys, usernames, verification codes, credentials, etc

  35. [43]

    Personal Identifiable Information (PII): Variables containing names, emails, addresses, social security numbers, health information, accounts, etc

  36. [44]

    Financial Information: Variables related to credit cards, bank account numbers, account IDs, payment IDs, CVV , etc

  37. [45]

    Files Containing Sensitive Information, Sensitive File Paths, URLs/URIs: Variables storing internal URLs/URIs or file paths to files that contain sensitive information or files themselves

  38. [46]

    Sensitive System and Configuration Information: Variables with database, cloud provider, or network connection strings, database schemas, configuration details, environment variables, sensitive settings, controllers, and managers

  39. [47]

    Security and Encryption Information: Variables holding encryption keys, seeds, or certificates

  40. [48]

    Application-Specific Sensitive Data: Variables storing sensitive information such as device details (Names, IDs, properties, objects), Application- specific IDs, email messages, notifications, etc

  41. [49]

    ### Note Exclude variables related to handlers, wrappers, loggers, listeners, generic file paths, and URLs

    Query Parameters: Variables storing sensitive data in HTTP GET requests. ### Note Exclude variables related to handlers, wrappers, loggers, listeners, generic file paths, and URLs. ### File Markers Each file begins with ”—–BEGIN FILE: [FileName]—–” and ends with ”—–END FILE: [...

  42. [50]

    Where it says ”variableName1” and ”variableDescription1” you should replace them with the actual name and description of the sensitive variable

  43. [51]

    • The ”name” field should be the sensitive information found in the variable

    Provide a JSON response for each file that matches the format below. • The ”name” field should be the sensitive information found in the variable. • The ”description” field should tell which category the variable belongs to

  44. [52]

    files": [ {

    Make sure there are no duplicate entries in the response. { "files": [ {"fileName": "FileName1.java", "sensitiveVariables": [ { "name": "variableName1", "description": "variableDescription1"}, { "name": "variableName2", "description": "variableDescription2"}] }]} Fig. 4: Promp...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.