Pith. sign in

REVIEW 4 major objections 5 minor 74 references

A sentence transformer trained on attack technique descriptions can name the CVE vulnerabilities behind an attack, reaching F1 89.0 while uncovering 275 links missing from MITRE's repositories.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-05 11:53 UTC pith:P2KVAWRL

load-bearing objection The F1=89 headline is attack-level triage, not per-vulnerability identification; with link-level metrics at ~58, the paper still deserves review but needs an honest reframing. the 4 major comments →

arxiv 2509.02077 v2 pith:P2KVAWRL submitted 2025-09-02 cs.CR cs.CLcs.LG

From Attack Descriptions to Vulnerabilities: A Sentence Transformer-Based Approach

Refat Othman , Diaeddin Rimawi , Bruno Rossi , Barbara Russo This is my paper
classification cs.CR cs.CLcs.LG
keywords Cyber Threat IntelligenceMITRE ATT&CKCAPECCWECVESentence TransformerAttack-vulnerability linkingsemantic similarity
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to show that the link between an observed cyberattack and the CVE vulnerabilities it exploits can be recovered automatically from text: a fine-tuned sentence transformer, given an ATT&CK technique description, ranks roughly 295,000 CVE descriptions by semantic similarity and cuts the list at a threshold. Across 14 sentence transformers and four kinds of attack text, the MMPNet model on Technique descriptions reaches F1 89.0 (precision 84.0, recall 94.7), and the authors argue that Technique descriptions carry the strongest linking signal. They further claim that the model's 'extra' predictions are not pure noise: manual review of 434 predicted-but-undocumented links confirmed 275 as genuinely valid technique–CVE links across 60 techniques, meaning MITRE's tables are incomplete by at least 12 percent and can be enriched automatically. If true, this shortens the path from 'we saw this technique' to 'patch these CVEs', turning a manually curated, fragmented set of repositories into a machine-readable, updatable mapping.

Core claim

The paper's central discovery, on its own terms, is that attack and vulnerability descriptions lie close enough in sentence-embedding space for cosine-similarity ranking to work as a linking rule. Fourteen pre-trained sentence transformers are fine-tuned on an annotated dataset in which each attack carries the CVEs reachable through explicit MITRE links (Technique to CAPEC pattern to CWE to example CVE); every CVE description is then scored against each attack description. MMPNet, a dot-product-trained MPNet model, performs best on Technique descriptions: F1 89.0, precision 84.0, recall 94.7, AUC 0.82, plus 56 percent mapping accuracy and 61 percent detection accuracy against the ground trut

What carries the argument

Two objects carry the argument. First, MMPNet (multi-qa-mpnet-base-dot-v1): a pre-trained MPNet sentence transformer, fine-tuned here with CosineSimilarityLoss, that embeds attack and CVE descriptions into 768-dimensional vectors; its dot-product training is credited with capturing context beyond mere angle, which is why it beats the other 13 models. Second, the ground-truth mapping M(a), built by chaining explicit links MITRE already publishes: an ATT&CK Technique is matched to the CAPEC attack patterns that mention it, each pattern to the CWEs that cite it, and each CWE to the CVE examples on its page. Predictions are made by ranking all CVE embeddings against an attack embedding with cosi

Load-bearing premise

The whole evaluation assumes that chaining MITRE's explicit links—technique to attack pattern to weakness to example CVE—produces the complete and correct set of real attack–vulnerability connections, even though MITRE's own pages describe their CVE lists as curated examples, not complete lists.

What would settle it

Take the Technique–CVE pairs MITRE added to its repositories after this paper's data snapshot (the paper reports at least one such case) and ask the fine-tuned MMPNet model at threshold rho=58 to rank them: if it does not retrieve most of these independently confirmed real links, the reported 94.7 recall is an artifact of evaluating against an incomplete oracle. A second check: have two independent analysts, blinded to the authors' decisions, re-review the 434 predicted-but-unlinked CVEs; the 275-link claim stands only if their agreement approaches the authors' 63 percent hit rate.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Technique descriptions can be turned into ranked CVE candidate lists automatically, so a defender who sees a technique can be handed a short patch list instead of a manual repository hunt.
  • Adopting the 275 validated Technique–CVE links into MITRE and downstream databases raises documented attack–vulnerability coverage by roughly 12 percent.
  • Because Technique text beats Tactic, Procedure, and Attack Pattern text across models, future attack-to-vulnerability automation should be built on technique-level descriptions.
  • The measured incompleteness of MITRE's curated links is itself a result: repository enrichment can become a continuous, machine-assisted process rather than pure manual curation.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • My extension: the same ranking pipeline should transfer to noisier, non-curated attack text such as incident reports and security-news articles; the paper lists this as future work, but the released code makes it immediately testable, and the transfer could be measured by how many of the model's top-ranked CVEs match NVD records for the affected product.
  • My caveat: precision and recall are computed against an oracle built from MITRE's own example links, so the numbers measure agreement with a curated, text-similarity-biased list, not with an absolute ground truth; a differently sourced oracle would likely shift them.
  • My extension: per-technique thresholds, which the paper flags as future work, are a cheap, high-value experiment given the released dataset, since a single global rho visibly trades precision for recall unequally across techniques.
  • My caveat: the 275-link enrichment count rests on agreement between two authors; a blinded multi-analyst replication, or tracking which of the submitted links MITRE accepts, would turn the enrichment claim into an auditable number.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes an automated approach to link MITRE ATT&CK attack descriptions to CVE vulnerability reports using sentence transformers. The authors construct a ground-truth dataset by chaining explicit links from ATT&CK Techniques through CAPEC attack patterns and CWE entries to CVE IDs, then evaluate 14 sentence-transformer models on four attack-description types (Tactic, Technique, Procedure, Attack Pattern). The central reported result is that the multi-qa-mpnet-base-dot-v1 (MMPNet) model, applied to Technique descriptions, achieves an F1-score of 89.0, precision of 84.0, and recall of 94.7 in identifying CVEs from attack texts. The paper also reports link-level mapping/detection accuracies of 56% and 61%, and claims to discover 275 previously undocumented Technique–CVE links through manual review. The main contributions are the comparative model evaluation, the annotated dataset, and the proposed method for enriching incomplete MITRE mappings.

Significance. If the headline F1 result held as stated, the paper would provide useful evidence that an off-the-shelf sentence transformer can automate a substantial portion of attack-to-vulnerability linking and enrich incomplete MITRE repositories. The comparative evaluation across 14 models and four attack-text types is a useful reference point, and the release of the dataset and code supports reproducibility and follow-up work. However, the central quantitative claim is not supported by the evaluation as designed: the F1=89.0 is computed at the level of attack texts, not at the level of individual Technique–CVE links, while the paper's own link-level accuracies imply an F1 of roughly 58. The oracle used for ground truth is explicitly incomplete, and the threshold and best-model selection involve fitting on data from the same distribution as the test set. These issues are load-bearing for the paper's main claim that transformers 'identify CVE vulnerabilities with an F1-score of 89.0.' The contribution remains potentially valuable, but the empirical claims need reframing and reanalysis.

major comments (4)
  1. [Abstract; Section 4.1, Table 7; Section 3.1.6, Table 6] The reported F1=89.0 is an attack-level classification metric, not a vulnerability-identification metric. In Table 6, a True Positive is defined as an attack a for which L_rho(a) ∩ M(a) ≠ ∅, i.e., a technique is counted as fully correct if it recovers at least one linked CVE. Consequently, a technique that retrieves one true CVE while missing dozens of linked CVEs still contributes a full positive. The paper's own link-level results in Section 4.2 (Mapping Accuracy = 56%, Detection Accuracy = 61%) give a harmonic mean of about 58, far below 89.0. The abstract and conclusion claim that the model 'identifies CVE vulnerabilities with an F1-score of 89.0' is therefore misleading. Please report link-level precision, recall, and F1 as the primary evaluation of vulnerability identification, or explicitly and consistently label the attack-level score as a triage/retrieval-at-technique metric.
  2. [Section 3.3; Section 3.1.4; Section 3.2] The threshold ρ is selected via ROC analysis on a balanced 50-positive/50-negative sample drawn from the original dataset, and the same threshold is then used to produce the reported results. This is a form of threshold fitting on data that overlaps the evaluation set, and it can inflate the reported precision/recall. In addition, the best model is selected by taking the maximum F1 over 14 models evaluated on the same 10% test split (Section 3.1.4, Section 3.2). With only about 10 positive Techniques in the test set (Table 4: 100 linked Techniques, 10% split), the expected maximum of 14 noisy F1 estimates is substantial. Please select the threshold and the best model using a validation fold only, and report confidence intervals or bootstrap estimates for all headline metrics.
  3. [Section 3.1.1; Section 3.1.2; Section 3.5] The ground truth M(a) is built from explicit links in MITRE pages, but the paper itself quotes CWE pages as stating that their CVE lists are 'a curated list of examples ... not a complete list' (Section 3.1.1). Thus the oracle is known to be incomplete. This incompleteness directly affects the reported precision/recall: a correct but undocumented prediction is counted as a false positive, while an incomplete M(a) can also distort recall and link-level accuracies. The manual validation in Section 3.5 (two authors agreeing on a 'clear semantic match') is not a substitute for a well-defined annotation protocol, and no inter-rater reliability is reported. As a result, the status of the 275 'new links' as true positives versus false positives cannot be fully resolved by the current evaluation design. Please either (a) treat M(a) as a partial oracle and report precision/recall bounds under alt
  4. [Table 4; Section 4.3; Section 5 (External validity)] The evaluation sample is very small: Table 4 lists 100 linked Techniques out of 625 total Technique entries, and with an 80/10/10 split the reported attack-level F1 rests on roughly 10 positive Techniques. This is acknowledged in Section 5, but the abstract and RQ summaries do not convey the fragility of the headline number. Moreover, Section 4.3 and Table 8 refer to '201 Techniques' while Table 4 reports 625 Technique rows; the paper also alternates between 'Techniques' and 'Subtechniques' (Section 2.2 vs. Section 5). These inconsistencies make it difficult to determine the exact population and sample. Please clarify the unit of analysis and provide exact counts, plus uncertainty measures for the reported F1.
minor comments (5)
  1. [Figures 5–7] In the manuscript version provided, the text inside Figures 5, 6, and 7 appears as corrupted vector-path code (e.g., '/uni00000013/uni00000011/...'), making the ROC curve and box plots unreadable. Please ensure the figures are embedded with proper text rendering.
  2. [Section 2.2 vs. Section 5] The paper uses 'Technique' and 'Subtechnique' inconsistently. Section 2.2 says the framework contains 201 Techniques and 424 Subtechniques, while Table 4 reports 625 Techniques; Section 5 says the ground truth is a subset of 100 ATT&CK subtechniques. Please standardize the terminology and counts.
  3. [Section 7, Conclusion] The conclusion calls the approach 'the first approach to automatically link attacks to vulnerabilities,' but the related work and Table 9 include prior attack-to-vulnerability efforts, including the authors' own VULDAT. This overclaim should be softened.
  4. [Section 4.1, Discussion] The discussion states that MMPNet and MSMBERT use dot-product loss functions, which allows them to 'capture the differences in embeddings based on their context semantics.' However, the inference stage uses cosine similarity, not raw dot-product similarity. Please clarify the distinction between training loss and inference metric.
  5. [Abstract; Section 4.1 RQ1 Summary] The phrase 'identifies CVE vulnerabilities with an F1-score of 89.0' should be qualified to specify the unit of analysis (attack text rather than individual CVE links). This is not just a wording issue: it is the main source of the overstated central claim.

Circularity Check

1 steps flagged

RQ3's manual validation of 275 new links uses the same semantic-match criterion the model optimizes, making the 'new link' claim partly self-confirming; RQ1/RQ2 evaluation is held-out and not circular.

specific steps
  1. self definitional [Section 3.5 (Manual validation), with Section 3.1.4 and Section 3.1.5]
    "Missing CVE IDs were considered 'to be linked' when both authors agreed that there was a clear semantic match between the Technique and the CVE descriptions after two iterations."

    The model defines a predicted link as cosine similarity above a threshold between Technique and CVE embeddings (Section 3.1.5), and is fine-tuned with CosineSimilarityLoss (Section 3.1.4). The RQ3 validation of whether an undocumented prediction is a real link uses the same construct: 'clear semantic match' between the very same Technique and CVE descriptions. Thus the 275 'recommended' links are accepted precisely when a human reproduces the semantic similarity the model was trained to compute. Since the MITRE oracle is acknowledged incomplete ('not a complete list'), the false-positive set is ambiguous, and the manual semantic-match check is the only arbiter. The claim that the model 'enriches' MITRE therefore reduces, for the 275 count, to the model's own similarity judgment being re-la

full rationale

The core RQ1/RQ2 evaluation is a supervised sentence-transformer benchmark on a held-out 10% test split (Section 3.1.4), so its F1/Jaccard numbers are not circular in themselves. No self-citation is load-bearing; VULDAT is prior work but the current evaluation stands on the new 14-model comparison. The main circularity is confined to RQ3: the '275 new links' are validated by human agreement on semantic match, which is the same construct the model's cosine similarity implements and was fine-tuned to optimize, making the novelty count partly self-confirming. Separately (a validity concern, not circularity), the headline F1=89 is computed at attack level (any correct CVE makes the technique a true positive, Section 3.1.6 Table 6), while link-level Mapping/Detection Accuracy are 56%/61%; this overstates 'identifying CVE vulnerabilities' but does not by itself make the derivation circular. Overall: partial circularity in the claim of discovering new links, with independent content in the retrieval comparison.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

No new postulated entities (particles, forces, dimensions) are introduced. The only artifacts are the annotated dataset and 275 candidate links, both empirically derived from existing repositories and manual review. The free parameters and domain assumptions above are what the central claim rests on beyond standard pretrained models.

free parameters (3)
  • Similarity threshold rho = 60 initially, ROC-optimal 58 on a balanced sample
    Defines the predicted link set L_rho(a) in Section 3.1.5; chosen by manual/ROC fit on the data and applied globally to all models, so the reported F1 depends on this fitted cutoff.
  • Fine-tuning hyperparameters = 4 epochs, 100 warmup steps
    Chosen by hand following sentence-transformers examples (Section 3.1.4); not systematically tuned, but applied uniformly across models.
  • Train/validation/test split ratios = 80/10/10
    Fixed split chosen for comparability (Section 3.1.4); affects the variance and stability of the reported performance numbers.
axioms (4)
  • domain assumption Explicit links found on MITRE CWE/CVE/CAPEC/ATT&CK pages are a reliable ground truth for attack-vulnerability relations.
    Sections 3.1.1 and 3.1.2 build M(a) from these links, yet the paper quotes CWE pages saying the lists are curated examples and not complete. If false, precision and recall are biased.
  • domain assumption The chain Technique -> CAPEC pattern -> CWE -> CVE captures the true set of CVEs exploitable by a technique.
    Section 3.1.2 constructs ground truth by retaining CAPEC patterns that mention a technique, then CWEs linked to those patterns, then CVEs linked to those CWEs; this many-to-many chain may omit or misattribute links.
  • domain assumption Cosine similarity between sentence embeddings is a valid measure of semantic attack-vulnerability relatedness after fine-tuning.
    Section 3.1.5 adopts cosine similarity based on prior NLP benchmarks; no task-specific calibration beyond the global threshold is shown.
  • domain assumption The 384-token truncation does not cut any relevant description.
    Section 5 asserts all attack and CVE descriptions used are within the 384-token limit; if false, longer descriptions lose context and scores change.

pith-pipeline@v1.4.0-alltime-deepseek-medium · 25640 in / 14347 out tokens · 155499 ms · 2026-08-05T11:53:27.853051+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of From Attack Descriptions to Vulnerabilities: A Sentence Transformer-Based Approach." pith.science (2026). https://pith.science/paper/P2KVAWRL

@misc{pith2026250902077,
  author       = {Pith},
  title        = {Pith review of: From Attack Descriptions to Vulnerabilities: A Sentence Transformer-Based Approach},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/P2KVAWRL}},
  note         = {Machine review of arXiv:2509.02077}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

In the domain of security, vulnerabilities frequently remain undetected even after their exploitation. In this work, vulnerabilities refer to publicly disclosed flaws documented in Common Vulnerabilities and Exposures (CVE) reports. Establishing a connection between attacks and vulnerabilities is essential for enabling timely incident response, as it provides defenders with immediate, actionable insights. However, manually mapping attacks to CVEs is infeasible, thereby motivating the need for automation. This paper evaluates 14 state-of-the-art (SOTA) sentence transformers for automatically identifying vulnerabilities from textual descriptions of attacks. Our results demonstrate that the multi-qa-mpnet-base-dot-v1 (MMPNet) model achieves superior classification performance when using attack Technique descriptions, with an F1-score of 89.0, precision of 84.0, and recall of 94.7. Furthermore, it was observed that, on average, 56% of the vulnerabilities identified by the MMPNet model are also represented within the CVE repository in conjunction with an attack, while 61% of the vulnerabilities detected by the model correspond to those cataloged in the CVE repository. A manual inspection of the results revealed the existence of 275 predicted links that were not documented in the MITRE repositories. Consequently, the automation of linking attack techniques to vulnerabilities not only enhances the detection and response capabilities related to software security incidents but also diminishes the duration during which vulnerabilities remain exploitable, thereby contributing to the development of more secure systems.

Figures

Figures reproduced from arXiv: 2509.02077 by Barbara Russo, Bruno Rossi, Diaeddin Rimawi, Refat Othman.

Figure 1
Figure 1. Figure 1: An example of the ATT&CK relations. Patterns allow us to see what techniques in the ATT&CK repository an adversary could use to target the weaknesses and vulnerabilities. For instance, the Pattern “privilege escalation (CAPEC-233)” elevates attackers’ privilege to per￾form an action they are not supposed to be authorized to perform. Related to this Pattern, one can link the Technique “Abuse Elevation Contr… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the methodology for linking attack descriptions to CVE reports. istic self-attention mechanism. These models are a new version of the BERT model [16], a popular model for natural language understanding, that has been created using knowledge distillation [48] to compress the large model BERT (called the teacher model) into a small model MiniLM (called the student model), which uses much fewer pa… view at source ↗
Figure 3
Figure 3. Figure 3: Workflow for prediction and performance evaluation by model and attack type. texts with the selected model (text embedding). The embedding of each attack text is compared for similarity with the embeddings of all vulnerabilities (text similarity analysis). Vulnerability embeddings that have a similarity score greater than the threshold with a given attack embedding are labeled as predicted vulnerabilities … view at source ↗
Figure 4
Figure 4. Figure 4: Graph representation of explicit links between attack types, weaknesses, and vulnerabilities in MITRE repositories, showing the density and direction of existing connections [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: ROC of varying threshold ρ for pair (MMPNet, Technique). The value nearest to the ideal ratios (0,1) corresponds to ρ = 58%. RQ1 Summary: Overall, sentence transformers are effective in identifying CVEs from attack descriptions, particularly for Techniques, where the MMPNet model achieves an F1-score of 89. Our study further indicates that the best-performing models either leverage a high-dimensional embed… view at source ↗
Figure 6
Figure 6. Figure 6: Jaccard similarity across SOTA models. Mapping Accuracy Detection Accuracy Jaccard Method 0.0 0.2 0.4 0.6 0.8 1.0 Min: 0.00 Max: 1.00 Mean: 0.59 Min: 0.05 Max: 1.00 Mean: 0.65 Min: 0.00 Max: 0.93 Mean: 0.44 [PITH_FULL_IMAGE:figures/full_fig_p016_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Performance of the MMPNet-based approach across attack techniques description. [PITH_FULL_IMAGE:figures/full_fig_p016_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

74 extracted references · 67 canonical work pages

  1. [1]

    W. S. Admass, Y . Y . Munaye, A. A. Diro, Cyber security: State of the art, challenges and future directions, Cyber Security and Applications 2 (2024) 100031

    work page 2024

  2. [2]

    M. M. Robert Muggah, Cybercrime to cost the world 10.5 trillion annually by 2025, accessed: January 28, 2024. https://www.weforum. org/agenda/2023/01/global-rules-crack-down-cybercrime/ (2023)

    work page 2025

  3. [3]

    Point, 38% increase in 2022 global cyberattacks, https://blog.checkpoint.com/2023/01/05/ 38-increase-in-2022-global-cyberattacks/ (2024)

    C. Point, 38% increase in 2022 global cyberattacks, https://blog.checkpoint.com/2023/01/05/ 38-increase-in-2022-global-cyberattacks/ (2024)

    work page 2022

  4. [4]

    M. R. Rahman, R. M. Hezaveh, L. Williams, What are the attackers doing now? automating cyberthreat intelligence extraction from text on pace with the changing threat landscape: A survey, ACM Computing Surveys 55 (12) (2023) 1–36

    work page 2023

  5. [5]

    MITRE, Attack, https://attack.mitre.org/ (2025). 20

    work page 2025

  6. [6]

    MITRE, Capec, https://capec.mitre.org/ (2025)

    work page 2025

  7. [7]

    MITRE, Cwe dataset, https://cwe.mitre.org/ (2025)

    work page 2025

  8. [8]

    MITRE, Cve, https://www.cve.org (2024)

    work page 2024

  9. [9]

    Refat, R

    O. Refat, R. Bruno, R. Barbara, A comparison of vulnerability feature extraction methods from textual attack patterns, in: 2024 50th Euromi- cro Conference on Software Engineering and Advanced Applications (SEAA), IEEE, 2024

    work page 2024

  10. [10]

    Armerding, Cve definitions, https://www.csoonline.com/article/3204884/what-is-cve-its-definition-and-purpose

    T. Armerding, Cve definitions, https://www.csoonline.com/article/3204884/what-is-cve-its-definition-and-purpose. html

    work page arXiv

  11. [11]

    R. T. Othman, Vulnerability detection for software-intensive system, in: Proceedings of the 28th International Conference on Evaluation and Assessment in Software Engineering, 2024, pp. 510–515

    work page 2024

  12. [12]

    F. ¨O. S¨onmez, Classifying common vulnerabilities and exposures database using text mining and graph theoretical analysis, Machine Intelli- gence and Big Data Analytics for Cybersecurity Applications (2021) 313–338

    work page 2021

  13. [13]

    Elder, N

    S. Elder, N. Zahan, R. Shu, M. Metro, V . Kozarev, T. Menzies, L. Williams, Do i really need all this work to find vulnerabilities? an empirical case study comparing vulnerability detection techniques on a java application, Empirical Software Engineering 27 (6) (2022) 154

    work page 2022

  14. [14]

    Othman, B

    R. Othman, B. Rossi, B. Russo, Cybersecurity defenses: Exploration of cve types through attack descriptions, in: 2024 50th Euromicro Conference on Software Engineering and Advanced Applications (SEAA), IEEE, 2024, pp. 415–418

    work page 2024

  15. [15]

    Othman, Att&ck2vul - automated vulnerability detection from cyberattack text, accessed: Feb 2, 2025

    R. Othman, Att&ck2vul - automated vulnerability detection from cyberattack text, accessed: Feb 2, 2025. https://github.com/ref3t/ Attack2VUL/tree/main (2025)

    work page 2025

  16. [16]

    Reimers, I

    N. Reimers, I. Gurevych, Sentence-bert: Sentence embeddings using siamese bert-networks, in: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP- IJCNLP), Association for Computational Linguistics, 2019

    work page 2019

  17. [17]

    Ra ffel, N

    C. Ra ffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y . Zhou, W. Li, P. J. Liu, Exploring the limits of transfer learning with a unified text-to-text transformer, Journal of machine learning research 21 (140) (2020) 1–67

    work page 2020

  18. [18]

    Othman, Vuldat- vulnerability dataset, accessed: Feb 2, 2025

    R. Othman, Vuldat- vulnerability dataset, accessed: Feb 2, 2025. figshare.Dataset.https://doi.org/10.6084/m9.figshare. 25828102.v1 (2025)

    doi:10.6084/m9.figshare 2025

  19. [19]

    Y . Dong, Y . Tang, X. Cheng, Y . Yang, Dekedver: A deep learning-based multi-type software vulnerability classification framework using vulnerability description and source code, Information and Software Technology 163 (2023) 107290

    work page 2023

  20. [20]

    Elder, M

    S. Elder, M. R. Rahman, G. Fringer, K. Kapoor, L. Williams, A survey on software vulnerability exploitability assessment, ACM Computing Surveys 56 (8) (2024) 1–41

    work page 2024

  21. [21]

    Dempsey, P

    K. Dempsey, P. Eavy, G. Moore, Automation support for security control assessments, V ol. 1: Overview (2017) 8011–1

    work page 2017

  22. [22]

    Esposito, D

    M. Esposito, D. Falessi, Validate: A deep dive into vulnerability prediction datasets, Information and Software Technology (2024) 107448

    work page 2024

  23. [23]

    Alevizopoulou, P

    S. Alevizopoulou, P. Koloveas, C. Tryfonopoulos, P. Raftopoulou, Social media monitoring for iot cyber-threats, in: 2021 IEEE International Conference on Cyber Security and Resilience (CSR), IEEE, 2021, pp. 436–441

    work page 2021

  24. [24]

    Gasmi, J

    H. Gasmi, J. Laval, A. Bouras, Information extraction of cybersecurity concepts: An lstm approach, Applied Sciences 9 (19) (2019) 3945

    work page 2019

  25. [25]

    Othman, B

    R. Othman, B. Russo, Vuldat: Automated vulnerability detection from cyberattack text, in: C. Silvano, C. Pilato, M. Reichenbach (Eds.), Embedded Computer Systems: Architectures, Modeling, and Simulation, Springer Nature Switzerland, Cham, 2023, pp. 494–501

    work page 2023

  26. [26]

    Theisen, N

    C. Theisen, N. Munaiah, M. Al-Zyoud, J. C. Carver, A. Meneely, L. Williams, Attack surface definitions: A systematic literature review, Information and Software Technology 104 (2018) 94–103

    work page 2018

  27. [27]

    Iorga, D.-G

    D. Iorga, D.-G. Corlatescu, O. Grigorescu, C. Sandescu, M. Dascalu, R. Rughinis, Yggdrasil—early detection of cybernetic vulnerabilities from twitter, in: 2021 23rd International Conference on Control Systems and Computer Science (CSCS), IEEE, 2021, pp. 463–468

    work page 2021

  28. [28]

    A. L. Queiroz, S. Mckeever, B. Keegan, Eavesdropping hackers: Detecting software vulnerability communication on social media using text mining, in: The Fourth International Conference on Cyber-Technologies and Cyber-Systems, 2019, pp. 41–48

    work page 2019

  29. [29]

    Baccar, Automated mapping of cve vulnerabilties to mitre att&ck framework, Ph.D

    K. Baccar, Automated mapping of cve vulnerabilties to mitre att&ck framework, Ph.D. thesis, Tekup (2021)

    work page 2021

  30. [30]

    Dion ´ısio, F

    N. Dion ´ısio, F. Alves, P. M. Ferreira, A. Bessani, Cyberthreat detection from twitter using deep neural networks, in: 2019 international joint conference on neural networks (IJCNN), IEEE, 2019, pp. 1–8

    work page 2019

  31. [31]

    W. Tang, M. Tang, M. Ban, Z. Zhao, M. Feng, Csgvd: A deep learning approach combining sequence and graph embedding for source code vulnerability detection, Journal of Systems and Software 199 (2023) 111623

    work page 2023

  32. [32]

    X. Sun, Z. Ye, L. Bo, X. Wu, Y . Wei, T. Zhang, B. Li, Automatic software vulnerability assessment by extracting vulnerability elements, Journal of Systems and Software 204 (2023) 111790

    work page 2023

  33. [33]

    NVD, Cvss, https://nvd.nist.gov/vuln-metrics/cvss/v3-calculator

    work page

  34. [34]

    M. R. Rahman, S. K. Basak, R. Mahdavi-Hezaveh, L. A. Williams, Attackers reveal their arsenal: An investigation of adversarial techniques in cti reports, CoRR (2024)

    work page 2024

  35. [35]

    S. B. Son, S. Park, H. Lee, Y . Kim, D. Kim, J. Kim, Introduction to mitre att&ck: Concepts and use cases, in: 2023 International Conference on Information Networking (ICOIN), IEEE, 2023, pp. 158–161

    work page 2023

  36. [36]

    Irshad, A

    E. Irshad, A. B. Siddiqui, Cyber threat attribution using unstructured reports in cyber threat intelligence, Egyptian Informatics Journal 24 (1) (2023) 43–59

    work page 2023

  37. [37]

    MITRE, Mitre att&ck, https://attack.mitre.org/ (2024)

    work page 2024

  38. [38]

    Satvat, R

    K. Satvat, R. Gjomemo, V . Venkatakrishnan, Extractor: Extracting attack behavior from threat reports, in: 2021 IEEE European Symposium on Security and Privacy (EuroS&P), IEEE, 2021, pp. 598–615

    work page 2021

  39. [39]

    Metasploit, https://www.metasploit.com/

    work page

  40. [40]

    Fin6 group, https://attack.mitre.org/groups/G0037/

    work page

  41. [41]

    Y . Wu, Q. Liu, X. Liao, S. Ji, P. Wang, X. Wang, C. Wu, Z. Li, Price tag: towards semi-automatically discovery tactics, techniques and procedures of e-commerce cyber threat intelligence, IEEE Transactions on Dependable and Secure Computing (2021)

    work page 2021

  42. [42]

    U. Noor, Z. Anwar, T. Amjad, K.-K. R. Choo, A machine learning-based fintech cyber threat attribution framework using high-level indicators of compromise, Future Generation Computer Systems 96 (2019) 227–242

    work page 2019

  43. [43]

    Muennigho ff, N

    N. Muennigho ff, N. Tazi, L. Magne, N. Reimers, Mteb: Massive text embedding benchmark, arXiv preprint arXiv:2210.07316 (2022). 21

    Pith/arXiv arXiv 2022

  44. [44]

    M. T. Colangelo, M. Meleti, S. Guizzardi, E. Calciolari, C. Galli, A comparative analysis of sentence transformer models for automated journal recommendation using pubmed metadata, Big Data and Cognitive Computing 9 (3) (2025) 67

    work page 2025

  45. [45]

    H. Choi, J. Kim, S. Joe, Y . Gwon, Evaluation of bert and albert sentence embedding performance on downstream nlp tasks, in: 2020 25th International conference on pattern recognition (ICPR), IEEE, 2021, pp. 5482–5487

    work page 2020

  46. [46]

    https://www.sbert.net/docs/pretrained_models.html (2024)

    Sentence transformers, accessed: May 2, 2024. https://www.sbert.net/docs/pretrained_models.html (2024)

    work page 2024

  47. [47]

    W. Wang, F. Wei, L. Dong, H. Bao, N. Yang, M. Zhou, Minilm: Deep self-attention distillation for task-agnostic compression of pre-trained transformers, Advances in Neural Information Processing Systems 33 (2020) 5776–5788

    work page 2020

  48. [48]

    Hinton, O

    G. Hinton, O. Vinyals, J. Dean, Distilling the knowledge in a neural network, stat 1050 (2015) 9

    work page 2015

  49. [49]

    Okonkwo, A

    O. Okonkwo, A. Dridi, E. Vakaj, Leveraging word embeddings and transformers to extract semantics from building regulations text, in: Proceedings of the 11th Linked Data in Architecture and Construction Workshop, 2023

    work page 2023

  50. [50]

    Siino, I

    M. Siino, I. Tinnirello, M. La Cascia, Is text preprocessing still worth the time? a comparative survey on the influence of popular preprocessing methods on transformers and traditional classifiers, Information Systems 121 (2024) 102342

    work page 2024

  51. [51]

    Hiniduma, S

    K. Hiniduma, S. Byna, J. L. Bez, Data readiness for ai: A 360-degree survey, ACM Comput. Surv. 57 (9) (Apr. 2025). doi:10.1145/3722214. URL https://doi.org/10.1145/3722214

    doi:10.1145/3722214 2025

  52. [52]

    Semantic textual similarity, https://github.com/UKPLab/sentence-transformers/blob/master/examples/sentence_ transformer/training/sts/README.md

    work page

  53. [53]

    A. Lobo, P. Oliveira, P. Sampaio, P. Novais, Cost-sensitive learning and threshold-moving approach to improve industrial lots release process on imbalanced datasets, in: International Symposium on Distributed Computing and Artificial Intelligence, Springer, 2022, pp. 280–290

    work page 2022

  54. [54]

    V . S. Sheng, C. X. Ling, Thresholding for making classifiers cost-sensitive, in: Aaai, V ol. 6, 2006, pp. 476–481

    work page 2006

  55. [55]

    J. Ni, C. Qu, J. Lu, Z. Dai, G. H. ´Abrego, J. Ma, V . Y . Zhao, Y . Luan, K. B. Hall, M.-W. Chang, Y . Yang, Large dual encoders are generalizable retrievers (2021). arXiv:2112.07899. URL https://arxiv.org/abs/2112.07899

    Pith/arXiv arXiv 2021

  56. [56]

    D. I. Sjøberg, G. R. Bergersen, Construct validity in software engineering, IEEE Transactions on Software Engineering 49 (3) (2022) 1374– 1396

    work page 2022

  57. [57]

    W. R. Shadish, T. D. Cook, D. T. Campbell, Experimental and quasi-experimental designs for generalized causal inference / William R. Shedish, Thomas D. Cook, Donald T. Campbell, Wadsworth, Cengage Learning, Belmont, CA, 2002

    work page 2002

  58. [58]

    Wohlin, P

    C. Wohlin, P. Runeson, M. H ¨ost, M. C. Ohlsson, B. Regnell, A. Wessl´en, et al., Experimentation in software engineering, V ol. 236, Springer, 2012

    work page 2012

  59. [59]

    O ffsec, Exploit-db, https://www.exploit-db.com/

    work page

  60. [60]

    Kuppa, L

    A. Kuppa, L. Aouad, N.-A. Le-Khac, Linking cve’s to mitre att&ck techniques, in: Proceedings of the 16th International Conference on Availability, Reliability and Security, 2021, pp. 1–12

    work page 2021

  61. [61]

    Devlin, M.-W

    J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, Bert: Pre-training of deep bidirectional transformers for language understanding, in: Proceed- ings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), 2019, pp. 4171–4186

    2019

  62. [62]

    J. Sun, Z. Xing, H. Guo, D. Ye, X. Li, X. Xu, L. Zhu, Generating informative cve description from exploitdb posts by extractive summariza- tion, CoRR (2021)

    work page 2021

  63. [63]

    Lakhdhar, S

    Y . Lakhdhar, S. Rekhis, Machine learning based approach for the automated mapping of discovered vulnerabilities to adversial tactics, in: 2021 IEEE Security and Privacy Workshops (SPW), IEEE, 2021, pp. 309–317

    work page 2021

  64. [64]

    Grigorescu, A

    O. Grigorescu, A. Nica, M. Dascalu, R. Rughinis, Cve2att&ck: Bert-based mapping of cves to mitre att&ck techniques, Algorithms 15 (9) (2022) 314

    work page 2022

  65. [65]

    Ampel, S

    B. Ampel, S. Samtani, S. Ullman, H. Chen, Linking common vulnerabilities and exposures to the mitre att&ck framework: A self-distillation approach, 1st KDD Workshop on AI-enabled Cybersecurity Analytics (2021)

    work page 2021

  66. [66]

    Y . Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, V . Stoyanov, Roberta: A robustly optimized bert pretraining approach, arXiv preprint arXiv:1907.11692 (2019)

    Pith/arXiv arXiv 1907

  67. [67]

    Hemberg, A

    E. Hemberg, A. Srinivasan, N. Rutar, U.-M. O’Reilly, Sourcing language models and text information for inferring cyber threat, vulnerability and mitigation relationships (2022)

    work page 2022

  68. [68]

    Kanakogi, H

    K. Kanakogi, H. Washizaki, Y . Fukazawa, S. Ogata, T. Okubo, T. Kato, H. Kanuka, A. Hazeyama, N. Yoshioka, Tracing cve vulnerability information to capec attack patterns using natural language processing techniques, Information 12 (8) (2021) 298

    work page 2021

  69. [69]

    Othman, B

    R. Othman, B. Rossi, B. Russo, A comparison of vulnerability feature extraction methods from textual attack patterns, in: 2024 50th Euromi- cro Conference on Software Engineering and Advanced Applications (SEAA), IEEE, 2024, pp. 419–422

    work page 2024

  70. [70]

    Hemberg, J

    E. Hemberg, J. Kelly, M. Shlapentokh-Rothman, B. Reinstadler, K. Xu, N. Rutar, U.-M. O’Reilly, Linking threat tactics, techniques, and patterns with defensive weaknesses, vulnerabilities and affected platform configurations for cyber hunting (2021). arXiv:2010.00533

    Pith/arXiv arXiv 2021

  71. [71]

    Ramos, et al., Using tf-idf to determine word relevance in document queries, in: Proceedings of the first instructional conference on machine learning, V ol

    J. Ramos, et al., Using tf-idf to determine word relevance in document queries, in: Proceedings of the first instructional conference on machine learning, V ol. 242, Citeseer, 2003, pp. 29–48

    work page 2003

  72. [72]

    J. H. Lau, T. Baldwin, An empirical evaluation of doc2vec with practical insights into document embedding generation, in: Proceedings of the 1st Workshop on Representation Learning for NLP, Association for Computational Linguistics, 2016

    work page 2016

  73. [73]

    Kanakogi, H

    K. Kanakogi, H. Washizaki, Y . Fukazawa, S. Ogata, T. Okubo, T. Kato, H. Kanuka, A. Hazeyama, N. Yoshioka, Comparative evaluation of nlp-based approaches for linking capec attack patterns from cve vulnerability information, Applied Sciences 12 (7) (2022) 3400

    work page 2022

  74. [74]

    Center of Excellence for Cybercrime, Cy- bersecurity and Protection of Critical Information Infrastructures,

    SecurityWeek, News vulnerabilities, https://www.securityweek.com/category/vulnerabilities/. 22 Refat Othman is a PhD candidate in Advanced-Systems Engineering at the Free University of Bozen-Bolzano, specializing in cybersecurity. He works as a researcher at the Cybersecu- rity Laboratory (CSLab), where his research focuses on automating vulnerability det...

    work page 2008