On a dataset of 973 Snort rules, traditional ML models (SVM, F1 up to 0.87) outperformed ChatGPT, Claude, and Gemini (best F1 0.62) at labeling rules with MITRE ATT&CK techniques.
Towards a better labeling process for network security datasets
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
Most network security datasets do not have comprehensive label assignment criteria, hindering the evaluation of the datasets, the training of models, the results obtained, the comparison with other methods, and the evaluation in real-life scenarios. There is no labeling ontology nor tools to help assign the labels, resulting in most analyzed datasets assigning labels in files or directory names. This paper addresses the problem of having a better labeling process by (i) reviewing the needs of stakeholders of the datasets, from creators to model users, (ii) presenting a new ontology of label assignment, (iii) presenting a new tool for assigning structured labels for Zeek network flows based on the ontology, and (iv) studying the differences between generating labels and consuming labels in real-life scenarios. We conclude that a process for structured label assignment is paramount for advancing research in network security and that the new ontology-based label assignation rules should be published as an artifact of every dataset.
citation-role summary
background 1
citation-polarity summary
fields
cs.CR 1years
2024 1verdicts
CONDITIONAL 1roles
background 1polarities
background 1representative citing papers
citing papers explorer
-
Labeling NIDS Rules with MITRE ATT&CK Techniques: Machine Learning vs. Large Language Models
On a dataset of 973 Snort rules, traditional ML models (SVM, F1 up to 0.87) outperformed ChatGPT, Claude, and Gemini (best F1 0.62) at labeling rules with MITRE ATT&CK techniques.