Pith. sign in

REVIEW 4 major objections 7 minor 1 cited by

Evaluating Language Models For Threat Detection in IoT Security Logs

T0 review · 4 major / 7 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Fine-tuned open-source LLMs, led by DeepSeek-R1, classify multi-class IoT attacks at roughly 0.75 F1, beating classical baselines near 0.50, and can also generate CAPEC-based mitigation guidance.

desk verdict The detection half is a useful, honest benchmark for fine-tuned small LLMs on Edge-IIoTset; the mitigation half is circular and should be treated as a memorization check, not evidence of security value. read the letter →

arxiv 2507.02390 v1 pith:RAKHJM35 submitted 2025-07-03 cs.CR cs.AI

Jorge J. Tejero-Fernández , Alfonso Sánchez-Macián This is my paper
classification cs.CRcs.AI
keywords largelanguagemodelsIoTsecuritylogsthreatdetectionanomalyfine-tuningmulti-classclassificationmitigationgenerationEdge-IIoTset
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that fine-tuned open-source large language models can do more than classical machine learning for IoT log analysis: they can identify which of fourteen attack types is present and, in the same response, suggest a mitigation adapted to an IoT environment. The authors compare three open LLMs under zero-shot prompting, few-shot prompting, and fine-tuning, against Random Forest, XGBoost, LightGBM, and an MLP, using a subset of the Edge-IIoTset dataset. Their headline finding is that fine-tuning lifts multi-class F1 to roughly 0.75 for the best model, DeepSeek-R1, while the best classical baseline stays near 0.50. The paper also shows that a second fine-tuning stage, built from a manual mapping of attack labels to CAPEC attack patterns, lets the model emit mitigation text with high semantic and lexical similarity to reference responses.

What carries the argument

The argument is carried by a prompt-and-fine-tune pipeline that turns tabular logs into text and then into actionable text. Each Edge-IIoTset row is reduced to seven features chosen by a Random Forest, including DNS query name length, MQTT protocol name, message, topic, and connection-ack flags, TCP options, and TCP destination port, then encoded, normalized, and written as a natural-language instruction. Low-Rank Adaptation (LoRA), a parameter-efficient fine-tuning method that updates small adapter matrices instead of full model weights, adapts three open models to classify those prompts. A second stage maps each of the fourteen attack labels to a CAPEC attack pattern, rewrites the generic countermeasures into IoT-specific mitigation examples using the best model, and fine-tunes on those examples so the output couples a predicted label with a mitigation recommendation. The machinery matters because it lets a structured dataset be handled by models not designed for tabular features, and it converts a detection label into the beginning of an incident response.

What would settle it

Run the same comparison under distribution shift: train on Edge-IIoTset rows from one set of protocols or devices and test on rows from a later time window, a software update, or an unseen device type, with feature selection performed only on the training portion; if the fine-tuned LLM's advantage over XGBoost narrows significantly or reverses, the paper's central claim fails.

Watch

Extended reading notes

Core claim

The paper claims that, after LoRA fine-tuning on a few thousand labeled rows, LLMs give better results on multi-class attack classification than the corresponding baseline models. DeepSeek-R1-Distill-Llama-8B reaches an F1 of 0.7479 when trained on the smaller set and 0.7154 on the larger set, while XGBoost, the strongest classical baseline, stays between about 0.49 and 0.50 in both settings. DeepSeek-R1 also outperforms XGBoost on balanced and imbalanced test draws, with the largest remaining failures on rare classes such as MITM and Fingerprinting. Binary threat detection is saturated, with both fine-tuned LLMs and classical models reaching perfect accuracy, so the claimed advantage is specifically about resolving which attack is present. Finally, by mapping each attack label to a CAPEC pattern and fine-tuning on DeepSeek-R1-generated IoT-specific mitigation examples, the models learn to output mitigation guidance; DeepSeek-R1 and LLaMA score at or near 100% ROUGE-L and cosine similarity in several classes, with Qwen close behind.

Load-bearing premise

The comparison assumes that a random 70/15/15 split of the Edge-IIoTset, using features selected from the whole dataset before splitting, represents real IoT deployment, even though the paper motivates LLMs by log drift; if the test logs come from a different distribution, the measured advantage may not survive.

Editorial extensions

If this is right

  • Fine-tuned open-source LLMs can raise multi-class IoT attack classification from roughly 0.50 to roughly 0.75 F1 using only a few thousand labeled rows, which makes them attractive where labeled data are scarce.
  • The same model can be trained to emit a mitigation recommendation alongside the attack label, giving security operations a single step from raw log to suggested response.
  • Because binary detection saturates for both model families, a practical deployment could use a cheap classical binary filter and reserve the LLM for the harder fourteen-way decision.
  • If the mitigation results hold, alert consumers get context-aware remediation suggestions tailored to IoT constraints, not just a bare attack label.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A stricter test of the paper's log-drift motivation would train on one slice of Edge-IIoTset and test on another with different devices, protocols, or time windows; the reported random-split numbers are likely an upper bound on deployment performance.
  • Because the Random Forest feature selection is fit on the full dataset before the train/test split, it could leak information about test labels into the prompts; a fully nested feature selection would give a cleaner measure of the LLM advantage.
  • The mitigation quality metrics reward reproducing a reference text, so they measure fidelity more than practical usefulness; judging whether the suggested actions are executable on real IoT hardware would be a stronger test.
  • The paper's own per-class tables show rare attacks like MITM and Fingerprinting are still missed almost entirely, so the practical next step is not broader multiclass accuracy but better handling of low-frequency classes.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes a pipeline for threat detection and mitigation recommendation in IoT security logs using the Edge-IIoTset dataset. It compares three open-source LLMs (DeepSeek-R1-Distill-Llama-8B, LLaMA 3.2 3B, Qwen 2.5 7B Instruct) under zero-shot, few-shot, and fine-tuning strategies against classical ML baselines (Random Forest, XGBoost, LightGBM, MLP), using seven features selected by Random Forest. The authors report that fine-tuned LLMs outperform classical baselines in multiclass attack classification (e.g., DeepSeek F1 = 0.7479 vs. XGBoost F1 = 0.4987 on the reduced sample setting), while all models reach perfect binary classification after fine-tuning. For mitigation, the authors map the 14 attack types to CAPEC patterns, use DeepSeek to generate IoT-adapted mitigation texts, fine-tune the three LLMs on those texts, and evaluate the generated mitigations using ROUGE-L and cosine similarity against the DeepSeek-generated references. The central claimed contribution is the combined detection-and-mitigation pipeline (Section 3, Step 5; Section 5.4; Conclusion).

Significance. If the classification results are reproducible, the paper would provide useful evidence that fine-tuned open LLMs can outperform classical ML on multiclass IoT attack classification under data-scarce conditions, and the release of code and use of a public dataset are positive features. The detection half is evaluated against external baselines and is, in principle, an independent contribution. However, the mitigation half is currently unsupported because the evaluation is circular, and the classification comparison is weakened by feature-selection leakage, single runs without error bars, and an evaluation protocol that does not address the log-drift motivation advanced in the Introduction. The mitigation claim is therefore not credible as presented, and the strength of the detection claim is substantially capped by the methodological issues.

major comments (4)
  1. [Sections 4.3 and 5.4] The mitigation quality evaluation is circular by construction. In Section 4.3, DeepSeek generates the IoT-adapted mitigation texts from CAPEC descriptions; after cleaning, those same texts become the supervised fine-tuning targets for the three LLMs. Section 5.4 then computes ROUGE-L and cosine similarity between model outputs and these same DeepSeek-generated references. Near-perfect scores therefore measure the models' ability to reproduce their training targets, not the correctness, actionability, or IoT-suitability of the mitigations. The paper itself concedes this confound in the final paragraph of Section 5.4. Without an external reference (e.g., human expert judgment, an independently authored playbook, or direct CAPEC countermeasures not filtered through DeepSeek), the claim of '100% quality' and 'context-aware responses' is unsupported. This is load-bearing because the combined detection-plus-mitigation pipeline is stated as the core contribution.
  2. [Section 3.1 and Section 4.1] Feature selection is performed before the data split. Section 3.1, step 2, states that a Random Forest classifier was used to select the seven most important features; Section 4.1 then describes the 70/15/15 split. Because the feature importance is computed on the full dataset, the test rows have implicitly influenced the feature choice, which is a form of leakage that can inflate both the ML and LLM results. The feature selection must be recomputed using only the training split (or inside a cross-validation loop) so that the test set remains truly held out.
  3. [Tables 8-15] All classification and mitigation results are reported as single point estimates with no error bars, confidence intervals, or multiple seeds. Fine-tuning is stochastic, and the training/testing subsets in Section 4.1 are randomly sampled. Reported differences such as DeepSeek F1 = 0.7479 vs. XGBoost F1 = 0.4987 in Table 10 may not be statistically robust. The paper should report means and standard deviations over at least several independent runs, or otherwise justify that the differences exceed run-to-run variability.
  4. [Section 5.3 and Introduction] The evaluation protocol does not match the paper's stated motivation. The Introduction and Section 2 emphasize log drift and instability as key challenges, yet all experiments use a random 70/15/15 split of Edge-IIoTset, so test rows come from the same distribution as training rows. This does not measure generalization to unseen log formats, new device types, or shifting class priors. Moreover, the per-class results in Tables 13 and 15 show that the model performs near zero on rare classes (e.g., MiTM F1 = 0.0000 and Fingerprinting F1 = 0.0217 in the imbalanced inference of Table 13), so the abstract's general claim that LLMs give better multiclass results should be qualified by per-class performance and by tests that actually probe distribution shift.
minor comments (7)
  1. [Table 8] The F1-score cell for LLaMA in zero-shot contains a leftover LaTeX command (\textbf{0.4028}) and should be formatted as a number.
  2. [Table 7] There is a typo: 'deconding' should be 'decoding' in the few-shot row.
  3. [Section 2] The phrase 'V olume of Hypersphere Minimization' has an erroneous space in 'Volume'.
  4. [Throughout] The notation is inconsistent: 'MiTM' appears in Tables 13 and 15 while 'MITM' appears in Table 3 and the text; 'Deepseek' and 'DeepSeek' are used interchangeably.
  5. [Section 5.4 and Table 16] Table 16 reports mitigation quality only for Qwen, but the text claims that DeepSeek and LLaMA achieved 100% in multiple classes. Per-class tables for all three models should be provided so the claim is directly verifiable.
  6. [Section 3.2 and 4.2] The number of few-shot examples and the exact prompt templates are not specified. Because few-shot performance is highly sensitive to example count and ordering, this information is needed for reproducibility.
  7. [Section 5.4] No concrete example of a generated mitigation is provided in the text. One or two representative examples for a frequent and a rare attack class would help the reader assess the qualitative claims of coherence and IoT adaptation.

Circularity Check

1 steps flagged · score 6.0 of 10

Mitigation-quality scores are circular: ROUGE-L/cosine references are the same DeepSeek-generated texts used as fine-tuning targets, so '100% quality' measures memorization, not security value.

  1. fitted input called prediction [Sections 4.1, 4.3, and 5.4 (mitigation evaluation, additional fine-tuning, and results)]
    "The generated responses were stored in a list, processed to eliminate formatting errors, style inconsistencies, and linguistic noise. These samples constituted the final set used for the supervised fine-tuning of the specific mitigation generation task. ... Cosine Similarity between the embeddings of the generated mitigations and the mitigations expected by the pre-trained model. ... ROUGE-L measures the overlap of word sequences between the generated mitigation and the expected one. ..."

    The 'expected' texts used for ROUGE-L and cosine similarity are the same DeepSeek-generated mitigation samples that were used as supervised fine-tuning targets in Section 4.3. The fine-tuned models are therefore scored on how well they reproduce their own training references. Near-perfect ROUGE-L and cosine similarity demonstrate memorization of the training set, not correctness, actionability, or IoT-suitability of the mitigations. No external playbook, human expert, or independently authored reference is used. The paper itself concedes this confound in Section 5.4, noting that DeepSeek and LLaMA may be favored because the initial samples were generated by DeepSeek, a LLaMA-based model.

full rationale

The classification half of the paper is self-contained and not circular: fine-tuned LLMs are compared against classical ML baselines on the public Edge-IIoTset dataset using standard held-out accuracy, precision, recall, and F1 metrics, with fixed hyperparameters. Those results stand independently of the mitigation evaluation. The circularity is confined to the mitigation-generation contribution. Section 3.3 uses DeepSeek to generate IoT-adapted mitigation texts from CAPEC descriptions; Section 4.3 turns those cleaned generated responses into the supervised fine-tuning targets for all three LLMs; Section 4.1 defines mitigation quality as ROUGE-L and cosine similarity against the 'expected' reference; Section 5.4 then reports near-perfect scores. Because the reference is the training target, the high scores are forced by construction. The paper's own final paragraph in Section 5.4 admits that the initial DeepSeek-generated, LLaMA-based samples may favor DeepSeek and LLaMA during training. The separate concern that the random 70/15/15 split does not test log drift or unseen log formats is a validity limitation, not a circularity, and does not affect this score. Overall, the detection claims retain independent content, but the combined detection-plus-mitigation claim depends on a circular metric, giving a partial-circularity score of 6.

Assumptions & free parameters 3 free parameters · 4 assumptions · 1 invented entities

The central detection comparison adds no fitted constants; it is a standard evaluation. The ledger entries that matter are methodological: the seven features are chosen by a Random Forest fit on the full dataset before any split (a soft leakage channel), the 12k/15k/7.5k subsets are drawn without a fully specified stratification scheme, and the mitigation corpus is synthetic, generated by DeepSeek from CAPEC descriptions, then used both as training target and as evaluation reference. The CAPEC mapping (Table 3) is a hand-made domain assumption with no external validation. These choices, not new physics or math, carry the paper's explanatory burden.

free parameters (3)
  • Random Forest feature-importance threshold = 7 features retained (Table 2)
    The seven input features are selected by a Random Forest fit on the full dataset before any train/test split (Section 3.1, step 2), so the choice encodes information from rows later used for testing; the retention threshold is not reported.
  • Training subset sizes and sampling scheme = 12,000 (binary); 15,000 and 7,500 (multiclass)
    Subsets are drawn from 20.95 million records under GPU memory constraints (Section 4.1); the sampling is described only as random, with no stratification statement, and per-class results in Tables 13 and 15 show extreme imbalance for rare classes.
  • Mitigation generation decoding parameters = temperature 0.7, top_p 0.9, max_new_tokens 200, repetition_penalty 1.1
    Hand-chosen generation settings in Section 4.3 that shape the synthetic mitigation corpus, which is then used both as the fine-tuning targets and as the evaluation reference.
assumptions (4)
  • domain assumption The Edge-IIoTset dataset labels are correct ground truth and representative of real IoT attack traffic.
    Section 3.1 adopts Edge-IIoTset without any external label audit; all reported metrics inherit dataset labeling errors and the simulated environment's fidelity limits.
  • domain assumption A random 70/15/15 split makes training, validation, and test sets i.i.d. samples of the same distribution.
    Section 4.1 defines the split; this contradicts the paper's own stated motivation that real IoT log streams drift over time (Section 2), so the transfer of the reported F1 gaps to deployment is untested.
  • ad hoc to paper The manual mapping of the 14 attack labels to CAPEC patterns in Table 3 is correct, complete, and sufficient.
    Constructed by hand without stated criteria, expert review, or validation; any mapping errors propagate into both the fine-tuning targets and the evaluation references for mitigation generation.
  • ad hoc to paper ROUGE-L and cosine similarity against a DeepSeek-generated reference corpus measure mitigation quality.
    Section 4.3 defines the references and Section 5.4 scores against them; no human security expert or external playbook validates correctness, actionability, or IoT suitability of the mitigations.
invented entities (1)
  • IoT-specific mitigation corpus generated by DeepSeek-R1-Distill-Llama-8B
    purpose: Used as fine-tuning targets for all three LLMs and as the reference for ROUGE-L and cosine-similarity scoring of mitigation quality
    Created by prompting DeepSeek with CAPEC descriptions (Section 4.3) and never checked against human experts, vendor playbooks, or external benchmarks; the evaluation in Section 5.4 therefore measures agreement with this self-generated corpus rather than security correctness.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Evaluating Language Models For Threat Detection in IoT Security Logs." pith.science (2026). https://pith.science/paper/RAKHJM35

@misc{pith2026250702390,
  author       = {Pith},
  title        = {Pith review of: Evaluating Language Models For Threat Detection in IoT Security Logs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RAKHJM35}},
  note         = {Machine review of arXiv:2507.02390}
}
read the original abstract

Log analysis is a relevant research field in cybersecurity as they can provide a source of information for the detection of threats to networks and systems. This paper presents a pipeline to use fine-tuned Large Language Models (LLMs) for anomaly detection and mitigation recommendation using IoT security logs. Utilizing classical machine learning classifiers as a baseline, three open-source LLMs are compared for binary and multiclass anomaly detection, with three strategies: zero-shot, few-shot prompting and fine-tuning using an IoT dataset. LLMs give better results on multi-class attack classification than the corresponding baseline models. By mapping detected threats to MITRE CAPEC, defining a set of IoT-specific mitigation actions, and fine-tuning the models with those actions, the models are able to provide a combined detection and recommendation guidance.

Figures

Figures reproduced from arXiv: 2507.02390 by the authors.

Figure 1
Figure 1. Methodology to evaluate the potential of ML and LLM for threat detection [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Just Testing, Move Along: Evasion of LLM-based System Log Interpretation by Prompt Injection

    cs.CR 2026-07 conditional novelty 6.0 of 10

    Prompt injections framing malicious system logs as authorized testing can flip multiple SOTA LLMs from attack to benign classifications, while their explanations often expose the manipulation.

Reference graph

Works this paper leans on

38 extracted references · 31 canonical work pages · cited by 1 Pith paper

  1. [1]

    Detecting large-scale system problems by mining console logs,

    W. Xu, L. Huang, A. Fox, D. Patterson, and M. I. Jordan, “Detecting large-scale system problems by mining console logs,” in Proceedings of the ACM SIGOPS 22nd Symposium on Operating Systems Principles, 2009, pp. 117–132

    work page 2009

  2. [2]

    Isolation forest,

    F. T. Liu, K. M. Ting, and Z.-H. Zhou, “Isolation forest,” in2008 Eighth IEEE International Conference on Data Mining. IEEE, 2008, pp. 413–422

    work page 2008

  3. [3]

    Anomaly intrusion detection using one class svm,

    Y . Wang, J. Wong, and A. Miner, “Anomaly intrusion detection using one class svm,” inProceedings from the Fifth Annual IEEE SMC Information Assurance Workshop. IEEE, 2004, pp. 358–364

    work page 2004

  4. [4]

    LogGPT: Log anomaly detection via GPT,

    X. Han, S. Yuan, and M. Trabelsi, “LogGPT: Log anomaly detection via GPT,” in 2023 IEEE International Conference on Big Data (BigData), 2023, pp. 1117–1122

    work page 2023

  5. [5]

    LogEval: A Comprehensive Benchmark Suite for Large Language Models In Log Analysis

    T. Cui, S. Ma, Z. Chen, T. Xiao, S. Tao, Y . Liu, S. Zhang, D. Lin, C. Liu, Y . Cai, W. Meng, Y . Sun, and D. Pei, “LogEval: A comprehensive benchmark suite for large language models in log analysis,” _eprint: 2407.01896. [Online]. Available: https://arxiv.org/abs/2407.01896

    work page Pith review arXiv

  6. [6]

    LLM-based event log analysis techniques: A survey,

    S. Akhtar, S. Khan, and S. Parkinson, “LLM-based event log analysis techniques: A survey,” _eprint: 2502.00677. [Online]. Available: https://arxiv.org/abs/2502.00677

    arXiv

  7. [7]

    Revolutionizing cyber threat detection with large language models: A privacy-preserving BERT-based lightweight model for IoT/IIoT devices,

    M. A. Ferrag, M. Ndhlovu, N. Tihanyi, L. C. Cordeiro, M. Debbah, T. Lestable, and N. S. Thandi, “Revolutionizing cyber threat detection with large language models: A privacy-preserving BERT-based lightweight model for IoT/IIoT devices,” IEEE Access, vol. 12, pp. 23 733–23 750, 2024

    work page 2024

  8. [8]

    Ton_iot datasets,

    A. A. et al., “Ton_iot datasets,” https://research.unsw.edu.au/projects/toniot-datasets, 2020, accessed: 2025-04-16

    work page 2020

Show all 38 references
  1. [9]

    Towards the development of realistic botnet dataset in the internet of things for network forensic analytics: Bot-iot dataset,

    N. Koroniotis, N. Moustafa, E. Sitnikova, and B. Turnbull, “Towards the development of realistic botnet dataset in the internet of things for network forensic analytics: Bot-iot dataset,” Future Generation Computer Systems, vol. 100, pp. 779–796, 2019. [Online]. Available: htt...

    2019

  2. [10]

    CLDTLog: System log anomaly detection method based on contrastive learning and dual objective tasks,

    G. Tian, N. Luktarhan, H. Wu, and Z. Shi, “CLDTLog: System log anomaly detection method based on contrastive learning and dual objective tasks,” Sensors, vol. 23, no. 11, 2023. [Online]. Available: https://www.mdpi.com/1424-8220/23/11/5042

    2023

  3. [11]

    LogBERT: Log anomaly detection via BERT,

    H. Guo, S. Yuan, and X. Wu, “LogBERT: Log anomaly detection via BERT,” in 2021 International Joint Conference on Neural Networks (IJCNN), 2021, pp. 1–8

    2021

  4. [12]

    Deeplog: Anomaly detection and diagnosis from system logs through deep learning,

    M. Du, F. Li, G. Zheng, and V . Srikumar, “Deeplog: Anomaly detection and diagnosis from system logs through deep learning,” in Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (CCS ’17) . Association for Computing Machinery, 2017, pp. 1285...

    2017

  5. [13]

    Loganomaly: Unsupervised detection of sequential and quantitative anomalies in unstructured logs,

    R. Zhou, P. Sun, S. Tao, R. Zhang, W. Meng, Y . Liu, Y . Zhu, Y . Liu, D. Pei, S. Zhang, and Y . Chen, “Loganomaly: Unsupervised detection of sequential and quantitative anomalies in unstructured logs,” in Proceedings of the 28th International Joint Conference on Artificial In...

    2019

  6. [14]

    What supercomputers say: A study of five system logs,

    A. Oliner and J. Stearley, “What supercomputers say: A study of five system logs,” in37th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN ’07). IEEE, 2007, pp. 575–584

    2007

  7. [15]

    LLM meets ML: Data-efficient anomaly detection on unseen unstable logs,

    F. Hadadi, Q. Xu, D. Bianculli, and L. Briand, “LLM meets ML: Data-efficient anomaly detection on unseen unstable logs,” _eprint: 2406.07467. [Online]. Available: https://arxiv.org/abs/2406.07467

  8. [16]

    Fine-tuning llms vs non-generative machine learning models: A comparative study of malware detection,

    G. Balan, C.-A. Simion, and D. T. Gavrilut, “Fine-tuning llms vs non-generative machine learning models: A comparative study of malware detection,” in Proceedings of the 17th International Conference on Agents and Artificial Intelligence (ICAART). ScitePress, 2025

    2025

  9. [17]

    Iot-23 dataset,

    S. G. et al., “Iot-23 dataset,” https://www.stratosphereips.org/datasets-iot23, 2020, accessed: 2025-04-16

    2020

  10. [18]

    Cic datasets,

    C. I. for Cybersecurity, “Cic datasets,” https://www.unb.ca/cic/datasets/index.html, 2021, accessed: 2025-04-16. 14 A PREPRINT - S EPTEMBER 10, 2025

    2021

  11. [19]

    Rt-iot 2022: Real-time internet of things dataset,

    “Rt-iot 2022: Real-time internet of things dataset,” https://archive.ics.uci.edu/dataset/942/rt-iot2022, accessed: 2025-04-16

    2022

  12. [20]

    Edge-iiotset: Cyber security dataset of iot & iiot,

    M. A. F. et al., “Edge-iiotset: Cyber security dataset of iot & iiot,” https://www.kaggle.com/datasets/ mohamedamineferrag/edgeiiotset-cyber-security-dataset-of-iot-iiot, accessed: 2025-04-16

    2025

  13. [21]

    Application of large language models to DDoS attack detection,

    M. Guastalla, Y . Li, A. Hekmati, and B. Krishnamachari, “Application of large language models to DDoS attack detection,” in Security and Privacy in Cyber-Physical Systems and Smart Vehicles, Y . Chen, C.-W. Lin, B. Chen, and Q. Zhu, Eds. Springer Nature Switzerland, 2024, pp. 83–99

    2024

  14. [22]

    HackMentor: Fine-tuning large language models for cybersecurity,

    J. Zhang, H. Wen, L. Deng, M. Xin, Z. Li, L. Li, H. Zhu, and L. Sun, “HackMentor: Fine-tuning large language models for cybersecurity,” in 2023 IEEE 22nd International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom), 2023, pp. 452–461

    2023

  15. [23]

    Lora: Low-rank adaptation of large language models,

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen, “Lora: Low-rank adaptation of large language models,” 2021. [Online]. Available: https://arxiv.org/abs/2106.09685

    2021 arXiv

  16. [24]

    LLM4itd: Insider threat detection with fine-tuned large language models,

    M. Zhang, X. Liang, F. Tian, Y . Yang, H. Yu, and B. Li, “LLM4itd: Insider threat detection with fine-tuned large language models,” in 2024 International Conference on Interactive Intelligent Systems and Techniques (IIST), 2024, pp. 236–241

    2024

  17. [25]

    Building cyber language models to unlock new cybersecurity capabilities

    G. Batmaz, “Building cyber language models to unlock new cybersecurity capabilities.” [Online]. Available: https://developer.nvidia.com/blog/building-cyber-language-models-to-unlock-new-cybersecurity-capabilities/

  18. [26]

    Benchmarking large language models for log analysis, security, and interpretation,

    E. Karlsen, X. Luo, N. Zincir-Heywood, and M. Heywood, “Benchmarking large language models for log analysis, security, and interpretation,”Journal of Network and Systems Management, vol. 32, no. 3, p. 59, 2024. [Online]. Available: https://doi.org/10.1007/s10922-024-09831-x

    2024 doi

  19. [27]

    Building a dynamic parserless network log and security alert platform with LLM and pydantic

    I. Labs, “Building a dynamic parserless network log and security alert platform with LLM and pydantic.” [Online]. Available: https://ithy.com/article/llm-pydantic-log-parser-security-kia8dv2t

  20. [28]

    Security vulnerability detection with multitask self-instructed fine-tuning of large language models,

    A. Z. H. Yang, H. Tian, H. Ye, R. Martins, and C. L. Goues, “Security vulnerability detection with multitask self-instructed fine-tuning of large language models,” _eprint: 2406.05892. [Online]. Available: https://arxiv.org/abs/2406.05892

    arXiv

  21. [29]

    RedChronos: A large language model-based log analysis system for insider threat detection in enterprises,

    C. Li, Z. Zhu, J. He, and X. Zhang, “RedChronos: A large language model-based log analysis system for insider threat detection in enterprises,” _eprint: 2503.02702. [Online]. Available: https://arxiv.org/abs/2503.02702

    arXiv

  22. [30]

    LogPrécis: Unleashing language models for automated malicious log analysis: Précis: A concise summary of essential points, statements, or facts,

    M. Boffa, I. Drago, M. Mellia, L. Vassio, D. Giordano, R. Valentim, and Z. B. Houidi, “LogPrécis: Unleashing language models for automated malicious log analysis: Précis: A concise summary of essential points, statements, or facts,” Computers & Security , vol. 141, p. 103805, ...

    2024

  23. [31]

    Mitre att&ck labeling of cyber threat intelligence via llm,

    T. O’Brien, “Mitre att&ck labeling of cyber threat intelligence via llm,” https://sansorg.egnyte.com/dl/ I7dMyUXrb3, 2024, sANS Institute, diciembre 2024

    2024

  24. [32]

    When llms meet cybersecurity: A systematic literature review,

    Y . Zhang, Z. Chen, X. Liu, B. Zhang, L. Wang, and J. Wu, “When llms meet cybersecurity: A systematic literature review,” Cybersecurity, vol. 8, p. 55, 2025. [Online]. Available: https: //cybersecurity.springeropen.com/articles/10.1186/s42400-025-00361-w

    2025 doi

  25. [33]

    Capec: Common attack pattern enumeration and classification,

    MITRE, “Capec: Common attack pattern enumeration and classification,” Último acceso: abril de 2024. [Online]. Available: https://capec.mitre.org/

    2024

  26. [34]

    Transformers: State-of-the-art machine learning for pytorch, tensorflow, and jax,

    H. Face, “Transformers: State-of-the-art machine learning for pytorch, tensorflow, and jax,” 2023, https:// huggingface.co

    2023

  27. [35]

    Unsloth,

    Unsloth Team, “Unsloth,” 2024, https://unsloth.ai/

    2024

  28. [36]

    Deepseek,

    DeepSeek AI, “Deepseek,” 2024, https://huggingface.co/deepseek-ai

    2024

  29. [37]

    Meta AI, “Llama,” 2024, https://huggingface.co/meta-llama

    2024

  30. [38]

    Alibaba DAMO Academy, “Qwen,” 2024, https://huggingface.co/Qwen. 15

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.