pith. machine review for the scientific record. sign in

arxiv: 2605.13337 · v1 · submitted 2026-05-13 · 💻 cs.CR · cs.LG

Recognition: 1 theorem link

· Lean Theorem

Context-Aware Web Attack Detection in Open-Source SIEM Systems via MITRE ATT&CK-Enriched Behavioral Profiling

Badr Alboushy , Assef Jafar , Mohamad Aljnidi , Mohamad Bashar Disoki , Aref Shaheed
Authors on Pith no claims yet

Pith reviewed 2026-05-14 18:29 UTC · model grok-4.3

classification 💻 cs.CR cs.LG
keywords SIEMweb attacksbehavioral profilingMITRE ATT&CKcontext-aware detectionWazuhgradient boostingconcept drift
0
0 comments X

The pith

Context features from prior events raise web attack detection F1 in open-source SIEM from 0.705 to 0.967.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper establishes that enriching SIEM event analysis with per-source-IP behavioral context vectors significantly improves the accuracy of machine learning classifiers in detecting web application attacks. The context includes distributions of HTTP response statuses, peak rule activations, and frequencies of MITRE ATT&CK techniques from recent events. A two-stage cascade using LightGBM for binary detection and XGBoost for categorization achieves high F1 scores on a dataset of over 46,000 events. This matters because traditional rule-based SIEMs often miss coordinated or novel attacks, and the approach includes retraining to handle changing patterns.

Core claim

Smart-SIEM adds a behavioural context vector per source IP that encodes HTTP response-status distributions, peak rule activation counts, and MITRE ATT&CK technique frequencies from the N most recent prior events. Combined with a hybrid cascade of LightGBM for binary attack detection and XGBoost for six-class categorisation, this yields F1 scores of 0.967 and 0.914 respectively on 46,454 Wazuh events, far exceeding the native rule engine's performance on attacks like brute force.

What carries the argument

Per-source-IP behavioural context vector enriched with MITRE ATT&CK frequencies from recent events

If this is right

  • Context features improve macro F1 from ~0.705 to 0.947-0.967 in binary detection across gradient boosting algorithms.
  • The hybrid cascade achieves 0.967 F1 binary and 0.914 F1 six-class.
  • Wazuh native rules detect 0% of Brute Force and Broken Authentication events while the AI module detects 100% and 98.3%.
  • Self-adaptive retraining recovers F1 from 0.465 to 0.814 after unseen attacks cause drift.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Similar context vectors could enhance detection in other open-source SIEM tools.
  • Extending the context to include user session patterns might address insider threats better.
  • Deploying this in production would require validation on diverse real-world traffic to confirm generalizability.

Load-bearing premise

The purpose-built dataset of 46,454 Wazuh security events accurately reflects real-world web attack distributions and behavioral patterns.

What would settle it

Running the model on a new dataset from a live SIEM deployment with different attack mixes and checking if F1 scores remain above 0.85 without retraining.

read the original abstract

Security Information and Event Management (SIEM) systems aggregate log data from heterogeneous sources to detect coordinated attacks. Traditional rule-based correlation engines struggle to classify multi-step web application attacks because they examine each event without reference to the behavioural history of the originating host. We present Smart-SIEM, an AI module for the open-source Wazuh SIEM platform with two contributions: (1) a per-source-IP behavioural context vector encoding HTTP response-status distributions, peak rule activation counts, and MITRE ATT&CK technique frequencies from the N most recent prior events; (2) a two-stage hybrid cascade combining LightGBM for binary attack detection and XGBoost for six-class attack categorisation. Evaluated on 46,454 purpose-built Wazuh security events, context features improve all tested gradient boosting algorithms from ~0.705 macro F1 to 0.947-0.967 (Stage 1) and 0.876-0.914 (Stage 2), an average gain of +0.254 and +0.324 respectively. The hybrid cascade achieves F1 of 0.967 (binary) and 0.914 (six-class). Wazuh's native rule engine detects 0% of Brute Force and Broken Authentication events; the AI module detects 100% and 98.3% respectively. A self-adaptive retraining mechanism recovers from concept drift: F1 drops from 0.905 to 0.465 when unseen attack types emerge, recovering to 0.814 after retraining on the combined corpus.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 1 minor

Summary. The paper introduces Smart-SIEM, an AI module for the open-source Wazuh SIEM platform. It defines a per-source-IP behavioral context vector that encodes HTTP response-status distributions, peak rule activation counts, and MITRE ATT&CK technique frequencies drawn from the N most recent prior events. This vector feeds a two-stage hybrid cascade (LightGBM for binary attack detection, XGBoost for six-class attack categorization). On a custom corpus of 46,454 Wazuh events the context features are reported to raise macro F1 from ~0.705 to 0.947-0.967 (Stage 1) and 0.876-0.914 (Stage 2); the cascade reaches 0.967 binary and 0.914 six-class F1. Wazuh native rules detect 0 % of Brute Force and Broken Authentication events while the AI module detects 100 % and 98.3 % respectively. A self-adaptive retraining loop is claimed to recover from concept drift.

Significance. If the performance gains are reproducible on organic logs, the work supplies a practical, open-source demonstration that behavioral context plus MITRE ATT&CK enrichment can materially improve detection of multi-step web attacks inside existing SIEM rule engines. The concrete F1 deltas, the explicit comparison against Wazuh’s native detector, and the retraining mechanism constitute usable empirical evidence for the community.

major comments (1)
  1. [Dataset Construction (evaluation section)] Dataset Construction (evaluation section): the manuscript supplies no description of attack generation, labeling procedure, temporal distribution of events per IP, or class balance for the 46,454-event corpus. Because the context vector is built from the N most recent prior events of the same source IP, any synthetic clustering of attack instances would allow the feature to encode label information that would not exist in real logs. This single unverified assumption underpins every reported F1 gain and the 0 % vs 100 % comparison with Wazuh rules.
minor comments (1)
  1. [Abstract] Abstract: the reported F1 numbers are given without any mention of cross-validation scheme, hyper-parameter search, or train/test split ratio, making it harder for readers to gauge robustness.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the constructive critique. The concern about insufficient dataset documentation is valid and we will address it directly in revision.

read point-by-point responses

  1. Referee: Dataset Construction (evaluation section): the manuscript supplies no description of attack generation, labeling procedure, temporal distribution of events per IP, or class balance for the 46,454-event corpus. Because the context vector is built from the N most recent prior events of the same source IP, any synthetic clustering of attack instances would allow the feature to encode label information that would not exist in real logs. This single unverified assumption underpins every reported F1 gain and the 0 % vs 100 % comparison with Wazuh rules.

    Authors: We agree the evaluation section is missing these details. In the revised manuscript we will insert a dedicated subsection (approximately 400 words) that specifies: (1) attack generation via a custom simulator that replays realistic web-application sequences mapped to MITRE ATT&CK techniques (T1110, T1190, T1078, etc.) with inter-event delays drawn from empirical distributions; (2) labeling performed by an independent rule-based oracle that tags an event only when the current log line contains attack indicators, without reference to future events; (3) per-IP temporal structure ensuring that attack bursts are separated by at least 30 s of benign traffic and that the N-window never crosses attack boundaries artificially; (4) explicit class counts (benign 28 412, brute-force 4 872, broken-auth 3 941, etc.) and the exact N=5 window size used. We will also add pseudocode for context-vector construction and a statement that the same temporal ordering is preserved in the train/test split. These additions will allow independent verification that no label leakage occurs and will substantiate the reported F1 deltas. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical evaluation on held-out test data with no reducing equations

full rationale

The paper reports macro-F1 gains from context features on a 46,454-event custom corpus using standard gradient-boosting classifiers. No equations, derivations, or fitted-parameter predictions appear in the described pipeline; the context vector is a direct aggregation of prior events, and all metrics are computed on separate test splits. The evaluation therefore remains independent of any self-referential definition or load-bearing self-citation chain. This is a conventional empirical ML setup whose central claims do not reduce to their inputs by construction.

Axiom & Free-Parameter Ledger

2 free parameters · 1 axioms · 0 invented entities

The central claims rest on standard supervised-learning assumptions plus the representativeness of a purpose-built dataset; no new physical entities are postulated.

free parameters (2)
  • N (number of recent prior events)
    Hyperparameter defining the length of the behavioral context window; value not stated in abstract and must be chosen or tuned.
  • LightGBM and XGBoost hyperparameters
    Model-specific parameters (learning rate, tree depth, etc.) fitted during training on the custom dataset.
axioms (1)
  • domain assumption Events in the 46,454-event corpus are independent and identically distributed with real-world traffic.
    Required for training and generalization claims.

pith-pipeline@v0.9.0 · 5614 in / 1382 out tokens · 39815 ms · 2026-05-14T18:29:41.191587+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

55 extracted references · 1 canonical work pages · 1 internal anchor

  1. [1]

    and Harris, Shon and Harper, Allen and VanDyke, Stephen and Blask, Chris , title =

    Miller, David R. and Harris, Shon and Harper, Allen and VanDyke, Stephen and Blask, Chris , title =. 2010 , isbn =

    2010

  2. [2]

    and Schmidt, Kevin J

    Chuvakin, Anton A. and Schmidt, Kevin J. and Phillips, Christopher , title =. 2012 , isbn =

    2012

  3. [3]

    Event Correlation Engine , school =

    M. Event Correlation Engine , school =. 2009 , note =

    2009

  4. [4]

    International Journal of Science and Research (IJSR) , volume =

    Agrawal, Kavita and Makwana, Hemant , title =. International Journal of Science and Research (IJSR) , volume =

  5. [5]

    Security Information and Event Management (

    Gonz. Security Information and Event Management (. Sensors , volume =. 2021 , publisher =

    2021

  6. [6]

    2017 , isbn =

    Stallings, William and Brown, Lawrie , title =. 2017 , isbn =

    2017

  7. [7]

    Halfond, William G. J. and Viegas, Jeremy and Orso, Alessandro , title =. Proceedings of the International Symposium on Secure Software Engineering (ISSSE) , pages =

  8. [8]

    Grossman, Jeremiah , title =

  9. [9]

    2011 , isbn =

    Stuttard, Dafydd and Pinto, Marcus , title =. 2011 , isbn =

    2011

  10. [10]

    , title =

    Denning, Dorothy E. , title =. IEEE Transactions on Software Engineering , volume =. 1987 , doi =

    1987

  11. [11]

    Journal of Network and Computer Applications , volume =

    Liao, Hung-Jen and Lin, Chun-Hung Richard and Lin, Ying-Chih and Tung, Kuang-Yuan , title =. Journal of Network and Computer Applications , volume =. 2013 , doi =

    2013

  12. [12]

    and Guven, Erhan , title =

    Buczak, Anna L. and Guven, Erhan , title =. IEEE Communications Surveys & Tutorials , volume =. 2016 , doi =

    2016

  13. [13]

    and Kim, Ikkyun and Kim, Kuinam J

    Kwon, Donghwoon and Kim, Hyunjoo and Kim, Jinoh and Suh, Sang C. and Kim, Ikkyun and Kim, Kuinam J. , title =. Cluster Computing , volume =. 2019 , doi =

    2019

  14. [14]

    , title =

    Tavallaee, Mahbod and Bagheri, Ebrahim and Lu, Wei and Ghorbani, Ali A. , title =. Proceedings of the 2009 IEEE Symposium on Computational Intelligence for Security and Defense Applications (CISDA) , pages =. 2009 , doi =

    2009

  15. [15]

    , title =

    Sharafaldin, Iman and Habibi Lashkari, Arash and Ghorbani, Ali A. , title =. Proceedings of the 4th International Conference on Information Systems Security and Privacy (ICISSP) , pages =. 2018 , doi =

    2018

  16. [16]

    2016 IEEE 2nd International Conference on Big Data Security on Cloud (BigDataSecurity) , pages =

    Veeramachaneni, Kalyan and Arnaldo, Ignacio and Korrapati, Vamsi and Bassias, Constantinos and Li, Ke , title =. 2016 IEEE 2nd International Conference on Big Data Security on Cloud (BigDataSecurity) , pages =. 2016 , doi =

    2016

  17. [17]

    and Abushark, Yoosef B

    Sarker, Iqbal H. and Abushark, Yoosef B. and Alsolami, Fawaz and Khan, Asif Irshad , title =. Symmetry , volume =. 2020 , doi =

    2020

  18. [18]

    Computers & Security , volume =

    Ring, Markus and Wunderlich, Sarah and Scheuring, Deniz and Landes, Dieter and Hotho, Andreas , title =. Computers & Security , volume =. 2019 , doi =

    2019

  19. [19]

    Machine Learning , volume =

    Breiman, Leo , title =. Machine Learning , volume =. 2001 , doi =

    2001

  20. [20]

    Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , pages =

    Chen, Tianqi and Guestrin, Carlos , title =. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , pages =. 2016 , doi =

    2016

  21. [21]

    Advances in Neural Information Processing Systems (NeurIPS) , volume =

    Ke, Guolin and Meng, Qi and Finley, Thomas and Wang, Taifeng and Chen, Wei and Ma, Weidong and Ye, Qiwei and Liu, Tie-Yan , title =. Advances in Neural Information Processing Systems (NeurIPS) , volume =

  22. [22]

    Advances in Neural Information Processing Systems (NeurIPS) , volume =

    Prokhorenkova, Liudmila and Gusev, Gleb and Vorobev, Aleksandr and Dorogush, Anna Veronika and Gulin, Andrey , title =. Advances in Neural Information Processing Systems (NeurIPS) , volume =

  23. [23]

    CatBoost: gradient boosting with categorical features support

    Dorogush, Anna Veronika and Ershov, Vasily and Gulin, Andrey , title =. arXiv preprint arXiv:1810.11363 , year =

    work page internal anchor Pith review Pith/arXiv arXiv

  24. [24]

    and Bowyer, Kevin W

    Chawla, Nitesh V. and Bowyer, Kevin W. and Hall, Lawrence O. and Kegelmeyer, W. Philip , title =. Journal of Artificial Intelligence Research , volume =. 2002 , doi =

    2002

  25. [25]

    Journal of Artificial Intelligence Research , volume =

    Fern. Journal of Artificial Intelligence Research , volume =. 2018 , doi =

    2018

  26. [26]

    , title =

    He, Haibo and Garcia, Edwardo A. , title =. IEEE Transactions on Knowledge and Data Engineering , volume =. 2009 , doi =

    2009

  27. [27]

    and Applebaum, Andy and Miller, Doug P

    Strom, Blake E. and Applebaum, Andy and Miller, Doug P. and Nickels, Kathryn C. and Pennington, Adam G. and Thomas, Cody B. , title =. 2018 , number =

    2018

  28. [28]

    Threat Modeling -- A Systematic Literature Review , journal =

    Xiong, Wenjun and Lagerstr. Threat Modeling -- A Systematic Literature Review , journal =. 2019 , doi =

    2019

  29. [29]

    and Lee, Su-In , title =

    Lundberg, Scott M. and Lee, Su-In , title =. Advances in Neural Information Processing Systems (NeurIPS) , volume =

  30. [30]

    A Survey on Concept Drift Adaptation , journal =

    Gama, Jo. A Survey on Concept Drift Adaptation , journal =. 2014 , doi =

    2014

  31. [31]

    IEEE Transactions on Knowledge and Data Engineering , volume =

    Lu, Jie and Liu, Anjin and Dong, Fan and Gu, Feng and Gama, Joao and Zhang, Guangquan , title =. IEEE Transactions on Knowledge and Data Engineering , volume =. 2019 , doi =

    2019

  32. [32]

    Wazuh: The Open Source Security Platform , year =

  33. [33]

    2021 , url =

    Kimminich, Bjoern , title =. 2021 , url =

    2021

  34. [34]

    2022 , url =

    sqlmap: Automatic. 2022 , url =

    2022

  35. [35]

    Acunetix Web Vulnerability Scanner , year =

  36. [36]

    Burp Suite: Web Application Security Testing , year =

  37. [37]

    2022 , url =

    Reeves, OJ and Mehlmauer, Christian , title =. 2022 , url =

    2022

  38. [38]

    Proceedings of the 2010

    Sommer, Robin and Paxson, Vern , title =. Proceedings of the 2010. 2010 , doi =

    2010

  39. [39]

    ACM Computing Surveys , volume =

    Chandola, Varun and Banerjee, Arindam and Kumar, Vipin , title =. ACM Computing Surveys , volume =. 2009 , doi =

    2009

  40. [40]

    Anomaly-Based Network Intrusion Detection: Techniques, Systems and Challenges , journal =

    Garcia-Teodoro, Pedro and Diaz-Verdejo, Jesus and Maci. Anomaly-Based Network Intrusion Detection: Techniques, Systems and Challenges , journal =. 2009 , doi =

    2009

  41. [41]

    Journal of Information Security and Applications , volume =

    Ferrag, Mohamed Amine and Maglaras, Leandros and Moschoyiannis, Stelios and Janicke, Helge , title =. Journal of Information Security and Applications , volume =. 2020 , doi =

    2020

  42. [42]

    and Alazab, Mamoun and Soman, K.P

    Vinayakumar, R. and Alazab, Mamoun and Soman, K.P. and Poornachandran, Prabaharan and Al-Nemrat, Ameer and Venkatraman, Sitalakshmi , title =. IEEE Access , volume =. 2019 , doi =

    2019

  43. [43]

    Expert Systems with Applications , volume =

    Guo, Haixiang and Li, Yijing and Shang, Jennifer and Gu, Mingyun and Huang, Yuanyue and Gong, Bing , title =. Expert Systems with Applications , volume =. 2017 , doi =

    2017

  44. [44]

    and Eshete, Birhanu and Gjomemo, Rigel and Venkatakrishnan, V.N

    Milajerdi, Sadegh M. and Eshete, Birhanu and Gjomemo, Rigel and Venkatakrishnan, V.N. , title =. Proceedings of the 2019. 2019 , doi =

    2019

  45. [45]

    Proceedings of the 33rd Annual Computer Security Applications Conference (

    Husari, Ghaith and Al-Shaer, Ehab and Ahmed, Mohammed and Chu, Bill and Niu, Xi , title =. Proceedings of the 33rd Annual Computer Security Applications Conference (. 2017 , doi =

    2017

  46. [46]

    Proceedings of the 30th

    Yang, Limin and Ciptadi, Arridhana and Laziuk, Ilya and Ahmadzadeh, Ali and Wang, Gang , title =. Proceedings of the 30th

  47. [47]

    and Manadhata, Pratyusa K

    Bhatt, Sandeep N. and Manadhata, Pratyusa K. and Zomlot, Loai , title =. 2014 , doi =

    2014

  48. [48]

    2014 , doi =

    Creech, Gideon and Hu, Jiankun , title =. 2014 , doi =

    2014

  49. [49]

    Proceedings of the 28th

    Pendlebury, Feargus and Pierazzi, Fabio and Jordaney, Roberto and Kinder, Johannes and Cavallaro, Lorenzo , title =. Proceedings of the 28th

  50. [50]

    and Katos, Vasilios , title =

    Nisioti, Antonia and Mylonas, Alexios and Yoo, Paul D. and Katos, Vasilios , title =. 2018 , doi =

    2018

  51. [51]

    2018 10th International Conference on Cyber Conflict (

    Apruzzese, Giovanni and Colajanni, Michele and Ferretti, Luca and Guido, Alessandro and Marchetti, Mirco , title =. 2018 10th International Conference on Cyber Conflict (. 2018 , doi =

    2018

  52. [52]

    Advances in Neural Information Processing Systems , volume =

    Why tree-based models still outperform deep learning on tabular data , author =. Advances in Neural Information Processing Systems , volume =. 2022 , publisher =

    2022

  53. [53]

    2003 , doi =

    Julisch, Klaus , title =. 2003 , doi =

    2003

  54. [54]

    2012 , doi =

    Settles, Burr , title =. 2012 , doi =

    2012

  55. [55]

    Proceedings of the 22nd International Conference on Machine Learning (

    Niculescu-Mizil, Alexandru and Caruana, Rich , title =. Proceedings of the 22nd International Conference on Machine Learning (. 2005 , doi =

    2005