Pith. sign in

REVIEW 3 major objections 7 minor 81 references

An Automated Attack Investigation Approach Leveraging Threat-Knowledge-Augmented Large Language Models

T0 review · 3 major / 7 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A kill-chain knowledge base lets an LLM rebuild full attack chains from raw logs

desk verdict The architecture is a genuine first and the EB/Linux generalization results are worth taking seriously, but the headline 97.1%/0.2% numbers are not supported because the knowledge base is built from the same ATLAS scenarios used for measurement. read the letter →

arxiv 2509.01271 v1 pith:4GQLPNYR submitted 2025-09-01 cs.CR

Rujie Dai , Peizhuo Lv , Yujiang Gui , Qiujian Lv , Yuanyuan Qiao , Yan Wang , Degang Sun , Weiqing Huang
show 2 more authors
Yingjiu Li XiaoFeng Wang
This is my paper
classification cs.CR
keywords attackinvestigationlargelanguagemodelsretrieval-augmentedgenerationCyberKillChainprovenancegraphthreatknowledgebaseAPTloganalysis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that an LLM, augmented with a knowledge base organized by the Cyber Kill Chain, can reconstruct complete multi-phase attack chains from millions of raw system-log events and turn them into analyst-ready reports. On 15 attack scenarios spanning Windows and Linux (4.3 million log events, 7.2 GB), the reported average true positive rate is 97.1% with a 0.2% false positive rate, against 79.2% and 29.1% for the prior sequence-learning baseline. The same setup is said to work on Linux logs and on unseen shellcode-injection attacks without retraining or platform-specific feature engineering. If these numbers hold, security operations teams could start from a single alert and receive a readable, phase-labelled attack narrative in about two hours instead of weeks of manual correlation. The practical stakes: faster incident response and defense planning against advanced persistent threats.

What carries the argument

The load-bearing object is the Kill Chain–aligned threat knowledge base: a store of small, phase-labelled units, each holding an annotated event sequence, a natural-language behavior summary, key malicious entities, and predicted links to adjacent phases. It is built automatically from known attack logs and queried by cosine similarity, so the LLM never sees the whole corpus at once. Its companion mechanism is adjacency-prioritized expansion: investigation grows one hop at a time from suspicious nodes, keeping every prompt within the model's context window, while a reasoning cache carries the attack story across iterations. The Cyber Kill Chain—a seven-phase model of intrusion progression fr

What would settle it

A concrete check: build the knowledge base from the four Windows scenarios plus the Linux datasets only, then investigate EB-P1 with no EB-related units in the base and record TPR and FPR. If balanced accuracy stays above 90% with near-zero false positives, the generalization claim holds; if it falls toward the baseline's ~60%, the reported edge depends on retrieving the target scenario's own annotations.

Watch

Extended reading notes

Core claim

ANANKE turns attack investigation into a retrieve–reason–expand loop rather than a one-shot detection problem. Known attack logs labelled with malicious entities are reduced to causally relevant events, segmented by an LLM into Cyber Kill Chain phases, and stored as embedded knowledge units. At investigation time, an alert node seeds a provenance graph; the system repeatedly converts one-hop adjacency subgraphs into time-ordered sequences, retrieves the most similar knowledge unit, and lets the LLM mark malicious nodes while updating a reasoning cache. Newly suspicious nodes drive the next expansion, and the cache is consolidated into a chronological report. Reported results: 97.1% average T

Load-bearing premise

The load-bearing premise is that a knowledge base built from known, labelled attack logs can guide investigation of attacks not materially present in it; the paper concedes (Section 6.1) that tactics wholly absent from the base can fail, and the headline numbers assume the measured scenarios were not themselves the source of the knowledge units that retrieval returns.

Editorial extensions

If this is right

  • Investigators can begin with one alert and receive a chronological, phase-labelled story of the intrusion rather than raw event snippets.
  • One knowledge base serves both Windows and Linux logs, removing the need for platform-specific vocabularies and retraining.
  • Novel attack techniques, such as shellcode injection into a benign process, are handled without updating the knowledge base, at balanced accuracy above 90% in the reported cases.
  • At a 0.2% average false positive rate, analysts are not overwhelmed by benign events, making full-chain reconstruction practical on multi-million-event corpora.
  • Adding new intelligence to the knowledge base is an append operation, so adaptation to evolving threats avoids retraining and catastrophic forgetting.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because knowledge-base construction uses ground-truth malicious entities from known scenarios, the strongest test of the generalization claim would be a clean holdout: build the base only from scenarios disjoint from the investigated one (no shared entities, logs, or phase annotations) and measure the drop, if any, in TPR/FPR.
  • The reported 1.9-hour average runtime and 247.6k average token cost suggest the method is an offline forensic tool; a natural extension would measure whether the same loop can be steered toward live triage by bounding the per-alert budget.
  • The same retrieve-reason-expand design could carry a finer attack taxonomy, which the paper itself names as future work; the open question is whether more granular knowledge units improve report precision faster than they inflate the knowledge base.
  • If organizations rely on shared public threat logs to populate the base, the practical ceiling is supply-chain quality: noisy or mislabeled knowledge units could propagate errors into the reasoning step even when retrieval remains fast.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper proposes ANANKE, an LLM-based attack investigation framework combining provenance-graph exploration with retrieval-augmented generation. Stage 1 builds a Cyber Kill Chain-aligned knowledge base from known attack scenarios: given a scenario (L, Emal), log entries are filtered by relevance to the ground-truth malicious-entity set (Eq. 2), segmented and phase-annotated by an LLM (Eq. 3), and embedded as subsequence units with metadata (Eq. 4). Stage 2 starts from an alert node, expands the provenance graph via adjacency-prioritized exploration, converts local subgraphs into bounded event sequences, retrieves the most similar knowledge unit (Eqs. 9-10), and performs iterative LLM causal reasoning with a cache (Eqs. 11-12), culminating in a generated forensic report (Eq. 13). Evaluation spans the ATLAS Windows dataset (S1-S4, M1-M6), two EternalBlue/Gamaredon cases (EB-P1/P2), and three Linux DepImpact datasets, with claims of 97.1% TPR and 0.2% FPR on average, outperforming ATLAS (79.2%/29.1%). The paper reports ablations, cost metrics, and an open-source release.

Significance. If the reported results were valid, ANANKE would be a significant contribution to provenance-based attack investigation: a training-free, cross-platform, RAG-based framework that reconstructs multi-phase attack chains and generates analyst-ready reports, with released artifacts (code, datasets, demo). The paper's strengths include a well-motivated pipeline, a sensible ablation progression, and a genuinely held-out test setting for EB-P1/P2 if the knowledge base indeed contains only S1-S4 material; those results (90.3%/94.8% balanced accuracy vs. ATLAS 59.8%/56.7%) are encouraging. The Linux and alert-type generalization results are likewise interesting, subject to the knowledge-base provenance question. However, the headline S/M results, and thus the abstract's average 97.1%/0.2%, are not credible as currently presented because the evaluation scenarios also populate the knowledge base. The contribution's significance is therefore conditional on a non-leaky re-evaluation.

major comments (3)
  1. [§4.1 Eqs. (2)-(3) with §5.2/Table 4] The central S1-S4/M1-M6 evaluation is circular. The knowledge base R is built from each scenario's log set L and ground-truth malicious set Emal: Eq. (2) filters T by rel(l,Emal); Eq. (3) annotates phases and 'key malicious entities' using Emal; Eq. (4) stores these annotated subsequences. Eq. (10) then retrieves the unit with maximum cosine similarity to the current suspicious subsequence. For a scenario whose own logs populate R, the top-1 unit is the ground-truth-annotated version of that same subsequence, and Appendix A.2 hands its entity names to the LLM. No split between scenarios populating R and scenarios measured is described, so the abstract's 97.1%/0.2% measures retrieval of the answer key rather than investigation capability. The ablation (Table 6) is consistent with this reading: FPR drops from 36.2% to 0.2% on the same S1-S4 scenarios as knowledge is added.
  2. [§5.2 (EB) and §5.3 (Linux)] The knowledge-base composition per evaluation set must be stated explicitly. For EB-P1/P2 the text says the knowledge base is 'built from the same scenarios [S1-S4],' which, if the EB scenarios were not added, is a valid held-out test; those results (balanced accuracy 90.3%/94.8%) are the most credible evidence in the paper. For the Linux DL/VF/SP datasets, however, no statement is made; since these are public datasets and §4.1 builds R from 'public attack logs,' the same overlap may apply. Also reconcile the abstract's '80.5% TPR and 0.1% FPR on EB-P1' with §5.2's 90.3% balanced accuracy. A per-dataset table listing exactly which scenarios populate R is required.
  3. [§5.1 (Evaluation Metrics)] TPR/FPR are defined over log events, but the system produces sets of suspicious entities M(t) at each iteration (Eq. 11) and a narrative report (Eq. 13). The mapping from entity-level suspicion to event-level TP/FP/TN/FN counts is never specified. Whether all events incident to a flagged node count as TP, or only events in the analyzed subsequence, can change the reported values materially given the 0.001%-16% malicious-event fractions in Table 3. The metric pipeline must be defined precisely for reproducibility and for interpreting any claim about TPR/FPR.
minor comments (7)
  1. [Abstract/§1] 'We organizes attack-relevant behaviors' should read 'We organize attack-relevant behaviors.'
  2. [Table 3] Several Log Size cells appear corrupted (e.g., M1 '711,102', M2 '671,112', M3 '336,138', M4 '533,91', M5 '726,113', M6 '55,142'); the Comp. Proc. column also uses ✗/✓ inconsistently.
  3. [§5.3] 'We evaluate ATLAS using the public AIRTAG implementation' is confusing, since ATLAS and AIRTAG are distinct systems; clarify how the ATLAS Linux numbers were produced.
  4. [Various] The text contains garbled/unreadable character runs at several points (e.g., after §3.2 and near Figures 7-9); these must be repaired in the camera-ready.
  5. [References] References [48] and [49] are the same Poirot paper; deduplicate.
  6. [§4.4/§5.6] Analyst-ready report quality is asserted but only anecdotally demonstrated; no rubric, human study, or automatic metric is provided.
  7. [§5.1/§5.2] Results appear to be single-run; given stochastic LLM APIs, repeated-run variance or statistical testing should be reported.

Circularity Check

1 steps flagged · score 7.0 of 10

Headline S1–S4/M1–M6 accuracy reduces to retrieving ground-truth-annotated knowledge units built from the same scenarios.

  1. self definitional [§4.1 Eqs. (2)–(3), §4.3 Eq. (10), §5.2]
    "To construct the knowledge base, we collect known attack scenarios, each consisting of a complete system log set L and a set of malicious entities E_mal. ... T = {l ∈ L | rel(l, E_mal) = 1}. (2) ... K = {k0,...,kn} = LLM(T, E_mal, P_kill). (3) ... Each subsequence u_j inherits the metadata m_i = ⟨Phase, Behavior, Entities, Neighbors⟩. ... r(t) = argmax_{ri∈R} ThreatScore(q(t), ri). (10)"

    The knowledge base is constructed from each known scenario using that scenario's ground-truth malicious entity set E_mal: Eq. (2) filters the trace with E_mal, Eq. (3) stores E_mal-derived entities and phase labels as metadata, and the resulting subsequences are the retrieval corpus. At investigation time, Eq. (10) retrieves the most similar stored unit for the current suspicious sequence. For the S1–S4/M1–M6 evaluation, the paper uses the same ATLAS scenarios with no stated train/test or KB holdout split. The top retrieved unit can therefore contain the exact malicious entity names and phase annotations for the scenario under investigation, and Eq. (12) feeds that answer-key content to the LLM. The reported 97.1% TPR / 0.2% FPR on these scenarios thus measures nearest-neighbor lookup from

full rationale

The central circularity is concrete and equation-level. The KB stores subsequences of known attack scenarios filtered by their own ground-truth E_mal, with metadata containing those entities and phase labels. Evaluation on the same ATLAS scenarios (S1–S4, M1–M6) without any described holdout means the retriever in Eq. (10) can return the exact annotation for the scenario under test, and the LLM is prompted with that retrieved knowledge. The headline average is therefore partly forced by construction. The EB-P1/P2 experiments are less affected because the text explicitly says the KB is built from S1–S4 and the EB scenarios are novel, though even there the knowledge base is built from the same ATLAS dataset used for ATLAS training, so some similarity exists. The Linux §5.3 results raise the same concern unless the DepImpact scenarios were excluded from KB construction; no such exclusion is stated. The self-citation dimension is not significant here; the problem is not citation but label leakage through the RAG knowledge base. Score 7 reflects that the main headline claim partially reduces to retrieval of ground-truth annotations, while the EB-P1/P2 results and the full pipeline still contain independent engineering and evaluation content.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

No new physical entities, mediators, or conserved quantities are introduced. The only new artifacts are data structures: Kill Chain knowledge units, reasoning cache entries, and suspicious/context queues, which are components of the system rather than invented entities requiring external falsification.

free parameters (3)
  • N (subsequence length) = 20
    Hand-chosen in Sections 4.1 and 4.2 to balance semantic completeness and embedding model capacity; controls the granularity of retrieval and LLM context.
  • top-k retrieval count = 1 (argmax, Eq. 10)
    The retriever always selects the single most similar knowledge unit; the paper does not test whether k > 1 changes results. This is a design choice, not fitted.
  • Knowledge base composition and test overlap = unknown
    The set of attack scenarios used to construct the knowledge base is not specified per experiment, and for the ATLAS benchmark appears to include the test scenarios themselves. This is the most consequential load-bearing 'parameter' of the evaluation.
assumptions (4)
  • domain assumption System logs are an accurate and complete record of system behavior
    Assumed in Section 3.2 threat model; if logs miss or corrupt events, reconstructed attack chains can be wrong regardless of reasoning quality.
  • domain assumption The Cyber Kill Chain phases are a natural and sufficient structure for attack reasoning
    Section 2.2 and 4.1; the paper itself notes in Section 6.3 that this high-level structure may overlook fine-grained techniques, and ATT&CK might be better.
  • domain assumption LLMs can perform reliable causal reasoning over subject-action-object triples when given retrieved context
    The entire pipeline delegates classification and causality to the LLM (Eqs. 3, 12, 13); no verification that LLM outputs are correct beyond aggregate metrics.
  • ad hoc to paper Evaluation scenarios used to build the knowledge base are independent of evaluation scenarios
    Required for the headline S1-S4/M1-M6 results; not stated as a split and contradicted by the construction procedure in Eq. (2). This is the load-bearing assumption that, if false, invalidates the main comparison.

how reviews work

0 comments
Cite this review

Pith. "Pith review of An Automated Attack Investigation Approach Leveraging Threat-Knowledge-Augmented Large Language Models." pith.science (2026). https://pith.science/paper/4GQLPNYR

@misc{pith2026250901271,
  author       = {Pith},
  title        = {Pith review of: An Automated Attack Investigation Approach Leveraging Threat-Knowledge-Augmented Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4GQLPNYR}},
  note         = {Machine review of arXiv:2509.01271}
}
read the original abstract

Advanced Persistent Threats (APTs) are prolonged, stealthy intrusions by skilled adversaries that compromise high-value systems to steal data or disrupt operations. Reconstructing complete attack chains from massive, heterogeneous logs is essential for effective attack investigation, yet existing methods suffer from poor platform generality, limited generalization to evolving tactics, and an inability to produce analyst-ready reports. Large Language Models (LLMs) offer strong semantic understanding and summarization capabilities, but in this domain they struggle to capture the long-range, cross-log dependencies critical for accurate reconstruction. To solve these problems, we present an LLM-empowered attack investigation framework augmented with a dynamically adaptable Kill-Chain-aligned threat knowledge base. We organizes attack-relevant behaviors into stage-aware knowledge units enriched with semantic annotations, enabling the LLM to iteratively retrieve relevant intelligence, perform causal reasoning, and progressively expand the investigation context. This process reconstructs multi-phase attack scenarios and generates coherent, human-readable investigation reports. Evaluated on 15 attack scenarios spanning single-host and multi-host environments across Windows and Linux (over 4.3M log events, 7.2 GB of data), the system achieves an average True Positive Rate (TPR) of 97.1% and an average False Positive Rate (FPR) of 0.2%, significantly outperforming the SOTA method ATLAS, which achieves an average TPR of 79.2% and an average FPR of 29.1%.

Figures

Figures reproduced from arXiv: 2509.01271 by the authors.

Figure 1
Figure 1. Extraction of an attack causal subgraph from the [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. An intrusion leveraging the EternalBlue vulnerabil [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Impact of mixed training data. 3.3 Technical Challenges APT attackers continually evolve their tactics, tools, and pro￾cedures across time and platforms. Many learning-based meth￾ods [18, 25, 27, 63], trained on past attack patterns, require costly retraining or fine-tuning to adapt to new attacks and risk forgetting previous knowledge [39] due to limited generaliza￾tion. For example, an adversary may switch from ex… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Overview of ANANKE. Stage 1 (Knowledge Base Construction) extracts and annotates causal events from known attacks to build a threat knowledge base. Stage 2 (Attack Investigation) explores the provenance graph from alert nodes, applies LLM-guided reasoning with retrieve…
Figure 5
Figure 5. Figure 5: Example of phase-aware annotation. Each sequence [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Illustration of iterative causal reasoning. (a) Identify [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 8
Figure 8. Figure 8: Balanced Accuracy comparison of ATLAS and [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]
Figure 9
Figure 9. Figure 9: Balanced Accuracy comparison of ATLAS and [PITH_FULL_IMAGE:figures/full_fig_p011_9.png]
Figure 10
Figure 10. Figure 10: Provenance summary graph of the EB-P1 attack [PITH_FULL_IMAGE:figures/full_fig_p012_10.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

81 extracted references · 73 canonical work pages

  1. [1]

    https://www.cisa.gov/news-events/ alerts/2020/12/13/active-exploitation- solarwinds-software

    Active Exploitation of SolarWinds Software | CISA — cisa.gov. https://www.cisa.gov/news-events/ alerts/2020/12/13/active-exploitation- solarwinds-software. [Accessed 05-08-2025]

    2020

  2. [2]

    https://anonymous.4open.science/r/Themis- LLM-B4EC

    Anonymous Github — anonymous.4open.science. https://anonymous.4open.science/r/Themis- LLM-B4EC. [Accessed 27-08-2025]

    work page 2025

  3. [3]

    https://docs.redhat.com/en/ documentation/red_hat_enterprise_linux/6/ html/security_guide/chap-system_auditing

    ChapterSystem Auditing | Security Guide | Red Hat Enterprise Linux | 6 | Red Hat Documentation — docs.redhat.com. https://docs.redhat.com/en/ documentation/red_hat_enterprise_linux/6/ html/security_guide/chap-system_auditing. [Accessed 17-07-2025]

    work page 2025

  4. [4]

    https://www

    Cisco Talos Intelligence Group - Comprehensive Threat Intelligence — talosintelligence.com. https://www. talosintelligence.com/. [Accessed 26-08-2025]

    work page 2025

  5. [5]

    https://csr.lanl.gov/data/

    Data Sets - Cyber Security Research — csr.lanl.gov. https://csr.lanl.gov/data/. [Accessed 26-08- 2025]

    work page 2025

  6. [6]

    https://community.broadcom

    Endpoint Protection - Symantec Enterprise — commu- nity.broadcom.com. https://community.broadcom. com/symantecenterprise/viewdocument/ wannacry-ransomware-attacks-show-s? CommunityKey=1ecf5f55-9545-44d6-b0f4- 4e4a7f5f5e68&tab=librarydocuments. [Accessed 18-07-2025]

    work page 2025

  7. [7]

    https://github.com/FiveDirections/OpTC- data

    GitHub - FiveDirections/OpTC-data — github.com. https://github.com/FiveDirections/OpTC- data. [Accessed 26-08-2025]

    work page 2025

  8. [8]

    https://github.com/ shramos/Awesome-Cybersecurity-Datasets

    GitHub - shramos/Awesome-Cybersecurity-Datasets: A curated list of amazingly awesome Cybersecu- rity datasets — github.com. https://github.com/ shramos/Awesome-Cybersecurity-Datasets. [Ac- cessed 26-08-2025]

    work page 2025

Show all 81 references
  1. [9]

    https://milvus.io/zh

    Milvus | High-Performance Vector Database Built for Scale — milvus.io. https://milvus.io/zh. [Ac- cessed 18-08-2025]

    2025

  2. [10]

    https: //sysdig.com/

    Security Tools for Containers, Kubernetes, and Cloud (Vulnerability Management) — sysdig.com. https: //sysdig.com/. [Accessed 17-07-2025]

    2025

  3. [11]

    https:// attack.mitre.org/campaigns/C0024/

    SolarWinds Compromise, Campaign C0024 | MITRE ATT&CK® — attack.mitre.org. https:// attack.mitre.org/campaigns/C0024/. [Accessed 17-07-2025]

    2025

  4. [12]

    https://cymulate.com/ cybersecurity-glossary/tactics-techniques- procedures/

    Tactics, Techniques, and Procedures (TTPs) Ex- plained — cymulate.com. https://cymulate.com/ cybersecurity-glossary/tactics-techniques- procedures/. [Accessed 10-08-2025]

    2025

  5. [13]

    https://www.crowdstrike.com/en-us/ platform/threat-intelligence/

    Threat Intelligence & Hunting | CrowdStrike — crowd- strike.com. https://www.crowdstrike.com/en-us/ platform/threat-intelligence/. [Accessed 26- 08-2025]

    2025

  6. [14]

    https://en.wikipedia.org/ wiki/WannaCry_ransomware_attack

    WannaCry ransomware attack - Wikipedia — en.wikipedia.org. https://en.wikipedia.org/ wiki/WannaCry_ransomware_attack. [Accessed 18-07-2025]

    2025

  7. [15]

    https://www.crowdstrike.com/en-us/ cybersecurity-101/threat-intelligence/

    What is Cyber Threat Intelligence? [Begin- ner’s Guide] | CrowdStrike — crowdstrike.com. https://www.crowdstrike.com/en-us/ cybersecurity-101/threat-intelligence/. [Accessed 17-07-2025]

    2025

  8. [16]

    https://www.upguard.com/ blog/cyber-incident-reporting? [Accessed 11- 08-2025]

    Why is Cyber Incident Reporting Important? | Up- Guard — upguard.com. https://www.upguard.com/ blog/cyber-incident-reporting? [Accessed 11- 08-2025]

    2025

  9. [17]

    https://www.lockheedmartin.com/en-us/ capabilities/cyber/cyber-kill-chain.html,

    Cyber Kill Chain ® — lockheedmartin.com. https://www.lockheedmartin.com/en-us/ capabilities/cyber/cyber-kill-chain.html,

  10. [18]

    Berkay Celik, Xiangyu Zhang, and Dongyan Xu

    Abdulellah Alsaheel, Yuhong Nan, Shiqing Ma, Le Yu, Gregory Walkup, Z. Berkay Celik, Xiangyu Zhang, and Dongyan Xu. ATLAS: A sequence-based learning ap- proach for attack investigation. In30th USENIX Security Symposium (USENIX Security 21) , pages 3005–3022. USENIX Association...

    2021

  11. [19]

    ANY.RUN - Interactive Online Malware Sand- box — any.run

    any.run. ANY.RUN - Interactive Online Malware Sand- box — any.run. https://any.run/. [Accessed 26-08- 2025]

    2025

  12. [20]

    Kairos: Practical intrusion detection and investigation using whole-system provenance

    Zijun Cheng, Qiujian Lv, Jinyuan Liang, Yan Wang, De- gang Sun, Thomas Pasquier, and Xueyuan Han. Kairos: Practical intrusion detection and investigation using whole-system provenance. In 2024 IEEE Symposium on Security and Privacy (SP), pages 3533–3551. IEEE, 2024

    2024

  13. [21]

    How to Use Threat Hunt- ing to Identify and Neutralize Advanced Per- sistent Threats — cloudoptics.ai

    cloudoptaiadmin. How to Use Threat Hunt- ing to Identify and Neutralize Advanced Per- sistent Threats — cloudoptics.ai. https: //cloudoptics.ai/cybersecurity-updates/ how-to-use-threat-hunting-to-identify-and- neutralize-advanced-persistent-threats . [Accessed 17-07-2025]

    2025

  14. [22]

    Cybersecurity study: SolarWinds attack cost affected companies an average of $12 million — techrepublic.com

    Veronica Combs. Cybersecurity study: SolarWinds attack cost affected companies an average of $12 million — techrepublic.com. https://www.techrepublic. com/article/cybersecurity-study-solarwinds- attack-cost-affected-companies-an-average- of-12-million. [Accessed 17-07-2025]

    2025

  15. [23]

    AIRTAG: Towards automated attack investigation by unsupervised learning with log texts

    Hailun Ding, Juan Zhai, Yuhong Nan, and Shiqing Ma. AIRTAG: Towards automated attack investigation by unsupervised learning with log texts. In 32nd USENIX Security Symposium (USENIX Security 23), pages 373– 390, Anaheim, CA, August 2023. USENIX Association

    2023

  16. [24]

    Event Tracing for Windows (ETW) - Windows drivers — learn.microsoft.com

    DOMARS. Event Tracing for Windows (ETW) - Windows drivers — learn.microsoft.com. https://learn.microsoft.com/en-us/windows- hardware/drivers/devtest/event-tracing-for- windows--etw- . [Accessed 17-07-2025]

    2025

  17. [25]

    {DISTDET}: A {Cost-Effective} distributed cyber threat detection system

    Feng Dong, Liu Wang, Xu Nie, Fei Shao, Haoyu Wang, Ding Li, Xiapu Luo, and Xusheng Xiao. {DISTDET}: A {Cost-Effective} distributed cyber threat detection system. In 32nd USENIX Security Symposium (USENIX Security 23), pages 6575–6592, 2023

    2023

  18. [26]

    Deeplog: Anomaly detection and diagnosis from system logs through deep learning

    Min Du, Feifei Li, Guineng Zheng, and Vivek Sriku- mar. Deeplog: Anomaly detection and diagnosis from system logs through deep learning. In Proceedings of the 2017 ACM SIGSAC conference on computer and communications security, pages 1285–1298, 2017

    2017

  19. [27]

    {Back-Propagating} system dependency impact for attack investigation

    Pengcheng Fang, Peng Gao, Changlin Liu, Erman Ay- day, Kangkook Jee, Ting Wang, Yanfang Fanny Ye, Zhuotao Liu, and Xusheng Xiao. {Back-Propagating} system dependency impact for attack investigation. In 31st USENIX security symposium (USENIX Security 22), pages 2461–2478, 2022

    2022

  20. [28]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    2025 arXiv

  21. [29]

    Unicorn: Runtime provenance-based detector for advanced persistent threats

    Xueyuan Han, Thomas Pasquier, Adam Bates, James Mickens, and Margo Seltzer. Unicorn: Runtime provenance-based detector for advanced persistent threats. arXiv preprint arXiv:2001.01525, 2020

    2001 arXiv

  22. [30]

    Tac- tical provenance analysis for endpoint detection and response systems

    Wajih Ul Hassan, Adam Bates, and Daniel Marino. Tac- tical provenance analysis for endpoint detection and response systems. In 2020 IEEE symposium on security and privacy (SP), pages 1172–1189. IEEE, 2020

    2020

  23. [31]

    Nodoze: Combatting threat alert fatigue with automated provenance triage

    Wajih Ul Hassan, Shengjian Guo, Ding Li, Zhengzhang Chen, Kangkook Jee, Zhichun Li, and Adam Bates. Nodoze: Combatting threat alert fatigue with automated provenance triage. In network and distributed systems security symposium, 2019

    2019

  24. [32]

    In 26th USENIX Security Symposium (USENIX Security 17), pages 487–504, 2017

    Md Nahid Hossain, Sadegh M Milajerdi, Junao Wang, Birhanu Eshete, Rigel Gjomemo, R Sekar, Scott Stoller, and VN Venkatakrishnan.{SLEUTH}: Real-time attack scenario reconstruction from {COTS} audit data. In 26th USENIX Security Symposium (USENIX Security 17), pages 487–504, 2017

    2017

  25. [33]

    Sellafield apologises after guilty plea over string of cybersecurity failings — theguardian.com

    https://www.theguardian.com/profile/alex-lawson https://www.theguardian.com/profile/anna isaac. Sellafield apologises after guilty plea over string of cybersecurity failings — theguardian.com. https://www.theguardian.com/business/ article/2024/aug/08/sellafield-apologises- gui...

    2024

  26. [34]

    Rain: Refinable attack investigation with on- demand inter-process information flow tracking

    Yang Ji, Sangho Lee, Evan Downing, Weiren Wang, Mat- tia Fazzini, Taesoo Kim, Alessandro Orso, and Wenke Lee. Rain: Refinable attack investigation with on- demand inter-process information flow tracking. In Proceedings of the 2017 ACM SIGSAC conference on computer and communic...

    2017

  27. [35]

    {MAGIC}: Detecting advanced per- sistent threats via masked graph representation learning

    Zian Jia, Yun Xiong, Yuhong Nan, Yao Zhang, Jinjing Zhao, and Mi Wen. {MAGIC}: Detecting advanced per- sistent threats via masked graph representation learning. In 33rd USENIX Security Symposium (USENIX Security 24), pages 5197–5214, 2024. 16

    2024

  28. [36]

    Orthrus: Achieving high quality of attribution in provenance-based intrusion de- tection systems

    Baoxiang Jiang, Tristan Bilot, Nour El Madhoun, Khal- doun Al Agha, Anis Zouaoui, Shahrear Iqbal, Xueyuan Han, and Thomas Pasquier. Orthrus: Achieving high quality of attribution in provenance-based intrusion de- tection systems. In Security Symposium (USENIX Sec’25). USENIX, 2025

    2025

  29. [37]

    Temporal decay loss for adaptive log anomaly detection in cloud environments

    Lelisa Adeba Jilcha, Deuk-Hun Kim, and Jin Kwak. Temporal decay loss for adaptive log anomaly detection in cloud environments. Sensors, 25(9):2649, 2025

    2025

  30. [38]

    Prov-gem: Au- tomated provenance analysis framework using graph embeddings

    Maya Kapoor, Joshua Melton, Michael Ridenhour, Sid- dharth Krishnan, and Thomas Moyer. Prov-gem: Au- tomated provenance analysis framework using graph embeddings. In 2021 20th IEEE International Confer- ence on Machine Learning and Applications (ICMLA), pages 1720–1727. IEEE, 2021

    2021

  31. [39]

    Measuring catastrophic forgetting in neural networks

    Ronald Kemker, Marc McClure, Angelina Abitino, Tyler Hayes, and Christopher Kanan. Measuring catastrophic forgetting in neural networks. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018

    2018

  32. [40]

    High accuracy attack provenance via binary-based exe- cution partition

    Kyu Hyung Lee, Xiangyu Zhang, and Dongyan Xu. High accuracy attack provenance via binary-based exe- cution partition. In NDSS, volume 16, 2013

    2013

  33. [41]

    LevelBlue Labs Open Threat Exchange — levelblue.com

    LevelBlue. LevelBlue Labs Open Threat Exchange — levelblue.com. https://levelblue.com/open- threat-exchange. [Accessed 26-08-2025]

    2025

  34. [42]

    Retrieval-augmented generation for knowledge- intensive nlp tasks

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. Retrieval-augmented generation for knowledge- intensive nlp tasks. Advances in neural information processing system...

    2020

  35. [43]

    A hierarchical approach for advanced persis- tent threat detection with attention-based graph neu- ral networks

    Zitong Li, Xiang Cheng, Lixiao Sun, Ji Zhang, and Bing Chen. A hierarchical approach for advanced persis- tent threat detection with attention-based graph neu- ral networks. Security and Communication Networks, 2021(1):9961342, 2021

    2021

  36. [44]

    Log2vec: A heteroge- neous graph embedding based approach for detecting cyber threats within enterprise

    Fucheng Liu, Yu Wen, Dongxue Zhang, Xihe Jiang, Xinyu Xing, and Dan Meng. Log2vec: A heteroge- neous graph embedding based approach for detecting cyber threats within enterprise. In Proceedings of the 2019 ACM SIGSAC conference on computer and com- munications security, pages ...

    2019

  37. [45]

    APT28 Cyber Espionage Campaign Targets Logistics and Tech Compa- nies, CISA Warns — reveal.security

    Christy Lynch. APT28 Cyber Espionage Campaign Targets Logistics and Tech Compa- nies, CISA Warns — reveal.security. https: //www.reveal.security/blog/apt28-cyber- espionage-campaign-targets-logistics-and- tech-companies-cisa-warns . [Accessed 17-07- 2025]

    2025

  38. [46]

    Pro- tracer: Towards practical provenance tracing by alter- nating between logging and tainting

    Shiqing Ma, Xiangyu Zhang, and Dongyan Xu. Pro- tracer: Towards practical provenance tracing by alter- nating between logging and tainting. In 23rd Annual Network and Distributed System Security Symposium, NDSS 2016, San Diego, California, USA, February 21- 24, 2016. The Inter...

    2016

  39. [47]

    Sysmon - Sysinternals — learn.microsoft.com

    markruss. Sysmon - Sysinternals — learn.microsoft.com. https://learn.microsoft. com/en-us/sysinternals/downloads/sysmon. [Accessed 17-07-2025]

    2025

  40. [49]

    Poirot: Aligning attack be- havior with kernel audit records for cyber threat hunting

    Sadegh M Milajerdi, Birhanu Eshete, Rigel Gjomemo, and VN Venkatakrishnan. Poirot: Aligning attack be- havior with kernel audit records for cyber threat hunting. In Proceedings of the 2019 ACM SIGSAC conference on computer and communications security, pages 1795– 1812, 2019

    2019

  41. [50]

    Holmes: real-time apt detection through correlation of suspicious information flows

    Sadegh M Milajerdi, Rigel Gjomemo, Birhanu Es- hete, Ramachandran Sekar, and VN Venkatakrishnan. Holmes: real-time apt detection through correlation of suspicious information flows. In 2019 IEEE symposium on security and privacy (SP), pages 1137–1152. IEEE, 2019

    2019

  42. [51]

    Mitre att&ck, 2020

    MITRE. Mitre att&ck, 2020. Accessed: 2025-06-30

    2020

  43. [52]

    Custos: Practical tamper-evident auditing of operating systems using trusted execution

    Riccardo Paccagnella, Pubali Datta, Wajih Ul Hassan, Adam Bates, Christopher Fletcher, Andrew Miller, and Dave Tian. Custos: Practical tamper-evident auditing of operating systems using trusted execution. In Network and distributed system security symposium, 2020

    2020

  44. [53]

    Logging to the danger zone: Race condition attacks and defenses on system audit frameworks

    Riccardo Paccagnella, Kevin Liao, Dave Tian, and Adam Bates. Logging to the danger zone: Race condition attacks and defenses on system audit frameworks. In Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security, pages 1551– 1574, 2020

    2020

  45. [54]

    Hercule: Attack story re- construction via community discovery on correlated log graph

    Kexin Pei, Zhongshu Gu, Brendan Saltaformaggio, Shiqing Ma, Fei Wang, Zhiwei Zhang, Luo Si, Xiangyu Zhang, and Dongyan Xu. Hercule: Attack story re- construction via community discovery on correlated log graph. In Proceedings of the 32Nd Annual Con- ference on Computer Securit...

    2016

  46. [55]

    No-doubt: Attack attribution based on threat intelligence reports

    Lior Perry, Bracha Shapira, and Rami Puzis. No-doubt: Attack attribution based on threat intelligence reports. In 2019 IEEE International Conference on Intelligence and Security Informatics (ISI), pages 80–85. IEEE, 2019

    2019

  47. [56]

    {ATTACK2VEC}: Leveraging temporal word embeddings to understand the evolution of cyberattacks

    Yun Shen and Gianluca Stringhini. {ATTACK2VEC}: Leveraging temporal word embeddings to understand the evolution of cyberattacks. In 28th USENIX Secu- rity Symposium (USENIX Security 19), pages 905–921, 2019

    2019

  48. [57]

    APT and financial attacks on industrial organizations in Q4 2024 | Kaspersky ICS CERT — ics-cert.kaspersky.com

    tsvetkovvladimir. APT and financial attacks on industrial organizations in Q4 2024 | Kaspersky ICS CERT — ics-cert.kaspersky.com. https://ics- cert.kaspersky.com/publications/reports/ 2025/03/25/apt-and-financial-attacks-on- industrial-organizations-in-q4-2024/ . [Ac- cessed 1...

    2024

  49. [58]

    Sur- vey on factuality in large language models: Knowl- edge, retrieval and domain-specificity

    Cunxiang Wang, Xiaoze Liu, Yuanhao Yue, Xiangru Tang, Tianhang Zhang, Cheng Jiayang, Yunzhi Yao, Wenyang Gao, Xuming Hu, Zehan Qi, et al. Sur- vey on factuality in large language models: Knowl- edge, retrieval and domain-specificity. arXiv preprint arXiv:2310.07521, 2023

    2023 arXiv

  50. [59]

    Threatrace: Detecting and tracing host-based threats in node level through provenance graph learn- ing

    Su Wang, Zhiliang Wang, Tao Zhou, Hongbin Sun, Xia Yin, Dongqi Han, Han Zhang, Xingang Shi, and Jiahai Yang. Threatrace: Detecting and tracing host-based threats in node level through provenance graph learn- ing. IEEE Transactions on Information Forensics and Security, 17:3972...

    2022

  51. [60]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information pro- cessing systems, 35:24824–24837, 2022

    2022

  52. [61]

    On the ef- fectiveness of log representation for log-based anomaly detection

    Xingfang Wu, Heng Li, and Foutse Khomh. On the ef- fectiveness of log representation for log-based anomaly detection. Empirical Software Engineering, 28(6):137, 2023

    2023

  53. [62]

    Depcomm: Graph sum- marization on system audit logs for attack investigation

    Zhiqiang Xu, Pengcheng Fang, Changlin Liu, Xusheng Xiao, Yu Wen, and Dan Meng. Depcomm: Graph sum- marization on system audit logs for attack investigation. In 2022 IEEE symposium on security and privacy (SP), pages 540–557. IEEE, 2022

    2022

  54. [63]

    {PROGRAPHER}: An anomaly detection system based on provenance graph embedding

    Fan Yang, Jiacen Xu, Chunlin Xiong, Zhou Li, and Ke- huan Zhang. {PROGRAPHER}: An anomaly detection system based on provenance graph embedding. In 32nd USENIX Security Symposium (USENIX Security 23) , pages 4355–4372, 2023

    2023

  55. [64]

    Shadewatcher: Recommendation-guided cyber threat analysis using system audit records

    Jun Zengy, Xiang Wang, Jiahao Liu, Yinfang Chen, Zhenkai Liang, Tat-Seng Chua, and Zheng Leong Chua. Shadewatcher: Recommendation-guided cyber threat analysis using system audit records. In 2022 IEEE Sym- posium on Security and Privacy (SP), pages 489–506. IEEE, 2022

    2022

  56. [65]

    A survey of large lan- guage models

    Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xi- aolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. A survey of large lan- guage models. arXiv preprint arXiv:2303.18223, 1(2), 2023

    2023 arXiv

  57. [66]

    Attack pattern discovery in forensic inves- tigation of network attacks

    Ying Zhu. Attack pattern discovery in forensic inves- tigation of network attacks. IEEE journal on selected areas in communications, 29(7):1349–1357, 2011. Appendix A Prompt Templates A.1 Prompt for Kill-Chain Annotation Prompt for Kill-Chain Annotation You are an expert in Ki...

    2011

  58. [68]

    Detect malicious file deployment patterns

  59. [69]

    Infer causality between the current and previous window based on behavioral continuity

  60. [70]

    Analyze invocation relationships between entities

  61. [71]

    Explicitly include the names of the relevant entities

  62. [72]

    Strictly preserve the original temporal order

    Processing Rules: 1. Strictly preserve the original temporal order

  63. [73]

    Merge operations with the same attack intent into a single phase

  64. [74]

    Separate different attack vectors into independent phases

  65. [75]

    18 5.evidence_set must fully retain the original log en- tries

    Merge multiple actions triggered by the same entity. 18 5.evidence_set must fully retain the original log en- tries

  66. [76]

    Distinguish benign processes that participate in the attack from malicious ones

  67. [77]

    A.2 Prompt for Causal Reasoning Prompt for Causal Reasoning You are a cyber forensic analyst investigating poten- tial attacks in preprocessed system logs

    Output Format: Return the result as a JSON array... A.2 Prompt for Causal Reasoning Prompt for Causal Reasoning You are a cyber forensic analyst investigating poten- tial attacks in preprocessed system logs... Given a sequence of attack-related behavior records... The logs are...

  68. [78]

    Analyze the current log sequence in temporal order

  69. [79]

    3.Infer causality between the current and previous window based on behavioral continuity

    Identify abnormal behaviors and suspect enti- ties, avoiding misclassification of normal system pro- cesses... 3.Infer causality between the current and previous window based on behavioral continuity... 4... Answer: Output Format: Return the result as a JSON array... A.3 Promp...

  70. [80]

    3.Summarize abnormal behaviors and their causal relationships, highlighting how each suspicious entity contributed to the attack progression

    Organize the scenario into Kill Chain phases (e.g., Initial Access, Persistence, Lateral Movement, Exfil- tration). 3.Summarize abnormal behaviors and their causal relationships, highlighting how each suspicious entity contributed to the attack progression

  71. [81]

    Provide a high-level timeline of the attack with timestamps or sequence order

  72. [82]

    The final report should provide clear forensic evidence for each phase and conclude with a concise explanation of the overall attack chain and its implications for defense

    Answer: Based on this information, generate a struc- tured forensic report that describes the chronological attack timeline, explains each stage in terms of the Kill Chain phases, highlights abnormal behaviors and their causal relationships, and summarizes the roles of suspici...

  73. [2025]

    [Accessed 06-08-2025]. 15

    2025

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.