Pith. sign in

REVIEW 3 major objections 5 minor 39 references

GARAGE: Characterizing the Automation Boundary in LLM-based Attack Graph Generation

T0 review · 3 major / 5 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read GARAGE claims that LLMs can reconstruct high-level attack paths for vehicle architectures never seen in their knowledge base, while low-level implementation details still require human experts.

desk verdict A well-supported automation-boundary result buried under a headline claim that the LOO protocol doesn't actually support. read the letter →

arxiv 2607.18108 v1 pith:K33DDMWO submitted 2026-07-20 cs.CR

Daekwon Pi , Sangho Lee , Young Hun Lee , Huy Kang Kim This is my paper · ORCID
classification cs.CR
keywords attackgraphgenerationlargelanguagemodelsretrieval-augmentedautomotivesecuritythreatanalysisandriskassessmentleave-one-outevaluationknowledgegranularityhuman-in-the-loop
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces GARAGE, a retrieval-augmented pipeline that turns public automotive vulnerability and incident data into a structured knowledge base, then prompts large language models to generate multi-step attack graphs for a target car. Its central claim is an automation boundary: LLMs can produce high-level tactical threat scenarios for vehicle architectures that were deliberately excluded from the knowledge base, but low-level implementation details—specific firmware, filesystem paths, and protocols unique to one vehicle—require human experts. The claim rests on 320 leave-one-out experiments across four well-documented real-world hacks and eight models, with proprietary models reconstructing valid attack paths at a mean practical-feasibility score near 59 out of 100. A sympathetic reader would care because, if correct, automotive threat analysis and risk assessment (TARA) can be front-loaded with machine-generated candidate paths while keeping expert review for feasibility.

What carries the argument

The load-bearing mechanism is the leave-one-out-plus-anonymization protocol: for each of four real-world attacks, all documents about the target are removed from both the graph and vector stores, and target-specific keywords are filtered from queries and retrieved results, so any successful reconstruction must come from cross-vehicle generalization or residual parametric memory. Around that protocol sits the hybrid RAG engine—structured graph traversal for component-to-vulnerability connectivity plus vector retrieval for semantic evidence—and a dual-metric judge that separates path viability from knowledge depth. This design is what lets the paper attribute performance differences to knowled

What would settle it

A concrete falsifier: repeat the exact leave-one-out protocol on a fifth attack that postdates the training corpora of all eight models (or on a synthetic vehicle with brand-new components and a synthetic exploit with no public footprint). If practical-feasibility scores collapse to near zero, the reported cross-architecture transfer was mostly parametric recall, not architectural reasoning; if high scores persist, the automation-boundary claim is robust.

Watch

Extended reading notes

Core claim

GARAGE is an end-to-end framework that converts 12,786 CVEs and 140 incident reports into a structured automotive-security knowledge base—a graph of roughly 8,800 entities and 9,200 relations alongside a vector store of rewritten evidence—and then uses hybrid retrieval, combining graph traversal with semantic search, to condition a large language model to write multi-step attack graphs for a given vehicle specification. The core empirical claim is an automation boundary: with the target incident removed from both stores and target-specific names filtered from queries, proprietary models reconstructed high-level tactical attack paths (mean practical-feasibility score near 59/100), while every

Load-bearing premise

The load-bearing premise is that removing the target incident from the knowledge stores and filtering target-specific keywords truly makes the attack 'unseen' to the LLM—since the four test hacks are among the most widely documented automotive exploits, their details almost certainly persist in the models' pretrained weights, so the measured 'transfer' may be partly remembered rather than inferred.

Editorial extensions

If this is right

  • TARA workflows can open with machine-generated tactical attack candidates, and the paper shows high coverage specifically at the entry-point and safety-impact stages, where analysts may otherwise miss scenarios.
  • Implementation-heavy stages—execution, command-and-control, persistence—remain human tasks, so the realistic deployment is a human-in-the-loop assistant rather than a fully autonomous analyzer.
  • When public knowledge about an architecture is rich, even small open-weight models approach proprietary performance; when coverage is sparse, model choice becomes decisive.
  • The knowledge base contributes less to raw accuracy than to evidence traceability: a no-retrieval baseline scores comparably, so GARAGE's case for deployment rests on auditability for regulatory compliance.
  • Cost-per-analysis results give concrete tier guidance: cheap models for large-scale enumeration, premium models for accuracy-critical compliance work.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the tactical/implementation boundary is a general property of security LLMs, the same split should appear in other regulated domains (medical devices, industrial control systems); a cross-domain replication of the leave-one-out protocol would test that.
  • Because ablation shows retrieval modality barely moves scores, a leaner alternative could skip the curated graph and simply require the LLM to cite its sources, then measure whether regulators accept citation-level grounding as compliance evidence.
  • Quantifying per-step grounding—which the paper leaves as future work—would create a practical trust cutoff: high-grounding steps skip review, low-grounding steps go to human experts.
  • The strong cost-performance of small open-weight models on tactical enumeration suggests air-gapped automotive teams could run initial threat scans locally and reserve premium APIs for high-stakes confirmations.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper presents GARAGE, a hybrid RAG pipeline that converts 12,786 automotive CVEs and 140 incident reports into a STIX/Auto-ISAC-aligned knowledge base, then generates vehicle-level attack graphs for user-supplied vehicle architectures. The main empirical claim is that, under Leave-One-Out (LOO) evaluation with entity anonymization, GARAGE accurately transfers security knowledge to entirely unseen vehicle architectures; the more nuanced contribution is an automation boundary: tactical-level threat scenarios can be generated automatically, while implementation-detail steps require human experts. The evaluation uses a dual PF/KR metric judged by Claude 4.5 Sonnet, 8 target LLMs, 4 attack cases, and 10 iterations per condition. The paper also provides a cost-performance analysis and an ablation isolating retrieval modality.

Significance. If the 'unseen architecture' claim were cleanly established, this would be a useful contribution to automotive TARA: a public domain KB, a dual-metric evaluation scheme, and fine-grained step-level evidence for where LLM automation stops. The paper is unusually candid in its limitations, and several empirical patterns (e.g., Jeep persistence 0/10, Execution tactic 15% coverage, KB Match Rate ranking cases) are internally consistent and support the automation-boundary claim regardless of the transfer claim. However, the headline transfer claim is currently not supported because the LOO protocol leaves a concrete retrieval channel open, and the authors' own ablation shows that parametric knowledge alone matches the RAG pipeline. The significance of the work thus depends on whether the contamination leak can be ruled out and the claims accordingly recalibrated.

major comments (3)
  1. [§3.1, §2.2.1, App. A.1] The LOO protocol is incomplete for the 'entirely unseen architecture' claim. §3.1 removes the target incident report and filters queries/retrieved results for make/model keywords, but §2.2.1 populates the KB with 12,786 CVE descriptions. The four benchmark attacks rely on publicly documented CVEs; e.g., Table 3 lists CVE-2017-7932 for Nissan. Nothing removes these CVE records from the vector store or KG, and keyword filtering cannot exclude a CVE ID or a CVE description that omits the vehicle name. This is a second, unremoved channel through which exact target-attack knowledge enters retrieval. The ablation in Table 11 (NO_RAG PF 47.8 vs Hybrid 46.6) makes this concrete: parametric memory alone reproduces the measured performance, so the observed scores cannot be attributed to KB-grounded cross-vehicle transfer. The authors should either (a) re-run the experiments with all target-case CV
  2. [§3.3.2, L2, Table 10] The LLM-as-a-judge setup has a residual circularity that is only partially mitigated. Claude 4.5 Sonnet is both a top-performing target model and the primary judge; the cross-judge validation in Table 10 covers only two of four cases (C1, C4; N=160), not the full 320-instance evaluation. The four Extraction & Inference models are also evaluated on KB instances they themselves constructed, so KB quality is not independent of the model being scored. The paper should provide cross-judge agreement for all four cases or a human-annotated subset, and explicitly state whether the self-built-KB design is intentional and what bias it could introduce.
  3. [§5.1, L3] The paper repeatedly claims that the KB-grounded pathway provides formal evidence traceability, but this is unquantified and partly contradicted by the authors' own data. §5.1 says 'Quantifying the grounding rate of each generated step remains future work,' and L3 reports 76.9% of attack graphs contained at least one hallucinated element, with a mean Knowledge Precision of 9.81/20 — approximately half of the generated knowledge elements lack direct traceability to retrieved context. Until a per-step grounding rate is reported and hallucinated elements are excluded, the manuscript should not describe the output as 'traceable evidence.' This does not undermine the automation-boundary finding, but it weakens the regulatory-compliance framing.
minor comments (5)
  1. [§4.1.2, Fig. 3] The text states Affect Vehicle Function coverage is 98.8%, while Figure 3's y-axis label shows 0.99 (99%). Reconcile the two values.
  2. [Table 4] The header 'V ALID Rate' has an extra space; also unify whether the table reports per-step VALID rates as percentages or decimals.
  3. [Fig. 6] The caption does not say which iteration or model produced the bottom graph, nor whether it is the best-scoring example. For consistency with Figures 7–10, state this explicitly.
  4. [§2.3, §2.3.6] Section 2.3 says all KB construction stages are performed by the same four LLMs, but §2.3.6 uses a separately fine-tuned GPT-4.1 model for ATM technique classification. Clarify that the technique classifier is an additional component, not one of the four extraction models.
  5. [§3.2.1] The sentence 'Since each model produces a unique knowledge representation, we created 16 hybrid KB instances (4 Extraction Models × 4 scenarios)' is confusing: scenario-specific KBs are not described earlier. Explain how the 16 instances relate to the general KB construction in §2.2–2.3.

Circularity Check

1 steps flagged · score 5.0 of 10

Leave-One-Out 'unseen' claim is partially circular: the target attacks' own CVE entries remain in the KB, so RQ1's transfer prediction can reduce to retrieving the target attack from the retained CVE corpus.

  1. self definitional [Section 3.1 (Ground Truth & Controls); Section 2.2.1; Table 3]
    "we applied a LOO strategy excluding target incident data from the KB ... strictly filtered out retrieved results from both the vector store and KG that contained target-specific keywords (e.g., jeep, uconnect for C1; bmw, connecteddrive for C4) ... General technical terms (e.g., SPI, CAN, Bluetooth) are intentionally retained as domain-common knowledge shared across vehicles. ... Table 3 ... CVE-2017-7932, libevo_stack.so"

    The 'entirely unseen vehicle architecture' condition is defined by removing the target incident document and keyword-filtering queries/results. But the KB is built from 12,786 CVE descriptions (§2.2.1), and the four benchmark attacks are documented via CVEs (Nissan's ground truth in Table 3 includes CVE-2017-7932). The LOO protocol does not remove CVE-derived triples, rewrites, or vector embeddings, and keyword filtering cannot catch a CVE ID or a CVE description that omits the vehicle name. Therefore the target attack's canonical vulnerability knowledge remains in the vector store and KG, so RQ1's 'prediction' for 'entirely unseen vehicle architectures' is not made from unseen inputs: the exact target CVE can be retrieved. The abstract's claim of transfer 'without direct knowledge of the

full rationale

GARAGE's core result is an empirical evaluation, not an equation chain, and I find no self-citation-based circularity: the reference list contains no load-bearing same-author citations, and the adopted components (Hybrid RAG [21], GraphRAG [7], LLM-as-judge [30]) are external. The main circularity-adjacent problem is the LOO control. The paper defines 'unseen' by removing the target incident document and keyword-filtering queries/results (§3.1), but the KB also contains 12,786 CVE descriptions (§2.2.1) and the four benchmark attacks are documented via CVEs (Table 3 lists CVE-2017-7932 for Nissan). Nothing in the LOO protocol removes CVE-derived triples or rewrites; keyword filtering cannot catch a CVE ID or a CVE text that omits the vehicle name. Consequently RQ1's headline—'transfer to entirely unseen vehicle architectures'—can be realized by retrieving the target attack's own CVE entries, making the 'unseen' prediction partially circular by construction. The paper's own ablation (Appendix A.1) shows NO_RAG ≈ Hybrid (PF 47.8 vs 46.6), so parametric recall alone reproduces the result, compounding the issue. The automation-boundary finding (tactical-level transfer works, implementation details require experts) is independent of this leak and rests on per-step analyses (Tables 4-6), so the circularity is partial. The LLM-as-judge overlap (Claude as both judge and target) is acknowledged in L2 with cross-judge correlations (ρ=0.74-0.84), so it is a disclosed risk rather than a load-bearing circular step. Similarly, evaluating extraction models on their own KB instances is a confound but not a definitional reduction. Overall score 5: one central 'prediction' (unseen transfer) is partially reduced to a retained input, while other contributions remain independent.

Assumptions & free parameters 7 free parameters · 6 assumptions · 4 invented entities

GARAGE is a systems/evaluation paper, so its ledger is dominated by evaluation-design choices rather than mathematical axioms. The three most consequential entries: (1) the four ground-truth attack graphs are author-reconstructed and taken as correct (L1 admits subjectivity, mitigated only by consensus, no κ reported); (2) the LOO protocol is assumed to make the target attack unknown to the models, which the paper's own ablation and §5.1 discussion undermine; (3) the LLM judge's rubric scores are assumed to track expert judgment, with cross-judge agreement as the only external check. The hand-chosen PF/KR weights (60/30/−20 and 50/30/20) and the τ=0.3 clustering threshold are free parameters that shape all headline numbers but are not fitted to the target result. No new physical entities are postulated; the invented constructs are the KB architecture, the PF/KR metrics, and the fine-tuned classifier.

free parameters (7)
  • Entity-normalization cosine threshold τ = 0.3
    Empirically tuned in §2.3.2 to balance under- and over-clustering; sets the canonical entity set that all retrieval and scoring rest on.
  • PF weights (completeness/feasibility/violations) = 60 / 30 / −20
    Hand-chosen in §3.3.1; with cascade failure, a broken early step zeroes later steps, so these weights shape the headline PF numbers.
  • KR weights (identification/structure/precision) = 50 / 30 / 20 (precision baseline 10)
    Hand-chosen in §3.3.2; determines how knowledge depth is credited independently of chain continuity.
  • Sampling temperature = 1.0
    Set in §3.2.2 to encourage path exploration; inflates the output variance that RQ2 then reports as consistency (CV 30–64%).
  • Chunk size / overlap = 500 / 150
    Text segmentation in §2.3.1; affects entity extraction and retrieval coverage.
  • KG traversal depth and rerank top-k = 2 hops; top-10
    Constants in §2.4.1–2.4.2 limiting the retrieved context available to inference.
  • Technique-classifier SFT data = LLM-synthesized virtual reports, 5 styles, GPT-4.1 base
    Classifier (§2.3.6) trained on self-generated narratives with blacklisted labels; accuracy never independently measured.
assumptions (6)
  • domain assumption Auto-ISAC ATM taxonomy is a correct and complete labeling of automotive attack techniques.
    ATM is the ground truth for technique classification and GT mapping (§2.3.6, §3.1); any taxonomy error propagates into PF/KR scoring.
  • domain assumption The four reconstructed ground-truth attack graphs are accurate.
    GT was author-reconstructed from white papers and ATM examples (§3.1, L1); consensus among three experts but no inter-rater statistic and no testbed validation.
  • domain assumption The Claude 4.5 judge with structured rubrics scores attack graphs the way human experts would.
    Justified by citation [30] and cross-judge LLM agreement (Table 10, ρ=0.74–0.84); no direct human-agreement study for these automotive rubrics.
  • ad hoc to paper LOO exclusion plus entity anonymization makes the target attack unknown to the model.
    §3.1 removes stored data and filters names, but cannot erase parametric memory of famous hacks; §5.1 itself credits "implicit recall from the LLM's parametric knowledge" as a transfer pathway, so the assumption is internally unsupported.
  • ad hoc to paper The conservative KG-over-vector conflict rule reflects real exploit status.
    §2.4.3: KG-recorded vulnerabilities are assumed present unless patch notes name make/model/year; the rule's false-positive rate is unmeasured.
  • domain assumption Judge scoring is well-defined when generated and GT graphs differ in granularity.
    PF/KR use 5-level semantic matching and accept alternative valid paths (L1); the boundary between "semantic match" and "valid alternative" is judge-interpreted.
invented entities (4)
  • Canonical-node KG (SAME_AS architecture)
    purpose: Consolidates duplicate LLM-extracted entity mentions into canonical representatives for retrieval
    Internal design artifact (§2.3.2); no external validation that canonicalization preserves security semantics.
  • PF/KR dual-metric evaluation scheme
    purpose: Separately scores attack-path viability and security-knowledge depth
    New scoring constructs (§3.3) validated only by internal cross-judge correlation; no calibration against external benchmarks or human scores.
  • Fine-tuned ATM technique classifier (GPT-4.1 SFT)
    purpose: Maps extracted events to ATM technique IDs
    Trained on LLM-synthesized narratives (§2.3.6); no held-out accuracy reported.
  • Automation boundary (knowledge-granularity constraint) independent evidence
    purpose: Claims the abstraction level of threat definition sets an upper bound on automated inference
    Falsifiable handle: high coverage for Initial Access/Affect Vehicle Function, near-zero for Execution/C2/Persistence — testable on new incidents.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GARAGE: Characterizing the Automation Boundary in LLM-based Attack Graph Generation." pith.science (2026). https://pith.science/paper/K33DDMWO

@misc{pith2026260718108,
  author       = {Pith},
  title        = {Pith review of: GARAGE: Characterizing the Automation Boundary in LLM-based Attack Graph Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/K33DDMWO}},
  note         = {Machine review of arXiv:2607.18108}
}
read the original abstract

While modern vehicle security depends on effective Cyber Threat Intelligence (CTI) synthesis, current automated tools struggle with unstructured data and automotive-specific architectural nuances. To bridge this gap, we introduce GARAGE, a RAG-powered framework that converts fragmented CTI into an actionable, domain-specific knowledge base for automated attack graph generation. GARAGE synthesizes a dataset of 12,786 CVEs and 140 incident reports into a STIX 2.1 and Auto-ISAC ATM-compliant knowledge base. By formalizing tactical-pattern-level scenarios through granular kill chain analysis, GARAGE achieves threat generation capabilities. Our 320 Leave-One-Out experiments reveal that the framework can accurately transfer security knowledge to entirely unseen vehicle architectures. Furthermore, we position GARAGE as a scalable TARA support tool within human-in-the-loop workflows, offering a comprehensive cost-performance analysis to guide its deployment across various LLM tiers.

Figures

Figures reproduced from arXiv: 2607.18108 by the authors.

Figure 1
Figure 1. Proposed GARAGE System Architecture and Data Flow. The system integrates graph-based structural retrieval with [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. PF/KR Score Heatmap across 8 models and 4 attack cases. Proprietary models (top three rows) consistently occupy [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Auto-ISAC Kill Chain tactic coverage. Initial Ac [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Distribution of PF scores across 10 independent trials for each model and case. Proprietary models (Claude, Gemini, [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: Cost vs. Performance trade-off across eight [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

39 extracted references · 3 linked inside Pith

  1. [1]

    Auto-isac atm

    Auto-ISAC. Auto-isac atm. https://atm.automoti veisac.com/. Accessed: 25 Feb 2026

    2026

  2. [2]

    Autosec-timeline

    AutoSec. Autosec-timeline. https://autosec-tim eline.delikely.eu.org/. Accessed: 25 Feb 2026

    2026

  3. [3]

    Remote exploitation of nissan leaf

    Black Hat. Remote exploitation of nissan leaf. https: //i.blackhat.com/Asia-25/Asia-25-Evdokim ov-Remote-Exploitation-of-Nissan-Leaf.pdf . Accessed: 25 Feb 2026

    2026

  4. [4]

    Ctinexus: Automatic cyber threat intelligence knowledge graph construction using large language models

    Yutong Cheng, Osama Bajaber, Saimon Amanuel Tsegai, Dawn Song, and Peng Gao. Ctinexus: Automatic cyber threat intelligence knowledge graph construction using large language models. In2025 IEEE 10th Euro- pean Symposium on Security and Privacy (EuroS&P), pages 923–938. IEEE, 2025

    2025

  5. [5]

    Threatget: towards automated at- tack tree analysis for automotive cybersecurity.Infor- mation, 14(1):14, 2022

    Sebastian Chlup, Korbinian Christl, Christoph Schmit- tner, Abdelkader Magdy Shaaban, Stefan Schauer, and Martin Latzenhofer. Threatget: towards automated at- tack tree analysis for automotive cybersecurity.Infor- mation, 14(1):14, 2022

    2022

  6. [6]

    Cti-hal: A human-annotated dataset for cyber threat intelligence analysis

    Sofia Della Penna, Roberto Natella, Vittorio Orbinato, Lorenzo Parracino, and Luciano Pianese. Cti-hal: A human-annotated dataset for cyber threat intelligence analysis. In2025 IEEE European Symposium on Secu- rity and Privacy Workshops (EuroS&PW), pages 69–78. IEEE, 2025

    2025

  7. [7]

    From local to global: A graph rag ap- proach to query-focused summarization.arXiv preprint arXiv:2404.16130, 2024

    Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, Dasha Metropolitansky, Robert Osazuwa Ness, and Jonathan Larson. From local to global: A graph rag ap- proach to query-focused summarization.arXiv preprint arXiv:2404.16130, 2024

    arXiv 2024

  8. [8]

    Llm-tikg: Threat intelligence knowledge graph 13 construction utilizing large language model.Computers & Security, 145:103999, 2024

    Yuelin Hu, Futai Zou, Jiajia Han, Xin Sun, and Yilei Wang. Llm-tikg: Threat intelligence knowledge graph 13 construction utilizing large language model.Computers & Security, 145:103999, 2024

    2024

Show all 39 references
  1. [9]

    Ctikg: Llm-powered knowledge graph construction from cyber threat intel- ligence

    Liangyi Huang and Xusheng Xiao. Ctikg: Llm-powered knowledge graph construction from cyber threat intel- ligence. InFirst Conference on Language Modeling, 2024

    2024

  2. [10]

    Open-cykg: An open cyber threat intelligence knowledge graph.Knowledge- Based Systems, 233(1):1–13, 2021

    Sarhan Injy and Spruit Marco. Open-cykg: An open cyber threat intelligence knowledge graph.Knowledge- Based Systems, 233(1):1–13, 2021

    2021

  3. [11]

    Jeep hacking

    IOActive. Jeep hacking. https://www.ioactive.c om/wp-content/uploads/pdfs/IOActive_Remote_ Car_Hacking.pdf. Accessed: 25 Feb 2026

    2026

  4. [12]

    Experimental security assessment of bmw cars

    KeenLab. Experimental security assessment of bmw cars. https://keenlab.tencent.com/en/whitepa pers/Experimental_Security_Assessment_of_B MW_Cars_by_KeenLab.pdf. Accessed: 25 Feb 2026

    2026

  5. [13]

    Mercedes benz security research report

    KeenLab. Mercedes benz security research report. ht tps://keenlab.tencent.com/en/whitepapers/M ercedes_Benz_Security_Research_Report_Fina l.pdf. Accessed: 25 Feb 2026

    2026

  6. [14]

    Kia open source software notice

    KIA. Kia open source software notice. http://webm anual.kia.com/STD_GEN5_WIDE/AVNT/EU/Italia n/opensourcesoftwarenotice.html. Accessed: 25 Feb 2026

    2026

  7. [15]

    Attackg: Constructing technique knowledge graph from cyber threat intelligence reports

    Zhenyuan Li, Jun Zeng, Yan Chen, and Zhenkai Liang. Attackg: Constructing technique knowledge graph from cyber threat intelligence reports. InEuropean Sympo- sium on Research in Computer Security, pages 589–609. Springer, 2022

    2022

  8. [16]

    Hyundai mobis open sources

    Hyundai MOBIS. Hyundai mobis open sources. https: //www.mobis.com/kr/tech/rnd.do . Accessed: 25 Feb 2026

    2026

  9. [17]

    Openrouter api

    OpenRouter. Openrouter api. https://openrouter .ai/. Accessed: 25 Feb 2026

    2026

  10. [18]

    Mulval: A logic-based network security ana- lyzer

    Xinming Ou, Sudhakar Govindavajhala, Andrew W Ap- pel, et al. Mulval: A logic-based network security ana- lyzer. InUSENIX security symposium, volume 8, pages 113–128. Baltimore, MD, 2005

    2005

  11. [19]

    A survey on security attacks and defense techniques for connected and au- tonomous vehicles.Computers & Security, 109:102269, 2021

    Minh Pham and Kaiqi Xiong. A survey on security attacks and defense techniques for connected and au- tonomous vehicles.Computers & Security, 109:102269, 2021

    2021

  12. [20]

    Springer, 2024

    Martin Salfer.Automotive Security Analyzer for Ex- ploitability Risks. Springer, 2024

    2024

  13. [21]

    Hybridrag: Integrating knowledge graphs and vector retrieval aug- mented generation for efficient information extraction

    Bhaskarjit Sarmah, Dhagash Mehta, Benika Hall, Ro- han Rao, Sunil Patel, and Stefano Pasquali. Hybridrag: Integrating knowledge graphs and vector retrieval aug- mented generation for efficient information extraction. InProceedings of the 5th ACM International Conference on AI ...

    2024

  14. [22]

    Graph-based automation of threat analy- sis and risk assessment for automotive security.Infor- mation, 16(6):449, 2025

    Mera Nizam-Edden Saulaiman, Miklos Kozlovszky, and Akos Csilling. Graph-based automation of threat analy- sis and risk assessment for automotive security.Infor- mation, 16(6):449, 2025

    2025

  15. [23]

    Uco: A unified cybersecurity ontol- ogy

    Zareen Syed, Ankur Padia, M Lisa Mathews, Tim Finin, Anupam Joshi, et al. Uco: A unified cybersecurity ontol- ogy. InProceedings of the AAAI Workshop on Artificial Intelligence for Cyber Security, pages 195–202, 2016

    2016

  16. [24]

    V olkswagen third-party licence notes.ht tps://www.volkswagen-newsroom.com/en/third -party-licence-notes-38

    V olkswagen. V olkswagen third-party licence notes.ht tps://www.volkswagen-newsroom.com/en/third -party-licence-notes-38. Accessed: 25 Feb 2026

    2026

  17. [25]

    Proactive security defense: cyber threat intelligence modeling for connected autonomous vehicles.arXiv preprint arXiv:2410.16016, 2024

    Yinghui Wang, Yilong Ren, Zhiyong Cui, and Haiyang Yu. Proactive security defense: cyber threat intelligence modeling for connected autonomous vehicles.arXiv preprint arXiv:2410.16016, 2024

    2024 arXiv

  18. [26]

    A dataset for cyber threat intelligence modeling of connected autonomous vehicles.Scientific Data, 12(1):366, 2025

    Yinghui Wang, Yilong Ren, Hongmao Qin, Zhiyong Cui, Yanan Zhao, and Haiyang Yu. A dataset for cyber threat intelligence modeling of connected autonomous vehicles.Scientific Data, 12(1):366, 2025

    2025

  19. [27]

    Cakg: A framework for cyber- security threat detection of automotive via knowledge graph

    Peng Yang, Wang Lijie, Li Yun, Song Xuedong, Wang Yaxin, and Guo Biheng. Cakg: A framework for cyber- security threat detection of automotive via knowledge graph. In2023 8th International Conference on Data Science in Cyberspace (DSC), pages 221–228. IEEE, 2023

    2023

  20. [28]

    Attackg+: Boosting attack graph con- struction with large language models.Computers & Security, 150:104220, 2025

    Yongheng Zhang, Tingwen Du, Yunshan Ma, Xiang Wang, Yi Xie, Guozheng Yang, Yuliang Lu, and Ee- Chien Chang. Attackg+: Boosting attack graph con- struction with large language models.Computers & Security, 150:104220, 2025

    2025

  21. [29]

    Mm-attackg: A multimodal approach to attack graph construction with large language models

    Yongheng Zhang, Xinyun Zhao, Yunshan Ma, Haokai Ma, Yingxiao Guan, Guozheng Yang, Yuliang Lu, and Xiang Wang. Mm-attackg: A multimodal approach to attack graph construction with large language models. arXiv preprint arXiv:2506.16968, 2025

    2025 arXiv

  22. [30]

    device",

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuo- han Li, Dacheng Li, Eric Xing, et al. Judging llm-as- a-judge with mt-bench and chatbot arena.Advances in neural information processing systems, 36:46595– 46623, 2023. 14 A App...

    2023

  23. [31]

    Relations

    USED_AGAINST ...[snip per-relation examples]... If no valid relations exist, return "Relations": []. 15 A.2.3 Rewriting Prompt The rewriting prompt converts structured data (entities and relations) into dense, factual natural language paragraphs op- timized for vector retrieva...

  24. [32]

    intercepted

    Action-Oriented: Use strong verbs describing the attack lifecycle (\textit{e.g.}, "intercepted", "spoofed", "flooded", "extracted", "modified")

  25. [33]

    Mechanism & Impact: Explicitly state HOW the vulnerability was exploited and WHAT the consequence was

  26. [34]

    Keyword Preservation: Keep specific entity names (ECUs, Protocols, CVEs) exact

  27. [35]

    Conciseness: Dense, technical summary

  28. [36]

    Contextual Linking: Connect the ’Tool’ used to the ’Vulnerability’ exploited and the ’Asset’ affected

  29. [37]

    style": one of [

    Grounding: Do not invent details not present in the input. ...[snip example and I/O format]... A.2.4 SFT Data Generation Prompt The SFT data generation prompt assigns the LLM the role of a Threat Intelligence Analyst. It instructs the model to synthesize virtual incident repor...

  30. [38]

    Prioritize VectorDB ONLY IF it explicitly mentions the target vehicle’s make, model, and year

  31. [39]

    nodes": [{

    Otherwise, assume the vulnerability exists 16 (Conservative Approach). === OUTPUT SCHEMA === { "nodes": [{"id", "label", "type", "group"}], "edges": [{"source", "target", "step_id", "type", "label", "protocol", "technique_ids": ["ATM-Txxxx"]}] } ...[snip requirements]... A.3 R...

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.