Pith. sign in

REVIEW 5 major objections 4 minor 40 references

LLM cyber-threat analysis is gated by knowledge access, not reasoning: CTIARENA shows closed-book CTI mappings score near zero, jump to near-perfect once authoritative sources are injected, and CTI-tailored retrieval beats generic RAG.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 10:00 UTC pith:G7LUQ3W3

load-bearing objection The body of the paper is a solid, detailed CTI RAG benchmark (CTIARENA, 691 QA pairs), but the arXiv abstract describes a different benchmark (CTIConnect, 1,860 pairs, temporal splits, rerank/IRCoT baselines) that the body never delivers; that mismatch blocks verification of the headline claims. the 5 major comments →

arxiv 2510.11974 v2 pith:G7LUQ3W3 submitted 2025-10-13 cs.CR cs.AI

CTIConnect: A Benchmark for Retrieval-Augmented LLMs over Heterogeneous Cyber Threat Intelligence

Yutong Cheng , Yang Liu , Changze Li , Dawn Song , Peng Gao This is my paper
classification cs.CR cs.AI
keywords cyber threat intelligenceLLM benchmarkretrieval-augmented generationknowledge-augmented LLMCVE-CWE-CAPEC-ATT&CKthreat report synthesisentity-centric retrievalmulti-source reasoning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper builds CTIARENA, which it presents as the first benchmark for evaluating LLMs on heterogeneous, multi-source cyber threat intelligence under knowledge-augmented settings: 691 expert-verified QA pairs spanning nine tasks that map CVE, CWE, CAPEC, and ATT&CK taxonomies, synthesize vendor reports and blogs, and bridge the two. Its central claim is that the cross-source semantic gap manifests differently across task categories, so no single retrieval recipe serves all of CTI. Closed-book models are nearly useless on structured mappings, yet the same models saturate to near-perfect accuracy once authoritative entries are injected, showing structured CTI reasoning is a grounding problem rather than a reasoning problem. Hybrid tasks demand query expansion that decomposes narratives into atomic security behaviors, while unstructured tasks demand entity-centric retrieval over a security knowledge graph instead of embedding similarity; the bottleneck shifts between retrieval infrastructure and evidence utilization depending on the task. If this holds, CTI copilots should be engineered around domain-tailored retrieval layers before model scale.

Core claim

CTIARENA's contribution is a benchmark and a diagnosis. The benchmark folds five CTI sources into 691 expert-verified QA pairs across nine tasks: structured mappings, unstructured report synthesis, and hybrid attribution of free-text behaviors to CWE and ATT&CK entries. The diagnosis: LLMs fail closed-book because cross-source correlations are long-tail enumerated knowledge that pretraining rarely absorbs; once authoritative entries are injected, structured accuracy saturates near 1.0 for all ten models. Query-expanded RAG beats vanilla semantic RAG on hybrid tasks; entity-centric retrieval over a security knowledge graph does the same on unstructured tasks. Residual errors trace to semantic

What carries the argument

Two mechanisms carry the argument. First, the three-stage construction pipeline: seed correlations from authoritative sources (CVE-CWE-CAPEC-ATT&CK mappings, adversary-centered blog clusters, blog-to-framework phrase alignments) become template-constrained QA pairs, then pass through an LLM judge and human expert verification. Second, two CTI-specific retrieval strategies: query-expanded RAG, which decomposes a narrative query into atomic security behaviors and retrieves per behavior before aggregating; and CSKG-guided RAG, which links reports through shared entities in a cyber security knowledge graph, keeping reports with at least 0.6 entity overlap. Against vanilla embedding retrieval, bo

Load-bearing premise

Every reported number inherits the assumption that the 691 gold answers are correct and complete: the seed correlations, blog clusters, and blog-to-framework annotations are taken as ground truth, and the LLM-judge-plus-expert filter is assumed to have removed all hallucinated or mislabeled QA pairs, with no inter-annotator agreement and no dataset release reported for independent audit.

What would settle it

Independently re-annotate a random sample of the 691 QA pairs with fresh experts who never saw the original labels and check agreement: if it is too low to sustain the headline margins (roughly 0.08 to 0.26 points for query-expanded RAG on hybrid tasks and CSKG-guided RAG on unstructured tasks), the rankings could be label noise. A second check: in structured tasks, withhold the gold entry from the injected candidate set; near-perfect accuracy should collapse if grounding, not parametric memory, is what closes the gap.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Structured CTI reasoning should be treated as a retrieval and grounding problem: authoritative entry injection lifts all ten models to near-perfect accuracy, so further gains from larger closed-book models will be marginal.
  • Hybrid retrieval should decompose narrative queries into atomic CTI behaviors before matching taxonomy entries; this consistently beat vanilla semantic RAG, most sharply for the strongest models (GPT-5 ATA 0.742 to 0.900, VCA 0.600 to 0.760).
  • Unstructured tasks need entity-centric retrieval over a security knowledge graph, yet even with the right evidence, cross-report synthesis stays the hard part: actor profiling and malware lineage scores remain low, so evidence utilization, not just retrieval, needs work.
  • Retrieval strategies are not model-agnostic: the query expansion that helps stronger models can destabilize smaller ones (the paper reports Phi-4 losing 12 percent accuracy with query-expanded RAG), so augmentation design must be matched to the model's ability to follow decomposition instructions.
  • CTI copilots should pair LLMs with source-grounded retrieval rather than parametric memorization, and should audit answers against retrieved evidence, since models sometimes produce correct answers while fabricating the supporting justification (3 percent of predictions, up to 7 percent for one model).

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the near-saturation of structured tasks implies the benchmark measures the retriever-plus-corpus at least as much as the model on those tasks; reporting structured accuracy as a function of retrieved-candidate coverage would let teams tune corpora rather than prompts.
  • Editorial inference: the unsupported-correctness finding could be converted into an automatic trust audit, requiring models to cite the exact evidence span behind each answer; the paper motivates this step but does not build it.
  • Editorial inference: the three-category design could be carried to other fragmented knowledge domains where authoritative enumerations and narrative sources coexist, such as medical or legal reference stacks; the claim that the bottleneck shifts by task category is testable there.
  • Editorial note on the manuscript: the front-matter abstract describes the benchmark as CTIConnect with 1,860 QA pairs and reports comparisons against retrieve-then-rerank and IRCoT baselines, while the body describes CTIARENA with 691 QA pairs and only vanilla-RAG baselines; this page follows the body.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The manuscript, as submitted under the CTIConnect title and abstract, actually contains a full text that presents a different benchmark, CTIARENA. The body defines 691 expert-verified QA pairs across nine tasks grouped into structured, unstructured, and hybrid categories, constructed via a three-stage pipeline (seed correlations, template-guided LLM synthesis, LLM-human curation), and evaluates ten LLMs under closed-book, vanilla RAG, CSKG-guided RAG, and query-expanded RAG. The reported findings are that closed-book performance is low on structured mappings, retrieval augmentation largely closes the structured gap, and the two CTI-specific retrieval variants improve hybrid/unstructured tasks relative to vanilla RAG, with some caveats for weaker models. The abstract, however, claims 1,860 QA pairs, temporal splits spanning 2008-2025, and comparisons against retrieve-then-rerank and IRCoT, none of which appear in the body.

Significance. If the CTIARENA results are reproducible, the benchmark is a useful step: no existing CTI benchmark combines nine tasks over structured, unstructured, and hybrid sources in a retrieval-augmented setup; the evaluation spans ten models; and the proposed CSKG-guided and query-expanded RAG are concrete, domain-tailored retrieval strategies. The paper also documents a human-verification pipeline and makes a credible effort to ground QA pairs in authoritative correlations. However, the current version is internally incoherent: the front matter advertises a different benchmark with unverifiable claims, the dataset is not released, inter-annotator agreement is not reported, and the 'consistently outperforms' statements are not supported by the tables. These issues must be resolved before the contribution can be fully assessed.

major comments (5)
  1. [Abstract vs. full text] The arXiv abstract defines CTIConnect with 1,860 expert-verified QA pairs, temporal splits 2008-2025, and comparisons against retrieve-then-rerank and IRCoT. The full text defines CTIARENA with 691 QA pairs and, in §4, evaluates only closed-book, vanilla RAG, CSKG-guided RAG, and query-expanded RAG. No section or table reports 1,860 pairs, temporal splits, or the two generic baselines. This is load-bearing: the paper's stated central claim cannot be checked. Please either substantiate the CTIConnect claims with data and experiments, or retitle/rewrite the front matter to match the CTIARENA content actually presented.
  2. [§4.2, Tables III–IV] The text claims CSKG-guided RAG 'consistently outperforms' vanilla RAG on unstructured tasks and query-expanded RAG 'consistently outperforms' vanilla RAG on hybrid tasks. Tables III and IV contradict this: e.g., GPT-5 CSC vanilla 0.721 > CSKG 0.671; Gemini-2.5-Pro CSC vanilla 0.612 > CSKG 0.609; Claude-3.5-Haiku MLA vanilla 0.443 > CSKG 0.410; Phi-4 VCA vanilla 0.480 > expansion 0.360; Gemini-2.5-Flash VCA vanilla 0.600 > expansion 0.540. Domain-specific gains are present in many cells but are not consistent. Report averages, per-task significance tests, or error bars, and soften the claim accordingly.
  3. [§4.1 vs. Table II] The evaluation protocol says structured and hybrid tasks use exact string matching and report accuracy, but Table II's header labels all numbers as F1-score. Please clarify the metric. Additionally, §4.2(1) says the best closed-book ATD score is 0.99, but Table II shows no closed-book ATD value near 0.99; the largest is about 0.06-0.09. This appears to be a typo that inverts CB and KW values, making the quantitative narrative difficult to interpret.
  4. [§D.3, RAG Expansion prompt; §4.1] The query-expanded RAG evaluation prompt instructs the model to 'answer based on the provided ATT&CK / CWE evidence, or use your internal knowledge if you're confident' and 'You don't have to select from the provided CWEs if you have a better answer.' This allows closed-book parametric knowledge to substitute for retrieval, so improvements attributed to query expansion may instead reflect prior knowledge. The main-text description of query-expanded RAG should be reconciled with this prompt, and the setup should be reported as a fallback/hybrid RAG rather than pure retrieval augmentation.
  5. [§3.2 Stages 1–3; §4.1] The benchmark's validity rests on the seed correlations, BlogCluster/B2F annotations, and the Stage 3 LLM-human curation. No inter-annotator agreement or post-hoc error audit is reported, and the GitHub link has no commit hash or data snapshot. In addition, GPT-5 is used both as an evaluated model and as the automatic judge for unstructured answers (§4.1), and GPT-5 is also used in data generation/filtering (§3.2). This creates a risk of circularity for the open-ended-task results. Please release the benchmark data with versioned code, report annotator agreement and remaining label-error rate, and consider using a judge model that is not among the evaluated systems.
minor comments (4)
  1. [§3.3.2] The dataset sizes for unstructured tasks (60/60/30) are very small, and no confidence intervals are reported for the GPT-5 judge scores. Please state variance or bootstrap intervals.
  2. [§4.1] The retrieval hyperparameters (top-k=5, similarity threshold 0.5, CSKG entity-overlap ≥0.6) are said to be chosen after exploratory trials; a sensitivity analysis would strengthen the benchmark.
  3. [Table II] The unstructured-task columns show only KW values, with no CB columns. Since the text says closed-book is not evaluated for unstructured tasks, this should be stated explicitly in the table caption.
  4. [Throughout] There are several small inconsistencies, e.g., 'compaign' in §3.3.2 and the mixed use of 'F1' vs. 'accuracy' terminology. A careful proofreading pass is needed.

Circularity Check

0 steps flagged

No circular derivation; benchmark findings are empirical, though the abstract and full text describe different artifacts.

full rationale

This paper is an empirical benchmark paper rather than a derivational one. Its central results—that closed-book LLMs perform poorly, that retrieval augmentation closes the structured-task gap, and that CSKG-guided or query-expanded RAG can outperform vanilla RAG on unstructured/hybrid tasks—come from executed experiments on the constructed datasets, not from a chain of equations or definitions that reduce to the inputs. The only self-citation, CTINexus for the CSKG, is used as a component for constructing a retrieval graph; it is not invoked as a uniqueness theorem or as an ansatz that forces the reported outcomes, and the comparison against vanilla RAG is empirically measured. The use of GPT-5 as both a data filter/judge and an evaluated model creates a self-evaluation validity concern, but it does not make any reported number equal to the input by construction; it is a measurement-bias issue rather than circular reasoning. The abstract/full-text mismatch (CTIConnect with 1,860 QA pairs and IRCoT/rerank baselines versus CTIARENA with 691 QA pairs and no such baselines or temporal splits) is a serious internal-consistency and supportability problem, but it is not an instance of the identified circularity patterns. No load-bearing step was found that reduces, by the paper's own construction or self-citation chain, to its own inputs, so the circularity score is 0.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

No new physical or conceptual entities are invented; the ledger captures the benchmark's reliance on label accuracy, judge validity, and self-cited infrastructure, plus retrieval thresholds tuned on the benchmark.

free parameters (4)
  • RAG top-k = 5
    Chosen after exploratory trials (Section 4.1); affects all retrieval results.
  • RAG similarity threshold = 0.5
    Chosen after exploratory trials (Section 4.1); determines what evidence is retained.
  • CSKG entity-overlap rate threshold = 0.6
    Hand-set threshold for CSKG-guided RAG (Section 4.1); no sensitivity analysis.
  • LLM judge filtering threshold = 3 (on 1-5 scale)
    Stage 3 filtering policy retains only QA pairs scored >=3 (Appendix D.2); affects dataset composition.
axioms (5)
  • domain assumption Authoritative CTI mappings (CVE→CWE, CWE→CAPEC, CAPEC→ATT&CK, blog-to-framework) used as seeds are correct and complete enough to serve as ground truth.
    Stage 1 (Section 3.2) builds all QA pairs on these correlations; errors propagate to every label.
  • domain assumption The LLM judge + two practitioner + senior annotator pipeline removes all hallucinated or ambiguous QA pairs.
    Stage 3 (Section 3.2); no inter-annotator agreement or audit of remaining errors is reported.
  • domain assumption GPT-5's rubric-based scoring of open-ended unstructured answers is a valid measure of answer quality.
    Section 4.1 evaluation protocol; no correlation with independent human scores is reported.
  • domain assumption Text-embedding-3-large semantic similarity is a fair representative of general-purpose retrieval.
    Section 4.1 baselines; only one embedding model is used.
  • domain assumption CTINexus knowledge graph (Cheng et al., 2025, same research group) is reliable for entity linking.
    CSKG-guided RAG (Section 4.1) relies on it; self-cited, no independent validation in this paper.

pith-pipeline@v1.3.0-alltime-deepseek · 26020 in / 15029 out tokens · 118423 ms · 2026-08-04T10:00:42.028666+00:00 · methodology

0 comments
read the original abstract

Cyber Threat Intelligence (CTI) is foundational to modern cybersecurity, enabling organizations to proactively defend against evolving threats. However, the sheer volume and heterogeneity of CTI data, spanning structured knowledge bases (CVE, CWE, CAPEC, MITRE ATT&CK) and unstructured threat reports, far exceed the capacity of manual analysis. The strong contextual understanding and reasoning of Large Language Models (LLMs) have driven growing interest in applying them to CTI tasks. Yet no existing benchmark evaluates LLMs in a retrieval-augmented setting with a proper evaluation harness that grants access to the heterogeneous domain knowledge sources analysts rely on in practice. To address this gap, we present CTIConnect, a benchmark for systematically evaluating retrieval-augmented LLMs across the CTI task landscape. We construct a unified evaluation environment integrating five heterogeneous CTI sources into 1,860 expert-verified QA pairs spanning nine tasks across three categories: Entity Linking, Multi-Document Synthesis, and Entity Attribution. Extensive experiments on ten state-of-the-art LLMs reveal that the cross-source semantic gap manifests differently across task categories, demanding fundamentally different retrieval strategies, and that the performance bottleneck shifts between retrieval infrastructure and evidence utilization depending on the task. Our domain-specific strategies further outperform stronger general-purpose retrieval paradigms (retrieve-then-rerank, IRCoT), showing that closing this gap requires structural interventions rather than generic retrieval improvements. These findings hold across all ten LLMs, remain consistent on the full benchmark, and stay stable under temporal splits spanning 2008-2025. Together, they provide actionable guidance for designing scalable retrieval architectures over heterogeneous CTI ecosystems.

Figures

Figures reproduced from arXiv: 2510.11974 by Changze Li, Dawn Song, Peng Gao, Yang Liu, Yutong Cheng.

Figure 1
Figure 1. Figure 1: Each CTI task in CTIARENA is created through a three-stage construction process. The task quality is controlled by the human-LLM collaboration. testing, showing that while LLMs can handle fundamental tasks and operate testing tools effectively, they struggle with context retention and attention management. Beyond penetration testing, LLMs have also been investigated for vulnerability detection (Fang et al.… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

40 extracted references · 10 linked inside Pith

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Ctibench: a benchmark for evaluating llms in cyber threat intelligence

    Md Tanvirul Alam, Le Nguyen, Dipkamal Bhusal, and Nidhi Rastogi. Ctibench: a benchmark for evaluating llms in cyber threat intelligence. In Proceedings of the 38th International Conference on Neural Information Processing Systems (NeurIPS), 2025

    2025

  3. [3]

    Claude 3.5 haiku

    Anthropic . Claude 3.5 haiku. https://www.anthropic.com/claude/haiku, 2024. Accessed: 2025-09-25

    2024

  4. [4]

    Claude sonnet 4

    Anthropic . Claude sonnet 4. https://www.anthropic.com/claude/sonnet, 2025. Accessed: 2025-09-25

    2025

  5. [5]

    Ctinexus: Automatic cyber threat intelligence knowledge graph construction using large language models, 2025

    Yutong Cheng, Osama Bajaber, Saimon Amanuel Tsegai, Dawn Song, and Peng Gao. Ctinexus: Automatic cyber threat intelligence knowledge graph construction using large language models, 2025. URL https://arxiv.org/abs/2410.21060

    Pith/arXiv arXiv 2025

  6. [6]

    Threat intelligence & hunting

    CrowdStrike . Threat intelligence & hunting. https://www.crowdstrike.com/en-us/platform/threat-intelligence/, 2025. Accessed: 2025-09-25

    2025

  7. [7]

    Towards effective identification of attack techniques in cyber threat intelligence reports using large language models

    Hoang Cuong Nguyen, Shahroz Tariq, Mohan Baruwal Chhetri, and Bao Quoc Vo. Towards effective identification of attack techniques in cyber threat intelligence reports using large language models. In Companion Proceedings of the ACM on Web Conference 2025, WWW '25, pp.\ 942–946, New York, NY, USA, 2025. Association for Computing Machinery. ISBN 979840071331...

    arXiv 2025

  8. [8]

    Gemini 1.5 flash, 2025

    Google DeepMind. Gemini 1.5 flash, 2025. URL https://deepmind.google/models/gemini/flash/. Accessed: September 18, 2025

    2025

  9. [9]

    Pentestgpt: An llm-empowered automatic penetration testing tool

    Gelei Deng, Yi Liu, V \' ctor Mayoral-Vilches, Peng Liu, Yuekang Li, Yuan Xu, Tianwei Zhang, Yang Liu, Martin Pinzger, and Stefan Rass. Pentestgpt: An llm-empowered automatic penetration testing tool. arXiv preprint arXiv:2308.06782, 2023

    Pith/arXiv arXiv 2023

  10. [10]

    Llm agents can autonomously exploit one-day vulnerabilities

    Richard Fang, Rohan Bindu, Akul Gupta, and Daniel Kang. Llm agents can autonomously exploit one-day vulnerabilities. arXiv preprint arXiv:2404.08144, 2024

    Pith/arXiv arXiv 2024

  11. [11]

    Gemini pro

    Google DeepMind . Gemini pro. https://deepmind.google/models/gemini/pro/, 2025. Accessed: 2025-09-25

    2025

  12. [12]

    Application of large language models to ddos attack detection

    Michael Guastalla, Yiyi Li, Arvin Hekmati, and Bhaskar Krishnamachari. Application of large language models to ddos attack detection. In Proceedings of the International Conference on Security and Privacy in Cyber-Physical Systems and Smart Vehicles (EAI SmartSP), 2023

    2023

  13. [13]

    CTIKG : LLM -powered knowledge graph construction from cyber threat intelligence

    Liangyi Huang and Xusheng Xiao. CTIKG : LLM -powered knowledge graph construction from cyber threat intelligence. In First Conference on Language Modeling, 2024. URL https://openreview.net/forum?id=DOMP5AgwQz

    2024

  14. [14]

    Sevenllm: Benchmarking, eliciting, and enhancing abilities of large language models in cyber threat intelligence, 2024

    Hangyuan Ji, Jian Yang, Linzheng Chai, Chaoren Wei, Liqun Yang, Yunlong Duan, Yunli Wang, Tianzhen Sun, Hongcheng Guo, Tongliang Li, Changyu Ren, and Zhoujun Li. Sevenllm: Benchmarking, eliciting, and enhancing abilities of large language models in cyber threat intelligence, 2024. URL https://arxiv.org/abs/2405.03446

    Pith/arXiv arXiv 2024

  15. [15]

    Detecting scams using large language models

    Liming Jiang. Detecting scams using large language models. arXiv preprint arXiv:2402.03147, 2024

    Pith/arXiv arXiv 2024

  16. [16]

    A case study of llm for automated vulnerability repair: Assessing impact of reasoning and patch validation feedback

    Ummay Kulsum, Haotian Zhu, Bowen Xu, and Marcelo d'Amorim. A case study of llm for automated vulnerability repair: Assessing impact of reasoning and patch validation feedback. arXiv preprint arXiv:2405.15690, 2024

    Pith/arXiv arXiv 2024

  17. [17]

    Multimodal large language models for phishing webpage detection and identification

    Jehyun Lee, Peiyuan Lim, Bryan Hooi, and Dinil Mon Divakaran. Multimodal large language models for phishing webpage detection and identification. arXiv preprint arXiv:2408.05941, 2024

    Pith/arXiv arXiv 2024

  18. [18]

    Dollm: How large language models understanding network flow data to detect carpet bombing ddos

    Qingyang Li, Yihang Zhang, Zhidong Jia, Yannan Hu, Lei Zhang, Jianrong Zhang, Yongming Xu, Yong Cui, Zongming Guo, and Xinggong Zhang. Dollm: How large language models understanding network flow data to detect carpet bombing ddos. arXiv preprint arXiv:2405.07638, 2024

    Pith/arXiv arXiv 2024

  19. [19]

    Grace: Empowering llm-based software vulnerability detection with graph structure and in-context learning

    Guilong Lu, Xiaolin Ju, Xiang Chen, Wenlong Pei, and Zhilong Cai. Grace: Empowering llm-based software vulnerability detection with graph structure and in-context learning. Journal of Systems and Software (JSS), 212: 0 112031, 2024

    2024

  20. [20]

    Large language model guided protocol fuzzing

    Ruijie Meng, Martin Mirchev, Marcel B \"o hme, and Abhik Roychoudhury. Large language model guided protocol fuzzing. In Proceedings of the 31st Annual Network and Distributed System Security Symposium (NDSS), 2024

    2024

  21. [21]

    Llama 3.1-405b

    Meta . Llama 3.1-405b. https://huggingface.co/meta-llama/Llama-3.1-405B, July 2024 a . Accessed: 2025-09-25

    2024

  22. [22]

    Meta llama 3 -- 8b

    Meta . Meta llama 3 -- 8b. https://huggingface.co/meta-llama/Meta-Llama-3-8B, April 2024 b . Accessed: 2025-09-25

    2024

  23. [23]

    Microsoft Research . Phi-4. https://huggingface.co/microsoft/phi-4, December 2024. Accessed: 2025-09-25

    2024

  24. [24]

    Common attack pattern enumeration and classification (capec), 2025 a

    MITRE. Common attack pattern enumeration and classification (capec), 2025 a . URL https://capec.mitre.org/. Accessed: September 18, 2025

    2025

  25. [25]

    Common weakness enumeration (cwe), 2025 b

    MITRE. Common weakness enumeration (cwe), 2025 b . URL https://cwe.mitre.org/. Accessed: September 18, 2025

    2025

  26. [26]

    Mitre att&ck, 2025 c

    MITRE. Mitre att&ck, 2025 c . URL https://attack.mitre.org/. Accessed: September 18, 2025

    2025

  27. [27]

    Hello gpt-4o

    OpenAI . Hello gpt-4o. https://openai.com/index/hello-gpt-4o/, May 2024. Accessed: 2025-09-25

    2024

  28. [28]

    Introducing gpt-5

    OpenAI . Introducing gpt-5. https://openai.com/index/introducing-gpt-5/, August 2025 a . Accessed: 2025-09-25

    2025

  29. [29]

    Text embedding 3 large — openai documentation

    OpenAI . Text embedding 3 large — openai documentation. https://platform.openai.com/docs/models/text-embedding-3-large, 2025 b . Accessed: 2025-09-25

    2025

  30. [30]

    Unit 42 -- threat intelligence research

    Palo Alto Networks . Unit 42 -- threat intelligence research. https://unit42.paloaltonetworks.com/, 2025. Accessed: 2025-09-25

    2025

  31. [31]

    Common vulnerabilities and exposures (cve), 2025

    CVE Program. Common vulnerabilities and exposures (cve), 2025. URL https://www.cve.org/. Accessed: September 18, 2025

    2025

  32. [32]

    Qwen3-235b-a22b

    Qwen Team . Qwen3-235b-a22b. https://huggingface.co/Qwen/Qwen3-235B-A22B, May 2025. Accessed: 2025-09-25

    2025

  33. [33]

    Time for action: Automated analysis of cyber threat intelligence in the wild, 2023

    Giuseppe Siracusano, Davide Sanvito, Roberto Gonzalez, Manikantan Srinivasan, Sivakaman Kamatchi, Wataru Takahashi, Masaru Kawakita, Takahiro Kakumaru, and Roberto Bifulco. Time for action: Automated analysis of cyber threat intelligence in the wild, 2023. URL https://arxiv.org/abs/2307.10214

    Pith/arXiv arXiv 2023

  34. [34]

    2018 cyber threat intelligence survey report

    ThreatConnect, Inc. 2018 cyber threat intelligence survey report. Technical report, ThreatConnect, January 2018. URL https://threatconnect.com/wp-content/uploads/20180122_Survey_CTI-2018_ThreatConnect_Final.pdf. Accessed: 2025-09-25

    arXiv 2018

  35. [35]

    Threat intelligence center

    Trend Micro . Threat intelligence center. https://www.trendmicro.com/vinfo/us/security/threat-intelligence-center, 2025. Accessed: 2025-09-25

    2025

  36. [36]

    Fuzz4all: Universal fuzzing with large language models

    Chunqiu Steven Xia, Matteo Paltenghi, Jia Le Tian, Michael Pradel, and Lingming Zhang. Fuzz4all: Universal fuzzing with large language models. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering (ICSE), pp.\ 1--13, 2024

    2024

  37. [37]

    Intelex: A llm-driven attack-level threat intelligence extraction framework, 2024

    Ming Xu, Hongtai Wang, Jiahao Liu, Yun Lin, Chenyang Xu Yingshi Liu, Hoon Wei Lim, and Jin Song Dong. Intelex: A llm-driven attack-level threat intelligence extraction framework, 2024. URL https://arxiv.org/abs/2412.10872

    arXiv 2024

  38. [38]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should not add it explicitly Type <Return> for now, but then later remove the command n...

  39. [39]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@first@sw \@firstoftwo \@ifundefined NAT@b*@#2 \@firstoftwo @num @NAT@ctr \@secondoft...

  40. [40]

    description

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibsetup #1 @NAT@ctr @ @openbib .11em \@plus.33em \@minus.07em 4000 4000 `\.\@m @bibit...

    Pith/arXiv arXiv 2025