Pith. sign in

REVIEW 4 major objections 8 minor 65 references

EventHunter: Dynamic Clustering and Ranking of Security Events from Hacker Forum Discussions

T0 review · 4 major / 8 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Hacker-forum posts become ranked security events via clustering

desk verdict Solid system paper with an unsupported clustering generalization claim until the train/test split is clarified. read the letter →

arxiv 2507.09762 v1 pith:3KK5NM7L submitted 2025-07-13 cs.CR cs.AIcs.CL

Yasir Ech-Chammakhy , Anas Motii , Anass Rabii , Jaafar Chbili This is my paper
classification cs.CRcs.AIcs.CL
keywords cyberthreatintelligencehackerforumssecurityeventdetectioncontrastivelearningHDBSCANclusteringprioritizationnamedentityrecognitiontransformerembeddings
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

EventHunter is an unsupervised pipeline that takes noisy, fragmented posts from underground forums and groups them into distinct security events, then ranks those events so an analyst knows what to investigate first. The paper's claim is that contrastively fine-tuned Transformer embeddings, built with an extra signal that marks named entities such as CVEs, malware names, and targeted organizations, make posts about the same incident cluster together far better than keyword, TF-IDF, or word-vector baselines. On 21 curated real-world events, the best configuration reaches NMI 0.712 and ARI 0.347, and a case-study ranking puts the Optus breach first while demoting older, thinner discussions. The payoff, if true, is moving from a firehose of forum chatter to a short, prioritized list of emerging threats.

What carries the argument

The mechanism is a three-stage representation-to-cluster-to-score chain. Posts are first encoded by a Transformer whose input layer adds an entity-presence embedding to the token, segment, and position embeddings, so named entities act as anchors; the model is then fine-tuned with a combined contrastive objective made of a triplet margin loss and a SimCSE-style pairwise cosine loss over in-batch negatives. HDBSCAN groups the resulting embeddings into clusters of at least five posts and labels the rest as noise. Each cluster receives a daily priority score $s_i = w_T T + w_V V + w_R R + w_C C$, where timeliness mixes exponential recency decay with log-scaled activity, relevance is semantic entity overlap with an optional analyst query, credibility averages the reputation of contributing authors, and completeness counts unique entities and entity types.

What would settle it

Re-run the pipeline with a strict holdout: remove every post belonging to the 21 ground-truth events from the contrastive fine-tuning set, train on the remaining posts, then cluster the held-out events. If NMI falls well below 0.712 and ARI below 0.347, the published numbers depended on training on the test events.

Watch

Extended reading notes

Core claim

The central discovery is that a fully unsupervised pipeline can turn fragmented forum posts into event clusters that align with known incidents. The best configuration uses a cybersecurity-domain Transformer fine-tuned with a pairwise contrastive loss: the embedding space groups posts by shared event, and HDBSCAN extracts clusters while marking off-topic posts as noise. The authors also show that a weighted priority score combining timeliness, relevance, credibility, and completeness produces a plausible ranking, with the Optus breach ranked first and an older LockBit infrastructure discussion ranked near the bottom. They conclude that the framework reduces noise and surfaces high-priority threats for security analysts.

Load-bearing premise

The clustering numbers assume the 21 test events were held out of the contrastive fine-tuning; the paper never states that split, so if the model was trained on posts from those same events, the reported NMI and ARI would overstate how well it detects new events.

Editorial extensions

If this is right

  • An analyst could monitor a hacker forum daily and receive a ranked event list without waiting for public vulnerability disclosures or vendor reports.
  • Events discussed across multiple threads or forums with different vocabulary can be merged by embedding proximity, even when no keyword is shared.
  • The priority score gives a transparent order: recent, active, author-credible, information-rich clusters rise, while older or sparse discussions fall, so triage effort concentrates on the top of the list.
  • The zero-shot LLM entity extraction makes the pipeline adaptable to new forum slang and entity types without retraining a domain-specific NER model.
  • The reported clustering metrics, NMI 0.712 and ARI 0.347 on 21 curated events, provide a testable benchmark for future event-detection work on underground-forum data.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper's evaluation, a sliding-window version of the clustering could test whether the framework detects when a new event begins and when an old one fades, which the static snapshot does not measure.
  • The credibility and timeliness metrics rely only on internal forum metadata; correlating the ranked clusters against an external incident database would show whether the top-ranked events correspond to major real-world breaches.
  • The failure of the output-level entity loss suggests that forcing post embeddings to align with individual entity embeddings distracts from event-level structure; a plausible alternative is entity-weighted attention rather than an auxiliary loss.
  • If actor scores were accumulated from contributions to high-ranked clusters, the framework would yield an event-centric measure of forum influence that could be checked against known threat-actor profiles.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 8 minor

Summary. EventHunter is presented as an unsupervised pipeline that processes CrimeBB hacker-forum posts through five stages: keyword-based pre-filtering and multi-class post classification (DarkBERT, average F1 0.739 over 14,386 manually annotated posts), zero-shot LLM-based entity extraction (mistral-nemo, mention-level F1 52.13 on the CyNER benchmark), contrastively fine-tuned Transformer post embeddings (triplet and pairwise SimCSE-style losses, with an optional entity-aware input mechanism), HDBSCAN clustering, and a weighted daily ranking over timeliness, relevance, credibility, and completeness. The clustering evaluation uses 21 events randomly selected from a curated 70-event ground truth and reports DarkBERT (Pairwise) with NMI 0.712 and ARI 0.347 and CySecBERT (Pairwise) with ARI 0.377, against TF-IDF and Word2Vec baselines of NMI 0.638 and 0.222. Ranking is demonstrated with three qualitative case studies and a top-3 comparison against recency and activity heuristics, and the authors conclude that EventHunter reduces forum noise and surfaces high-priority threats. The paper explicitly acknowledges the static (snapshot) nature of the clustering evaluation, sensitivity of the Priority Score to its weights, and the absence of external ground truth for ranking as limitations.

Significance. If the clustering numbers survive a correctly held-out evaluation, this is a valuable end-to-end contribution: it is one of the few large-scale studies of forum event detection with an explicit analyst-facing prioritization stage, and it ships public code, evaluates classification on 14,386 labeled posts, compares five Transformer architectures and three contrastive objectives, and benchmarks zero-shot NER on the standard CyNER corpus. The decisive weakness is that the reported NMI/ARI values may measure memorization of events seen during contrastive fine-tuning rather than generalization to new events; because the system's stated purpose is early detection of emerging events, the value of the contribution depends on resolving this. The ranking contribution is plausible but is currently supported only by qualitative evidence, and the paper's own limitation list stops short of acknowledging the train/test leakage risk.

major comments (4)
  1. [§III-D, §IV-A, §IV-D] On reading the paper, the holdout concern is confirmed: §III-D, §IV-A, and §IV-D contain no statement that the 21 events used as the clustering test set in §IV-D were excluded from the contrastive fine-tuning of §III-D, in which positive pairs are drawn from posts of the same ground-truth event and negative pairs from different ground-truth events. Because §IV-D selects the test events from the same 70-event ground-truth set described in §IV-A, the reported NMI 0.712 and ARI 0.347 (Table IV) may reflect the embedding model having been trained on pairs drawn from the exact events on which it is later evaluated. This is load-bearing: if the 21 events were not held out, the numbers measure memorization of the ground-truth clusters, and the central claim that EventHunter detects new, previously unseen events is unsupported. The authors must state the split explicitly and, unless the 21 events were genuinely excluded from pair construction, re-run the evaluation on events held out from fine-tuning and report those results. The same absence of a stated train/test split also affects the classification evaluation in §IV-B/Table III for the 14,386 annotated posts. The ground-truth curation in §IV-A (clusters defined by unique shared markers such as CVEs, malware names, and target names) makes the holdout essential rather than cosmetic, since the contrastive objective is trained to reproduce exactly the entity-sharing structure that defines the target clusters.
  2. [§IV-D, Tables IV–VII] All clustering results are single-run values with no variance estimate, significance test, or stated random seed, and the paper does not specify how HDBSCAN noise points (posts labeled as noise) are treated when computing ARI and NMI with scikit-learn; options such as dropping noise points, labeling them as one additional cluster, or giving each noise point its own label materially change both metrics. The 'Clusters Found' column in Tables IV and V (e.g., 14 clusters found by DarkBERT against 21 true events, 9 by BERT) shows substantial under-segmentation, which the paper does not interpret. Reporting mean and standard deviation over multiple random subsets of the 70 events, a seed, and explicit noise handling is necessary before the quantitative ranking of configurations (e.g., 0.712 vs. 0.714) can be taken at face value.
  3. [§IV-D, Table IV vs. Overall Performance Summary] Table IV reports RoBERTa (Triplet+Pairwise) with ARI 0.402 and NMI 0.714, while the Overall Performance Summary in the same section states that DarkBERT (Pairwise Loss) achieved the highest NMI (0.712); both statements cannot be true as written. No row in Tables V or VI corresponds to the Table IV RoBERTa configuration, so the reader cannot reconcile the conflict or determine which configuration is actually best. The manuscript's headline result is therefore ambiguous, and the tables, findings, and summary must be made mutually consistent.
  4. [§IV-E, Eq. (9)] The prioritization claim relies on three qualitative case studies (Table VIII) and a top-3 comparison against recency and activity baselines (Table IX). A distinct ordering is not evidence of a better ordering: the weights in Eq. (9) are fixed by the authors (wT=0.35, wV=0.25, wR=0.20, wC=0.20), the relevance term is disabled (V=0) during evaluation, and no quantitative link between the Priority Score and any external measure of event importance (verified incident severity, public disclosure timing, or analyst judgment) is provided. The weight-sensitivity concern is acknowledged in §V-A, but that does not supply the missing evidence for the ranking contribution; at minimum, a sensitivity analysis over the Eq. (9) weights and a small external-validity check (e.g., correlation with later public reporting dates) are needed.
minor comments (8)
  1. [§IV-C] The paper calls CyNER 'the standard CyNER dataset [2]', but reference [2] is a Python library; please specify the underlying corpus (e.g., SAM, CNN, DNS, or MITRE) and the entity schema used in the zero-shot evaluation.
  2. [Table IV] The row labeled 'Word2Vec (GloVe)' conflates two distinct embedding families; please state which method actually produced the reported ARI -0.024 and NMI 0.222.
  3. [§IV-D, Finding 5] The pairwise-loss comparison values quoted in Finding 5 (RoBERTa NMI 0.702, DarkBERT NMI 0.592) conflict with Table VI (RoBERTa Pairwise 0.664, DarkBERT Pairwise 0.712), and the detailed results are omitted; the discrepancy and the omitted numbers must be resolved.
  4. [Table III] The column header 'DosAttack' differs from the 'DoS/DDoS' category name used in Table I; please harmonize the terminology.
  5. [§III-A and §I] The abstract and problem statement call the framework 'unsupervised' even though §III-D fine-tunes the embedding model using ground-truth event clusters; please clarify that the clustering stage is unsupervised while the representation-learning stage is supervised by the curated event labels.
  6. [Abstract and §III-C] The claim of identifying incidents 'without relying on predefined keywords' is in tension with the keyword-based filtering used to build the classification dataset; please clarify whether keyword filtering is applied at inference time in the deployed pipeline or only during dataset construction.
  7. [§IV-A and §IV-D] To make the evaluation reproducible from the public code alone (since CrimeBB raw posts are not shareable), please release or describe the ground-truth post-ID sets for the 70 events, the classification labels, and the random seed used to select the 21 test events.
  8. [§IV-A] min_cluster_size=5 for HDBSCAN directly controls the number and granularity of event clusters and is a free parameter; a short sensitivity statement for at least the best configuration would show that the headline NMI/ARI numbers are not artifacts of this choice.

Circularity Check

1 steps flagged · score 6.0 of 10

As written, the contrastive fine-tuning and the 21-event clustering test draw on the same ground-truth events with no stated holdout, so the reported NMI/ARI may measure training fit rather than detection of new events.

  1. fitted input called prediction [Section III-D (contrastive fine-tuning objective, Eqs. 1-2) and Section IV-D (clustering test set)]
    "Section III-D: "This loss operates on triplets of (anchor post, positive post, negative post), where the positive post belongs to the same ground-truth event as the anchor, and the negative post belongs to a different ground-truth event." Section IV-D: "From the initially identified set of 70 distinct security events, we randomly selected a subset of 21 events to serve as the test set for this clustering evaluation.""

    The contrastive objective is supervised by exactly the ground-truth event membership: positive pairs are posts from the same ground-truth event and negatives come from different ground-truth events. The clustering evaluation then draws its 21 test events from the same 70-event ground-truth pool described in Section IV-A, and the paper never states that those 21 events or their posts were held out of the triplet/pair construction. As written, therefore, the model may have been fine-tuned to make the very test clusters compact and separated. In that case, the reported DarkBERT NMI 0.712 / ARI 0.347 are a measure of fit to the training labels, not of the ability to group unseen fragmented forum posts into new events.

full rationale

The framework's components—entity-presence input embeddings, HDBSCAN clustering, and the weighted priority score of Eq. (9)—are described and are not themselves circular; the ranking evaluation is applied directly to curated ground-truth clusters, which is an evaluation-scope limitation rather than a circular derivation. The main circularity concern is the training/evaluation split. Section III-D defines the post-level contrastive losses using positive pairs from the same ground-truth event and negatives from different ground-truth events, i.e., the embedding is explicitly optimized to reproduce ground-truth event membership. Section IV-D then selects the 21-event test set from the same 70-event ground-truth source, and no sentence indicates that these events or their posts were held out from fine-tuning. If they were not, the reported clustering metrics measure in-sample fit, and the central claim of detecting emerging events is unsupported by the experiment as written. The self-citations (refs. 4, 5, 20) are background and not load-bearing, and no uniqueness theorem is imported. Score 6 reflects that the central empirical claim has this partial, unguarded circularity; the non-evaluation contributions are independent.

Assumptions & free parameters 8 free parameters · 7 assumptions · 0 invented entities

The framework rests on many hand-chosen hyperparameters and unvalidated domain assumptions; the most consequential is the unstated train/test event split. No invented entities in the particle or mediator sense are introduced.

free parameters (8)
  • HDBSCAN min_cluster_size = 5
    Set by configuration; clusters with fewer than 5 posts are treated as noise, which affects which events can be discovered at all.
  • Contrastive margin alpha = 0.5
    Used in the triplet loss (Eq. 1); chosen as a common value, with no sensitivity analysis reported.
  • Pairwise loss temperature tau = not reported
    The pairwise contrastive loss (Eq. 2) depends on this temperature, but the value is not given.
  • Contrastive loss weights w_t, w_p = not reported
    The combined objective in Section III-D is a weighted sum of triplet and pairwise losses; the weights used in experiments are not disclosed.
  • Ranking weights wT, wV, wR, wC = 0.35, 0.25, 0.20, 0.20
    Default weights in Section IV-E selected by the authors; relevance is disabled in the evaluation and no sensitivity analysis is performed.
  • Timeliness decay constant tau_Rec and weights wRec, wAct = not reported
    Equations 3-5 require these values for the ranking score, but the paper does not give them, making the daily ranking non-reproducible.
  • Completeness weights w_alpha, w_beta = not reported
    Equation 8 requires these weights; values are not disclosed.
  • Contrastive fine-tuning hyperparameters (learning rate, epochs, dropout) = 2e-5, 5 epochs, 0.1
    These are given in Section IV-A, but no search or sensitivity analysis is reported.
assumptions (7)
  • domain assumption CrimeBB is an appropriate and accurate source of hacker forum posts.
    All experiments rely on the integrity of the CrimeBB corpus [44]; no independent verification of the collected forum text or metadata is performed.
  • domain assumption Ground-truth event clusters are correctly identified by CVE IDs, malware names, and targeted entities.
    The ground truth in Section IV-A is constructed using unique markers plus manual inspection; if these markers do not correspond to true event boundaries, the clustering evaluation is invalid.
  • domain assumption HDBSCAN with a fixed min_cluster_size can discover event clusters of any size or density above the threshold.
    The framework assumes density-based clustering in embedding space is a valid model of event structure; no alternative clustering algorithm is compared.
  • domain assumption Zero-shot LLM NER performance on CyNER transfers to hacker forum language.
    Mistral-nemo is selected using the standard CyNER benchmark in Section IV-C, but no evaluation is reported on forum posts, despite the paper noting a linguistic disparity between formal CTI texts and forums.
  • domain assumption Contrastive fine-tuning with ground-truth event pairs generalizes to unseen events and does not memorize training clusters.
    This is the unstated holdout assumption in Sections III-D and IV-D; if violated, the clustering results are circular.
  • domain assumption Forum reputation metadata is a valid credibility signal for event priority.
    The credibility metric in Eq. 7 averages CrimeBB reputation scores; no validation is provided that reputation correlates with trustworthy information.
  • domain assumption Weighted linear combination of timeliness, relevance, credibility, and completeness reflects operational priority.
    The priority score in Eq. 9 is a hand-designed aggregation; no external ground truth or analyst validation is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of EventHunter: Dynamic Clustering and Ranking of Security Events from Hacker Forum Discussions." pith.science (2026). https://pith.science/paper/3KK5NM7L

@misc{pith2026250709762,
  author       = {Pith},
  title        = {Pith review of: EventHunter: Dynamic Clustering and Ranking of Security Events from Hacker Forum Discussions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3KK5NM7L}},
  note         = {Machine review of arXiv:2507.09762}
}
read the original abstract

Hacker forums provide critical early warning signals for emerging cybersecurity threats, but extracting actionable intelligence from their unstructured and noisy content remains a significant challenge. This paper presents an unsupervised framework that automatically detects, clusters, and prioritizes security events discussed across hacker forum posts. Our approach leverages Transformer-based embeddings fine-tuned with contrastive learning to group related discussions into distinct security event clusters, identifying incidents like zero-day disclosures or malware releases without relying on predefined keywords. The framework incorporates a daily ranking mechanism that prioritizes identified events using quantifiable metrics reflecting timeliness, source credibility, information completeness, and relevance. Experimental evaluation on real-world hacker forum data demonstrates that our method effectively reduces noise and surfaces high-priority threats, enabling security analysts to mount proactive responses. By transforming disparate hacker forum discussions into structured, actionable intelligence, our work addresses fundamental challenges in automated threat detection and analysis.

Figures

Figures reproduced from arXiv: 2507.09762 by the authors.

Figure 1
Figure 1. Overview of the EventHunter framework: Forum posts are processed through data preparation (including classification and entity extraction) and [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Illustration of CTI entity identification within a forum post announcing a data breach. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 4
Figure 4. Illustration of the combined contrastive fine-tuning process for the [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figures from the paper (2 more)
Figure 5
Figure 5. Figure 5: NMI Scores by Model and Loss Configuration (Input Entity Experi [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: Comparison of NMI (A) and ARI (B) for BERT using Input vs. [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

65 extracted references · 59 canonical work pages

  1. [1]

    Securebert: A domain-specific language model for cybersecurity

    Ehsan Aghaei, Xi Niu, Waseem Shadid, and Ehab Al-Shaer. Securebert: A domain-specific language model for cybersecurity. arXiv, October 2022

    work page 2022

  2. [2]

    Cyner: A python library for cybersecurity named entity recognition,

    Md Tanvirul Alam, Dipkamal Bhusal, Youngja Park, and Nidhi Rastogi. Cyner: A python library for cybersecurity named entity recognition,

    work page

  3. [3]

    Alkhodair, Steven H

    Sarah A. Alkhodair, Steven H. H. Ding, Benjamin C. M. Fung, and Junqiang Liu. Detecting breaking news rumors of emerging topics in social media. Information Processing & Management , 57(2):102018, March 2020

    work page 2020

  4. [4]

    EUREKHA: Enhancing User Representation for Key Hackers Identification in Underground Forums

    Abdoul Nasser Hassane Amadou, Anas Motii, Saida Elouardi, and El Houcine Bergou. EUREKHA: Enhancing User Representation for Key Hackers Identification in Underground Forums. In 2024 IEEE 23rd International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom) , pages 387–398. IEEE, 2024

    work page 2024

  5. [5]

    HC-HackerRank: Identifying Key Hackers in Cybercrime Social Net- work Forums

    Abdoul Nasser Hassane Amadou, Anas Motii, and Mohammed Jouhari. HC-HackerRank: Identifying Key Hackers in Cybercrime Social Net- work Forums. In 2024 7th International Conference on Advanced Communication Technologies and Networking (CommNet) , pages 1–8. IEEE, 2024

    work page 2024

  6. [6]

    Arikkat, Serena Nicolazzo, Antonino Nocera, Rafidha Rehiman K

    Marco Arazzi, Dincy R. Arikkat, Serena Nicolazzo, Antonino Nocera, Rafidha Rehiman K. A., P. Vinod, and Mauro Conti. NLP-Based Tech- niques for Cyber Threat Intelligence. arXiv preprint, arXiv:2311.08807, November 15 2023

    arXiv 2023

  7. [7]

    Using twitter data to monitor natural disaster social dynamics: A recurrent neural network approach with word embeddings and kernel density estimation

    Zahra Ashktorab, Christopher Brown, Manojit Nandi, and Aron Culotta. Using twitter data to monitor natural disaster social dynamics: A recurrent neural network approach with word embeddings and kernel density estimation. Sensors, 19(7):1746, 2019

    work page 2019

  8. [8]

    Dhananjay Ashok and Zachary C. Lipton. PromptNER: Prompting For Named Entity Recognition. arXiv, June 2023. arXiv preprint arXiv:2305.15444

    arXiv 2023

Show all 65 references
  1. [9]

    Avishek Bose, Vahid Behzadan, Carlos Aguirre, and William H. Hsu. A novel approach for detection and ranking of trendy and emerging cyber threat events in twitter streams. In Proceedings of the 2019 IEEE/ACM International Conference on Advances in Social Networks Analysis and ...

    2019

  2. [10]

    Density- based clustering based on hierarchical density estimates

    Ricardo JGB Campello, Davoud Moulavi, and Joerg Sander. Density- based clustering based on hierarchical density estimates. In Advances in Knowledge Discovery and Data Mining , volume 7819, pages 160–172. Springer, 2013

    2013

  3. [11]

    Improving quality of indicators of compromise using stix graphs

    Sheng-Shan Chen, Ren-Hung Hwang, Asad Ali, Ying-Dar Lin, Yu-Chih Wei, and Tun-Wen Pai. Improving quality of indicators of compromise using stix graphs. Computers & Security, 144:103972, September 2024

    2024

  4. [12]

    Enhancing cyber threat intelligence with named entity recognition using bert-crf

    Sheng-Shan Chen, Ren-Hung Hwang, Chin-Yu Sun, Ying-Dar Lin, and Tun-Wen Pai. Enhancing cyber threat intelligence with named entity recognition using bert-crf. In GLOBECOM 2023 - 2023 IEEE Global Communications Conference , pages 7532–7537, Kuala Lumpur, Malaysia, 2023. IEEE

    2023

  5. [13]

    A survey of large language models for cyber threat detection

    Yiren Chen, Mengjiao Cui, Ding Wang, Yiyang Cao, Peian Yang, Bo Jiang, Zhigang Lu, and Baoxu Liu. A survey of large language models for cyber threat detection. Computers & Security , 145:104016, October 2024

    2024

  6. [14]

    A Novel Prompting Method for Few-Shot NER via LLMs.Natural Language Processing Journal, 8:100099, September 2024

    Qi Cheng, Liqiong Chen, Zhixing Hu, Juan Tang, Qiang Xu, and Binbin Ning. A Novel Prompting Method for Few-Shot NER via LLMs.Natural Language Processing Journal, 8:100099, September 2024

    2024

  7. [15]

    Tweezers: A framework for security event detection via event attribution-centric tweet embedding

    Jian Cui, Hanna Kim, Eugene Jang, Dayeon Yim, Kicheol Kim, Yongjae Lee, Jin-Woo Chung, Seungwon Shin, and Xiaojing Liao. Tweezers: A framework for security event detection via event attribution-centric tweet embedding. arXiv preprint arXiv:2409.08221, 2024

    2024 arXiv

  8. [16]

    Cyber threat intelligence survey, 2024

    Cybersixgill. Cyber threat intelligence survey, 2024. Accessed: 2025- 02-27

    2024

  9. [17]

    Collecting cyber threat intelligence from hacker forums via a two-stage, hybrid process using support vector machines and latent dirichlet allocation

    Isuf Deliu, Carl Leichter, and Katrin Franke. Collecting cyber threat intelligence from hacker forums via a two-stage, hybrid process using support vector machines and latent dirichlet allocation. In 2018 IEEE International Conference on Big Data (Big Data) , pages 5008–5013, ...

    2018

  10. [18]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin et al. Bert: Pre-training of deep bidirectional transformers for language understanding. In NAACL, 2019

    2019

  11. [19]

    New cyber threat discovery from darknet marketplaces

    Fangzhou Dong, Shaoxian Yuan, and Liang Liu. New cyber threat discovery from darknet marketplaces. In 2018 IEEE International Conference on Big Data and Artificial Intelligence (ICBDAI) , pages 69–

    2018

  12. [20]

    A survey on hybrid-cnn and llms for intrusion detection systems: Recent iot datasets

    Saida Elouardi, Anas Motii, Mohammed Jouhari, Abdoul Amadou, and Mustapha Hedabou. A survey on hybrid-cnn and llms for intrusion detection systems: Recent iot datasets. IEEE Access , PP:1–1, 2024. Published: November 26, 2024

    2024

  13. [21]

    Detecting cyber threat event from twitter using idcnn and bilstm

    Yong Fang, Jian Gao, Zhonglin Liu, and Cheng Huang. Detecting cyber threat event from twitter using idcnn and bilstm. Applied Sciences , 10(17):5922, 2020

    2020

  14. [22]

    Analyzing and identifying data breaches in underground forums

    Yong Fang, Yusong Guo, Cheng Huang, and Liang Liu. Analyzing and identifying data breaches in underground forums. IEEE Access , 7:48770–48777, 2019

    2019

  15. [23]

    Real-time event detection on social data streams

    Mateusz Fedoryszak, Brent Frederick, Vijay Rajaram, and Changtao Zhong. Real-time event detection on social data streams. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining , pages 2774–2782, New York, NY , USA,

  16. [24]

    Cysecbert: Cybersecurity language model

    Nicolas Fiorini et al. Cysecbert: Cybersecurity language model. In IEEE International Conference on Big Data , 2023

    2023

  17. [25]

    Simcse: Sim- ple contrastive learning of sentence embeddings

    Tianyu Gao, Xingcheng Yao, and Danqi Chen. Simcse: Sim- ple contrastive learning of sentence embeddings. arXiv preprint arXiv:2104.08821, 2021

    2021 arXiv

  18. [26]

    The ‘big beast to tackle’: Practices in quality assurance for cyber threat intelligence

    Thomas Geras and Thomas Schreck. The ‘big beast to tackle’: Practices in quality assurance for cyber threat intelligence. In The 27th Inter- national Symposium on Research in Attacks, Intrusions and Defenses , pages 337–352, Padua, Italy, 2024. ACM

    2024

  19. [27]

    Towards natural disasters detection from twitter using topic modelling

    Mohamed Hagras, Ghada Hassan, and Nadine Farag. Towards natural disasters detection from twitter using topic modelling. In 2017 European Conference on Electrical Engineering and Computer Science (EECS) , pages 272–279. IEEE, 2017

    2017

  20. [28]

    Comparing partitions

    Lawrence Hubert and Phipps Arabie. Comparing partitions. Journal of Classification, 2(1):193–218, 1985

    1985

  21. [29]

    2023 cti survey: Keeping up with a changing threat landscape, 2023

    SANS Institute. 2023 cti survey: Keeping up with a changing threat landscape, 2023

    2023

  22. [30]

    Darkbert: A language model for the dark side of the internet

    Seungjun Jin et al. Darkbert: A language model for the dark side of the internet. In ACL, 2023

    2023

  23. [31]

    Exploring the dark web for cyber threat intelligence using machine learning

    Masashi Kadoguchi, Shota Hayashi, Masaki Hashimoto, and Akira Ot- suka. Exploring the dark web for cyber threat intelligence using machine learning. In 2019 IEEE International Conference on Intelligence and Security Informatics (ISI) , pages 200–202. IEEE, 2019

    2019

  24. [32]

    Crowdsourcing cybersecurity: Cyber attack detection using social media

    Rupinder Paul Khandpur, Taoran Ji, Steve Jan, Gang Wang, Chang- Tien Lu, and Naren Ramakrishnan. Crowdsourcing cybersecurity: Cyber attack detection using social media. https://doi.org/10.48550/arXiv.1702. 07745, 2017. arXiv:1702.07745

    2017 doi arXiv

  25. [33]

    Event graph-based news cluster- ing: The role of named entity-centered subgraphs

    Basak K ¨omec ¸oglu and Burcu Yilmaz. Event graph-based news cluster- ing: The role of named entity-centered subgraphs. IEEE Access, PP:1–1, January 2024

    2024

  26. [34]

    Nedetector: Automatically extracting cybersecurity neologisms from hacker forums

    Ying Li, Jiaxing Cheng, Cheng Huang, Zhouguo Chen, and Weina Niu. Nedetector: Automatically extracting cybersecurity neologisms from hacker forums. Journal of Information Security and Applications , 58:102784, May 2021

    2021

  27. [35]

    Roberta: A robustly optimized bert pretraining approach

    Yinhan Liu et al. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 , 2019

    1907 arXiv

  28. [36]

    Zero-shot entity linking by reading entity descriptions

    Lajanugen Logeswaran, Ming-Wei Chang, Kenton Lee, Kristina Toutanova, Jacob Devlin, and Honglak Lee. Zero-shot entity linking by reading entity descriptions. In Anna Korhonen, David Traum, and Llu´ıs M `arquez, editors, Proceedings of the 57th Annual Meeting of the Association...

    2019

  29. [37]

    Decoupled weight decay regulariza- tion

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regulariza- tion. arXiv preprint arXiv:1711.05101 , 2019

    2019 arXiv

  30. [38]

    The rapidly evolving threat landscape of 2024, 2024

    Cyber Magazine. The rapidly evolving threat landscape of 2024, 2024. Accessed: 2025-02-27

    2024

  31. [39]

    Understand your shady neighborhood: An approach for detecting and investigating hacker communities

    Dalyapraz Manatova, Charles DeVries, and Sagar Samtani. Understand your shady neighborhood: An approach for detecting and investigating hacker communities. Decision Support Systems, 184:114271, September 2024

    2024

  32. [40]

    Cream skimming the un- derground: Identifying relevant information points from online forums

    Felipe Moreno-Vera, Mateus Nogueira, Cain ˜a Figueiredo, Daniel Sadoc Menasch´e, Miguel Bicudo, Ashton Woiwood, Enrico Lovat, Anton Kocheturov, and Leandro Pfleger de Aguiar. Cream skimming the un- derground: Identifying relevant information points from online forums. arXiv pr...

    2023 arXiv

  33. [41]

    Entity and relation extractions for threat intelligence knowledge graphs

    Inoussa Mouiche and Sherif Saad. Entity and relation extractions for threat intelligence knowledge graphs. Computers & Security , 148:104120, January 2025

    2025

  34. [42]

    You might have known it earlier: Analyzing the role of underground forums in threat intelligence

    Tommaso Paladini, Lara Ferro, Mario Polino, Stefano Zanero, and Michele Carminati. You might have known it earlier: Analyzing the role of underground forums in threat intelligence. In Proceedings of the 27th International Symposium on Research in Attacks, Intrusions and Defens...

    2024

  35. [43]

    Event detection from social media for epidemic prediction

    Tanmay Parekh, Anh Mac, Jiarui Yu, Yuxuan Dong, Syed Shahriar, Bonnie Liu, Eric Yang, et al. Event detection from social media for epidemic prediction. In Kevin Duh, Helena Gomez, and Steven Bethard, editors, Proceedings of the 2024 Conference of the North American Chapter of ...

    2024

  36. [44]

    Thomas, Alice Hutchings, and Richard Clayton

    Sergio Pastrana, Daniel R. Thomas, Alice Hutchings, and Richard Clayton. CrimeBB: Enabling Cybercrime Research on Underground Forums at Scale. In Proceedings of the 2018 World Wide Web Conference on World Wide Web - WWW ’18 , pages 1845–1854. ACM Press, 2018

    2018

  37. [45]

    Pytorch: An imperative style, high-performance deep learning library

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Brad- bury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems...

    2019

  38. [46]

    Scikit-learn: Machine learning in python

    Fabian Pedregosa, Gael Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Pretten- hofer, Ron Weiss, Vincent Dubourg, et al. Scikit-learn: Machine learning in python. Journal of Machine Learning Research, 12:2825–2830, 2011

    2011

  39. [47]

    What are the attackers doing now? automating cyber threat intelligence extraction from text on pace with the changing threat landscape: A survey

    Md Rayhanur Rahman, Rezvan Mahdavi-Hezaveh, and Laurie Williams. What are the attackers doing now? automating cyber threat intelligence extraction from text on pace with the changing threat landscape: A survey. ACM Computing Surveys , 55(12):1–36, 2023

    2023

  40. [48]

    Severity-based triage of cybersecurity incidents using kill chain attack graphs

    Luk ´aˇs Sadlek, Muhammad Mudassar Yamin, Pavel ˇCeleda, and Basel Katt. Severity-based triage of cybersecurity incidents using kill chain attack graphs. Journal of Information Security and Applications , 89:103956, March 2025

    2025

  41. [49]

    Ex- ploring emerging hacker assets and key hackers for proactive cyber threat intelligence

    Sagar Samtani, Ryan Chinn, Hsinchun Chen, and Jay Nunamaker. Ex- ploring emerging hacker assets and key hackers for proactive cyber threat intelligence. Journal of Management Information Systems , 34(4):1023– 1053, 2017

    2017

  42. [50]

    Early warnings of cyber threats in online discussions

    Anna Sapienza, Alessandro Bessi, Saranya Damodaran, Paulo Shakarian, Kristina Lerman, and Emilio Ferrara. Early warnings of cyber threats in online discussions. In 2017 IEEE International Conference on Data Mining Workshops (ICDMW), pages 667–674, New Orleans, LA, 2017. IEEE

    2017

  43. [51]

    Event-driven news stream clustering using entity-aware contextual embeddings

    Kailash Karthik Saravanakumar, Miguel Ballesteros, Muthu Kumar Chandrasekaran, and Kathleen McKeown. Event-driven news stream clustering using entity-aware contextual embeddings. In Paola Merlo, J¨org Tiedemann, and Reut Tsarfaty, editors, Proceedings of the 16th Conference of...

    2021

  44. [52]

    SONAR: Automatic Detection of Cyber Security Events over the Twitter Stream

    Quentin Sceller, Elmouatez Karbab, Mourad Debbabi, and Farkhund Iqbal. SONAR: Automatic Detection of Cyber Security Events over the Twitter Stream. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management (CIKM) , 2017

    2017

  45. [53]

    Cybersecurity event detection with new and re-emerging words

    Hyejin Shin, WooChul Shim, Jiin Moon, Jae Seo, Sol Lee, and Yong Hwang. Cybersecurity event detection with new and re-emerging words. In Proceedings of the 2020 ACM SIGKDD Workshop on Cybersecurity and Intelligence. ACM, 2020

    2020

  46. [54]

    Dense vs

    Todor Staykovski, Alberto Barr ´on-Cede˜no, Giovanni Da San Martino, and Preslav Nakov. Dense vs. sparse representations for news stream clustering. In Proceedings of the 2nd International Workshop on Narra- tive Extraction from Texts (Text2Story@ECIR) , pages 43–48, Cologne, ...

    2019

  47. [55]

    Trigger- free cybersecurity event detection based on contrastive learning

    Mengmeng Tang, Yuanbo Guo, Qingchun Bai, and Han Zhang. Trigger- free cybersecurity event detection based on contrastive learning. The Journal of Supercomputing , 79:20984–21007, 2023

    2023

  48. [56]

    A feature-driven method for automating the assessment of osint cyber threat sources

    Andrea Tundis, Samuel Ruppert, and Max M ¨uhlh¨auser. A feature-driven method for automating the assessment of osint cyber threat sources. Computers & Security , 113:102576, 2022

    2022

  49. [57]

    Information theoretic measures for clusterings comparison: Variants, properties, normalization and correction for chance

    Nguyen Xuan Vinh, Julien Epps, and James Bailey. Information theoretic measures for clusterings comparison: Variants, properties, normalization and correction for chance. In Journal of Machine Learning Research, volume 11, pages 2837–2854, 2010

    2010

  50. [58]

    Aptner: A specific dataset for ner missions in cyber threat intelligence field

    Xuren Wang, Songheng He, Zihan Xiong, Xinxin Wei, Zhengwei Jiang, Sihan Chen, and Jun Jiang. Aptner: A specific dataset for ner missions in cyber threat intelligence field. In 2022 IEEE 25th International Con- ference on Computer Supported Cooperative Work in Design (CSCWD) , ...

    2022

  51. [59]

    Dnrti: A large-scale dataset for named entity recognition in threat intelligence

    Xuren Wang, Xinpei Liu, Shengqin Ao, Ning Li, Zhengwei Jiang, Zongyi Xu, Zihan Xiong, Xiong Mengbo, and Xiaoqing Zhang. Dnrti: A large-scale dataset for named entity recognition in threat intelligence. In 2020 IEEE 19th International Conference on Trust, Security and Privacy i...

    2020

  52. [60]

    Detecting cybersecurity events from noisy short text

    Semih Yagcioglu, Mehmet Saygin Seyfioglu, Begum Citamak, Batuhan Bardak, Seren Guldamlasioglu, Azmi Yuksel, and Emin Islam Tatli. Detecting cybersecurity events from noisy short text. pages 1366–1372, 2019

    2019

  53. [61]

    The assessment of twitter’s potential for outbreak detection: Avian influenza case study

    Samira Yousefinaghani, Rozita Dara, Zvonimir Poljak, Theresa Bernardo, and Shayan Sharif. The assessment of twitter’s potential for outbreak detection: Avian influenza case study. Scientific Reports , 9, 2019

    2019

  54. [62]

    iDetector: Automate underground forum analysis based on heterogeneous information network

    Yiming Zhang, Yujie Fan, Shifu Hou, Jian Liu, Yanfang Ye, and Thirimachos Bourlai. iDetector: Automate underground forum analysis based on heterogeneous information network. In 2018 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), ...

    2018

  55. [63]

    Adam Zibak, Clemens Sauerwein, and Andrew C. Simpson. Threat intelligence quality dimensions for research and practice. Digital Threats: Research and Practice , 3(4):1–22, 2022

    2022

  56. [2019]

    Association for Computing Machinery

  57. [2022]

    Accessed: 2025-03-15

    2025

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.