Pith. sign in

REVIEW 2 major objections 1 minor 22 references

TeleHunt: A Framework and Tool for Efficient Cybercriminal Community Discovery on Telegram

T0 review · 2 major / 1 minor · reviewed 2026-06-28 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read TeleHunt evaluates reference-driven snowballing strategies to discover and label cybercriminal communities on Telegram.

desk verdict TeleHunt gives a new labeled Telegram dataset and the first systematic comparison of discovery strategies, but the labels depend on an unvalidated classifier with no reported accuracy or ground-truth details. read the letter →

arxiv 2606.04657 v1 pith:5CHD5ODQ submitted 2026-06-03 cs.CR

Roy Ricaldi , Victor Asanache , Luca Allodi This is my paper · ORCID
classification cs.CR
keywords Telegramcybercrimecommunitydiscoverysnowballingcontentclassificationdarkweblabeleddatasetmarketsegmentation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces TeleHunt as a modular framework that combines reference-driven snowballing from open- and dark-web seeds with message-level classification, contextual filtering, and market-segment labeling. It conducts the first systematic comparison of how seed source, pointer type, and exploration strategy affect discovery in terms of efficiency, accessibility, and rediscovery. The work also releases a labeled dataset of over 172 million messages from 6,022 Telegram communities. A sympathetic reader would care because better discovery methods can improve monitoring of cybercrime on messaging platforms where traditional web crawlers fall short.

What carries the argument

reference-driven snowballing strategies combined with message-level classification, contextual filtering, and market-segment labeling

What would settle it

A manual review of randomly sampled labeled communities showing a high rate of non-criminal groups or the systematic absence of known large cybercriminal communities from the collected set.

Watch

Extended reading notes

Core claim

TeleHunt employs reference-driven snowballing strategies integrating message-level classification, contextual filtering, and market-segment labeling. Using seeds from open- and dark-web sources, the framework systematically evaluates how seed source, pointer type, and exploration strategy influence discovery outcomes across efficiency, accessibility, and rediscovery dimensions, delivering a modular pipeline, the first such comparison with empirical market-segment characterization, and a labeled dataset of over 172 million messages from 6,022 communities.

Load-bearing premise

The chosen seeds and filtering methods accurately identify cybercriminal communities without substantial false positives or selection bias.

Editorial extensions

If this is right

  • Seed source and exploration strategy directly influence the accessibility of different cybercrime market segments.
  • Different discovery approaches produce measurable differences in efficiency and community rediscovery rates.
  • The modular pipeline supports consistent, repeatable evaluation of new Telegram discovery methods.
  • The released dataset enables downstream analysis of labeled cybercriminal content at scale.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same pipeline structure could be tested on other encrypted messaging apps to compare cross-platform discovery performance.
  • Law enforcement agencies might prioritize monitoring based on which seed types yield higher accessibility to specific market segments.
  • The dataset could be used to train improved classifiers that reduce reliance on manual seed curation over time.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 1 minor

Summary. The paper presents TeleHunt, a framework and tool for discovering cybercriminal communities on Telegram via reference-driven snowballing strategies that integrate message-level classification, contextual filtering, and market-segment labeling. It systematically compares discovery strategies (varying seed source, pointer type, and exploration strategy) along efficiency, accessibility, and rediscovery dimensions and releases a labeled dataset of over 172 million messages from 6,022 communities.

Significance. If the labeling pipeline is shown to be reliable, the work would deliver the first systematic empirical comparison of Telegram discovery strategies together with a large, labeled dataset that could support downstream cybercrime research and market-segment analysis.

major comments (2)
  1. [§3–4 (classification and labeling pipeline)] The section describing the message-level classification (and the associated pipeline in §3–4) reports no precision, recall, F1, or other performance metrics, nor any description of how ground-truth labels were obtained or how the classifier was validated. This directly undermines the reliability of the released dataset and the market-segment accessibility claims.
  2. [Evaluation and dataset release sections] No numbers are given for the fraction of communities discarded by contextual filtering or for the false-positive rate in the final labeled set of 6,022 communities. Without these quantities the empirical characterization of market-segment accessibility cannot be assessed.
minor comments (1)
  1. [Abstract] The abstract asserts 'the first systematic comparison' without citing prior Telegram discovery studies to substantiate the novelty claim.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for these focused comments on the validation of the classification and labeling components. We address each point below and will revise the manuscript to incorporate the requested details.

read point-by-point responses

  1. Referee: [§3–4 (classification and labeling pipeline)] The section describing the message-level classification (and the associated pipeline in §3–4) reports no precision, recall, F1, or other performance metrics, nor any description of how ground-truth labels were obtained or how the classifier was validated. This directly undermines the reliability of the released dataset and the market-segment accessibility claims.

    Authors: We agree that the manuscript currently omits performance metrics and validation details for the message-level classifier. In the revised version we will add a dedicated subsection in §3–4 that (i) describes the ground-truth collection process (manual annotation of a stratified sample of messages by multiple annotators with inter-annotator agreement reported), (ii) specifies the classifier architecture and training procedure, and (iii) reports precision, recall, and F1 on held-out test data. These additions will directly support the reliability claims for the released dataset. revision: yes

  2. Referee: [Evaluation and dataset release sections] No numbers are given for the fraction of communities discarded by contextual filtering or for the false-positive rate in the final labeled set of 6,022 communities. Without these quantities the empirical characterization of market-segment accessibility cannot be assessed.

    Authors: We acknowledge that the current text does not quantify the impact of contextual filtering or the false-positive rate in the final set. In the revision we will report (a) the exact fraction of candidate communities removed by each contextual filter and (b) an empirical false-positive estimate obtained by manual review of a random sample of the 6,022 communities. These figures will be added to the evaluation and dataset-release sections to allow readers to assess the market-segment accessibility results. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity in empirical framework and dataset

full rationale

The paper presents TeleHunt as a modular pipeline for Telegram community discovery via reference-driven snowballing, message-level classification, contextual filtering, and market-segment labeling. It reports an empirical comparison of discovery strategies across efficiency, accessibility, and rediscovery dimensions, plus release of a 172M-message labeled dataset from 6,022 communities. No equations, derivations, fitted parameters, or predictions appear; the central claims rest on described empirical methods and external seed sources rather than any self-definitional, fitted-input, or self-citation reduction. The work is therefore self-contained against external benchmarks with no load-bearing circular steps.

Assumptions & free parameters 0 free parameters · 0 assumptions · 0 invented entities

Review based solely on abstract; no free parameters, axioms, or invented entities are specified in the provided text.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TeleHunt: A Framework and Tool for Efficient Cybercriminal Community Discovery on Telegram." pith.science (2026). https://pith.science/paper/5CHD5ODQ

@misc{pith2026260604657,
  author       = {Pith},
  title        = {Pith review of: TeleHunt: A Framework and Tool for Efficient Cybercriminal Community Discovery on Telegram},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5CHD5ODQ}},
  note         = {Machine review of arXiv:2606.04657}
}
read the original abstract

This paper presents TeleHunt, a framework and tool for evaluating the effectiveness of different strategies to discover cybercriminal communities on Telegram. TeleHunt employs a set of reference-driven snowballing strategies, integrating message-level classification, contextual filtering, and market-segment labeling. Using open- and dark-web seeds, we systematically evaluate how seed source, pointer type, and exploration strategy influence discovery outcomes in three dimensions: efficiency, accessibility, and rediscovery. Our work provides (i) a modular cybercrime content discovery pipeline, (ii) the first systematic comparison of Telegram discovery strategies with an empirical characterization of market-segment accessibility, and (iii) a labeled dataset of over 172 million messages from 6,022 Telegram communities.

Figures

Figures reproduced from arXiv: 2606.04657 by the authors.

Figure 1
Figure 1. Methodology overview. [8] examine stolen data markets on Telegram but focus on crime script analy￾sis rather than discovery methodology. Large Language Models have improved automated classification of underground content [13,21], and fine-tuned models achieve high accuracy in market-segment categorization [20]. However, existing work primarily focuses on content classification and ecosystem characterization rather t… view at source ↗
Figure 2
Figure 2. TeleHunt pipeline. Message and Community Classification For each iteration, we scrape all text messages within a 30-day window, together with metadata (author, times￾tamp, replies, forwards). Communities with fewer than 30 messages in this win￾dow are treated as dormant and excluded from further processing. Message-level classification uses two fine-tuned RoBERTa-Large models: a binary cybercriminal classifier and a… view at source ↗
Figure 3
Figure 3. Discovery rate per iteration [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figures from the paper (4 more)
Figure 5
Figure 5. Figure 5: Precision per iteration [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 7
Figure 7. Figure 7: Market segment accessibility characteristics. Shading depicts public, private, and vetted communities from the bottom to the top. 4.4 Rediscovery Extent (RQ3) Rediscovery remains low across all configurations ( [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: O-HLF-C-N access paths across segments [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: Per-visit redundancy across configurations. Accessibility. Not all market segments are equally accessible. Fraud Tools and Cyberattacks dominate discovery flows and function as central transition hubs, consistent with their prominence in prior Telegram cybercrime studi…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

22 extracted references · 11 canonical work pages

  1. [1]

    In: Workshop on the Economics of Information Security (WEIS) (2012), updated versions or follow-up papers exist, but this is a key foundational text

    Anderson, R., Böhme, R., Clayton, R., Moore, T.: Measuring the Cost of Cyber- crime. In: Workshop on the Economics of Information Security (WEIS) (2012), updated versions or follow-up papers exist, but this is a key foundational text

    2012

  2. [2]

    org/ethics/(2015)

    British Society of Criminology: Statement of ethics.https://www.britsoccrim. org/ethics/(2015)

    2015

  3. [3]

    Computers & Security111, 102489 (2021).https://doi.org/https://doi.org/10.1016/j.cose.2021.102489

    Cabrero-Holgueras, J., Pastrana, S.: A methodology for large-scale identification of related accounts in underground forums. Computers & Security111, 102489 (2021).https://doi.org/https://doi.org/10.1016/j.cose.2021.102489

    work page doi:10.1016/j.cose.2021.102489 2021

  4. [4]

    In: Proceedings of the 17th Annual Symposium on Electronic Crime Re- search (APWG eCrime 2022)

    Campobasso, M., Allodi, L.: THREAT/crawl: a Trainable, Highly-Reusable, and Extensible Automated Method and Tool to Crawl Criminal Underground Fo- rums. In: Proceedings of the 17th Annual Symposium on Electronic Crime Re- search (APWG eCrime 2022). pp. 1–11. APWG (2022).https://doi.org/10. 1109/eCrime57793.2022.10142081

    work page arXiv 2022

  5. [5]

    In: 32nd USENIX Security Symposium (USENIX Security 23)

    Campobasso, M., Allodi, L.: Know Your Cybercriminal: Evaluating Attacker Pref- erences by Measuring Profile Sales on an Active, Leading Criminal Market for User Impersonation at Scale. In: 32nd USENIX Security Symposium (USENIX Security 23). pp. 553–570. USENIX Association (2023)

    2023

  6. [6]

    In: Proceedings of the 18th Annual SymposiumonElectronicCrime Research(APWGeCrime2023).pp.1–12.APWG (2023).https://doi.org/10.1109/eCrime59882.2023.00004

    Campobasso, M., Radulescu, R., Brons, S.H., Allodi, L.: You Can Tell a Cyber- criminalbytheCompanytheyKeep:AFrameworktoInfertheRelevanceofUnder- ground Communities to the Threat Landscape. In: Proceedings of the 18th Annual SymposiumonElectronicCrime Research(APWGeCrime2023).pp.1–12.APWG (2023).https://doi.org/10.1109/eCrime59882.2023.00004

    work page doi:10.1109/ecrime59882.2023.00004 2023

  7. [7]

    The British Journal of Crim- inology61(5), 1407–1423 (04 2021).https://doi.org/10.1093/bjc/azab026, https://doi.org/10.1093/bjc/azab026

    Collier, B., Clayton, R., Hutchings, A., Thomas, D.: Cybercrime is (often) boring: Infrastructure and alienation in a deviant subculture. The British Journal of Crim- inology61(5), 1407–1423 (04 2021).https://doi.org/10.1093/bjc/azab026, https://doi.org/10.1093/bjc/azab026

    work page doi:10.1093/bjc/azab026 2021

  8. [8]

    Trends in Organized Crime pp

    Garkava, T., Moneva, A., Leukfeldt, E.: Stolen data markets on telegram: a crime script analysis and situational crime prevention measures. Trends in Organized Crime pp. 1–25 (04 2024).https://doi.org/10.1007/s12117-024-09532-6

    work page doi:10.1007/s12117-024-09532-6 2024

Show all 22 references
  1. [9]

    The Annals of Mathematical Statistics32(1), 148–170 (1961)

    Goodman, L.A.: Snowball sampling. The Annals of Mathematical Statistics32(1), 148–170 (1961)

    1961

  2. [10]

    Hughes, J., Caines, A., Hutchings, A.: Argot as a trust signal: Slang, jargon & reputation on a large cybercrime forum (Nov 2023)

    2023

  3. [11]

    In: 2023 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW)

    Hughes, J., Hutchings, A.: Digital Drift and the Evolution of a Large Cyber- crime Forum. In: 2023 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW). pp. 407–416. IEEE (2023).https://doi.org/10.1109/ EuroSPW59781.2023.00049

    2023

  4. [12]

    ACM Computing Surveys56(6), 1–26 (2024).https://doi.org/10.1145/3638407

    Hughes, J., Pastrana, S., Hutchings, A., Afroz, S., Samtani, S., Li, W.: The Sci- ence/Art of Cybercrime Community Research. ACM Computing Surveys56(6), 1–26 (2024).https://doi.org/10.1145/3638407

    2024 doi

  5. [13]

    Jin, Y., Jang, E., Cui, J., Chung, J.W., Lee, Y., Shin, S.: Darkbert: A language model for the dark side of the internet (May 2023) Efficient Cybercriminal Community Discovery on Telegram 17

    2023

  6. [14]

    In: 46th IEEE Security and Privacy (IEEE SP) (2025)

    Marjanov, T., Hutchings, A.: SoK: Digging into the Digital Underworld of Stolen Data Markets. In: 46th IEEE Security and Privacy (IEEE SP) (2025)

    2025

  7. [15]

    In: 23rd Workshop on the Economics of Information Security (WEIS) (2024)

    Marjanov, T., Ioannidis, K., Hyndman, T., Seyedzadeh, N., Hutchings, A.: Break- ing the Ice: Using Transparency to Overcome the Cold Start Problem in an Un- derground Market. In: 23rd Workshop on the Economics of Information Security (WEIS) (2024)

    2024

  8. [16]

    In: International Symposium onResearchinAttacks,Intrusions,andDefenses(RAID).pp.3–23.Springer,Cham (2018).https://doi.org/10.1007/978-3-030-00470-5_1

    Pastrana, S., Hutchings, A., Caines, A., Buttery, P.: Characterizing Eve: Analysing Cybercrime Actors in a Large Underground Forum. In: International Symposium onResearchinAttacks,Intrusions,andDefenses(RAID).pp.3–23.Springer,Cham (2018).https://doi.org/10.1007/978-3-030-00470-5_1

    2018 doi

  9. [17]

    Ricaldi, R., Allodi, L., Wientjes, J., Radu, A.: Where is dmitry going? framing mi- gratory decisions in the cybercriminal underground, accepted in the 2025 Security Protocols Workshop

    2025

  10. [18]

    In: 2025 APWG Symposium on Elec- tronic Crime Research (eCrime)

    Ricaldi, R., Marjanov, T., Allodi, L., Hutchings, A.: Uncovering the trust signals supporting telegram’s cybercrime economy. In: 2025 APWG Symposium on Elec- tronic Crime Research (eCrime). pp. 1–17. IEEE (2025)

    2025

  11. [19]

    Ricaldi, R., Schafer, M., Zech, P., Allodi, L., Groner, R., Pekaric, I.: Topical shifts in the dark web: A longitudinal analysis of content from the cybercrime ecosystem (2026),https://arxiv.org/abs/2605.15345

    2026 arXiv

  12. [20]

    In: 34th USENIX Security Symposium (USENIX Security 25)

    Roy, S.S., Vafa, E.P., Khanmohamaddi, K., Nilizadeh, S.: Darkgram: A large-scale analysis of cybercriminal activity channels on telegram. In: 34th USENIX Security Symposium (USENIX Security 25). pp. 4839–4858 (2025)

    2025

  13. [21]

    ACM Trans

    Xu, H., Wang, S., Li, N., Wang, K., Zhao, Y., Chen, K., Yu, T., Liu, Y., Wang, H.: Large language models for cyber security: A systematic literature review. ACM Trans. Softw. Eng. Methodol. (Sep 2025).https://doi.org/10.1145/3769676, just Accepted

    2025 doi

  14. [22]

    In: Proceedings of the 5th Annual ACM Web Science Conference

    Yip, M., Shadbolt, N., Webber, C.: Why forums? an empirical analysis into the facilitating factors of carding forums. In: Proceedings of the 5th Annual ACM Web Science Conference. p. 453–462. WebSci ’13, Association for Computing Machinery, New York, NY, USA (2013).https://doi...

    2013 doi

Pith tools

Reviewed June 28, 2026 · model on record in the stance chip above.