Zero-shot text LLMs detect conspiracy YouTube videos with high recall but low precision, a fine-tuned RoBERTa remains competitive, and thumbnails add little value.
Conspiracy theories and where to find them on TikTok
1 Pith paper cite this work. Polarity classification is still indexing.
abstract
TikTok has skyrocketed in popularity over recent years, especially among younger audiences. However, there are public concerns about the potential of this platform to promote and amplify harmful content. This study presents the first systematic analysis of conspiracy theories on TikTok. By leveraging the official TikTok Research API we collect a longitudinal dataset of 1.5M videos shared in the U.S. over three years. We estimate a lower bound on the prevalence of conspiratorial videos (up to 1000 new videos per month) and evaluate the effects of TikTok's Creativity Program for monetization, observing an overall increase in video duration regardless of content. Lastly, we evaluate the capabilities of state-of-the-art open-weight Large Language Models to identify conspiracy theories from audio transcriptions of videos. While these models achieve high precision in detecting harmful content (up to 96%), their overall performance remains comparable to fine-tuned traditional models such as RoBERTa. Our findings suggest that Large Language Models can serve as an effective tool for supporting content moderation strategies aimed at reducing the spread of harmful content on TikTok.
citation-role summary
citation-polarity summary
fields
cs.CL 1years
2025 1verdicts
CONDITIONAL 1roles
background 1polarities
background 1representative citing papers
citing papers explorer
-
Evaluating AI capabilities in detecting conspiracy theories on YouTube
Zero-shot text LLMs detect conspiracy YouTube videos with high recall but low precision, a fine-tuned RoBERTa remains competitive, and thumbnails add little value.