Fine-tuned LLMs generate social media text that evades state-of-the-art detectors and human readers, dropping detection accuracy from up to 99.9% to near chance.
MultiSocial: Multilingual Benchmark of Machine-Generated Text Detection of Social-Media Texts
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
Recent LLMs are able to generate high-quality multilingual texts, indistinguishable for humans from authentic human-written ones. Research in machine-generated text detection is however mostly focused on the English language and longer texts, such as news articles, scientific papers or student essays. Social-media texts are usually much shorter and often feature informal language, grammatical errors, or distinct linguistic items (e.g., emoticons, hashtags). There is a gap in studying the ability of existing methods in detection of such texts, reflected also in the lack of existing multilingual benchmark datasets. To fill this gap we propose the first multilingual (22 languages) and multi-platform (5 social media platforms) dataset for benchmarking machine-generated text detection in the social-media domain, called MultiSocial. It contains 472,097 texts, of which about 58k are human-written and approximately the same amount is generated by each of 7 multilingual LLMs. We use this benchmark to compare existing detection methods in zero-shot as well as fine-tuned form. Our results indicate that the fine-tuned detectors have no problem to be trained on social-media texts and that the platform selection for training matters.
fields
cs.CL 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
When Detection Fails: The Power of Fine-Tuned Models to Generate Human-Like Social Media Text
Fine-tuned LLMs generate social media text that evades state-of-the-art detectors and human readers, dropping detection accuracy from up to 99.9% to near chance.