The reviewed record of science sign in
Pith

arxiv: 2605.24000 · v1 · pith:TOCT7ILW · submitted 2026-05-18 · cs.CL

Toxicity in Twitch Chats: An LLM-Based Analysis Across Gaming Communities

Ronja Fuchs , Florian Rupp , Timo Bertram , Kai Eckert , Alexander Dockhorn This is my paper

Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel 2026-06-30 18:24 UTCgrok-4.3pith:TOCT7ILWrecord.jsonopen to challenge →

classification cs.CL
keywords toxicity analysisTwitch chatgaming communitiesLLM classificationMOBA gamessports gameschat moderationgenre differences
0
0 comments X

The pith

Twitch chat toxicity reaches 3.2 percent in MOBA game streams but only 2 percent in sports game streams.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper measures how often toxic messages appear in Twitch chats across seven game genres by processing roughly 20 million messages from 4,452 streams. It applies a pre-trained language model to label each message according to Twitch's own four-category toxicity taxonomy. Overall, 2.4 percent of messages are labeled toxic, yet the rate changes markedly by genre and even by individual game within a genre. These patterns suggest that specific game rules and player communities create distinct norms around acceptable chat behavior. The results point toward moderation approaches that can be tuned to particular games rather than applied uniformly.

Core claim

Analysis of 20 million Twitch chat messages shows that 2.4 percent are toxic, with MOBA streams at the high end of 3.2 percent and sports streams at the low end of 2 percent; individual games within the same genre also differ significantly, indicating game-specific community norms and mechanics shape toxic behavior beyond genre effects.

What carries the argument

Zero-shot classification by a pre-trained LLM that assigns each chat message to one of Twitch's four toxicity categories and eight subclasses on a sample of 4,452 streams.

If this is right

  • Moderation tools can be adjusted to focus more resources on MOBA streams than on sports streams.
  • Game designers may examine which in-game mechanics correlate with higher chat toxicity.
  • Community guidelines could be written at the level of individual titles rather than whole genres.
  • Platform-wide toxicity statistics may mask large differences between games.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same genre differences might appear in chat on other streaming or social platforms that host game communities.
  • If game mechanics drive toxicity, changing rules inside a title could reduce toxic chat without changing the player base.
  • Long-term tracking of the same streams could reveal whether toxicity rates shift when a game receives updates or new players arrive.

Load-bearing premise

The language model applies Twitch's toxicity rules to gaming chat messages in the same way a human moderator would.

What would settle it

A large set of Twitch chat messages labeled by multiple human raters using the same taxonomy would show whether the model's 2.4 percent overall rate and the genre differences match human judgments.

Figures

Figures reproduced from arXiv: 2605.24000 by Alexander Dockhorn, Florian Rupp, Kai Eckert, Ronja Fuchs, Timo Bertram.

Figure 1
Figure 1. Figure 1: Our pipeline for toxicity detection follows four sequential steps: data [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Distribution of toxicity subclasses per game. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
read the original abstract

Toxicity in online gaming communities remains a persistent challenge, manifesting across genres, platforms, and player interactions. While much research is focused on in-game toxicity, less is known about how toxic behavior varies between gaming communities on streaming platforms. To address this shortcoming, we analyze approximately 20 million chat messages from 4,452 streams, spanning seven game genres on Twitch. We categorize messages according to Twitch's toxicity taxonomy with a pre-trained Large Language Model using zero-shot classification. The taxonomy comprises four categories and eight subclasses, including harassment, discrimination, sexual content, and profanity. Our approach achieves an F1 score of 94.5% on the TextDetox dataset and demonstrates human-model agreement comparable to inter-human agreement. Our analysis reveals that 2.4% of all messages are classified as toxic, with notable differences across genres: streams of MOBA games exhibit the highest relative rate of toxicity (3.2%), and sports games show the lowest rate (2%). Furthermore, results indicate that individual games differ significantly in their toxicity distributions, even within genres, suggesting the existence of game-specific community norms and mechanics that shape toxic behavior beyond genre-level effects. These findings offer empirical insights into genre- and game-specific toxicity patterns on Twitch and can inform more targeted moderation strategies for gaming communities.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper analyzes approximately 20 million Twitch chat messages from 4,452 streams across seven game genres using zero-shot classification by a pre-trained LLM according to Twitch's toxicity taxonomy (four categories, eight subclasses including harassment, discrimination, sexual content, and profanity). It reports an overall toxicity rate of 2.4%, with MOBA streams highest at 3.2% and sports lowest at 2%, plus significant within-genre differences across individual games that the authors attribute to game-specific community norms and mechanics. Validation is reported as F1=94.5% on TextDetox and human-model agreement comparable to inter-human agreement.

Significance. If the zero-shot labels prove reliable for informal, slang-heavy Twitch chats, the scale of the dataset and the genre/game breakdowns would provide useful empirical evidence on toxicity variation in streaming communities, potentially supporting more targeted moderation. The use of an established platform taxonomy and the focus on chat (rather than in-game) toxicity are strengths for computational social science.

major comments (2)
  1. [Abstract / Methods] Abstract and Methods: The classifier is validated solely via F1=94.5% on TextDetox and a human agreement check; no domain-matched Twitch gaming data (with emotes, abbreviations, or trash-talk norms) is used for validation. If error rates differ systematically by genre or game, the headline differences (MOBA 3.2% vs. sports 2%) and within-genre claims cannot be guaranteed to reflect actual toxicity patterns.
  2. [Results] Results: The representativeness of the 4,452 streams and the statistical tests supporting 'significant' differences across games within genres are not described; without stratification details or effect sizes, it is unclear whether the observed distributions are robust or driven by sampling imbalance.
minor comments (2)
  1. [Methods] Specify the exact pre-trained LLM, prompt template, and any post-processing rules used for zero-shot classification.
  2. [Results] Provide per-genre message counts and per-game sample sizes to allow readers to assess the reliability of the reported percentages.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their detailed and constructive comments on our manuscript. We address each of the major comments below and indicate the revisions we plan to make.

read point-by-point responses

  1. Referee: [Abstract / Methods] Abstract and Methods: The classifier is validated solely via F1=94.5% on TextDetox and a human agreement check; no domain-matched Twitch gaming data (with emotes, abbreviations, or trash-talk norms) is used for validation. If error rates differ systematically by genre or game, the headline differences (MOBA 3.2% vs. sports 2%) and within-genre claims cannot be guaranteed to reflect actual toxicity patterns.

    Authors: We acknowledge the validity of this concern. The validation reported in the manuscript is based on the TextDetox dataset and a human agreement check, without explicit domain-matched validation on Twitch chat data featuring emotes and gaming-specific language. We will revise the Methods section to more clearly describe the human agreement evaluation and add a limitations paragraph in the Discussion acknowledging that differential error rates across genres could affect the reported differences. We will also note this as an area for future work. revision: yes

  2. Referee: [Results] Results: The representativeness of the 4,452 streams and the statistical tests supporting 'significant' differences across games within genres are not described; without stratification details or effect sizes, it is unclear whether the observed distributions are robust or driven by sampling imbalance.

    Authors: We agree that providing more details on the sampling and statistical analysis would enhance the robustness of our findings. The streams were sampled from popular Twitch categories to ensure coverage across genres, but we will add a subsection in the Methods or Results describing the selection criteria, any stratification used, the statistical tests (including p-values and effect sizes) used to determine significance of differences within genres. This will allow readers to assess potential sampling effects. revision: yes

Circularity Check

0 steps flagged

No circularity: purely empirical measurement with external validation

full rationale

The paper collects ~20M Twitch messages from 4,452 streams and applies zero-shot LLM classification using a pre-trained model. Reported toxicity rates (2.4% overall, 3.2% MOBA, 2% sports) and game-specific differences are direct empirical outputs. Validation uses the external TextDetox dataset (F1=94.5%) with no fitted parameters, self-referential predictions, or load-bearing self-citations. No equations, ansatzes, or derivations exist that could reduce to inputs by construction. This is a standard observational study whose claims rest on data and external benchmark performance.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

Based solely on abstract; full paper may detail additional modeling choices.

axioms (2)
  • domain assumption Twitch's toxicity taxonomy (four categories, eight subclasses) is a suitable and complete framework for classifying chat messages.
    Directly used as the classification target for the LLM.
  • domain assumption The TextDetox dataset serves as a valid external benchmark for evaluating the LLM's toxicity classification performance.
    Cited to support the reported F1 score of 94.5%.

pith-pipeline@v0.9.1-grok · 5768 in / 1588 out tokens · 37429 ms · 2026-06-30T18:24:39.473337+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

31 extracted references · 31 canonical work pages

  1. [1]

    Phi-4 Technical Report,

    M. Abdin, J. Aneja, H. Behl, S. Bubeck, R. Eldan, and S. Gunasekar et al., “Phi-4 Technical Report,” Dec. 2024, arXiv.org

    work page 2024

  2. [2]

    The enemy hates best? toxicity in league of legends and its content moderation implications,

    J. C. Aguerri, M. Santisteban, and F. Miró-Llinares, “The enemy hates best? toxicity in league of legends and its content moderation implications,”European Journal on Criminal Policy and Research, vol. 29, no. 3, pp. 437–456, 2023

    work page 2023

  3. [3]

    Hate speech detection using large language models: A comprehensive review,

    A. Albladi, M. Islam, A. Das, M. Bigonah, Z. Zhang, F. Jamshidi, M. Rahgouy, N. Raychawdhary, D. Marghitu, and C. Seals, “Hate speech detection using large language models: A comprehensive review,”IEEE Access, vol. 13, pp. 20 871–20 892, 2025

    work page 2025

  4. [4]

    Distance-based tests for homogeneity of multivariate dispersions,

    M. J. Anderson, “Distance-based tests for homogeneity of multivariate dispersions,”Biometrics, vol. 62, no. 1, pp. 245–253, 2006

    work page 2006

  5. [5]

    Permutational multivariate analysis of variance (permanova),

    ——, “Permutational multivariate analysis of variance (permanova),” Wiley statsref: statistics reference online, pp. 1–15, 2014

    work page 2014

  6. [6]

    Content moderation in online platforms: A study of annotation methods for inappropriate language,

    B. Barbarestani, I. Maks, and P. T. V ossen, “Content moderation in online platforms: A study of annotation methods for inappropriate language,” inProceedings of the Fourth Workshop on Threat, Aggression & Cyberbullying @ LREC-COLING-2024. ELRA and ICCL, May 2024, pp. 96–104

    work page 2024

  7. [7]

    A coefficient of agreement for nominal scales,

    J. Cohen, “A coefficient of agreement for nominal scales,”Educational and Psychological Measurement, vol. 20, no. 1, pp. 37–46, 1960

    work page 1960

  8. [8]

    Overview of the multilingual text detoxification task at pan 2024

    D. Dementieva, D. Moskovskiy, N. Babakov, A. A. Ayele, N. Rizwan, F. Schneider, X. Wang, S. M. Yimam, D. Ustalov, E. Stakovskiiet al., “Overview of the multilingual text detoxification task at pan 2024.” in CLEF (Working Notes), 2024, pp. 2432–2461

    work page 2024

  9. [9]

    Overview of the multilingual text detoxification task at pan 2025,

    D. Dementieva, V . Protasov, N. Babakov, N. Rizwan, I. Alimova, and C. Brune et al., “Overview of the multilingual text detoxification task at pan 2025,” inWorking Notes of CLEF 2025 – Conference and Labs of the Evaluation Forum, ser. CEUR Workshop Proceedings, Sep. 2025

    work page 2025

  10. [10]

    Toxicity in twitch live stream chats: Towards understanding the impact of gender, size of community and game genre,

    L. Dreier and J. Pirker, “Toxicity in twitch live stream chats: Towards understanding the impact of gender, size of community and game genre,” in2023 IEEE Conference on Games (CoG). IEEE, 2023, pp. 1–4

    work page 2023

  11. [11]

    Women’s experiences with general and sexual harassment in online video games: Rumination, organizational responsiveness, withdrawal, and coping strategies,

    J. Fox and W. Y . Tang, “Women’s experiences with general and sexual harassment in online video games: Rumination, organizational responsiveness, withdrawal, and coping strategies,”New media & society, vol. 19, no. 8, pp. 1290–1307, 2017

    work page 2017

  12. [12]

    How perceived toxicity of gaming communities is associated with social capital, satisfaction of relatedness, and loneliness,

    J. Frommel, D. Johnson, and R. L. Mandryk, “How perceived toxicity of gaming communities is associated with social capital, satisfaction of relatedness, and loneliness,”Computers in Human Behavior Reports, vol. 10, 2023

    work page 2023

  13. [13]

    Challenges to combating toxicity and harassment in multiplayer games: Involving the hci games research community,

    J. Frommel, R. L. Mandryk, and M. Klarkowski, “Challenges to combating toxicity and harassment in multiplayer games: Involving the hci games research community,” inExtended Abstracts of the 2022 Annual Symposium on Computer-Human Interaction in Play, 2022

    work page 2022

  14. [14]

    Supplementary Material,

    Fuchs, Rupp, Bertram, Eckert, and Dockhorn, “Supplementary Material,” 2026, available at: https://github.com/ronjafuchs/twitch_toxicity_analysis

    work page 2026

  15. [15]

    How do players perceive gender discrimination? on the differences of harassment in online games,

    R. Fuchs, J. Droste, and A. Dockhorn, “How do players perceive gender discrimination? on the differences of harassment in online games,” in 2025 IEEE Conference on Games (CoG). IEEE, 2025, pp. 1–8

    work page 2025

  16. [16]

    Sharing dark sides on game service platforms: Disruptive behaviors and toxicity in dota2 through a platform lens,

    E. Gandolfi and R. E. Ferdig, “Sharing dark sides on game service platforms: Disruptive behaviors and toxicity in dota2 through a platform lens,”Convergence, vol. 28, no. 2, pp. 468–487, 2022

    work page 2022

  17. [17]

    The Llama 3 Herd of Models,

    A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey, A. Kadian, and A. Al- Dahle et al., “The Llama 3 Herd of Models,” Jul. 2024, arXiv.org

    work page 2024

  18. [18]

    Zero-shot Topical Text Classification with LLMs - an Experimental Study,

    S. Gretz, A. Halfon, I. Shnayderman, O. Toledo-Ronen, A. Spector, and L. Dankin et al., “Zero-shot Topical Text Classification with LLMs - an Experimental Study,” inFindings of the Association for Computational Linguistics: EMNLP 2023. Association for Computational Linguistics, 2023, pp. 9647–9676

    work page 2023

  19. [19]

    Exploring the potential of an ai-based twitch moderation and toxicity detection bot,

    J. Huth, C. Eichhorn, D. A. Plecher, and J. Pirker, “Exploring the potential of an ai-based twitch moderation and toxicity detection bot,” in2025 IEEE Conference on Games (CoG). IEEE, 2025, pp. 1–4

    work page 2025

  20. [20]

    Understanding and identifying the use of emotes in toxic chat on twitch,

    J. Kim, D. Y . Wohn, and M. Cha, “Understanding and identifying the use of emotes in toxic chat on twitch,”Online Social Networks and Media, vol. 27, p. 100180, 2022

    work page 2022

  21. [21]

    Can llms recognize toxicity? a structured investigation framework and toxicity metric,

    H. Koh, D. Kim, M. Lee, and K. Jung, “Can llms recognize toxicity? a structured investigation framework and toxicity metric,” inFindings of the Association for Computational Linguistics: EMNLP 2024, 2024, pp. 6092–6114

    work page 2024

  22. [22]

    Llm-based synthetic datasets: Applications and limitations in toxicity detection,

    U. Kruschwitz and M. Schmidhuber, “Llm-based synthetic datasets: Applications and limitations in toxicity detection,” inProceedings of the Fourth Workshop on Threat, Aggression & Cyberbullying@ LREC- COLING-2024, 2024, pp. 37–51

    work page 2024

  23. [23]

    The measurement of observer agreement for categorical data,

    J. R. Landis and G. G. Koch, “The measurement of observer agreement for categorical data,”Biometrics, vol. 33, no. 1, pp. 159–174, 1977

    work page 1977

  24. [24]

    Gaming for equity: The power of diversity within gender and race in gamers,

    L. E. Ma, C. Dickson-Deane, W. Raffe, A. R. Murphy, and J. Garcia, “Gaming for equity: The power of diversity within gender and race in gamers,” in2024 IEEE Conference on Games (CoG). IEEE, 2024

    work page 2024

  25. [25]

    Toxicity detection in multiplayer online games,

    M. Märtens, S. Shen, A. Iosup, and F. Kuipers, “Toxicity detection in multiplayer online games,” in2015 International Workshop on Network and Systems Support for Games (NetGames). IEEE, 2015, pp. 1–6

    work page 2015

  26. [26]

    The unreasonable effectiveness of large language models in zero-shot semantic annotation of legal texts,

    J. Savelka and K. D. Ashley, “The unreasonable effectiveness of large language models in zero-shot semantic annotation of legal texts,” Frontiers in Artificial Intelligence, vol. 6, Nov. 2023

    work page 2023

  27. [27]

    Help, my game is toxic! first insights from a systematic literature review on intervention systems for toxic behaviors in online video games,

    M. Wijkstra, K. Rogers, R. L. Mandryk, R. C. Veltkamp, and J. Frommel, “Help, my game is toxic! first insights from a systematic literature review on intervention systems for toxic behaviors in online video games,” in Companion Proceedings of the Annual Symposium on Computer-Human Interaction in Play, 2023, pp. 3–9

    work page 2023

  28. [28]

    Wohlinet al.,Experimentation in software engineering

    C. Wohlinet al.,Experimentation in software engineering. Springer, 2012, vol. 236

    work page 2012

  29. [29]

    Unified game moderation: Soft- prompting and llm-assisted label transfer for resource-efficient toxicity detection,

    Z. Yang, D. Tullo, and R. Rabbany, “Unified game moderation: Soft- prompting and llm-assisted label transfer for resource-efficient toxicity detection,” inProceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V . 2, 2025, pp. 5161–5170

    work page 2025

  30. [30]

    Efficient toxic content detection by bootstrapping and distilling large language models,

    J. Zhang, Q. Wu, Y . Xu, C. Cao, Z. Du, and K. Psounis, “Efficient toxic content detection by bootstrapping and distilling large language models,” inProceedings of the AAAI conference on artificial intelligence, vol. 38, no. 19, 2024, pp. 21 779–21 787

    work page 2024

  31. [31]

    Enhancing llm-based hatred and toxicity detection with meta-toxic knowledge graph,

    Y . Zhao, J. Zhu, C. Xu, Y . Liu, and X. Li, “Enhancing llm-based hatred and toxicity detection with meta-toxic knowledge graph,” inFindings of the Association for Computational Linguistics: ACL 2025, 2025, pp. 24 747–24 760

    work page 2025