Pith. sign in

REVIEW 6 major objections 5 minor 1 cited by

Unified Game Moderation: Soft-Prompting and LLM-Assisted Label Transfer for Resource-Efficient Toxicity Detection

T0 review · 6 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A single toxicity-detection model, steered by game-identity tokens, can replace per-game and per-language fleets: soft prompting matches curriculum learning at 43.16 vs 43.35 macro F1, and LLM-assisted label transfer extends coverage to…

desk verdict A useful industrial report with a valuable dataset, but the headline multilingual claims rest on a 450-line, single-annotator human eval and a metric mismatch that the paper itself does not resolve. read the letter →

arxiv 2506.06347 v1 pith:DRJMN32J submitted 2025-06-01 cs.CL cs.AI

Zachary Yang , Domenico Tullo , Reihaneh Rabbany This is my paper
classification cs.CLcs.AI
keywords toxicitydetectionchatmoderationsoft-promptinglabeltransfermulti-lingualgameLLM-assistedannotation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that a single small model can police toxic chat in many games and languages at production scale, replacing the per-game, per-language model fleet that its predecessor ToxBuster required. The two load-bearing tricks are soft-prompting — a GAME_TYPE_TOKEN prepended to each chat line so one shared model knows which game it is moderating — and an LLM-assisted label-transfer pipeline that re-annotates fifteen existing human-labeled datasets into the company's own toxicity taxonomy with GPT-4o-mini. On two Ubisoft games, the soft-prompted model reaches a macro F1 of 43.16%, matching curriculum learning at 43.35% while needing only a single training pass. On real game chat in French, German, Japanese, Portuguese, and Russian, the multi-lingual model scores 19.07–58.88% macro F1, with German beating the English benchmark of 45.39%. If the claims hold, game companies can cut the compute and maintenance overhead of moderation while still catching sanctionable behavior — the paper reports about 50 players flagged per game per day in production.

What carries the argument

The central object is the GAME_TYPE_TOKEN, a learned soft-prompt token prepended before the context section of each chat line, which conditions a shared ToxBuster model on the game being moderated (with GAME_UNKNOWN as a fallback); the paper's ablation shows this placement matters, since putting the token before the current line drops overall F1 from 43.16 to 41.84. The second mechanism is the agreement filter in LLM-assisted label transfer: GPT-4o-mini re-annotates existing human-labeled datasets, and only lines where the LLM and the human binary labels agree are kept, which is what lifts toxic-category F1 from 38.36% to 79.12% on the golden dataset. Together these two mechanisms let one model keep game-specific knowledge and one pipeline repurpose any human-annotated toxicity corpus into a new taxonomy.

What would settle it

Re-run the paper's human-eval at scale: sample at least 2,000 lines of raw chat per language from both games, label them with three independent annotators per language, and recompute per-language macro F1. If German no longer beats the English benchmark, or if any language's score falls outside the reported 19.07–58.88% range, the language-level generalization claim fails; a parallel 20-seed paired comparison of soft prompting versus curriculum learning on the two game datasets would settle whether the F1 match is real or within noise.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central discovery is that two engineering choices make unified, resource-efficient toxicity detection possible. First, soft prompting closes the gap with curriculum learning: prepending game tokens (GAME_1, GAME_2, GAME_UNKNOWN) to the input, before the context section, yields macro F1 of 43.16% across two games against 43.35% for the best curriculum sequence, while training only once on the combined dataset. Second, the LLM-assisted label transfer framework shows that keeping only entries where GPT-4o-mini agrees with the original human binary label raises toxic-category weighted F1 from 38.36% to 79.12% on a golden set — a 40-point gain that makes low-cost, definition-updatable multilingual annotation practical. Using XLM-RoBERTa-base adapted on one week of game chat, the resulting single model retains English performance (42.82/39.09 macro F1 on the two games) while reaching 32.96–58.88% macro F1 on human-evaluated French, Portuguese, Russian, and German chat, with German exceeding the English benchmark. The paper also claims a production result: the unified model identifies an average of 50 players per game per day at Ubisoft engaging in sanctionable behavior.

Load-bearing premise

The multilingual performance numbers rest on the paper's human-eval protocol (Sections 3.2.6 and 4.2.5): only 450 lines per game and language, one annotator per language, sampled to a 50/20/80 target balance, and the paper's own note that the Japanese chat was 'mostly adversarial / translated from another language and did not actually contain normal comprehensible Japanese'; if those samples do not reflect real chat distributions, the language-level F1 rankings and the production figure of 50 players per game per day do not follow.

Editorial extensions

If this is right

  • Adding a new game reduces to introducing one new GAME_TYPE_TOKEN and retraining on the combined dataset, replacing the per-game model training and model selection that curriculum learning requires.
  • The released MLSNT dataset—15 source datasets across seven languages—gives other teams a pre-filtered, taxonomy-aligned multilingual resource without new annotation.
  • Language choice matters more than model choice: German (58.88% macro F1) surpasses the English benchmark (45.39%), while Japanese (19.07%) lags far behind, so English-centric evaluation can misorder production priorities.
  • A single unified model can sustain a moderation pipeline that flags roughly 50 players per game per day in live production, a scale the authors tie to reduced compute and maintenance overhead.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The GAME_TYPE_TOKEN mechanism could plausibly extend to other context variables such as platform, region, or game mode, and the GAME_UNKNOWN token offers a path to unseen games, but the paper evaluates only the two training games, so cross-game transfer to a third title remains an untested projection of the design.
  • The human-LLM agreement filter keeps only clear-cut cases; since 10–70% of lines are discarded per dataset, the released MLSNT is likely biased toward unambiguous examples, which could inflate downstream F1 relative to a model trained on full distributions.
  • Because the multilingual human-eval used a 50/20/80 target balance and a single annotator per language, the reported language-level F1s describe an engineered sample; measuring the same model on raw live traffic per language, where toxic lines are far rarer, is a natural next test.
  • The label-transfer prompt stacks meta prompting, chain of thought, and self-consistency; ablating each component on the golden set would show which one actually drives the 79% toxic-agreement F1, a comparison the paper does not report.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

6 major / 5 minor

Summary. The paper addresses two production challenges at Ubisoft: unifying per-game toxicity detectors into a single model and extending detection to seven non-English languages. The authors evaluate four training schemes (single-game, mixed-dataset, curriculum learning, and soft-prompting with a game token) and report that soft-prompting achieves macro F1 comparable to curriculum learning (43.16 vs 43.35 overall) with a single training pass. For multilingual extension, they propose an LLM-assisted label-transfer framework using GPT-4o-mini to relabel 15 open-source datasets into a new Multi-Lingual Social Network Toxicity (MLSNT) corpus, train XLM-RoBERTa variants, and report a human evaluation on sampled in-game chat (macro F1 19.07%–58.88%), claiming German surpasses the English benchmark of 45.39%. The paper also states the model is deployed at Ubisoft, identifying an average of 50 sanctionable players per game per day.

Significance. The practical problem is important: scaling toxicity detection across games and languages with low computational cost has clear industrial value. The soft-prompting comparison is a useful engineering contribution, and the release of the MLSNT dataset is a concrete resource for future work. If the headline multilingual and production claims were rigorously supported, this would be a compelling systems paper. However, the current evidentiary base for the multilingual results and for the label-transfer filter's '40% improvement' is substantially weaker than the abstract and conclusion suggest. These issues limit the paper's scientific impact until corrected.

major comments (6)
  1. [§4.1, Table 1] The central claim that soft-prompting 'matches' curriculum learning rests on point estimates whose differences are far smaller than the reported standard deviations (43.16±1.06 vs 43.35±1.16, and 43.16 vs 42.48±0.74 for mixed training). No significance or equivalence tests are reported, so the ordering of methods in the text ('second-highest', 'third-highest') is not statistically meaningful. The authors should report confidence intervals for the differences or a non-inferiority test, particularly because the paper's abstract makes a matching claim.
  2. [§4.2.5, Table 8] The headline multilingual result—German (58.88% macro F1) surpassing the English benchmark of 45.39%—is not supported by the evidence as reported. The German score comes from a 450-line, single-annotator human-eval set sampled with a 50/20/80 toxic-to-nontoxic target, while the English 45.39% is a token-level macro F1 on the full GAME_1 test set from Table 7; Table 8 does not state whether the human-eval scores are token-level or line-level, binary or category-level, or whether the same gold-label source and scoring script were used. With only about 50 toxic lines per cell, the exact binomial 95% confidence interval for toxic recall is roughly ±11–14 points, so the 13.5-point German–English gap is within sampling noise. The abstract's inclusive range (19.07%–58.88%) also includes the Japanese GAME_1 score, which §4.2.5 admits is based on chat that was 'mostly adversarial / translated from another language and did not actually contain normal comprehensible Japanese.' The language-level rankings and the 'surpassing English' claim must be either withdrawn or re-supported with an adequate evaluation set, clear protocol reporting, and confidence intervals.
  3. [§3.2.3, Table 4] The '40% increase in performance' from the human–LLM agreement filter is an artifact of selecting on the label being evaluated. Table 4 compares the LLM's class-wise F1 on all its 'Toxic' predictions (38.36%) with the F1 on the subset where the human also agreed (79.12%); the latter is trivially higher because the filter removes most of the LLM's false positives. The no-filter weighted F1 (84.48%) is actually higher than the agreed-toxic F1, and the filter's coverage cost (the fraction of lines discarded, as high as 69.35% in Table 6) must be reported alongside the precision gain. As written, the claim overstates the framework's label-transfer accuracy.
  4. [§3.2.3, Tables 3–4] The prompt-version and temperature choices (v1 vs v2, 0.7 vs 1.0) are selected using the same golden dataset on which feasibility is then reported, with no held-out split. This makes the feasibility numbers optimistic and likely overfit. Please add a description of the development/test split or a cross-validation scheme, and re-report the label-transfer results on a held-out set.
  5. [§4.2.4, Table 7] The statement 'we see that going from English only to multi-lingual does not hurt the performance' is contradicted by the reported GAME_1 macro F1: bert-base scores 45.39±1.01, while xlm-roberta-base and xlm-roberta-base-adapted score 42.78±1.75 and 42.82±1.42. A drop of 2.6 points may or may not be significant, but the equivalence claim requires a statistical test. This is also relevant to the Conclusion's assertion that the multilingual model is built 'without sacrificing its performance on the English dataset.'
  6. [Abstract and §5] The production claim of identifying an average of 50 players per game per day is reported without any precision, recall, or confidence interval, and without any definition of 'sanctionable behavior' or how the deployed threshold was set. This claim should be removed or replaced with an account of the deployed model's operating point and any monitoring of false positives.
minor comments (5)
  1. [§3.1] The GAME_UNKNOWN token is described as 'not trained'; please clarify what the model does with it at inference, since an untrained embedding may produce degenerate behavior.
  2. [Table 7] The model name 'xm-roberta-base-adapted' in Table 7 is a typo; the text elsewhere correctly writes 'xlm-roberta-base-adapted'.
  3. [Figure 2] The example JSON in the prompt has a missing comma and inconsistent spacing in 'overall_category'; this should be fixed for readability.
  4. [§4.2.2, Table 5] The table's abbreviated model names (e.g., '...-multilingual-cased') are hard to follow; please use exact model names to avoid ambiguity.
  5. [Conclusion] The phrase 'it's performance' should be 'its performance'.

Circularity Check

1 steps flagged · score 4.0 of 10

Label-transfer '40% improvement' is a self-selection artifact, but the multilingual and soft-prompting claims rest on independent human-eval and head-to-head comparisons.

  1. self definitional [Section 3.2.3 (filter rule) / Section 4.2.1, Table 4]
    "Retain only the entries where human and LLM binary classifications agree. ... The most significant finding emerges when we filter for cases where both human annotators and GPT-4o-mini classify content as toxic: the performance dramatically improves to 79.12%, representing a remarkable 40% increase in performance."

    The filter is defined by agreement between the LLM and the human gold labels, and the reported post-filter F1 is computed against those same gold labels. On the 'Agreed Toxic' subset, every retained binary label is by definition a true positive under the reference labels, so removing disagreement removes the errors themselves. The jump from 38.36% (all LLM-toxic predictions) to 79.12% (human-LLM agreed toxic) is therefore a restatement of the selection rule rather than evidence that transferred labels generalize. Additionally, the prompt version and temperature were tuned on this same golden dataset (Section 4.2.1) before reporting the filtered numbers, so the feasibility claim is doubly self-referential.

full rationale

The paper's two headline results — soft-prompting matching curriculum learning (43.16 vs 43.35 in Table 1) and multilingual extension with MLSNT — are empirical comparisons or holdout evaluations and are not circular. The soft-prompting numbers come from models trained on combined game datasets and evaluated on held-out game test sets; no fitted parameter is renamed as a prediction. The multilingual model in Table 8 is evaluated on human-annotated real game chat (450 lines per language) that was not used in training, so the German 58.88% figure is external evidence rather than a reconstruction of training labels. The human-eval protocol has substantial limitations (one annotator per language, small toxic sample, and the authors' admission that the Japanese chat was mostly adversarial/translated), but these are measurement-validity concerns, not circularity. The one concrete circular step is in Section 4.2.1: the agreement filter retains exactly the entries where the LLM already matches the golden human labels, and the same golden labels are then used to report the post-filter F1, making the '40% increase' a definitional consequence of removing disagreement cases. Prompt/temperature selection on that same golden set further inflates the reported numbers. This circularity is confined to the filter-quality sub-claim; it does not force the downstream human-eval results, so the overall score is moderate rather than high.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claims rest on the toxicity taxonomy and the three label-transfer assumptions from Section 3.2.3, plus the assumption that SNS-derived labels transfer to game chat. The free parameters are design choices: the agreement filter, prompt configuration, domain-adaptation epochs, sampling protocol, and token placement. No new theoretical entities are introduced.

free parameters (5)
  • GPT-4o-mini prompt version and temperature = v1, temperature 0.7
    Selected in Section 4.2.1, Table 3 to maximize true label count and weighted F1 on the golden dataset; used for all 15 source datasets.
  • Domain adaptation epochs = 74
    Early stopping on MLM loss plateau (max 100); Section 3.2.5.
  • Human-eval sampling targets = 50 per category, 20% spillover, 450 lines per game/language
    Section 3.2.6; hand-chosen balance for the evaluation sets.
  • Agreement filter (retain only human-LLM agreed binary labels) = None (hard filter)
    Defined in Section 3.2.3, step 4; no confidence threshold; discarded 10-69% of lines per dataset.
  • GAME_TYPE_TOKEN placement = before CONTEXT section
    Ablation in Table 2; chosen because before-context placement beats before-current-line placement.
assumptions (5)
  • domain assumption Original labels in existing datasets (both proprietary and open-source) are considered ground truth, having been human-annotated.
    Stated in Section 3.2.3 assumption (1). If source labels are noisy or use different definitions, the entire MLSNT label transfer inherits those errors.
  • domain assumption There is sufficient agreement in binary classification across datasets: content labeled toxic/non-toxic in source datasets aligns with our use case definitions.
    Stated in Section 3.2.3 assumption (2). The framework only transfers binary labels, so cross-dataset definitional drift is not corrected.
  • domain assumption For fine-grained toxic categories, lower performance (accuracy, F1) is acceptable.
    Section 3.2.3 assumption (3); the paper says it accepts lower performance on fine-grained labels, meaning the category taxonomy fidelity is not validated.
  • domain assumption A single human annotator per language provides reliable gold labels for human-eval.
    Section 3.2.6 and 4.2.5; no inter-annotator agreement reported, and Japanese eval was noted as mostly adversarial or translated.
  • domain assumption Toxicity defined on social-network text transfers to gaming chat after domain adaptation.
    Sections 3.2.5 and 4.2.4; the model trains on MLSNT (mostly SNS text) and is evaluated on game chat, with large performance gaps across languages indicating this transfer is not uniform.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Unified Game Moderation: Soft-Prompting and LLM-Assisted Label Transfer for Resource-Efficient Toxicity Detection." pith.science (2026). https://pith.science/paper/DRJMN32J

@misc{pith2026250606347,
  author       = {Pith},
  title        = {Pith review of: Unified Game Moderation: Soft-Prompting and LLM-Assisted Label Transfer for Resource-Efficient Toxicity Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DRJMN32J}},
  note         = {Machine review of arXiv:2506.06347}
}
read the original abstract

Toxicity detection in gaming communities faces significant scaling challenges when expanding across multiple games and languages, particularly in real-time environments where computational efficiency is crucial. We present two key findings to address these challenges while building upon our previous work on ToxBuster, a BERT-based real-time toxicity detection system. First, we introduce a soft-prompting approach that enables a single model to effectively handle multiple games by incorporating game-context tokens, matching the performance of more complex methods like curriculum learning while offering superior scalability. Second, we develop an LLM-assisted label transfer framework using GPT-4o-mini to extend support to seven additional languages. Evaluations on real game chat data across French, German, Portuguese, and Russian achieve macro F1-scores ranging from 32.96% to 58.88%, with particularly strong performance in German, surpassing the English benchmark of 45.39%. In production, this unified approach significantly reduces computational resources and maintenance overhead compared to maintaining separate models for each game and language combination. At Ubisoft, this model successfully identifies an average of 50 players, per game, per day engaging in sanctionable behavior.

Figures

Figures reproduced from arXiv: 2506.06347 by the authors.

Figure 1
Figure 1. LLM-assisted Label Transfer Framework for Cross-dataset Toxic Content Classification. The framework leverages [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. System prompt (v1) used for label generation. [PITH_FULL_IMAGE:figures/full_fig_p011_2.png] view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Prosocial Behavior Detection in Player Game Chat: From Aligning Human-AI Definitions to Efficient Annotation at Scale

    cs.CL 2025-08 conditional novelty 6.0 of 10

    A hybrid system using a calibrated SVM plus selective GPT-4o fallback detects prosocial game chat at roughly 0.90 precision while cutting LLM inference cost by about 70%.

Reference graph

Works this paper leans on

64 extracted references · 31 canonical work pages · cited by 1 Pith paper

  1. [1]

    ADL. 2023. Hate is no game: hate and harassment in online games 2023.Anti- Defamation League, (Feb. 2023). https://www.adl.org/resources/report/hate-no -game-hate-and-harassment-online-games-2023

    work page 2023

  2. [2]

    ADL. 2021. Online hate and harassment: the american experience 2021. (Mar. 2021). https://www.adl.org/online-hate-2021

    work page 2021

  3. [3]

    Akiko Aizawa et al. 2024. Llm-jp: a cross-organizational project for the research and development of fully open japanese llms.CoRR, abs/2407.03963. https://do i.org/10.48550/arXiv.2407.03963

    arXiv doi:10.48550/arxiv.2407.03963 2024

  4. [4]

    Hind Almerekhi, Haewoon Kwak, Joni Salminen, and Bernard J. Jansen. 2022. Provoke: toxicity trigger detection in conversations from the top 100 subreddits. Data and Information Management, 100019. doi:https://doi.org/10.1016/j.dim.20 22.100019

    work page doi:10.1016/j.dim.20 2022

  5. [5]

    Bohdan Andrusyak, Mykhailo Rimel, and Roman Kern. 2018. Detection of abusive speech for mixed sociolects of russian and ukrainian languages. InThe 12th Workshop on Recent Advances in Slavonic Natural Languages Processing, RASLAN 2018, Karlova Studanka, Czech Republic, December 7-9, 2018. Ales Horák, Pavel Rychlý, and Adam Rambousek, (Eds.) Tribun EU, 77–84

    work page 2018

  6. [6]

    Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. 2009. Curriculum learning. InProceedings of the 26th Annual International Confer- ence on Machine Learning(ICML ’09). Association for Computing Machinery, Montreal, Quebec, Canada, 41–48.isbn: 9781605585161. doi:10.1145/1553374.1 553380

    work page doi:10.1145/1553374.1 2009

  7. [7]

    Rishi Bommasani et al. 2022. On the opportunities and risks of foundation models. (2022). https://arxiv.org/abs/2108.07258 arXiv: 2108.07258[cs.LG]

    arXiv 2022

  8. [8]

    Brown et al

    Tom B. Brown et al. 2020. Language models are few-shot learners. (2020). https://arxiv.org/abs/2005.14165 arXiv: 2005.14165[cs.CL]

    arXiv 2020

Show all 64 references
  1. [9]

    Minh Duc Bui, Katharina von der Wense, and Anne Lauscher. 2024. Multi3hate: multimodal, multilingual, and multicultural hate speech detection with vision- language models. (2024). https://arxiv.org/abs/2411.03888 arXiv: 2411.03888 [cs.CL]

    2024 arXiv

  2. [10]

    Mazurek, Katie Shilton, and Hal Daumé Iii

    Yang Trista Cao, Lovely-Frances Domingo, Sarah Gilbert, Michelle L. Mazurek, Katie Shilton, and Hal Daumé Iii. 2024. Toxicity detection is NOT all you need: measuring the gaps to supporting volunteer content moderators through a user-centric method. InProceedings of the 2024 C...

    2024 doi

  3. [11]

    Justin Cheng, Michael Bernstein, Cristian Danescu-Niculescu-Mizil, and Jure Leskovec. 2017. Anyone can become a troll: causes of trolling behavior in online discussions. InProceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing(CSCW ’...

    2017

  4. [12]

    I Chung and Chuan-Jie Lin. 2021. Tocab: a dataset for chinese abusive language processing. In2021 IEEE 22nd International Conference on Information Reuse and Integration for Data Science (IRI). IEEE Press, Las Vegas, NV, USA, 445–452. doi:10.1109/IRI51335.2021.00069

    2021

  5. [13]

    Tuba Ciftci, Liridona Gashi, René Hoffmann, David Bahr, Aylin Ilhan, and Kaja Fietkiewicz. 2017. Hate speech on facebook. In Fourth European Conference on Social Media Research, (Jan. 2017), 425–433

    2017

  6. [14]

    Rogers de Pelle and Viviane Moreira. 2017. Offensive comments in the brazilian web: a dataset and baseline results. InProceedings of the 6th Brazilian Workshop on Social Network Analysis and Mining. SBC, São Paulo, 510–519. doi:10.5753/b rasnam.2017.3260

    2017

  7. [15]

    Jiawen Deng, Jingyan Zhou, Hao Sun, Chujie Zheng, Fei Mi, Helen Meng, and Minlie Huang. 2022. COLD: a benchmark for Chinese offensive language detection. InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. Yoav Goldberg, Zornitsa Kozareva,...

    2022 doi

  8. [16]

    Nicola Döring and M. Mohseni. 2020. Gendered hate speech in youtube and younow comments: results of two content analyses.Studies in Communication and Media, 9, (Mar. 2020), 62–88. doi:10.5771/2192-4007-2020-1-62. KDD ’25, August 3–7, 2025, Toronto, ON, Canada. Yang et al

    2020 doi

  9. [17]

    Abolfazl Farahani, Sahar Voghoei, Khaled Rasheed, and Hamid R. Arabnia. 2020. A brief review of domain adaptation. (2020). https://arxiv.org/abs/2010.03978 arXiv: 2010.03978[cs.LG]

    2020 arXiv

  10. [18]

    Björn Gambäck and Utpal Kumar Sikdar. 2017. Using convolutional neural networks to classify hate-speech. InProceedings of the First Workshop on Abusive Language Online. Zeerak Waseem, Wendy Hui Kyong Chung, Dirk Hovy, and Joel Tetreault, (Eds.) Association for Computational Li...

    2017 doi

  11. [19]

    Janis Goldzycher, Paul Röttger, and Gerold Schneider. 2024. Improving adver- sarial data collection by supporting annotators: lessons from GAHD, a German hate speech dataset. InProceedings of the 2024 Conference of the North American Chapter of the Association for Computationa...

    2024 doi

  12. [20]

    Yukun Huang, Kun Qian, and Zhou Yu. 2023. Learning a better initialization for soft prompts via meta-learning. InProceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Comp...

    2023

  13. [21]

    Manan Jhaveri, Devanshu Ramaiya, and Harveen Singh Chadha. 2022. Toxicity detection for indic multilingual social media content.CoRR, abs/2201.00598. https://arxiv.org/abs/2201.00598 arXiv: 2201.00598

    2022 arXiv

  14. [22]

    Aiqi Jiang, Xiaohan Yang, Yang Liu, and Arkaitz Zubiaga. 2022. Swsr: a chinese dataset and lexicon for online sexism detection.Online Social Networks and Media, 27, 100182. doi:https://doi.org/10.1016/j.osnem.2021.100182

    2022

  15. [23]

    Jared Kaplan et al. 2020. Scaling laws for neural language models. (2020). https: //arxiv.org/abs/2001.08361 arXiv: 2001.08361[cs.LG]

    2020 arXiv

  16. [24]

    Zixuan Ke, Yijia Shao, Haowei Lin, Tatsuya Konishi, Gyuhak Kim, and Bing Liu

  17. [25]

    Hyukhun Koh, Dohyung Kim, Minwoo Lee, and Kyomin Jung. 2024. Can llms recognize toxicity? a structured investigation framework and toxicity metric. (2024). https://arxiv.org/abs/2402.06900 arXiv: 2402.06900[cs.CL]

    2024 arXiv

  18. [26]

    Tran, Yi Tay, Jeffrey Sorensen, Jai Gupta, Donald Metzler, and Lucy Vasserman

    Alyssa Lees, Vinh Q. Tran, Yi Tay, Jeffrey Sorensen, Jai Gupta, Donald Metzler, and Lucy Vasserman. 2022. A new generation of perspective api: efficient mul- tilingual character-level transformers. InProceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data...

    2022

  19. [27]

    João Augusto Leite, Diego Silva, Kalina Bontcheva, and Carolina Scarton. 2020. Toxic language detection in social media for Brazilian Portuguese: new dataset and multilingual analysis. InProceedings of the 1st Conference of the Asia- Pacific Chapter of the Association for Comp...

    2020

  20. [28]

    Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: optimizing continuous prompts for generation. (2021). https : / / arxiv . org / abs / 2101 . 00190 arXiv: 2101.00190[cs.CL]

    2021 arXiv

  21. [29]

    Junyu Lu, Bo Xu, Xiaokun Zhang, Changrong Min, Liang Yang, and Hongfei Lin. 2023. Facilitating fine-grained detection of Chinese toxic language: hi- erarchical taxonomy, resources, and benchmarks. InProceedings of the 61st Annual Meeting of the Association for Computational Li...

    2023 doi

  22. [30]

    Thomas Mandl, Sandip Modha, Prasenjit Majumder, Daksh Patel, Mohana Dave, Chintak Mandlia, and Aditya Patel. 2019. Overview of the hasoc track at fire 2019: hate speech and offensive content identification in indo-european languages. InProceedings of the 11th Annual Meeting of...

    2019

  23. [31]

    Shruthi Mohan, Apala Guha, Michael Harris, Fred Popowich, Ashley Schuster, and Chris Priebe. 2017. The impact of toxic language on the health of reddit communities. In Canadian Conference on Artificial Intelligence, (Apr. 2017), 51–56.isbn: 978-3-319-57350-2. doi:10.1007/978-3...

    2017 doi

  24. [32]

    Nedjma Ousidhoum, Zizheng Lin, Hongming Zhang, Yangqiu Song, and Dit-Yan Yeung. 2019. Multilingual and multi-aspect hate speech analysis. InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natu...

    2019 doi

  25. [33]

    Ronghao Pan, José Antonio García-Díaz, and Rafael Valencia-García. 2024. Com- paring fine-tuning, zero and few-shot strategies with large language models in hate speech detection in english.CMES - Computer Modeling in Engineering and Sciences, 140, 3, 2849–2868. doi:https://do...

    2024

  26. [34]

    Flor Miriam Plaza-del-arco, Debora Nozza, and Dirk Hovy. 2023. Respectful or toxic? using zero-shot learning with language models to detect hate speech. In The 7th Workshop on Online Abuse and Harms (WOAH). Yi-ling Chung, Paul R{\”ottger}, Debora Nozza, Zeerak Talat, and Aida ...

    2023 doi

  27. [35]

    Kamil Saitov and Leon Derczynski. 2021. Abusive language recognition in Russian. InProceedings of the 8th Workshop on Balto-Slavic Natural Language Processing. Bogdan Babych et al., (Eds.) Association for Computational Linguis- tics, Kiyv, Ukraine, (Apr. 2021), 20–25. https://...

    2021

  28. [36]

    Omar Shaikh, Hongxin Zhang, William Held, Michael Bernstein, and Diyi Yang

  29. [37]

    Bruno Silva, Mirian Tavares, Filipa Cerol, Susana Silva, Paulo Alves, and Beatriz Isca. 2020. Playing against hate speech -how teens see hate speech in video games and online gaming communities.Journal of Digital Media and Interaction, 3, (Jan. 2020), 34–52. doi:https://doi.or...

    2020 doi

  30. [38]

    InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

    On second thought, let‘s not think step by step! bias and toxicity in zero- shot reasoning. InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Anna Rogers, Jordan Boyd- Graber, and Naoaki Okazaki, (Eds.) Associati...

    2023 doi

  31. [39]

    Bordini, and Renata Vieira

    Douglas Trajano, Rafael H. Bordini, and Renata Vieira. 2023. Olid-br: offensive language identification dataset for brazilian portuguese.Lang. Resour. Eval., 58, 4, (May 2023), 1263–1289. doi:10.1007/s10579-023-09657-0

    2023 doi

  32. [40]

    Petru Soviany, Radu Tudor Ionescu, Paolo Rota, and Nicu Sebe. 2022. Cur- riculum learning: a survey. (2022). https://arxiv.org/abs/2101.10382 arXiv: 2101.10382[cs.LG]

    2022 arXiv

  33. [41]

    Bertie Vidgen and Leon Derczynski. 2020. Directions in abusive language training data, a systematic review: garbage in, garbage out.PLOS ONE, 15, 12, e0243300. doi:10.1371/journal.pone.0243300

    2020 doi

  34. [42]

    Betty Van Aken, Julian Risch, Ralf Krestel, and Alexander Löser. 2018. Chal- lenges for toxic comment classification: an in-depth error analysis. (2018). doi:10.48550/ARXIV.1809.07572

    2018 doi arXiv

  35. [43]

    Hajime Watanabe, Mondher Bouazizi, and Tomoaki Ohtsuki. 2018. Hate speech on twitter: a pragmatic approach to collect hateful and offensive expressions and perform hate speech detection.IEEE Access, 6, 13825–13835. doi:10.1109 /ACCESS.2018.2806394

    2018

  36. [44]

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. Self-consistency improves chain of thought reasoning in language models. (2023). https://arxiv.org/abs/2 203.11171 arXiv: 2203.11171[cs.CL]

    2023 arXiv

  37. [45]

    Michael Wiegand, Melanie Siegel, and Josef Ruppenhofer. 2018. Overview of the germeval 2018 shared task on the identification of offensive language. In Proceedings of GermEval 2018, 14th Conference on Natural Language Processing (KONVENS 2018). (Sept. 2018)

    2018

  38. [46]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. 2023. Chain-of-thought prompting elicits reasoning in large language models. (2023). https://arxiv.org/abs/2201.1 1903 arXiv: 2201.11903[cs.CL]

    2023 arXiv

  39. [47]

    Zachary Yang, Nicolas Grenon-Godbout, and Reihaneh Rabbany. 2023. Towards detecting contextual real-time toxicity for in-game chat. InFindings of the Association for Computational Linguistics: EM dNLP 2023. Houda Bouamor, Juan Pino, and Kalika Bali, (Eds.) Association for Comp...

    2023 doi

  40. [48]

    Zachary Yang, Nicolas Grenon-Godbout, and Reihaneh Rabbany. 2024. Game on, hate off: a study of toxicity in online multiplayer environments.ACM Games, 2, 2, Article 14, (Aug. 2024), 13 pages. doi:10.1145/3675805

    2024 doi

  41. [49]

    Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Sara Rosenthal, Noura Farra, and Ritesh Kumar. 2019. SemEval-2019 task 6: identifying and catego- rizing offensive language in social media (OffensEval). InProceedings of the 13th International Workshop on Semantic Evaluation. J...

    2019 doi

  42. [50]

    Hao Yu, Zachary Yang, Kellin Pelrine, Jean Francois Godbout, and Reihaneh Rabbany. 2023. Open, closed, or small language models for text classification? (2023). https://arxiv.org/abs/2308.10092 arXiv: 2308.10092[cs.CL]

    2023 arXiv

  43. [51]

    Yifan Zhang, Yang Yuan, and Andrew Chi-Chih Yao. 2024. Meta prompting for ai systems. (2024). https://arxiv.org/abs/2311.11482 arXiv: 2311.11482 [cs.AI]. Unified Game Moderation KDD ’25, August 3–7, 2025, Toronto, ON, Canada

    2024 arXiv

  44. [52]

    Jiang Zhang, Qiong Wu, Yiming Xu, Cheng Cao, Zheng Du, and Konstantinos Psounis. 2023. Efficient toxic content detection by bootstrapping and distilling large language models. (2023). https : / / arxiv . org / abs / 2312 . 08303 arXiv: 2312.08303[cs.CL]

    2023 arXiv

  45. [54]

    Miller, and Cornelia Caragea

    Haoti Zhong, Hao Li, Anna Cinzia Squicciarini, Sarah Michele Rajtmajer, Christopher Griffin, David J. Miller, and Cornelia Caragea. 2016. Content-driven detection of cyberbullying on the instagram social network. InInternational Joint Conference on Artificial Intelligence. A A...

    2016

  46. [55]

    Threats ( Life Threatening )

    " Threats ( Life Threatening ) " - concrete intention to cause harm , injury or other hostile actions to someone or to oneself

  47. [56]

    Minor Endangerment

    " Minor Endangerment " - intention or mentioning of where a child is / will be in a potentially harmful , dangerous , or inappropriate situation ( e . g . grooming , neglect , pedophilia , child pornography , threats to a child , a child's personally identifiable information )

  48. [57]

    Threats ( Non - Life Threatening )

    " Threats ( Non - Life Threatening ) " - provocative statement that would cause a disruption in another player's life or game experience , without a clear intent to cause real - life injury or damage

  49. [58]

    " Hate " - verbal abuse ( e . g . intimidation , ridicule , derogatory or insulting remarks ) based on another person's or group of people's actual or perceived identity ( e . g . , race , religion , color , sex , gender identity , national origin , age , disability , sexual o...

  50. [59]

    Sexual Content / Harassment

    " Sexual Content / Harassment " - any text that contains , mentions or coerces other players with implicit / explicit sexual language , overtones , behavior or favors ( e . g . euphemisms , flirting , innuendos , unwanted sexual advances , sexual requests )

  51. [60]

    Extremism

    " Extremism " - political term , describes activities that are not in accordance with norms of the state ; those who are fully intolerant toward others and reject the way of problem solving and existing social order

  52. [61]

    Insults

    " Insults " - intent to damage the dignity of another person ( or group of people ) , undermining their reputation or undermining their self - esteem

  53. [62]

    Controversial / Potentially Toxic Topic

    " Controversial / Potentially Toxic Topic " - Any text that contain any of the following subtopic ( s ) : a . " Abortion ": any mentions pro or against the termination of pregnancy b . " Religion ": any text relating to / believing in a religion c . " Politics ": any text rela...

  54. [63]

    o verall_c ategory

    {" o verall_c ategory ": " toxic " " spans ": [ {" text ": " retard " , " category ": [" Insults "]} ] }

  55. [64]

    o verall_c ategory

    {" o verall_c ategory ": " non - toxic "} Figure 2: System prompt (v1) used for label generation

  56. [2023]

    Continual pre-training of language models. (2023). https://arxiv.org/abs /2302.03241 arXiv: 2302.03241[cs.CL]

    2023 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.