Pith. sign in

REVIEW 3 major objections 7 minor 37 references

Silencing Empowerment, Allowing Bigotry: Auditing the Moderation of Hate Speech on Twitch

T0 review · 3 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Twitch's automated moderator AutoMod catches only 22 percent of hateful comments overall and 6 to 7 percent of implicit hate, while blocking most benign uses of sensitive terms.

desk verdict First audit of Twitch AutoMod: credible finding of low hate-speech recall and slur dependence, but the headline percentages rest on unvalidated label alignment with Twitch's policy. read the letter →

arxiv 2506.07667 v2 pith:BWY2Z64X submitted 2025-06-09 cs.CL cs.HCcs.LG

Prarabdh Shukla , Wei Yin Chong , Yash Patel , Brennan Schaffner , Danish Pruthi , Arjun Bhagoji This is my paper
classification cs.CLcs.HCcs.LG
keywords contentmoderationaudithatespeechdetectionTwitchAutoModimplicitalgorithmauditingcounterfactualanalysisbiaslivechat
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper reports a large-scale black-box audit of Twitch's automated moderation tool, AutoMod, conducted by sending more than 107,000 comments from four hate-speech datasets into siloed live chats. The central finding is that AutoMod, even at its strictest setting, flags only 22% of hateful comments overall and only 6 to 7% on datasets built to capture implicit hate, so most hateful content bypasses moderation. The audit also finds that swapping a demographic term for a slur triggers 100% removal, while benign pedagogical or empowering sentences containing sensitive words are blocked up to 98.5% of the time. The paper argues these patterns show AutoMod depends on slurs and profanity rather than context, and that this gap matters because Twitch's own guidelines promise context-sensitive moderation.

What carries the argument

The argument is carried by a three-bot audit pipeline running inside siloed Twitch streams: a messenger bot sends dataset comments within chat rate limits, a receiver bot logs every non-moderated message, and a Pubsub bot records AutoMod's moderation queue events together with Twitch's internal category labels such as Ableism, Misogyny, Racism, and Homophobia. A third outcome, labeled 'pre-filtered,' covers messages visible to neither bot and is attributed to a service-level blocklist that runs before AutoMod. The pipeline is used to measure filter-wise recall and precision for four AutoMod filters by toggling them individually, and to run controlled counterfactual, policy-adherence, and perturbation case studies.

What would settle it

Re-run the audit on fresh accounts while confirming every sent message appears in at least one independent IRC log; if many 'pre-filtered' messages turn out to be undelivered, or if moderation rates change materially, the reported recall figures are not stable measurements of AutoMod.

Watch

Extended reading notes

Core claim

The central claim is that AutoMod's moderation behavior is both under- and over-inclusive in a way that tracks surface offensive words, not hateful intent. On the four datasets, recall at maximum filtering is 22% overall, with race, ethnicity, and religion hate caught only 12.3% of the time and hate targeting mentally disabled people escaping up to 98% of the time on some datasets; on ToxiGen and the Implicit Hate Corpus, implicit-hate recall is 6 to 7%. In counterfactual testing, 110 previously unflagged hateful SBIC messages become 100% moderated once demographic terms are replaced with slurs, and in robustness testing, simple spelling or spacing changes drop moderation of sensitive fragments from 100% to as low as 4%. At the same time, 89.5% at alpha=2 and 98.5% at alpha=4 of benign model-generated sentences that use sensitive words in pedagogical or empowering contexts are blocked, contrary to Twitch's community guidelines.

Load-bearing premise

The recall and precision measurements assume the four datasets' human labels and the chosen offensiveness thresholds are the right definition of hateful content for Twitch's policy, and that messages missing from both observation bots were actually pre-filtered by Twitch rather than lost to a connection glitch.

Editorial extensions

If this is right

  • If these measurements hold, Twitch's most stringent AutoMod setting is not a safety guarantee: most explicitly hateful comments and nearly all implicit hate pass through into chat.
  • The counterfactual result implies AutoMod's moderation decisions are largely driven by the presence of slurs, so hate that avoids slurs is unlikely to be caught.
  • Benign speech about marginalized groups, including education and empowerment, faces a high risk of being held or blocked, imposing costs on the very communities the moderation policy is meant to protect.
  • Because switching filter levels from alpha=2 to alpha=4 changes recall by only about 1% on the Implicit Hate Corpus, turning the strictness dials up does little to close the gap on implicit hate.
  • The reported pre-filtering rates, up to 54.8% for SSG-related examples, mean a large share of catches occur before AutoMod, so the channel-level tool's own performance is even weaker than the headline recall suggests.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension is to run the same counterfactual and perturbation battery on other platforms' moderation systems; the paper's results predict a similar slur-dependence, consistent with the concurrent findings it cites for commercial moderation APIs.
  • The blocklist-based 'pre-filtering' pattern implies a disparate-impact risk: groups whose slurs sit on the blocklist receive disproportionate protection, while groups whose hate is expressed implicitly are left exposed; balanced cross-community datasets could test this directly.
  • A practical, testable consequence is that attackers can systematically collect AutoMod's false negatives and apply small semantic-preserving edits to keep hate below the moderation threshold, since the robustness case study already demonstrates the mechanism at small scale.
  • If Twitch moved toward a context-aware classifier, one would expect recall on ToxiGen and the Implicit Hate Corpus to rise toward the 70 to 85% levels that modern language models reach on the same inputs; this is a direct, falsifiable prediction from the paper's comparison.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper presents a large-scale algorithmic audit of Twitch's AutoMod hate-speech moderation tool. The authors created siloed test streams and, using three bots per pipeline, sent over 100,000 (they later state around 300,000) messages drawn from four hate-speech datasets (SBIC, DynaHate, ToxiGen, IHC) to measure AutoMod's moderation decisions. They report that even at the maximum filtering level AutoMod flags only 22% of hateful content overall, with particularly low recall on implicit-hate datasets (6-7% on ToxiGen and IHC). Case studies show that replacing demographic terms with explicit slurs leads to 100% removal, that semantic-preserving perturbations sharply reduce moderation rates (from 100% to 4% for some obfuscations), and that 89.5%-98.5% of benign pedagogical or empowering uses of sensitive terms are blocked at alpha=2 and alpha=4, respectively. The paper also analyzes filter-wise recall, filter precision, target-group differences, and the contribution of an inferred 'pre-filtering' stage.

Significance. If the findings hold, this is a valuable third-party audit of a widely deployed commercial moderation system, one that is rarely audited in the algorithmic-accountability literature. The paper's strengths include a controlled, siloed experimental design that limits harm from posting hateful content, a large message corpus, a reproducible pipeline with code and data released, and careful reporting of case studies that connect the quantitative results to plausible underlying mechanisms. The observation that AutoMod relies on explicit slurs rather than context aligns with concurrent work on commercial moderation APIs (Hartmann et al., 2025), lending external consistency. The main limitation is that the headline recall figures treat dataset offensiveness/toxicity labels as ground truth for Twitch's actual hate-speech policy, which is a real but fixable gap in the central measurement.

major comments (3)
  1. [§3.2, §B.2, Table 1] The headline recall numbers (22% overall; 19% on SBIC; 6-7% on ToxiGen/IHC) are computed using dataset labels as ground truth for Twitch's 'Discrimination & Slurs' policy. SBIC is an offensiveness corpus, and the chosen threshold of 1.0 (and 0.5 for filter subsets) selects examples that all annotators found offensive, not necessarily examples that violate the policy quoted in §1. ToxiGen positives are selected using RoBERTa toxicity scores, not Twitch's policy. No validation is provided that these labels align with the policy-relevant concept of hateful conduct toward a protected group. If a substantial fraction of dataset positives are offensive but not policy-relevant hate, the reported recall measures agreement with offensiveness labels rather than hate-speech recall. This is load-bearing because the abstract and introduction state the conclusion in policy terms. The authors should either (i) manually annotate a random sample of positives against Twitch's policy to demonstrate high agreement, or (ii) consistently re-frame all recall figures as 'recall with respect to dataset offensiveness/toxicity labels' and temper the policy-level conclusions in the title, abstract, and introduction.
  2. [§4.1, §C.4, Table 2] The 'pre-filtered' category is inferred from messages that appear in neither the receiver bot's IRC stream nor the Pubsub moderation queue. The authors note that they 'suspect' and 'speculate' this is service-level moderation, and they manually verified only 'some' such messages. Messages lost to transient connection failures or rate-limit drops would be misclassified as pre-filtered, inflating recall (because these messages are counted as moderated) and distorting the filter-wise analysis in Table 2 and Figure 3, where pre-filtering accounts for 54.8% of SSG-related moderation and large shares of other categories. The paper needs a direct control: e.g., send a set of known-benign messages through the same pipeline and measure the disappearance rate, or re-send unobserved messages to see whether they appear on retry. Without such a check, the pre-filtering mechanism itself is confounded with message loss.
  3. [§5, Tables 6-7] The three case studies rely on small, manually selected samples: 110 counterfactual false negatives, 20 sensitive fragments for the policy-adherence study, and 50 fragments for perturbation robustness. The paper reports point estimates (100% counterfactual recall, 89.5%/98.5% blocking of pedagogical/empowering examples, and drops to 4% for partial obfuscation) without confidence intervals, selection protocols, or the total number of generated examples actually tested. A handful of selection or labeling choices could materially change these percentages. The authors should report full sample sizes after generation, the criteria for manual selection, and bootstrap confidence intervals for the moderation rates. Without these, the case-study conclusions are indicative but not quantitatively robust; as written they are used as direct evidence for the paper's central claim that AutoMod lacks contextual understanding.
minor comments (7)
  1. [Abstract vs. §4.1] The abstract states 'over 107,000 comments' while §4.1 says 'around 300,000 messages' for the experiments; the total number of unique messages sent versus the number of moderation decisions logged should be reconciled, and the discrepancy explained.
  2. [Figure 1] The figure labels a send rate of 0.43 msg/s, but §C.1 describes sending 5 messages with a 4-second wait between each message plus a 3.5-second pause, which corresponds to roughly 0.26 msg/s; the figure and text should be made consistent.
  3. [Table 1] The table uses two F1 definitions ('P,R F1' and 'TPR,TNR F1'); the caption should define both and explain why both are reported.
  4. [§C.2] The text refers to 'tmi.js servers'; this appears to be a typo, as tmi.js is a JavaScript library. The intended meaning should be clarified.
  5. [§E.3] The word 'compherensibility' should be 'comprehensibility'.
  6. [References] The citation 'Gilens, Naomi and Williams, Jamie' is not a standard bibliographic entry; the court decision or the news article should be cited properly so that the legal claim can be verified.
  7. [§D.8] The 'quality control' analysis uses AutoMod's own decisions to claim that the filter-specific subsets are correctly categorized; this is a useful filter-specificity sanity check, but it does not validate that the subsets align with Twitch's policy. The text should not present this as evidence for the ground-truth operationalization used in the main recall calculation.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the audit measures AutoMod against external hate-speech datasets and independent benchmarks; no fitted parameter or self-citation drives the central results.

full rationale

The paper's central claims—22% overall recall, 6–7% recall on ToxiGen/IHC, 100% counterfactual slur removal, and 89.5–98.5% blocking of pedagogical/empowering examples—are empirical measurements obtained by sending externally sourced messages to AutoMod and recording its binary decisions. The ground-truth labels come from independent benchmark datasets (SBIC, DynaHate, ToxiGen, IHC) with stated thresholds (e.g., SBIC offensiveness score 1.0 in §3.2 and 0.5 in §B.2); these thresholds are not fitted to AutoMod's outputs. The pre-filtered category in §4.1 is an inference about messages absent from both receiver and Pubsub channels, but this is a measurement assumption subject to error, not a definition that makes the conclusion true by construction. The counterfactual analysis in §5 is a controlled comparison: false negatives are selected first, then demographic terms are replaced with slurs, and the observed increase to 100% recall is an empirical result, not an equivalence. The policy-adherence case study similarly compares generated benign statements to Twitch's own quoted policy; the blocking rate is measured, not derived. The only self-referential element is the quality-control check in Appendix D.8, which uses AutoMod's decisions with the corresponding filter turned off to validate the manual filter subsets; this is a secondary validation and does not enter the headline recall computations. Self-citations, such as Schaffner et al. (2024) for platform selection and policy background, are contextual and not load-bearing for the core findings. Concerns that dataset labels may not perfectly operationalize Twitch's hate-speech policy are external-validity caveats, not circularity. The derivation chain is therefore self-contained and independent of its own conclusions.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The audit rests on four domain assumptions about dataset labels and Twitch's internal behavior; it introduces no new theoretical entities.

free parameters (3)
  • SBIC offensiveness threshold for overall evaluation = 1.0 (overall); 0.5 (filter subsets)
    Hateful examples in SBIC are defined only when all annotators agree the post is offensive (§3.2); using 0.5 for filter subsets changes which examples count as hateful, and recall falls as the threshold relaxes (Figure 10).
  • AutoMod filter level alpha = 4
    All main experiments run filters at Maximum Filtering (§4.1); the 22% recall and 94% bypass figures correspond to this strictest setting, not to typical streamer configuration.
  • Case-study sample sizes = 110 counterfactual examples; 20 sensitive fragments; 50 perturbed fragments
    Hand-selected subsets drive the 100% counterfactual removal, 89.5%/98.5% over-blocking, and perturbation results in §5; no confidence intervals are reported.
assumptions (4)
  • domain assumption Dataset ground-truth labels are correct and align with Twitch's hate speech policy
    All recall/precision metrics are computed against SBIC, DynaHate, ToxiGen, and IHC labels; label errors or policy mismatch would invalidate the quantitative findings.
  • domain assumption Missing messages are pre-filtered by Twitch, not lost to failures
    In §4.1 and §C.4 the authors infer a service-level pre-filtering category from messages that appear in neither the receiver bot nor Pubsub; manual checks covered only some messages.
  • domain assumption AutoMod internal Pubsub categories map injectively to documented content categories
    The paper infers Ableism→Disability, Misogyny→Misogyny, Racism→RER, Homophobia→SSG from API output (§4.1, Figure 6); Twitch documentation does not confirm this mapping.
  • domain assumption Manual mapping of dataset target groups to Twitch subfilters is correct
    Appendix F hand-maps DynaHate, SBIC, and ToxiGen targets to Disability, SSG, Misogyny, and RER; errors would bias filter-wise and community-wise recall.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Silencing Empowerment, Allowing Bigotry: Auditing the Moderation of Hate Speech on Twitch." pith.science (2026). https://pith.science/paper/BWY2Z64X

@misc{pith2026250607667,
  author       = {Pith},
  title        = {Pith review of: Silencing Empowerment, Allowing Bigotry: Auditing the Moderation of Hate Speech on Twitch},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BWY2Z64X}},
  note         = {Machine review of arXiv:2506.07667}
}
abstract

To meet the demands of content moderation, online platforms have resorted to automated systems. Newer forms of real-time engagement($\textit{e.g.}$, users commenting on live streams) on platforms like Twitch exert additional pressures on the latency expected of such moderation systems. Despite their prevalence, relatively little is known about the effectiveness of these systems. In this paper, we conduct an audit of Twitch's automated moderation tool ($\texttt{AutoMod}$) to investigate its effectiveness in flagging hateful content. For our audit, we create streaming accounts to act as siloed test beds, and interface with the live chat using Twitch's APIs to send over $107,000$ comments collated from $4$ datasets. We measure $\texttt{AutoMod}$'s accuracy in flagging blatantly hateful content containing misogyny, racism, ableism and homophobia. Our experiments reveal that a large fraction of hateful messages, up to $94\%$ on some datasets, $\textit{bypass moderation}$. Contextual addition of slurs to these messages results in $100\%$ removal, revealing $\texttt{AutoMod}$'s reliance on slurs as a moderation signal. We also find that contrary to Twitch's community guidelines, $\texttt{AutoMod}$ blocks up to $89.5\%$ of benign examples that use sensitive words in pedagogical or empowering contexts. Overall, our audit points to large gaps in $\texttt{AutoMod}$'s capabilities and underscores the importance for such systems to understand context effectively.

Figures

Figures reproduced from arXiv: 2506.07667 by the authors.

Figure 1
Figure 1. The audit pipeline: (1) Setting up bots to interact with AutoMod and data collation, (2) Recording moderation decisions at scale, (3) Analysis of AutoMod moderation decisions. (2.1) shows actual instances of moderated and non-moderated messages from our ex￾periments in the Twitch interface. The moderation deci￾sions clearly show the limitations of AutoMod. or no legal regulations dictating what is accept￾able conten… view at source ↗
Figure 2
Figure 2. Comparison of AutoMod to State-of-The￾Art (SoTA) language models. Language models were prompted with a zero-shot instruction containing Twitch’s community guidelines. For experimental setup, see §D.6. data subsets is provided in Appendix D.8. The primary objective of this analysis is to assess how well a filter Fi models the criterion ci associated with it. Note that we receive AutoMod’s internal categories for any … view at source ↗
Figure 3
Figure 3. AutoMod’s recall for subsets of hate examples directed towards various communities (All filters under the Discrimination & Slurs category turned on). The opaque portion of each bar indicates the fraction of messages that are pre-filtered. ering utterances use n-grams that are frequently seen in hate speech. Measuring the specificity of filters is therefore as important as recall. The filter precision is shown in [P… view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Official Twitch documentation of moderation tools: (a) Different levels at which Twitch moderates content along with their respective methods of moderation (b) Twitch documentation on AutoMod moderation levels (referred to as α in this work) from 0 to 4. ated and label…
Figure 5
Figure 5. Figure 5: Moderation view as seen from a streamer’s account. (Note: The original screenshot, which was in dark mode UI was manually adapted to light mode for print clarity; minor visual differences from the actual light mode may exist.) C.1 Sending Messages Programatically The f…
Figure 6
Figure 6. Figure 6: Correlating the PubSub output with moderation decisions: (left) Sample JSON output received from PubSub; (top right) Mapping of JSON output to AutoMod’s content categories (described in [PITH_FULL_IMAGE:figures/full_fig_p018_6.png]
Figure 7
Figure 7. Figure 7: Example of user-chat with and without chat filters. We observe that the chat filter behaves similar to AutoMod whereby false positives are prevalent and fails our target group analysis standards as seen in (a). Original Messages are provided in (b) data. We use the dev…
Figure 8
Figure 8. Figure 8: The precision of filters in AutoMod. Precision measures the specificity of a filter. We find that the RER (Race, Ethnicity and Religion) filter is the most specific, while the Misogyny filter is the least specific. D.3 False Negative and False Positive Analysis D.3.1 F…
Figure 10
Figure 10. Figure 10: for Moderation Rate(Recall) on different values of threshold for SBIC [PITH_FULL_IMAGE:figures/full_fig_p020_10.png]
Figure 12
Figure 12. Figure 12: Top 10 pre-filtered unigrams based on frequency. Stop words were excluded while ranking unigrams according to frequency. We observe that both variants of the n-word combined together appear close to 800 times. D.8 Quality Control analysis for data subsets In §4.2 we’v…
Figure 11
Figure 11. Figure 11: Moderation prompt for SoTA language models based on Twitch’s community guidelines [PITH_FULL_IMAGE:figures/full_fig_p022_11.png]
Figure 13
Figure 13. Figure 13: Subset-wise recall with CA = C˜ \ {ci} for each subset Dci . This corresponds to the case when all filters are turned on except the one that shares the same category as the subset. All data subsets except SSG have low recall when their corresponding filter is turned o…
Figure 14
Figure 14. Figure 14: Prompt used for generating examples to evaluate AutoMod’s context-awareness when sensitive terms are used in an empowering/pedagogical way. This prompt was fed to the GPT-4o model via the ChatGPT web interface (www.chatgpt.com). Content Warning: The table below contai…
Figure 15
Figure 15. Figure 15: Prompt to apply semantic-preserving perturbations. This prompt was fed to the GPT-4o model via the ChatGPT web interface (www.chatgpt.com). Perturbation Example Moderation Rate (%) Unperturbed bitch 100.0 Phonetic Play bittch 42.0 Adding Spaces bit ch 22.0 Adding Punc…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

37 extracted references · 37 canonical work pages

  1. [1]

    Platform Choice and Twitch Moderation Details(§A): Provides additional details about Twitch Moderation options referenced in §3.1

    work page

  2. [2]

    Dataset Details(§B): Detailed descriptions of the 4 datasets of hate speech content intro- duced in §3.2

    work page

  3. [3]

    Experimental Setup Details(§C): Break- down of the steps followed to send and receive messages from Twitch, expanding upon §4.1

    work page

  4. [4]

    Further Results(§D): Building on results in §4.2 with analysis of false negatives and false positives from AutoMod, SBIC data thresh- old, different AutoMod filter levels, prefilter- ing bias, and quality control analysis of filter- specific datasets

    work page

  5. [5]

    Ablation Details(§E): Information on the methods used to construct counterfactuals, policy adherence samples and perturbed ex- amples in §5

    work page

  6. [6]

    ML powered tech

    Filter- and Community-Specific Subset Ex- traction(§F): Dataset-wise breakdown of the procedure followed to obtain subsets for ex- periments in §4.2. A Platform Choice and Twitch Moderation Details As an extension of the Platform Exploration and Twitch Moderation section (§3.1), we provide more details about the investigation guiding our platform choice, ...

    work page 2025

  7. [7]

    Expressions of contempt, hatred, or disgust based on a protected characteristic

    work page

  8. [8]

    Mock the event/victims or deny the occurrence of well-documented hate crimes, or deny the existence of documented acts of mass murder/genocide against a protected group

    work page

Show all 37 references
  1. [9]

    Make unfounded claims assigning blame to a protected group, or that otherwise intends to incite fear about a protected group as it relates to health and safety

  2. [10]

    For our experiments, we additionally include a random sample of 6,000 be- nign examples from the dataset

    is a dataset containing approximately 6,000 examples of implicit hate. For our experiments, we additionally include a random sample of 6,000 be- nign examples from the dataset. It includes both the target and implied meaning of hateful statements, collected from hate communiti...

  3. [11]

    label": ‘hate’ if the comment violates community guidelines, else ‘nothate’ }

    Support, promote, or be a member of a hate group, including sharing hate group propaganda materials. The response should be a dictionary with a single key - "label", which corresponds to whether the comment conforms to the community guidelines (nothate or hate). You should onl...

  4. [12]

    Promote, glorify, threaten, or advocate violence, physical harm, or death against individual(s) or groups on the basis of a protected characteristic, including age

  5. [13]

    We allow certain words or terms, which might otherwise violate our policy, to be used in an empowering way or as terms of endearment when such intent is clear

    Use hateful slurs, either untargeted or directed towards another individual. We allow certain words or terms, which might otherwise violate our policy, to be used in an empowering way or as terms of endearment when such intent is clear. We also make exceptions for slurs in mus...

  6. [14]

    Post a comment that dehumanizes or perpetuates negative stereotypes and/or memes

  7. [15]

    Post a comment that expresses inferiority based on a protected characteristic, for example, statements related to physical, mental, and moral deficiencies

  8. [16]

    Call for subjugation, segregation or exclusion, including political, economic, and social exclusion/segregation, based on a protected characteristic, including age. We do, however, allow discussions on certain topics such as immigration policy, voting rights for non-citizens, ...

  9. [17]

    This does not include support for self-determination movements

    Encourage or support the political or economic dominance of any race, ethnicity, or religious group, including support for white supremacist/nationalist ideologies. This does not include support for self-determination movements

  10. [21]

    Encourage the use of or generally endorsing sexual orientation conversion therapy

  11. [23]

    All LGBTQ people should live freely and authentically in their own way

    “All LGBTQ people should live freely and authentically in their own way"

  12. [24]

    We are a group of strong independent bitches and we get things done

    “We are a group of strong independent bitches and we get things done."

  13. [25]

    ISIS is a terrorist organization and we should educate people on its impact and dangers

    “ISIS is a terrorist organization and we should educate people on its impact and dangers"

  14. [26]

    We love you just the way you are if you’re gay or not you’re family

    “We love you just the way you are if you’re gay or not you’re family" E.3 Robustness to semantic-preserving perturbations As described in §5, we conduct experiments to as- sessAutoMod’s robustness to semantic-preserving perturabtions on input text. For obtaining these perturbe...

  15. [27]

    Adding Punctuation: Introducing symbols within the word to disrupt recognition (e.g., b.itches)

  16. [28]

    Adding Spaces: Breaking up the word with spaces (e.g.,bit ches)

  17. [29]

    Partial Obfuscation: Replacing some letters with symbols or asterisks (e.g.,b***ches)

  18. [30]

    Phonetic Play: Modifying spelling to re- tain pronunciation but evade filters (e.g., bittches)

  19. [31]

    Reversed Letters: Rearranging letters in the fragment (e.g.,sehctib)

  20. [32]

    Combination of Methods: Using two or more techniques simultaneously (e.g., b.it ches). Pedagogical Example Generation Prompt Using the Twitch policy: ‘At Twitch, we allow certain words or terms, which might otherwise violate our policy, to be used in an empowering way or as te...

  21. [33]

    Use the phrase in an empowering or endearing way

  22. [34]

    Use the phrase in educational contexts to raise awareness

  23. [35]

    jewish folks

    Use the phrase in neutral or harmless contexts, aligning with Twitch’s policy Fragments: <list of fragments> Figure 14:Prompt used for generating examples to evaluate AutoMod’s context-awareness when sensitive terms are used in an empowering/pedagogical way. This prompt was fe...

  24. [36]

    The first sentence should use the fragment exactly as it is

  25. [37]

    b.itches

    The next 6 sentences should use adversarial perturbations of the fragment as follows: (a) Adding Punctuation: Introduce periods or other symbols within the word to break its recognition while keeping it readable (e.g., "b.itches"). (b) Adding Spaces: Insert spaces between lett...

  26. [2021]

    Software-supported audits of decision-making systems: Testing google and facebook’s political ad- vertising policies.Preprint, arXiv:2103.00064. Meta. 2024. Community Standards Enforcement Re- port. [Online; accessed 19. Jan. 2025]. Danaë Metaxa, Michelle A Gan, Su Goh, Jeff H...

    2024 arXiv

  27. [2022]

    In31st USENIX Security Symposium (USENIX Security 22), pages 607–624, Boston, MA

    An audit of facebook’s political ad policy enforcement. In31st USENIX Security Symposium (USENIX Security 22), pages 607–624, Boston, MA. USENIX Association. Manoel Horta Ribeiro, Justin Cheng, and Robert West

  28. [2023]

    community guidelines make this the best party on the internet

    Automated content moderation increases adherence to community guidelines.Preprint, arXiv:2210.10454. Nadia Ruiz Bravo and Maryam Roshan. 2022. The political turn of twitch – understanding live chat as an emergent political space. Christian Sandvig, Kevin Hamilton, Karrie Karah...

    2022 arXiv

  29. [2025]

    has_swear_words

    allowing for less than 20 messages per 30 seconds for normal chat bot accounts. This ensures that each message is sent and processed into the stream chat without duplication or misses due to connection delays. We conduct iterative tests and conclude that this configuration of ...

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.