REVIEW 5 cited by
Watch Your Language: Investigating Content Moderation with Large Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large language models (LLMs) have exploded in popularity due to their ability to perform a wide array of natural language tasks. Text-based content moderation is one LLM use case that has received recent enthusiasm, however, there is little research investigating how LLMs perform in content moderation settings. In this work, we evaluate a suite of commodity LLMs on two common content moderation tasks: rule-based community moderation and toxic content detection. For rule-based community moderation, we instantiate 95 subcommunity specific LLMs by prompting GPT-3.5 with rules from 95 Reddit subcommunities. We find that GPT-3.5 is effective at rule-based moderation for many communities, achieving a median accuracy of 64% and a median precision of 83%. For toxicity detection, we evaluate a suite of commodity LLMs (GPT-3, GPT-3.5, GPT-4, Gemini Pro, LLAMA 2) and show that LLMs significantly outperform currently widespread toxicity classifiers. However, recent increases in model size add only marginal benefit to toxicity detection, suggesting a potential performance plateau for LLMs on toxicity detection tasks. We conclude by outlining avenues for future work in studying LLMs and content moderation.
Forward citations
Cited by 5 Pith papers
-
Compass: Continuously Aligning Social Media Feeds via In-Situ Reflections
A browser extension that embeds reflection prompts into short-form video feeds and automatically re-aligns recommendations led to more preference adjustments and better feed alignment than a manual baseline in a 10-da...
-
Lost in Pronunciation: Detecting Chinese Offensive Language Disguised by Phonetic Cloaking Replacement
A new 500-post benchmark of naturally occurring phonetic cloaking shows LLMs detect such Chinese offensive language with F1 at most 0.672, and Pinyin-augmented prompting partially repairs the gap.
-
It is not enough to give your moderation rules to ChatGPT: Policy-as-Prompt Moderation and Its Potential Impacts on Community Governance
Writing a moderation policy as an LLM prompt cannot by itself ensure meaningful community governance.
-
Dynamic Content Moderation in Livestreams: Combining Supervised Classification with MLLM-Boosted Similarity Matching
A deployed hybrid moderation system combining supervised classification and reference-based similarity matching, boosted by MLLM distillation, reduces unwanted livestream views by 6–8%.
-
Assessing and Refining ChatGPT's Performance in Identifying Targeting and Inappropriate Language: A Comparative Study
Prompt-engineering ChatGPT on a self-created Reddit benchmark raises Cohen's Kappa for targeting detection to 0.66 on the tuning set, but the evaluation is in-sample and the crowd baseline itself is only moderately reliable.
Discussion (0). Continue with ORCID to comment.