REVIEW 10 cited by
Adapting Large Language Models for Content Moderation: Pitfalls in Data Engineering and Supervised Fine-tuning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Nowadays, billions of people engage in communication and express their opinions on the internet daily. Unfortunately, not all of these expressions are friendly or compliant, making content moderation an indispensable task. A common approach is to use a discriminative model to classify the content, but this method often requires strict data engineering, otherwise it will face unacceptable overfitting. With the successful development of Large Language Models (LLMs) in recent years, LLM-based methods have become a feasible solution for handling tasks in various domains. Thanks to the knowledge of the foundation models, we can develop more robust privately deployed models with limited data via fine-tuning these foundation models. Moreover, as a generative model, it can provide detailed analysis of the review process, enhancing interpretability. In this paper, we introduce how to fine-tune a LLM model that can be privately deployed for content moderation. Specifically, we discuss the differences between discriminative and generative models using content moderation as an example. Additionally, we reveal that incorporating reasoning processes during the fine-tuning of LLMs can effectively alleviate overfitting, even if the model is not allowed to directly output reasoning processes during deployment. We present a complete process, from data collection and construction to model training and overfitting elimination, for fine-tuning LLMs in vertical domain deployments. We report the entire research process and the key findings in this paper, hoping to provide valuable experience for researchers who are fine-tuning privately deployed models in their domain-specific research.
Forward citations
Cited by 10 Pith papers
-
Lost in Pronunciation: Detecting Chinese Offensive Language Disguised by Phonetic Cloaking Replacement
A new 500-post benchmark of naturally occurring phonetic cloaking shows LLMs detect such Chinese offensive language with F1 at most 0.672, and Pinyin-augmented prompting partially repairs the gap.
-
ChineseHarm-Bench: A Chinese Harmful Content Detection Benchmark
ChineseHarm-Bench is a six-category, 6,000-sample Chinese harmful content detection benchmark with a human-annotated knowledge rule base, and a knowledge-augmented fine-tuning baseline that brings small models to near...
-
VLM as Policy: Common-Law Content Moderation Framework for Short Video Platform
KuaiMod is a VLM-based content moderation system for short video platforms, trained on chain-of-thought case data and refined with user feedback, claiming a 20% drop in user reports in online A/B tests.
-
Clear Preferences Leave Traces: Reference Model-Guided Sampling for Preference Learning
Filtering DPO training data by the absolute length-normalized reference model log-probability gap between preferred and rejected responses improves MT-Bench scores with 30-50% of the data.
-
ICM-Assistant: Instruction-tuning Multimodal Large Language Models for Rule-based Explainable Image Content Moderation
A rule-decomposition data pipeline and 246K question-answer pairs let instruction-tuned multimodal LLMs classify and explain image content moderation more accurately than their untuned versions.
-
Embedding-based Retrieval in Multimodal Content Moderation
Similarity-based retrieval with contrastively trained embeddings catches emerging harmful video trends far better than a fixed classifier, according to the paper's production experiments.
-
Socio-Culturally Aware Evaluation Framework for LLM-Based Content Moderation
A persona-based generation pipeline creates culturally varied content moderation test sets, but its central 'greater challenge' claim depends on unvalidated synthetic labels and unreleased data.
-
Filter-And-Refine: A MLLM Based Cascade System for Industrial-Scale Video Content Moderation
A cascade of an embedding router and a fine-tuned multimodal LLM ranker is claimed to improve content moderation F1 by 66.5% while using 1.5% of the compute of direct LLM deployment.
-
Guardians and Offenders: A Survey on Harmful Content Generation and Safety Mitigation of LLM
The submission's abstract promises an LLM safety survey, but the provided body is the opening page of an unrelated arithmetic-dynamics paper, so the artifact is internally inconsistent.
-
Bridging the Safety Gap: A Guardrail Pipeline for Trustworthy LLM Inferences
A guardrail pipeline combining detection, retrieval grounding, rule-based wrappers, and a repair model is reported to match OpenAI moderation and fix 80.7 percent of hallucinated HaluEval answers.
Discussion (0). Continue with ORCID to comment.