Pith. sign in

REVIEW 31 cited by

ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2203.09509 v4 pith:444SPTCJ submitted 2022-03-17 cs.CL

Thomas Hartvigsen , Saadia Gabriel , Hamid Palangi , Maarten Sap , Dipankar Ray , Ece Kamar This is my paper · ORCID
classification cs.CL
keywords toxictoxigentextlanguagemachine-generateddatagroupshate
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Toxic language detection systems often falsely flag text that contains minority group mentions as toxic, as those groups are often the targets of online hate. Such over-reliance on spurious correlations also causes systems to struggle with detecting implicitly toxic language. To help mitigate these issues, we create ToxiGen, a new large-scale and machine-generated dataset of 274k toxic and benign statements about 13 minority groups. We develop a demonstration-based prompting framework and an adversarial classifier-in-the-loop decoding method to generate subtly toxic and benign text with a massive pretrained language model. Controlling machine generation in this way allows ToxiGen to cover implicitly toxic text at a larger scale, and about more demographic groups, than previous resources of human-written text. We conduct a human evaluation on a challenging subset of ToxiGen and find that annotators struggle to distinguish machine-generated text from human-written language. We also find that 94.5% of toxic examples are labeled as hate speech by human annotators. Using three publicly-available datasets, we show that finetuning a toxicity classifier on our data improves its performance on human-written data substantially. We also demonstrate that ToxiGen can be used to fight machine-generated toxicity as finetuning improves the classifier significantly on our evaluation subset. Our code and data can be found at https://github.com/microsoft/ToxiGen.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 31 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. STATE ToxiCN: A Benchmark for Span-level Target-Aware Toxicity Extraction in Chinese Hate Speech Detection

    cs.CL 2025-01 conditional novelty 8.0 of 10

    STATE ToxiCN provides the first span-level Chinese hate speech dataset with 9,533 target-argument-hateful-group quadruples and a 830-term annotated hateful slang lexicon, and baseline results show fine-tuned models ou...

  2. Fine-Grained Chinese Hate Speech Understanding: Span-Level Resources, Coded Term Lexicon, and Enhanced Detection Frameworks

    cs.CL 2025-07 reject novelty 7.0 of 10

    The paper creates a span-level Chinese hate speech dataset and a 830-term coded hate lexicon, but its two-stage training method's reported superiority is contradicted by the paper's own COLD results.

  3. S2T-RLHF: Hierarchical Credit Assignment for Stable Preference-Based RLHF

    cs.AI 2026-05 conditional novelty 6.0 of 10

    S2T-RLHF splits each response-level RLHF reward into sentence shares and then token shares, via bargaining and Dirichlet weighting, yielding steadier training with competitive preference alignment.

  4. YouthSafe: A Youth-Centric Safety Benchmark and Safeguard Model for Large Language Models

    cs.HC 2025-09 conditional novelty 6.0 of 10

    Introduces YAIR, a youth-GenAI risk benchmark, and YouthSafe, a fine-tuned classifier with AUPRC 0.94 on it, though it compares a trained model to untrained baselines.

  5. Model Misalignment and Language Change: Traces of AI-Associated Language in Unscripted Spoken English

    cs.CL 2025-08 conditional novelty 6.0 of 10

    After ChatGPT's release, science and tech podcast speakers used AI-associated words like 'surpass' and 'align' more often, while control synonyms showed no average shift.

  6. Trade-offs in Image Generation: How Do Different Dimensions Interact?

    cs.CV 2025-07 conditional novelty 6.0 of 10

    A new benchmark and VLM-as-judge metric map trade-offs among ten image-generation dimensions across 14 models, with a visualization called DTM.

  7. PL-Guard: Benchmarking Language Model Safety for Polish

    cs.CL 2025-06 reject novelty 6.0 of 10

    A small Polish BERT classifier proved more robust than larger fine-tuned LLMs at classifying safe versus unsafe Polish content, including under character-level adversarial perturbations.

  8. MCP Safety Training: Learning to Refuse Falsely Benign MCP Exploits using Improved Preference Alignment

    cs.LG 2025-05 conditional novelty 6.0 of 10

    The paper introduces TRADE, an online-only MCP attack, and RAG-Pref, a retrieval-based preference alignment method that together with DPO improves strict refusal of falsely benign MCP exploits from 6.7% to 24.1% on average.

  9. Large Language Models Often Know When They Are Being Evaluated

    cs.CL 2025-05 conditional novelty 6.0 of 10

    Frontier language models distinguish evaluation transcripts from deployment transcripts with AUC up to 0.83, below the authors' human baseline of 0.92.

  10. Lifelong Safety Alignment for Language Models

    cs.CR 2025-05 conditional novelty 6.0 of 10

    A co-evolutionary attacker-defender loop, warmed up by strategies extracted from jailbreak papers, reduces jailbreak success rate on a robust model from 73% to 7% in two iterations.

  11. Examining the Expanding Role of Synthetic Data Throughout the AI Development Pipeline

    cs.HC 2025-01 conditional novelty 6.0 of 10

    Twenty-nine interviews show AI practitioners rely on synthetic data across nearly every pipeline stage while validation remains mostly manual spot-checking.

  12. Cannot or Should Not? Automatic Analysis of Refusal Composition in IFT/RLHF Datasets and Refusal Behavior of Black-Box LLMs

    cs.AI 2024-12 conditional novelty 6.0 of 10

    A unified 16-category refusal taxonomy with human and synthetic datasets and a low-cost classifier for auditing refusal behavior in LLMs.

  13. Chained Tuning Leads to Biased Forgetting

    cs.CL 2024-12 conditional novelty 6.0 of 10

    Fine-tuning a safety-tuned LLM on a capability task erases safety behavior more than the reverse order, and this forgetting is worse for specific groups such as Muslim people in the authors' tests.

  14. Fence: Specialized SLM Guardrails for LLM Applications

    cs.AI 2026-05 conditional novelty 5.0 of 10

    Small language models trained on GAN-style adversarial synthetic data outperform prompted large models on use-case-specific guardrails for off-topic and prompt-injection detection.

  15. Decoding the Rule Book: Extracting Hidden Moderation Criteria from Reddit Communities

    cs.CL 2025-09 conditional novelty 5.0 of 10

    Lexical scores from an interpretable classifier reveal that moderation criteria differ sharply across subreddits, though the extracted criteria only partially match the accuracy of full-text neural models.

  16. Surfer-H Meets Holo1: Cost-Efficient Web Agent Powered by Open Weights

    cs.AI 2025-06 reject novelty 5.0 of 10

    The paper claims a state-of-the-art, low-cost open-weight web agent, but its WebVoyager result is inflated by training on the evaluation benchmark.

  17. AutoMixAlign: Adaptive Data Mixing for Multi-Task Preference Optimization in LLMs

    cs.LG 2025-05 conditional novelty 5.0 of 10

    AutoMixAlign adaptively reweights or resamples task data during DPO training to match specialist-model losses, improving average performance on helpfulness, coding, and safety benchmarks compared to standard DPO and m...

  18. Towards High-Fidelity Synthetic Multi-platform Social Media Datasets via Large Language Models

    cs.CL 2025-05 conditional novelty 5.0 of 10

    LLM-generated multi-platform social media posts approximate real data on some metrics, but all three tested models show platform-specific biases in URLs, hashtags, sentiment, and topics.

  19. Mitigating Heterogeneous Token Overfitting in LLM Knowledge Editing

    cs.CL 2025-02 conditional novelty 5.0 of 10

    OVERTONE replaces uniform cross-entropy targets with adaptively filtered, per-token smoothed targets and improves portability and locality of four knowledge editing methods.

  20. Unveiling the Secret Recipe: A Guide For Supervised Fine-Tuning Small LLMs

    cs.LG 2024-12 conditional novelty 5.0 of 10

    For 3B-7B LLMs, larger batch sizes with lower learning rates improve instruction-tuning benchmarks, early gradient and loss signals predict final quality, and stacked training matches phased training with fewer samples.

  21. Making FETCH! Happen: Finding Emergent Dog Whistles Through Common Habitats

    cs.CL 2024-12 conditional novelty 5.0 of 10

    A new benchmark shows existing NLP systems find almost no novel dog whistles in social media, while the proposed EarShot pipeline raises F0.5 scores to 14.6 on synthetic Reddit, 5.7 on Gab, and 4.6 on Twitter.

  22. ChineseWebText 2.0: Large-Scale High-quality Chinese Web Text with Multi-dimensional and fine-grained information

    cs.CL 2024-11 conditional novelty 5.0 of 10

    ChineseWebText2.0 is a 3.8TB Chinese web text dataset with per-text quality scores, domain labels, toxicity labels and toxicity scores, built and released by the paper's MDFG-tool pipeline.

  23. The Dark Side of Trust: Authority Citation-Driven Jailbreak Attacks on Large Language Models

    cs.LG 2024-11 conditional novelty 5.0 of 10

    Fake authoritative citations matched to the type of harmful request can bypass safety alignment in several commercial and open LLMs.

  24. Beyond Explainable AI (XAI): An Overdue Paradigm Shift and Post-XAI Research Directions

    cs.CY 2026-02 unverdicted novelty 4.0 of 10

    Current XAI methods for DNNs and LLMs rest on paradoxes and false assumptions that demand a paradigm shift to verification protocols, scientific foundations, context-aware design, and faithful model analysis rather th...

  25. SweEval: Do LLMs Really Swear? A Safety Benchmark for Testing Limits for Enterprise Use

    cs.CL 2025-05 conditional novelty 4.0 of 10

    A new cross-lingual benchmark shows large language models comply with explicit requests to use swear words far more often in Indic languages than in English, revealing a safety alignment gap.

  26. Toward Generalizable Evaluation in the LLM Era: A Survey Beyond Benchmarks

    cs.CL 2025-04 conditional novelty 4.0 of 10

    A survey that frames the core problem of LLM evaluation as 'evaluation generalization': finite test sets cannot scale with unbounded model capabilities, and proposes two transitions in evaluation design.

  27. MODP: Multi Objective Directional Prompting

    cs.CC 2025-04 reject novelty 4.0 of 10

    MODP is a metrics-driven, multi-objective prompt engineering framework that, in this paper, improves ReCoRD fill-in-the-blank accuracy from 48% to 73% on Mixtral by adding instructions, toxicity handling, and model-sp...

  28. Parameter-Efficient Checkpoint Merging via Metrics-Weighted Averaging

    cs.LG 2025-04 conditional novelty 4.0 of 10

    Merging LoRA checkpoints with weights based on inverse training loss, plus a rank-based penalty, beats uniform averaging on three fine-tuning tasks.

  29. Compromising Honesty and Harmlessness in Language Models via Deception Attacks

    cs.CL 2025-02 conditional novelty 4.0 of 10

    Fine-tuning LLMs on a handful of misleading answers creates selectively deceptive models that stay accurate elsewhere and also become more toxic.

  30. The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMs

    cs.CL 2025-06 conditional novelty 3.0 of 10

    A structured survey of LLM safety evaluation that proposes a why/what/where/how taxonomy and catalogs metrics, datasets, benchmarks, evaluators, and frameworks.

  31. A Survey on Backdoor Threats in Large Language Models (LLMs): Attacks, Defenses, and Evaluations

    cs.CR 2025-02 conditional novelty 2.0 of 10

    A literature review that taxonomizes LLM backdoor attacks and defenses by model construction phase, with no new experimental results.

Pith tools