REVIEW 7 cited by
RAID: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectors
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
RAID: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectors
read the original abstract
Many commercial and open-source models claim to detect machine-generated text with extremely high accuracy (99% or more). However, very few of these detectors are evaluated on shared benchmark datasets and even when they are, the datasets used for evaluation are insufficiently challenging-lacking variations in sampling strategy, adversarial attacks, and open-source generative models. In this work we present RAID: the largest and most challenging benchmark dataset for machine-generated text detection. RAID includes over 6 million generations spanning 11 models, 8 domains, 11 adversarial attacks and 4 decoding strategies. Using RAID, we evaluate the out-of-domain and adversarial robustness of 8 open- and 4 closed-source detectors and find that current detectors are easily fooled by adversarial attacks, variations in sampling strategies, repetition penalties, and unseen generative models. We release our data along with a leaderboard to encourage future research.
Forward citations
Cited by 7 Pith papers
-
Triospect: A Three-Dimensional Framework for Robust Statistical AI-Generated Text Detection Against Diverse Attacks
Triospect combines statistical, content, and expression views to detect AI text more robustly, reporting AUROC gains of 22.3% and 9.1% on two attacked benchmarks across 17 attacks and 17 models.
-
MAGA-Bench: Machine-Augment-Generated Text via Alignment Detection Benchmark
Adding human-alignment augmentation (roleplaying, BPO, self-refine, RLDF) to machine-generated text both fools existing detectors and improves the generalization of detectors fine-tuned on it.
-
Small Is Enough: Per-User Style Rewriting of AI-Edited Text via LoRA Adapters
LoRA-adapted 0.5B-7B language models all reach the same automatic rewriting score (0.69), indicating model size does not change measured quality for this single-user style-rewriting task.
-
Hijacking Text Heritage: Hiding the Human Signature through Homoglyphic Substitution
Replacing characters in at least 37.5% of words with visual homoglyphs degrades authorship verification scores enough to obfuscate style, with diminishing returns past 50%.
-
Hijacking Text Heritage: Hiding the Human Signature through Homoglyphic Substitution
Homoglyph substitution on text degrades stylometric systems to hide author signatures and personal information.
-
Lightweight Stylistic Consistency Profiling: Robust Detection of LLM-Generated Textual Content for Multimedia Moderation
LiSCP detects LLM-generated text via stylistic consistency profiling across paraphrased variants and reports up to 11.79% better cross-domain accuracy plus robustness to adversarial attacks.
-
A Comprehensive Dataset for Human vs. AI Generated Text Detection
A dataset of ~58k NYT articles plus AI rewrites from six LLMs, evaluated with a rewrite-distance baseline reaching 58.35% detection and 8.92% attribution accuracy.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.