Deepfake detection must shift from classifying media realism to detecting communicative deception by applying Speech Act Theory, Grice's Cooperative Principle, and Cialdini's influence principles.
hub Mixed citations
Deepfake-Eval-2024: A Multi-Modal In-the-Wild Benchmark of Deepfakes Circulated in 2024
Mixed citation behavior. Most common role is background (40%).
17
Pith papers citing it
Background
40% of classified citations
abstract
In the age of increasingly realistic generative AI, robust deepfake detection is essential for mitigating fraud and disinformation. While many deepfake detectors report high accuracy on academic datasets, we show that these academic benchmarks are out of date and not representative of real-world deepfakes. We introduce Deepfake-Eval-2024, a new deepfake detection benchmark consisting of in-the-wild deepfakes collected from social media and deepfake detection platform users in 2024. Deepfake-Eval-2024 consists of 45 hours of videos, 56.5 hours of audio, and 1,975 images, encompassing the latest manipulation technologies. The benchmark contains diverse media content from 88 different websites in 52 different languages. We find that the performance of open-source state-of-the-art deepfake detection models drops precipitously when evaluated on Deepfake-Eval-2024, with AUC decreasing by 50% for video, 48% for audio, and 45% for image models compared to previous benchmarks. We also evaluate commercial deepfake detection models and models finetuned on Deepfake-Eval-2024, and find that they have superior performance to off-the-shelf open-source models, but do not yet reach the accuracy of deepfake forensic analysts. The dataset is available at https://github.com/nuriachandra/Deepfake-Eval-2024.
hub tools
citation-role summary
background 3
baseline 1
dataset 1
citation-polarity summary
years
2026 17representative citing papers
An automated fact-check-based pipeline for in-the-wild AI image data, when mixed with generator data in continual learning, lets detectors adapt to new generators while avoiding forgetting and delivers 8-9% accuracy gains on two existing models.
ICLAD combines in-context learning and comparison guidance in audio language models with a routing detector to boost generalization and explanations for audio deepfake detection, achieving up to 2x F1 gains on wild data.
By mid-2025 roughly 35% of new websites are AI-generated or AI-assisted, correlating with lower semantic diversity and higher positive sentiment but showing no significant drop in factual accuracy or stylistic diversity.
Spoof-SUPERB benchmark shows large-scale discriminative SSL models such as XLS-R, UniSpeech-SAT, and WavLM Large outperform others in audio deepfake detection and maintain robustness under acoustic degradations.
Linear probes on frozen self-supervised representations closely approach bespoke deepfake detector performance on benchmarks, indicating benchmarks largely measure general modality understanding.
Introduces the CIFAR Synthetic Evidence Corpus, a multi-family dataset of AI-manipulated documents with source-separated train/test splits for evaluating detectors of AI-generated legal evidence.
COF fuses epistemic, aleatoric, calibration, conformal and distributional uncertainties via simplex optimization of Pearson correlation with errors, outperforming alternatives under distribution shift on CelebDF but collapsing with all methods on cross-dataset tests.
Alethia is a pretrained audio encoder using continuous embedding prediction and generative flow-matching reconstruction that outperforms existing speech foundation models on voice deepfake tasks with better robustness and zero-shot generalization.
PhyLAA-X embeds physics-derived feature volumes into localized artifact attention for improved cross-generator generalization and adversarial robustness in deepfake detection.
DiCoME decomposes entangled representations into decorrelated semantic and artifact views using geometric purification and uncertainty-aware evidential learning to improve generalization and produce calibrated uncertainty in deepfake detection.
Omni-Fake delivers a unified multimodal deepfake benchmark dataset and RL-driven detector that reports gains in accuracy, cross-modal generalization, and explainability over prior baselines.
A 30-minute training intervention increased US intelligence analysts' accuracy at distinguishing real from AI-generated images by 9 percentage points from a 72% baseline, mainly by improving identification of real images.
The SAFE challenge shows measurable progress in detecting synthetic videos across different generators but persistent weaknesses against post-processing operations.
A 3B-parameter vision-language model trained on continuously curated social media data detects AI-generated content with state-of-the-art accuracy on benchmarks and shows positive engagement effects in production deployment.
The paper analyzes evolving security and safety threats in generative AI from content generation to agentic actions, noting that attack surfaces expand faster than defenses and that many safeguards require institutional coordination not yet in place.
citing papers explorer
-
Detecting Deception, Not Deepfakes: Why Media Forensics Needs Social Theories
Deepfake detection must shift from classifying media realism to detecting communicative deception by applying Speech Act Theory, Grice's Cooperative Principle, and Cialdini's influence principles.
-
Automated In-the-Wild Data Collection for Continual AI Generated Image Detection
An automated fact-check-based pipeline for in-the-wild AI image data, when mixed with generator data in continual learning, lets detectors adapt to new generators while avoiding forgetting and delivers 8-9% accuracy gains on two existing models.
-
ICLAD: In-Context Learning with Comparison-Guidance for Audio Deepfake Detection
ICLAD combines in-context learning and comparison guidance in audio language models with a routing detector to boost generalization and explanations for audio deepfake detection, achieving up to 2x F1 gains on wild data.
-
The Impact of AI-Generated Text on the Internet
By mid-2025 roughly 35% of new websites are AI-generated or AI-assisted, correlating with lower semantic diversity and higher positive sentiment but showing no significant drop in factual accuracy or stylistic diversity.
-
A SUPERB-Style Benchmark of Self-Supervised Speech Models for Audio Deepfake Detection
Spoof-SUPERB benchmark shows large-scale discriminative SSL models such as XLS-R, UniSpeech-SAT, and WavLM Large outperform others in audio deepfake detection and maintain robustness under acoustic degradations.
-
What Do Deepfake Benchmarks Measure? An Audit Using Frozen Self-Supervised Representations
Linear probes on frozen self-supervised representations closely approach bespoke deepfake detector performance on benchmarks, indicating benchmarks largely measure general modality understanding.
-
The CIFAR Synthetic Evidence Corpus for Detecting AI-Generated Evidence
Introduces the CIFAR Synthetic Evidence Corpus, a multi-family dataset of AI-manipulated documents with source-separated train/test splits for evaluating detectors of AI-generated legal evidence.
-
Architecture-Adaptive Uncertainty Fusion for Deepfake Detection
COF fuses epistemic, aleatoric, calibration, conformal and distributional uncertainties via simplex optimization of Pearson correlation with errors, outperforming alternatives under distribution shift on CelebDF but collapsing with all methods on cross-dataset tests.
-
Alethia: A Foundational Encoder for Voice Deepfakes
Alethia is a pretrained audio encoder using continuous embedding prediction and generative flow-matching reconstruction that outperforms existing speech foundation models on voice deepfake tasks with better robustness and zero-shot generalization.
-
Aletheia: Physics-Conditioned Localized Artifact Attention (PhyLAA-X) for End-to-End Generalizable and Robust Deepfake Video Detection
PhyLAA-X embeds physics-derived feature volumes into localized artifact attention for improved cross-generator generalization and adversarial robustness in deepfake detection.
-
Divide and Conquer: Reliable Multi-View Evidential Learning for Deepfake Detection
DiCoME decomposes entangled representations into decorrelated semantic and artifact views using geometric purification and uncertainty-aware evidential learning to improve generalization and produce calibrated uncertainty in deepfake detection.
-
Omni-Fake: Benchmarking Unified Multimodal Social Media Deepfake Detection
Omni-Fake delivers a unified multimodal deepfake benchmark dataset and RL-driven detector that reports gains in accuracy, cross-modal generalization, and explainability over prior baselines.
-
Generative AI Literacy Training Improves Intelligence Analysts' Discrimination of Real and AI-Generated Images
A 30-minute training intervention increased US intelligence analysts' accuracy at distinguishing real from AI-generated images by 9 percentage points from a 72% baseline, mainly by improving identification of real images.
-
Advancing Reliable Synthetic Video Detection: Insights from the SAFE Challenge
The SAFE challenge shows measurable progress in detecting synthetic videos across different generators but persistent weaknesses against post-processing operations.
-
Detecting AI-Generated Content on Social Media with Multi-modal Language Models
A 3B-parameter vision-language model trained on continuously curated social media data detects AI-generated content with state-of-the-art accuracy on benchmarks and shows positive engagement effects in production deployment.
-
From AI-Generated Content to Agentic Action: Security and Safety Threats in Generative AI
The paper analyzes evolving security and safety threats in generative AI from content generation to agentic actions, noting that attack surfaces expand faster than defenses and that many safeguards require institutional coordination not yet in place.
- The Calibrated Deepfake Trust Score (CDTS): Competence-Coupled Trust Degradation Across Deepfake Detectors