RECORD
Understanding the Security of Deepfake Detection
Cryptography and Security · Submitted 5 Jul 2021
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim Our results highlight that deepfake detection should consider the adversarial nature of the problem.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
Deepfakes pose growing challenges to the trust of information on the Internet. Thus, detecting deepfakes has attracted increasing attentions from both academia and industry. State-of-the-art deepfake detection methods consist of two key components, i.e., face extractor and face classifier, which extract the face region in an image and classify it to be real/fake, respectively. Existing studies mainly focused on improving the detection performance in non-adversarial settings, leaving security of deepfake detection in adversarial settings largely unexplored. In this work, we aim to bridge the gap. In particular, we perform a systematic measurement study to understand the security of the state-of-the-art deepfake detection methods in adversarial settings. We use two large-scale public deepfakes data sources including FaceForensics++ and Facebook Deepfake Detection Challenge, where the deepfakes are fake face images; and we train state-of-the-art deepfake detection methods. These detection methods can achieve 0.94--0.99 accuracies in non-adversarial settings on these datasets. However, our measurement results uncover multiple security limitations of the deepfake detection methods in adversarial settings. First, we find that an attacker can evade a face extractor, i.e., the face extractor fails to extract the correct face regions, via adding small Gaussian noise to its deepfake images. Second, we find that a face classifier trained using deepfakes generated by one method cannot detect deepfakes generated by another method, i.e., an attacker can evade detection via generating deepfakes using a new method. Third, we find that an attacker can leverage backdoor attacks developed by the adversarial machine learning community to evade a face classifier. Our results highlight that deepfake detection should consider the adversarial nature of the problem.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | Deepfakes pose growing challenges to the trust of information on the Internet. | background 1.00 | 0.03 | 0.02 | asserted 1.00 |
| s1 | Thus, detecting deepfakes has attracted increasing attentions from both academia and industry. | background 1.00 | 0.04 | 0.02 | asserted 1.00 |
| s2 | State-of-the-art deepfake detection methods consist of two key components, i.e., face extractor and face classifier, which extract the face region in an image and classify it to be real/fake, respectively. | background 0.99 | 0.04 | 0.02 | asserted 1.00 |
| s3 | Existing studies mainly focused on improving the detection performance in non-adversarial settings, leaving security of deepfake detection in adversarial settings largely unexplored. | gap 1.00 | 0.05 | 0.04 | asserted 1.00 |
| s4 | In this work, we aim to bridge the gap. | contribution 0.89 | 0.31 | 0.03 | asserted 0.65 |
| s5 | In particular, we perform a systematic measurement study to understand the security of the state-of-the-art deepfake detection methods in adversarial settings. | contribution 0.94 | 0.92 | 0.05 | asserted 0.91 |
| s6 | We use two large-scale public deepfakes data sources including FaceForensics++ and Facebook Deepfake Detection Challenge, where the deepfakes are fake face images; and we train state-of-the-art deepfake detection methods. | method 0.99 | 0.94 | 0.02 | asserted 0.99 |
| s7 | These detection methods can achieve 0.94--0.99 accuracies in non-adversarial settings on these datasets. | result 0.96 | 0.81 | 0.03 | asserted 1.00 |
| s8 | However, our measurement results uncover multiple security limitations of the deepfake detection methods in adversarial settings. | result 0.99 | 0.95 | 0.35 | asserted 1.00 |
| s9 | First, we find that an attacker can evade a face extractor, i.e., the face extractor fails to extract the correct face regions, via adding small Gaussian noise to its deepfake images. | result 1.00 | 0.95 | 0.08 | asserted 1.00 |
| s10 | Second, we find that a face classifier trained using deepfakes generated by one method cannot detect deepfakes generated by another method, i.e., an attacker can evade detection via generating deepfakes using a new method. | result 0.99 | 0.92 | 0.10 | asserted 1.00 |
| s11 | Third, we find that an attacker can leverage backdoor attacks developed by the adversarial machine learning community to evade a face classifier. | result 1.00 | 0.90 | 0.07 | asserted 1.00 |
| s12 | Our results highlight that deepfake detection should consider the adversarial nature of the problem. | interpretation 0.97 | 0.46 | 0.88 | asserted 0.68 |
What was examined, and what was not
- arXiv listing abstract v3
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
Deepfakes pose growing challenges to the trust of information on the Internet. Thus, detecting deepfakes has attracted increasing attentions from both academia and industry. State-of-the-art deepfake detection methods consist of two key components, i.e., face extractor and face classifier, which extract the face region in an image and classify it to be real/fake, respectively. Existing studies mainly focused on improving the detection performance in non-adversarial settings, leaving security of deepfake detection in adversarial settings largely unexplored. In this work, we aim to bridge the gap. In particular, we perform a systematic measurement study to understand the security of the state-of-the-art deepfake detection methods in adversarial settings. We use two large-scale public deepfakes data sources including FaceForensics++ and Facebook Deepfake Detection Challenge, where the deepfakes are fake face images; and we train state-of-the-art deepfake detection methods. These detection methods can achieve 0.94--0.99 accuracies in non-adversarial settings on these datasets. However, our measurement results uncover multiple security limitations of the deepfake detection methods in adversarial settings. First, we find that an attacker can evade a face extractor, i.e., the face extractor fails to extract the correct face regions, via adding small Gaussian noise to its deepfake images. Second, we find that a face classifier trained using deepfakes generated by one method cannot detect deepfakes generated by another method, i.e., an attacker can evade detection via generating deepfakes using a new method. Third, we find that an attacker can leverage backdoor attacks developed by the adversarial machine learning community to evade a face classifier. Our results highlight that deepfake detection should consider the adversarial nature of the problem.
Human review
How the literature uses it
1 arXiv paper cite this in 3 sentences
Papers citing it · 2 sentences without a confident label
Contests it 1 paper
-
The attack method adopted in Cao et al. [ref] belongs to the category of BadNets and the trigger pattern is not stealthy, making it easy to be detected and defended by multiple backdoor defense methods.
Is It Possible to Backdoor Face Forgery Detection with Natural Triggers? · Backdoor Attacks · contest 0.82
What it cites · 1 sentence without a confident label
Cites as background 2 papers
-
Heuristic-based face classifiers [ref] rely on some heuristics to distinguish between fake faces and real faces.
In Ictu Oculi: Exposing AI Generated Fake Face Videos by Detecting Eye Blinking · Introduction · 0.81
-
Neural-network-based face classifiers [ref] train binary neural network classifiers to detect fake faces.
Use of a Capsule Network to Detect Fake Images and Videos · Detecting Deepfakes · 0.96
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.