Pith. sign in

REVIEW 4 major objections 5 minor 53 references

DeepFake Doctor: Diagnosing and Treating Audio-Video Fake Detection

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Audio-video deepfake detectors exploit leading silence in fake clips rather than detecting manipulation; temporal jittering during training removes the shortcut, and new splits give cleaner generalization numbers.

desk verdict A genuinely useful benchmarking paper; the silence-shortcut mitigation claim needs a precise trim specification and error bars before it is fully checkable. read the letter →

arxiv 2506.05851 v1 pith:BSH6MQMF submitted 2025-06-06 cs.MM cs.AIcs.SDeess.AS

Marcel Klemt , Carlotta Segna , Anna Rohrbach This is my paper
classification cs.MMcs.AIcs.SDeess.AS
keywords audio-videodeepfakedetectionleadingsilenceshortcuttemporaljitteringDeepSpeakv1FakeAVCelebcross-manipulationgeneralizationevaluationprotocolSIMBAmultimodalbaseline
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper takes on three interlocking problems in audio-video deepfake detection — datasets with exploitable shortcuts, irreproducible methods, and evaluation protocols with blind spots — and argues all three are distorting published progress. Its central diagnosis is the 'leading silence' artifact: fake-audio clips in FakeAVCeleb begin with milliseconds of silence that real clips do not, and supervised models latch onto that cue instead of learning to detect manipulation. Its proposed treatment is temporal jittering, sampling every training clip at a random starting time, which shrinks the untrimmed-versus-trimmed performance gap from drops of tens of AUC points to at most a few points. The paper also gives DeepSpeak v1 its first evaluation protocol, proposes method and family splits that remove leakage from the established FakeAVCeleb leave-one-out scheme, and shows that DeepSpeak v1's newer manipulations form a harder, more out-of-distribution benchmark.

What carries the argument

The load-bearing artifact is SIMBA, a simple supervised baseline pairing a Kinetics-pretrained R(2+1)D video encoder with a BYOL-A audio encoder followed by self-attention and max-pooling, with late fusion into a binary or multiclass head. Two further instruments carry the argument: temporal jittering, the training-time augmentation that samples a clip at a random starting point and serves as the proposed treatment for the shortcut, and the silence detector, which flags audio that stays more than 20 dB below normal for at least 20 ms at clip start and serves as the diagnostic tool for measuring shortcut reliance. The third instrument is the evaluation design: method splits that hold out one manipulation regardless of its audio condition, and family splits that hold out entire manipulation families, built to remove the blind spots and leakage of the previous leave-one-out scheme.

What would settle it

Train a supervised audio-video detector with temporal jittering and evaluate it on FakeAVCeleb clips whose leading silence has been trimmed away; if the untrimmed-versus-trimmed AUC drop stays above a few points — the paper reports near-zero deltas — the mitigation claim is wrong. A second check varies the silence detector itself, recomputing the same deltas at 10 dB and 30 dB thresholds and at 10 ms and 50 ms durations; if the shortcut pattern and its post-jittering near-zero deltas do not persist across thresholds, the leading-silence artifact is partly a construct of the chosen operational definition.

Watch

Extended reading notes

Core claim

The central claim is that a one-line training-time change makes supervised audio-video detectors stop depending on the leading-silence shortcut: sampling each clip from a random starting point during training, instead of always from the beginning, reduces the untrimmed-versus-trimmed AUC delta on FakeAVCeleb from drops of up to 46 points to at most a few points on every manipulation split, and turns most DeepSpeak v1 deltas slightly positive. On the diagnosis side, the paper shows the shortcut is broader than previously reported: every FakeAVCeleb manipulation involving fake audio carries the silence artifact, and in DeepSpeak v1 the Wav2Lip and Retalking splits show a similar though weaker imbalance. On the evaluation side, it uncovers that the established FakeAVCeleb leave-one-out protocol omitted the single-manipulation FaceSwap and FSGAN samples entirely and leaked Wav2Lip between training and test, and it states it is the first to propose an evaluation protocol for DeepSpeak v1, holding out one manipulation at a time in method splits and entire lip-synthesis or face-animation families in family splits.

Load-bearing premise

The entire shortcut diagnosis rests on the hand-chosen rule that leading silence means audio staying more than 20 dB below normal for at least 20 milliseconds at the start of a clip, and that trimming removes exactly that artifact and nothing else, so if the threshold mischaracterizes the artifact the measured performance deltas would not reflect shortcut reliance and the jittering cure could be an artifact of the preprocessing.

Editorial extensions

If this is right

  • Prior supervised scores on FakeAVCeleb should be read as partly measuring shortcut use; training with temporal jittering and reporting trimmed-video performance yields the honest numbers.
  • Because the mitigation is an augmentation rather than an architectural change, it can be dropped into most supervised audio-video detectors without redesigning the model.
  • Family splits on both datasets are harder than method splits, so generalization claims that report only method splits are optimistic; the previously hidden FaceSwap real-audio subset is the hardest FakeAVCeleb target.
  • DeepSpeak v1's more recent manipulations (FaceFusion, FaceFusion GAN, FaceFusion Live, Retalking) form a harder and more out-of-distribution benchmark than the saturated FakeAVCeleb.
  • Cross-dataset evaluation across all manipulations, not just the shared Wav2Lip, is feasible and shows video-only models currently generalize better on average than multimodal ones, underscoring the datasets' tilt toward the video modality.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the diagnosis is right, a detector could have scored near-perfect on FakeAVCeleb by timing the start of the audio rather than detecting manipulation at all, which suggests the shortcut drove a large share of the near-saturated results published on this benchmark.
  • The 20 dB / 20 ms operationalization of leading silence is the one un-tested dial in the diagnosis; a sensitivity analysis varying both thresholds would show whether the shortcut and the jittering cure are threshold-independent or partly artifacts of the measurement.
  • The same recipe — silence detection, trimming, and jittering — could be applied to other datasets suspected of temporal boundary artifacts, such as AV-Deepfake1M or datasets with trailing silence; the paper does not test this.
  • Because multiclass SIMBA resists the shortcut on some splits even without jittering, forcing distinct manipulation clusters during training may itself discourage shortcut reliance; this design lesson is visible in the tables but is not isolated as a separate experiment.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper examines benchmarking issues in audio-video DeepFake detection along three axes: datasets, methods, and evaluation protocols. It diagnoses the leading-silence shortcut in FakeAVCeleb (extending prior work to all manipulations) and analyzes DeepSpeak v1, introducing the first evaluation protocol for that dataset. The authors propose SIMBA, a simple multimodal baseline, and show that temporal jittering during training reduces performance drops on trimmed videos. They also propose method and family splits for cross-manipulation evaluation and carry out cross-dataset experiments between FakeAVCeleb and DeepSpeak v1.

Significance. If the empirical claims hold, the paper makes a useful contribution: it provides a new benchmark protocol for a recent dataset, exposes blind spots and leakage in the established FakeAVCeleb leave-one-out protocol, and offers a simple and potentially transferable mitigation for a known shortcut. The temporal jittering result is striking and actionable, and the proposed splits are more realistic than previous practice. However, the strength of the central claims is currently limited by reproducibility gaps: the silence-trimming procedure is not fully specified, results are single-run with no error bars, and no code or exact splits are released. These issues are fixable within the scope of the manuscript, but they must be addressed before the quantitative conclusions can be accepted.

major comments (4)
  1. [§3.1, §6.3, Tables 2–3] The central quantitative evidence relies on comparing AUC on untrimmed versus trimmed videos, yet the trimming procedure is never fully specified. Section 3.1 only says that silence is defined by a 20 dB threshold lasting at least 20 ms; it does not state whether the threshold is applied to RMS or peak amplitude, how the leading silence boundary is located, whether only the audio is trimmed or the video is trimmed as well, how audio–video synchronization is maintained after trimming, or what is done for samples with no detected silence. Without this specification, the delta values in Tables 2 and 3 cannot be reproduced and could reflect preprocessing artifacts rather than shortcut reliance. Please provide the exact algorithm, including parameter choices, and consider releasing the trimming code.
  2. [§6.3, Tables 2–3] The conclusion that temporal jittering reduces the trim-induced drop to 'an almost insignificant delta' is not supported statistically. All reported AUCs are single-run values with no error bars or confidence intervals, and the significance thresholds differ between tables (>10 in Table 2, >3 in Table 3) without justification. Moreover, several deltas in the jittered models are non-negligible, e.g., -4.14 on the FSGAN split in Table 2 and -2.07 on the FaceFusion split in Table 3; these are the same order as some deltas that are flagged as significant in the other table. The paper should report results over multiple seeds (at least 3), provide error bars or standard deviations, and use a consistent, pre-specified criterion for what counts as significant.
  3. [§6.4–6.5, Figures 5–6] The benchmark comparisons with prior models (LipForensics, RealForensics, AVAD, AVFF) rely on numbers taken from published papers rather than on running those models under the new protocols in this paper. This is acknowledged in the Limitations, but it undermines the fairness of the cross-manipulation and cross-dataset comparisons, because different training setups, preprocessing, and sampling strategies may affect the results. For example, in Figure 6, unimodal models are assigned 50% AUC on realVideo-fakeAudio splits by construction; this is a convention, not a measurement, and should be stated clearly. Please either rerun the prior methods under the exact proposed protocol or clearly mark all results that are taken from other papers and discuss the potential for protocol mismatch.
  4. [§5, §A, Figures 3–4] The proposed method and family splits are a valuable contribution, but the paper does not release the exact split definitions, and the construction details are incomplete. In particular, Section A states that splits are identity-based for FakeAVCeleb but does not specify how identity overlaps are handled in the DeepSpeak v1 splits, which use a provided test set. For reproducibility, the exact file lists or split-generation code should be released, and the 'method' grouping (e.g., FaceSwap and FaceSwap+Wav2Lip as one method) should be justified in terms of what artifacts are shared and what the grouping means for audio-only vs. video-only manipulations.
minor comments (5)
  1. [Throughout] There are inconsistent spellings such as 'FakeA VCeleb' and '20db'; fix to 'FakeAVCeleb' and '20 dB' for consistency.
  2. [§6.3, Tables 2–3] The caption states that positive deltas are shown in green in Table 3, but the text in §6.3 describes only negative deltas as significant; please clarify the exact color-coding rule and use the same significance criterion across both tables.
  3. [§6.1, Appendix C] The audio preprocessing keeps different sampling rates for the two datasets (16 kHz for FakeAVCeleb and 48 kHz for DeepSpeak v1); this is a design choice that should be discussed, since it may affect the audio encoder's behavior in the cross-dataset evaluation.
  4. [§4, Figure 2] The notation for the fusion layer is clear, but the multiclass variant is described only in the caption; consider adding a sentence in the main text explaining how the multiclass head is supervised when some manipulation types are combined (e.g., FaceSwap+Wav2Lip).
  5. [Appendix F] In the last sentence, 'beginningschema' should read 'beginning schema'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central claims are empirical, held-out generalization tests with no fitted-input-as-prediction steps.

full rationale

The paper's central claims are empirical rather than derivational: (1) the proposed method/family evaluation splits are genuine held-out generalization tests, with training/validation/test disjoint by manipulation, and are benchmarked against external SOTA models; (2) the silence-shortcut diagnosis is an operational diagnostic (20 dB / 20 ms leading silence) and the mitigation claim is supported by comparing AUC deltas between untrimmed and trimmed videos across SIMBA variants, where trimming removes the defined artifact rather than a fitted parameter; (3) temporal jittering is a training-time augmentation whose benefit is measured on held-out trimmed data, not a quantity fitted to the same test deltas and then reported as a prediction; and (4) the proposed DeepSpeak v1 protocol is presented as a first proposal and is evaluated independently. The only self-citation is reference [24], co-authored by Rohrbach, used to support the general statement that deepfake detectors overfit to training artifacts; this is motivational and not load-bearing for the quantitative results. Concerns raised about the unspecified trimming procedure and single-run AUCs are reproducibility and statistical-support issues, not circularity: even if the trim specification were incomplete, the comparison would not reduce by construction to the paper's own inputs. No equation or claimed result is equivalent to its input by definition, and no fitted constant is renamed as a prediction. Therefore the appropriate finding is no significant circularity.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

This is an empirical benchmarking paper. The only hand-chosen number that directly shapes the central shortcut analysis is the silence threshold; the other choices (N=16 frames, step M=5, learning rate, etc.) are standard hyperparameters. The evaluation protocol rests on domain assumptions about what the splits measure. There are no invented physical or conceptual entities beyond the SIMBA architecture, which is a combination of known components.

free parameters (1)
  • Silence detection threshold = 20 dB below reference, minimum 20 ms duration
    Hand-chosen operationalization used to define the leading-silence shortcut and to create trimmed videos for evaluation; the shortcut analysis would shift if this threshold were different.
assumptions (3)
  • domain assumption The 20 dB / 20 ms silence definition captures the dataset shortcut rather than a benign artifact
    The shortcut analysis in Section 3.1 and the mitigation evaluation in Section 6.3 rest on this measurement assumption; it is not validated against an external standard.
  • domain assumption Method and family splits are meaningful proxies for real-world generalization
    The proposed evaluation protocols in Section 5 assume that holding out a manipulation method, or a family, measures generalization to unseen manipulations rather than some confound.
  • domain assumption Reported SOTA results from prior papers are comparable to the authors' re-runs
    Section 6.2 compares SIMBA to published numbers for prior models; comparability depends on consistent preprocessing and protocol, which cannot be fully guaranteed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DeepFake Doctor: Diagnosing and Treating Audio-Video Fake Detection." pith.science (2026). https://pith.science/paper/BSH6MQMF

@misc{pith2026250605851,
  author       = {Pith},
  title        = {Pith review of: DeepFake Doctor: Diagnosing and Treating Audio-Video Fake Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BSH6MQMF}},
  note         = {Machine review of arXiv:2506.05851}
}
read the original abstract

Generative AI advances rapidly, allowing the creation of very realistic manipulated video and audio. This progress presents a significant security and ethical threat, as malicious users can exploit DeepFake techniques to spread misinformation. Recent DeepFake detection approaches explore the multimodal (audio-video) threat scenario. In particular, there is a lack of reproducibility and critical issues with existing datasets - such as the recently uncovered silence shortcut in the widely used FakeAVCeleb dataset. Considering the importance of this topic, we aim to gain a deeper understanding of the key issues affecting benchmarking in audio-video DeepFake detection. We examine these challenges through the lens of the three core benchmarking pillars: datasets, detection methods, and evaluation protocols. To address these issues, we spotlight the recent DeepSpeak v1 dataset and are the first to propose an evaluation protocol and benchmark it using SOTA models. We introduce SImple Multimodal BAseline (SIMBA), a competitive yet minimalistic approach that enables the exploration of diverse design choices. We also deepen insights into the issue of audio shortcuts and present a promising mitigation strategy. Finally, we analyze and enhance the evaluation scheme on the widely used FakeAVCeleb dataset. Our findings offer a way forward in the complex area of audio-video DeepFake detection.

Figures

Figures reproduced from arXiv: 2506.05851 by the authors.

Figure 1
Figure 1. Leading silence distribution of the FAVC and DeepSpeak v1 datasets. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. SIMBA is composed of two encoders, green for audio and blue for the video. A self￾attention and a max pooling layer follow the audio encoder. Each encoder has a modality-specific classifier on top. In the fusion, the embedding vectors are concatenated, ⊕, followed by a fusion layer and a multimodal classifier. We show both the binary and a multiclass variant. (Best viewed in color.) this research field, this discove… view at source ↗
Figure 3
Figure 3. (a) The established vs. (b) our proposed cross-manipulation generalization evaluation for [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: The proposed evaluation protocol for DeepSpeak v1. The first five rows show method splits, the last two rows are the family splits. Similarly, we propose the same evaluation concept for DeepSpeak v1, depicted in [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Cross-manipulation comparison using the proposed methods splits (as AUC). [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Cross-dataset evaluation across all manipulations (as AUC) [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Number of videos for each method split in training, validation, and test for FakeAVCeleband [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: Cross-manipulation comparison using the proposed family splits (as AUC). [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]
Figure 9
Figure 9. Figure 9: Visualization of the embedding space for SIMBA binary and multiclass on FakeAVCeleb. [PITH_FULL_IMAGE:figures/full_fig_p019_9.png]
Figure 10
Figure 10. Figure 10: Visualization of the embedding space for SIMBA binary and multiclass on DeepSpeak v1. [PITH_FULL_IMAGE:figures/full_fig_p021_10.png]
Figure 11
Figure 11. Figure 11: Examples of the video manipulation types in FakeAVCeleb. [PITH_FULL_IMAGE:figures/full_fig_p022_11.png]
Figure 12
Figure 12. Figure 12: Examples of the video manipulation types in DeepSpeak v1. [PITH_FULL_IMAGE:figures/full_fig_p023_12.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

53 extracted references · 28 canonical work pages

  1. [1]

    Senior, Oriol Vinyals, and Andrew Zisser- man

    Triantafyllos Afouras, Joon Son Chung, Andrew W. Senior, Oriol Vinyals, and Andrew Zisser- man. Deep audio-visual speech recognition.IEEE Trans. Pattern Anal. Mach. Intell., 44(12): 8717–8727, 2022. doi: 10.1109/TPAMI.2018.2889052. URL https://doi.org/10.1109/ TPAMI.2018.2889052

    arXiv 2022

  2. [2]

    Detecting deep-fake videos from phoneme-viseme mismatches

    Shruti Agarwal, Hany Farid, Ohad Fried, and Maneesh Agrawala. Detecting deep-fake videos from phoneme-viseme mismatches. In2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR Workshops 2020, Seattle, WA, USA, June 14-19, 2020, pages 2814–

    work page 2020

  3. [3]

    Deepspeak dataset v1.0.CoRR, abs/2408.05366, 2024

    Sarah Barrington, Matyas Bohacek, and Hany Farid. Deepspeak dataset v1.0.CoRR, abs/2408.05366, 2024. doi: 10.48550/ARXIV .2408.05366. URL https://doi.org/10. 48550/arXiv.2408.05366

    arXiv doi:10.48550/arxiv.2408.05366 2024

  4. [4]

    Lost in translation: Lip-sync deepfake detection from audio- video mismatch

    Matyas Bohacek and Hany Farid. Lost in translation: Lip-sync deepfake detection from audio- video mismatch. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pages 4315–4323, June 2024

    work page 2024

  5. [5]

    Circumventing shortcuts in audio-visual deepfake detection datasets with unsupervised learning.To appear in CVPR, 2025

    Dragos-Alexandru Boldisor, Stefan Smeu, Dan Oneata, and Elisabeta Oneata. Circumventing shortcuts in audio-visual deepfake detection datasets with unsupervised learning.To appear in CVPR, 2025

    work page 2025

  6. [6]

    Do you really mean that? content driven audio-visual deepfake dataset and multimodal method for temporal forgery localization

    Zhixi Cai, Kalin Stefanov, Abhinav Dhall, and Munawar Hayat. Do you really mean that? content driven audio-visual deepfake dataset and multimodal method for temporal forgery localization. In2022 International Conference on Digital Image Computing: Techniques and Applications (DICTA), pages 1–10. IEEE, 2022

    work page

  7. [7]

    Av-deepfake1m: A large-scale llm-driven audio-visual deepfake dataset

    Zhixi Cai, Shreya Ghosh, Aman Pankaj Adatia, Munawar Hayat, Abhinav Dhall, Tom Gedeon, and Kalin Stefanov. Av-deepfake1m: A large-scale llm-driven audio-visual deepfake dataset. In Jianfei Cai, Mohan S. Kankanhalli, Balakrishnan Prabhakaran, Susanne Boll, Ramanathan Subramanian, Liang Zheng, Vivek K. Singh, Pablo César, Lexing Xie, and Dong Xu, editors, P...

    arXiv 2024

  8. [8]

    Trusted media challenge dataset and user study

    Weiling Chen, Sheng Lun Benjamin Chua, Stefan Winkler, and See-Kiong Ng. Trusted media challenge dataset and user study. In Mohammad Al Hasan and Li Xiong, editors,Proceedings of the 31st ACM International Conference on Information & Knowledge Management, Atlanta, GA, USA, October 17-21, 2022, pages 3873–3877. ACM, 2022. doi: 10.1145/3511808.3557715. URLh...

    arXiv 2022

Show all 53 references
  1. [9]

    V oice-face homogeneity tells deepfake.ACM Trans

    Harry Cheng, Yangyang Guo, Tianyi Wang, Qi Li, Xiaojun Chang, and Liqiang Nie. V oice-face homogeneity tells deepfake.ACM Trans. Multim. Comput. Commun. Appl., 20(3):76:1–76:22,

  2. [10]

    Videoretalking: Audio-based lip synchronization for talking head video editing in the wild

    Kun Cheng, Xiaodong Cun, Yong Zhang, Menghan Xia, Fei Yin, Mingrui Zhu, Xuan Wang, Jue Wang, and Nannan Wang. Videoretalking: Audio-based lip synchronization for talking head video editing in the wild. In Soon Ki Jung, Jehee Lee, and Adam W. Bargteil, editors, SIGGRAPH Asia 20...

    2022

  3. [11]

    Not made for each other- audio-visual dissonance-based deepfake detection and localization

    Komal Chugh, Parul Gupta, Abhinav Dhall, and Ramanathan Subramanian. Not made for each other- audio-visual dissonance-based deepfake detection and localization. In Chang Wen Chen, Rita Cucchiara, Xian-Sheng Hua, Guo-Jun Qi, Elisa Ricci, Zhengyou Zhang, and Roger Zimmermann, ed...

    2020

  4. [12]

    J. S. Chung, A. Nagrani, and A. Zisserman. V oxceleb2: Deep speaker recognition. InINTER- SPEECH, 2018

    2018

  5. [13]

    The deepfake detection challenge (DFDC) preview dataset.CoRR, abs/1910.08854, 2019

    Brian Dolhansky, Russ Howes, Ben Pflaum, Nicole Baram, and Cristian Canton-Ferrer. The deepfake detection challenge (DFDC) preview dataset.CoRR, abs/1910.08854, 2019. URL http://arxiv.org/abs/1910.08854

    1910 arXiv

  6. [14]

    ElevenLabs: Free Text to Speech & AI Voice Generator

    ElevenLabs. ElevenLabs: Free Text to Speech & AI Voice Generator. https://elevenlabs. io/, 2024. [Accessed 10-11-2024]

    2024

  7. [15]

    Self-supervised video forensics by audio-visual anomaly detection

    Chao Feng, Ziyang Chen, and Andrew Owens. Self-supervised video forensics by audio-visual anomaly detection. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10491–10503, 2023

    2023

  8. [16]

    I. J. Good. Rational decisions.Journal of the Royal Statistical Society. Series B (Methodological), 14(1):107–114, 1952. ISSN 00359246. URLhttp://www.jstor.org/stable/2984087

    1952

  9. [17]

    Spatiotemporal inconsistency learning for deepfake video detection

    Zhihao Gu, Yang Chen, Taiping Yao, Shouhong Ding, Jilin Li, Feiyue Huang, and Lizhuang Ma. Spatiotemporal inconsistency learning for deepfake video detection. In Heng Tao Shen, Yueting Zhuang, John R. Smith, Yang Yang, Pablo César, Florian Metze, and Balakrishnan Prabhakaran, ...

    2021

  10. [19]

    Leveraging real talking faces via self-supervision for robust forgery detection

    Alexandros Haliassos, Rodrigo Mira, Stavros Petridis, and Maja Pantic. Leveraging real talking faces via self-supervision for robust forgery detection. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14930–14942. IEEE, 2022. doi: 10.1109/CVPR52688. 2022...

    2022

  11. [20]

    Forgerynet: A versatile benchmark for comprehensive forgery analysis

    Yinan He, Bei Gan, Siyu Chen, Yichun Zhou, Guojun Yin, Luchuan Song, Lu Sheng, Jing Shao, and Ziwei Liu. Forgerynet: A versatile benchmark for comprehensive forgery analysis. InIEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 4360–4369. Computer Vision ...

    2021

  12. [21]

    Polyglotfake: A novel multilingual and multimodal deepfake dataset.CoRR, abs/2405.08838, 2024

    Yang Hou, Haitao Fu, Chuankai Chen, Zida Li, Haoyu Zhang, and Jianjun Zhao. Polyglotfake: A novel multilingual and multimodal deepfake dataset.CoRR, abs/2405.08838, 2024. doi: 10.48550/ARXIV .2405.08838. URLhttps://doi.org/10.48550/arXiv.2405.08838

    2024 doi arXiv

  13. [22]

    Deepfakes and their impact on society, Feb 2024

    Neill Jacobson. Deepfakes and their impact on society, Feb 2024. URL https://www.openfox.com/deepfakes-and-their-impact-on-society/#:~:text= The%20Growth%20of%20Deepfakes,social%20media%20around%20the%20world

    2024

  14. [23]

    Weiss, Quan Wang, Jonathan Shen, Fei Ren, Zhifeng Chen, Patrick Nguyen, Ruoming Pang, Ignacio López-Moreno, and Yonghui Wu

    Ye Jia, Yu Zhang, Ron J. Weiss, Quan Wang, Jonathan Shen, Fei Ren, Zhifeng Chen, Patrick Nguyen, Ruoming Pang, Ignacio López-Moreno, and Yonghui Wu. Transfer learning from speaker verification to multispeaker text-to-speech synthesis. In Samy Bengio, Hanna M. Wallach, Hugo Lar...

    2018

  15. [24]

    Revisiting generalizability in deepfake detection: Improving metrics and stabilizing transfer

    Sarthak Kamat, Shruti Agarwal, Trevor Darrell, and Anna Rohrbach. Revisiting generalizability in deepfake detection: Improving metrics and stabilizing transfer. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 426–435, 2023

    2023

  16. [25]

    Hasam Khalid, Shahroz Tariq, Minha Kim, and Simon S. Woo. FakeA VCeleb: A novel audio- video multimodal deepfake dataset. InThirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2021. URL https://openreview.net/ forum?id=TAXFsg6ZaOl

    2021

  17. [26]

    Fast face-swap using con- volutional neural networks

    Iryna Korshunova, Wenzhe Shi, Joni Dambre, and Lucas Theis. Fast face-swap using con- volutional neural networks. InIEEE International Conference on Computer Vision, ICCV, pages 3697–3705. IEEE Computer Society, 2017. doi: 10.1109/ICCV .2017.397. URL https://doi.org/10.1109/IC...

    2017 doi

  18. [27]

    Dimodif: Discourse modality-information differen- tiation for audio-visual deepfake detection and localization.CoRR, abs/2411.10193, 2024

    Christos Koutlis and Symeon Papadopoulos. Dimodif: Discourse modality-information differen- tiation for audio-visual deepfake detection and localization.CoRR, abs/2411.10193, 2024. doi: 10.48550/ARXIV .2411.10193. URLhttps://doi.org/10.48550/arXiv.2411.10193

    2024 doi arXiv

  19. [28]

    Kodf: A large-scale korean deepfake detection dataset

    Patrick Kwon, Jaeseong You, Gyuhyeon Nam, Sungwoo Park, and Gyeongsu Chae. Kodf: A large-scale korean deepfake detection dataset. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 10744–10753, October 2021

    2021

  20. [30]

    Speechforensics: Audio-visual speech represen- tation learning for face forgery detection

    Yachao Liang, Min Yu, Gang Li, Jianguo Jiang, Boquan Li, Feng Yu, Ning Zhang, Xiang Meng, and Weiqing Huang. Speechforensics: Audio-visual speech represen- tation learning for face forgery detection. In A. Globerson, L. Mackey, D. Bel- grave, A. Fan, U. Paquet, J. Tomczak, and...

    2024

  21. [31]

    Lips are lying: Spotting the temporal inconsistency between audio and visual in lip-syncing deepfakes

    Weifeng Liu, Tianyi She, Jiawei Liu, Boheng Li, Dongyu Yao, Ziyou Liang, and Run Wang. Lips are lying: Spotting the temporal inconsistency between audio and visual in lip-syncing deepfakes. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, e...

    2024

  22. [32]

    Emotions don’t lie: An audio-visual deepfake detection method using affective cues

    Trisha Mittal, Uttaran Bhattacharya, Rohan Chandra, Aniket Bera, and Dinesh Manocha. Emotions don’t lie: An audio-visual deepfake detection method using affective cues. InACM International Conference on Multimedia, 2020

    2020

  23. [33]

    Df-platter: Multi-face heterogeneous deepfake dataset

    Kartik Narayan, Harsh Agarwal, Kartik Thakral, Surbhi Mittal, Mayank Vatsa, and Richa Singh. Df-platter: Multi-face heterogeneous deepfake dataset. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9739–9748. IEEE, 2023. doi: 10.1109/CVPR52729. 2023.00939...

    2023

  24. [34]

    BYOL for audio: Self-supervised learning for general-purpose audio representation

    Daisuke Niizumi, Daiki Takeuchi, Yasunori Ohishi, Noboru Harada, and Kunio Kashino. BYOL for audio: Self-supervised learning for general-purpose audio representation. InInternational Joint Conference on Neural Networks, IJCNN 2021, Shenzhen, China, July 18-22, 2021, pages 1–8....

    2021

  25. [35]

    FSGAN: subject agnostic face swapping and reenactment

    Yuval Nirkin, Yosi Keller, and Tal Hassner. FSGAN: subject agnostic face swapping and reenactment. In2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019, pages 7183–7192. IEEE, 2019. doi: 10.1109/ICCV .2019....

    2019

  26. [36]

    A VFF: audio-visual feature fusion for video deep- fake detection

    Trevine Oorloff, Surya Koppisetti, Nicolò Bonettini, Divyaraj Solanki, Ben Colman, Yaser Yacoob, Ali Shahriyari, and Gaurav Bharaj. A VFF: audio-visual feature fusion for video deep- fake detection. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024, S...

    2024

  27. [37]

    K. R. Prajwal, Rudrabha Mukhopadhyay, Vinay P. Namboodiri, and C. V . Jawahar. A lip sync expert is all you need for speech to lip generation in the wild. In Chang Wen Chen, Rita Cucchiara, Xian-Sheng Hua, Guo-Jun Qi, Elisa Ricci, Zhengyou Zhang, and Roger Zimmermann, editors,...

    2020

  28. [38]

    Multimodaltrace: Deepfake detection using audiovisual representation learning

    Muhammad Anas Raza and Khalid Mahmood Malik. Multimodaltrace: Deepfake detection using audiovisual representation learning. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023 - Workshops, Vancouver, BC, Canada, June 17-24, 2023, pages 993–1000. IEEE, 2...

    2023

  29. [39]

    Faceforensics++: Learning to detect manipulated facial images

    Andreas Rössler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, and Matthias Nießner. Faceforensics++: Learning to detect manipulated facial images. In2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 - No...

    2019

  30. [40]

    Facefusion.https://github.com/facefusion/facefusion, 2024

    Henry Ruhs. Facefusion.https://github.com/facefusion/facefusion, 2024

    2024

  31. [41]

    Asha S, Vinod P, and Varun G. Menon. MMDFD- A multimodal custom dataset for deepfake detection. InProceedings of the 2023 Fifteenth International Conference on Contemporary Computing, IC3-2023, Noida, India, August 3-5, 2023, pages 322–327. ACM, 2023. doi: 10.1145/3607947.3608...

    2023

  32. [42]

    Unsupervised multi- modal deepfake detection using intra- and cross-modal inconsistencies.CoRR, abs/2311.17088,

    Mulin Tian, Mahyar Khayatkhoei, Joe Mathai, and Wael AbdAlmageed. Unsupervised multi- modal deepfake detection using intra- and cross-modal inconsistencies.CoRR, abs/2311.17088,

    arXiv

  33. [43]

    A closer look at spatiotemporal convolutions for action recognition

    Du Tran, Heng Wang, Lorenzo Torresani, Jamie Ray, Yann LeCun, and Manohar Paluri. A closer look at spatiotemporal convolutions for action recognition. In2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, pages...

    2018

  34. [44]

    Deepfakes audio detection leveraging audio spectrogram and convolutional neural networks

    Taiba Majid Wani and Irene Amerini. Deepfakes audio detection leveraging audio spectrogram and convolutional neural networks. In Gian Luca Foresti, Andrea Fusiello, and Edwin R. Hancock, editors,Image Analysis and Processing - ICIAP 2023 - 22nd International Conference, ICIAP ...

    2023 doi

  35. [45]

    Abc-capsnet: Attention based cascaded capsule network for audio deepfake detection

    Taiba Majid Wani, Reeva Gulzar, and Irene Amerini. Abc-capsnet: Attention based cascaded capsule network for audio deepfake detection. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pages 2464–2472, June 2024

    2024

  36. [46]

    Audio deepfake detection system with neural stitching for ADD 2022

    Rui Yan, Cheng Wen, Shuran Zhou, Tingwei Guo, Wei Zou, and Xiangang Li. Audio deepfake detection system with neural stitching for ADD 2022. InIEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2022, Virtual and Singapore, 23-27 May 2022, pages 922...

    2022

  37. [47]

    Avoid-df: Audio-visual joint learning for detecting deepfake.IEEE Trans

    Wenyuan Yang, Xiaoyu Zhou, Zhikai Chen, Bofei Guo, Zhongjie Ba, Zhihua Xia, Xiaochun Cao, and Kui Ren. Avoid-df: Audio-visual joint learning for detecting deepfake.IEEE Trans. Inf. Forensics Secur., 18:2015–2029, 2023. doi: 10.1109/TIFS.2023.3262148. URL https://doi.org/10.110...

    2015

  38. [48]

    Joint face detection and alignment using multitask cascaded convolutional networks.IEEE signal processing letters, 23(10): 1499–1503, 2016

    Kaipeng Zhang, Zhanpeng Zhang, Zhifeng Li, and Yu Qiao. Joint face detection and alignment using multitask cascaded convolutional networks.IEEE signal processing letters, 23(10): 1499–1503, 2016

    2016

  39. [49]

    Exploring temporal coherence for more general video face forgery detection

    Yinglin Zheng, Jianmin Bao, Dong Chen, Ming Zeng, and Fang Wen. Exploring temporal coherence for more general video face forgery detection. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 15044–15054, 2021

    2021

  40. [50]

    Shangchen Zhou, Kelvin C. K. Chan, Chongyi Li, and Chen Change Loy. Towards ro- bust blind face restoration with codebook lookup transformer. In Sanmi Koyejo, S. Mo- hamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh, editors,Advances in Neu- ral Information Processing Sy...

    2022

  41. [51]

    real”/“fake

    Yipin Zhou and Ser-Nam Lim. Joint audio-visual deepfake detection. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 14800–14809, October 2021. 14 DeepFake Doctor: Diagnosing and Treating Audio-Video Fake Detection Supplementary Material T...

    2021

  42. [2020]

    URLhttps://arxiv.org/abs/2005.00214

    2005 arXiv

  43. [2023]

    URL https://doi.org/10.48550/arXiv.2311

    doi: 10.48550/ARXIV .2311.17088. URL https://doi.org/10.48550/arXiv.2311. 17088

    doi arXiv

  44. [2024]

    URLhttps://doi.org/10.1145/3625231

    doi: 10.1145/3625231. URLhttps://doi.org/10.1145/3625231

    doi

  45. [2822]

    doi: 10.1109/CVPRW50498.2020.00338

    Computer Vision Foundation / IEEE, 2020. doi: 10.1109/CVPRW50498.2020.00338. URL https://openaccess.thecvf.com/content_CVPRW_2020/html/w39/Agarwal_ Detecting_Deep-Fake_Videos_From_Phoneme-Viseme_Mismatches_CVPRW_2020_ paper.html

    2020

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.