Pith. sign in

REVIEW 4 major objections 4 minor 49 references

Lightweight Joint Audio-Visual Deepfake Detection via Single-Stream Multi-Modal Learning Framework

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A single-stream network that fuses audio and visual features at every layer claims to beat larger two-stream deepfake detectors while using only 0.48M parameters.

desk verdict The architecture is genuinely interesting and the parameter count is striking, but the random video-level splits with identity overlap make the reported superiority and 'unseen deepfake' claims unsupported. read the letter →

arxiv 2506.07358 v1 pith:WZQH6UQZ submitted 2025-06-09 cs.SD cs.AIcs.LGeess.AS

Kuiyuan Zhang , Wenjie Pei , Rushi Lan , Yifang Guo , Zhongyun Hua This is my paper
classification cs.SDcs.AIcs.LGeess.AS
keywords audio-visualdeepfakedetectionsingle-streammulti-modallearninglightweightnetworkself-attentionfusionspatialattentionstyle-shuffleaugmentationcross-methodgeneralizationforensics
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that joint audio-visual deepfake detection does not require two separate neural networks that learn audio and visual features in isolation and fuse them only at the end. It proposes a single-stream network, SS-AVD, that fuses audio and visual features at every layer through iterative multi-modal learning blocks, and claims this design is both more accurate and far smaller than existing joint detectors. With only 0.48M parameters, the authors report state-of-the-art results on the DF-TIMIT, FakeAVCeleb, and DFDC benchmarks, including higher accuracy on audio-only, visual-only, and whole-video classification, and better cross-method generalization to unseen forgery types. The practical payoff is that a capable detector could run on resource-limited devices where larger two-stream models cannot.

What carries the argument

The Collaborative Audio-Visual Learning (CAVL) block is the central object: it combines a Visual Preprocessing Module (VPM), which applies frame-based spatial attention to visual features, with a Self-Attention-based Audio-Visual Module (SAAVM), which chunks visual frames into tokens and pools audio into matching tokens, then feeds both through a shared self-attention layer to learn spatial-temporal correlations between the modalities. Stacking CAVL blocks across four pyramid stages produces iterative audio-visual fusion throughout the network. A multi-modal classification module then adds two training strategies: Multi-Modal Style-Shuffle Augmentation (MMSSA), which shuffles feature style statistics between samples so classifiers rely on content rather than style, and Latent-Shuffle Augmentation (LSA), which pairs visual and audio features from different samples to make the whole-video classifier resistant to audio-visual mismatches. The loss combines classification, adversarial, and contrast terms.

What would settle it

Run the same SS-AVD training on FakeAVCeleb or DFDC with an identity-disjoint split where no speaker or face appears in both train and test, then compare AUC scores to the paper's random-split numbers; if the gap to the baselines collapses or the absolute AUC drops sharply, the claimed generalization does not survive identity separation.

Watch

Extended reading notes

Core claim

The central claim is that continuous cross-modal fusion during feature learning, rather than late fusion of independently learned audio and visual features, yields better deepfake detection with a fraction of the parameters. The authors report that SS-AVD, a single-stream network with 0.48M parameters, outperforms or matches state-of-the-art joint audio-visual detectors on DF-TIMIT, FakeAVCeleb, and DFDC, and also outperforms them in a cross-method evaluation where training and test deepfakes are produced by different face-swap methods. The design rests on the premise that fusing modalities throughout the network exploits their natural correlations and avoids redundant layers, which is what makes the model small and effective at the same time.

Load-bearing premise

The random video-level train and test splits assume no person appears in both training and testing, so the model's apparent generalization to unseen deepfakes could partly come from recognizing familiar identities rather than general forgery cues.

Editorial extensions

If this is right

  • A detector with 0.48M parameters is practical for mobile and edge deployment, where existing joint detectors with more than 5M parameters are too heavy.
  • Iterative multi-modal fusion can flag partial forgeries, such as fake visual with real audio or real visual with fake audio, using correlations established early in the network rather than only at the final classification stage.
  • The style-shuffle and latent-shuffle augmentations should make the classifier less dependent on compression traces or device fingerprints, improving robustness to unseen deepfake generation methods.
  • The reported results suggest the approach transfers across datasets with different manipulation types, since it leads on DF-TIMIT, FakeAVCeleb, and DFDC simultaneously.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The single-stream iterative-fusion principle is a general recipe: the same CAVL-style architecture could be applied to other audio-visual reasoning tasks, such as lip-sync verification or audio-visual speech recognition, where cross-modal correlation is the signal.
  • A natural stress test the authors do not run is a strictly identity-disjoint split; measuring performance under that condition would isolate whether the reported generalization comes from general fake traces or from speaker and face familiarity.
  • Because the paper's hyperparameter search was deliberately small, the reported 0.48M-parameter model may be a lower bound on achievable accuracy; tuning the loss weights and the style-shuffle mixing factor could push scores higher.
  • The style-shuffle and latent-shuffle ideas could be plugged into existing two-stream detectors as a drop-in training scheme, potentially improving their generalization without architectural changes.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes SS-AVD, a single-stream audio-visual deepfake detector. It uses stacked collaborative audio-visual learning (CAVL) blocks, each containing a visual preprocessing module and a self-attention-based audio-visual module, to fuse audio and visual features throughout the network rather than only at the classification stage. A multi-modal classification module with multi-modal style-shuffle augmentation (MMSSA) and latent-shuffle augmentation (LSA) is introduced to improve modality-specific and whole-video predictions. The model has only 0.48M parameters. Experiments on DF-TIMIT, FakeAVCeleb, and DFDC report accuracy/AUC values for visual, audio, and whole-video detection, and a cross-method evaluation on FakeAVCeleb is used to support a claim of generalization to unseen deepfake types. The paper concludes that SS-AVD outperforms prior audio-visual joint detectors while being substantially lighter.

Significance. If the reported results hold under a rigorous evaluation protocol, the paper makes a useful contribution: it demonstrates that a parameter-efficient detector (0.48M parameters, at least 11x smaller than the next smallest baseline) can perform iterative multimodal fusion and achieve competitive or superior performance, which is relevant for deployment on resource-constrained devices. The architecture is clearly motivated, and the ablation study gives some evidence for the contribution of each proposed component. However, the central empirical claims are currently supported only by an evaluation protocol with random video-level splits, test-set hyperparameter tuning, and single-run results with no variance estimates. Until these issues are addressed, the quantitative superiority and the 'unseen deepfake' generalization claims are not established.

major comments (4)
  1. [Splits paragraph] The evaluation protocol uses random video-level splits (0.75/0.1/0.15) and never checks for identity overlap. Since DF-TIMIT has only 32 subjects, the same subject is almost guaranteed to appear in both training and test; FakeAVCeleb and DFDC are also built from repeated actors/speakers, and the cross-method evaluation uses forgery methods applied to the same underlying real videos. Under these splits, the model can succeed by recognizing identity- or recording-specific traces rather than general forgery artifacts, which would inflate the results in Tables 1 and 3 and would invalidate the 'unseen types of deepfakes' claim. Please re-run with subject-disjoint (identity-exclusive) splits and report the degree of identity overlap in the current splits.
  2. [Table 5] The hyperparameters gamma2 and gamma3 are tuned on the DFDC test set, and the best configuration from this tuning ({0.1, 1.0}, ACC 86.48) is then used for the DFDC results reported in Table 1. This is test-set leakage: the reported DFDC gains over baselines partially reflect selection on the test set rather than an unbiased evaluation. The hyperparameters should be selected on the validation split and the test set used only once, or the DFDC numbers should be explicitly labeled as tuning results.
  3. [Comparison Methods] The authors state that Emotions and VFD results are taken directly from their original papers, while the other baselines are retrained on the authors' random splits. Because the original papers use their own train/test protocols, the entries in Table 1 are not directly comparable, and the claimed superiority over these methods (e.g., whole-video AUC improvements on FakeAVCeleb) may reflect protocol mismatch rather than model quality. All baselines should be evaluated under the same identity-disjoint splits, or entries from other protocols should be clearly marked and excluded from superiority claims.
  4. [Experiments (Tables 1, 3, 4, 5)] All results appear to be from a single training run with no error bars or multiple seeds. Many of the reported advantages are small (e.g., Table 3 FaceSwap whole-video AUC 85.39 vs 83.87 for 2+1 Stream, and Table 4 differences around 0.5-1%), so without variance estimates the 'outperforms' claim is not statistically grounded. Please report mean and standard deviation over at least three seeds.
minor comments (4)
  1. [Design of SS-AVD] In the sentence defining input shapes, 'donates' should be 'denotes'; additionally, Eq. (1) contains an unmatched parenthesis in 'P= ReLU(Projc(F''v),'.
  2. [Adversarial Loss] The phrase 'y_1/2 indicts a pseudo-label' should be 'indicates', and the definition of the pseudo-label could be made more explicit.
  3. [Datasets (DFDC)] Generating audio labels for DFDC by comparing hash values of audio tracks is a heuristic that may introduce label noise; please discuss its accuracy or provide a validation on a subset with known audio manipulations.
  4. [Table 4] The checkmark/cross notation in the ablation table is hard to read; using explicit 'with/without' columns or a legend would improve clarity.

Circularity Check

1 steps flagged · score 2.0 of 10

Minor circularity: γ2/γ3 are tuned on the DFDC test set before the DFDC results are reported; otherwise the derivation is self-contained.

  1. fitted input called prediction [Experiments, Hyperparameters subsection and Table 5; Eq. (14); Table 1 (DFDC column)]
    "We fine-tune the hyperparameters {γ1, γ2, γ3} to assess the sensitivity of the model's performance to them. ... As illustrated in Table 5, our method exhibits sensitivity to these hyperparameters. ... While our default settings ({0.1,1.0}) may not be optimal, they indicate that the performance of our method can be enhanced further through improved hyperparameter combinations. Table 5: ACC (%) scores of entire video detection on DFDC."

    The final loss weights (γ2=0.1, γ3=1.0) in Eq. (14) are selected by comparing ACC on the DFDC test set in Table 5, where (0.1,1.0) gives the highest whole-video ACC (86.48). Table 1 then reports SS-AVD's DFDC whole-video result (86.55/93.61) as an independent benchmark superiority. The DFDC test set is therefore not held out for these hyperparameters; part of the reported gain over baselines on DFDC is a test-set selection artifact rather than an independent prediction. This is a minor, localized form of fitted-input-called-prediction, not a construction-level circularity of the model itself.

full rationale

The paper's central derivation—the SS-AVD architecture, CAVL block, MMSSA/LSA augmentation strategies, and the combined loss—is not circular: no component is defined in terms of the evaluation metrics, and no load-bearing result is imported from the authors' own prior work. The evaluation uses external benchmarks with held-out test partitions, so the model architecture is not fitted to test labels. The one concrete circular/leakage element is the selection of loss weights γ2 and γ3 on the DFDC test set before reporting the DFDC benchmark numbers: Table 5 measures ACC on the DFDC test set, and the final settings (0.1,1.0) are the best tested configuration on that set; Table 1 then reports DFDC superiority with those settings. This makes part of the DFDC comparison a test-set selection result rather than an independent prediction, but it does not by construction determine the DF-TIMIT or FakeAVCeleb results, nor the cross-method generalization results. The identity-overlap concern with random video-level splits is a generalization/validity risk, not a circularity, so it is not counted in the score. Overall, the circularity is minor and localized.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The paper contributes an architecture and training recipe, not a derivation. What it pulls from upstream are standard deep learning components, the three datasets, and a proxy labeling method for DFDC audio. The main 'unpaid' inputs are the hand-chosen and test-set-tuned hyperparameters and the assumptions about split cleanliness and label fidelity.

free parameters (5)
  • gamma2, adversarial loss weight = 0.1 (selected on DFDC test set)
    In Table 5, the combination (gamma2=0.1, gamma3=1.0) gives the highest whole-video ACC of 86.48 on the DFDC test set and is adopted as the default.
  • gamma3, contrast loss weight = 1.0 (selected on DFDC test set)
    Same selection process as gamma2; part of the best-performing pair in Table 5.
  • beta, LSA loss weight = 0.5
    Hand-chosen in 'Experiment Settings'; no sensitivity analysis is shown.
  • alpha, contrast margin = 0.4
    Hand-chosen in 'Experiment Settings'.
  • Architecture hyperparameters (stages, blocks per stage, channels, Cm) = [2,2,6,2] blocks, [8,16,32,64] channels, stages=4, Cm=1
    Selected by the authors without a reported search; these determine the 0.48M parameter count and the accuracy.
assumptions (3)
  • domain assumption The three benchmark datasets (DF-TIMIT, FakeAVCeleb, DFDC) plus VoxCeleb2 real videos provide valid and representative samples of audio-visual deepfakes.
    The paper relies entirely on these datasets to support its generalization claims; if the datasets or the added real videos are not representative, the reported superiority may not transfer.
  • domain assumption The DFDC audio labels produced by comparing hash values of audio tracks are accurate enough for per-modality evaluation.
    Section 'Datasets' states that audio labels are generated via hash comparison (Hosler et al. 2021); label noise from this proxy affects the audio and whole-video metrics for all methods compared on DFDC.
  • domain assumption Random video-level splitting prevents identity leakage between training and test sets.
    The 'Splits' paragraph describes random selection and split; identity overlap could inflate performance, and this is not checked.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Lightweight Joint Audio-Visual Deepfake Detection via Single-Stream Multi-Modal Learning Framework." pith.science (2026). https://pith.science/paper/WZQH6UQZ

@misc{pith2026250607358,
  author       = {Pith},
  title        = {Pith review of: Lightweight Joint Audio-Visual Deepfake Detection via Single-Stream Multi-Modal Learning Framework},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WZQH6UQZ}},
  note         = {Machine review of arXiv:2506.07358}
}
read the original abstract

Deepfakes are AI-synthesized multimedia data that may be abused for spreading misinformation. Deepfake generation involves both visual and audio manipulation. To detect audio-visual deepfakes, previous studies commonly employ two relatively independent sub-models to learn audio and visual features, respectively, and fuse them subsequently for deepfake detection. However, this may underutilize the inherent correlations between audio and visual features. Moreover, utilizing two isolated feature learning sub-models can result in redundant neural layers, making the overall model inefficient and impractical for resource-constrained environments. In this work, we design a lightweight network for audio-visual deepfake detection via a single-stream multi-modal learning framework. Specifically, we introduce a collaborative audio-visual learning block to efficiently integrate multi-modal information while learning the visual and audio features. By iteratively employing this block, our single-stream network achieves a continuous fusion of multi-modal features across its layers. Thus, our network efficiently captures visual and audio features without the need for excessive block stacking, resulting in a lightweight network design. Furthermore, we propose a multi-modal classification module that can boost the dependence of the visual and audio classifiers on modality content. It also enhances the whole resistance of the video classifier against the mismatches between audio and visual modalities. We conduct experiments on the DF-TIMIT, FakeAVCeleb, and DFDC benchmark datasets. Compared to state-of-the-art audio-visual joint detection methods, our method is significantly lightweight with only 0.48M parameters, yet it achieves superiority in both uni-modal and multi-modal deepfakes, as well as in unseen types of deepfakes.

Figures

Figures reproduced from arXiv: 2506.07358 by the authors.

Figure 1
Figure 1. Illustration of two feature learning frameworks in [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of our SS-AVD. Given the input visual and audio clips, our SS-AVD sequentially fuses the visual and audio [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Architecture of the visual preprocessing module. [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Architecture of the self-attention-based audio-visual module. [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 3
Figure 3. Figure 3: Assuming that the input video features is [PITH_FULL_IMAGE:figures/full_fig_p004_3.png]
Figure 5
Figure 5. Figure 5: Visualization of the regions focused on by our SS-VAD for prediction. The figures from the top to bottom rows are [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: t-SNE results of the latent features on the test set of FakeAVCeleb. [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

49 extracted references · 40 canonical work pages

  1. [1]

    Arora, S.; Hu, W.; and Kothari, P. K. 2018. An analysis of the t-sne algorithm for data visualization. In Conference on Learning Theory, 1455--1462

    work page 2018

  2. [2]

    Cai, Z.; Stefanov, K.; Dhall, A.; and Hayat, M. 2022. Do You Really Mean That ? Content Driven Audio - Visual Deepfake Dataset and Multimodal Method for Temporal Forgery Localization . In 2022 International Conference on Digital Image Computing : Techniques and Applications ( DICTA ) , 1--10

    work page 2022

  3. [3]

    D.; Junior, A

    Casanova, E.; Weber, J.; Shulby, C. D.; Junior, A. C.; G \"o lge, E.; and Ponti, M. A. 2022. Yourtts: Towards zero-shot multi-speaker tts and zero-shot voice conversion for everyone. In International Conference on Machine Learning, 2709--2720

    work page 2022

  4. [4]

    Cheng, H.; Guo, Y.; Wang, T.; Li, Q.; Chang, X.; and Nie, L. 2022. Voice-face homogeneity tells deepfake. arXiv preprint arXiv:2203.02195

    work page Pith review arXiv 2022

  5. [5]

    Chollet, F. 2017. Xception: Deep learning with depthwise separable convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 1251--1258

    work page 2017

  6. [6]

    Chugh, K.; Gupta, P.; Dhall, A.; and Subramanian, R. 2020. Not made for each other: Audio - Visual Dissonance -based Deepfake Detection and Localization . In Proceedings of the 28th ACM International Conference on Multimedia , 439--447

    work page 2020

  7. [7]

    S.; Nagrani, A.; and Zisserman, A

    Chung, J. S.; Nagrani, A.; and Zisserman, A. 2018. Voxceleb2: Deep speaker recognition. arXiv preprint arXiv:1806.05622

    arXiv 2018

  8. [8]

    Cozzolino, D.; Pianese, A.; Nie ner, M.; and Verdoliva, L. 2023. Audio-visual person-of-interest deepfake detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 943--952

    work page 2023

Show all 49 references
  1. [9]

    Dolhansky, B.; Bitton, J.; Pflaum, B.; Lu, J.; Howes, R.; Wang, M.; and Ferrer, C. C. 2020. The DeepFake Detection Challenge ( DFDC ) Dataset . arXiv preprint arXiv:2006.07397

    2020 arXiv

  2. [10]

    Dong, X.; Bao, J.; Chen, D.; Zhang, T.; Zhang, W.; Yu, N.; Chen, D.; Wen, F.; and Guo, B. 2022. Protecting celebrities from deepfake with identity consistency transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9468--9478

    2022

  3. [11]

    Frank, J.; and Sch \"o nherr, L. 2021. WaveFake: A Data Set to Facilitate Audio Deepfake Detection . In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track

    2021

  4. [12]

    Groshev, A.; Maltseva, A.; Chesakov, D.; Kuznetsov, A.; and Dimitrov, D. 2022. GHOST—A New Face Swap Approach for Image and Video Domains. IEEE Access, 10: 83452--83462

    2022

  5. [13]

    Guo, M.-H.; Lu, C.-Z.; Liu, Z.-N.; Cheng, M.-M.; and Hu, S.-M. 2022. Visual attention network. arXiv preprint arXiv:2202.09741

    2022 arXiv

  6. [14]

    Güera, D.; and Delp, E. J. 2018. Deepfake Video Detection Using Recurrent Neural Networks. In 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), 1--6

    2018

  7. [15]

    Haliassos, A.; Vougioukas, K.; Petridis, S.; and Pantic, M. 2021. Lips Don 't Lie : A Generalisable and Robust Approach to Face Forgery Detection . In 2021 IEEE / CVF Conference on Computer Vision and Pattern Recognition ( CVPR ) , 5037--5047

    2021

  8. [16]

    He, Z.; Wang, W.; Guan, W.; Dong, J.; and Tan, T. 2022. Defeating DeepFakes via Adversarial Visual Reconstruction . In Proceedings of the 30th ACM International Conference on Multimedia , 2464--2472

    2022

  9. [17]

    Hosler, B.; Salvi, D.; Murray, A.; Antonacci, F.; Bestagini, P.; Tubaro, S.; and Stamm, M. C. 2021. Do deepfakes feel emotions? A semantic approach to detecting deepfakes via emotional inconsistencies. In Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Re...

    2021

  10. [18]

    Huang, J.; and Ling, C. X. 2005. Using AUC and accuracy in evaluating learning algorithms. IEEE Transactions on knowledge and Data Engineering, 17(3): 299--310

    2005

  11. [19]

    Huang, J.; Wang, X.; Du, B.; Du, P.; and Xu, C. 2021. DeepFake MNIST +: A DeepFake Facial Animation Dataset . In Proceedings of the IEEE/CVF International Conference on Computer Vision, 1973--1982

    2021

  12. [20]

    Huang, X.; and Belongie, S. 2017. Arbitrary style transfer in real-time with adaptive instance normalization. In Proceedings of the IEEE International Conference on Computer Vision ( ICCV ) , 1501--1510

    2017

  13. [21]

    Jeong, Y.; Kim, D.; Min, S.; Joe, S.; Gwon, Y.; and Choi, J. 2022. BiHPF: bilateral high-pass filters for robust deepfake detection. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 48--57

    2022

  14. [22]

    Khalid, H.; Tariq, S.; Kim, M.; and Woo, S. S. 2021. FakeAVCeleb: A novel audio-video multimodal deepfake dataset. arXiv preprint arXiv:2108.05080

    2021 arXiv

  15. [23]

    Kim, J.; Kong, J.; and Son, J. 2021. Conditional Variational Autoencoder with Adversarial Learning for End -to- End Text -to- Speech . In Proceedings of the 38th International Conference on Machine Learning , 5530--5540

    2021

  16. [24]

    Korshunov, P.; and Marcel, S. 2018. DeepFakes : a New Threat to Face Recognition ? Assessment and Detection . arXiv preprint arXiv:1812.08685

    2018 arXiv

  17. [25]

    KoÇak, A.; and Alkan, M. 2022. Deepfake Generation, Detection and Datasets: a Rapid-review. In 2022 15th International Conference on Information Security and Cryptography (ISCTURKEY), 86--91

    2022

  18. [26]

    Kwon, P.; You, J.; Nam, G.; Park, S.; and Chae, G. 2021. KoDF : A Large -scale Korean DeepFake Detection Dataset . In 2021 IEEE / CVF International Conference on Computer Vision ( ICCV ) , 10724--10733

    2021

  19. [27]

    Li, J.; Xie, H.; Yu, L.; and Zhang, Y. 2022. Wavelet-enhanced Weakly Supervised Local Feature Learning for Face Forgery Detection . In Proceedings of the 30th ACM International Conference on Multimedia , 1299--1308

    2022

  20. [28]

    Li, Y.; Yang, X.; Sun, P.; Qi, H.; and Lyu, S. 2020. Celeb- DF : A Large - Scale Challenging Dataset for DeepFake Forensics . In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 3207--3216

    2020

  21. [29]

    Loshchilov, I.; and Hutter, F. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101

    2017 arXiv

  22. [30]

    Lyu, S. 2020. Deepfake Detection: Current Challenges and Next Steps. In 2020 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), 1--6

    2020

  23. [31]

    Mittal, T.; Bhattacharya, U.; Chandra, R.; Bera, A.; and Manocha, D. 2020. Emotions Don 't Lie : An Audio - Visual Deepfake Detection Method using Affective Cues . In Proceedings of the 28th ACM International Conference on Multimedia , 2823--2832

    2020

  24. [32]

    Nam, H.; Lee, H.; Park, J.; Yoon, W.; and Yoo, D. 2021. Reducing domain gap by reducing style bias. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8690--8699

    2021

  25. [33]

    A.; and Malik, K

    Raza, M. A.; and Malik, K. M. 2023. Multimodaltrace: Deepfake Detection Using Audiovisual Representation Learning . In Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition , 993--1000

    2023

  26. [34]

    Sanderson, C. 2002. The vidtimit database. Technical report, IDIAP

    2002

  27. [35]

    R.; Cogswell, M.; Das, A.; Vedantam, R.; Parikh, D.; and Batra, D

    Selvaraju, R. R.; Cogswell, M.; Das, A.; Vedantam, R.; Parikh, D.; and Batra, D. 2017. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE International Conference on Computer Vision ( ICCV ) , 618--626

    2017

  28. [36]

    Tan, M.; and Le, Q. 2019. Efficientnet: Rethinking model scaling for convolutional neural networks. In International Conference on Machine Learning, 6105--6114

    2019

  29. [37]

    N.; Kaiser, .; and Polosukhin, I

    Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, .; and Polosukhin, I. 2017. Attention is all you need. Advances in Neural Information Processing Systems, 30

    2017

  30. [38]

    Wang, G.; Zhang, P.; Xie, L.; Huang, W.; and Zha, Y. 2022 a . Attention-Based Lip Audio-Visual Synthesis for Talking Face Generation in the Wild. arXiv preprint arXiv:2203.03984

    2022 arXiv

  31. [39]

    Wang, Q.; Zhang, X.; Wang, J.; Cheng, N.; and Xiao, J. 2022 b . Drvc: A framework of any-to-any voice conversion with self-supervised learning. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 3184--3188

    2022

  32. [40]

    Wang, S.-Y.; Wang, O.; Zhang, R.; Owens, A.; and Efros, A. A. 2020. CNN-generated images are surprisingly easy to spot... for now. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 8695--8704

    2020

  33. [41]

    Yang, T.; Huang, Z.; Cao, J.; Li, L.; and Li, X. 2022. Deepfake network architecture attribution. In Proceedings of the AAAI Conference on Artificial Intelligence, 4662--4670

    2022

  34. [42]

    Yang, W.; Zhou, X.; Chen, Z.; Guo, B.; Ba, Z.; Xia, Z.; Cao, X.; and Ren, K. 2023. AVoiD - DF : Audio - Visual Joint Learning for Detecting Deepfake . IEEE Transactions on Information Forensics and Security, 18: 2015--2029

    2023

  35. [43]

    Zhang, S.; Zhu, X.; Lei, Z.; Shi, H.; Wang, X.; and Li, S. Z. 2017. S3fd: Single shot scale-invariant face detector. In Proceedings of the IEEE International Conference on Computer Vision ( ICCV ) , 192--201

    2017

  36. [44]

    Zhao, C.; Wang, C.; Hu, G.; Chen, H.; Liu, C.; and Tang, J. 2023. ISTVT: Interpretable Spatial-Temporal Video Transformer for Deepfake Detection. IEEE Transactions on Information Forensics and Security, 18: 1335--1348

    2023

  37. [45]

    Zhao, H.; Zhou, W.; Chen, D.; Zhang, W.; and Yu, N. 2022. Self-supervised transformer for deepfake detection. arXiv preprint arXiv:2203.01265

    2022 arXiv

  38. [46]

    Zhao, T.; Xu, X.; Xu, M.; Ding, H.; Xiong, Y.; and Xia, W. 2021. Learning self-consistency for deepfake detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision ( ICCV ) , 15023--15033

    2021

  39. [47]

    Zhou, Y.; and Lim, S.-N. 2021. Joint Audio - Visual Deepfake Detection . In Proceedings of the IEEE/CVF International Conference on Computer Vision, 14800--14809

    2021

  40. [48]

    S.; and Rajan, D

    Zou, H.; Shen, M.; Hu, Y.; Chen, C.; Chng, E. S.; and Rajan, D. 2024. Cross-Modality and Within-Modality Regularization for Audio-Visual DeepFake Detection. arXiv preprint arXiv:2401.05746

    2024 arXiv

  41. [49]

    write newline

    " write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence '...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.