Pith. sign in

REVIEW 4 major objections 5 minor 62 references

HOLA: Enhancing Audio-visual Deepfake Detection via Hierarchical Contextual Aggregations and Efficient Pre-training

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read HOLA, a two-stage audio-visual model pre-trained on 1.81M videos and fine-tuned with hierarchical gated fusion, claims first place in the 2025 1M-Deepfakes video track with 0.9991 AUC on validation and a 0.0476 AUC margin on TestA.

desk verdict HOLA is a capable deepfake detector with a real challenge win, but its headline TestA margin is compromised by using TestA for pseudo-labeling and iteration selection, so the reported generalization claim doesn't hold as written. read the letter →

arxiv 2507.22781 v1 pith:WDDIOEN5 submitted 2025-07-30 cs.CV

Xuecheng Wu , Danlei Huang , Heli Sun , Xinyi Yin , Yifan Wang , Hao Wang , Jia Zhang , Fei Wang
show 4 more authors
Peihao Guo Suyu Xing Junxiao Xue Liang He
This is my paper
classification cs.CV
keywords deepfakedetectionaudio-visuallearningself-supervisedpre-trainingmultimodalfusionvideo-levelAV-Deepfake1M++pseudo-labelingmaskedautoencoder
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that video-level deepfake detection can be solved by treating it as a two-stage audio-visual learning problem: first pre-train audio and video encoders on 1.81 million talking-face videos with masked autoencoding, then fine-tune with three hierarchically organized fusion modules. On AV-Deepfake1M++, the resulting system, HOLA, reaches 98.63% accuracy and 0.9991 AUC on the validation set and reports first place on the 2025 1M-Deepfakes Detection Challenge video track, 0.0476 AUC ahead of the runner-up on TestA. If the result holds, the paper's recipe — scaling self-supervised audio-visual pre-training and adding gated, multi-scale cross-modal fusion — makes a large benchmark's manipulated videos nearly fully separable from real ones. The authors also show that standard expert models and even large vision-language models lag far behind, which suggests the gap comes from learning deepfake-specific audio-visual correlations rather than from raw reasoning ability.

What carries the argument

The load-bearing machinery is a two-stage pipeline. Stage one is audio-visual self-supervised pre-training in the style of a masked autoencoder: a video encoder and an audio encoder are trained with 90% and 80% masking, a cross-modal fusion encoder, modality-specific decoders, and MSE reconstruction loss, on 1.81M videos. Stage two is fine-tuning with three named modules: the iterative-aware cross-modal interaction (three stacked bidirectional cross-attention layers with a sigmoid gate that controls how much each modality updates from the other), the local-global contextual fusion module (a [CLS] token and self-attention over the concatenated modality streams, with a gate mixing local and global features), and the pyramid-like refiner (three stride-2 Conv1D scales that downsample the sequence and pool into a multi-scale global feature for classification). A pseudo-supervised signal injection adds high-confidence TestA predictions back into training across iterations.

What would settle it

Run the same HOLA pipeline but with the TestA set completely withheld — no AUC-based iteration selection, no TestA samples added as pseudo-labels — and compare the resulting AUC on a fresh, untouched evaluation split. If the gap over the runner-up collapses or the validation-level 0.9991 AUC drops substantially, the reported first-place margin is an artifact of test-set feedback rather than a generalizable detector.

Watch

Extended reading notes

Core claim

The paper's central claim is that HOLA, built from large-scale audio-visual self-supervised pre-training followed by task-specific fine-tuning, achieves near-perfect performance on the video-level deepfake detection track of AV-Deepfake1M++: 98.63% ACC, 98.62% WA-F1, 98.07% UAR, and 0.9991 AUC on the validation set, and first place with 0.0476 AUC margin over second on TestA. The three fine-tuning components — iterative-aware cross-modal interaction, local-global contextual fusion with gated aggregation, and a pyramid-like refiner — each add measurable gains in the ablation, and the pseudo-supervised signal injection contributes the final 0.42% ACC. The authors state that the large pre-training corpus (1.81M samples from several talking-face and speaker datasets) plus dual-masking audio-visual masked autoencoding is what lets the downstream model learn generalizable facial and vocal representations, and that zero-shot vision-language models remain far below this expert model.

Load-bearing premise

The reported TestA margin assumes that using TestA itself to choose the pseudo-labeling iteration count and to select high-confidence pseudo-labeled training samples does not materially inflate the result; if TestA is effectively leaked into model selection, the 0.0476 AUC gap is not evidence of generalization.

Editorial extensions

If this is right

  • Large-scale audio-visual self-supervised pre-training transfers to video deepfake detection: the HOLA baseline already beats all compared expert models trained from scratch, reaching 95.27% ACC versus 86.04% for the best baseline.
  • Each hierarchical component contributes to the final result: the ablation shows gains of 0.95% ACC for iterative-aware cross-modal interaction, 1.18% for local-global contextual fusion, 0.81% for the pyramid-like refiner, and 0.42% for pseudo-supervised signal injection, cumulatively reaching 98.63% ACC.
  • Zero-shot vision-language models, including reasoning-capable ones, underperform the expert HOLA by tens of percentage points in accuracy on a 1,000-sample validation subset, implying deepfake detection requires dedicated training data rather than general visual reasoning.
  • The pseudo-supervised signal injection strategy improves TestA AUC by about 1.69 points at its peak before performance degrades with further iterations, indicating that repeated reinforcement of incorrect pseudo-labels eventually hurts.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the result transfers, the HOLA pre-training recipe should also improve other audio-visual understanding tasks such as emotion recognition, speaker verification, and talking-head generation, because the 1.81M-video encoder learns speaker-generic audiovisual correspondence rather than dataset-specific artifacts.
  • The reported near-perfect AUC on AV-Deepfake1M++ likely reflects that this benchmark's generation pipeline leaves detectable manipulation traces; real-world deepfakes outside this protocol should be tested before concluding the method generalizes to arbitrary generated video.
  • A clean test of the pseudo-supervised strategy would be to select the iteration count and pseudo-labeled samples using a held-out validation set instead of TestA; the paper's Fig. 2 suggests the peak at iteration 5 is chosen by TestA feedback, so a fresh evaluation is needed to confirm the benefit is not an artifact of test-set leakage.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper presents HOLA, a two-stage audio-visual framework for video-level deepfake detection on the AV-Deepfake1M++ benchmark. Stage one performs audio-visual self-supervised pre-training on a self-built 1.81M-sample dataset, following the designs of AVF-MAE++ [48] and HiCMAE [40]. Stage two fine-tunes the pre-trained encoders on the challenge training set using three proposed modules: an iterative-aware cross-modal interaction module, a local-global contextual fusion module with gated aggregation, and a pyramid-like refiner. The paper also introduces a pseudo-supervised signal injection strategy that adds high-confidence test-subset samples to the training set over multiple iterations. The authors report first place in the 2025 1M-Deepfakes Detection Challenge video-level track, outperforming the second-place entry by 0.0476 AUC on TestA, and 0.9991 AUC on the Validation set. Ablation studies and comparisons with expert models and VLMs are also reported.

Significance. If the central empirical claim were cleanly established, the paper would make a useful contribution: it would demonstrate that large-scale audio-visual self-supervised pre-training combined with hierarchical cross-modal fusion can achieve near-perfect video-level deepfake detection on AV-Deepfake1M++. The three proposed modules are plausible and are evaluated incrementally, and the inclusion of VLM baselines and qualitative examples adds context. However, the headline claim on TestA is not supported by the reported protocol because TestA is used both as a source of pseudo-labels and as the model-selection set. The absence of multiple seeds, error bars, or code release further limits the strength of the quantitative claims. The paper's strengths are the clarity of the architectural description and the explicit ablation of each proposed component.

major comments (4)
  1. [Sec. 3.3 and Sec. 4.3 / Fig. 2] The central performance claim on TestA is not a clean held-out measurement. Section 3.3 states that after each training iteration, samples from the unlabeled test subset with confidence equal to 1 are added to the training set, and Section 4.3 / Fig. 2 reports that the number of pseudo-labeling iterations was chosen by monitoring AUC on TestA, with the peak at iteration 5. This makes TestA partially a training set and fully a model-selection set. Consequently, the reported 0.0476 AUC margin over the second-place entry does not measure generalization to unseen data. The authors must report TestA AUC without pseudo-label injection, with the iteration count fixed without test-set peeking, and ideally evaluate on TestB or another truly held-out set. They should also disclose how many TestA samples were added in each iteration and whether the official challenge rules permitted test-set adaptation.
  2. [Sec. 4.3 / Fig. 2 and Table 3] The paper reports single-run results without seed variance or error bars. In Fig. 2, the TestA AUC curve varies by only about 1.5 percentage points across iterations, and the claimed improvement from the pseudo-label strategy is 1.69% AUC, while Table 3 shows ACC gains of 0.41% and 0.42% for the last two components. With no repeated runs or confidence intervals, these differences could be within run-to-run noise, especially given that the stopping point is selected on the same test set. The central comparisons should be repeated with at least three seeds and reported with means and standard deviations.
  3. [Sec. 4.2.1 / Table 1 and Sec. 4.4 / Table 3] The comparison against expert baselines is not controlled. HOLA benefits from 1.81M pre-training samples, whereas the baseline methods are described as trained from scratch on the challenge dataset. The ablation in Table 3 starts from a pre-trained baseline, so the incremental gains of the three modules are measured on top of that pre-training; however, there is no ablation that isolates the contribution of pre-training data scale or pre-training itself. Given that the paper's title and abstract emphasize 'efficient pre-training,' the authors should provide evidence that the 1.81M-sample pre-training, rather than the fine-tuning architecture alone, is responsible for the gains, for example by comparing against a from-scratch fine-tuning baseline with the same modules.
  4. [Sec. 4.2.2 / Table 2] The VLM comparison in Table 2 is performed on a randomly sampled 1,000-instance subset of the Validation set, but the paper does not report the sampling seed, the overlap with the Validation set used in Table 1, or any confidence intervals. The 'HOLA(Reference)' row reports 98.83% ACC, yet Table 1 reports 98.63% ACC on the full Validation set; the paper should clarify whether these numbers are comparable and should describe how the random subset was selected.
minor comments (5)
  1. [Abstract and throughout] There are several typos, including 'singal injection' in the abstract (should be 'signal'), 'Interative-aware' in Fig. 1 and Section 3.2.3, 'Performace' in Tables 1 and 2, and 'Linera + sigmoid' in Fig. 1. These should be corrected.
  2. [Sec. 3.2.5 / Eq. (13)] Equation (13) uses F_{s_i-1} without defining F_{s_0} or the initial input feature. Please clarify the notation for the first scale and the dimension changes after each stride-2 Conv1D.
  3. [Sec. 3.3 / Fig. 2] Figure 2 shows 'Incremental Samples' on the x-axis with labels such as '319211' and '334740', but the text says the iterations are indexed from 1 to 9. The relationship between the iteration index and the number of incremental samples should be explained.
  4. [Sec. 4.1.1] The paper states that the pre-training settings follow AVF-MAE++ and HiCMAE, but it does not provide the model size, number of parameters, or pre-training compute budget. Adding these details would make the 'efficient pre-training' claim more concrete.
  5. [References] The paper does not provide a link to the challenge website or the official leaderboard, nor a statement about code/model release. A reproducibility statement would be helpful.

Circularity Check

2 steps flagged · score 6.0 of 10

TestA is used both as a source of pseudo-labeled training samples and as the model-selection set for the pseudo-label iteration count, so the reported 0.0476 AUC margin is partially self-fulfilling.

  1. fitted input called prediction [Section 3.3 (Pseudo Supervised Singal Injection Strategy); Section 4.1.2 (Dataset split)]
    "To make better use of the unlabeled test subset, we introduce the pseudo-supervised signal injection strategy to further boost model performance. Specifically, after each training iteration, we select all the samples with the confidence score = 1 for both real and fake categories based on the final predictions. These absolute samples are then added to the training set, leading to more samples for next iteration."

    The unlabeled test subset is TestA (Sec. 4.1.2). Samples whose predicted confidence is exactly 1 are cherry-picked from TestA and added to the training set; any later TestA evaluation therefore includes samples the model has already been trained on, with labels generated by the same model. The 'absolute' samples are self-confirming: they are selected because the model is certain, then used to make the model more certain on the same inputs. The reported 0.0476 AUC margin over second place is thus not a clean held-out measurement, and the paper does not state how many TestA samples were injected or how the official protocol treated such test-set adaptation.

  2. fitted input called prediction [Section 4.3, Figure 2]
    "As shown in Fig. 2, after five iterations, HOLA achieves a peak AUC of 97.83% on testA, exhibiting 1.69% AUC improvement over the previous best performance. However, performance begins to decline beyond 5 iterations, likely due to the repeated reinforcement of incorrect pseudo-labels."

    The pseudo-label iteration count is selected by monitoring AUC on testA and stopping at the peak. This makes testA a model-selection set: the iteration number encodes information from testA, so reporting the resulting testA AUC as the system's predictive performance double-counts the same data. This is the 'fitted parameter renamed as prediction' pattern: the number of iterations is fit to testA, and the testA result is then presented as the outcome of the fitted pipeline. Because the headline margin over the runner-up is only 0.0476 AUC, the selection channel alone could plausibly account for the reported advantage.

full rationale

The architecture-level components (iterative-aware cross-modal interaction, local-global contextual fusion, pyramid-like refiner) are evaluated by ablations on the Validation set, so those components are not circular: their gains are measured against a set not used for the reported TestA margin. The reuse of the authors' own AVF-MAE++ [48] and HiCMAE [40] as pretraining designs is a self-citation, but it functions as engineering inheritance rather than as a logical premise that forces the empirical outcome, so I do not count it as load-bearing circularity. The genuine circularity is confined to the evaluation protocol for the TestA headline: Section 3.3 injects TestA-derived pseudo-labels into training, and Section 4.3/Fig. 2 selects the iteration count by TestA AUC. Both operations make the later '1st place, 0.0476 AUC margin' claim partly self-fulfilling rather than a clean generalization test. Score 6 reflects partial circularity: the modular contributions have independent validation support, but the central TestA prediction is contaminated by construction.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The framework rests on transfer from reconstruction pre-training, pseudo-labels from the test set, and several hand-set architectural hyperparameters. The main uncharged premise is that self-supervised audio-visual reconstruction on real speech videos transfers to forgery detection; the main methodological risk is that the pseudo-labeling procedure is tuned to TestA. No new physical entities are proposed.

free parameters (5)
  • Pre-training masking ratios = 90% video tube masking, 80% audio random masking, 50% decoder masking
    Chosen for efficiency and 'acceptable drops' in Section 3.1.1; inherited from AVF-MAE++ settings, not derived.
  • Pseudo-label iteration count = 5
    Selected because TestA AUC peaks at 5 iterations in Fig. 2; performance declines after. This is fitted to the test benchmark.
  • Pseudo-label confidence threshold = confidence score = 1
    Samples with exactly 1.0 confidence are selected as absolutely confident in Section 3.3; no sensitivity analysis is provided.
  • Number of interaction layers in iterative-aware module = 3
    Set by design in Section 3.2.3; no ablation over depth is presented.
  • Number of pyramid scales = 3
    Set by design in Section 3.2.5; no ablation over the number of scales is presented.
assumptions (4)
  • domain assumption AV-Deepfake1M++ labels are correct and the official AUC metric is a valid measure of deepfake detection performance.
    Used throughout experiments; if labels or metric are flawed, all comparisons inherit the flaw.
  • domain assumption Reconstruction-based self-supervised pre-training on real talking-face videos (CelebV-HQ, CN-Celeb, HDTF, MSD-Wild-DB) yields representations that transfer to binary real/fake audio-visual detection.
    This premise justifies the entire two-stage framework and is not independently validated outside the reported fine-tuning results.
  • domain assumption The 1.81M pretraining samples do not overlap with AV-Deepfake1M++ test samples in a way that leaks identity or content.
    No leak analysis is provided; speaker-recognition datasets may contain some of the same speakers or videos.
  • ad hoc to paper High-confidence pseudo-labels (confidence = 1.0) on the unlabeled test subset are correct enough to improve training.
    This premise justifies the pseudo-supervised signal injection and is not validated on a held-out set; the authors themselves note later iterations reinforce incorrect labels in Section 4.3.

how reviews work

0 comments
Cite this review

Pith. "Pith review of HOLA: Enhancing Audio-visual Deepfake Detection via Hierarchical Contextual Aggregations and Efficient Pre-training." pith.science (2026). https://pith.science/paper/WDDIOEN5

@misc{pith2026250722781,
  author       = {Pith},
  title        = {Pith review of: HOLA: Enhancing Audio-visual Deepfake Detection via Hierarchical Contextual Aggregations and Efficient Pre-training},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WDDIOEN5}},
  note         = {Machine review of arXiv:2507.22781}
}
read the original abstract

Advances in Generative AI have made video-level deepfake detection increasingly challenging, exposing the limitations of current detection techniques. In this paper, we present HOLA, our solution to the Video-Level Deepfake Detection track of 2025 1M-Deepfakes Detection Challenge. Inspired by the success of large-scale pre-training in the general domain, we first scale audio-visual self-supervised pre-training in the multimodal video-level deepfake detection, which leverages our self-built dataset of 1.81M samples, thereby leading to a unified two-stage framework. To be specific, HOLA features an iterative-aware cross-modal learning module for selective audio-visual interactions, hierarchical contextual modeling with gated aggregations under the local-global perspective, and a pyramid-like refiner for scale-aware cross-grained semantic enhancements. Moreover, we propose the pseudo supervised singal injection strategy to further boost model performance. Extensive experiments across expert models and MLLMs impressivly demonstrate the effectiveness of our proposed HOLA. We also conduct a series of ablation studies to explore the crucial design factors of our introduced components. Remarkably, our HOLA ranks 1st, outperforming the second by 0.0476 AUC on the TestA set.

Figures

Figures reproduced from arXiv: 2507.22781 by the authors.

Figure 1
Figure 1. The overall illustrations of introduced HOLA , in which our core design is three hierarchical fusion components. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 3
Figure 3. Qualitative comparisons of HOLA and SOTA VLMs. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 2
Figure 2. The results of pseudo-supervised signal injection. [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

62 extracted references · 42 canonical work pages

  1. [48]

    Xuecheng Wu, Heli Sun, Yifan Wang, Jiayu Nie, Jie Zhang, Yabing Wang, Junxiao Xue, and Liang He. 2025. AVF-MAE++: Scaling Affective Video Facial Masked Autoencoders via Efficient Audio-Visual Self-Supervised Learning. InProceedings of the Computer Vision and Pattern Recognition Conference. 9142–9153

    work page 2025

  2. [40]

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems 32 (2019)

    2019

  3. [1]

    Marah Abdin, Sam Ade Jacobs, Ammar Ahmad Awan, Jyoti Aneja, Ahmed Awadallah, Hany Awadalla, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Harki- rat Behl, et al. 2024. Phi-3 technical report: A highly capable language model locally on your phone. arXiv preprint arXiv:2404.14219 (2024)

    arXiv 2024

  4. [2]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. 2025. Qwen2.5-vl technical report. arXiv preprint arXiv:2502.13923 (2025)

    arXiv 2025

  5. [3]

    Gedas Bertasius, Heng Wang, and Lorenzo Torresani. 2021. Is space-time attention all you need for video understanding?. In ICML, Vol. 2. 4

    2021

  6. [4]

    ByteDance. 2025. Doubao-1.5-vision-pro-32k. https://volcengine.com/product/ doubao

    work page 2025

  7. [5]

    Zhixi Cai, Shreya Ghosh, Aman Pankaj Adatia, Munawar Hayat, Abhinav Dhall, Tom Gedeon, and Kalin Stefanov. 2024. AV-Deepfake1M: A large-scale LLM-driven audio-visual deepfake dataset. In Proceedings of the 32nd ACM International Conference on Multimedia. 7414–7423

    work page 2024

  8. [6]

    Zhixi Cai, Kartik Kuckreja, Shreya Ghosh, Akanksha Chuchra, Muhammad Haris Khan, Usman Tariq, Tom Gedeon, and Abhinav Dhall. 2025. AV-Deepfake1M++: A Large-Scale Audio-Visual Deepfake Benchmark with Real-World Perturbations. arXiv:2507.20579 [cs.CV] https://arxiv.org/abs/2507.20579

    arXiv 2025

Show all 62 references
  1. [7]

    Zhixi Cai, Kalin Stefanov, Abhinav Dhall, and Munawar Hayat. 2022. Do you really mean that? content driven audio-visual deepfake dataset and multimodal method for temporal forgery localization. In 2022 International Conference on Digital Image Computing: Techniques and Applica...

    2022

  2. [8]

    Joao Carreira and Andrew Zisserman. 2017. Quo vadis, action recognition? a new model and the kinetics dataset. In proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 6299–6308

    2017

  3. [9]

    Davide Cozzolino, Alessandro Pianese, Matthias Nießner, and Luisa Verdoliva

  4. [10]

    Brecht Desplanques, Jenthe Thienpondt, and Kris Demuynck. 2020. Ecapa- tdnn: Emphasized channel attention, propagation and aggregation in tdnn based speaker verification. arXiv preprint arXiv:2005.07143 (2020)

    2020 arXiv

  5. [11]

    Yue Fan, JW Kang, LT Li, KC Li, HL Chen, ST Cheng, PY Zhang, ZY Zhou, YQ Cai, and Dong Wang. 2020. Cn-celeb: a challenging chinese speaker recogni- tion dataset. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 7604–7608

    2020

  6. [12]

    Liangyu Fu, Junbo Wang, Qiangguo Jin, Yining Zhu, Hongsong Wang, Yuke Li, Xuecheng Wu, and Kun Hu. 2025. PTSR: A Unified Patch Tokenization, Selection and Representation Framework for Efficient Micro-expression Recognition. In Proceedings of the 2025 International Conference o...

    2025

  7. [13]

    Shreyan Ganguly, Sk Mohiuddin, Samir Malakar, Erik Cuevas, and Ram Sarkar

  8. [14]

    Shang-Hua Gao, Ming-Ming Cheng, Kai Zhao, Xin-Yu Zhang, Ming-Hsuan Yang, and Philip Torr. 2019. Res2net: A new multi-scale backbone architecture. IEEE transactions on pattern analysis and machine intelligence 43, 2 (2019), 652–662

    2019

  9. [15]

    T M Geethanjali, T S Darshan, K Surya, H U Rahul, and Ipshika N Sheety

  10. [16]

    Shoutao Guo, Xiang Li, Shaolei Zhang, Mengge Liu, Wei Chen, and Yang Feng

  11. [17]

    Yinan He, Bei Gan, Siyu Chen, Yichun Zhou, Guojun Yin, Luchuan Song, Lu Sheng, Jing Shao, and Ziwei Liu. 2021. Forgerynet: A versatile benchmark for comprehensive forgery analysis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 4360–4369

    2021

  12. [18]

    Jie Hu, Li Shen, and Gang Sun. 2018. Squeeze-and-excitation networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 7132–7141

    2018

  13. [19]

    Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. 2020. Analyzing and improving the image quality of stylegan. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 8110–8119

    2020

  14. [20]

    Hasam Khalid, Shahroz Tariq, Minha Kim, and Simon S Woo. 2021. FakeAVCeleb: A novel audio-video multimodal deepfake dataset. arXiv preprint arXiv:2108.05080 (2021)

    2021 arXiv

  15. [21]

    Qiuqiang Kong, Yin Cao, Turab Iqbal, Yuxuan Wang, Wenwu Wang, and Mark D Plumbley. 2020. Panns: Large-scale pretrained audio neural networks for audio pattern recognition. IEEE/ACM Transactions on Audio, Speech, and Language Processing 28 (2020), 2880–2894

    2020

  16. [22]

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. 2024. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326 (2024)

    2024 arXiv

  17. [23]

    Yuezun Li, Xin Yang, Pu Sun, Honggang Qi, and Siwei Lyu. 2020. Celeb-df: A large-scale challenging dataset for deepfake forensics. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 3207–3216

    2020

  18. [24]

    Tao Liu, Shuai Fan, Xu Xiang, Hongbo Song, Shaoxiong Lin, Jiaqi Sun, Tianyuan Han, Siyuan Chen, Binwei Yao, Sen Liu, et al . 2022. MSDWild: Multi-modal Speaker Diarization Dataset in the Wild.. In INTERSPEECH. 1476–1480

    2022

  19. [25]

    Ze Liu, Jia Ning, Yue Cao, Yixuan Wei, Zheng Zhang, Stephen Lin, and Han Hu

  20. [26]

    Ilya Loshchilov and Frank Hutter. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 (2017)

    2017 arXiv

  21. [27]

    Florian Lugstein, Simon Baier, Gregor Bachinger, and Andreas Uhl. 2021. PRNU- based Deepfake Detection. In Proc. ACM Workshop on Information Hiding and Multimedia Security (IH&MMSec). 7–12. https://doi.org/10.1145/3437880. 3460400

    2021 doi

  22. [28]

    Run Luo, Ting-En Lin, Haonan Zhang, Yuchuan Wu, Xiong Liu, Min Yang, Yong- bin Li, Longze Chen, Jiaming Li, Lei Zhang, et al. 2025. OpenOmni: Advancing Open-Source Omnimodal Large Language Models with Progressive Multimodal Alignment and Real-Time Self-Aware Emotional Speech S...

    2025

  23. [29]

    Changtao Miao, Yi Zhang, Weize Gao, Man Luo, Weiwei Feng, Zhiya Tan, Jianshu Li, Ajian Liu, Yunfeng Diao, Qi Chu, et al. 2025. DDL: A Dataset for Interpretable Deepfake Detection and Localization in Real-World Scenarios. arXiv preprint arXiv:2506.23292 (2025)

    2025

  24. [30]

    In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Video swin transformer. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 3202–3211

  25. [31]

    Dat Nguyen, Nesryne Mejri, Inder Pal Singh, Polina Kuleshova, Marcella Astrid, Anis Kacem, Enjie Ghorbel, and Djamila Aouada. 2024. LAA-Net: Localized Artifact Attention Network for Quality-Agnostic and Generalizable Deepfake Detection. In Proceedings of the IEEE/CVF Conferenc...

    2024

  26. [32]

    Trevine Oorloff, Surya Koppisetti, Nicolò Bonettini, Divyaraj Solanki, Ben Col- man, Yaser Yacoob, Ali Shahriyari, and Gaurav Bharaj. 2024. Avff: Audio-visual feature fusion for video deepfake detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern ...

    2024

  27. [33]

    OpenAI. 2024. Hello GPT-4o. https://openai.com/index/hello-gpt-4o/

    2024

  28. [34]

    OpenAI. 2025. OpenAI o3. https://openai.com/index/introducing-o3-and-o4- mini/

    2025

  29. [35]

    Gillian Murphy, Didier Ching, John Twomey, and Conor Linehan. 2023. Face/Off: Changing the face of movies with deepfakes. Plos one 18, 7 (2023), e0287503

    2023

  30. [36]

    Zhiwu Qing, Shiwei Zhang, Ziyuan Huang, Xiang Wang, Yuehuan Wang, Yiliang Lv, Changxin Gao, and Nong Sang. 2023. Mar: Masked autoencoders for efficient action recognition. IEEE Transactions on Multimedia 26 (2023), 218–233

    2023

  31. [37]

    Andreas Rossler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, and Matthias Nießner. 2019. Faceforensics++: Learning to detect manipu- lated facial images. In Proceedings of the IEEE/CVF international conference on computer vision. 1–11

    2019

  32. [38]

    Yiwei Ru, Wanting Zhou, Yunfan Liu, Jianxin Sun, and Qi Li. 2021. Bita-Net: Bi-temporal Attention Network for Facial Video Forgery Detection. In Proc. of IEEE International Joint Conference on Biometrics (IJCB). 1–8. https://doi.org/ 10.1109/IJCB52358.2021.9484408

    2021

  33. [39]

    Sahil Sharma, Ashima Sood, and Vijay Kumar. 2024. Deepfake Synthetic-20K Dataset. https://doi.org/10.21227/67x4-9g14

    2024 doi

  34. [41]

    Gemini Team. 2025. Gemini-2.5-pro-preview-03-25. https://deepmind.google/ technologies/gemini/pro/

    2025

  35. [42]

    Qwen Team. 2024. QVQ: To See the World with Wisdom. https://qwenlm.github. io/blog/qvq-72b-preview/

    2024

  36. [43]

    Du Tran, Lubomir Bourdev, Rob Fergus, et al. 2015. Learning spatiotemporal fea- tures with 3d convolutional networks. In Proceedings of the IEEE international conference on computer vision. 4489–4497

    2015

  37. [44]

    Gaojian Wang, Qian Jiang, Xin Jin, and Xiaohui Cui. 2022. FFR_FD: Effective and Fast Detection of DeepFakes via Feature Point Defects. Information Sciences 596 (2022), 472–488. https://doi.org/10.1016/j.ins.2022.03.026

    2022 doi

  38. [45]

    Licai Sun, Zheng Lian, Bin Liu, and Jianhua Tao. 2024. Hicmae: Hierarchical contrastive masked autoencoder for self-supervised audio-visual emotion recog- nition. Information Fusion 108 (2024), 102382

    2024

  39. [46]

    Yujia Wang and Hua Huang. 2024. Audio–visual deepfake detection using articu- latory representation learning. Computer Vision and Image Understanding 248 (2024), 104133

    2024

  40. [47]

    Xuecheng Wu, Jiaxing Liu, Danlei Huang, Xiaoyu Li, et al . 2025. ViC-Bench: Benchmarking Visual-Interleaved Chain-of-Thought Capability in MLLMs with Free-Style Intermediate State Representations. arXiv preprint arXiv:2505.14404 (2025). MM ’25, October 27-October 31, 2025, Dub...

    2025

  41. [49]

    Xuecheng Wu, Heli Sun, Junxiao Xue, Jiayu Nie, Xiangyan Kong, Ruofan Zhai, Danlei Huang, and Liang He. 2025. Towards Emotion Analysis in Short- form Videos: A Large-Scale Dataset and Baseline. In Proceedings of the 2025 International Conference on Multimedia Retrieval. 1497–1506

    2025

  42. [50]

    Hui Wang, Siqi Zheng, Yafeng Chen, Luyao Cheng, and Qian Chen. 2023. CAM++: A Fast and Efficient Network For Speaker Verification Using Context-Aware Masking. arXiv preprint arXiv:2303.00332 (2023)

    2023 arXiv

  43. [51]

    Zhiyuan Yan, Taiping Yao, Shen Chen, Yandan Zhao, Xinghe Fu, Junwei Zhu, Donghao Luo, Chengjie Wang, Shouhong Ding, Yunsheng Wu, et al. 2024. Df40: Toward next-generation deepfake detection. Advances in Neural Information Processing Systems 37 (2024), 29387–29434

    2024

  44. [52]

    Yibo Zhang et al. 2024. Joint Audio-Visual Attention with Contrastive Learning for More General Deepfake Detection. ACM Trans. Multimedia Comput. Commun. Appl. 20, 5 (2024), 137. https://doi.org/10.1145/3625100

    2024 doi

  45. [53]

    Zhimeng Zhang, Lincheng Li, Yu Ding, and Changjie Fan. 2021. Flow-guided one-shot talking face generation with a high-resolution audio-visual dataset. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 3661–3670

    2021

  46. [54]

    Zijian Zhang, Xuhui Zheng, Xuecheng Wu, et al. 2025. TokenFocus-VQA: Enhanc- ing Text-to-Image Alignment with Position-Aware Focus and Multi-Perspective Aggregations on LVLMs. In Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR) Workshops. 1279–1288

    2025

  47. [55]

    xAI. 2025. xAI Grok 4. https://x.ai/news/grok-4

    2025

  48. [56]

    Yinglin Zheng, Jianmin Bao, Dong Chen, Ming Zeng, and Fang Wen. 2021. Ex- ploring Temporal Coherence for More General Video Face Forgery Detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 15044–15054

    2021

  49. [57]

    Hao Zhu, Wayne Wu, Wentao Zhu, Liming Jiang, Siwei Tang, Li Zhang, Ziwei Liu, and Chen Change Loy. 2022. CelebV-HQ: A large-scale video facial attributes dataset. In European conference on computer vision. Springer, 650–667

    2022

  50. [60]

    Hanqing Zhao, Wenbo Zhou, Dongdong Chen, Tianyi Wei, Weiming Zhang, and Nenghai Yu. 2021. Multi-Attentional Deepfake Detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2185–2194

    2021

  51. [2022]

    Pattern Analysis and Applications 25, 4 (2022), 981–992

    Visual attention-based deepfake video forgery detection. Pattern Analysis and Applications 25, 4 (2022), 981–992

    2022

  52. [2023]

    In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Audio-visual person-of-interest deepfake detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 943–952

  53. [2024]

    In 2024 International Conference on Signal Processing, Computation, Electronics, Power and Telecommunication (IConSCEPT)

    Detectify : Image Tampering Detection using Error Level Analysis (ELA) and Convolutional Neural Network (CNN). In 2024 International Conference on Signal Processing, Computation, Electronics, Power and Telecommunication (IConSCEPT). 1–6. https://doi.org/10.1109/IConSCEPT61884....

    2024

  54. [2025]

    arXiv preprint arXiv:2507.07803 (2025)

    StreamUni: Achieving Streaming Speech Translation with a Unified Large Speech-Language Model. arXiv preprint arXiv:2507.07803 (2025)

    2025 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.