Pith. sign in

REVIEW 3 major objections 5 minor 84 references

Visual Language Models as Zero-Shot Deepfake Detectors

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that zero-shot vision-language models, scored by normalized yes/no token probabilities, can outperform deepfake detectors trained specifically for the task.

desk verdict A new 60k SimSwap benchmark and a practical token-probability scoring trick are the real assets; the headline claim that zero-shot VLMs beat trained detectors is not yet supported because VLM accuracy is measured at test-set-tuned thresholds while baselines are fixed. read the letter →

arxiv 2507.22469 v1 pith:WLUVP6G5 submitted 2025-07-30 cs.CV cs.AIcs.LG

Viacheslav Pirogov This is my paper
classification cs.CVcs.AIcs.LG
keywords deepfakedetectionvision-languagemodelszero-shotclassificationtokenprobabilityscoringfaceswappingInstructBLIPDFDC-PCelebA-HQ
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that off-the-shelf vision-language models (VLMs) can detect deepfaked faces without any training, if their yes/no token probabilities are converted into a normalized confidence score. Evaluated on a new 60,000-image fake-face dataset built from CelebA-HQ with SimSwap swaps, this zero-shot scoring matched or beat all specialized detectors except one, and the best model, InstructBLIP, reached near-perfect metrics after a single-epoch fine-tune on the DFDC-P dataset. The practical stake is real: a detector that needs no training data and returns a calibrated confidence score could be integrated into liveness checks and identity verification, where thresholds on false acceptance and false rejection matter.

What carries the argument

The load-bearing mechanism is the normalized token-probability score (Eq. 2): for a prompt such as 'Is this photo real?', the model defines fake probability as $P_{\text{no}}/(P_{\text{no}}+P_{\text{yes}})$, where the probabilities come from summing the VLM's first-step token distributions over semantically equivalent yes and no tokens, and similarly for the real probability $P_{\text{yes}}/(P_{\text{no}}+P_{\text{yes}})$. This turns a hard binary answer into a continuous confidence score that supports threshold-based metrics (AUC, EER, FAR/FRR), and it is what the comparison experiments use in place of the earlier binary-answer baseline. The same idea extends to multi-token answer strings and multiple classes via Algorithm 1, which multiplies per-token autoregressive probabilities including an EOS term and normalizes class sums.

What would settle it

Run the same Table 2 comparison with thresholds chosen from a held-out validation split and GPT-4o's prompt frozen before any evaluation image is seen; if a specialized detector such as SBI then matches or exceeds the zero-shot VLM accuracy on the SimSwap dataset, the claimed zero-shot superiority is falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is that a VLM, prompted with a simple 'Is this photo real?' question, yields a usable deepfake score from the first decoding step: instead of taking the argmax token, the authors sum the probabilities of all 'real' tokens and all 'fake' tokens, normalize the two sums to one, and treat the normalized fake probability as the image's deepfake confidence (Eq. 2). In zero-shot use on a novel SimSwap-based dataset, this normalized score lifts accuracy well above the binary argmax baseline and, for most models, above specialized detectors that were not trained on that distribution; InstructBLIP with this scoring is the best VLM, behind only the SBI self-blended-image detector among the specialized methods. Fine-tuning only the Q-Former of InstructBLIP for one epoch on a fraction of DFDC-P yields near-perfect image-level metrics, and the same fine-tuned model retains and even improves its out-of-domain performance on the new CelebA-HQ dataset. The authors generalize the scoring to multi-token answers and multi-class labels (Algorithm 1), giving a route to calibrated probability vectors for several manipulation types at once.

Load-bearing premise

The reported superiority assumes that picking each VLM's threshold from the test set and tuning GPT-4o's prompt on a 100-image sample gives VLMs no unfair advantage over the fixed pretrained detectors they are compared with.

Editorial extensions

If this is right

  • Zero-shot VLM detection can be deployed without collecting or labeling training data, so it can react to a new fake-generation technique as soon as the model is updated.
  • Because the method returns a confidence score rather than a hard yes/no, it can be plugged directly into biometric workflows that set operating thresholds on false acceptance and false rejection rates.
  • A single-epoch fine-tune of only the Q-Former component on a known data distribution yields near-perfect image-level accuracy on that distribution, and the fine-tuned model stays strong out of distribution.
  • The multi-class extension allows one prompt to output a calibrated probability vector over manipulation types (real, face-swap, GAN, diffusion, compression), which could feed downstream risk or compliance decisions.
  • Four of six specialized detectors scored poorly on the unseen SimSwap dataset, so the paper's comparison suggests zero-shot VLMs generalize across generators better than detectors trained on other deepfake datasets.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The authors do not draw out that, if this holds across generators, deepfake detection becomes a prompt-engineering and scoring problem around foundation models rather than a bespoke-classifier training problem.
  • A testable extension the paper does not run: score the same zero-shot VLMs on fakes from a diffusion-based face swap or on re-encoded video to see whether the normalized score keeps its margin over SBI.
  • The paper leaves implicit a deployment caveat: the method's cost (24GB-class GPUs or API fees above $5 per thousand images) may restrict it to high-value checks such as KYC onboarding rather than real-time social-media moderation.
  • The threshold-sensitivity of the reported accuracies suggests that a validation-split threshold protocol could change the ranking; that test is not in the paper.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a zero-shot deepfake detection method based on Visual Language Models (VLMs). Instead of taking the arg-max token from a VQA prompt, the authors sum the token probabilities for 'yes'/'no' answer sets and normalize them to obtain a continuous fake score (Eq. 2). The method is evaluated on a new 60,000-image CelebA-HQ/SimSwap dataset and on DFDC-P. The authors report that zero-shot VLMs outperform most specialized deepfake detectors on the new dataset, and that InstructBLIP fine-tuned for one epoch on DFDC-P achieves near-perfect metrics. The central claim is that VLMs beat trained detectors because of their generalizability and zero-shot capabilities, and that they can be quickly adapted to in-domain data.

Significance. If the comparison were sound, the result would be practically valuable: a simple probabilistic reformulation of VLM outputs would make zero-shot deepfake detection competitive with specialized models, and a light fine-tuning procedure would allow rapid adaptation to new data distributions. The method is simple and potentially reproducible across many open VLMs, which is a strength. However, the paper does not ship code, the main benchmark dataset is from the authors' parallel work and its public availability is unclear, and the reported comparisons are weakened by methodological asymmetries described below. The core idea—using normalized token probabilities rather than binary answers—is reasonable and could be a useful contribution, but the experimental evidence as presented does not yet substantiate the superiority claim.

major comments (3)
  1. [Section 4.1, Tables 1-2] VLM accuracy is reported at the optimal threshold for each model, selected from the grid [0.1, ..., 0.9] on the test set, while the trained detectors (FF, MAT, M2TR, RECCE, CADDM, SBI) are presented as fixed pretrained models with no indication that their thresholds were tuned in the same way. This asymmetry makes the ACC comparisons in Table 2 not apples-to-apples and directly undermines the claim in Section 5 that VLMs 'outperform specifically trained deepfake detectors.' The authors should either fix the threshold to 0.5 for all methods or select thresholds on a held-out split disjoint from the test set, and then re-report both ACC and AUC.
  2. [Section 3.3] The GPT-4o prompt was engineered on a 100-image sample ('We sampled 100 fake and real images'), but the paper never states whether those images were part of the evaluation set. If the tuning sample overlaps the test set, the reported GPT-4o accuracy of 69.2 (Table 1) is optimistically biased. The authors must clarify the provenance of the tuning images and, if necessary, re-evaluate on a disjoint test set or demonstrate that the prompt design did not use test labels.
  3. [Section 4.2, Figure 2] The DFDC-P few-shot experiment fine-tunes InstructBLIP on a training split but compares it only to pretrained detectors that were not fine-tuned on DFDC-P. This does not support the claim that VLMs can be 'quickly and efficiently fine-tuned to achieve near-perfect metrics' relative to traditional methods. A traditional detector (e.g., Xception or SBI) fine-tuned on the same training split should be included as a baseline, otherwise the comparison conflates the benefit of in-domain training with the benefit of the VLM architecture.
minor comments (5)
  1. [Equation 2] The final expression '= ePno' is notationally unclear; presumably it denotes the normalized probability \tilde{P}_{fake}, but as written it appears to introduce a spurious exponential. Please rewrite the equation with explicit definitions.
  2. [Section 3.2] The sentence 'what can be seen at 1 Algorithm 1' is incomplete; the algorithm reference should be integrated grammatically (e.g., 'as shown in Algorithm 1').
  3. [References] Reference (Fraser, 2023) is a tweet URL; this is not an archival citation and should be replaced or removed.
  4. [Table 1] The GPT-4o row has dashes for AUC and EER with no explanation; the paper should state that closed APIs do not expose full token distributions in this setting, so these metrics could not be computed.
  5. [Section 5 and Table 1] The statement that the fine-tuned model 'improving all metrics (see Table 1)' is not directly supported, because Table 1 does not report results for the fine-tuned InstructBLIP on the CelebA-HQ dataset; the referenced result in Section 4.2 is described qualitatively without numbers. Please add the actual metrics.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the normalized-token score is a neutral transformation, the headline comparison is against external pretrained detectors, and the self-cited dataset is an evaluation input rather than a derived result.

full rationale

The paper's derivation chain is Eq. 1 (argmax hard label) to Eq. 2 (normalized first-token probabilities) to thresholded classification. Eq. 2 is a deterministic normalization of the VLM's own output distribution, with no parameter fitted to the labels, so it is not circular by construction. The empirical claims are supported by comparisons against external baselines (FF, MAT, M2TR, RECCE, CADDM, SBI) on a new CelebA-HQ/SimSwap dataset, where the baselines are pretrained detectors taken from prior work. The dataset is sourced from the authors' parallel work (Pirogov & Artemev, 2025), but that citation supplies an evaluation input and motivation, not a theorem or fitted value to which the paper's conclusion reduces. The reported 'optimal threshold' chosen from the grid in Section 4.1 and the prompt engineering on a 100-image sample in Section 3.3 are genuine threats to the fairness of the accuracy comparison: they can inflate the VLM numbers relative to the fixed baselines. However, this is an experimental-design flaw involving test-set tuning, not a self-referential derivation; the AUC columns in Table 2 are threshold-independent and still show the qualitative ordering used in the conclusion. No equation is defined in terms of the quantity it claims to predict, and no load-bearing claim is justified solely by a self-citation. The Limitations section acknowledges the narrow scope of the dataset and unexamined VLM biases, which further supports treating the reported superiority as an empirical evaluation rather than a circular derivation. Therefore no significant circularity is found.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The claims rest on benchmark representativeness, VLM probability calibration, and fair baseline comparison. No new physical or theoretical entities are introduced.

free parameters (2)
  • classification threshold per model = optimal value from the grid [0.1,0.2,...,0.9] on the test set
    Section 4.1 reports accuracy at the optimal threshold on the test set, making the threshold a test-fitted parameter that inflates reported accuracy.
  • GPT-4o prompt template = long role prompt printed in Section 3.3
    The prompt was hand-engineered on a 100-image sample; if those images overlap the evaluation set, the GPT-4o results are tuned rather than zero-shot.
assumptions (4)
  • domain assumption The CelebA-HQ + SimSwap images with gender matching form a representative deepfake benchmark.
    Section 4.1 uses this dataset as the main evaluation ground truth; it only covers one face-swap method and one source dataset.
  • domain assumption First-token VLM probabilities are meaningful confidence estimates for classification.
    Equation 2 in Section 3.1 builds the fake score directly from these probabilities without calibration.
  • domain assumption The SOTA detector results in Table 2 were obtained under the same test-time protocol as the VLM results.
    No evaluation harness is provided; if VLM accuracy uses test-tuned thresholds and baselines do not, the comparison is unfair.
  • domain assumption DFDC-P video labels are valid for image-level frame evaluation.
    Section 4.2 evaluates frames sampled from DFDC-P videos at the image level.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Visual Language Models as Zero-Shot Deepfake Detectors." pith.science (2026). https://pith.science/paper/WLUVP6G5

@misc{pith2026250722469,
  author       = {Pith},
  title        = {Pith review of: Visual Language Models as Zero-Shot Deepfake Detectors},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WLUVP6G5}},
  note         = {Machine review of arXiv:2507.22469}
}
read the original abstract

The contemporary phenomenon of deepfakes, utilizing GAN or diffusion models for face swapping, presents a substantial and evolving threat in digital media, identity verification, and a multitude of other systems. The majority of existing methods for detecting deepfakes rely on training specialized classifiers to distinguish between genuine and manipulated images, focusing only on the image domain without incorporating any auxiliary tasks that could enhance robustness. In this paper, inspired by the zero-shot capabilities of Vision Language Models, we propose a novel VLM-based approach to image classification and then evaluate it for deepfake detection. Specifically, we utilize a new high-quality deepfake dataset comprising 60,000 images, on which our zero-shot models demonstrate superior performance to almost all existing methods. Subsequently, we compare the performance of the best-performing architecture, InstructBLIP, on the popular deepfake dataset DFDC-P against traditional methods in two scenarios: zero-shot and in-domain fine-tuning. Our results demonstrate the superiority of VLMs over traditional classifiers.

Figures

Figures reproduced from arXiv: 2507.22469 by the authors.

Figure 1
Figure 1. Fake samples from deepfake dataset created from CelebA-HQ (Karras et al., 2018) with SimSwap (Chen et al., 2020) 4.1. Unseen dataset To ensure a fair comparison, we used a new deepfake dataset containing 30,000 fake and 30,000 real images based on the CelebA-HQ dataset (Karras et al., 2018), created by using the SOTA face-swapping model SimSwap (Chen et al., 2020), and ensuring gender matching to create more realist… view at source ↗
Figure 2
Figure 2. Performance metrics for InstructBLIP and pretrained deepfake detectors on the DFDC-P (Dolhansky et al., 2020) dataset at the image level. deepfakes originate is known (such as part of the dataset), a Visual Language Model can be easily fine-tuned to achieve exceptional metrics. Notably, when we employed this fine￾tuned InstructBLIP model on our CelebA-HQ deepfake dataset, it demonstrated near state-of-the-art perfor… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

84 extracted references · 32 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Flamingo: a visual language model for few-shot learning, 2022

    Alayrac, J.-B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., Ring, R., Rutherford, E., Cabi, S., Han, T., Gong, Z., Samangooei, S., Monteiro, M., Menick, J., Borgeaud, S., Brock, A., Nematzadeh, A., Sharifzadeh, S., Binkowski, M., Barreira, R., Vinyals, O., Zisserman, A., and Simonyan, K. Flaming...

    arXiv 2022

  3. [3]

    Large scale GAN training for high fidelity natural image synthesis

    Brock, A., Donahue, J., and Simonyan, K. Large scale GAN training for high fidelity natural image synthesis. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019 . OpenReview.net, 2019. URL https://openreview.net/forum?id=B1xsqj09Fm

    2019

  4. [4]

    Roop unleashed

    C0untFloyd. Roop unleashed. https://github.com/C0untFloyd/roop-unleashed, 2023

    2023

  5. [5]

    End-to-end reconstruction-classification learning for face forgery detection

    Cao, J., Ma, C., Yao, T., Chen, S., Ding, S., and Yang, X. End-to-end reconstruction-classification learning for face forgery detection. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 4103--4112, 2022. doi:10.1109/CVPR52688.2022.00408

    work page arXiv 2022

  6. [6]

    M., and Zisserman, A

    Cao, Q., Shen, L., Xie, W., Parkhi, O. M., and Zisserman, A. Vggface2: A dataset for recognising faces across pose and age. In 13th IEEE International Conference on Automatic Face & Gesture Recognition, FG 2018, Xi'an, China, May 15-19, 2018 , pp.\ 67--74. IEEE Computer Society, 2018. doi:10.1109/FG.2018.00020. URL https://doi.org/10.1109/FG.2018.00020

    arXiv 2018

  7. [7]

    Antifakeprompt: Prompt-tuned vision-language models are fake image detectors

    Chang, Y.-M., Yeh, C., Chiu, W.-C., and Yu, N. Antifakeprompt: Prompt-tuned vision-language models are fake image detectors. arXiv preprint arXiv:2310.17419, 2023

    arXiv 2023

  8. [8]

    Simswap: An efficient framework for high fidelity face swapping

    Chen, R., Chen, X., Ni, B., and Ge, Y. Simswap: An efficient framework for high fidelity face swapping. In Proceedings of the 28th ACM International Conference on Multimedia, MM '20, pp.\ 2003–2011, New York, NY, USA, 2020. Association for Computing Machinery. ISBN 9781450379885. doi:10.1145/3394171.3413630. URL https://doi.org/10.1145/3394171.3413630

    arXiv 2003

Show all 84 references
  1. [9]

    Local relation learning for face forgery detection, 2021

    Chen, S., Yao, T., Chen, Y., Ding, S., Li, J., and Ji, R. Local relation learning for face forgery detection, 2021. URL https://arxiv.org/abs/2105.02577

    2021 arXiv

  2. [10]

    Xception: Deep learning with depthwise separable convolutions

    Chollet, F. Xception: Deep learning with depthwise separable convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.\ 1251--1258, 2017

    2017

  3. [11]

    ComfyUI . Comfyui. https://github.com/comfyanonymous/ComfyUI, 2023. Accessed: 2025-07-29

    2023

  4. [12]

    Dai, W., Li, J., Li, D., Tiong, A. M. H., Zhao, J., Wang, W., Li, B., Fung, P., and Hoi, S. Instructblip: Towards general-purpose vision-language models with instruction tuning, 2023. URL https://arxiv.org/abs/2305.06500

    2023 arXiv

  5. [13]

    DeepSeek-AI, Guo, D., Yang, D., Zhang, H., Song, J., Zhang, R., Xu, R., Zhu, Q., Ma, S., Wang, P., Bi, X., Zhang, X., Yu, X., Wu, Y., Wu, Z. F., Gou, Z., Shao, Z., Li, Z., Gao, Z., Liu, A., Xue, B., Wang, B., Wu, B., Feng, B., Lu, C., Zhao, C., Deng, C., Zhang, C., Ruan, C., D...

    2025 arXiv

  6. [14]

    Arcface: Additive angular margin loss for deep face recognition

    Deng, J., Guo, J., Xue, N., and Zafeiriou, S. Arcface: Additive angular margin loss for deep face recognition. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019 , pp.\ 4690--4699. Computer Vision Foundation / IEEE ,...

    2019

  7. [15]

    Dolhansky, B., Bitton, J., Pflaum, B., Lu, J., Howes, R., Wang, M., and Ferrer, C. C. The deepfake detection challenge (dfdc) dataset, 2020. URL https://arxiv.org/abs/2006.07397

    2020 arXiv

  8. [16]

    Implicit identity leakage: The stumbling block to improving deepfake detection generalization

    Dong, S., Wang, J., Ji, R., Liang, J., Fan, H., and Ge, Z. Implicit identity leakage: The stumbling block to improving deepfake detection generalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 3994--4004, 2023

    2023

  9. [17]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020

    2010 arXiv

  10. [18]

    Asking chatgpt to generate a random number

    Fraser, C. Asking chatgpt to generate a random number. https://x.com/colin_fraser/status/1636755134679224320, 2023. Accessed: 2025-07-29

    2023

  11. [19]

    J., Pouget - Abadie, J., Mirza, M., Xu, B., Warde - Farley, D., Ozair, S., Courville, A

    Goodfellow, I. J., Pouget - Abadie, J., Mirza, M., Xu, B., Warde - Farley, D., Ozair, S., Courville, A. C., and Bengio, Y. Generative adversarial nets. In Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N. D., and Weinberger, K. Q. (eds.), Advances in Neural Information Pro...

    2014

  12. [20]

    Google colab

    Google LLC . Google colab. https://colab.research.google.com/, 2020. Accessed: 2025-07-29

    2020

  13. [21]

    Google LLC . Veo 3. https://deepmind.google/models/veo/, 2025. Accessed: 2025-07-29

    2025

  14. [22]

    Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., and Courville, A. C. Improved training of wasserstein gans. In Guyon, I., von Luxburg, U., Bengio, S., Wallach, H. M., Fergus, R., Vishwanathan, S. V. N., and Garnett, R. (eds.), Advances in Neural Information Processing Sy...

    2017

  15. [23]

    Lips don't lie: A generalisable and robust approach to face forgery detection, 2021

    Haliassos, A., Vougioukas, K., Petridis, S., and Pantic, M. Lips don't lie: A generalisable and robust approach to face forgery detection, 2021. URL https://arxiv.org/abs/2012.07657

    2021 arXiv

  16. [24]

    Deepfake detection using deep learning methods: A systematic and comprehensive review

    Heidari, A., Jafari Navimipour, N., Dag, H., and Unal, M. Deepfake detection using deep learning methods: A systematic and comprehensive review. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 14 0 (2): 0 e1520, 2024

    2024

  17. [25]

    The curious case of neural text degeneration, 2020

    Holtzman, A., Buys, J., Du, L., Forbes, M., and Choi, Y. The curious case of neural text degeneration, 2020. URL https://arxiv.org/abs/1904.09751

    2020 arXiv

  18. [26]

    A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions, 2023

    Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., and Liu, T. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions, 2023. URL https://arxiv.org/abs/2311.05232

    2023 arXiv

  19. [27]

    and Belongie, S

    Huang, X. and Belongie, S. J. Arbitrary style transfer in real-time with adaptive instance normalization. In IEEE International Conference on Computer Vision, ICCV 2017, Venice, Italy, October 22-29, 2017 , pp.\ 1510--1519. IEEE Computer Society, 2017. doi:10.1109/ICCV.2017.16...

    2017 doi

  20. [28]

    Isola, P., Zhu, J., Zhou, T., and Efros, A. A. Image-to-image translation with conditional adversarial networks. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017 , pp.\ 5967--5976. IEEE Computer Society, 2017. d...

    2017 doi

  21. [29]

    Perceiver: General perception with iterative attention, 2021

    Jaegle, A., Gimeno, F., Brock, A., Zisserman, A., Vinyals, O., and Carreira, J. Perceiver: General perception with iterative attention, 2021. URL https://arxiv.org/abs/2103.03206

    2021 arXiv

  22. [30]

    Can chatgpt detect deepfakes? a study of using multimodal large language models for media forensics, 2024

    Jia, S., Lyu, R., Zhao, K., Chen, Y., Yan, Z., Ju, Y., Hu, C., Li, X., Wu, B., and Lyu, S. Can chatgpt detect deepfakes? a study of using multimodal large language models for media forensics, 2024. URL https://arxiv.org/abs/2403.14077

    2024 arXiv

  23. [31]

    Jia Guo, J. D. Insightface: 2d and 3d face analysis project. https://github.com/deepinsight/insightface, 2017

    2017

  24. [32]

    Jiang, L., Li, R., Wu, W., Qian, C., and Loy, C. C. Deeperforensics-1.0: A large-scale dataset for real-world face forgery detection, 2020. URL https://arxiv.org/abs/2001.03024

    2020 arXiv

  25. [33]

    Progressive growing of gans for improved quality, stability, and variation, 2018

    Karras, T., Aila, T., Laine, S., and Lehtinen, J. Progressive growing of gans for improved quality, stability, and variation, 2018. URL https://arxiv.org/abs/1710.10196

    2018 arXiv

  26. [34]

    A style-based generator architecture for generative adversarial networks

    Karras, T., Laine, S., and Aila, T. A style-based generator architecture for generative adversarial networks. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019 , pp.\ 4401--4410. Computer Vision Foundation / IEEE , ...

    2019

  27. [35]

    Y., Salakhutdinov, R., and Fried, D

    Koh, J. Y., Salakhutdinov, R., and Fried, D. Grounding language models to images for multimodal inputs and outputs, 2023. URL https://arxiv.org/abs/2301.13823

    2023 arXiv

  28. [36]

    What matters when building vision-language models?, 2024

    Laurençon, H., Tronchon, L., Cord, M., and Sanh, V. What matters when building vision-language models?, 2024. URL https://arxiv.org/abs/2405.02246

    2024 arXiv

  29. [37]

    Why do facial deepfake detectors fail? In Proceedings of the 2nd Workshop on Security Implications of Deepfakes and Cheapfakes, pp.\ 24--28, 2023

    Le, B., Tariq, S., Abuadbba, A., Moore, K., and Woo, S. Why do facial deepfake detectors fail? In Proceedings of the 2nd Workshop on Security Implications of Deepfakes and Cheapfakes, pp.\ 24--28, 2023

    2023

  30. [38]

    Le, B. M. and Woo, S. S. Add: Frequency attention and multi-view based knowledge distillation to detect low-quality compressed deepfake images, 2021. URL https://arxiv.org/abs/2112.03553

    2021 arXiv

  31. [39]

    Le, B. M. and Woo, S. S. Quality-agnostic deepfake detection with intra-model collaborative learning, 2023. URL https://arxiv.org/abs/2309.05911

    2023 arXiv

  32. [40]

    M., Kim, J., Tariq, S., Moore, K., Abuadbba, A., and Woo, S

    Le, B. M., Kim, J., Tariq, S., Moore, K., Abuadbba, A., and Woo, S. S. Sok: Facial deepfake detectors. arXiv preprint arXiv:2401.04364, 2024

    2024 arXiv

  33. [41]

    Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation, 2022

    Li, J., Li, D., Xiong, C., and Hoi, S. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation, 2022. URL https://arxiv.org/abs/2201.12086

    2022 arXiv

  34. [42]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models, 2023

    Li, J., Li, D., Savarese, S., and Hoi, S. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models, 2023. URL https://arxiv.org/abs/2301.12597

    2023 arXiv

  35. [43]

    Faceshifter: Towards high fidelity and occlusion aware face swapping

    Li, L., Bao, J., Yang, H., Chen, D., and Wen, F. Faceshifter: Towards high fidelity and occlusion aware face swapping. arXiv preprint arXiv:1912.13457, 2019

    1912 arXiv

  36. [44]

    Celeb-df: A large-scale challenging dataset for deepfake forensics, 2020

    Li, Y., Yang, X., Sun, P., Qi, H., and Lyu, S. Celeb-df: A large-scale challenging dataset for deepfake forensics, 2020. URL https://arxiv.org/abs/1909.12962

    2020 arXiv

  37. [45]

    Fakebench: Uncover the achilles' heels of fake images with large multimodal models, 2024

    Li, Y., Liu, X., Wang, X., Wang, S., and Lin, W. Fakebench: Uncover the achilles' heels of fake images with large multimodal models, 2024. URL https://arxiv.org/abs/2404.13306

    2024 arXiv

  38. [46]

    L., and Dollár, P

    Lin, T.-Y., Maire, M., Belongie, S., Bourdev, L., Girshick, R., Hays, J., Perona, P., Ramanan, D., Zitnick, C. L., and Dollár, P. Microsoft coco: Common objects in context, 2015. URL https://arxiv.org/abs/1405.0312

    2015 arXiv

  39. [47]

    Liu, H., Li, C., Wu, Q., and Lee, Y. J. Visual instruction tuning, 2023. URL https://arxiv.org/abs/2304.08485

    2023 arXiv

  40. [48]

    Liu, H., Li, C., Li, Y., and Lee, Y. J. Improved baselines with visual instruction tuning, 2024 a . URL https://arxiv.org/abs/2310.03744

    2024 arXiv

  41. [49]

    Few-shot unsupervised image-to-image translation

    Liu, M., Huang, X., Mallya, A., Karras, T., Aila, T., Lehtinen, J., and Kautz, J. Few-shot unsupervised image-to-image translation. In 2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019 , pp.\ 10550--10559....

    2019

  42. [50]

    Liu, P., Tao, Q., and Zhou, J. T. Evolving from single-modal to multi-modal facial deepfake detection: A survey. arXiv preprint arXiv:2406.06965, 2024 b

    2024 arXiv

  43. [51]

    and Hutter, F

    Loshchilov, I. and Hutter, F. Decoupled weight decay regularization, 2019. URL https://arxiv.org/abs/1711.05101

    2019 arXiv

  44. [52]

    Fooocus-inswapper

    machineminded. Fooocus-inswapper. https://github.com/machineminded/Fooocus-inswapper, 2024

    2024

  45. [53]

    H., Yamagishi, J., and Echizen, I

    Nguyen, H. H., Yamagishi, J., and Echizen, I. Capsule-forensics: Using capsule networks to detect forged images and videos, 2018. URL https://arxiv.org/abs/1810.11215

    2018 arXiv

  46. [54]

    FSGAN : Subject agnostic face swapping and reenactment

    Nirkin, Y., Keller, Y., and Hassner, T. FSGAN : Subject agnostic face swapping and reenactment. In Proceedings of the IEEE International Conference on Computer Vision, pp.\ 7184--7193, 2019

    2019

  47. [55]

    Fsganv2: Improved subject agnostic face swapping and reenactment

    Nirkin, Y., Keller, Y., and Hassner, T. Fsganv2: Improved subject agnostic face swapping and reenactment. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022

    2022

  48. [56]

    Introducing chatgpt

    OpenAI. Introducing chatgpt. https://openai.com/index/chatgpt/, 2022. Accessed: 2025-07-29

    2022

  49. [57]

    S., RP, L., Jiang, J., et al

    Perov, I., Gao, D., Chervoniy, N., Liu, K., Marangonda, S., Um \'e , C., Dpfks, M., Facenheim, C. S., RP, L., Jiang, J., et al. Deepfacelab: Integrated, flexible and extensible face-swapping framework. arXiv preprint arXiv:2005.05535, 2020

    2005 arXiv

  50. [58]

    and Artemev, M

    Pirogov, V. and Artemev, M. Evaluating deepfake detectors in the wild, 2025. URL https://arxiv.org/abs/2507.21905

    2025 arXiv

  51. [59]

    Thinking in frequency: Face forgery detection by mining frequency-aware clues, 2020

    Qian, Y., Yin, G., Sheng, L., Chen, Z., and Shao, J. Thinking in frequency: Face forgery detection by mining frequency-aware clues, 2020. URL https://arxiv.org/abs/2007.09355

    2020 arXiv

  52. [60]

    W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., and Sutskever, I

    Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., and Sutskever, I. Learning transferable visual models from natural language supervision. CoRR, abs/2103.00020, 2021. URL https://arxiv.org/abs/2...

    2021 arXiv

  53. [61]

    Towards the detection of diffusion model deepfakes

    Ricker, J., Damm, S., Holz, T., and Fischer, A. Towards the detection of diffusion model deepfakes. arXiv preprint arXiv:2210.14571, 2022

    2022 arXiv

  54. [62]

    Faceforensics++: Learning to detect manipulated facial images

    Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., and Nie ner, M. Faceforensics++: Learning to detect manipulated facial images. In Proceedings of the IEEE/CVF international conference on computer vision, pp.\ 1--11, 2019

    2019

  55. [63]

    Salesforce. Lavis. https://github.com/salesforce/LAVIS, 2022. Accessed: 2025-07-29

    2022

  56. [64]

    Sangwan, S. Roop. https://github.com/s0md3v/roop, 2023 a

    2023

  57. [65]

    Roop for stablediffusion

    Sangwan, S. Roop for stablediffusion. https://github.com/s0md3v/sd-webui-roop, 2023 b

    2023

  58. [66]

    Shield : An evaluation benchmark for face spoofing and forgery detection with multimodal large language models, 2024

    Shi, Y., Gao, Y., Lai, Y., Wang, H., Feng, J., He, L., Wan, J., Chen, C., Yu, Z., and Cao, X. Shield : An evaluation benchmark for face spoofing and forgery detection with multimodal large language models, 2024. URL https://arxiv.org/abs/2402.04178

    2024 arXiv

  59. [67]

    and Yamasaki, T

    Shiohara, K. and Yamasaki, T. Detecting deepfakes with self-blended images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 18720--18729, 2022

    2022

  60. [68]

    Adaptive face forgery detection in cross domain

    Song, L., Fang, Z., Li, X., Dong, X., Jin, Z., Chen, Y., and Lyu, S. Adaptive face forgery detection in cross domain. In Avidan, S., Brostow, G., Ciss \'e , M., Farinella, G. M., and Hassner, T. (eds.), Computer Vision -- ECCV 2022, pp.\ 467--484, Cham, 2022. Springer Nature S...

    2022

  61. [69]

    Fraud report 2024

    Sumsub . Fraud report 2024. https://sumsub.com/fraud-report-2024/, 2024. Accessed: 2025-07-29

    2024

  62. [70]

    Domain general face forgery detection by learning to weight

    Sun, K., Liu, H., Ye, Q., Gao, Y., Liu, J., Shao, L., and Ji, R. Domain general face forgery detection by learning to weight. Proceedings of the AAAI Conference on Artificial Intelligence, 35 0 (3): 0 2638--2646, May 2021 a . doi:10.1609/aaai.v35i3.16367. URL https://ojs.aaai....

    2021 doi

  63. [71]

    Dual contrastive learning for general face forgery detection, 2021 b

    Sun, K., Yao, T., Chen, S., Ding, S., L, J., and Ji, R. Dual contrastive learning for general face forgery detection, 2021 b . URL https://arxiv.org/abs/2112.13522

    2021 arXiv

  64. [72]

    Towards general visual-linguistic face forgery detection, 2024

    Sun, K., Chen, S., Yao, T., Yang, H., Sun, X., Ding, S., and Ji, R. Towards general visual-linguistic face forgery detection, 2024. URL https://arxiv.org/abs/2307.16545

    2024 arXiv

  65. [73]

    One detector to rule them all: Towards a general deepfake attack detection framework

    Tariq, S., Lee, S., and Woo, S. One detector to rule them all: Towards a general deepfake attack detection framework. In Proceedings of the Web Conference 2021, WWW ’21. ACM, April 2021. doi:10.1145/3442381.3449809. URL http://dx.doi.org/10.1145/3442381.3449809

    2021

  66. [74]

    Gemini: A family of highly capable multimodal models, 2024 a

    Team, G. Gemini: A family of highly capable multimodal models, 2024 a . URL https://arxiv.org/abs/2312.11805

    2024 arXiv

  67. [75]

    Gpt-4 technical report, 2024 b

    Team, O. Gpt-4 technical report, 2024 b . URL https://arxiv.org/abs/2303.08774

    2024 arXiv

  68. [76]

    N., Kaiser, ., and Polosukhin, I

    Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, ., and Polosukhin, I. Attention is all you need. Advances in neural information processing systems, 30, 2017

    2017

  69. [77]

    M2tr: Multi-modal multi-scale transformers for deepfake detection

    Wang, J., Wu, Z., Ouyang, W., Han, X., Chen, J., Jiang, Y.-G., and Li, S.-N. M2tr: Multi-modal multi-scale transformers for deepfake detection. In Proceedings of the 2022 international conference on multimedia retrieval, pp.\ 615--623, 2022

    2022

  70. [78]

    Chain-of-thought prompting elicits reasoning in large language models, 2023

    Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., and Zhou, D. Chain-of-thought prompting elicits reasoning in large language models, 2023. URL https://arxiv.org/abs/2201.11903

    2023 arXiv

  71. [79]

    Few-shot classification with feature map reconstruction networks

    Wertheimer, D., Tang, L., and Hariharan, B. Few-shot classification with feature map reconstruction networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.\ 8012--8021, 2021

    2021

  72. [80]

    C., and Nie, Y

    Wu, Y., Zhang, P., Xiong, W., Oguz, B., Gee, J. C., and Nie, Y. The role of chain-of-thought in complex vision-language reasoning task, 2023. URL https://arxiv.org/abs/2311.09193

    2023 arXiv

  73. [81]

    Hallucination is inevitable: An innate limitation of large language models, 2024

    Xu, Z., Jain, S., and Kankanhalli, M. Hallucination is inevitable: An innate limitation of large language models, 2024. URL https://arxiv.org/abs/2401.11817

    2024 arXiv

  74. [82]

    Coca: Contrastive captioners are image-text foundation models, 2022

    Yu, J., Wang, Z., Vasudevan, V., Yeung, L., Seyedhosseini, M., and Wu, Y. Coca: Contrastive captioners are image-text foundation models, 2022. URL https://arxiv.org/abs/2205.01917

    2022 arXiv

  75. [83]

    Common sense reasoning for deep fake detection, 2024

    Zhang, Y., Colman, B., Shahriyari, A., and Bharaj, G. Common sense reasoning for deep fake detection, 2024. URL https://arxiv.org/abs/2402.00126

    2024 arXiv

  76. [84]

    Multi-attentional deepfake detection

    Zhao, H., Zhou, W., Chen, D., Wei, T., Zhang, W., and Yu, N. Multi-attentional deepfake detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.\ 2185--2194, 2021

    2021

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.