Pith. sign in

REVIEW 4 major objections 5 minor 98 references

From Prediction to Explanation: Multimodal, Explainable, and Interactive Deepfake Detection Framework for Non-Expert Users

T0 review · 4 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A deepfake detector can explain its verdicts in heatmaps, captions, and narratives while keeping competitive accuracy.

desk verdict A useful integration of existing pieces for explainable deepfake detection, but the explanation-quality claims outrun the evidence. read the letter →

arxiv 2508.07596 v1 pith:75BMKIQP submitted 2025-08-11 cs.CV

Shahroz Tariq , Simon S. Woo , Priyanka Singh , Irena Irmalasari , Saakshi Gupta , Dev Gupta This is my paper
classification cs.CV
keywords deepfakedetectionexplainableAIGrad-CAMimagecaptioninglargelanguagemodelsmultimodalhuman-centeredDF40benchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that deepfake detection can be made trustworthy for non-experts by wrapping a standard classifier in a three-stage explanation pipeline: a Grad-CAM heatmap localizes the decision, an image captioner translates that heatmap into words, and a fine-tuned large language model turns the caption into a narrative explanation. The authors instantiate this on the DF40 benchmark with a CLIP-large backbone and show the detection accuracy stays competitive (average AUC 0.913) while the generated captions align with the saliency maps, and a six-person user study rates the explanations useful (4.5/5). The point is that interpretability does not have to be sacrificed for accuracy if the explanation is layered and grounded in the model's own activations.

What carries the argument

The carrying mechanism is the three-module pipeline (classifier with Grad-CAM → image captioning → LLM narrative refinement). Grad-CAM produces the saliency map $A_{cam}$ that anchors all downstream text; the captioner $g$ maps (image, heatmap) to a caption; the LLM $h$ refines caption, image, and heatmap into a user-adapted explanation. The claim's plausibility rests on this grounding chain: visual localization is translated into semantic description and then into narrative, so every sentence traces back to a heatmap region.

What would settle it

Measure the overlap between Grad-CAM-highlighted regions and ground-truth manipulation masks on a large set of annotated deepfakes (e.g., Face X-Ray masks); if average IoU is near zero, the textual explanations are not grounded in the true artefacts. A complementary test: run the classifier with randomized weights and show that the generated captions remain confident, which would reveal the text is not actually tied to the model's reasoning.

Watch

Extended reading notes

Core claim

The central claim is that a deepfake detector can be made transparent without losing accuracy by treating explanation as a first-class output. Concretely, the DF-P2E system maps an input image to a tuple (prediction, Grad-CAM heatmap, caption, narrative), where each explanation layer is derived from the previous one. On DF40, the CLIP-large backbone achieves 0.913 average AUC, the BLIP2-Flan-T5-xxl captioner scores best on CIDEr (1.461), and the deployed BLIP-large captioner produces descriptions that refer to the same facial regions the heatmap highlights. The authors take this as evidence that a saliency map can be semantically grounded into natural language that non-experts can understand

Load-bearing premise

The whole explanation chain assumes Grad-CAM's heatmap actually points at the manipulation cues that drove the classifier; if the heatmap is not faithful, the captions and narratives will be fluent but disconnected from the real decision.

Editorial extensions

If this is right

  • If the grounding chain holds, a non-expert can receive a deepfake verdict with a transparent reason: the heatmap shows where, the caption says what, and the narrative says why it matters.
  • The framework can be deployed in forensic, journalistic, or legal workflows where a bare probability score is not actionable, because each output trace links back to a visual region.
  • The modular design means the classifier, captioner, or narrative layer can be swapped independently, allowing future improvements in any one module without rebuilding the pipeline.
  • Competitive detection performance (0.913 AUC with CLIP-large) suggests that adding explanation layers does not have to degrade accuracy, addressing a common accuracy-versus-interpretability trade-off.
  • The interactive interface lets users ask follow-up questions, turning a one-shot prediction into a dialogue that can resolve doubts before a decision is made.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An editorial inference: the six-person user study measures perceived usefulness, not decision quality; a natural next experiment is to test whether the explanations actually improve a user's ability to distinguish real from fake, and whether they induce over-trust in the classifier's verdict.
  • A testable extension would be to replace Grad-CAM with a more principled attribution method (e.g., integrated gradients or concept-based explanations) and measure whether caption fidelity and user trust improve, exactly the direction the paper flags as future work.
  • Because the captioner is trained on MSCOCO plus a small custom Grad-CAM dataset, the framework may inherit a bias toward generic facial descriptions; evaluating on unseen generator families would reveal whether the linguistic alignment is robust.
  • If the grounding assumption fails, the framework still produces helpful-looking narratives that could mislead non-experts more than a plain confidence score would — an editorial caution that follows from the paper's own caveat about Grad-CAM faithfulness.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. DF-P2E is a modular pipeline for explainable deepfake image detection. It combines: (1) a binary classifier (Xception, CLIP-base, CLIP-large) fine-tuned on DF40, with Grad-CAM saliency; (2) an image-captioning module (14 model variants, BLIP-large selected for deployment) fine-tuned on a corpus of Grad-CAM overlays with human annotations; and (3) a LLaMA-3.2-11B-Vision narrative module fine-tuned with QLoRA to turn captions into user-targeted narratives. Evaluation includes frame-level AUC on DF40 subsets (Table 1), captioning metrics (Table 2), and a six-participant usability study (Table 3). The paper claims competitive detection performance and high-quality, trustworthy explanations for non-expert users.

Significance. The detection component is credible: reusing the DF40 public weights reproduces the benchmark ranking, with CLIP-large reaching average AUC 0.913, and the captioning comparison across 14 models is a useful engineering contribution. If the explanation-fidelity claims were substantiated, the framework would be a valuable step toward human-centered forensic tools. However, the explanation-quality evaluation is currently too weak to support the central claim: the captioning evaluation is partly circular, the narrative layer is unevaluated beyond perceived-usefulness ratings, and the user study is too small and lacks controls. As it stands, the paper is a promising proof-of-concept rather than a validated explainability method.

major comments (4)
  1. [Section 6; Section 3, Eqs. (3)-(4)] The Limitations subsection ('Dependence on visual explanations') concedes that Grad-CAM maps are not guaranteed to be faithful, citing Adebayo et al. This is load-bearing because the captioning and narrative modules consume Acam directly (Eqs. (3)-(4)); any unfaithful heatmap propagates into fluent but misleading language. The paper provides no quantitative check on this risk: no comparison to ground-truth manipulation regions, no model-randomization sanity check, and no perturbation test linking caption content to the classifier's decision. The abstract's 'high-quality explanations aligned with Grad-CAM activations' therefore equates alignment with the heatmap to fidelity to actual manipulation cues, which is unestablished.
  2. [Section 4, 'Visual Captioning Module'; Table 2] The captioning models are fine-tuned on a custom corpus of 'Grad-CAM overlays paired with region-level human annotations.' Table 2 then reports BLEU/CIDEr/SPICE against annotations of those overlays. High scores therefore measure how well the model describes the heatmap's appearance, not whether the caption identifies true manipulation cues. This is a partly self-referential evaluation for the claim of forensic specificity. An external signal is needed: ground-truth manipulation masks, perturbation/faithfulness tests, or at minimum a baseline comparing captions generated with and without the heatmap.
  3. [Section 4, 'Human-Centred Evaluation'; Table 3] The user study has n=6, no baseline condition, no error bars, and no significance testing. Section 6 admits it is not sufficient for 'statistically significant generalization,' yet Section 5 concludes 'strong user confidence across all dimensions' and the Introduction claims the system 'significantly improving user trust.' These statements are not supported by the presented data. A larger controlled study comparing against a saliency-only or label-only condition, or appropriately hedged language, is needed before the human-interpretability claim can be assessed.
  4. [Section 4, 'Narrative Refinement Module'] The fine-tuned LLaMA-3.2-11B-Vision narrative layer is a core contribution, but the paper reports no direct evaluation of the narratives themselves. The only evidence is the n=6 perceived-usefulness ratings; there is no hallucination check, no comparison to the unfine-tuned base model, and no quantitative or qualitative measure of whether the narrative statements are grounded in the image and heatmap. Given the acknowledged 'LLM hallucination risk' in Section 6, the claim of 'high-quality explanations' at the narrative level is currently unsupported.
minor comments (5)
  1. [Section 7, Conclusion] 'Extensive experimental evaluation across twelve benchmark datasets' is not supported by the experimental section, which uses DF40 (with subsets reported in Table 1). Please correct or rephrase.
  2. [Section 2, 'LLMs for Explanation Enrichment'] Reference [19] is cited as 'GPT [19]', but [19] is VisionGPT2, a vision-language model. Please update the citation to the intended GPT work.
  3. [Table 2 caption] The note on metric ranges is confusing ('the best score is the closest to 1 while for CIDEr is 10'), and the reported PaliGemma-448 CIDEr value of 0.067 appears inconsistent with the stated range. Clarify.
  4. [Table 2, time columns] Loading time, per-image processing time, all-image total, and total time are presented without consistent units, making latency comparisons difficult. Specify units clearly and report end-to-end per-sample latency for the deployed pipeline.
  5. [Section 4, 'Narrative Refinement Module'] The QLoRA fine-tuning corpus is not described in terms of size, source, or train/validation split. Please include basic statistics and hyperparameters for reproducibility.

Circularity Check

1 steps flagged · score 6.0 of 10

Explanation-quality evaluation is self-referential: captions are fine-tuned on and evaluated against descriptions of the same Grad-CAM heatmap that is their input, so 'alignment with Grad-CAM' is true by construction.

  1. self definitional [Section 4 (Visual Captioning Module) and Section 5 (Captioning and Explanation Quality); abstract claim 'high-quality explanations aligned with Grad-CAM activations']
    "All models are fine-tuned on a hybrid corpus of MSCOCO [43] and a task-specific dataset curated for this study. Our custom dataset includes Grad-CAM overlays paired with region-level human annotations highlighting manipulation artefacts (e.g., warped eye edges, texture inconsistencies)."

    The captioning module is defined as c = g_phi(x, Acam), and its fine-tuning data are Grad-CAM overlays paired with human descriptions of those overlays. Table 2's BLEU/CIDEr scores compare generated captions against these overlay-derived references, while Figure 6's 'qualitative alignment' is explicitly with the Grad-CAM heatmaps. Thus the abstract's 'high-quality explanations aligned with Grad-CAM activations' is guaranteed by the training/evaluation setup: the reference describes the very heatmap that is the model's input. The scores show the model can verbalize heatmap color layout, not that the text identifies true manipulation cues. The paper itself concedes in Section 6 that Grad-CAM 'are not guaranteed to be faithful in all cases [1]', so this alignment metric cannot independently e

full rationale

The detection component is externally validated: CLIP-large is fine-tuned and evaluated on the DF40 benchmark following its protocol, and the AUC results are consistent with the benchmark's published ranking. That portion is not circular. The circularity is confined to the explanation-quality claim: the captioning model consumes the Grad-CAM heatmap Acam and is trained on human annotations of Grad-CAM overlays; the paper then reports that generated captions align with Acam and calls this 'high-quality explanations.' Because Acam's faithfulness to the classifier's true manipulation cues is explicitly conceded as unguaranteed, the alignment metric is a self-description of the model's input rather than independent evidence that the text explains why the image is fake. The narrative module inherits this issue, and the human study (n=6, no baseline, no significance testing) measures perceived usefulness rather than factual fidelity. No load-bearing self-citation chain or imported uniqueness theorem was found; the central weakness is the self-referential explanation evaluation, while detection accuracy itself remains an external benchmark result.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The framework introduces no new physical or conceptual entities; it composes existing models. The main choices the central claim rests on are model selections made after seeing evaluation results (CLIP-large, BLIP-large) and an undisclosed data mixture for captioning fine-tuning. The faithfulness of Grad-CAM and the representativeness of the six-participant user study are assumptions the paper itself acknowledges.

free parameters (3)
  • Explanation backbone model = CLIP-large
    Selected post hoc as the highest-AUC classifier on DF40 (0.913), then used to generate every Grad-CAM map in the pipeline.
  • Deployed captioning model = BLIP-large
    Selected post hoc from Table 2 as the latency-quality trade-off; BLIP2-Flan-T5-xxl had higher scores but was much slower.
  • Fine-tuning corpus mixture = Unspecified
    The captioning module is fine-tuned on an undisclosed mixture of MSCOCO and a custom Grad-CAM annotation set; the ratio and split are not reported.
assumptions (4)
  • domain assumption Grad-CAM provides a faithful attribution of the classifier's decision.
    The whole explanation pipeline feeds Grad-CAM activations into captioning and narrative generation; the paper itself notes this is not guaranteed (Discussion, citing Adebayo et al.).
  • domain assumption The custom captioning dataset with Grad-CAM overlays and human annotations is representative of the test distribution and was split appropriately.
    The paper does not describe the dataset construction, size, or train/test splits for the fine-tuned captioning models.
  • domain assumption Six self-selected non-expert participants provide meaningful average ratings of usefulness, understandability, and explainability.
    The human evaluation uses n=6 without sampling criteria or statistical analysis, yet average scores are reported as evidence of user confidence.
  • domain assumption The LLM narrative module does not introduce factual errors that undermine the explanation.
    The paper acknowledges LLM hallucination risk in Discussion, but the evaluation does not fact-check generated narratives.

how reviews work

0 comments
Cite this review

Pith. "Pith review of From Prediction to Explanation: Multimodal, Explainable, and Interactive Deepfake Detection Framework for Non-Expert Users." pith.science (2026). https://pith.science/paper/75BMKIQP

@misc{pith2026250807596,
  author       = {Pith},
  title        = {Pith review of: From Prediction to Explanation: Multimodal, Explainable, and Interactive Deepfake Detection Framework for Non-Expert Users},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/75BMKIQP}},
  note         = {Machine review of arXiv:2508.07596}
}
read the original abstract

The proliferation of deepfake technologies poses urgent challenges and serious risks to digital integrity, particularly within critical sectors such as forensics, journalism, and the legal system. While existing detection systems have made significant progress in classification accuracy, they typically function as black-box models, offering limited transparency and minimal support for human reasoning. This lack of interpretability hinders their usability in real-world decision-making contexts, especially for non-expert users. In this paper, we present DF-P2E (Deepfake: Prediction to Explanation), a novel multimodal framework that integrates visual, semantic, and narrative layers of explanation to make deepfake detection interpretable and accessible. The framework consists of three modular components: (1) a deepfake classifier with Grad-CAM-based saliency visualisation, (2) a visual captioning module that generates natural language summaries of manipulated regions, and (3) a narrative refinement module that uses a fine-tuned Large Language Model (LLM) to produce context-aware, user-sensitive explanations. We instantiate and evaluate the framework on the DF40 benchmark, the most diverse deepfake dataset to date. Experiments demonstrate that our system achieves competitive detection performance while providing high-quality explanations aligned with Grad-CAM activations. By unifying prediction and explanation in a coherent, human-aligned pipeline, this work offers a scalable approach to interpretable deepfake detection, advancing the broader vision of trustworthy and transparent AI systems in adversarial media environments.

Figures

Figures reproduced from arXiv: 2508.07596 by the authors.

Figure 1
Figure 1. Comparison of DF-P2E with existing deepfake detection pipelines. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The overall workflow of the framework. Image Captioning for Visual Interpretation. Image-to￾text models offer a pathway to bridge the visual-linguistic divide by translating images into natural language descriptions. Modern approaches such as BLIP [38], ViT-GPT2 [19], and OFA [84] employ pre-trained vision encoders and autoregressive language decoders to generate semantically rich captions. These systems are typical… view at source ↗
Figure 3
Figure 3. The user interface of our deployed application. One of the motivating scenarios (bottom-right). [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (1 more)
Figure 5
Figure 5. Figure 5: The image-to-text model validation result of the [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

98 extracted references · 56 canonical work pages

  1. [1]

    Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, and Been Kim. 2018. Sanity checks for saliency maps. Advances in neural information processing systems 31 (2018)

    2018

  2. [2]

    Darius Afchar, Vincent Nozick, Junichi Yamagishi, and Isao Echizen. 2018. Mesonet: a compact facial video forgery detection network. In 2018 IEEE in- ternational workshop on information forensics and security (WIFS) . IEEE, 1–7

    2018

  3. [3]

    Basem H Ahmed and Motaz Saad. 2021. The Use of Machine Translation to Provide Resources for Under-Resourced Languages-Image Captioning Task. In 2021 Palestinian International Conference on Information and Communication Technology (PICICT). IEEE, 25–29

    2021

  4. [4]

    Ahmed Alqaraawi, Martin Schuessler, Philipp Weiß, Enrico Costanza, and Nadia Berthouze. 2020. Evaluating saliency map explanations for convolutional neural networks: a user study. In Proceedings of the 25th international conference on intelligent user interfaces. 275–285

    2020

  5. [5]

    Ahmed Alqaraawi, Martin Schuessler, Philipp Weiß, Enrico Costanza, and Nadia Berthouze. 2020. Evaluating saliency map explanations for convolutional neural networks: a user study. In Proceedings of the 25th International Conference on Intelligent User Interfaces (Cagliari, Italy) (IUI ’20). Association for Computing Ma- chinery, New York, NY, USA, 275–285...

    work page arXiv 2020

  6. [6]

    Grigoris Antoniou, Katie Atkinson, George Baryannis, Sotiris Batsakis, Luigi Di Caro, Guido Governatori, Livio Robaldo, Giovanni Siragusa, and Ilias Tach- mazidis. 2022. Explainable Reasoning with Legal Big Data: A Layered Framework. Journal of Applied Logic 9 (2022), 1155–170

    2022

  7. [7]

    Lucas Beyer, Andreas Steiner, André Susano Pinto, Alexander Kolesnikov, Xiao Wang, Daniel Salz, Maxim Neumann, Ibrahim Alabdulmohsin, Michael Tschan- nen, Emanuele Bugliarello, et al. 2024. Paligemma: A versatile 3b vlm for transfer. arXiv preprint arXiv:2407.07726 (2024)

    arXiv 2024

  8. [8]

    Shannon Bond. 2024. How deepfakes and AI memes affected global elections in 2024 . https://www.npr.org/2024/12/21/nx-s1-5220301/deepfakes-memes- artificial-intelligence-elections Accessed: 2025-04-12

    2024

Show all 98 references
  1. [9]

    Bartosz Brożek, Michał Furman, Marek Jakubiec, and Bartłomiej Kucharzyk. 2024. The black box problem revisited. Real and imaginary challenges for automated legal decision making. Artificial Intelligence and Law 32, 2 (2024), 427–440. https://doi.org/10.1007/s10506-023-09356-9

    2024 doi

  2. [10]

    Case IQ. 2024. What is Deepfake Identity Theft? https://www.caseiq.com/ resources/what-is-deepfake-identity-theft/ Accessed: 2025-04-12

    2024

  3. [11]

    Ilias Chalkidis and Dimitrios Kampas. 2019. Deep learning in law: early adapta- tion and legal word embeddings trained on large corpora. Artif. Intell. Law 27, 2 (2019), 171–198. https://doi.org/10.1007/s10506-018-9238-9

    2019 doi

  4. [12]

    Heather Chen and Kathleen Magramo. 2024. Finance worker pays out $25 million after video call with deepfake CFO. CNN (2024). https://edition.cnn.com/2024/ 02/04/asia/deepfake-cfo-scam-hong-kong-intl-hnk/index.html Accessed: 2025- 04-10

    2024

  5. [13]

    Jin Chen, Zheng Liu, Xu Huang, Chenwang Wu, Qi Liu, Gangwei Jiang, Yuanhao Pu, Yuxuan Lei, Xiaolong Chen, Xingmei Wang, et al. 2024. When large language models meet personalization: Perspectives of challenges and opportunities.World Wide Web 27, 4 (2024), 42

    2024

  6. [14]

    Yuwei Chen. 2025. Towards Human Explainable Digital Forensics: Generating Human Interpretable Evidence for Semantic Understanding in Manipulated Images and Text. (2025)

    2025

  7. [15]

    Beomsang Cho, Binh M Le, Jiwon Kim, Simon Woo, Shahroz Tariq, Alsharif Abuadbba, and Kristen Moore. 2023. Towards understanding of deepfake videos in the wild. In Proceedings of the 32nd ACM International Conference on Informa- tion and Knowledge Management . 4530–4537

    2023

  8. [16]

    François Chollet. 2017. Xception: Deep learning with depthwise separable con- volutions. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1251–1258

    2017

  9. [17]

    Ben Colman. 2025. 2,100% Surge in AI Fraud as Deepfake Regulation Lags. https://www.realitydefender.com/blog/2-100-surge-in-ai-fraud-as- deepfake-regulation-lags Accessed: 2025-04-12

    2025

  10. [18]

    Hoang Cuong Nguyen, Shahroz Tariq, Mohan Baruwal Chhetri, and Bao Quoc Vo

  11. [19]

    Shreyansh Dani and contributors. 2024. VisionGPT2: A Vision-to-Language Model. https://github.com/shreydan/VisionGPT2. Accessed: 2024-12-23

    2024

  12. [20]

    Shichao Dong, Jin Wang, Renhe Ji, Jiajun Liang, Haoqiang Fan, and Zheng Ge

  13. [21]

    Joel Frank, Thorsten Eisenhofer, Lea Schönherr, Asja Fischer, Dorothea Kolossa, and Thorsten Holz. 2020. Leveraging frequency analysis for deep fake image recognition. In International conference on machine learning . PMLR, 3247–3258

    2020

  14. [22]

    Yingqiang Ge, Wenyue Hua, Kai Mei, Juntao Tan, Shuyuan Xu, Zelong Li, Yongfeng Zhang, et al. 2023. Openagi: When llm meets domain experts.Advances in Neural Information Processing Systems 36 (2023), 5539–5568

    2023

  15. [23]

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2020. Generative adversarial networks. Commun. ACM 63, 11 (2020), 139–144

    2020

  16. [24]

    Brian Hosler, Davide Salvi, Anthony Murray, Fabio Antonacci, Paolo Bestagini, Stefano Tubaro, and Matthew C Stamm. 2021. Do deepfakes feel emotions? A semantic approach to detecting deepfakes via emotional inconsistencies. In Proceedings of the IEEE/CVF conference on computer ...

    2021

  17. [25]

    Mousa Jafar, Mohammad Ababneh, Mohammed Al-Zoube, and Ammar Elhassan

  18. [26]

    Tero Karras, Samuli Laine, and Timo Aila. 2019. A style-based generator ar- chitecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 4401–4410

    2019

  19. [27]

    Hasam Khalid, Minha Kim, Shahroz Tariq, and Simon S Woo. 2021. Evaluation of an Audio-Video Multimodal Deepfake Dataset using Unimodal and Multimodal Detectors. In Proceedings of the 1st Workshop on Synthetic Multimedia-Audiovisual Deepfake Generation and Detection . 7–15

    2021

  20. [28]

    Hasam Khalid, Shahroz Tariq, Minha Kim, and Simon S Woo. 2021. FakeAVCeleb: A Novel Audio-Video Multimodal Deepfake Dataset. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track

    2021

  21. [29]

    Jeongho Kim, Shahroz Tariq, and Simon S Woo. 2022. PTD: Privacy-Preserving Human Face Processing Framework using Tensor Decomposition. In Proceedings of the 37th ACM/SIGAPP Symposium on Applied Computing . 1296–1303. https: //doi.org/10.1145/3477314.3507036

    2022

  22. [30]

    Minha Kim, Shahroz Tariq, and Simon S Woo. 2021. Cored: Generalizing fake media detection with continual representation using distillation. In Proceedings of the 29th ACM International Conference on Multimedia . 337–346

    2021

  23. [31]

    Minha Kim, Shahroz Tariq, and Simon S Woo. 2021. FReTAL: Generalizing Deepfake Detection using Knowledge Distillation and Representation Learn- ing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1001–1012

    2021

  24. [32]

    Vejay Lalla, Adine Mitrani, and Zach Harned. 2022. Artificial In- telligence: Deepfakes in the Entertainment Industry. WIPO Maga- zine (2022). https://www.wipo.int/web/wipo-magazine/articles/artificial- intelligence-deepfakes-in-the-entertainment-industry-42620 Accessed: 2025- 04-10

    2022

  25. [34]

    Le, Jiwon Kim, Simon S

    Binh M. Le, Jiwon Kim, Simon S. Woo, Kristen Moore, Alsharif Abuadbba, and Shahroz Tariq. 2025. SoK: Systematization and Benchmarking of Deepfake Detectors in a Unified Framework. arXiv:2401.04364 [cs.CV] https://arxiv.org/ abs/2401.04364

    2025 arXiv

  26. [35]

    Sangyup Lee, Shahroz Tariq, Junyaup Kim, and Simon S Woo. 2021. TAR: Gen- eralized Forensic Framework to Detect Deepfakes Using Weakly Supervised Learning. In IFIP International Conference on ICT Systems Security and Privacy Protection. Springer, 351–366

    2021

  27. [36]

    Sangyup Lee, Shahroz Tariq, Youjin Shin, and Simon S Woo. 2021. Detecting handcrafted facial image manipulations and GAN-generated facial images using Shallow-FakeFaceNet. Applied Soft Computing 105 (2021), 107256

    2021

  28. [37]

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning . PMLR, 19730–19742

    2023

  29. [38]

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. 2022. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In International conference on machine learning. PMLR, 12888–12900

    2022

  30. [39]

    Lingzhi Li, Jianmin Bao, Ting Zhang, Hao Yang, Dong Chen, Fang Wen, and Baining Guo. [n. d.]. Face x-ray for more general face forgery detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 5001–5010

  31. [40]

    Xuhong Li, Haoyi Xiong, Xingjian Li, Xuanyu Wu, Xiao Zhang, Ji Liu, Jiang Bian, and Dejing Dou. 2022. Interpretable deep learning: Interpretation, interpretability, trustworthiness, and beyond. Knowledge and Information Systems 64, 12 (2022), 3197–3234

    2022

  32. [41]

    Y Li. 2018. Exposing deepfake videos by detecting face warping artif acts. arXiv preprint arXiv:1811.00656 (2018). Tariq et al

    2018 arXiv

  33. [42]

    Yuezun Li, Xin Yang, Pu Sun, Honggang Qi, and Siwei Lyu. 2020. Celeb-df: A large-scale challenging dataset for deepfake forensics. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 3207–3216

    2020

  34. [43]

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. 2014. Microsoft coco: Common objects in context. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proce...

    2014

  35. [44]

    Marco Lippi, Przemysław Pałka, Giuseppe Contissa, Francesca Lagioia, Hans- Wolfgang Micklitz, Giovanni Sartor, and Paolo Torroni. 2019. CLAUDETTE: an automated detector of potentially unfair clauses in online terms of service. Artificial Intelligence and Law 27 (2019), 117–139

    2019

  36. [45]

    Jiawei Liu, Fanrui Zhang, Jiaying Zhu, Esther Sun, Qiang Zhang, and Zheng-Jun Zha. 2024. Forgerygpt: Multimodal large language model for explainable image forgery detection and localization. arXiv preprint arXiv:2410.10238 (2024)

    2024 arXiv

  37. [46]

    Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. 2018. Large-scale celebfaces attributes (celeba) dataset. Retrieved August 15, 2018 (2018), 11

    2018

  38. [47]

    Scott Lundberg. 2017. A unified approach to interpreting model predictions. arXiv preprint arXiv:1705.07874 (2017)

    2017 arXiv

  39. [48]

    Arpan Mahara and Naphtali Rishe. 2025. Methods and Trends in Detecting Generated Images: A Comprehensive Review. arXiv preprint arXiv:2502.15176 (2025)

    2025

  40. [49]

    Manoj Kumar Mahto, Durgesh Srivastava, Ranjit Kumar, Basant Sah, Hare Ram Singh, and Sunil Kr. Maakar. 2025. Personalized User Interaction in Web Ap- plications using Adaptive LLM Model. In 2025 International Conference on Per- vasive Computational Technologies (ICPCT). 962–96...

    2025

  41. [50]

    Abdullahi, and Ahmad Neyaz Khan

    Asad Malik, Minoru Kuribayashi, Sani M. Abdullahi, and Ahmad Neyaz Khan

  42. [51]

    Yisroel Mirsky and Wenke Lee. 2021. The creation and detection of deepfakes: A survey. ACM computing surveys (CSUR) 54, 1 (2021), 1–41

    2021

  43. [52]

    Trisha Mittal, Uttaran Bhattacharya, Rohan Chandra, Aniket Bera, and Dinesh Manocha. 2020. Emotions don’t lie: An audio-visual deepfake detection method using affective cues. In Proceedings of the 28th ACM international conference on multimedia. 2823–2832

    2020

  44. [53]

    Fania Mokhayeri, Eric Granger, and Guillaume-Alexandre Bilodeau. 2015. Syn- thetic face generation under various operational conditions in video surveillance. In 2015 IEEE International Conference on Image Processing (ICIP) . 4052–4056. https://doi.org/10.1109/ICIP.2015.7351567

    2015

  45. [54]

    Yunsheng Ni, Depu Meng, Changqian Yu, Chengbin Quan, Dongchun Ren, and Youjian Zhao. [n. d.]. Core: Consistent representation learning for face forgery detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 12–21

  46. [55]

    Cécile Paris and Andrew Reeson. 2024. What’s the Secret to Making Sure AI Does Not Steal Your Job? Work with It, Not Against It. In The Conversation on Work, Ian O. Williamson (Ed.). Johns Hopkins University Press, Baltimore, 177–181. First published in The Conversation on Nov...

    2024

  47. [56]

    Yogesh Patel, Sudeep Tanwar, Rajesh Gupta, Pronaya Bhattacharya, Inno- cent Ewean Davidson, Royi Nyameko, Srinivas Aluvala, and Vrince Vimal. 2023. Deepfake Generation and Detection: Case Study and Challenges. IEEE Access 11 (2023), 143296–143323. https://doi.org/10.1109/ACCES...

    2023

  48. [57]

    D Philmlee. 2023. Practice Innovations: Seeing is no longer believing—The rise of deepfakes. Thomson Reuters 18 (2023)

    2023

  49. [58]

    Ben Pinhasov, Raz Lapid, Rony Ohayon, Moshe Sipper, and Yehudit Aperstein

  50. [59]

    Yuyang Qian, Guojun Yin, Lu Sheng, Zixuan Chen, and Jing Shao. 2020. Think- ing in frequency: Face forgery detection by mining frequency-aware clues. In European conference on computer vision . Springer, 86–103

    2020

  51. [60]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sand- hini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al

  52. [61]

    Victoria Reed. 2024. The Bright Side of Deepfakes: Edu, Films & Digital Lega- cies. https://aicompetence.org/the-bright-side-of-deepfakes-education-films/ Accessed: 2025-04-12

    2024

  53. [62]

    Why Should I Trust You?

    Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. "Why Should I Trust You?": Explaining the Predictions of Any Classifier. , 1135–1144 pages. https://doi.org/10.1145/2939672.2939778

    2016

  54. [63]

    Andreas Rossler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, and Matthias Nießner. 2019. Faceforensics++: Learning to detect manipulated facial images. In Proceedings of the IEEE/CVF international conference on computer vision. 1–11

    2019

  55. [64]

    Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedan- tam, Devi Parikh, and Dhruv Batra. 2017. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE inter- national conference on computer vision . 618–626

    2017

  56. [65]

    Samuel Henrique Silva, Mazal Bethany, Alexis Megan Votto, Ian Henry Scarff, Nicole Beebe, and Peyman Najafirad. 2022. Deepfake forensics analysis: An explainable hierarchical ensemble of weakly supervised models. Forensic Science International: Synergy 4 (2022), 100217

    2022

  57. [66]

    Eicher-Miller, Toby Jia-Jun Li, Meng Jiang, and Ronald A

    Annalisa Szymanski, Noah Ziems, Heather A. Eicher-Miller, Toby Jia-Jun Li, Meng Jiang, and Ronald A. Metoyer. 2025. Limitations of the LLM-as-a-Judge Approach for Evaluating LLM Outputs in Expert Knowledge Tasks. In Pro- ceedings of the 30th International Conference on Intelli...

    2025

  58. [67]

    Woo, and Shahroz Tariq

    Razaib Tariq, Minji Heo, Simon S. Woo, and Shahroz Tariq. 2024. Beyond the Screen: Evaluating Deepfake Detectors under Moiré Pattern Effects. In Workshop on Media Forensics (CVPR) . Seattle

    2024

  59. [68]

    Woo, and Shahroz Tariq

    Razaib Tariq, Simon S. Woo, and Shahroz Tariq. 2024. Exploring the Impact of Moiré Pattern on Deepfake Detectors. In Special Session Recent Advances in Multimedia Forensics in the AI-Generative Era (ICIP) . Abu Dhabi, UAE

    2024

  60. [69]

    Shahroz Tariq, Alsharif Abuadbba, and Kristen Moore. 2023. Deepfake in the Metaverse: Security Implications for Virtual Gaming, Meetings, and Offices. In Proceedings of the 2nd Workshop on Security Implications of Deepfakes and Cheapfakes (Melbourne, VIC, Australia) (WDC ’23)....

    2023

  61. [70]

    Shahroz Tariq, Mohan Baruwal Chhetri, Surya Nepal, and Cecile Paris. 2025. A2C: A modular multi-stage collaborative decision framework for human–AI teams. Expert Systems with Applications 282 (2025), 127318

    2025

  62. [71]

    Shahroz Tariq, Sowon Jeon, and Simon Woo. 2021. Am I a Real or Fake Celebrity? Measuring Commercial Face Recognition Web APIs under Deepfake Imperson- ation Attack. arXiv preprint arXiv:2103.00847 (2021)

    2021 arXiv

  63. [72]

    Shahroz Tariq, Sowon Jeon, and Simon S Woo. 2023. Evaluating Trustworthiness and Racial Bias in Face Recognition APIs Using Deepfakes.Computer 56, 5 (2023), 51–61

    2023

  64. [73]

    Shahroz Tariq, Sangyup Lee, Hoyoung Kim, Youjin Shin, and Simon S Woo. 2018. Detecting both machine and human created fake face images in the wild. In Proceedings of the 2nd International Workshop on Multimedia Privacy and Security . ACM, 81–87

    2018

  65. [74]

    Shahroz Tariq, Sangyup Lee, Hoyoung Kim, Youjin Shin, and Simon S Woo. 2019. GAN is a friend or foe?: a framework to detect various fake face images. In Proceedings of the 34th ACM/SIGAPP Symposium on Applied Computing . ACM, 1296–1303

    2019

  66. [75]

    Shahroz Tariq, Sangyup Lee, and Simon Woo. 2021. One detector to rule them all: Towards a general deepfake attack detection framework. In Proceedings of the web conference 2021

    2021

  67. [76]

    Shahroz Tariq, Sangyup Lee, and Simon S Woo. 2020. A Convolutional LSTM based Residual Network for Deepfake Video Detection. arXiv preprint arXiv:2009.07480 (2020)

    2020 arXiv

  68. [77]

    Shahroz Tariq, David Nguyen, MAP Chamikara, Tingmin Wu, Alsharif Abuadbba, and Kristen Moore. 2025. LLMs Are Not Yet Ready for Deepfake Image Detection. arXiv preprint arXiv:2506.10474 (2025)

    2025 arXiv

  69. [78]

    Shahroz Tariq, Priyanka Singh, and Simon S. Woo. 2025. DF-P2E Framework Demo: Interactive Deepfake Detection for Non-Expert Users. https://doi.org/10. 5281/zenodo.15198666

    2025

  70. [79]

    Shahroz Tariq, Ronal Singh, Mohan Baruwal Chhetri, Surya Nepal, and Cecile Paris. 2025. Bridging expertise gaps: The role of llms in human-ai collaboration for cybersecurity. arXiv preprint arXiv:2505.03179 (2025)

    2025 arXiv

  71. [80]

    Mulin Tian, Mahyar Khayatkhoei, Joe Mathai, and Wael AbdAlmageed. 2023. Unsupervised multimodal deepfake detection using intra-and cross-modal in- consistencies. arXiv preprint arXiv:2311.17088 (2023)

    2023 arXiv

  72. [81]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, and Faisal Azhar. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 (2023)

    2023 arXiv

  73. [82]

    Jianfeng Wang, Zhengyuan Yang, Xiaowei Hu, Linjie Li, Kevin Lin, Zhe Gan, Zicheng Liu, Ce Liu, and Lijuan Wang. 2022. Git: A generative image-to-text transformer for vision and language. arXiv preprint arXiv:2205.14100 (2022)

    2022 arXiv

  74. [83]

    Luozhou Wang, Shuai Yang, Shu Liu, and Ying-cong Chen. 2023. Not All Steps are Created Equal: Selective Diffusion Distillation for Image Manipulation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) . 7472–7481

    2023

  75. [84]

    Peng Wang, An Yang, Rui Men, Junyang Lin, Shuai Bai, Zhikang Li, Jianxin Ma, Chang Zhou, Jingren Zhou, and Hongxia Yang. 2022. Ofa: Unifying architectures, tasks, and modalities through a simple sequence-to-sequence learning framework. In International conference on machine le...

    2022

  76. [85]

    Saima Waseem, Syed Abdul Rahman Syed Abu Bakar, Bilal Ashfaq Ahmed, Zaid Omar, Taiseer Abdalla Elfadil Eisa, and Mhassen Elnour Elneel Dalam

  77. [86]

    Jun Wei, Shuhui Wang, and Qingming Huang. 2020. F 3Net: fusion, feedback and focus for salient object detection. In Proceedings of the AAAI conference on artificial intelligence, Vol. 34. 12321–12328. DF-P2E: From Prediction to Explanation

    2020

  78. [87]

    Liangbin Xie, Xintao Wang, Honglun Zhang, Chao Dong, and Ying Shan. 2022. Vfhq: A high-quality dataset and benchmark for video face super-resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion. 657–666

    2022

  79. [88]

    Wenju Xu, Shawn Keshmiri, and Guanghui Wang. 2019. Adversarially Approxi- mated Autoencoder for Image Generation and Manipulation.IEEE Transactions on Multimedia 21, 9 (2019), 2387–2396. https://doi.org/10.1109/TMM.2019.2898777

    2019

  80. [89]

    Zhiyuan Yan, Taiping Yao, Shen Chen, Yandan Zhao, Xinghe Fu, Junwei Zhu, Donghao Luo, Li Yuan, Chengjie Wang, Shouhong Ding, et al . 2024. DF40: Toward Next-Generation Deepfake Detection. arXiv preprint arXiv:2406.13495 (2024)

    2024 arXiv

  81. [90]

    Xin Yang, Yuezun Li, and Siwei Lyu. 2019. Exposing deep fakes using inconsistent head poses. In ICASSP 2019-2019 IEEE international conference on acoustics, speech and signal processing (ICASSP) . IEEE, 8261–8265

    2019

  82. [91]

    Yifan Yao, Jinhao Duan, Kaidi Xu, Yuanfang Cai, Zhibo Sun, and Yue Zhang

  83. [92]

    IEEE Access 11 (2023), 117865–117906

    DeepFake on Face and Expression Swap: A Review. IEEE Access 11 (2023), 117865–117906. https://doi.org/10.1109/ACCESS.2023.3324403

    2023

  84. [99]

    High-Confidence Computing 4, 2 (2024), 100211

    A survey on large language model (LLM) security and privacy: The Good, The Bad, and The Ugly. High-Confidence Computing 4, 2 (2024), 100211. https: //doi.org/10.1016/j.hcc.2024.100211

    2024

  85. [2020]

    053–058 pages

    Forensics and Analysis of Deepfake Videos . 053–058 pages. https://doi.org/ 10.1109/ICICS49469.2020.239493

    2020

  86. [2021]

    In International conference on machine learning

    Learning transferable visual models from natural language supervision. In International conference on machine learning . PmLR, 8748–8763

  87. [2022]

    IEEE Access 10 (2022), 18757–18775

    DeepFake Detection for Human Face Images and Videos: A Survey. IEEE Access 10 (2022), 18757–18775. https://doi.org/10.1109/ACCESS.2022.3151186

    2022

  88. [2023]

    In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Implicit identity leakage: The stumbling block to improving deepfake detection generalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 3994–4004

  89. [2024]

    arXiv preprint arXiv:2403.02955 (2024)

    Xai-based detection of adversarial attacks on deepfake detectors. arXiv preprint arXiv:2403.02955 (2024)

    2024 arXiv

  90. [2025]

    In Companion Proceedings of the ACM on Web Conference 2025

    Towards effective identification of attack techniques in cyber threat intel- ligence reports using large language models. In Companion Proceedings of the ACM on Web Conference 2025. 942–946

    2025

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.