Pith. sign in

REVIEW 3 major objections 5 minor 62 references

HAMLET-FFD: Hierarchical Adaptive Multi-modal Learning Embeddings Transformation for Face Forgery Detection

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A frozen CLIP fitted with a bidirectional text-vision loop detects face forgeries from manipulation techniques it never saw, reaching 90.07% average AUC on seven DeepfakeBench benchmarks and 6.68 points above the prior best.

desk verdict Novel bidirectional fusion architecture with a coherent ablation story, but the appendix code doesn't freeze CLIP's text encoder and the headline numbers rely on single runs tuned on the evaluation sets. read the letter →

arxiv 2507.20913 v1 pith:IHUKDWTT submitted 2025-07-28 cs.CV cs.AI

Jialei Cui , Jianwei Du , Yanzhe Li , Lei Gao , Hui Jiang , Chenfu Bao This is my paper
classification cs.CVcs.AI
keywords faceforgerydetectioncross-domaingeneralizationvision-languagemodelsCLIPbidirectionalcross-modalfusionpromptlearninghierarchicalvisualfeaturesdeepfake
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Face forgery detectors trained as binary classifiers tend to memorize artifacts of the manipulation techniques in their training set, so they fail when a new generation of forgery methods appears. HAMLET-FFD proposes instead that the authenticity knowledge needed for generalization is already latent in CLIP's vision-language representations, and that it can be extracted by a closed bidirectional loop in which text embeddings guide visual features and visual features, in turn, refine the text embeddings. The paper reports that a fully frozen CLIP with this plugin reaches 90.07% average AUC on seven unseen manipulation datasets in DeepfakeBench, 6.68 percentage points above the previous best method, and keeps the lead on diffusion-based and in-the-wild forgeries. The practical stake is that forgery detectors could be updated for new attack families without retraining or degrading the underlying general-purpose model.

What carries the argument

The load-bearing object is the bidirectional modal fusion module operating on hierarchical patch tokens from a frozen CLIP vision transformer. For each of the three embedding types, the text tokens are average-pooled, passed through SiLU and a linear layer, and turned into per-patch scale and shift parameters, a text-to-visual conditioning step. The modulated patch embeddings from multiple transformer blocks are integrated by self-attention and average pooling, and a cross-attention layer then uses the integrated visual features as keys and values to refine each text embedding, the visual-to-text step. These conditioned prompts are encoded by the frozen text transformer. Around this loop sits a progressive cross-entropy loss that enforces a similarity hierarchy—real above context above fake above the prior prompt, or the reverse for fakes—which is what presses the embeddings to encode graded authenticity rather than a single binary score.

What would settle it

Retrain HAMLET-FFD with every architectural choice fixed before inspecting any cross-domain result (for example, (2,2,16) tokens, every-4th-block extraction, 32 heads, geometric decay) and evaluate on the same seven DeepfakeBench benchmarks plus a face generator released after training; if the average AUC falls toward RepDFD's 83.39% or the margin disappears on a genuinely post-training generator, the claimed generalization is driven by selection on the test set rather than by bidirectional fusion.

Watch

Extended reading notes

Core claim

The paper's central claim is that CLIP implicitly encodes authenticity cues and that a strategic refinement loop, rather than further feature learning, can turn them into a strong cross-domain face forgery detector. The mechanism is a bidirectional fusion circuit. Three learnable token sets—real, fake, and context—each produce scale and shift parameters that modulate patch tokens collected from blocks 4, 8, 12, 16, 20, and 24 of the frozen ViT; after self-attention and pooling, the aggregated visual features become keys and values in a cross-attention step that updates the text embeddings, which are then concatenated with context tokens and run through CLIP's frozen text encoder. Training orders the four similarities—image against real, fake, context, and a fixed prior prompt—into a hierarchy and applies a progressively weighted cross-entropy loss; inference simply compares the real and fake similarities. On the seven unseen manipulation benchmarks of DeepfakeBench, the method reports 90.07% average AUC versus 83.39% for the strongest prior CLIP-based method, and attention visualizations show real embeddings attending to global facial harmony, fake embeddings to local artifact-prone regions, and context embeddings shifting focus per image.

Load-bearing premise

The headline 90.07% cross-domain score assumes the architectural choices reported by the paper (token counts, selected transformer blocks, attention-head count, and loss weighting) were not selected on the same unseen benchmarks used to report the result; if those choices were tuned against the test sets, the 'unseen manipulation' claim is partly a tuning artifact.

Editorial extensions

If this is right

  • If the reported numbers hold, a frozen CLIP with this plugin surpasses existing CLIP-based deepfake detectors on seven unseen manipulation benchmarks, 90.07% versus 83.39% average AUC, with the largest gains on DFDC and DFDCP.
  • The plugin also generalizes to manipulation families that appeared after CLIP's pretraining, reporting 99.03% on DiffSwap and 92.34% average on WildDeepfake, FFIW, DiffSwap, and DF40 subsets.
  • The ablation ordering indicates the visual-to-text cross-attention step is the largest single contributor: removing it costs 3.90 percentage points of cross-domain AUC, while reducing the method to prompt tuning alone costs 9.88 points.
  • Because CLIP's parameters stay frozen and features are read through forward hooks, the same backbone can host this plugin alongside other CLIP-based applications without retraining, which the paper frames as a deployment advantage.
  • Backbone scale directly affects generalization: switching from ViT-B/16 to ViT-L/14 raises cross-domain AUC by 3.90 points, and original CLIP outperforms OpenCLIP and SigLIP variants at equal size.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable consequence the paper leaves implicit is that the shared context embeddings should carry most of the domain-invariant signal; reusing the same context tokens across different CLIP backbones or across other image-forensics tasks would isolate that claim from the face-specific real and fake prompts.
  • The modular-plugin framing suggests a deployment path in which new forgery families are handled by shipping a new fusion module rather than retraining the vision-language backbone; a natural stress test is to train only on FF++ and swap in a newer VLM at test time to see whether the margin over unidirectional baselines persists.
  • Because the progressive similarity hierarchy is the only training signal that orders the context prompt between real and fake, an experiment holding the hierarchy fixed while randomizing the geometric weights would separate the contribution of the ranking from the contribution of the specific decay schedule.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes HAMLET-FFD, a face forgery detection framework built on CLIP. The method extracts multi-level patch embeddings from frozen CLIP visual transformer blocks, introduces learnable Real/Fake/Context authenticity embeddings, and performs bidirectional cross-modal fusion: text embeddings modulate visual features via scale/shift, and aggregated visual features refine text embeddings through cross-attention. A structured similarity hierarchy loss with progressive cross-entropy is used for training; inference compares similarity with conditioned real and fake prompts. Experiments on DeepfakeBench and extended benchmarks report 90.07% average cross-domain AUC on seven unseen manipulation techniques, surpassing a strong baseline by 6.68pp, with additional results on diffusion and in-the-wild forgeries, ablations, robustness analyses, and attention visualizations.

Significance. If the claims hold, the paper makes a useful contribution: a parameter-preserving plugin that leverages hierarchical CLIP features with bidirectional cross-modal reasoning, achieving strong cross-domain generalization while leaving the backbone untouched. The extensive evaluation across multiple benchmarks, the ablations isolating the visual-to-text pathway, and the careful attention visualizations are valuable. However, the central 'fully frozen backbone' claim is not substantiated by the released code, and the hyperparameter selection procedure on the same cross-domain benchmarks weakens the 'unseen manipulation' interpretation. The absence of error bars or multiple-seed reporting further limits the confidence in the reported margins.

major comments (3)
  1. [Appendix D, Code 2 / §3.1] The code in Appendix D does not implement the advertised fully frozen text encoder. In `_bind_encoder_functions`, the function `__encode_text_embed` executes `clip_model.transformer(x)`, `clip_model.ln_final(x)`, and `x @ clip_model.text_projection` without any `torch.no_grad()` or `requires_grad_(False)` on the text-encoder parameters. The `HAMLET.forward` method calls `__encode_text_embed` to produce `real_prompt_features`, `fake_prompt_features`, and `context_features`, and these are used to compute the Eq. (9) loss, so gradients will flow into CLIP's text transformer during training. This contradicts the abstract and §3.1 claims that 'all pretrained parameters' are frozen and that HAMLET-FFD is an external plugin preserving CLIP's original capabilities. The authors must either supply a training loop that explicitly freezes the text encoder or rerun the experiments with a genuinely frozen text encoder and report the resulting numbers.
  2. [§4.2, Fig. 4, Fig. 7, Tables 10, 11] The key hyperparameters are selected using cross-domain AUC on the same benchmark sets that produce the headline result (90.07%). Specifically, token counts are chosen via the grid search in Figure 4, the block stride via the single-block analysis in Figure 7, the attention head count via Table 10, and the loss weights via Table 11, all evaluated on the target cross-domain datasets. This selection-on-test-set protocol makes the 'unseen manipulation' generalization claim partly a tuning artifact. The authors should use a validation split held out from the reported test benchmarks, or present the full distribution of results across configurations and show that the superiority is not due to selection.
  3. [Tables 1, 2, 3, 4, 5, 6, 7] No error bars, confidence intervals, or multiple-seed results are reported anywhere in the experimental section. Given that several ablation differences are small (e.g., Table 4 shows 0.2–0.35pp differences between embedding arrangements, and Table 6 shows a 0.69pp gain from prior regularization), it is impossible to assess whether these differences and the reported 6.68pp improvement over RepDFD are statistically meaningful. The authors should provide mean and standard deviation over at least three independent training runs for all main tables.
minor comments (5)
  1. [§3.3.1, Eq. (9)] The definition of the progressive cross-entropy loss is unclear: the notation '0 indicates that the first position in each subsequence should have the highest similarity' and the cross-entropy over a subsequence $S[i:]$ need a precise mathematical definition of the target distribution, since standard cross-entropy expects class probabilities.
  2. [Figure 4] The heatmap axes appear transposed: the text says the y-axis is 'Number of Tokens in Real/Fake Embeddings' and the x-axis is 'Number of Tokens in Context Embeddings', but the tick labels show the opposite arrangement. Please check and relabel for readability.
  3. [§4.2.1 and Table 2] The paper alternates between 'seven cross-domain benchmarks' and 'seven unseen manipulation techniques'; clarify that DeepfakeBench includes multiple datasets, each with its own manipulation types, and also specify which subsets of DiffusionFace and DF40 are used in Table 2 (the table lists DiffSwap, UniFace, E4S, BlendFace, MobileSwap, with a reference to [10] for the DiffusionFace subset).
  4. [Figure 1 and Section 4.4.2] The t-SNE visualizations are computed from detection features using $v*fc.weight+fc.bias$, but it is not stated how the linear classifier fc is trained or on which features. Please specify the protocol used to produce these visualizations.
  5. [Table 7] The diagonal values are bolded, but the table caption does not explain the convention consistently with other tables; consider a caption note.

Circularity Check

1 steps flagged · score 6.0 of 10

The headline cross-domain AUC is the maximum of a grid search conducted on the same target benchmarks, so the 'unseen manipulation' evidence is partially circular; no self-citation or definitional circularity is otherwise present.

  1. fitted input called prediction [Section 4.1 (Implementation details); Figure 4; Appendix A.4 (Table 10); Appendix A.6 (Figure 7); Appendix A.5 (Table 11)]
    "The optimal configuration (16, 2) achieves 90.07% cross-domain AUC (right). ... Performance peaks at 32 attention heads for both within-domain (97.73% AUC) and cross-domain (90.07% AUC) evaluations. ... The geometric decay scheme (our default configuration) achieves the best performance with 90.07% cross-domain AUC."

    The reported 90.07% cross-domain AUC is not an independent evaluation of a fixed architecture: it is the best value of the same cross-domain metric used to select token counts (Figure 4), attention-head count (A.4/Table 10), block spacing (A.6/Figure 7), and loss-weight scheme (A.5/Table 11), with all variants evaluated on the same DeepfakeBench target sets. By construction, the headline number is the maximum of the selection criterion rather than an unbiased estimate of generalization to unseen manipulations, so the claim of superior cross-domain generalization is partly a tuning artifact.

full rationale

The only circularity I can substantiate from the paper's own text is methodological: several design hyperparameters—token counts, block spacing, attention-head count, and loss weights—are selected by comparing cross-domain AUC on the exact benchmarks later used to report the headline 90.07%. The reported number is therefore the best value of the selection metric, making the 'unseen manipulation' claim partially circular. This is not a purely mathematical Eq-X-equals-Eq-Y circularity, but it fits the 'fitted input called prediction' pattern because the evaluation metric is used as the fitting criterion for the configuration. There are no load-bearing self-citations, imported uniqueness theorems, or ansatz-by-citation patterns; the bidirectional fusion mechanism retains independent ablative support, so the circularity is partial rather than total. Separately, the Appendix D code does not by itself verify the promised frozen text encoder—the text-transformer forward is not inside torch.no_grad() and no optimizer freeze is shown—which is a correctness/reproducibility caveat rather than a circularity finding.

Assumptions & free parameters 5 free parameters · 4 assumptions · 2 invented entities

The central claim rests on a handful of design choices, each of which is a free parameter selected using the cross-domain benchmarks. The core domain assumption is that CLIP's frozen representations contain transferable authenticity cues, which is the premise for the whole frozen-backbone plugin. No new physical entities are proposed; the only new objects are trained parameter sets whose evidence is internal to the paper's experiments.

free parameters (5)
  • Authenticity token counts (Real=2, Fake=2, Context=16) = 2, 2, 16
    Selected by grid search on the cross-domain benchmarks (Figure 4); the reported 90.07% uses the best cell of this search, so the headline is partly a selected maximum.
  • Number of attention heads in fusion module = 32
    Chosen by sweep on cross-domain AUC (Table 10); differences between 16 and 128 heads are small (89.67-90.07).
  • Vision transformer block selection = blocks 4, 8, 12, 16, 20, 24 (every 4th)
    Chosen from single-block analysis and cross-domain AUC (Table 5, Figure 7); 'All' blocks gives 90.42, so the default is an efficiency/performance compromise.
  • Loss weighting coefficients = geometric decay (0.5)^i
    Chosen by comparing five weighting schemes on cross-domain AUC (Table 11); default beats uniform by 1.13pp.
  • Training schedule = 20 epochs, AdamW lr=1e-3, 2xA100
    Standard choices; no ablation on schedule is reported, and batch size is not stated.
assumptions (4)
  • domain assumption CLIP's frozen representations encode transferable authenticity cues.
    Invoked in the introduction (Figure 1) and Section 3.1; if false, the frozen-backbone plugin cannot work as claimed.
  • domain assumption Training on FF++ (c23) and testing on the seven/eight target benchmarks is a valid measure of unseen manipulation generalization.
    Section 4.2 assumes the target datasets and manipulation types are sufficiently disjoint from FF++ training data; no domain-overlap or provenance analysis is provided.
  • ad hoc to paper The ordered-similarity training objective (sr>sc>sf>spr for real, sf>sc>sr>spr for fake) is a valid inductive bias.
    Section 3.3.1; the ranking is assumed to capture the right relationships among real, fake, and context prompts, and is not independently motivated.
  • domain assumption The standardized preprocessing in DeepfakeBench (including face cropping and resizing to 224x224) is uniform across train and test domains.
    Section 4.1; unknown face-cropping details could introduce cross-domain artifacts that help detection.
invented entities (2)
  • Specialized authenticity embeddings (Real/Fake/Context)
    purpose: Learnable token vectors used as CLIP text prompts to represent face authenticity classes.
    These are model parameters, not physical entities. Their effect is only measured through the paper's own benchmarks; no external falsifiable prediction is associated with them.
  • HAMLET bidirectional fusion module
    purpose: Trainable plugin that modulates frozen CLIP features and refines text embeddings.
    A computational module; its claimed specialization is supported only by qualitative attention maps (Figures 5, 8-10) and internal ablations.

how reviews work

0 comments
Cite this review

Pith. "Pith review of HAMLET-FFD: Hierarchical Adaptive Multi-modal Learning Embeddings Transformation for Face Forgery Detection." pith.science (2026). https://pith.science/paper/IHUKDWTT

@misc{pith2026250720913,
  author       = {Pith},
  title        = {Pith review of: HAMLET-FFD: Hierarchical Adaptive Multi-modal Learning Embeddings Transformation for Face Forgery Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IHUKDWTT}},
  note         = {Machine review of arXiv:2507.20913}
}
read the original abstract

The rapid evolution of face manipulation techniques poses a critical challenge for face forgery detection: cross-domain generalization. Conventional methods, which rely on simple classification objectives, often fail to learn domain-invariant representations. We propose HAMLET-FFD, a cognitively inspired Hierarchical Adaptive Multi-modal Learning framework that tackles this challenge via bidirectional cross-modal reasoning. Building on contrastive vision-language models such as CLIP, HAMLET-FFD introduces a knowledge refinement loop that iteratively assesses authenticity by integrating visual evidence with conceptual cues, emulating expert forensic analysis. A key innovation is a bidirectional fusion mechanism in which textual authenticity embeddings guide the aggregation of hierarchical visual features, while modulated visual features refine text embeddings to generate image-adaptive prompts. This closed-loop process progressively aligns visual observations with semantic priors to enhance authenticity assessment. By design, HAMLET-FFD freezes all pretrained parameters, serving as an external plugin that preserves CLIP's original capabilities. Extensive experiments demonstrate its superior generalization to unseen manipulations across multiple benchmarks, and visual analyses reveal a division of labor among embeddings, with distinct representations specializing in fine-grained artifact recognition.

Figures

Figures reproduced from arXiv: 2507.20913 by the authors.

Figure 1
Figure 1. Domain-invariant features of CLIP facilitate cross [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. HAMLET-FFD architecture. Left: CLIP’s vision transformer (frozen) processes input images and provides multi-level [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Robustness comparison under various perturbations. Each plot shows video-level AUC (%) versus perturbation severity [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Grid Search across Token Counts. Heatmaps show [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Attention visualization across embedding categories on manipulated faces. Real embeddings (top-left) attend to [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: t-SNE visualization of features from CLIPping and [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Performance analysis of individual transformer blocks. The light blue bars and dark blue trend line show within [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]
Figure 8
Figure 8. Figure 8: Attention patterns on FF++ manipulations (FF-DF, FF-F2F, FF-FS, FF-NT). [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]
Figure 9
Figure 9. Figure 9: Attention patterns on DeepfakeBench cross-domain datasets (CDFv1, CDFv2, DFD, DFDC, DFDCP, Fsh, UADFV). [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]
Figure 10
Figure 10. Figure 10: Attention patterns on recent and emerging forgery techniques: in-the-wild forgeries (WDF, FFIW), diffusion-based [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

62 extracted references · 39 canonical work pages

  1. [1]

    [n. d.]. DeepFakes. www.github.com/deepfakes/faceswap. Accessed: 2021-04-24

    work page 2021

  2. [2]

    [n. d.]. DFD. https://ai.googleblog.com/2019/09/contributing-data-to-deepfake- detection.html. Accessed 2021-04-24

    work page 2019

  3. [3]

    [n. d.]. FaceSwap. www.github.com/MarekKowalski/FaceSwap. Accessed 2021- 04-24

    work page 2021

  4. [4]

    Darius Afchar, Vincent Nozick, Junichi Yamagishi, and Isao Echizen. 2018. Mesonet: a compact facial video forgery detection network. In 2018 IEEE in- ternational workshop on information forensics and security (WIFS) . IEEE, 1–7

    work page 2018

  5. [5]

    Shruti Agarwal, Hany Farid, Ohad Fried, and Maneesh Agrawala. 2020. De- tecting deep-fake videos from phoneme-viseme mismatches. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops . 660–661

    work page 2020

  6. [6]

    Lalith Bharadwaj Baru, Rohit Boddeda, Shilhora Akshay Patel, and Sai Mohan Gajapaka. 2025. Wavelet-Driven Generalizable Framework for Deepfake Face Forgery Detection. In Proceedings of the Winter Conference on Applications of Computer Vision (W ACV) Workshops. 1661–1669

    work page 2025

  7. [7]

    Junyi Cao, Chao Ma, Taiping Yao, Shen Chen, Shouhong Ding, and Xiaokang Yang. 2022. End-to-end reconstruction-classification learning for face forgery detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 4113–4122

    work page 2022

  8. [8]

    Liang Chen, Yong Zhang, Yibing Song, Lingqiao Liu, and Jue Wang. 2022. Self- supervised learning of adversarial example: Towards good generalizations for deepfake detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 18710–18719

    work page 2022

Show all 62 references
  1. [9]

    Liang Chen, Yong Zhang, Yibing Song, Jue Wang, and Lingqiao Liu. [n. d.]. OST: Improving Generalization of DeepFake Detection via One-Shot Test-Time Train- ing. ([n. d.])

  2. [10]

    Zhongxi Chen, Ke Sun, Ziyin Zhou, Xianming Lin, Xiaoshuai Sun, Liujuan Cao, and Rongrong Ji. 2024. DiffusionFace: Towards a Comprehensive Dataset for Diffusion-Based Face Forgery Analysis. arXiv preprint arXiv:2403.18471 (2024)

    2024 arXiv

  3. [11]

    Jikang Cheng, Zhiyuan Yan, Ying Zhang, Yuhao Luo, Zhongyuan Wang, and Chen Li. 2024. Can We Leave Deepfake Data Behind in Training Deepfake Detector?. In Advances in Neural Information Processing Systems

    2024

  4. [12]

    Francois Chollet. 2017. Xception: Deep Learning with Depthwise Separable Convolutions. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). doi:10.1109/cvpr.2017.195

    2017 doi

  5. [13]

    Brian Dolhansky, Joanna Bitton, Ben Pflaum, Jikuo Lu, Russ Howes, Menglin Wang, and Cristian Canton Ferrer. 2020. The deepfake detection challenge (dfdc) dataset. arXiv preprint arXiv:2006.07397 (2020)

    2020 arXiv

  6. [14]

    Ricard Durall, Margret Keuper, and Janis Keuper. 2020. Watch your up- convolution: Cnn based generative deep neural networks are failing to reproduce spectral distributions. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 7890–7899

    2020

  7. [15]

    Joel Frank, Thorsten Eisenhofer, Lea Schönherr, Asja Fischer, Dorothea Kolossa, and Thorsten Holz. 2020. Leveraging frequency analysis for deep fake image recognition. In International conference on machine learning . PMLR, 3247–3258

    2020

  8. [16]

    Peng Gao, Shijie Geng, Renrui Zhang, Teli Ma, Rongyao Fang, Yanwen Zhang, Hongsheng Li, and Yu Qiao. 2021. CLIP-Adapter: Better Vision-Language Models with Feature Adapters. Cornell University - arXiv,Cornell University - arXiv (Oct 2021)

    2021

  9. [17]

    Luca Guarnera, Oliver Giudice, Francesco Guarnera, Alessandro Ortis, Giovanni Puglisi, Antonino Paratore, Linh MQ Bui, Marco Fontani, Davide Alessandro Coccomini, Roberto Caldelli, et al. 2022. The face deepfake detection challenge. Journal of Imaging 8, 10 (2022), 263

    2022

  10. [18]

    Alexandros Haliassos, Konstantinos Vougioukas, Stavros Petridis, and Maja Pan- tic. 2021. Lips Don’t Lie: A Generalisable and Robust Approach to Face Forgery Detection. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion (CVPR). doi:10.1109/cvpr46437.2021.00500

    2021

  11. [19]

    Gabriel Ilharco, Mitchell Wortsman, Ross Wightman, Cade Gordon, Nicholas Carlini, Rohan Taori, Achal Dave, Vaishaal Shankar, Hongseok Namkoong, John Miller, Hannaneh Hajishirzi, Ali Farhadi, and Ludwig Schmidt. 2021. OpenCLIP. doi:10.5281/zenodo.5143773 If you use this softwar...

    2021 doi

  12. [20]

    Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. 2021. Scaling up visual and vision- language representation learning with noisy text supervision. In International conference on machine learning . PMLR, 4904–4916

    2021

  13. [21]

    Liming Jiang, Ren Li, Wayne Wu, Chen Qian, and Chen Change Loy. 2020. Deeperforensics-1.0: A large-scale dataset for real-world face forgery detec- tion. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2889–2898

    2020

  14. [22]

    Sohail Ahmed Khan and Duc-Tien Dang-Nguyen. 2024. Clipping the deception: Adapting vision-language models for universal deepfake detection. InProceedings of the 2024 International Conference on Multimedia Retrieval . 1006–1015

    2024

  15. [23]

    Lingzhi Li, Jianmin Bao, Hao Yang, Dong Chen, and Fang Wen. 2020. Advanc- ing high fidelity identity swapping for forgery detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 5074–5083

    2020

  16. [24]

    Lingzhi Li, Jianmin Bao, Ting Zhang, Hao Yang, Dong Chen, Fang Wen, and Bain- ing Guo. 2020. Face x-ray for more general face forgery detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 5001–5010

    2020

  17. [25]

    Y Li. 2018. Exposing deepfake videos by detecting face warping artif acts. arXiv preprint arXiv:1811.00656 (2018)

    2018 arXiv

  18. [26]

    Yuezun Li, Ming-Ching Chang, and Siwei Lyu. 2018. In Ictu Oculi: Exposing AI Created Fake Videos by Detecting Eye Blinking. In 2018 IEEE International Workshop on Information Forensics and Security (WIFS) . doi:10.1109/wifs.2018. 8630787

    2018 doi

  19. [27]

    Yuezun Li, Ming-Ching Chang, and Siwei Lyu. 2018. In ictu oculi: Exposing ai created fake videos by detecting eye blinking. In2018 IEEE International workshop on information forensics and security (WIFS) . Ieee, 1–7

    2018

  20. [28]

    Yuezun Li, Xin Yang, Pu Sun, Honggang Qi, and Siwei Lyu. 2020. Celeb-df: A large-scale challenging dataset for deepfake forensics. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 3207–3216

    2020

  21. [29]

    Kaiqing Lin, Yuzhen Lin, Weixiang Li, Taiping Yao, and Bin Li. 2024. Standing on the Shoulders of Giants: Reprogramming Visual-Language Model for General Deepfake Detection. arXiv preprint arXiv:2409.02664 (2024)

    2024 arXiv

  22. [30]

    Honggu Liu, Xiaodan Li, Wenbo Zhou, Yuefeng Chen, Yuan He, Hui Xue, Weiming Zhang, and Nenghai Yu. 2021. Spatial-phase shallow learning: rethinking face forgery detection in frequency domain. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition ...

    2021

  23. [31]

    Kot Life

    Anwei Luo, Rizhao Cai, Chenqi Kong, Yakun Ju, Xiangui Kang, Jiwu Huang, and Alex C. Kot Life. 2024. Forgery-aware Adaptive Learning with Vision Transformer for Generalized Face Forgery Detection.IEEE Transactions on Circuits and Systems for Video Technology (2024), 1–1. doi:10...

    2024

  24. [32]

    Yuchen Luo, Yong Zhang, Junchi Yan, and Wei Liu. 2021. Generalizing face forgery detection with high-frequency features. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 16317–16326

    2021

  25. [33]

    Huy H Nguyen, Junichi Yamagishi, and Isao Echizen. 2019. Capsule-forensics: Using capsule networks to detect forged images and videos. In ICASSP 2019-2019 IEEE international conference on acoustics, speech and signal processing (ICASSP) . IEEE, 2307–2311

    2019

  26. [34]

    Yunsheng Ni, Depu Meng, Changqian Yu, Chengbin Quan, Dongchun Ren, and Youjian Zhao. 2022. Core: Consistent representation learning for face forgery detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 12–21

    2022

  27. [35]

    Utkarsh Ojha, Yuheng Li, and Yong Jae Lee. 2023. Towards universal fake image detectors that generalize across generative models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 24480–24489

    2023

  28. [36]

    Utkarsh Ojha, Yuheng Li, and Yong Jae Lee. 2023. Towards Universal Fake Image Detectors that Generalize Across Generative Models. In CVPR

    2023

  29. [37]

    Yuyang Qian, Guojun Yin, Lu Sheng, Zixuan Chen, and Jing Shao. 2020. Think- ing in frequency: Face forgery detection by mining frequency-aware clues. In European conference on computer vision . Springer, 86–103

    2020

  30. [38]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learni...

    2021

  31. [39]

    Andreas Rossler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, and Matthias Nießner. 2019. Faceforensics++: Learning to detect manipulated facial images. In Proceedings of the IEEE/CVF international conference on computer vision. 1–11

    2019

  32. [40]

    Kaede Shiohara and Toshihiko Yamasaki. 2022. Detecting deepfakes with self- blended images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 18720–18729

    2022

  33. [41]

    Ke Sun, Shen Chen, Taiping Yao, Haozhe Yang, Xiaoshuai Sun, Shouhong Ding, and Rongrong Ji. 2023. Towards general visual-linguistic face forgery detection. arXiv preprint arXiv:2307.16545 (2023)

    2023 arXiv

  34. [42]

    Chuangchuang Tan, Renshuai Tao, Huan Liu, Guanghua Gu, Baoyuan Wu, Yao Zhao, and Yunchao Wei. 2024. C2P-CLIP: Injecting Category Common Prompt in CLIP to Enhance Generalization in Deepfake Detection. arXiv preprint arXiv:2408.09647 (2024)

    2024 arXiv

  35. [43]

    Mingxing Tan and Quoc Le. 2019. Efficientnet: Rethinking model scaling for convolutional neural networks. In International conference on machine learning . PMLR, 6105–6114

    2019

  36. [44]

    Justus Thies, Michael Zollhöfer, and Matthias Nießner. 2019. Deferred neural rendering: Image synthesis using neural textures. Acm Transactions on Graphics (TOG) 38, 4 (2019), 1–12

    2019

  37. [45]

    Justus Thies, Michael Zollhofer, Marc Stamminger, Christian Theobalt, and Matthias Nießner. 2016. Face2face: Real-time face capture and reenactment of rgb videos. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2387–2395

    2016

  38. [46]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017)

    2017

  39. [47]

    Chengrui Wang and Weihong Deng. 2021. Representative forgery mining for fake face detection. In Proceedings of the IEEE/CVF conference on computer vision MM ’25, October 27–31, 2025, Dublin, Ireland Jialei Cui, Jianwei Du, Yanzhe Li, Lei Gao, Hui Jiang, and Chenfu Bao and patt...

    2021

  40. [48]

    Mitchell Wortsman, Gabriel Ilharco, Jong Wook Kim, Mike Li, Simon Korn- blith, Rebecca Roelofs, Raphael Gontijo Lopes, Hannaneh Hajishirzi, Ali Farhadi, Hongseok Namkoong, and Ludwig Schmidt. 2022. Robust fine-tuning of zero-shot models. In 2022 IEEE/CVF Conference on Computer...

    2022

  41. [49]

    Zhiyuan Yan, Yuhao Luo, Siwei Lyu, Qingshan Liu, and Baoyuan Wu. 2024. Transcending forgery specificity with latent space augmentation for generalizable deepfake detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    2024

  42. [50]

    Zhiyuan Yan, Taiping Yao, Shen Chen, Yandan Zhao, Xinghe Fu, Junwei Zhu, Donghao Luo, Li Yuan, Chengjie Wang, Shouhong Ding, et al. 2024. DF40: Toward Next-Generation Deepfake Detection. arXiv preprint arXiv:2406.13495 (2024)

    2024 arXiv

  43. [51]

    Zhiyuan Yan, Yong Zhang, Yanbo Fan, and Baoyuan Wu. 2023. Ucf: Uncovering common features for generalizable deepfake detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision . 22412–22423

    2023

  44. [52]

    Zhiyuan Yan, Yong Zhang, Xinhang Yuan, Siwei Lyu, and Baoyuan Wu. 2023. DeepfakeBench: A Comprehensive Benchmark of Deepfake Detection. In Ad- vances in Neural Information Processing Systems , A. Oh, T. Neumann, A. Glober- son, K. Saenko, M. Hardt, and S. Levine (Eds.), Vol. 3...

    2023

  45. [53]

    Xin Yang, Yuezun Li, and Siwei Lyu. 2019. Exposing Deep Fakes Using Inconsis- tent Head Poses. In ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . doi:10.1109/icassp.2019.8683164

    2019

  46. [54]

    Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. 2023. Sig- moid loss for language image pre-training. In Proceedings of the IEEE/CVF inter- national conference on computer vision . 11975–11986

    2023

  47. [55]

    Xu Zhang, Svebor Karaman, and Shih-Fu Chang. 2019. Detecting and simulating artifacts in gan fake images. In 2019 IEEE international workshop on information forensics and security (WIFS) . IEEE, 1–6

    2019

  48. [56]

    Hanqing Zhao, Wenbo Zhou, Dongdong Chen, Tianyi Wei, Weiming Zhang, and Nenghai Yu. 2021. Multi-attentional deepfake detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 2185–2194

    2021

  49. [57]

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. 2022. Conditional prompt learning for vision-language models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 16816–16825

    2022

  50. [58]

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. 2022. Learning to prompt for vision-language models. International Journal of Computer Vision 130, 9 (2022), 2337–2348

    2022

  51. [59]

    Tianfei Zhou, Wenguan Wang, Zhiyuan Liang, and Jianbing Shen. 2021. Face Forensics in the Wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 5778–5788

    2021

  52. [60]

    Emergence of Spe- cialized Detectors

    Bojia Zi, Minghao Chang, Jingjing Chen, Xingjun Ma, and Yu-Gang Jiang. 2020. Wilddeepfake: A challenging real-world dataset for deepfake detection. In Pro- ceedings of the 28th ACM International Conference on Multimedia . 2382–2390. HAMLET-FFD: Hierarchical Adaptive Multi-moda...

    2020

  53. [61]

    c r o s s _ m o d a l _ f u s i o n _ m o d u l e ( self

    ) , m u l t i _ l a y e r _ p a t c h _ e m b ) fake_embeddings = self . c r o s s _ m o d a l _ f u s i o n _ m o d u l e ( self . f a k e_ e m be d _ dr o p ou t ( self . fake_embeds . repeat ( batch_size , 1 ,

  54. [62]

    image_features

    ) , m u l t i _ l a y e r _ p a t c h _ e m b ) return { " image_features " : image_features , " r e a l _ p r o m p t _ f e a t u r e s " : self . _ _ e n c o d e _ t e x t _ e m b e d ( torch . cat ([ real_embeddings , c o n te x t _e m b ed d i ng s ] , dim =1) ) , " f a k ...

    2025

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.