REVIEW 4 major objections 5 minor 26 references
MGFFD-VLM: Multi-Granularity Prompt Learning for Face Forgery Detection with VLM
T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read A compact vision-language model, MGFFD-VLM, detects forged faces and gives human-readable reasons, reporting 90.72% accuracy and 0.9420 F1 on the extended DD-VQA text task.
desk verdict A sensible VLM-for-deepfake design, but the headline 90.72% text accuracy is unverifiable because no held-out split is stated and the baseline numbers conflict across tables. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
Multi-Granularity Prompt Learning and the Attribute-Driven Hybrid LoRA Strategy carry the argument. The Probability Prompt is $P^{\text{probability}}_i = F_{\text{concat}}([\hat y_i \times P_{\text{fake}}, (1-\hat y_i) \times P_{\text{real}}])$, where $\hat y_i$ is the binary fake logit and $P_{\text{fake}}, P_{\text{real}}$ are learnable text embeddings along the lines of CoOp. The Location Prompt is $P^{\text{segment}}_i = t^s_i + [V]_{2M+1}\cdots [V]_{2M+L}$, with $t^s_i$ the segmentation feature from a DeepLabv3 decoder projected into text space. These prompt tokens are prepended to the question and vision embeddings, so the LLM literally sees the model's evidence as text. The hybrid LoRA module computes a 4-way softmax over concatenated image features and quality indicators, selects one specialized LoRA expert, and mixes it with a global LoRA expert via $f' = p^* A^*B^*(f) + (1-p^*) A_g B_g(f) + l(f)$; the $A,B$ matrices are low-rank adaptations of the original linear layer $l(f)$. The training strategy ties these modules together with stage-wise unfreezing and the losses $L_{\text{text}}, L_b, L_s, L_f$, and $L_{\text{tcs}}$.
What would settle it
Re-train MGFFD-VLM on an explicitly documented partition of DD-VQA+ with no image or identity in both the training and test sets, and measure test accuracy; if it falls well below 90.72% and no longer beats the BLIP-TI baseline, the paper's central claim of superior text-based forgery judgment is falsified.
Extended reading notes
Core claim
The paper's central claim is that the VLM's own outputs can be turned into prompts to make the LLM focus on forgery cues. A probability prompt is formed by scaling learnable text embeddings for 'it is fake' and 'it is real' by the binary fake logit, and a location prompt is formed by projecting the segmentation decoder's features into text space and adding learnable embeddings. At the same time, the LLM's linear layers are modified with LoRA experts selected by a mixture of image features and face-quality indicators, so low-quality and high-quality faces are handled by different expert branches. A three-stage training schedule first teaches the vision side to classify and segment, then aligns prompts with answers, then instruction-tunes the whole model with a fine-grained contrastive loss and a soft-label cross-entropy on the real/fake word. The paper reports that this design reaches 90.72% accuracy and 0.9420 F1 on DD-VQA text classification, improves answer-generation metrics such as CIDEr and ROUGE-L, and that fusing its vision features into classical detectors raises their AUC on Celeb-DF, DFDC and WDF.
Load-bearing premise
The load-bearing premise is that the DD-VQA+ evaluation uses genuine held-out test samples; the paper says it follows prior settings but never specifies the split, and if the same samples appear in training and testing, the reported 90.72% accuracy is not a measure of generalization.
Editorial extensions
If this is right
- Text-based deepfake classification reaches 90.72% accuracy and 0.9420 F1, so a VLM can act as an interpretable detector rather than only a black-box classifier.
- Adding the probability and location prompts improves both accuracy and answer-generation metrics, meaning a model's own segmentation and confidence estimates can be recycled as inputs to sharpen its final answer.
- Fusing the VLM's vision features into classical detectors raises their AUC on Celeb-DF, DFDC and WDF for Xception and RECCE, and for SBI on most settings, suggesting the VLM features are complementary to specialized forgery detectors.
- The three-stage schedule with the fine-grained contrastive loss and text calibration loss contributes the largest part of the accuracy gain, showing that auxiliary losses targeted at real/fake semantics matter beyond standard text generation.
Reading between the lines
- Because the probability prompt weights 'it is fake' and 'it is real' by the model's own logit, the prompt itself makes the decision boundary somewhat inspectable; one could read the weighting as the model's confidence and use it to flag uncertain cases.
- The self-supervised blending recipe for forgery regions and types means DD-VQA+ can be expanded to new manipulations without manual annotation; a natural next test is whether retraining with newly blended manipulations improves generalization to unseen forgery methods.
- The paper evaluates text accuracy on DD-VQA+ but does not state the train/test split; if the split is not held out, the reported 90.72% would overstate generalization, so a documented disjoint split is needed before taking the headline number at face value.
- The fine-grained contrastive loss treats blended images as hard positives pulled toward fake images; a testable extension is whether this loss also helps robustness to compressed or low-resolution video, where forgery cues are faint.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes MGFFD-VLM, a vision-language framework for interpretable face forgery detection. It introduces DD-VQA+, an extension of the DD-VQA dataset with self-supervised forgery-region/type labels, face quality attributes, and classification text; multi-granularity prompt learning that converts classification logits and segmentation features into textual prompts; an attribute-driven hybrid LoRA expert-selection module; and a three-stage forgery-aware training strategy with auxiliary losses. The authors report state-of-the-art text-based forgery detection accuracy (90.72% Acc, 0.9420 F1) on DD-VQA and improved AUC for Xception, RECCE, and SBI on Celeb-DF, DFDC, and WDF.
Significance. If the reported results are valid, the paper would make a useful contribution to interpretable deepfake detection by combining coarse classification, pixel-level localization, and natural-language explanations in a single VLM, and by showing that such a VLM can boost conventional detectors. The proposed DD-VQA+ dataset and the multi-granularity prompting idea are potentially of interest to the community. However, the central claims currently rest on an evaluation protocol whose data split is not described, and on an internal inconsistency between tables, so the significance cannot be assessed without substantial clarification and likely re-evaluation.
major comments (4)
- [Training Dataset / Evaluation Dataset] The paper trains on DD-VQA+ ('mainly includes' DD-VQA plus self-generated attributes) and evaluates on DD-VQA, but never specifies a held-out split. Since DD-VQA+ is constructed by extending DD-VQA with labels on the same face images, any overlap between training and test samples would directly inflate the Table 1 accuracy (90.72%) and F1 (0.9420). Please report the exact train/validation/test split (e.g., number of images and QA pairs per split, whether the split is at the video/identity level), and demonstrate that the test set was not used to construct the training labels. Without this, the central claim is not verifiable.
- [Table 1 vs Table 3] Table 1 reports Mobile-VLM fine-tuned on DD-VQA with Acc 0.7699, while Table 3 row 1, described as 'the baseline trained only with the DD-VQA dataset,' reports Acc 0.8745 on the same metric. These two entries cannot both describe the same configuration. This inconsistency undermines the ablation study in Table 3, since every subsequent row builds on that baseline. Please clarify the exact configuration of each row and correct the tables accordingly.
- [Results on Deepfake Detection Models and Figure 7] The protocol for combining MGFFD-VLM with Xception, RECCE, and SBI is described in a single sentence ('we blend the output prediction from the small model (Prediction 1) with the fused result (Prediction 2)'). There is no specification of how the vision features are extracted and fused, how the mixture weight is chosen, whether the base detectors are frozen or fine-tuned, or how frame-level versus video-level results are aggregated. The improvements in Table 2 (e.g., +12.98 AUC for Xception on Celeb-DF) cannot be reproduced or assessed without this information.
- [Attribute-Driven Hybrid LoRA Strategy] The pre-trained image quality model Mq that produces the quality indicators QI is not identified anywhere in the paper (no architecture, checkpoint, or reference). Because the expert-selection mechanism is a core contribution, the method is not reproducible without specifying Mq, its training data, and how its outputs are scaled for the quality linear layer. Please provide these details or remove the dependence on an unspecified external model.
minor comments (5)
- [Tables 1 and 3] The metric name 'BLUE-4' should be 'BLEU-4' (Papineni et al. is BLEU).
- [Equation (7)] The text says 'M, H are hyperparameters', but the equation uses M and L; H is never defined. Please correct to the intended notation.
- [Figure 2] Figure 2 contains unexplained labels and question marks (e.g., 'b, c', '?????') and the loss names in the figure do not match the loss names used in the text. Please harmonize the figure with the manuscript.
- [Results on the DD-VQA dataset] The sentence 'due to the insufficient richness of DD-VQA+, in the deepfake detection task, using DD-VQA+ increases the Acc by 10%' is internally contradictory: if DD-VQA+ were insufficient, adding it would not improve performance. The intended meaning is likely that DD-VQA itself is insufficient; please rephrase.
- [DD-VQA+ Construction] The phrase 'which can generate forgery locations and attributes in a self-supervised' is incomplete; it should read 'in a self-supervised manner.'
Circularity Check
The headline text-accuracy claim is largely the binary classifier re-issued as language: the probability prompt inserts the fitted 'real/fake' logits into the LLM input, and the metric then detects those words in the output.
-
fitted input called prediction
[Methodology, 'Multi-Granularity Prompt Learning' subsection, Eq. (6); Table 1 caption/note]
"We calculate the Probability Prompt as follows: P probability i = Fconcat([ ˆyi × Pfake, (1 − ˆyi) × Preal]). (6) / As our analysis relies on identifying the terms "real" or "fake" within the text, AUC scores are not provided."
The deepfake-detection Acc/F1 in Table 1 is computed by detecting 'real'/'fake' in the generated answer (Table 1 note). The same words are inserted into the LLM input by Eq. (6) as text embeddings scaled by the trained classifier logit ŷ_i. Thus the textual verdict is a reformatting of the classification head's own output: a model can satisfy the metric by echoing the prompt, and the 'text-based forgery judgment' is not an independent prediction. The answer-generation metrics are also affected because the prompt contains the target words, though they retain some independent content about explanation quality.
full rationale
Step 1 is a genuine reduction: the headline text-based forgery judgment is not measured independently of the fitted classifier, because the probability prompt embeds that classifier's logits as 'it is fake'/'it is real' text embeddings and the metric scores the presence of those words in the output. That makes the central Table 1 accuracy partially circular; the model is being credited for re-stating its own classification signal. The paper does contain non-circular components: the cross-dataset enhancements in Table 2 (Celeb-DF/DFDC/WDF) are external benchmarks, the answer-generation metrics measure explanatory text beyond the real/fake word, and the ablations test the modules against each other. There is no load-bearing self-citation chain. Two validity concerns should be noted separately: (i) the training text data 'mainly includes' DD-VQA while evaluation is reported on DD-VQA, with no stated held-out split, so overlap cannot be ruled out; this is a data-protocol gap rather than a demonstrated equation-level circularity. (ii) Table 1 lists Mobile-VLM on DD-VQA at 0.7699 Acc while Table 3's baseline 'trained only with the DD-VQA dataset' shows 0.8745, an inconsistency that should be resolved. Overall the text-classification claim is substantially forced by the prompt construction, giving 6/10.
Assumptions & free parameters
free parameters (5)
- loss weights lambda_1-lambda_4 =
not reported
- quality score thresholds =
e.g., CPBD 0.9, illumination <0.3
- number of LoRA experts =
4 specialized + 1 global
- LoRA rank and alpha =
rank 64, alpha 16
- prompt lengths M and L =
not reported
assumptions (5)
- domain assumption Face quality indicators (CPBD-based visibility, CNN-based illumination) are correlated with forgery cues and are safe to use as training targets and expert-selection features.
- domain assumption Blended images with random region replacement are representative surrogate forgeries, and their ground-truth masks are usable for segmentation supervision.
- ad hoc to paper GPT-4-generated synonym sentences are valid and diverse ground-truth text for face quality evaluation.
- ad hoc to paper The image quality model Mq used for LoRA expert selection is available and produces the stated quality indicators.
- domain assumption The evaluation protocol of Zhang et al. includes a clean, non-overlapping train/test split for DD-VQA, and extending the training set with DD-VQA+ does not contaminate the test set.
invented entities (2)
-
DD-VQA+ dataset
-
Attribute-Driven Hybrid LoRA module
Cite this review
Pith. "Pith review of MGFFD-VLM: Multi-Granularity Prompt Learning for Face Forgery Detection with VLM." pith.science (2026). https://pith.science/paper/67C6XAD5
@misc{pith2026250712232,
author = {Pith},
title = {Pith review of: MGFFD-VLM: Multi-Granularity Prompt Learning for Face Forgery Detection with VLM},
year = {2026},
howpublished = {\url{https://pith.science/paper/67C6XAD5}},
note = {Machine review of arXiv:2507.12232}
}
read the original abstract
Recent studies have utilized visual large language models (VLMs) to answer not only "Is this face a forgery?" but also "Why is the face a forgery?" These studies introduced forgery-related attributes, such as forgery location and type, to construct deepfake VQA datasets and train VLMs, achieving high accuracy while providing human-understandable explanatory text descriptions. However, these methods still have limitations. For example, they do not fully leverage face quality-related attributes, which are often abnormal in forged faces, and they lack effective training strategies for forgery-aware VLMs. In this paper, we extend the VQA dataset to create DD-VQA+, which features a richer set of attributes and a more diverse range of samples. Furthermore, we introduce a novel forgery detection framework, MGFFD-VLM, which integrates an Attribute-Driven Hybrid LoRA Strategy to enhance the capabilities of Visual Large Language Models (VLMs). Additionally, our framework incorporates Multi-Granularity Prompt Learning and a Forgery-Aware Training Strategy. By transforming classification and forgery segmentation results into prompts, our method not only improves forgery classification but also enhances interpretability. To further boost detection performance, we design multiple forgery-related auxiliary losses. Experimental results demonstrate that our approach surpasses existing methods in both text-based forgery judgment and analysis, achieving superior accuracy.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
, " * write output.state after.block = add.period write newline
ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...
-
[2]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
-
[3]
Anderson, P.; Fernando, B.; Johnson, M.; and Gould, S. 2016. Spice: Semantic propositional image caption evaluation. In Computer Vision--ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part V 14, 382--398. Springer
2016
-
[4]
Banerjee, S.; and Lavie, A. 2005. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, 65--72
2005
-
[5]
Cao, J.; Ma, C.; Yao, T.; Chen, S.; Ding, S.; and Yang, X. 2022. End-to-end reconstruction-classification learning for face forgery detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 4113--4122
work page 2022
-
[6]
Chollet, F. 2017. Xception: Deep learning with depthwise separable convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, 1251--1258
work page 2017
-
[7]
Chu, X.; Qiao, L.; Lin, X.; Xu, S.; Yang, Y.; Hu, Y.; Wei, F.; Zhang, X.; Zhang, B.; Wei, X.; et al. 2023. Mobilevlm: A fast, reproducible and strong vision language assistant for mobile devices. arXiv preprint arXiv:2312.16886
arXiv 2023
-
[8]
Dolhansky, B.; Bitton, J.; Pflaum, B.; Lu, J.; Howes, R.; Wang, M.; and Ferrer, C. C. 2020. The deepfake detection challenge (dfdc) dataset. arXiv preprint arXiv:2006.07397
arXiv 2020
Show all 26 references
-
[9]
Hua, Y.; Shi, R.; Wang, P.; and Ge, S. 2023. Learning patch-channel correspondence for interpretable face forgery detection. IEEE Transactions on Image Processing, 32: 1668--1680
2023
-
[10]
Li, J.; Li, D.; Xiong, C.; and Hoi, S. 2022. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In International conference on machine learning, 12888--12900. PMLR
2022
-
[11]
Li, Y.; Yang, X.; Sun, P.; Qi, H.; and Lyu, S. 2020. Celeb-df: A large-scale challenging dataset for deepfake forensics. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 3207--3216
2020
-
[12]
Lin, C.-Y. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, 74--81
2004
-
[13]
D.; and Karam, L
Narvekar, N. D.; and Karam, L. J. 2011. A no-reference image blur metric based on the cumulative probability of blur detection (CPBD). IEEE Transactions on Image Processing, 20(9): 2678--2683
2011
-
[14]
Ou, F.-Z.; Li, C.; Wang, S.; and Kwong, S. 2024. CLIB-FIQA: Face Image Quality Assessment with Confidence Calibration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 1694--1704
2024
-
[15]
Papineni, K.; Roukos, S.; Ward, T.; and Zhu, W.-J. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, 311--318
2002
-
[16]
Rossler, A.; Cozzolino, D.; Verdoliva, L.; Riess, C.; Thies, J.; and Nie ner, M. 2019. Faceforensics++: Learning to detect manipulated facial images. In Proceedings of the IEEE/CVF international conference on computer vision, 1--11
2019
-
[17]
Shiohara, K.; and Yamasaki, T. 2022. Detecting deepfakes with self-blended images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 18720--18729
2022
-
[18]
Sun, K.; Chen, S.; Yao, T.; Yang, H.; Sun, X.; Ding, S.; and Ji, R. 2023. Towards general visual-linguistic face forgery detection. arXiv preprint arXiv:2307.16545
2023 arXiv
-
[19]
Vedantam, R.; Lawrence Zitnick, C.; and Parikh, D. 2015. Cider: Consensus-based image description evaluation. In Proceedings of the IEEE conference on computer vision and pattern recognition, 4566--4575
2015
-
[20]
Wu, X.; Huang, S.; and Wei, F. 2023. Mole: Mixture of lora experts. In The Twelfth International Conference on Learning Representations
2023
-
[21]
Yang, S.; Luo, P.; Loy, C.-C.; and Tang, X. 2016. Wider face: A face detection benchmark. In Proceedings of the IEEE conference on computer vision and pattern recognition, 5525--5533
2016
-
[22]
Yu, B.; Li, W.; Li, X.; Zhou, J.; and Lu, J. 2024. Uncertainty-aware hierarchical labeling for face forgery detection. Pattern Recognition, 153: 110526
2024
-
[23]
C.; S ahin, Y
Yurtkulu, S. C.; S ahin, Y. H.; and Unal, G. 2019. Semantic segmentation with extended DeepLabv3 architecture. In 2019 27th Signal Processing and Communications Applications Conference (SIU), 1--4. IEEE
2019
-
[24]
Zhang, Y.; Colman, B.; Shahriyari, A.; and Bharaj, G. 2024. Common Sense Reasoning for Deep Fake Detection. arXiv preprint arXiv:2402.00126
2024 arXiv
-
[25]
C.; and Liu, Z
Zhou, K.; Yang, J.; Loy, C. C.; and Liu, Z. 2022. Learning to prompt for vision-language models. International Journal of Computer Vision, 130(9): 2337--2348
2022
-
[26]
Zi, B.; Chang, M.; Chen, J.; Ma, X.; and Jiang, Y.-G. 2020. Wilddeepfake: A challenging real-world dataset for deepfake detection. In Proceedings of the 28th ACM international conference on multimedia, 2382--2390
2020
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.