Pith. sign in

REVIEW 3 major objections 4 minor 82 references

DFBench: Benchmarking Deepfake Image Detection Capability of Large Multimodal Models

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A new 540,000-image benchmark tests deepfake detectors against 12 modern generators, and an ensemble of vision-language models sets the top score.

desk verdict DFBench is a genuinely useful benchmark resource, but every headline accuracy number is macro-averaged recall mislabeled as accuracy; fix the metric, the duplicate row, and the GenImage omission before trusting any comparison. read the letter →

arxiv 2506.03007 v1 pith:EZGV776Y submitted 2025-06-03 cs.CV

Jiarui Wang , Huiyu Duan , Juntong Wang , Ziheng Jia , Woo Yi Yang , Xiaorong Zhu , Yu Zhao , Jiaying Qian
show 3 more authors
Yuke Xing Guangtao Zhai Xiongkuo Min
This is my paper
classification cs.CV
keywords deepfakedetectionlargemultimodalmodelsAI-generatedimagebenchmarkdatasetmixtureofagentsdiffusionforensicszero-shot
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

DFBench is a new benchmark built to test how well deepfake detectors keep up with current image generators. It contains 540,000 images: 45,000 real photographs (many with natural distortions), 15,000 partially AI-edited images, and 480,000 fully synthetic images produced by 12 state-of-the-art diffusion and autoregressive models from 40,000 prompts. On this benchmark the paper argues that large multimodal models, queried zero-shot, generalize to modern fakes better than conventional detectors trained on older deepfake datasets. It then proposes MoA-DF, which averages the softmax probabilities of three such models, and reports that this ensemble beats both individual LMMs and fine-tuned detectors on nearly every subset. If correct, the result gives the community a current stress test for detectors and evidence that ensembled vision-language models are a viable detection route.

What carries the argument

The dataset construction pipeline: 40K prompts from Flickr8k drive 12 generators (e.g., SD3.5, Flux, PixArt-$\sigma$) to produce 480K fully synthetic images; real images come from seven public quality databases (LIVE, CSIQ, TID2013, KADID-10k, CLIVE, KonIQ-10k, Flickr8k) and AI-edited images from EPAIQA-15K. The MoA-DF method: three LMMs (Qwen2.5-VL, InternVL2.5, InternVL3) each answer a forced-choice prompt, their log-probabilities for "real" versus "AI" are softmax-normalized as $p_A^{(i)} = e^{\log p_A}/(e^{\log p_A}+e^{\log p_B})$, and the final decision is the summed-probability comparison $P_A > P_B$ with $P_A = \sum_i p_A^{(i)}$. The evaluation protocol defines accuracy via Eq. (4), $Acc = \frac{TP}{TP+FN}$ (per-class recall), and this definition carries every headline comparison in the paper.

What would settle it

Recompute every headline table using true accuracy, $\frac{TP+TN}{TP+TN+FP+FN}$, or balanced accuracy (the mean of recall and specificity), and compare model rankings; if MoA-DF no longer leads, or if an always-real classifier that scores near 100% under $Acc = \frac{TP}{TP+FN}$ on real subsets drops to chance under true accuracy, the reported state-of-the-art result is an artifact of the per-class-recall metric.

Watch

Extended reading notes

Core claim

The central claim is that DFBench provides the largest and most diverse public benchmark for deepfake image detection, covering real, AI-edited, and fully AI-generated content, and that MoA-DF, a mixture of agents that sums the softmax probabilities of Qwen2.5-VL, InternVL2.5, and InternVL3, achieves state-of-the-art detection accuracy on it. The paper further claims that the benchmark's bidirectional protocol lets detection accuracy double as a measure of a generator's evasion capability, and that by this measure current diffusion models such as SD3.5-Large produce images that fool detectors far more than earlier models. The authors also report that zero-shot LMMs generalize better than conventional detectors to modern generators, though fine-tuned LMMs (including MoA-DF) are needed for top scores.

Load-bearing premise

The entire benchmark comparison, including the claim that MoA-DF is state of the art, rests on treating per-class recall ($Acc = \frac{TP}{TP+FN}$) as "accuracy" for each dataset and macro-averaging it across classes for the Overall rows.

Editorial extensions

If this is right

  • Conventional detectors trained on earlier deepfake datasets (CnnSpott, Gram-Net, UnivFD) score near zero zero-shot against the DFBench AI-generated subsets, so methods built on old generators will not transfer to current ones.
  • Zero-shot LMMs, in particular larger ones such as InternVL2.5 (78B), detect modern fakes far better than specialized detectors without any task-specific training.
  • Fine-tuned LMMs need only one LoRA epoch to outperform the best conventional detector fine-tuned for 50 epochs, indicating that LMMs are substantially more sample- and compute-efficient for deepfake detection.
  • MoA-DF's summed-probability ensemble beats each individual LMM and every baseline on the real, AI-edited, and AI-generated subsets, supporting the paper's claim that ensembling LMMs is an effective detection strategy.
  • On the generation side, the benchmark's per-generator detection accuracy ranks SD3.5-Large as the most evasion-capable generator and LaVi-Bridge as the least.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the Overall columns are macro-averages of per-class recall, the tables should be read as recall-per-class, not as conventional accuracy; a detector biased toward "real" appears excellent on real subsets while scoring near zero on fakes.
  • DFBench's design holds prompts constant across all 12 generators, so an extension of the paper's analysis could measure whether a detector's failures come from generator-specific artifacts or from content the prompt set exposes.
  • The MoA-DF combination rule is a fixed sum of softmax probabilities; a learned weighting over the same three LMMs, trained on a held-out split of DFBench, could show whether the ensemble gain comes from the aggregation rule itself or from the particular mix of models.
  • A direct evasion index, such as one minus the average detection recall across all detectors for each generator, would make the paper's implicit generator-ranking comparison from Figure 5 an explicit, reproducible metric.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces DFBench, a large-scale benchmark for deepfake image detection, containing 540,000 images spanning real images from eight sources (including distorted real images), 15,000 AI-edited images, and 480,000 AI-generated images from 12 state-of-the-art generation models. It proposes MoA-DF, a mixture-of-agents method that combines softmax-normalized log-probabilities from three large multimodal models (LMMs) for real/fake classification. The paper reports benchmark results for conventional detectors and LMMs on real, AI-edited, and AI-generated subsets, and uses detection accuracy as an indirect measure of generator evasion capability. The central claims are that DFBench is the largest and most diverse such benchmark and that MoA-DF achieves state-of-the-art detection performance.

Significance. If the benchmark and its evaluation are sound, DFBench would be a useful community resource: it provides a broad set of current generative models, includes partially edited images and naturally distorted real images, and offers a bidirectional evaluation protocol. The public release of the database and code is a concrete strength. The proposed MoA-DF ensemble is a plausible approach to leveraging LMMs for deepfake detection, and the fine-tuned results suggest strong performance. However, the paper's quantitative claims are currently undermined by a mislabeled evaluation metric and by inconsistencies between the text and the reported tables, so the significance cannot be fully assessed without revision.

major comments (3)
  1. [Section 5.1, Eq. (4)] The metric defined as Acc = TP/(TP+FN) is recall, not accuracy. This is not a naming quibble: the Overall columns in Tables 2-4 are macro-averages of per-class recall values, not the proportion of correctly classified images. For example, in Table 4, the Llama3.2-Vision Overall of 78.11% equals the mean of its 13 per-class recall values (sum = 1015.48 over 13 classes). Every headline 'Accuracy' number, the MoA-DF state-of-the-art claim, the detector rankings, and the generator evasion ranking in Figure 5 are therefore expressed in a quantity that does not match the text. The authors must relabel the metric as macro-averaged class recall (or recompute true accuracy) and re-examine all conclusions, including the abstract's SOTA claim, under the corrected metric.
  2. [Section 5.4, Table 4] The text states that 'InternVL2.5 (78B) achieves the best results' among zero-shot LMMs on AI-generated subsets, but Table 4 shows Llama3.2-Vision (11B) with an Overall of 78.11%, higher than InternVL2.5 (78B) at 68.89%. This contradiction is material to the paper's narrative about model scaling and must be resolved, either by correcting the text or by clarifying the subset/condition under which InternVL2.5 (78B) is best.
  3. [Table 4, InternVL3 (78B) row] The InternVL3 (78B) row appears to be partially duplicated from the InternVL3 (9B) row: columns for Playground (41.28), PixArt-Sigma (48.66), Infinity (22.63), Kandinsky-3 (30.24), Flux_Schnell (13.69), Kolors (26.88), and the SD3.5_Large values (6.235 vs. 6.325) are essentially identical, while later columns differ. This suggests a copy-and-paste or execution error. The 78B evaluation must be rerun and the row corrected, or the duplication must be explicitly justified.
minor comments (4)
  1. [Section 4 and Section 5.1] The MoA-DF method is presented as a zero-shot ensemble, but in Tables 3 and 4 it is listed in the fine-tuned block (marked with ♦). Section 5.1 states that three LMMs are fine-tuned with LoRA, yet it is not explicitly stated that MoA-DF uses these fine-tuned versions. Please clarify the exact configuration of MoA-DF's components.
  2. [Throughout] There are several typos and inconsistencies: 'Natuaral' in Figure 1, 'TID2023' in Figure 1 versus TID2013 in the text, 'NOV A' in Table 4, 'Avalibility' in Table 1, and 'Intern2.5' in Figure 1. These should be corrected.
  3. [Section 5.2, Table 2] The text says that 'most models exhibit strong zero-shot identification capabilities on real image datasets,' but Llama3.2-Vision (11B) achieves only 56.27% on the real subset, which is near chance. The statement should be qualified to reflect the wide performance range shown in Table 2.
  4. [Section 3.2] The feature distribution analysis in Figure 3 is qualitative; no quantitative measure (e.g., separability statistics) is provided to support the claim that the distributions establish a 'comprehensive benchmark.' Adding a simple quantitative summary would strengthen the analysis.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the benchmark and MoA-DF results are measured on held-out splits with no fitted parameter renamed as prediction.

full rationale

I walked the derivation chain and found no step where a claimed prediction reduces to an input by construction, no fitted parameter renamed as a prediction, and no load-bearing self-citation chain. The central empirical claims are (i) DFBench is large and diverse and (ii) MoA-DF achieves state-of-the-art performance on it. MoA-DF is an ensemble of softmax probabilities from three LMMs (Eqs. 1-3) with no trainable parameters of its own; the fine-tuned variants (MoA-DF, InternVL2.5*, InternVL3*, Qwen2.5-VL*) are LoRA-adapted on a 4:1 train/test split of the benchmark and then evaluated on the held-out test portion. That is a standard held-out evaluation, not a fitted-input-called-prediction loop. The AI-edited images are taken from EPAIQA-15K [49], a same-group dataset, but this is a data-source choice rather than a derivation step, and it does not by itself force any detection result. The intro's citations to same-group papers [9, 59-64, 72] support background claims about AI-generated image quality, not the benchmark's evaluation outcomes. One substantive concern is not circularity but metric validity: Eq. (4) defines Acc = TP/(TP+FN), which is recall, not accuracy, and the printed Overall columns in Tables 2-4 reproduce macro-averaged per-class recall. This mislabels the headline quantity and weakens the 'state-of-the-art' claim as stated, but it does not make the result circular; the numbers still come from independent held-out test inference. If the metric is relabeled or recomputed as true accuracy, the comparisons would need to be re-derived, but that is a correctness issue outside the circularity rubric. Therefore the circularity score is 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

No fitted parameters feed the central claim; the hand-chosen design elements (prompt template, N=3, LoRA settings) affect results but are not fitted to data. The central claims rest on the evaluation protocol (Eq. 4 mislabels recall as accuracy), on unverified generation quality of the 480K fake images, and on the representativeness of the chosen real sources. No new physical or mathematical entities are postulated. 'Mixture of Agents' is a name for an equal-weight probability sum over three existing models, not a new entity with independent evidence.

free parameters (3)
  • Zero-shot prompt template for LMMs
    The exact wording 'Is this a real image... Just answer with A/B. A: real, B: AI' (Figure 4) is chosen by hand with no prompt ablation; LMM accuracy is known to be prompt-sensitive.
  • MoA-DF ensemble size and weights = N=3, equal weights
    Eqs. (2)-(3) sum softmax probabilities of exactly three models with equal weight; no ablation of N, weightings, or alternative fusion strategies.
  • LoRA fine-tuning hyperparameters = r=8, lr=1e-5, 1 epoch
    Section 5.1 reports these choices without sensitivity analysis, and only one training run is described for each fine-tuned model.
assumptions (4)
  • domain assumption All 480K generated images are valid, complete, correctly labeled fakes; generation failures and filtering are negligible.
    Section 3.1 states 12 models x 40K prompts = 480K images with no QC, no failure-rate report, and no discussion of degenerate outputs; detectors could exploit artifacts and inflate per-model evasion scores.
  • ad hoc to paper Per-class recall averaged across classes (the paper's Eq. 4 'Accuracy') adequately represents detection accuracy for benchmarking.
    Section 5.1 defines Acc = TP/(TP+FN), which is recall, and the tables report means of per-class values; benchmark conclusions are phrased in terms of this quantity.
  • domain assumption The 4:1 train/test split and the public and API-based inference protocols have no leakage or non-determinism that materially changes rankings.
    Section 5.1 describes the split but not file-level composition, seeds, API sampling parameters, or test-set sizes; closed-source models (Gemini, Grok) may be updated between runs.
  • ad hoc to paper Distortions in LIVE, CSIQ, TID2013, KADID-10k, CLIVE, and KonIQ-10k represent real-world in-the-wild authentic image distortions.
    Section 3.1 frames these as real-world degradations, but these IQA databases contain synthetic distortions applied to pristine images, not natively captured camera artifacts.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DFBench: Benchmarking Deepfake Image Detection Capability of Large Multimodal Models." pith.science (2026). https://pith.science/paper/EZGV776Y

@misc{pith2026250603007,
  author       = {Pith},
  title        = {Pith review of: DFBench: Benchmarking Deepfake Image Detection Capability of Large Multimodal Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EZGV776Y}},
  note         = {Machine review of arXiv:2506.03007}
}
read the original abstract

With the rapid advancement of generative models, the realism of AI-generated images has significantly improved, posing critical challenges for verifying digital content authenticity. Current deepfake detection methods often depend on datasets with limited generation models and content diversity that fail to keep pace with the evolving complexity and increasing realism of the AI-generated content. Large multimodal models (LMMs), widely adopted in various vision tasks, have demonstrated strong zero-shot capabilities, yet their potential in deepfake detection remains largely unexplored. To bridge this gap, we present \textbf{DFBench}, a large-scale DeepFake Benchmark featuring (i) broad diversity, including 540,000 images across real, AI-edited, and AI-generated content, (ii) latest model, the fake images are generated by 12 state-of-the-art generation models, and (iii) bidirectional benchmarking and evaluating for both the detection accuracy of deepfake detectors and the evasion capability of generative models. Based on DFBench, we propose \textbf{MoA-DF}, Mixture of Agents for DeepFake detection, leveraging a combined probability strategy from multiple LMMs. MoA-DF achieves state-of-the-art performance, further proving the effectiveness of leveraging LMMs for deepfake detection. Database and codes are publicly available at https://github.com/IntMeGroup/DFBench.

Figures

Figures reproduced from arXiv: 2506.03007 by the authors.

Figure 1
Figure 1. We present the DFBench, a large dataset for benchmarking deepfake image detection capabilities. (a) 45K real and 15K [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Visualization of images on the DFBench dataset. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Feature distribution of the DFBench. (a) Feature distribution of real images with no distortion. (b) Feature distribution [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Overview of the MoA-DF architecture. Three LMMs [PITH_FULL_IMAGE:figures/full_fig_p003_4.png]
Figure 5
Figure 5. Figure 5: (a) Performance comparison of image generation models (b) Performance comparison of image detection models [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

82 extracted references · 49 canonical work pages

  1. [1]

    Martin Arjovsky, Soumith Chintala, and Léon Bottou. 2017. Wasserstein GAN. In Proceedings of the 32nd International Conference on Machine Learning (ICML) . 214–223

    2017

  2. [2]

    Vladimir Arkhipkin, Viacheslav Vasilev, Andrei Filatov, Igor Pavlov, Julia Aga- fonova, Nikolai Gerasimenko, Anna Averchenkova, Evelina Mironova, Anton Bukashkin, Konstantin Kulikov, et al. 2024. Kandinsky 3: Text-to-image synthesis for multifunctional generative framework.arXiv preprint arXiv:2410.21061 (2024)

    work page Pith review arXiv 2024

  3. [3]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. 2025. Qwen2.5-VL Technical Rep...

    arXiv 2025

  4. [4]

    Xiuli Bi, Bo Liu, Fan Yang, Bin Xiao, Weisheng Li, Gao Huang, and Pamela C. Cosman. 2023. Detecting Generated Images by Real Images Only. Arxiv (2023)

    2023

  5. [5]

    Jordan J Bird and Ahmad Lotfi. 2023. CIFAKE: Image Classification and Ex- plainable Identification of AI-Generated Synthetic Images. arXiv preprint arXiv:2303.14126 (2023)

    arXiv 2023

  6. [6]

    Andrew Brock, Jeff Donahue, and Karen Simonyan. 2018. Large scale GAN training for high fidelity natural image synthesis. arXiv preprint arXiv:1809.11096 (2018)

    arXiv 2018

  7. [7]

    You-Ming Chang, Chen Yeh, Wei-Chen Chiu, and Ning Yu. 2023. AntifakePrompt: Prompt-Tuned Vision-Language Models are Fake Image Detectors. Arxiv (2023)

    work page 2023

  8. [8]

    Junsong Chen, Chongjian Ge, Enze Xie, Yue Wu, Lewei Yao, Xiaozhe Ren, Zhong- dao Wang, Ping Luo, Huchuan Lu, and Zhenguo Li. 2024. Pixart- 𝜎: Weak-to- strong training of diffusion transformer for 4k text-to-image generation. In Proceedings of the European Conference on Computer Vision (ECCV) . 74–91

    work page 2024

Show all 82 references
  1. [9]

    Zijian Chen, Wei Sun, Yuan Tian, Jun Jia, Zicheng Zhang, Jiarui Wang, Ru Huang, Xiongkuo Min, Guangtao Zhai, and Wenjun Zhang. 2024. GAIA: Rethinking Ac- tion Quality Assessment for AI-Generated Videos. In Proceedings of the Advances in Neural Information Processing Systems (N...

    2024

  2. [10]

    Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al . 2024. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.05271 (2024)

    2024 arXiv

  3. [11]

    Guillaume Couairon, Jakob Verbeek, Holger Schwenk, and Matthieu Cord. 2022. Diffedit: Diffusion-based semantic image editing with mask guidance. arXiv preprint arXiv:2210.11427 (2022)

    2022 arXiv

  4. [12]

    Hao Dang, Feng Liu, Joel Stehouwer, Xiaoming Liu, and Anil K Jain. 2020. On the detection of digital face manipulation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern recognition (CVPR) . 5781–5790

    2020

  5. [13]

    Haoge Deng, Ting Pan, Haiwen Diao, Zhengxiong Luo, Yufeng Cui, Huchuan Lu, Shiguang Shan, Yonggang Qi, and Xinlong Wang. 2024. Autoregressive Video Generation without Vector Quantization. arXiv preprint arXiv:2412.14169 (2024)

    2024 arXiv

  6. [14]

    Prafulla Dhariwal and Alexander Nichol. 2021. Diffusion models beat gans on image synthesis. In Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), Vol. 34. 8780–8794

    2021

  7. [15]

    Huiyu Duan, Qiang Hu, Jiarui Wang, Liu Yang, Zitong Xu, Lu Liu, Xiongkuo Min, Chunlei Cai, Tianxiao Ye, Xiaoyun Zhang, et al. 2025. FineVQ: Fine-Grained User Generated Content Video Quality Assessment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Re...

    2025

  8. [16]

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. 2024. Scaling rectified flow transformers for high-resolution image synthesis. In Pro- ceedings of the International Confere...

    2024

  9. [17]

    Joel Frank, Thorsten Eisenhofer, Lea Schönherr, Asja Fischer, Dorothea Kolossa, and Thorsten Holz. 2020. Leveraging frequency analysis for deep fake image recognition. In International Conference on Machine Learning (ICML) . PMLR, 3247–3258

    2020

  10. [18]

    Joel Frank and Thorsten Holz. 2021. CNN-generated images are surprisingly easy to spot...for now. Arxiv (2021)

    2021

  11. [19]

    Apurva Gandhi and Shomik Jain. 2020. Adversarial perturbations fool deepfake detectors. In International Joint Conference on Neural Networks (IJCNN) . IEEE, 1–8

    2020

  12. [20]

    Deepti Ghadiyaram and Alan C Bovik. 2015. Massive online crowdsourced study of subjective and objective picture quality. IEEE Transactions on Image Processing (TIP) 25 (2015)

    2015

  13. [21]

    Shuyang Gu, Dong Chen, Jianmin Bao, Fang Wen, Bo Zhang, Dongdong Chen, Lu Yuan, and Baining Guo. 2022. Vector quantized diffusion model for text-to-image synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 10696–10706

    2022

  14. [22]

    Jian Han, Jinlai Liu, Yi Jiang, Bin Yan, Yuqi Zhang, Zehuan Yuan, Bingyue Peng, and Xiaobing Liu. 2024. Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesis. arXiv preprint arXiv:2412.04431 (2024)

    2024 arXiv

  15. [23]

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising Diffusion Probabilistic Models. In Proceedings of the Advances in Neural Information Processing Systems (NeurIPS). 6840–6851

    2020

  16. [24]

    Micah Hodosh, Peter Young, and Julia Hockenmaier. 2013. Framing image de- scription as a ranking task: Data, models and evaluation metrics. Journal of Artificial Intelligence Research (JAIR) 47 (2013), 853–899

    2013

  17. [25]

    Wenyi Hong, Weihan Wang, Qingsong Lv, Jiazheng Xu, Wenmeng Yu, Junhui Ji, Yan Wang, Zihan Wang, Yuxuan Zhang, Juanzi Li, Bin Xu, Yuxiao Dong, Ming Ding, and Jie Tang. 2024. CogAgent: A Visual Language Model for GUI Agents. arXiv preprint arXiv:2312.08914 (2024)

    2024 arXiv

  18. [26]

    Vlad Hosu, Hanhe Lin, Tamas Sziranyi, and Dietmar Saupe. 2020. KonIQ-10k: An ecologically valid database for deep learning of blind image quality assessment. IEEE Transactions on Image Processing (TIP) (2020), 4041–4056

    2020

  19. [27]

    Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. 2022. LoRA: Low-Rank Adaptation of Large Language Models. In Proceedings of the International Conference on Learning Representations (ICLR), Vol. 1. 3

    2022

  20. [28]

    Zhenglin Huang, Jinwei Hu, Xiangtai Li, Yiwei He, Xingyu Zhao, Bei Peng, Baoyuan Wu, Xiaowei Huang, and Guangliang Cheng. 2025. SIDA: Social Media Image Deepfake Detection, Localization and Explanation with Large Multimodal Model. arXiv preprint arXiv:2412.04292 (2025)

    2025 arXiv

  21. [29]

    Yan Ju, Shan Jia, Lipeng Ke, Hongfei Xue, Koki Nagano, and Siwei Lyu. 2022. Fusing global and local features for generalized ai-synthesized image detection. In IEEE International Conference on Image Processing (ICIP) . IEEE, 3465–3469

    2022

  22. [30]

    Minguk Kang, Jun-Yan Zhu, Richard Zhang, Jaesik Park, Eli Shechtman, Sylvain Paris, and Taesung Park. 2023. Scaling up GANs for Text-to-Image Synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 1179–1189

    2023

  23. [31]

    Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. 2017. Progressive growing of gans for improved quality, stability, and variation. arXiv preprint arXiv:1710.10196 (2017)

    2017 arXiv

  24. [32]

    Tero Karras, Miika Aittala, Samuli Laine, Erik Härkönen, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. 2021. Alias-Free Generative Adversarial Networks. In Proceedings of the Advances in Neural Information Processing Systems (NeurIPS) , Vol. 34. 852–863

    2021

  25. [33]

    Black Forest Labs. 2024. FLUX. https://github.com/black-forest-labs/flux

    2024

  26. [34]

    Eric Cooper Larson and Damon Michael Chandler. 2010. Most apparent distortion: full-reference image quality assessment and the role of strategy. Journal of Electronic Imaging (JEI) 19, 1 (2010)

    2010

  27. [35]

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. 2024. LLaVA- OneVision: Easy Visual Task Transfer. arXiv preprint arXiv:2408.03326 (2024)

    2024 arXiv

  28. [36]

    Daiqing Li, Aleks Kamko, Ehsan Akhgari, Ali Sabet, Linmiao Xu, and Suhail Doshi

  29. [37]

    Feng Li, Renrui Zhang, Hao Zhang, Yuanhan Zhang, Bo Li, Wei Li, Zejun Ma, and Chunyuan Li. 2024. LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Models. arXiv preprint arXiv:2407.07895 (2024)

    2024 arXiv

  30. [38]

    Hanhe Lin, Vlad Hosu, and Dietmar Saupe. 2019. Kadid-10k: A large-scale artificially distorted iqa database. In Proceedings of the International Conference on Quality of Multimedia Experience (QoMEX) . IEEE

    2019

  31. [39]

    Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C

    Tsung-Yi Lin, Michael Maire, Serge J. Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. 2014. Microsoft COCO: Common Objects in Context. In Proceedings of the European Conference on Computer Vision (ECCV)

    2014

  32. [40]

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2024. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 26296–26306

    2024

  33. [41]

    Zhengzhe Liu, Xiaojuan Qi, and Philip HS Torr. 2020. Global texture enhancement for fake face detection in the wild. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR) . 8060–8069

    2020

  34. [42]

    Haoyu Lu, Wen Liu, Bo Zhang, Bingxuan Wang, Kai Dong, Bo Liu, Jingxiang Sun, Tongzheng Ren, Zhuoshu Li, Hao Yang, Yaofeng Sun, Chengqi Deng, Hanwei Xu, Zhenda Xie, and Chong Ruan. 2024. DeepSeek-VL: Towards Real-World Vision-Language Understanding. arXiv preprint arXiv:2403.05...

    2024 arXiv

  35. [43]

    AI Meta. 2024. Llama 3.2: Revolutionizing edge AI and vision with open, cus- tomizable models. Meta AI Blog. Retrieved December 20 (2024), 2024

    2024

  36. [44]

    Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. 2021. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprint arXiv:2112.10741 (2021)

    2021 arXiv

  37. [45]

    Utkarsh Ojha, Yuheng Li, and Yong Jae Lee. 2023. Towards universal fake image detectors that generalize across generative models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 24480–24489

    2023

  38. [46]

    Bryan A Plummer, Liwei Wang, Chris M Cervantes, Juan C Caicedo, Julia Hock- enmaier, and Svetlana Lazebnik. 2015. Flickr30k entities: Collecting region-to- phrase correspondences for richer image-to-sentence models. In Proceedings of the IEEE International Conference on Comput...

    2015

  39. [47]

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. 2023. Sdxl: Improving latent diffusion Conference acronym ’25, October 27–31, 2025, Dublin, Ireland Jiarui Wang, et al. models for high-resolution image synthe...

    2023 arXiv

  40. [48]

    Nikolay Ponomarenko, Lina Jin, Oleg Ieremeiev, Vladimir Lukin, Karen Egiazar- ian, Jaakko Astola, Benoit Vozel, Kacem Chehdi, Marco Carli, Federica Battisti, et al. 2015. Image database TID2013: Peculiarities, results and perspectives.Signal Processing: Image Communication (SP...

    2015

  41. [49]

    Jiaying Qian, Ziheng Jia, Zicheng Zhang, Zeyu Zhang, Guangtao Zhai, and Xiongkuo Min. 2025. Towards Explainable Partial-AIGC Image Quality Assess- ment. arXiv preprint arXiv:2504.09291 (2025)

    2025 arXiv

  42. [50]

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022. High-Resolution Image Synthesis With Latent Diffusion Models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 10684–10695

    2022

  43. [51]

    Zeyang Sha, Zheng Li, Ning Yu, and Yang Zhang. 2022. DE-FAKE: Detection and Attribution of Fake Images Generated by Text-to-Image Diffusion Models. arXiv preprint arXiv:2210.06998 (2022)

    2022 arXiv

  44. [52]

    Hamid R Sheikh, Muhammad F Sabir, and Alan C Bovik. 2006. A statistical evaluation of recent full reference image quality assessment algorithms. IEEE Transactions on Image Processing (TIP) (2006), 3440–3451

    2006

  45. [53]

    Weiss, Niru Maheswaranathan, and Surya Ganguli

    Jascha Sohl-Dickstein, Eric A. Weiss, Niru Maheswaranathan, and Surya Ganguli

  46. [54]

    Haixu Song, Shiyu Huang, Yinpeng Dong, and Wei-Wei Tu. 2023. Robustness and Generalizability of Deepfake Detection: A Study with Diffusion Models. Arxiv (2023)

    2023

  47. [55]

    Chuangchuang Tan, Yao Zhao, Shikui Wei, Guanghua Gu, and Yunchao Wei. 2023. Learning on Gradients: Generalized Artifacts Representation for GAN-Generated Images Detection. In CVPR

    2023

  48. [56]

    Google Team. 2024. Gemini1.5-pro. https://gemini.google.com/. Accessed: 2025-03-08

    2024

  49. [57]

    Kolors Team. 2024. Kolors: Effective Training of Diffusion Model for Photorealistic Text-to-Image Synthesis. arXiv preprint (2024)

    2024

  50. [58]

    Luisa Verdoliva, Davide Cozzolino, and Koki Nagano. 2022. 2022 IEEE Image and Video Processing Cup Synthetic Image Detection

    2022

  51. [59]

    Jiarui Wang, Huiyu Duan, Ziheng Jia, Yu Zhao, Woo Yi Yang, Zicheng Zhang, Zijian Chen, Juntong Wang, Yuke Xing, Guangtao Zhai, and Xiongkuo Min. 2025. LOVE: Benchmarking and Evaluating Text-to-Video Generation and Video-to- Text Interpretation. arXiv preprint arXiv:2505.12098 (2025)

    2025 arXiv

  52. [60]

    Jiarui Wang, Huiyu Duan, Jing Liu, Shi Chen, Xiongkuo Min, and Guangtao Zhai. 2023. Aigciqa2023: A large-scale image quality assessment database for ai generated images: from the perspectives of quality, authenticity and correspon- dence. In Proceedings of the CAAI Internation...

    2023

  53. [61]

    Jiarui Wang, Huiyu Duan, Guangtao Zhai, and Xiongkuo Min. 2025. Quality Assessment for AI Generated Images with Instruction Tuning. arXiv preprint arXiv:2405.07346 (2025)

    2025 arXiv

  54. [62]

    Jiarui Wang, Huiyu Duan, Guangtao Zhai, Juntong Wang, and Xiongkuo Min

  55. [63]

    Jiarui Wang, Huiyu Duan, Yu Zhao, Juntong Wang, Guangtao Zhai, and Xiongkuo Min. 2025. LMM4LMM: Benchmarking and Evaluating Large-multimodal Image Generation with LMMs. arXiv preprint arXiv:2504.08358 (2025)

    2025 arXiv

  56. [64]

    Juntong Wang, Jiarui Wang, Huiyu Duan, Guangtao Zhai, and Xiongkuo Min

  57. [65]

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. 2024. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191 (2024)

    2024 arXiv

  58. [66]

    Run Wang, Felix Juefei-Xu, Lei Ma, Xiaofei Xie, Yihao Huang, Jian Wang, and Yang Liu. 2019. Fakespotter: A simple yet robust baseline for spotting ai-synthesized fake faces. arXiv preprint arXiv:1909.06122 (2019)

    2019 arXiv

  59. [67]

    Weiyun Wang, Zhe Chen, Wenhai Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Jinguo Zhu, Xizhou Zhu, Lewei Lu, Yu Qiao, and Jifeng Dai. 2024. Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization. arXiv preprint arXiv:2411.10442 (2024)

    2024 arXiv

  60. [68]

    TDVE-Assessor: Benchmarking and Evaluating the Quality of Text-Driven Video Editing with LMMs

  61. [69]

    Yabin Wang, Zhiwu Huang, and Xiaopeng Hong. 2023. Benchmarking Deepart Detection. arXiv preprint arXiv:2302.14475 (2023)

    2023 arXiv

  62. [70]

    Chengyue Wu, Xiaokang Chen, Zhiyu Wu, Yiyang Ma, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, Chong Ruan, et al. 2024. Janus: Decoupling visual encoding for unified multimodal understanding and generation. arXiv preprint arXiv:2410.13848 (2024)

    2024 arXiv

  63. [71]

    xAI Team. 2024. Grok2 Vision. https://grok.com/. Accessed: 2025-03-08

    2024

  64. [72]

    Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jin- sheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, et al. 2024. Emu3: Next-token prediction is all you need. arXiv preprint arXiv:2409.18869 (2024)

    2024 arXiv

  65. [73]

    Woo Yi Yang, Jiarui Wang, Sijing Wu, Huiyu Duan, Yuxin Zhu, Liu Yang, Kang Fu, Guangtao Zhai, and Xiongkuo Min. 2025. LMME3DHF: Benchmarking and Evaluating Multimodal 3D Human Face Generation with LMMs. arXiv preprint arXiv:2504.20466 (2025)

    2025 arXiv

  66. [74]

    Xin Yang, Yuezun Li, and Siwei Lyu. 2019. Exposing deep fakes using inconsistent head poses. InICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 8261–8265

    2019

  67. [75]

    Jiabo Ye, Haiyang Xu, Haowei Liu, Anwen Hu, Ming Yan, Qi Qian, Ji Zhang, Fei Huang, and Jingren Zhou. 2024. mplug-owl3: Towards long image-sequence understanding in multi-modal large language models. In Proceedings of the Inter- national Conference on Learning Representations (ICLR)

    2024

  68. [76]

    Zitong Xu, Huiyu Duan, Guangji Ma, Liu Yang, Jiarui Wang, Qingbo Wu, Xiongkuo Min, Guangtao Zhai, and Patrick Le Callet. 2025. HarmonyIQA: Pio- neering Benchmark and Model for Image Harmonization Quality Assessment. arXiv preprint arXiv:2501.01116 (2025)

    2025 arXiv

  69. [77]

    Shihao Zhao, Shaozhe Hao, Bojia Zi, Huaizhe Xu, and Kwan-Yee K Wong. 2024. Bridging different language models and generative vision models for text-to- image generation. In Proceedings of the European Conference on Computer Vision (ECCV). 70–86

    2024

  70. [78]

    Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. 2017. Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks. In Proceedings of the IEEE International Conference on Computer Vision (ICCV) . 2223– 2232

    2017

  71. [80]

    Pan Zhang, Xiaoyi Dong, Bin Wang, Yuhang Cao, Chao Xu, Linke Ouyang, Zhiyuan Zhao, Shuangrui Ding, Songyang Zhang, Haodong Duan, Wenwei Zhang, Hang Yan, Xinyue Zhang, Wei Li, Jingwen Li, Kai Chen, Conghui He, Xingcheng Zhang, Yu Qiao, Dahua Lin, and Jiaqi Wang. 2023. InternLM-...

    2023 arXiv

  72. [2015]

    In Proceedings of the 32nd International Conference on Machine Learning (ICML)

    Deep Unsupervised Learning using Nonequilibrium Thermodynamics. In Proceedings of the 32nd International Conference on Machine Learning (ICML) . 2256–2265

  73. [2024]

    arXiv preprint arXiv:2402.17245 (2024)

    Playground v2.5: Three Insights towards Enhancing Aesthetic Quality in Text-to-Image Generation. arXiv preprint arXiv:2402.17245 (2024)

    2024 arXiv

  74. [2025]

    In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    AIGV-Assessor: Benchmarking and Evaluating the Perceptual Quality of Text-to-Video Generation with LMM. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.