Recognition: unknown
AI-Generated Images: What Humans and Machines See When They Look at the Same Image
Pith reviewed 2026-05-08 13:56 UTC · model grok-4.3
The pith
Human surveys of 100 people identify which of 16 XAI methods best match human judgments on why images are flagged as AI-generated.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Training detectors on AIText2Image and integrating sixteen XAI methods, then refining the resulting visual explanations through human survey responses, yields measurable alignment between machine outputs and human preferences; this process surfaces visual-language cues that both detectors and people rely on when identifying AI-generated images.
What carries the argument
The human-preference alignment evaluation that collects textual and visual responses from 100 participants to score how well each of the sixteen XAI outputs matches human understanding of AI-generated image cues.
If this is right
- Detectors can be tuned to favor XAI methods that humans find clearest.
- Identified visual-language cues can guide improvements in future detection architectures.
- The alignment metric offers a concrete way to compare XAI techniques for any image-classification task.
- Systems using the best-aligned explanations become more transparent for non-expert users.
Where Pith is reading between the lines
- The same survey-based alignment procedure could be adapted to evaluate explanations in other AI domains such as text or audio generation.
- Automating parts of the preference collection might allow continuous updating of which XAI methods are preferred as detectors improve.
- The work implies that purely technical performance metrics are insufficient; human clarity must be treated as a primary design goal.
Load-bearing premise
That the judgments of 100 survey participants give a reliable and generalizable picture of how humans understand and prefer explanations for AI image detections.
What would settle it
A larger or demographically broader study that ranks the same sixteen XAI methods in a substantially different order of human clarity would falsify the alignment results.
Figures
read the original abstract
The misuse of generative AI in online disinformation campaigns highlights the urgent need for transparent and explainable detection systems. In this work, we investigate how detectors for AI-generated images can be more effective in providing human-understandable explanations for their predictions. To this end, we develop a suite of detectors with various architectures and fine-tuning strategies, trained on our large-scale photorealistic fake image dataset, AIText2Image, and assess their performance on state-of-the-art text-to-image AI generators. We integrate 16 different explainable AI (XAI) methods into our detection framework, and the visual explanations are comprehensively refined and evaluated through a novel approach that prioritizes human understanding of AI-generated images, using both textual and visual responses collected from a survey of 100 participants. This framework offers insights into visual-language cues in fake image detection and into the clarity of XAI methods from a human perspective, measuring the alignment of XAI outputs with human preferences.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper develops a suite of AI-generated image detectors trained on the AIText2Image dataset, integrates 16 XAI methods into the detection pipeline, and evaluates the resulting visual explanations via textual and visual responses collected from a survey of 100 participants. The central claim is that this human-centered evaluation framework measures the alignment between XAI outputs and human preferences, yielding insights into visual-language cues for fake-image detection and the clarity of different XAI techniques.
Significance. A robust human-evaluation component for XAI in image forensics could help close the gap between automated detectors and human-interpretable explanations, which is relevant for combating disinformation. The multi-architecture detector suite and the explicit focus on human alignment are positive design choices. However, because the manuscript supplies neither quantitative detector performance numbers nor any tabulated survey results or statistical measures, it is not yet possible to judge whether the claimed insights are actually delivered.
major comments (2)
- [Abstract] Abstract: the manuscript states that responses from a survey of 100 participants are used to 'refine and evaluate' the 16 XAI outputs and to 'measure the alignment' with human preferences, yet supplies no information on participant demographics, recruitment, task design, statistical power, or inter-rater agreement. Because the central claim that the framework 'prioritizes human understanding' rests on these responses being a reliable proxy, the omission is load-bearing.
- [Abstract] Abstract: despite asserting that detectors are assessed on state-of-the-art text-to-image generators and that XAI explanations are 'comprehensively refined and evaluated,' the text contains no quantitative performance metrics, confusion matrices, alignment scores, or error analysis. Without these data it is impossible to determine whether the empirical results support the stated claims.
minor comments (1)
- [Abstract] The abstract introduces the acronym XAI without an initial expansion; a single parenthetical definition on first use would improve readability.
Simulated Author's Rebuttal
We thank the referee for the constructive feedback highlighting the need for greater transparency in our survey methodology and empirical results. We agree that these elements are critical to substantiate the central claims and will incorporate the requested details and metrics in the revised manuscript.
read point-by-point responses
-
Referee: [Abstract] Abstract: the manuscript states that responses from a survey of 100 participants are used to 'refine and evaluate' the 16 XAI outputs and to 'measure the alignment' with human preferences, yet supplies no information on participant demographics, recruitment, task design, statistical power, or inter-rater agreement. Because the central claim that the framework 'prioritizes human understanding' rests on these responses being a reliable proxy, the omission is load-bearing.
Authors: We acknowledge that the current manuscript provides only a high-level description of the survey in the abstract and methods without the requested specifics. In the revision, we will expand the survey methodology section to include participant demographics (age, gender, education, and AI familiarity distributions), recruitment details (via online platforms and academic networks with inclusion criteria), task design (rating scales for explanation clarity, relevance to generative artifacts, and perceived alignment, with example interfaces), a priori statistical power analysis (targeting 80% power), and inter-rater reliability measures (Fleiss' kappa and Cronbach's alpha). These additions will directly support the reliability of the human evaluation as a proxy for preferences. revision: yes
-
Referee: [Abstract] Abstract: despite asserting that detectors are assessed on state-of-the-art text-to-image generators and that XAI explanations are 'comprehensively refined and evaluated,' the text contains no quantitative performance metrics, confusion matrices, alignment scores, or error analysis. Without these data it is impossible to determine whether the empirical results support the stated claims.
Authors: We agree that the lack of quantitative metrics limits the ability to assess the empirical support for the claims. The manuscript prioritizes the human-centered framework but does not present specific detector performance numbers, matrices, or tabulated alignment scores. In the revision, we will add a dedicated results section with detector performance metrics (accuracy, precision, recall, F1 across architectures on the AIText2Image test set and external generators), confusion matrices, quantitative alignment scores (mean survey ratings per XAI method with standard deviations and significance tests), and an error analysis of misalignment cases. This will enable evaluation of the insights on visual-language cues and XAI clarity. revision: yes
Circularity Check
No circularity: empirical pipeline relies on external survey data and standard training
full rationale
The paper trains detectors on the authors' AIText2Image dataset, applies 16 off-the-shelf XAI methods, collects fresh textual/visual responses from 100 participants, and computes alignment scores between XAI outputs and those responses. No equations, predictions, or central claims reduce by construction to fitted parameters, self-definitions, or prior self-citations. The survey constitutes independent external input rather than a renamed or fitted output of the model itself. The framework is therefore self-contained against external benchmarks.
Axiom & Free-Parameter Ledger
Reference graph
Works this paper leans on
-
[1]
https://firefly.adobe.com (2024), accessed: 2025-01-16
Adobe: Adobe firefly text to image. https://firefly.adobe.com (2024), accessed: 2025-01-16
2024
-
[2]
IEEE Open Journal of Signal Processing (2023)
Bammey, Q.: Synthbuster: Towards detection of diffusion model generated images. IEEE Open Journal of Signal Processing (2023)
2023
-
[3]
IEEE Access (2024)
Bird, J.J., Lotfi, A.: Cifake: Image classification and explainable identification of ai-generated synthetic images. IEEE Access (2024)
2024
-
[4]
Image and Vision Computing137, 104771 (2023) AI-Generated Images: Human versus Machines 11
Borji, A.: Qualitative failures of image generation models and their application in detecting deepfakes. Image and Vision Computing137, 104771 (2023) AI-Generated Images: Human versus Machines 11
2023
-
[5]
In: 2018 IEEE winter conference on applications of computer vision (WACV)
Chattopadhay, A., Sarkar, A., Howlader, P., Balasubramanian, V.N.: Grad- cam++: Generalized gradient-based visual explanations for deep convolutional networks. In: 2018 IEEE winter conference on applications of computer vision (WACV). pp. 839–847. IEEE (2018)
2018
-
[6]
ImageNet: A large- scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Ima- genet: A large-scale hierarchical image database. In: 2009 IEEE Confer- ence on Computer Vision and Pattern Recognition. pp. 248–255 (2009). https://doi.org/10.1109/CVPR.2009.5206848
-
[7]
In: Proceedings of the 2nd ACM International Workshop on Multimedia AI against Disinformation
Dogoulis, P., Kordopatis-Zilos, G., Kompatsiaris, I., Papadopoulos, S.: Improving synthetically generated image detection in cross-concept settings. In: Proceedings of the 2nd ACM International Workshop on Multimedia AI against Disinformation. pp. 28–35 (2023)
2023
-
[8]
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., Houlsby, N.: An image is worth 16x16 words: Transformers for image recognition at scale (2021), https://arxiv.org/abs/2010.11929
work page internal anchor Pith review arXiv 2021
- [9]
-
[10]
University of Montreal1341(3), 1 (2009)
Erhan, D., Bengio, Y., Courville, A., Vincent, P.: Visualizing higher-layer features of a deep network. University of Montreal1341(3), 1 (2009)
2009
-
[11]
http://www.pascal-network.org/challenges/VOC/voc2012/workshop/index.html (2015)
Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The PASCAL Visual Object Classes Challenge 2012 (VOC2012) Results. http://www.pascal-network.org/challenges/VOC/voc2012/workshop/index.html (2015)
2012
- [12]
-
[13]
In: Arai, K
Guarnera, L., Giudice, O., Battiato, S.: Level Up the Deepfake Detection: A Method to Effectively Discriminate Images Generated by GAN Architectures and Diffusion Models. In: Arai, K. (ed.) Intelligent Systems and Applications, vol. 1067, pp.615–625.SpringerNatureSwitzerland,Cham(2024),seriesTitle:LectureNotes in Networks and Systems
2024
- [14]
-
[15]
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition (2015), https://arxiv.org/abs/1512.03385
work page internal anchor Pith review arXiv 2015
- [16]
- [17]
-
[18]
URL http: //dx.doi.org/10.1007/s11263-020-01316-z
Kuznetsova, A., Rom, H., Alldrin, N., Uijlings, J., Krasin, I., Pont-Tuset, J., Kamali, S., Popov, S., Malloci, M., Kolesnikov, A., Duerig, T., Ferrari, V.: The open images dataset v4: Unified image classification, object detection, and visual relationship detection at scale. International Journal of Computer Vi- sion128(7), 1956–1981 (Mar 2020). https://...
-
[19]
https://github.com/black-forest-labs/flux (2024) 12 S
Labs, B.F.: Flux. https://github.com/black-forest-labs/flux (2024) 12 S. Poletti et al
2024
- [20]
-
[21]
In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T
Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft coco: Common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) Computer Vision – ECCV 2014. pp. 740–755. Springer International Publishing, Cham (2014)
2014
-
[22]
Neural Networks177, 106392 (2024)
Liu, G., Zhang, J., Chan, A.B., Hsiao, J.H.: Human attention guided explain- able artificial intelligence for computer vision models. Neural Networks177, 106392 (2024). https://doi.org/https://doi.org/10.1016/j.neunet.2024.106392, https://www.sciencedirect.com/science/article/pii/S0893608024003162
-
[23]
Information Fusion 106, 102301 (2024)
Longo, L., Brcic, M., Cabitza, F., Choi, J., Confalonieri, R., Ser, J.D., Guidotti, R., Hayashi, Y., Herrera, F., Holzinger, A., Jiang, R., Khosravi, H., Lecue, F., Malgieri, G., Páez, A., Samek, W., Schneider, J., Speith, T., Stumpf, S.: Explainable artificial intelligence (xai) 2.0: A manifesto of open challenges and interdisciplinary research direction...
-
[24]
In: Proceedings of the IEEE/CVF International Conference on Computer Vision
Lorenz, P., Durall, R.L., Keuper, J.: Detecting images generated by deep diffusion models using their local intrinsic dimensionality. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 448–459 (2023)
2023
- [25]
-
[26]
In: Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R
Lundberg, S.M., Lee, S.I.: A unified approach to interpreting model predictions. In: Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. (eds.) Advances in Neural Information Processing Systems. vol. 30. Curran Associates, Inc. (2017)
2017
- [27]
-
[28]
https://docs.midjourney.com/docs/model-version-5 (2024), accessed: 2025-01-16
Midjourney: Midjourney 5.2. https://docs.midjourney.com/docs/model-version-5 (2024), accessed: 2025-01-16
2024
-
[29]
https://docs.midjourney.com/docs/models (2024), accessed: 2025-01-16
Midjourney: Midjourney 6.1. https://docs.midjourney.com/docs/models (2024), accessed: 2025-01-16
2024
-
[30]
In: 2020 international joint conference on neural networks (IJCNN)
Muhammad, M.B., Yeasin, M.: Eigen-cam: Class activation map using principal components. In: 2020 international joint conference on neural networks (IJCNN). pp. 1–7. IEEE (2020)
2020
-
[31]
Nichol, A., Dhariwal, P., Ramesh, A., Shyam, P., Mishkin, P., McGrew, B., Sutskever, I., Chen, M.: Glide: Towards photorealistic image generation and editing with text-guided diffusion models (2022), https://arxiv.org/abs/2112.10741
work page internal anchor Pith review arXiv 2022
-
[32]
Nightingale, S.J., Wade, K.A., Watson, D.G.: Can people identify original and manipulated photos of real-world scenes? Cognitive research: principles and impli- cations2, 1–21 (2017)
2017
-
[33]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Ojha, U., Li, Y., Lee, Y.J.: Towards Universal Fake Image Detectors That Gener- alize Across Generative Models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 24480–24489 (2023)
2023
-
[34]
https://openai.com/index/dall-e-2 (2022), accessed: 2025-01-16
OpenAI: Dall-e 2: Generating images from text. https://openai.com/index/dall-e-2 (2022), accessed: 2025-01-16
2022
-
[35]
https://openai.com/index/dall-e-3 (2023), accessed: 2025-01-16 AI-Generated Images: Human versus Machines 13
OpenAI: Dall-e 3: Generating images from text. https://openai.com/index/dall-e-3 (2023), accessed: 2025-01-16 AI-Generated Images: Human versus Machines 13
2023
-
[36]
SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis
Podell, D., English, Z., Lacey, K., Blattmann, A., Dockhorn, T., Müller, J., Penna, J., Rombach, R.: Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952 (2023), https://arxiv.org/abs/2307.01952
work page internal anchor Pith review arXiv 2023
-
[37]
In: proceedings of the IEEE/CVF winter conference on applications of computer vision
Ramaswamy,H.G.,etal.:Ablation-cam:Visualexplanationsfordeepconvolutional network via gradient-free localization. In: proceedings of the IEEE/CVF winter conference on applications of computer vision. pp. 983–991 (2020)
2020
-
[38]
why should i trust you?
Ribeiro, M.T., Singh, S., Guestrin, C.: " why should i trust you?" explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD interna- tional conference on knowledge discovery and data mining. pp. 1135–1144 (2016)
2016
-
[39]
In- ternational journal of computer vision128, 336–359 (2020)
Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad- cam: visual explanations from deep networks via gradient-based localization. In- ternational journal of computer vision128, 336–359 (2020)
2020
-
[40]
In: Proceed- ings of the 2023 ACM SIGSAC Conference on Computer and Communica- tions Security
Sha, Z., Li, Z., Yu, N., Zhang, Y.: DE-FAKE: Detection and Attribution of Fake Images Generated by Text-to-Image Generation Models. In: Proceed- ings of the 2023 ACM SIGSAC Conference on Computer and Communica- tions Security. pp. 3418–3432. CCS ’23, Association for Computing Machin- ery, New York, NY, USA (Nov 2023). https://doi.org/10.1145/3576915.36165...
- [41]
-
[42]
Sundararajan, M., Taly, A., Yan, Q.: Axiomatic attribution for deep networks (2017), https://arxiv.org/abs/1703.01365
work page Pith review arXiv 2017
-
[43]
In: Proceedings of the IEEE/CVF conference on computer vision and pat- tern recognition workshops
Wang, H., Wang, Z., Du, M., Yang, F., Zhang, Z., Ding, S., Mardziel, P., Hu, X.: Score-cam: Score-weighted visual explanations for convolutional neural net- works. In: Proceedings of the IEEE/CVF conference on computer vision and pat- tern recognition workshops. pp. 24–25 (2020)
2020
-
[44]
In: Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision
Wang, Z., Bao, J., Zhou, W., Wang, W., Hu, H., Chen, H., Li, H.: DIRE for Diffusion-Generated Image Detection. In: Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision. pp. 22445–22455 (2023)
2023
-
[45]
Machine Learning111(11), 4295–4327 (Nov 2022)
Wolter, M., Blanke, F., Heese, R., Garcke, J.: Wavelet-packets for deepfake image analysis and detection. Machine Learning111(11), 4295–4327 (Nov 2022). https://doi.org/10.1007/s10994-022-06225-5, https://link.springer.com/10.1007/s10994-022-06225-5
- [46]
-
[47]
In: European conference on computer vision/arXiv
Zeiler, M.: Visualizing and understanding convolutional networks. In: European conference on computer vision/arXiv. vol. 1311 (2014)
2014
- [48]
-
[49]
Advances in Neural Information Processing Systems36(2024)
Zhu, M., Chen, H., Yan, Q., Huang, X., Lin, G., Li, W., Tu, Z., Hu, H., Hu, J., Wang, Y.: Genimage: A million-scale benchmark for detecting ai-generated image. Advances in Neural Information Processing Systems36(2024)
2024
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.