Pith. sign in

REVIEW 3 major objections 1 cited by

XPlainVerse: A Million-Scale Benchmark for Explainable Deepfake Detection

T0 review · 3 major / 0 minor · reviewed 2026-07-12 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read XPlainVerse makes grounded explanation quality a measurable part of deepfake detection at million-image scale.

desk verdict Solid million-scale explainable-deepfake benchmark with real engineering and a clear OOD diagnosis; the VLM-on-VLM scoring loop is the main soft spot, not a reason to ignore the work. read the letter →

arxiv 2607.03562 v1 pith:AYZ4YC25 submitted 2026-07-03 cs.CV

Abhijeet Narang , Kartik Kuckreja , Shreya Ghosh , Muhammad Haris Khan , Jianfei Cai , Abhinav Dhall This is my paper · ORCID
classification cs.CV
keywords deepfakedetectionexplainableAIvision-languagemodelsimagemanipulationbenchmarkdatasetgroundedexplanationsEntityScoreEvidence
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Deepfake detectors increasingly output natural-language reasons, but those reasons often read as fluent stories rather than evidence tied to what was actually changed in the image. Existing benchmarks mostly score real-versus-fake accuracy and leave that gap unmeasured, so progress on trustworthy, user-facing systems is hard to track. XPlainVerse is a one-million-image benchmark built to close it: real photos from five public sources are paired with forgeries from twelve editing and synthesis models, then filtered by Edit-Check so retained fakes really show the intended edit. The release supplies both technical and simplified explanations, plus EntityScore and EvidenceScore that test whether an explanation names the right manipulated parts and the right visual cues. Human checks on two thousand pairs support the claim that this supervision is usable at scale, and experiments show high in-domain scores collapse under generator shift—pointing to shortcut learning rather than stable visual reasoning.

What carries the argument

Edit-Check: a multi-stage filter that extracts visible differences between real and generated images, then judges whether those differences match the intended edit and discards off-target rewrites; EntityScore and EvidenceScore then score whether predicted explanations recover the same manipulated entities and visual evidence claims as the reference.

What would settle it

Train and score the same models using only the 2,000 human gold explanations as targets and as the sole Entity/Evidence reference, then check whether the reported ID-to-OOD collapse and the ranking of methods reverse or disappear relative to the silver-label results.

Watch

Extended reading notes

Core claim

The paper’s central claim is that explainable deepfake detection becomes a well-posed research problem only when manipulations are edit-verified, explanations are dual-level (expert and lay), and evaluation measures entity and evidence fidelity rather than surface text similarity. Under that regime, fine-tuned vision–language models look strong in distribution but fail under held-out generators, while EntityScore and EvidenceScore drop more than BERTScore—exposing fluent but ungrounded reasoning.

Load-bearing premise

The load-bearing premise is that large-scale VLM-written silver explanations, and VLM judges used for filtering and Entity/Evidence scoring, stay faithful enough to human-visible manipulation evidence to train and rank models.

Editorial extensions

If this is right

  • Grounded explanation quality becomes a reported metric alongside detection F1, not an optional narrative.
  • Generator-shift evaluation is required; strong in-domain fine-tuning no longer counts as success if Entity/Evidence collapse.
  • Dual-level (technical vs simple) explanations become standard so systems can serve experts and non-experts without one style for all.
  • Edit-aware filtering is treated as necessary for scalable reasoning supervision, not optional data cleaning.
  • Detectors are pushed toward content-level cues (geometry, lighting, placement) as low-level generator artifacts fade.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If silver labels and VLM judges share the same blind spots, reported gains on Entity/Evidence may partly measure agreement with that judge rather than human-grounded forensics.
  • The same edit-instruction-plus-difference protocol could transfer to video and audio-visual deepfakes once temporal consistency checks replace still-image difference JSON.
  • Deployment systems may need to expose both explanation styles and let users pick, because preference splits by role are large enough that a single default leaves many users underserved.
  • Spectral convergence across newer generators implies that pure embedding probes will keep failing; methods that localize entities before explaining may be the practical next step.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 0 minor

Summary. The paper introduces XPlainVerse, a ~1M-image benchmark for joint deepfake detection and natural-language explanation. Real images from five public sources are paired with manipulations from twelve editing/synthesis models; a multi-stage Edit-Check pipeline (VLM difference extraction + text-only alignment judge) retains only edits that match intended instructions and discards off-target rewrites (~56.5% discard). The release provides dual-level explanations (complex forensic and simple lay explanations) plus authenticity explanations for reals, totaling ~1.53M image–explanation pairs, with human gold explanations on 2,000 fakes. The authors propose EntityScore and EvidenceScore—bidirectional VLM-judged coverage of diagnostic entities and visual evidence claims—alongside BERTScore and SLE, and report that fine-tuned VLMs reach high ID detection/explanation scores but drop sharply under generator-held-out OOD, while Entity/Evidence degrade more than surface similarity. Linear probes and spectral analyses suggest reliance on generator-specific low-level artifacts rather than stable visual evidence.

Significance. If the resource and metrics hold up, this is a substantial contribution to explainable deepfake detection: scale far exceeds prior explanation-oriented sets (Table 1), Edit-Check addresses a real source of weakly grounded supervision, dual-level explanations operationalize human-centered XAI needs, and Entity/Evidence scores target a genuine evaluation gap beyond fluency. The ID–OOD collapse and spectral/probe analyses are useful negative results for the community. Strengths include concrete inventories (Tables 8–10), kept-vs-discarded fidelity (Table 2), multi-stage human preference/quality studies (Figures 4–5, Tables 4 and 18), and a detailed appendix of prompts and protocols. The main significance risk is that explanation targets and automatic fidelity metrics are themselves largely VLM-produced, so the claim that grounded quality is now reliably measurable depends on how well those silver labels and judges track human visual evidence.

major comments (3)
  1. §3.1 Step 5 and §4 / Appendix I: Complex silver explanations are generated with privileged auxiliaries (paired real image, edit instruction, and difference JSON), while EntityScore/EvidenceScore extraction and bidirectional coverage are judged by Qwen3.5-4B. Human gold covers only 2,000 of ~530K fakes; inter-annotator EntityScore/EvidenceScore on a 10-image overlap are 0.5 and 0.44 (§3.2). Preference wins vs SIDA/human (Table 4) do not calibrate whether auto metrics recover the same diagnostic entities and evidence claims humans would mark. Without a reported correlation (or human re-annotation of metric components) on a larger held-out set, the central claim that Entity/Evidence make grounded quality measurable—and that their sharper OOD drop vs BERTScore proves loss of visual grounding rather than shift in VLM-style rationales—is only weakly supported. Please add metric–human agreement
  2. Table 9 and §5 / Table 5: Training fakes are extremely imbalanced—Gemini-2.0-flash accounts for 354,256 of 360,000 train fakes (~98.4%), with other generators almost entirely deferred to val/test. The reported >50-point F1 OOD drops for fine-tuned models are therefore largely “train-on-Gemini, test-on-other-generators.” That is still a valid robustness finding, but it confounds “generator-specific artifact memorization” with simple single-source overfitting. Please report per-generator ID/OOD breakdowns, retrain or ablate with more balanced multi-generator training subsets, and clarify how much of the collapse remains under multi-source training before attributing failure primarily to low-level spectral shortcuts (Figure 6).
  3. §3.1 Edit-Check and Appendix E: Filtering and KEEP/DISCARD decisions are fully VLM-driven (difference extractors + DeepSeek/GPT-OSS judges; KEEP if final_score ≥ 4). Table 2 and human edit-fidelity (Figure 4) support that retained pairs are cleaner, but there is no independent non-VLM audit of false KEEP/false DISCARD rates stratified by edit type (Table 11) or generator. Because explanation supervision is only as grounded as retained edit compliance, a quantified error analysis of the judge (e.g., human audit of a few thousand KEEP/DISCARD decisions, agreement with BORDERLINE handling) is load-bearing for the claim that Edit-Check enables reliable reasoning supervision at scale.

Circularity Check

1 steps flagged · score 1.0 of 10

No derivation-by-construction circularity; only a mild self-referential measurement loop (VLM silver labels scored by a VLM judge) that does not force the main empirical claims.

  1. other [§3.1 Step 5; §4; Appendix I (Entity/Evidence extraction & coverage)]
    "For manipulated images, we generate two types of explanations. A complex explanation... one model randomly sampled from a pool of four (Gemini-3-Flash, GPT-5-mini, GLM-4.6V, and Qwen3.5-27B) receives the fake image alongside three auxiliary references: the paired real image, the edit-instruction, and the visual-difference... Given a predicted explanation and a reference explanation, we first use Qwen3.5-4B to extract... diagnostic entities and... evidence claims... compute coverage in two directional passes using the same Qwen3.5-4B evaluator."

    Silver complex references are VLM-written with privileged edit/difference context, and EntityScore/EvidenceScore are then judged by another VLM (Qwen3.5-4B) for entity/claim coverage against those references. High automatic explanation scores can partly reflect agreement with VLM-preferred cue phrasing rather than independently verified human-grounded evidence. This is a mild self-referential measurement loop, not a definitional identity that forces detection F1, OOD drops, or spectral findings.

full rationale

XPlainVerse is a benchmark/dataset paper, not a first-principles derivation. Detection labels come from real sources plus generator outputs; Edit-Check verifies edit intent against image pairs rather than defining success as matching a fitted parameter; dual-level explanations and EntityScore/EvidenceScore are proposed evaluation constructs, not predictions forced by their own inputs. The main experimental findings—large ID→OOD F1 drops for fine-tuned VLMs, sharper Entity/Evidence degradation than BERTScore, and spectral/probe evidence of generator-specific shortcuts—are empirical comparisons on held-out generators and frozen embeddings, not tautologies. Human preference studies (Table 4, Figs. 4–5) and a 2,000-image gold subset provide external (if limited) anchors. Self-citations (e.g., LayLens, MultiFakeVerse) motivate dual-level explanations and supply some candidates but do not import a uniqueness theorem that forces the results. The only mild circularity-adjacent structure is that silver complex explanations and Entity/Evidence coverage are both VLM-mediated, so automatic explanation scores partly measure consistency with VLM-style rationales; that is a validity risk for the “grounded quality is measurable” claim, not a reduction of a claimed prediction to its defining inputs. Score 1 reflects that minor self-referential measurement structure without elevating it to load-bearing circular derivation.

Assumptions & free parameters 4 free parameters · 4 assumptions · 3 invented entities

The central scientific claim rests on treating edit-instruction compliance plus VLM difference extraction as a scalable proxy for ‘grounded’ manipulation evidence, and on treating VLM-extracted entity/evidence coverage as a valid explanation-quality measure. Free parameters are mostly pipeline thresholds and perturbation schedules; axioms are standard CV/XAI practice plus domain trust in commercial editors and judge models; invented entities are the named pipeline and metrics rather than physical objects.

free parameters (4)
  • Edit-Check KEEP threshold (final_score >= 4; BORDERLINE at 3)
    Discrete keep/discard cutoffs in the alignment judge directly determine which ~265K of ~609K candidates enter the benchmark and thus all downstream supervision.
  • Difference salience scale (1–5) and unrelated-change penalty (0–2)
    Salience and off-target penalties gate hard discards; values are design choices of the judge prompt, not measured physical constants.
  • Perturbation mixture rates and severity schedules by split
    Train/val/test single/pair/chunk percentages and parameter ranges (JPEG quality, noise σ, FOA/xTransfer ε, etc.) are hand-set and define the robustness evaluation distribution.
  • SLE clip/normalize bounds [-1,4] → [0,1]
    Reporting transform for simple-explanation readability is an ad hoc normalization choice affecting metric comparability.
assumptions (4)
  • domain assumption Visible compliance with a natural-language edit instruction (core intent present, no dominant off-target rewrite) is a valid proxy for high-quality manipulation supervision for explanation learning.
    Load-bearing for Edit-Check (Section 3.1 Step 3); without it, discarded/kept sets do not define ‘grounded’ labels.
  • domain assumption Ensemble VLMs can extract verifiable visual differences and produce explanations that, after filtering, approximate human forensic grounding at scale.
    Used throughout generation, filtering, and silver explanations; only partially validated on 2K human annotations.
  • ad hoc to paper Bidirectional semantic coverage of extracted entities and evidence claims (via Qwen3.5-4B) measures explanation fidelity better than surface similarity alone.
    Defines EntityScore/EvidenceScore (Section 4, Appendix I); metric validity is internal to this evaluation design.
  • standard math Standard deepfake/XAI evaluation practices: binary F1, BERTScore, human Likert/preference studies, frozen embedding probes, FFT spectral distance to real images.
    Background methods assumed valid for supporting the empirical claims in Sections 3–5.
invented entities (3)
  • Edit-Check multi-stage filtering pipeline independent evidence
    purpose: Scale reliable fake-image retention by verifying intended edits and discarding off-target generations before explanation supervision.
    Named contribution of the paper; independent evidence is empirical (kept vs discarded fidelity and human edit-fidelity scores), not an external physical discovery.
  • EntityScore and EvidenceScore
    purpose: Score whether explanations name the correct manipulated entities and supporting visual evidence beyond BERTScore.
    New metrics defined via VLM extraction and coverage; validated only within this paper’s human/auto comparisons.
  • Dual-level (complex vs simple) explanation tracks with SLE for simplicity independent evidence
    purpose: Serve technical vs non-expert users with different explanation styles on the same images.
    Dataset design choice motivated by human-centered XAI and LayLens; preference studies provide internal support.

how reviews work

0 comments
Cite this review

Pith. "Pith review of XPlainVerse: A Million-Scale Benchmark for Explainable Deepfake Detection." pith.science (2026). https://pith.science/paper/AYZ4YC25

@misc{pith2026260703562,
  author       = {Pith},
  title        = {Pith review of: XPlainVerse: A Million-Scale Benchmark for Explainable Deepfake Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AYZ4YC25}},
  note         = {Machine review of arXiv:2607.03562}
}
read the original abstract

As deepfake detection models increasingly produce natural language explanations, their reasoning often remains weakly grounded in visual artifacts, limiting reliability and user trust. Existing benchmarks mainly evaluate classification accuracy, overlooking whether explanations reflect the actual manipulations. This gap hinders progress toward deployable, explainable deepfake detection systems. To this end, we introduce XPlainVerse, a large-scale benchmark designed for joint deepfake detection and human-centered explanation. XPlainVerse comprises one million real and manipulated images, pairing authentic images from five established sources with forgeries generated by twelve off-the-shelf image editing and synthesis models. We further propose a multi-stage filtering pipeline, Edit-Check, to verify if manipulations satisfy their intended edits, enabling reliable reasoning supervision at scale. Beyond dataset scale, XPlainVerse provides two complementary explanation styles: technical explanations for expert analysis and simplified explanations optimized for non-technical users. To evaluate explanation quality beyond surface similarity, we propose novel metrics, EntityScore and EvidenceScore, that measure reasoning fidelity by checking whether explanations correctly identify manipulated entities and visual evidence. Human annotations on 2,000 explanation pairs validate our dataset quality against human judgment. We believe XPlainVerse will establish grounded explanation quality as a measurable dimension of deepfake detection and support scalable research on trustworthy, interpretable models.

Figures

Figures reproduced from arXiv: 2607.03562 by the authors.

Figure 1
Figure 1. The proposed large-scale deepfake explainability dataset XPlainVerse created with (a) Edit [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of the XPlainVerse data generation pipeline. For each real image, we generate [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. a) Distribution for the real data sources subset across the Train, Validation, and Test splits. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: (a) Realism comparison between real, filtered, and discarded fakes. (b) Edit-fidelity scores. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: (a) Use-case preference comparison between complex and simple explanations. (b) Prefer [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: (a) Linear probes on frozen CLIP, DINOv2, and SigLIP embeddings. (b) Increasing probe [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Visual-difference extraction examples. The examples show that the pipeline records [PITH_FULL_IMAGE:figures/full_fig_p022_7.png]
Figure 8
Figure 8. Figure 8: Judge scoring and rationale examples [PITH_FULL_IMAGE:figures/full_fig_p023_8.png]
Figure 9
Figure 9. Figure 9: Qualitative perturbation examples. Perturbations alter low-level image statistics while [PITH_FULL_IMAGE:figures/full_fig_p025_9.png]
Figure 10
Figure 10. Figure 10: Fake-image explanation examples. Complex explanations preserve multiple grounded cues; Simple explanations select the single clearest cue and use non-technical language. 27 [PITH_FULL_IMAGE:figures/full_fig_p027_10.png]
Figure 11
Figure 11. Figure 11: Authenticity explanation examples for real images. Each explanation should point to [PITH_FULL_IMAGE:figures/full_fig_p028_11.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Explainable Deepfake Detection Challenge

    cs.CV 2026-07 conditional novelty 5.0 of 10

    A joint deepfake detection-and-explanation benchmark shows that classification accuracy and grounded, audience-appropriate explanations are separate capabilities, with Pixel Sleuth winning at 0.7612.

Reference graph

Works this paper leans on

110 extracted references · 2 canonical work pages · cited by 1 Pith paper

  1. [1]

    Prompt-to-prompt image editing with cross attention control.arXiv preprint arXiv:2208.01626, 2022

    Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to-prompt image editing with cross attention control.arXiv preprint arXiv:2208.01626, 2022

    arXiv 2022

  2. [2]

    Instructpix2pix: Learning to follow image editing instructions

    Tim Brooks, Aleksander Holynski, and Alexei A Efros. Instructpix2pix: Learning to follow image editing instructions. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 18392–18402, 2023

    2023

  3. [3]

    Qwen-image technical report, 2025

    Chenfei Wu, Jiahao Li, Jingren Zhou, Junyang Lin, Kaiyuan Gao, Kun Yan, Sheng ming Yin, Shuai Bai, Xiao Xu, Yilei Chen, Yuxiang Chen, Zecheng Tang, Zekai Zhang, Zhengyi Wang, An Yang, Bowen Yu, Chen Cheng, Dayiheng Liu, Deqing Li, Hang Zhang, Hao Meng, Hu Wei, Jingyuan Ni, Kai Chen, Kuan Cao, Liang Peng, Lin Qu, Minggang Wu, Peng Wang, Shuting Yu, Tingkun...

    arXiv 2025

  4. [4]

    Faceforensics++: Learning to detect manipulated facial images, 2019

    Andreas Rössler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, and Matthias Nießner. Faceforensics++: Learning to detect manipulated facial images, 2019. URL https: //arxiv.org/abs/1901.08971

    arXiv 2019

  5. [5]

    The deepfake detection challenge (dfdc) dataset, 2020

    Brian Dolhansky, Joanna Bitton, Ben Pflaum, Jikuo Lu, Russ Howes, Menglin Wang, and Cristian Canton Ferrer. The deepfake detection challenge (dfdc) dataset, 2020. URL https: //arxiv.org/abs/2006.07397

    arXiv 2020

  6. [6]

    Sida: Social media image deepfake de- tection, localization and explanation with large multimodal model, 2025

    Zhenglin Huang, Jinwei Hu, Xiangtai Li, Yiwei He, Xingyu Zhao, Bei Peng, Baoyuan Wu, Xiaowei Huang, and Guangliang Cheng. Sida: Social media image deepfake de- tection, localization and explanation with large multimodal model, 2025. URL https: //arxiv.org/abs/2412.04292

    arXiv 2025

  7. [7]

    Genimage: A million-scale benchmark for detecting ai-generated image, 2023

    Mingjian Zhu, Hanting Chen, Qiangyu Yan, Xudong Huang, Guanyu Lin, Wei Li, Zhijun Tu, Hailin Hu, Jie Hu, and Yunhe Wang. Genimage: A million-scale benchmark for detecting ai-generated image, 2023. URLhttps://arxiv.org/abs/2306.08571

    arXiv 2023

  8. [8]

    Dire for diffusion-generated image detection, 2023

    Zhendong Wang, Jianmin Bao, Wengang Zhou, Weilun Wang, Hezhen Hu, Hong Chen, and Houqiang Li. Dire for diffusion-generated image detection, 2023. URL https://arxiv.org/ abs/2303.09295

    arXiv 2023

Show all 110 references
  1. [9]

    Common sense reasoning for deepfake detection, 2024

    Yue Zhang, Ben Colman, Xiao Guo, Ali Shahriyari, and Gaurav Bharaj. Common sense reasoning for deepfake detection, 2024. URLhttps://arxiv.org/abs/2402.00126

    2024 arXiv

  2. [10]

    Ffaa: Multimodal large language model based explainable open-world face forgery analysis assistant,

    Zhengchao Huang, Bin Xia, Zicheng Lin, Zhun Mou, Wenming Yang, and Jiaya Jia. Ffaa: Multimodal large language model based explainable open-world face forgery analysis assistant,

  3. [11]

    URLhttps://arxiv.org/abs/2408.10072

    arXiv

  4. [12]

    Fakebench: Probing explainable fake image detection via large multimodal models, 2024

    Yixuan Li, Xuelin Liu, Xiaoyang Wang, Bu Sung Lee, Shiqi Wang, Anderson Rocha, and Weisi Lin. Fakebench: Probing explainable fake image detection via large multimodal models, 2024. URLhttps://arxiv.org/abs/2404.13306

    2024 arXiv

  5. [13]

    X-transfer attacks: Towards super transferable adversarial attacks on clip, 2025

    Hanxun Huang, Sarah Erfani, Yige Li, Xingjun Ma, and James Bailey. X-transfer attacks: Towards super transferable adversarial attacks on clip, 2025. URL https://arxiv.org/abs/ 2505.05528

    2025 arXiv

  6. [14]

    Adversarial attacks against closed-source mllms via feature optimal alignment, 2025

    Xiaojun Jia, Sensen Gao, Simeng Qin, Tianyu Pang, Chao Du, Yihao Huang, Xinfeng Li, Yiming Li, Bo Li, and Yang Liu. Adversarial attacks against closed-source mllms via feature optimal alignment, 2025. URLhttps://arxiv.org/abs/2505.21494

    2025 arXiv

  7. [15]

    Reveal: Reasoning-enhanced forensic evidence analysis for explainable ai-generated image detection, 2026

    Huangsen Cao, Qin Mei, Zhiheng Li, Yuxi Li, Zhan Meng, Ying Zhang, Chen Li, Zhimeng Zhang, Xin Ding, Yongwei Wang, Jing Lyu, and Fei Wu. Reveal: Reasoning-enhanced forensic evidence analysis for explainable ai-generated image detection, 2026. URL https: //arxiv.org/abs/2511.23158. 10

    2026 arXiv

  8. [16]

    Interpretable and reliable detection of ai-generated images via grounded reasoning in mllms, 2025

    Yikun Ji, Hong Yan, Jun Lan, Huijia Zhu, Weiqiang Wang, Qi Fan, Liqing Zhang, and Jianfu Zhang. Interpretable and reliable detection of ai-generated images via grounded reasoning in mllms, 2025. URLhttps://arxiv.org/abs/2506.07045

    2025 arXiv

  9. [17]

    Aigi-holmes: Towards explainable and generalizable ai-generated image detection via multimodal large language models, 2025

    Ziyin Zhou, Yunpeng Luo, Yuanchen Wu, Ke Sun, Jiayi Ji, Ke Yan, Shouhong Ding, Xiaoshuai Sun, Yunsheng Wu, and Rongrong Ji. Aigi-holmes: Towards explainable and generalizable ai-generated image detection via multimodal large language models, 2025. URL https: //arxiv.org/abs/2507.02664

    2025 arXiv

  10. [18]

    Spot the fake: Large multimodal model-based synthetic image detection with artifact explanation, 2025

    Siwei Wen, Junyan Ye, Peilin Feng, Hengrui Kang, Zichen Wen, Yize Chen, Jiang Wu, Wenjun Wu, Conghui He, and Weijia Li. Spot the fake: Large multimodal model-based synthetic image detection with artifact explanation, 2025. URLhttps://arxiv.org/abs/2503.14905

    2025

  11. [19]

    Locate-then-examine: Grounded region reasoning improves detection of ai-generated images, 2026

    Yikun Ji, Yan Hong, Bowen Deng, Jun Lan, Huijia Zhu, Weiqiang Wang, Liqing Zhang, and Jianfu Zhang. Locate-then-examine: Grounded region reasoning improves detection of ai-generated images, 2026. URLhttps://arxiv.org/abs/2510.04225

    2026 arXiv

  12. [20]

    Explanation in artificial intelligence: Insights from the social sciences.Artificial Intelligence, 267:1–38, 2019

    Tim Miller. Explanation in artificial intelligence: Insights from the social sciences.Artificial Intelligence, 267:1–38, 2019. ISSN 0004-3702. doi: https://doi.org/10.1016/j.artint.2018.07.007. URLhttps://www.sciencedirect.com/science/article/pii/S0004370218305988

    2019 doi

  13. [21]

    Can we do better explanations? a proposal of user-centered explainable ai

    Mireia Ribera Turró and Agata Lapedriza. Can we do better explanations? a proposal of user-centered explainable ai. 03 2019

    2019

  14. [22]

    Vera Liao, Daniel Gruen, and Sarah Miller

    Q. Vera Liao, Daniel Gruen, and Sarah Miller. Questioning the ai: Informing design practices for explainable ai user experiences. InProceedings of the 2020 CHI Conference on Human Factors in Computing Systems, CHI ’20, page 1–15, New York, NY , USA, 2020. Association for Compu...

    2020 doi

  15. [23]

    Upol Ehsan and Mark O. Riedl. Human-centered explainable ai: Towards a reflective sociotech- nical approach, 2020. URLhttps://arxiv.org/abs/2002.01092

    2020 arXiv

  16. [24]

    Pentyala, Eric D

    Rhema Linder, Sina Mohseni, Fan Yang, Shiva K. Pentyala, Eric D. Ragan, and Xia Ben Hu. How level of explanation detail affects human performance in interpretable intelligent systems: A study on explainable fact checking.Applied AI Letters, 2(4):e49, 2021. doi: https://doi.org...

    2021 doi

  17. [25]

    Laylens: Improving deepfake understanding through simplified explanations

    Abhijeet Narang, Parul Gupta, Liuyijia Su, and Abhinav Dhall. Laylens: Improving deepfake understanding through simplified explanations. InProceedings of the 27th International Conference on Multimodal Interaction, ICMI ’25, page 788–790, New York, NY , USA, 2025. Association ...

    2025 doi

  18. [26]

    On the detection of digital face manipulation, 2020

    Hao Dang, Feng Liu, Joel Stehouwer, Xiaoming Liu, and Anil Jain. On the detection of digital face manipulation, 2020. URLhttps://arxiv.org/abs/1910.01717

    2020 arXiv

  19. [27]

    Multi- verse through deepfakes: The multifakeverse dataset of person-centric visual and concep- tual manipulations

    Parul Gupta, Shreya Ghosh, Tom Gedeon, Thanh-Toan Do, and Abhinav Dhall. Multi- verse through deepfakes: The multifakeverse dataset of person-centric visual and concep- tual manipulations. InProceedings of the 33rd ACM International Conference on Multi- media, MM ’25, page 132...

    2025 doi

  20. [28]

    Semi-truths: A large-scale dataset of ai-augmented images for evaluating robustness of ai-generated image detectors, 2024

    Anisha Pal, Julia Kruk, Mansi Phute, Manognya Bhattaram, Diyi Yang, Duen Horng Chau, and Judy Hoffman. Semi-truths: A large-scale dataset of ai-augmented images for evaluating robustness of ai-generated image detectors, 2024. URL https://arxiv.org/abs/2411. 07472

    2024

  21. [29]

    Spatial-phase shallow learning: Rethinking face forgery detection in frequency domain, 2021

    Honggu Liu, Xiaodan Li, Wenbo Zhou, Yuefeng Chen, Yuan He, Hui Xue, Weiming Zhang, and Nenghai Yu. Spatial-phase shallow learning: Rethinking face forgery detection in frequency domain, 2021. URLhttps://arxiv.org/abs/2103.01856. 11

    2021 arXiv

  22. [30]

    Face x-ray for more general face forgery detection, 2020

    Lingzhi Li, Jianmin Bao, Ting Zhang, Hao Yang, Dong Chen, Fang Wen, and Baining Guo. Face x-ray for more general face forgery detection, 2020. URL https://arxiv.org/abs/ 1912.13458

    2020 arXiv

  23. [31]

    In ictu oculi: Exposing ai generated fake face videos by detecting eye blinking, 2018

    Yuezun Li, Ming-Ching Chang, and Siwei Lyu. In ictu oculi: Exposing ai generated fake face videos by detecting eye blinking, 2018. URLhttps://arxiv.org/abs/1806.02877

    2018 arXiv

  24. [32]

    Mesonet: a compact facial video forgery detection network

    Darius Afchar, Vincent Nozick, Junichi Yamagishi, and Isao Echizen. Mesonet: a compact facial video forgery detection network. In2018 IEEE International Workshop on Information Forensics and Security (WIFS), page 1–7. IEEE, December 2018. doi: 10.1109/wifs.2018.8630761. URL ht...

    2018 doi

  25. [33]

    David Güera and Edward J. Delp. Deepfake video detection using recurrent neural networks. In 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pages 1–6, 2018. doi: 10.1109/A VSS.2018.8639163

    2018 doi

  26. [34]

    Multi-attentional deepfake detection, 2021

    Hanqing Zhao, Wenbo Zhou, Dongdong Chen, Tianyi Wei, Weiming Zhang, and Nenghai Yu. Multi-attentional deepfake detection, 2021. URLhttps://arxiv.org/abs/2103.02406

    2021 arXiv

  27. [35]

    Patch diffusion: a general module for face manipulation detection

    Baogen Zhang, Sheng Li, Guorui Feng, Zhenxing Qian, and Xinpeng Zhang. Patch diffusion: a general module for face manipulation detection. InProceedings of the AAAI conference on artificial intelligence, volume 36, pages 3243–3251, 2022

    2022

  28. [36]

    M2tr: Multi-modal multi-scale transformers for deepfake detection

    Junke Wang, Zuxuan Wu, Wenhao Ouyang, Xintong Han, Jingjing Chen, Yu-Gang Jiang, and Ser-Nam Li. M2tr: Multi-modal multi-scale transformers for deepfake detection. InProceedings of the 2022 International Conference on Multimedia Retrieval, ICMR ’22, page 615–623, New York, NY ...

    2022 doi

  29. [37]

    Fakeshield: Explainable image forgery detection and localization via multi-modal large language models,

    Zhipei Xu, Xuanyu Zhang, Runyi Li, Zecheng Tang, Qing Huang, and Jian Zhang. Fakeshield: Explainable image forgery detection and localization via multi-modal large language models,

  30. [38]

    URLhttps://arxiv.org/abs/2410.02761

    arXiv

  31. [39]

    Busterx++: Towards unified cross-modal ai-generated content detection and explanation with mllm, 2026

    Haiquan Wen, Tianxiao Li, Zhenglin Huang, Yiwei He, and Guangliang Cheng. Busterx++: Towards unified cross-modal ai-generated content detection and explanation with mllm, 2026. URLhttps://arxiv.org/abs/2507.14632

    2026 arXiv

  32. [40]

    Pixels don’t lie (but your detector might): Bootstrapping mllm-as-a-judge for trustworthy deepfake detection and reasoning supervision, 2026

    Kartik Kuckreja, Parul Gupta, Muhammad Haris Khan, and Abhinav Dhall. Pixels don’t lie (but your detector might): Bootstrapping mllm-as-a-judge for trustworthy deepfake detection and reasoning supervision, 2026. URLhttps://arxiv.org/abs/2602.19715

    2026

  33. [41]

    Towards human-centered explainable ai: A survey of user studies for model explanations.IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(4):2104–2122, April 2024

    Yao Rong, Tobias Leemann, Thai-Trang Nguyen, Lisa Fiedler, Peizhu Qian, Vaibhav Unhelkar, Tina Seidel, Gjergji Kasneci, and Enkelejda Kasneci. Towards human-centered explainable ai: A survey of user studies for model explanations.IEEE Transactions on Pattern Analysis and Machi...

    2024 doi

  34. [42]

    Vera Liao and Kush R

    Q. Vera Liao and Kush R. Varshney. Human-centered explainable ai (xai): From algorithms to user experiences, 2022. URLhttps://arxiv.org/abs/2110.10790

    2022 arXiv

  35. [43]

    Alina Kuznetsova, Hassan Rom, Neil Alldrin, Jasper Uijlings, Ivan Krasin, Jordi Pont-Tuset, Shahab Kamali, Stefan Popov, Matteo Malloci, Alexander Kolesnikov, Tom Duerig, and Vittorio Ferrari. The open images dataset v4: Unified image classification, object detection, and visu...

    1956 doi

  36. [44]

    Context based emotion recognition using emotic dataset.IEEE Transactions on Pattern Analysis and Machine In- telligence, page 1–1, 2019

    Ronak Kosti, Jose Alvarez, Adria Recasens, and Agata Lapedriza. Context based emotion recognition using emotic dataset.IEEE Transactions on Pattern Analysis and Machine In- telligence, page 1–1, 2019. ISSN 1939-3539. doi: 10.1109/tpami.2019.2916866. URL http://dx.doi.org/10.11...

    2019 doi

  37. [45]

    Person recognition in personal photo collections, 2015

    Seong Joon Oh, Rodrigo Benenson, Mario Fritz, and Bernt Schiele. Person recognition in personal photo collections, 2015. URLhttps://arxiv.org/abs/1509.03502. 12

    2015 arXiv

  38. [46]

    Human-centric relation segmentation: Dataset and solution, 2021

    Si Liu, Zitian Wang, Yulu Gao, Lejian Ren, Yue Liao, Guanghui Ren, Bo Li, and Shuicheng Yan. Human-centric relation segmentation: Dataset and solution, 2021. URL https://arxiv. org/abs/2105.11168

    2021 arXiv

  39. [47]

    Kankanhalli

    Junnan Li, Yongkang Wong, Qi Zhao, and Mohan S. Kankanhalli. Visual social relationship recognition, 2018. URLhttps://arxiv.org/abs/1812.05917

    2018 arXiv

  40. [48]

    Gpt-4o mini: Advancing cost-efficient intelligence

    OpenAI. Gpt-4o mini: Advancing cost-efficient intelligence. OpenAI Blog, 2024. URL https: //openai.com/index/gpt-4o-mini-advancing-cost-efficient-intelligence/

    2024

  41. [49]

    Flux.2-dev

    Black Forest Labs. Flux.2-dev. Hugging Face Model Card, 2026. URL https:// huggingface.co/black-forest-labs/FLUX.2-dev

    2026

  42. [50]

    Hunyuanimage 3.0 technical report, 2026

    Siyu Cao, Hangting Chen, Peng Chen, Yiji Cheng, Yutao Cui, Xinchi Deng, Ying Dong, Kipper Gong, Tianpeng Gu, Xiusen Gu, Tiankai Hang, Duojun Huang, Jie Jiang, Zhengkai Jiang, Weijie Kong, Changlin Li, Donghao Li, Junzhe Li, Xin Li, Yang Li, Zhenxi Li, Zhimin Li, Jiaxin Lin, Li...

    2026 arXiv

  43. [51]

    Longcat-image technical report, 2025

    Meituan LongCat Team, Hanghang Ma, Haoxian Tan, Jiale Huang, Junqiang Wu, Jun-Yan He, Lishuai Gao, Songlin Xiao, Xiaoming Wei, Xiaoqi Ma, Xunliang Cai, Yayong Guan, and Jie Hu. Longcat-image technical report, 2025. URLhttps://arxiv.org/abs/2512.07584

    2025 arXiv

  44. [52]

    Gpt image 1.5 model documentation

    OpenAI. Gpt image 1.5 model documentation. OpenAI API Documentation, 2025. URL https://developers.openai.com/api/docs/models/gpt-image-1.5

    2025

  45. [53]

    Seedream 4.5

    ByteDance Seed Team. Seedream 4.5. Official Model Page, 2026. URL https://seed. bytedance.com/en/seedream4_5

    2026

  46. [54]

    Wan2.6 image generation and editing

    Alibaba Cloud. Wan2.6 image generation and editing. Alibaba Cloud Model Studio Documentation, 2026. URL https://www.alibabacloud.com/help/en/model-studio/ wan-image-generation-api-reference

    2026

  47. [55]

    Build with nano banana 2

    Google. Build with nano banana 2. Google Blog, 2026. URL https: //blog.google/innovation-and-ai/technology/developers-tools/ build-with-nano-banana-2/

    2026

  48. [56]

    Introducing nano banana pro

    Google DeepMind. Introducing nano banana pro. Google Blog, 2025. URL https://blog. google/innovation-and-ai/products/nano-banana-pro/

    2025

  49. [57]

    Experiment with gemini 2.0 flash native image generation

    Google. Experiment with gemini 2.0 flash native image generation. Google Developers Blog, 2025. URL https://developers.googleblog.com/ experiment-with-gemini-20-flash-native-image-generation/ . Accessed: 2026-05-06

    2025

  50. [58]

    Gpt image 1 model

    OpenAI. Gpt image 1 model. OpenAI API Documentation, 2025. URL https://developers. openai.com/api/docs/models/gpt-image-1. Accessed: 2026-05-06

    2025

  51. [59]

    In-context edit: Enabling instructional image editing with in-context generation in large scale diffusion transformer, 2025

    Zechuan Zhang, Ji Xie, Yu Lu, Zongxin Yang, and Yi Yang. In-context edit: Enabling instructional image editing with in-context generation in large scale diffusion transformer, 2025. URLhttps://arxiv.org/abs/2504.20690

    2025 arXiv

  52. [60]

    Gemini 3 flash: Frontier intelligence built for speed

    Google. Gemini 3 flash: Frontier intelligence built for speed. Google Blog, 2025. URL https: //blog.google/products-and-platforms/products/gemini/gemini-3-flash/

    2025

  53. [61]

    Gpt-5 mini model documentation

    OpenAI. Gpt-5 mini model documentation. OpenAI API Documentation, 2026. URL https: //developers.openai.com/api/docs/models/gpt-5-mini. 13

    2026

  54. [62]

    Glm-4.6v: Open source multimodal models with native tool use

    Z.ai. Glm-4.6v: Open source multimodal models with native tool use. Z.ai Developer Docu- mentation, 2025. URLhttps://docs.z.ai/guides/vlm/glm-4.6v

    2025

  55. [63]

    Qwen3.5: Towards native multimodal agents, February 2026

    Qwen Team. Qwen3.5: Towards native multimodal agents, February 2026. URL https: //qwen.ai/blog?id=qwen3.5

    2026

  56. [64]

    DeepSeek-AI, Aixin Liu, Aoxue Mei, Bangcai Lin, Bing Xue, Bingxuan Wang, Bingzheng Xu, Bochao Wu, Bowei Zhang, Chaofan Lin, Chen Dong, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenhao Xu, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Erhang Li, Fangqi Zhou, Fangyun...

    2025 arXiv

  57. [65]

    Introducing gpt-oss

    OpenAI. Introducing gpt-oss. OpenAI Blog, 2025. URL https://openai.com/index/ introducing-gpt-oss/

    2025

  58. [66]

    Gemini 2.5 pro

    Google. Gemini 2.5 pro. Google AI for Developers Documentation, 2025. URL https: //ai.google.dev/gemini-api/docs/models/gemini-2.5-pro

    2025

  59. [67]

    Weinberger, and Yoav Artzi

    Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. Bertscore: Evaluating text generation with bert, 2020. URLhttps://arxiv.org/abs/1904.09675

    2020 arXiv

  60. [68]

    Simplicity level estimate (sle): A learned reference-less metric for sentence simplification, 2023

    Liam Cripwell, Joël Legrand, and Claire Gardent. Simplicity level estimate (sle): A learned reference-less metric for sentence simplification, 2023. URLhttps://arxiv.org/abs/2310. 08170

    2023

  61. [69]

    Qwen3-vl technical report, 2025

    Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie 14 Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhaohai Li, Mingsheng Li, Mei Li, Kaixin Li, Zicheng Lin, Ju...

    2025 arXiv

  62. [70]

    Internvl3.5: Advancing open-source multimodal models in versatility, reasoning, and efficiency, 2025

    Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, Zhaokai Wang, Zhe Chen, Hongjie Zhang, Ganlin Yang, Haomin Wang, Qi Wei, Jinhui Yin, Wenhao Li, Erfei Cui, Guanzhou Chen, Zichen Ding, Changyao Tian, Z...

    2025 arXiv

  63. [71]

    Visual instruction tuning, 2023

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, 2023. URLhttps://arxiv.org/abs/2304.08485

    2023 arXiv

  64. [72]

    Llava-next: Improved reasoning, ocr, and world knowledge

    LLaV A Team. Llava-next: Improved reasoning, ocr, and world knowledge. Project Blog, 2024. URLhttps://llava-vl.github.io/blog/2024-01-30-llava-next/. 15 Appendix XPlainVerse: A Million-Scale Benchmark for Explainable Deepfake Detection Supplementary Material A Appendix Roadmap...

    2024

  65. [73]

    Image real- ism Judge whether individ- ual images appear real or manipulated Real/fake judgment, confidence, visual realism Choice + 1–5 scale

  66. [74]

    Edit quality Evaluate paired origi- nal and edited images Core edit presence, edit realism, unrelated off-target changes 1–5 scale

  67. [75]

    Explana- tion quality Rate one explanation for one image Grounding, specificity, completeness, convincingness, unsupported claims 1–5 scale

  68. [76]

    Ex- planation preference Compare two explana- tions for the same im- age Grounding, specificity, completeness, fewer unsupported claims, overall usefulness A/B/Tie

  69. [77]

    generated comparison Compare human- written and generated complex explanations Grounding, specificity, completeness, fewer unsupported claims, overall preference A/B/Tie

    Human vs. generated comparison Compare human- written and generated complex explanations Grounding, specificity, completeness, fewer unsupported claims, overall preference A/B/Tie

  70. [78]

    Readability Evaluate simple/user- facing explanations Ease of understanding, memorability, cognitive load, liking 1–5 scale

  71. [79]

    not sure

    Audience usefulness Compare explanations for different users Personal preference, usefulness for ordinary users, usefulness for technical analysts, clarity/evidence balance, default system choice Choice 26 Original Edited Edit/diff:Add a large construction crane in the backgro...

  72. [80]

    Subtlety / under-edit risk (MOST IMPORTANT) - Is the intended change small, fine-grained, or easy for models to ignore/half-apply? - If yes, reduce P(success) aggressively, even if the edit sounds “small”

  73. [81]

    Instance/identity lock (replacement not allowed) - Must remain the same subject/object instance (not swapped for a different-looking one)? - Strongly reduce P(success) if identity must remain stable while only one attribute changes

  74. [82]

    Locality + collateral-damage intolerance - Must be confined to a specific region while everything else stays identical? - Higher intolerance to spillover => lower P(success)

  75. [83]

    Structure continuity / reveal-cover requirements - Does the edit require plausible reconstruction of previously occluded areas or seamless continuation of edges/textures/geometry? - If yes, lower P(success), especially when the reconstructed region must align with existing structures

  76. [84]

    difficulty_scores

    Caption underspecification 38 - If the caption doesn’t provide enough detail to judge what “correct” means (placement/appearance), treat the instruction as harder (lower P(success)). Magnitude rule (secondary): - Do NOT automatically rate large changes as hard. If the instruct...

  77. [85]

    K.4 Visual-Difference Extraction Prompt System prompt You are a Visual Difference Extractor

    {instruction_2} ... K.4 Visual-Difference Extraction Prompt System prompt You are a Visual Difference Extractor. INPUTS: - Image A: original/real image - Image B: generated/edited image GOAL: List only the differences that are VISUALLY VERIFIABLE between Image A and Image B. D...

  78. [86]

    overall_summary

    "overall_summary": string

  79. [87]

    changes": array of objects Each object in

    "changes": array of objects Each object in "changes" MUST have exactly: - "entity": string - "salience": integer from 1 to 5 - "differences": array of strings FORMAT for each string in "differences": "A: <what is visible in Image A>; B: <what is visible in Image B>." HOW TO CO...

  80. [88]

    before/after

    Do NOT mention or imply: any other image, comparisons, “before/after”, “edit instruction”, “difference JSON”, “Image A/B”, “original/real/reference”, or “compared to”

  81. [89]

    - If you only know it because of the auxiliary references, DO NOT include it

    Every claim must be directly and clearly supported by visible evidence in the GENERATED_IMAGE alone. - If you only know it because of the auxiliary references, DO NOT include it

  82. [90]

    explanation

    You may use the auxiliary references ONLY to decide where to inspect more carefully. - Do NOT copy wording from DIFFERENCE_JSON. - Do NOT introduce claims that you cannot see in the GENERATED_IMAGE. WHAT TO LOOK FOR (examples): - warped geometry (hands/face asymmetry, bent str...

  83. [91]

    Write the final explanation as if you only saw the GENERATED_IMAGE

  84. [92]

    A” or “B

    Do NOT mention or hint at: - the original image - a reference image - before/after - edit instruction - difference JSON - intended change - “A” or “B”

  85. [93]

    Do NOT use comparison phrases like: - changed from - used to - originally - compared to - should be - restored

  86. [94]

    Do NOT introduce a totally new main person, object, or story that is not connected to the COMPLEX_EXPLANATION and/or DIFFERENCE_JSON

  87. [95]

    You MAY say new simple things about the same changed thing, as long as they are easy to see in the GENERATED_IMAGE

  88. [96]

    If you are unsure, choose the clearest and most obvious reason only

  89. [97]

    explanation

    Do not try to sound smart. Sound clear. OUTPUT FORMAT: Return ONLY valid JSON with exactly one key: { "explanation": string } FINAL CHECK BEFORE ANSWERING: - Is this about just one main changed thing? - Is this easy for a five-year-old to understand? - Did I avoid hard words? ...

  90. [98]

    diagnostic_entities: the entities that are actually used as evidence for the authenticity judgment

  91. [99]

    the image looks fake

    evidence_claims: the specific claims that explain why those entities make the image look fake or real Important: Do NOT extract all entities mentioned in the explanation. Do NOT extract all factual statements. Extract only the entities and claims that function as evidence for ...

  92. [100]

    Use only information explicitly stated in the explanation

  93. [101]

    Do not infer unstated causes, intentions, or visual details

  94. [102]

    Extract only evidence-bearing entities

  95. [103]

    Extract only evidence-bearing claims

  96. [104]

    Each claim must be specific and checkable

  97. [105]

    Keep wording normalized and concise

  98. [106]

    Do not create entities for minor sub-parts unless the explanation is clearly centered on them

  99. [107]

    Do not include non-evidential context

  100. [108]

    diagnostic_entities

    Return valid JSON only Output schema: { "diagnostic_entities": [ { "entity_id": "E1", "name": "..." } ], "evidence_claims": [ { "claim_id": "C1", "entity_id": "E1", "claim": "..." } ] } Additional guidance: - Prefer the entity that actually carries the evidence - Do not includ...

  101. [109]

    a reference JSON containing: - diagnostic_entities - evidence_claims

  102. [110]

    Would a reasonable human judge say that the candidate explanation covers this same evidence, even if phrased differently?

    a candidate explanation Your job is to determine, for each diagnostic entity and each evidence claim in the reference JSON, whether it is present in the candidate explanation. This is a LENIENT SEMANTIC MATCHING task. Do NOT require exact wording. Match based on core meaning. ...

Pith tools

Reviewed July 12, 2026 · model on record in the stance chip above.