Pith. sign in

REVIEW 5 major objections 5 minor 29 references

METER: Multi-modal Evidence-based Thinking and Explainable Reasoning -- Algorithm and Benchmark

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A single benchmark demands evidence chains for fake media across images, video, audio, and audio-visual content, and a three-stage training pipeline is claimed to produce state-of-the-art explainable detections.

desk verdict A well-structured proposal for a full-modal explainable forgery benchmark, but with no experiments, no data release, and no annotation validation, the SOTA claim is a promise, not a result. read the letter →

arxiv 2507.16206 v1 pith:ZOG7AC6K submitted 2025-07-22 cs.LG cs.AI

Xu Yang , Qi Zhang , Shuming Jiang , Yaowen Xu , Zhaofan Zou , Hao Sun , Xuelong Li This is my paper
classification cs.LGcs.AI
keywords multi-modalforgerydetectionexplainableAIevidencechainChain-of-ThoughtreasoningbenchmarkdatasetGRPOreinforcementlearningdeepfakeaudio-visualforensics
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Binary real-versus-fake labels are not enough for trustworthy media forensics; a detector must say where a forgery is, why it is a forgery, and how it was made. This paper claims METER is the first benchmark to demand and evaluate such evidence chains across images, video, audio, and audio-visual content in one framework, covering both digital and physical attacks. It also claims a three-stage training recipe (supervised fine-tuning, preference optimization, and group-relative reinforcement learning with a human-aligned rationality evaluator) produces state-of-the-art evidence chains that are accurate in verdict, precise in localization, and convincing in rationale. If the claims hold, METER gives the field a common yardstick for explainable, cross-modal forgery detection.

What carries the argument

The central object is the evidence chain: a structured model output containing the real/fake verdict, spatio-temporal bounding boxes or intervals, the forgery type and generation method, and natural-language clues with a concluding summary. The training machinery is a three-stage pipeline that starts with supervised fine-tuning on the annotated chains, applies direct preference optimization on human-selected versus rejected chains, and finishes with group-relative policy optimization whose reward is a composite of format, rationality (from a fine-tuned multimodal evaluator), traceability, and length terms. The group-normalized advantage makes the final stage learn from relative quality within each sampled group, which is what the paper credits for the final jump in evidence quality.

What would settle it

Measure inter-annotator agreement on the rationality labels (e.g., Cohen's kappa over the two annotators' independent judgments) and have a panel of forensic media experts re-label a random sample; if agreement is near chance level or experts disagree with the majority labels, the human-aligned ground truth that anchors the whole pipeline is not reliable. A complementary test: shuffle the irrational/rational labels and retrain the GRPO stage; if the reported improvement over SFT persists, the reward signal is not the source of the gain.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central discovery is that full-modal explainable forgery detection can be organized as a single evidence-chain task, and that such chains can be generated and scored consistently across modalities. The benchmark defines four tracks (image, video, audio, audio-visual) and three interdependent criteria: spatial/temporal IoU for localization, multi-class traceability accuracy for source attribution, and a learned rationality score for the textual clues. Paired with this, the training pipeline first imitates curated annotations, then aligns to human preference pairs, then optimizes a composite reward under group-relative policy optimization. The reported experiments are claimed to outperform existing approaches on verdict accuracy, localization precision, and rationale quality.

Load-bearing premise

Ground truth for the benchmark is produced by having large language models propose clues, non-expert annotators correct boxes and judge rationality by majority vote, and reviewers pick preferred chains; the paper reports no agreement statistics or expert validation for these human judgments, so the benchmark's accuracy and the trained evaluator both depend on that unverified annotation quality.

Editorial extensions

If this is right

  • Detectors can move from a binary verdict to a checkable, localized explanation, which is what journalists, auditors, and courts need before they can act on a flag.
  • A single benchmark now exists for comparing explainable detection across image, video, audio, and audio-visual inputs, including physical attacks such as prints, screens, masks, and replay.
  • The SFT-DPO-GRPO recipe with a learned rationality reward provides a reusable template for aligning multimodal models to human judgment on reasoning-heavy tasks.
  • Traceability accuracy gives a concrete way to track which forgery techniques (T2I, face swap, lip-sync, voice cloning, etc.) current models can still attribute correctly.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the benchmark's validity rests on the unmeasured reliability of the non-expert annotation pipeline; a small expert-annotator agreement study would directly test whether the 'human-aligned' ground truth is aligned with expert forensic judgment.
  • Editorial inference: the rationality evaluator, trained on binary rational/irrational labels, could serve as a general reward model for other multimodal explanation tasks, since it encodes a reusable notion of evidence quality.
  • Editorial inference: the benchmark opens a concrete research question the paper leaves open: which forgery types and which attack vectors (digital vs physical) are systematically harder to localize and attribute, and whether the SOTA gap is driven by a few modalities.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper introduces METER, a proposed multi-modal benchmark and training pipeline for explainable forgery detection covering image, video, audio, and audio-visual tracks. The benchmark is described as including binary real/fake labels, spatio-temporal localization, forgery-type traceability, and human-aligned textual evidence chains. The method is a three-stage training strategy combining SFT, DPO, and GRPO, where the GRPO reward is computed from a composite signal that includes a learned rationality evaluator. The paper claims that the resulting method significantly outperforms existing approaches and sets a new state of the art, but the manuscript contains no experimental section, no quantitative results, and no baseline comparisons. The dataset construction, annotation pipeline, and evaluation metrics are described at a high level, with the full annotation protocol deferred to Appendix A.

Significance. If fully realized and validated, a unified multi-modal explainable forgery benchmark with spatial/temporal localization, traceability, and human-aligned rationales would be a valuable community resource, and the proposed three-stage training recipe is a reasonable direction. The paper also deserves credit for explicitly including both digital and physical attack types and for defining measurable criteria such as IoU, traceability accuracy, and evidence rationality. However, in its current form the central claims are unsupported: there are no experimental results, no dataset statistics, no validation of annotation quality, and no release details. The value of the contribution therefore remains conditional on substantial additional evidence that the manuscript does not provide.

major comments (5)
  1. [Section 5, Conclusion] The paper's load-bearing claim is stated in the Conclusion: 'Our extensive experiments show that this method significantly outperforms existing approaches, setting a new state of the art.' The manuscript, however, contains no experimental section, no tables, no baseline comparisons, and no quantitative results of any kind. This unsupported assertion is central to the paper's contribution and cannot be evaluated or reproduced from the text.
  2. [Section 3.2, Figure 3, Appendix A] The validity of the entire benchmark depends on the hybrid annotation pipeline, but no annotation quality statistics are reported. There is no inter-annotator agreement measure, no expert validation, no quantification of bounding-box corrections, no pass rates for the rationality filtering, and no example of annotation disagreements. Without these, the assertion that the dataset is 'human-aligned' and 'rigorous' is unfalsifiable, and errors in these labels propagate into every evaluation metric and every training stage.
  3. [Section 4.3.1 and Section 3.4.3] The rationality evaluation model is trained on the same binary rationality labels produced by the annotation pipeline, then used both as the GRPO reward component R_rationality and as the benchmark's evidence rationality metric. This creates a closed loop between the data, the reward, and the evaluation: the metric measures agreement with the annotators whose reliability is itself assumed. Independent validation against expert forensic judgments or a held-out human preference set is needed to support the 'human-aligned' claim.
  4. [Section 3.3 and Section 3.2] The paper describes dataset distribution and balance in qualitative terms but provides no quantitative statistics: no number of samples per track, no class balance per forgery type, no train/test/validation splits, no duration or resolution statistics, and no counts of evidence chains or annotations. For a benchmark paper, these numbers are essential for assessing coverage, difficulty, and usability, and their absence makes the proposed benchmark impossible to compare with or use.
  5. [Section 4.3.2] The composite reward function R includes several free coefficients and thresholds that are not specified, such as the -5 penalty for duplicated clues, the IoU>0.85 duplicate threshold, the length reward bounds in [-0.5, 0.5], and the relative weighting of the reward components. The GRPO hyperparameters (group size k, KL coefficient beta, clipping epsilon) are also omitted. The method therefore cannot be reproduced, and no sensitivity or ablation analysis is provided to justify these choices.
minor comments (5)
  1. [Section 2.2] There is a typographical duplication at the end of the LOKI comparison paragraph: the phrase 'detection capabilities.detection capabili- ties.' repeats and should be corrected.
  2. [Section 2.2] The reference to 'FakeA VCeleb' contains an extra space and should be written as 'FakeAVCeleb'.
  3. [Section 3.4.1] The text before Equation (2) reads 'and itbridges the semantic gap' and 'weuse temporal IoU (tIoU)' with missing spaces; these should be corrected.
  4. [General] The paper states that the benchmark is released to the community, but no dataset URL, code repository, or download link is provided; this should be added.
  5. [Section 3.3] Figure 2 is described as showing the dataset distribution, but no actual numbers are visible in the figure caption or referenced in the text; adding a table with per-category counts would make the distribution quantitative.

Circularity Check

1 steps flagged · score 6.0 of 10

Rationality evaluation is a closed loop: the evaluator trained on the benchmark's own rationality labels is used both as the GRPO reward and as the benchmark's rationality metric.

  1. self definitional [Sections 3.4.3, 4.3.1, and 4.3.2]
    "We measure this using a dedicated evaluation model (see Section 4.3.1), trained to mirror human judgment, which assigns a rationality score to each piece of textual evidence. ... The training data consists of the positive (rational) and negative (irrational) samples identified during our human annotation process."

    The benchmark's rationality metric is not independent: Section 3.4.3 defines it as the output of the evaluation model, and Section 4.3.1 states that this model is fine-tuned on the very rationality labels produced by the same human annotation pipeline that defines the benchmark ground truth. The same evaluator then supplies the GRPO reward (Section 4.3.2: 'Rationality Reward (R_rationality): The core signal from our evaluation model'), so the trained model is optimized to satisfy the fitted evaluator and is later assessed by that same fitted evaluator.

full rationale

The only clear derivation-chain circularity is the rationality evaluation loop. The paper trains a Qwen2.5-omni evaluator on positive/negative rationality samples from its own annotation pipeline, uses that evaluator as the GRPO reward signal, and then uses the same evaluator as the benchmark's rationality metric. Thus the headline claim of producing rationales that are 'convincing in their rationale' is evaluated by a model fitted to the same annotations that define the benchmark, with no external human-alignment validation. This is a genuine, quotable reduction, though it does not affect the spatial/temporal IoU or traceability metrics, which are defined independently. I did not find load-bearing self-citation, imported uniqueness theorems, or ansatz smuggling. The conclusion's statement that 'our extensive experiments show that this method significantly outperforms existing approaches' is unsupported by any experimental section in the manuscript, and the annotation quality is unvalidated (no inter-annotator agreement statistics), but these are correctness and completeness concerns rather than circularity. Overall score 6: one central evaluation component reduces by construction to its own training labels.

Assumptions & free parameters 2 free parameters · 5 assumptions · 1 invented entities

The benchmark ground truth rests on subjective human judgments and LLM proposals with no reported reliability statistics. The GRPO reward is defined by hand-chosen coefficients. No external or quantitative validation is provided for any component, and the promised 'evidence consistency' metric is undefined.

free parameters (2)
  • GRPO reward coefficients = +1 for format, +1/-1 for rationality, -5 for duplicate clues, +1 per correct traceability level, length in [-0.5, 0.5]
    Hand-specified in Section 4.3.2; no tuning procedure or ablation is reported, and they directly define the training objective.
  • Duplicate-clue IoU penalty threshold = 0.85
    Set in Section 4.3.2 to prevent reward hacking; no sensitivity analysis is provided.
assumptions (5)
  • domain assumption Human-LLM hybrid annotation with non-expert annotators yields accurate ground-truth evidence chains.
    Section 3.2 and Appendix A describe the process but provide no inter-annotator agreement, no expert validation, and no quantitative quality analysis, so the reliability of the benchmark's ground truth is assumed.
  • domain assumption Human annotators' binary rationality judgments are a reliable gold standard for evidence quality.
    Used to filter evidence and to train the rationality evaluator (Appendix A, Stage 2; Section 4.3.1); no calibration or external benchmark is presented.
  • domain assumption The rationality evaluator trained on METER annotations generalizes to unseen evidence and provides a valid reward.
    Section 4.3.1 fine-tunes Qwen2.5-omni on the dataset's own rational/irrational pairs; generalization is assumed without evaluation.
  • standard math Standard metrics (IoU, traceability accuracy) capture the quality of explainable forgery detection.
    Equations (1)-(3) in Section 3.4 define the metrics but do not demonstrate their validity as measures of explanation quality; this is a modeling assumption.
  • domain assumption Synthetic content generated by the listed open-source and commercial models is representative of real-world forgeries.
    Section 3.2 lists sources; no distribution analysis or external adversarial evaluation is given.
invented entities (1)
  • Evidence consistency metric
    purpose: Abstract and introduction promise richer interpretability metrics including 'evidence consistency', but the metric is never defined in the body.
    The term appears in the abstract and the contributions list but not in Section 3.4's metric definitions; as defined nowhere, it cannot be computed or validated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of METER: Multi-modal Evidence-based Thinking and Explainable Reasoning -- Algorithm and Benchmark." pith.science (2026). https://pith.science/paper/ZOG7AC6K

@misc{pith2026250716206,
  author       = {Pith},
  title        = {Pith review of: METER: Multi-modal Evidence-based Thinking and Explainable Reasoning -- Algorithm and Benchmark},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZOG7AC6K}},
  note         = {Machine review of arXiv:2507.16206}
}
read the original abstract

With the rapid advancement of generative AI, synthetic content across images, videos, and audio has become increasingly realistic, amplifying the risk of misinformation. Existing detection approaches predominantly focus on binary classification while lacking detailed and interpretable explanations of forgeries, which limits their applicability in safety-critical scenarios. Moreover, current methods often treat each modality separately, without a unified benchmark for cross-modal forgery detection and interpretation. To address these challenges, we introduce METER, a unified, multi-modal benchmark for interpretable forgery detection spanning images, videos, audio, and audio-visual content. Our dataset comprises four tracks, each requiring not only real-vs-fake classification but also evidence-chain-based explanations, including spatio-temporal localization, textual rationales, and forgery type tracing. Compared to prior benchmarks, METER offers broader modality coverage and richer interpretability metrics such as spatial/temporal IoU, multi-class tracing, and evidence consistency. We further propose a human-aligned, three-stage Chain-of-Thought (CoT) training strategy combining SFT, DPO, and a novel GRPO stage that integrates a human-aligned evaluator with CoT reasoning. We hope METER will serve as a standardized foundation for advancing generalizable and interpretable forgery detection in the era of generative media.

Figures

Figures reproduced from arXiv: 2507.16206 by the authors.

Figure 1
Figure 1. Overview of the METER Framework. Our framework takes a multi-modal input (e.g., an audio-visual clip) and performs a comprehensive, explainable forensic analysis. The central Omni-modal Analysis block shows how our model dissects the media across four tracks: audio-only, image-only (per-frame), video-only, and joint audio-visual. For each track, it generates a Chain-of-Thought evi￾dence trail, identifying specific f… view at source ↗
Figure 2
Figure 2. Distribution of the METER dataset. Left: Classification by modality, showing the breakdown of forgery types within Audio, Video, Image, and Audio-Visual tracks. Right: Classification by attack vector, distinguishing between Digital Attacks (e.g., Text-to-Image, Lip-Sync) and Physical Attacks (e.g., Replay, Print, Mask). Mix Annotation GPT4o Human Expert Gemini2.5 Pro Manual Refinement Rationality Assessment Bbox Cor… view at source ↗
Figure 3
Figure 3. The METER Annotation Pipeline. This process starts with hybrid initial annotation from leading LLMs (GPT-4o, Gemini 2.5 [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

29 extracted references · 27 canonical work pages

  1. [1]

    2noise. Chattts. https://github.com/2noise/ ChatTTS, 2024. Accessed: 11 Jul 2025. 4

    work page 2024

  2. [2]

    Z. Cai, S. Ghosh, A. P. Adatia, et al. Av-deepfake1m: A large-scale llm-driven audio-visual deepfake dataset. InPro- ceedings of the 32nd ACM International Conference on Mul- timedia, pages 7414–7423, 2024. 3

    work page 2024

  3. [3]

    Scal- ing rectified flow transformers for high-resolution image synthesis

    Patrick Esser, Suhail Kulal, Andreas Blattmann, et al. Scal- ing rectified flow transformers for high-resolution image synthesis. In Forty-first International Conference on Ma- chine Learning, 2024. 1, 4

    work page 2024

  4. [4]

    Y . Gao, D. Chang, B. Yu, et al. Fakereasoning: Towards gen- eralizable forgery detection and reasoning. arXiv preprint,

    work page

  5. [5]

    Y . He, B. Gan, S. Chen, et al. Forgerynet: A versatile bench- mark for comprehensive forgery analysis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4360–4369, 2021. 4

    work page 2021

  6. [6]

    Hasam Khalid, Shahroz Tariq, and Simon S. Woo. Fakeavceleb: A novel audio-video multimodal deepfake dataset, 2021. 3

    work page 2021

  7. [7]

    Y . Li, X. Liu, X. Wang, et al. Fakebench: Probing ex- plainable fake image detection via large multimodal models. arXiv preprint, 2024. 2, 3

    work page 2024

  8. [8]

    Asvspoof 2021: Towards spoofed and deepfake speech detection in the wild

    Xuechen Liu, Xin Wang, Md Sahidullah, Jose Patino, H´ector Delgado, Tomi Kinnunen, Massimiliano Todisco, Junichi Yamagishi, Nicholas Evans, Andreas Nautsch, and Kong Aik Lee. Asvspoof 2021: Towards spoofed and deepfake speech detection in the wild. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 31:2507–2522, 2023. 4

    work page 2021

Show all 29 references
  1. [9]

    Video generation models as world simulators,

    OpenAI. Video generation models as world simulators,

  2. [10]

    Rafailov, A

    R. Rafailov, A. Sharma, E. Mitchell, et al. Direct prefer- ence optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36:53728–53741, 2023. 4, 6

    2023

  3. [11]

    Z. Shao, P. Wang, Q. Zhu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint, 2024. 4

    2024

  4. [12]

    Rethinking the up-sampling op- erations in cnn-based generative network for generalizable deepfake detection

    Chuangchuang Tan, Yao Zhao, Shikui Wei, Guanghua Gu, Ping Liu, and Yunchao Wei. Rethinking the up-sampling op- erations in cnn-based generative network for generalizable deepfake detection. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition,...

    2024

  5. [13]

    Neural codec language models are zero - shot text to speech synthesizers

    Chengyi Wang, Sanyuan Chen, Yu Wu, Ziqiang Zhang, Long Zhou, Shujie Liu, Zhuo Chen, Yanqing Liu, Huaming Wang, Jinyu Li, Lei He, Sheng Zhao, and Furu Wei. Neural codec language models are zero - shot text to speech synthesizers. arXiv preprint arXiv:2301.02111, 2023. 1

    2023 arXiv

  6. [14]

    Z. Wang, J. Bao, W. Zhou, et al. Dire for diffusion-generated image detection. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pages 22445–22455,

  7. [15]

    Chain-of-thought prompting elicits reasoning in large language models, 2023

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models, 2023. 1, 3

    2023

  8. [16]

    Spot the fake: Large multimodal model-based synthetic image detection with artifact explanation, 2025

    Siwei Wen, Junyan Ye, Peilin Feng, Hengrui Kang, Zichen Wen, Yize Chen, Jiang Wu, Wenjun Wu, Conghui He, and Weijia Li. Spot the fake: Large multimodal model-based synthetic image detection with artifact explanation, 2025. 3

    2025

  9. [17]

    J. Xu, J. Chen, X. Song, et al. Identity-driven multimedia forgery detection via reference assistance. In Proceedings of the 32nd ACM International Conference on Multimedia , pages 3887–3896, 2024. 3

    2024

  10. [18]

    Qwen2.5-omni technical report, 2025

    Jin Xu, Zhifang Guo, Jinzheng He, Hangrui Hu, Ting He, Shuai Bai, Keqin Chen, Jialin Wang, Yang Fan, Kai Dang, Bin Zhang, Xiong Wang, Yunfei Chu, and Junyang Lin. Qwen2.5-omni technical report, 2025. 7

    2025

  11. [19]

    Z. Xu, X. Zhang, R. Li, et al. Fakeshield: Explainable im- age forgery detection and localization via multi-modal large language models. arXiv preprint, 2024. 1, 3

    2024

  12. [20]

    A sanity check for ai- generated image detection, 2025

    Shilin Yan, Ouxiang Li, Jiayin Cai, Yanbin Hao, Xiaolong Jiang, Yao Hu, and Weidi Xie. A sanity check for ai- generated image detection, 2025. 3, 4

    2025

  13. [21]

    Cogvideox: Text-to-video dif- fusion models with an expert transformer

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, et al. Cogvideox: Text-to-video dif- fusion models with an expert transformer. arXiv preprint arXiv:2408.06072, 2024. 4

    2024 arXiv

  14. [22]

    J. Ye, B. Zhou, Z. Huang, et al. Loki: A comprehensive synthetic data detection benchmark using large multimodal models. arXiv preprint, 2024. 3, 4

    2024

  15. [23]

    Facecot: A benchmark dataset for face anti-spoofing with chain-of- thought reasoning, 2025

    Honglu Zhang, Zhiqin Fang, Ningning Zhao, Saihui Hou, Long Ma, Renwang Pei, and Zhaofeng He. Facecot: A benchmark dataset for face anti-spoofing with chain-of- thought reasoning, 2025. 4

    2025

  16. [24]

    Zhang, H

    R. Zhang, H. Wang, M. Du, et al. Ummaformer: A uni- versal multimodal-adaptive transformer framework for tem- poral forgery localization. In Proceedings of the 31st ACM International Conference on Multimedia, pages 8749–8759,

  17. [25]

    Ivy-fake: A unified explainable framework and benchmark for image and video aigc detection, 2025

    Wayne Zhang, Changjiang Jiang, Zhonghao Zhang, Chenyang Si, Fengchang Yu, and Wei Peng. Ivy-fake: A unified explainable framework and benchmark for image and video aigc detection, 2025. 2, 3

    2025

  18. [26]

    Genimage: A million-scale benchmark for de- tecting ai-generated image, 2023

    Mingjian Zhu, Hanting Chen, Qiangyu Yan, Xudong Huang, Guanyu Lin, Wei Li, Zhijun Tu, Hailin Hu, Jie Hu, and Yunhe Wang. Genimage: A million-scale benchmark for de- tecting ai-generated image, 2023. 3 A. Detailed Annotation Protocol Our annotation protocol was meticulously des...

    2023

  19. [27]

    Their primary task was to meticulously adjust the BBox to ensure it tightly and accurately encompassed the region corresponding to the textual clue

    BBox Correction: Annotators were presented with the source media, the LLM’s textual explanation, and its proposed BBox (spatial for images, temporal for audio/video). Their primary task was to meticulously adjust the BBox to ensure it tightly and accurately encompassed the reg...

  20. [28]

    Rationality Assessment: In this critical stage, annota- tors were shown the source media, the textual explana- tion, and the now-corrected BBox. They provided a bi- nary judgment: is this clue a ”rational” and convincing piece of evidence for forgery, or is it ”irrational” (e....

  21. [29]

    A final reviewer was tasked with comparing these com- plete evidence sets and selecting the single ”best” one, which then became the ground-truth ”Evidence Chain”

    Preference Selection: For each media file, we col- lected multiple sets of rational evidence (e.g., one set from GPT-4o, one from Gemini 2.5 Pro, and some- times one from an independent human annotator). A final reviewer was tasked with comparing these com- plete evidence sets...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.