pith. sign in

arxiv: 2603.01993 · v2 · pith:MLIGWK43new · submitted 2026-03-02 · 💻 cs.CV

Cultivating Forensic Reasoning for Generalizable Multimodal Manipulation Detection

Yuchen Zhang , Yaxiong Wang , Kecheng Han , Yujiao Wu , Lianwei Wu , Li Zhu , Zhedong Zheng This is my paper

Pith reviewed 2026-05-21 11:10 UTC · model grok-4.3

classification 💻 cs.CV
keywords multimodal manipulation detectionforensic reasoninggeneralizationdeepfake detectionreasoning curriculummanipulation groundingcurriculum learningreinforcement learning
0
0 comments X

The pith

Explicit forensic reasoning in a staged curriculum lets manipulation detectors generalize to unseen patterns instead of overfitting to known artifacts.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper aims to establish that generalizable multimodal manipulation detection requires incorporating explicit forensic reasoning rather than classifying a limited set of manipulation types under result-oriented supervision. If this holds, detectors would avoid overfitting to superficial artifacts and handle novel techniques generated by advancing AI models. The authors introduce REFORM, a framework that follows a three-stage curriculum: first inducing forensic rationales, then aligning those rationales with final judgments, and finally refining logical consistency through reinforcement learning. To enable this shift, they also release the ROM dataset containing rich reasoning annotations alongside the media examples.

Core claim

REFORM is a reasoning-driven framework that shifts learning from outcome fitting to process modeling for generalizable multimodal manipulation detection. It adopts a three-stage curriculum that first induces forensic rationales, then aligns reasoning with final judgments, and finally refines logical consistency via reinforcement learning. Supported by the ROM dataset with rich reasoning annotations, this approach enables superior generalization to unseen manipulation patterns compared with standard classification methods.

What carries the argument

The three-stage curriculum that induces forensic rationales, aligns them with judgments, and refines logical consistency via reinforcement learning.

If this is right

  • Detection systems become capable of handling manipulation patterns never encountered in training data.
  • Decisions gain interpretability because the model produces explicit forensic rationales rather than opaque classifications.
  • Logical consistency between reasoning steps and final judgments improves through the reinforcement learning refinement stage.
  • Datasets that include reasoning annotations become necessary resources for training generalizable detectors.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same curriculum structure could be adapted to improve generalization in related tasks such as generated-text or generated-audio detection.
  • Models trained this way might require fewer retraining cycles when new generative methods appear in the wild.
  • The explicit rationales could support human-in-the-loop verification workflows in forensic or moderation settings.

Load-bearing premise

Inducing explicit forensic rationales and aligning them with judgments via a three-stage curriculum will produce better generalization to unseen manipulation patterns than standard result-oriented supervision.

What would settle it

A direct comparison where both a standard result-oriented classifier and the three-stage reasoning curriculum are trained on identical data and then evaluated on a held-out test set containing only novel manipulation types absent from training; equal or lower performance by the reasoning model would falsify the claim.

Figures

Figures reproduced from arXiv: 2603.01993 by Kecheng Han, Lianwei Wu, Li Zhu, Yaxiong Wang, Yuchen Zhang, Yujiao Wu, Zhedong Zheng.

Figure 1
Figure 1. Figure 1: Comparison between learning paradigms. (a) [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the ROM dataset. Left: Representative samples spanning 9 manipulated and 1 real categories, ranging from face-centric edits to scene-level synthesis, each accompanied by a detailed reasoning annotation. Right: Statistical distribution showing the diversity of manipulation types and the coverage of news media domains. Qwen-VL (Bai et al., 2025) and InternVL (Wang et al., 2025b), have revolutioni… view at source ↗
Figure 3
Figure 3. Figure 3: Probability Density of Token Count for An [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Overview of the REFORM framework and its three-stage training curriculum. (a) The primary pipeline [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: The user interface of the human evaluation study where each participant is given pairs of news images and [PITH_FULL_IMAGE:figures/full_fig_p013_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Human evaluation statistics on multimodal fake news identification. (a) Per-class accuracy across four [PITH_FULL_IMAGE:figures/full_fig_p014_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Overview of the generalization evaluation datasets. The Unseen News Dataset category includes the [PITH_FULL_IMAGE:figures/full_fig_p016_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Overview of the ROM dataset construction pipeline and statistics. (a) Distribution of real vs. fake data [PITH_FULL_IMAGE:figures/full_fig_p020_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Qualitative results of REFORM on the ROM test set. The figure displays representative samples across [PITH_FULL_IMAGE:figures/full_fig_p024_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Zero-shot generalization results on unseen benchmarks. We visualize REFORM’s performance on [PITH_FULL_IMAGE:figures/full_fig_p025_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: User interface for the human evaluation of reasoning quality. Participants review the news sample [PITH_FULL_IMAGE:figures/full_fig_p026_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Distribution of human ratings on reasoning [PITH_FULL_IMAGE:figures/full_fig_p026_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Qualitative Failure Analysis. We analyze limitations across four cognitive dimensions: (a) Logical [PITH_FULL_IMAGE:figures/full_fig_p027_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: The prompt template for REFORM. The prompt strictly concatenates the system instruction, caption, [PITH_FULL_IMAGE:figures/full_fig_p028_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: The prompt template used to generate reasoning chains. The [PITH_FULL_IMAGE:figures/full_fig_p029_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: The prompt template used for General-purpose Model. Note the change in coordinate format to [PITH_FULL_IMAGE:figures/full_fig_p029_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: The MMD-Agent prompt workflow. The agent sequentially performs Fact-Checking (utilizing external [PITH_FULL_IMAGE:figures/full_fig_p030_17.png] view at source ↗
read the original abstract

Recent advances in generative AI have significantly enhanced the realism of multimodal media manipulation, thereby posing substantial challenges to manipulation detection. Existing manipulation detection and grounding approaches predominantly focus on manipulation type classification under result-oriented supervision, which not only lacks interpretability but also tends to overfit superficial artifacts. In this paper, we argue that generalizable detection requires incorporating explicit forensic reasoning, rather than merely classifying a limited set of manipulation types, which fails to generalize to unseen manipulation patterns. To this end, we propose REFORM, a reasoning-driven framework that shifts learning from outcome fitting to process modeling. REFORM adopts a three-stage curriculum that first induces forensic rationales, then aligns reasoning with final judgments, and finally refines logical consistency via reinforcement learning. To support this paradigm, we introduce ROM, a large-scale dataset with rich reasoning annotations. Extensive experiments show that REFORM establishes new state-of-the-art performance with superior generalization, achieving 81.52% ACC on ROM, 76.65% ACC on DGM4, and 74.9 F1 on MMFakeBench.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 2 minor

Summary. The paper proposes REFORM, a reasoning-driven framework for multimodal manipulation detection that shifts from result-oriented supervision to explicit forensic reasoning via a three-stage curriculum (rationale induction, alignment with judgments, and RL consistency refinement). It introduces the ROM dataset with rich reasoning annotations and reports new state-of-the-art results with claimed superior generalization: 81.52% ACC on ROM, 76.65% ACC on DGM4, and 74.9 F1 on MMFakeBench.

Significance. If the central claim holds after addressing controls, the work would advance the field by showing that modeling forensic reasoning processes can yield more interpretable and generalizable detection than standard classification, particularly for unseen manipulation patterns in generative AI media.

major comments (3)
  1. [§4 (Experiments)] §4 (Experiments) and Table 2: The generalization improvements on DGM4 and MMFakeBench are reported after training on ROM, but no matched baseline is shown that applies standard result-oriented supervision (e.g., cross-entropy loss) to the identical ROM data, model capacity, and annotations. This leaves open whether gains stem from the three-stage curriculum or from richer supervision signals in ROM.
  2. [§3.2 (ROM Dataset)] §3.2 (ROM Dataset Construction): The ROM dataset is introduced by the authors and used for both training and one of the primary benchmarks (81.52% ACC). Additional details on train/test splits, annotation process, and safeguards against overlap with DGM4 or MMFakeBench are required to substantiate the generalization claim.
  3. [§4.3 (Ablation Studies)] §4.3 (Ablation Studies): The ablations on the three stages do not include a control that removes the reasoning components while retaining the full ROM annotations and scale; without this, the necessity of the curriculum for the reported cross-dataset gains cannot be isolated from dataset effects.
minor comments (2)
  1. [Abstract] The abstract and §1 would benefit from explicitly naming the prior SOTA baselines and their scores on DGM4 and MMFakeBench for direct comparison.
  2. [§3.3] Notation for the RL consistency stage (e.g., reward formulation) could be clarified with a short equation or pseudocode in §3.3.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive and detailed comments. We address each major point below and will revise the manuscript to strengthen the experimental controls and dataset documentation.

read point-by-point responses

  1. Referee: §4 (Experiments) and Table 2: The generalization improvements on DGM4 and MMFakeBench are reported after training on ROM, but no matched baseline is shown that applies standard result-oriented supervision (e.g., cross-entropy loss) to the identical ROM data, model capacity, and annotations. This leaves open whether gains stem from the three-stage curriculum or from richer supervision signals in ROM.

    Authors: We agree this control is important for isolating the curriculum's contribution. Our existing comparisons are against prior methods trained on their original datasets. In the revision we will add a matched baseline that applies standard cross-entropy supervision to the identical ROM data and model backbone, then report its performance on DGM4 and MMFakeBench to clarify the source of the observed generalization gains. revision: yes

  2. Referee: §3.2 (ROM Dataset Construction): The ROM dataset is introduced by the authors and used for both training and one of the primary benchmarks (81.52% ACC). Additional details on train/test splits, annotation process, and safeguards against overlap with DGM4 or MMFakeBench are required to substantiate the generalization claim.

    Authors: We will expand §3.2 with the requested details: an explicit 80/20 train/test split with media-level deduplication, a step-by-step description of the multi-expert annotation protocol used to generate the reasoning annotations, and the verification procedures (including cross-dataset similarity checks) that confirm no overlap with DGM4 or MMFakeBench. The revised manuscript will also state that the dataset and splits will be released publicly. revision: yes

  3. Referee: §4.3 (Ablation Studies): The ablations on the three stages do not include a control that removes the reasoning components while retaining the full ROM annotations and scale; without this, the necessity of the curriculum for the reported cross-dataset gains cannot be isolated from dataset effects.

    Authors: We acknowledge the value of this additional control. The current ablations show incremental gains from adding each curriculum stage. In the revision we will insert a new row in the ablation table that trains the same model on the full ROM scale using only the final judgment labels under standard supervision (no rationale annotations or curriculum stages) and evaluates cross-dataset performance, thereby separating curriculum effects from dataset richness. revision: yes

Circularity Check

0 steps flagged

No significant circularity; derivation remains self-contained

full rationale

The paper introduces REFORM as a three-stage curriculum (rationale induction, alignment, RL refinement) and the ROM dataset to enable process-oriented supervision rather than result-oriented classification. Performance is reported on the new ROM set but also on independent external benchmarks DGM4 and MMFakeBench. No equations, definitions, or steps are shown where the claimed generalization, SOTA numbers, or forensic reasoning outputs reduce by construction to the training inputs, fitted parameters, or self-referential citations. The central argument rests on empirical comparison rather than tautological renaming or load-bearing self-citation chains, making the derivation self-contained against external data.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 2 invented entities

The central claim rests on the assumption that explicit reasoning annotations in ROM are both available and sufficient to train generalizable forensic logic; no free parameters or invented physical entities are described.

axioms (1)
  • domain assumption Forensic reasoning can be elicited and aligned with final judgments through staged supervision and reinforcement learning.
    Invoked in the description of the three-stage curriculum.
invented entities (2)
  • REFORM framework no independent evidence
    purpose: Shifts learning from outcome fitting to process modeling of forensic reasoning
    Newly proposed architecture and training procedure.
  • ROM dataset no independent evidence
    purpose: Provides rich reasoning annotations to support the new training paradigm
    Introduced to enable the proposed curriculum.

pith-pipeline@v0.9.0 · 5734 in / 1326 out tokens · 34383 ms · 2026-05-21T11:10:56.323223+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

12 extracted references · 12 canonical work pages · 5 internal anchors

  1. [1]

    Dif- fusionface: Towards a comprehensive dataset for diffusion-based face forgery analysis

    Diffusionface: Towards a comprehensive dataset for diffusion-based face forgery analysis. arXiv:2403.18471. Harry Cheng, Yangyang Guo, Tianyi Wang, Liqiang Nie, and Mohan Kankanhalli. 2024. Diffusion facial forgery detection. InACM MM. Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V . Le, Sergey Levine, and Yi ...

    work page arXiv 2024

  2. [2]

    Scaling rectified flow transformers for high- resolution image synthesis. InICML. Liu Fuxiao, Wang Yinghan, Wang Tianlu, and Ordonez Vicente. 2021. Visual news: Benchmark and chal- lenges in news image captioning. InEMNLP. Asso- ciation for Computational Linguistics. Jing Huang, Zhiya Tan, Shutao Gong, Fanwei Zeng, Joey Tianyi Zhou, Changtao Miao, Huazhe ...

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  3. [3]

    Blip-2: Bootstrapping language-image pre- training with frozen image encoders and large lan- guage models. InICML. Zongxia Li, Wenhao Yu, Chengsong Huang, Rui Liu, Zhenwen Liang, Fuxiao Liu, Jingxi Che, Dian Yu, Jordan Boyd-Graber, Haitao Mi, and Dong Yu. 2025b. Self-rewarding vision-language model via reasoning decomposition.arXiv preprint arXiv:2508.196...

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  4. [4]

    DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models

    Fake news on social media: the impact on soci- ety.Information Systems Frontiers, 26(2):443–458. Yikang Pan, Liangming Pan, Wenhu Chen, Preslav Nakov, Min-Yen Kan, and William Wang. 2023. On the risk of misinformation pollution with large lan- guage models. InEMNLP Findings. Association for Computational Linguistics. Chan Young Park, Julia Mendelsohn, Anj...

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  5. [5]

    Cosmos: Catching out-of-context image mis- use using self-supervised learning. InAAAI. StabilityAI. 2023. Introducing stable diffusion 3.5. Ac- cessed: 2025-08-24. Yixuan Su, Tian Lan, Huayang Li, Jialu Xu, Yan Wang, and Deng Cai. 2023. PandaGPT: One model to instruction-follow them all. InProceedings of the 1st Workshop on Taming Large Language Models. A...

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  6. [6]

    Transform and tell: Entity-aware news image captioning. InCVPR. JunXi Wang, Yaxiong Wang, Lechao Cheng, and Zhun Zhong. 2025a. FakeSV-VLM: Taming VLM for de- tecting fake short-video news via progressive mixture- of-experts adapter. InEMNLP Findings. Association for Computational Linguistics. Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xing...

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  7. [7]

    In EMNLP

    TRUST-VL: An explainable news assistant for general multimodal misinformation detection. In EMNLP. Association for Computational Linguistics. Jiabo Ye, Haiyang Xu, Haowei Liu, Anwen Hu, Ming Yan, Qi Qian, Ji Zhang, Fei Huang, and Jingren Zhou

    work page

  8. [8]

    mplug-owl3: Towards long image-sequence understanding in multi-modal large language models. InICLR. Ruohong Zhang, Bowen Zhang, Yanghao Li, Haotian Zhang, Zhiqing Sun, Zhe Gan, Yinfei Yang, Ruom- ing Pang, and Yiming Yang. 2025a. Improve vision language model chain-of-thought reasoning. InACL. Wenhua Zhang, Weicheng Li, Xuanrong Rao, Lixin Zou, Xiangyang ...

    work page arXiv

  9. [9]

    Bilateral reference for high-resolution dichotomous image segmentation.arXiv preprint arXiv:2401.03407, 2024

    Bilateral reference for high-resolution di- chotomous image segmentation.arXiv preprint arXiv:2401.03407. Tong Zheng, Hongming Zhang, Wenhao Yu, Xiaoyang Wang, Runpeng Dai, Rui Liu, Huiwen Bao, Cheng- song Huang, Heng Huang, and Dong Yu. 2025. Parallel-r1: Towards parallel thinking via reinforce- ment learning.arXiv preprint arXiv:2509.07980. Deyao Zhu, j...

    work page arXiv 2025

  10. [10]

    Diffusion Artifact Detection: REFORM ex- hibits exceptional robustness against diffusion arti- facts, achieving near-perfect accuracy on DiffFace- DDIM (98.12) and DiFF-Image2Image (98.15)

    work page

  11. [11]

    Prometheus

    Generalization from News to General Vision: Although trained on news-oriented manipulations, REFORM generalizes effectively to these specific high-fidelity forgery benchmarks. On the DiFF benchmark, it dominates the FaceEdit and Im- age2Image categories and achieves the highest overall average. This suggests that REFORM has successfully learned intrinsic ...

    work page arXiv 2025

  12. [12]

    Acceptable

    REFORM demonstrates a high level of inter- pretability across all news domains. On average, 85.8%of the reasoning chains were rated as ef- fective (summing “Acceptable” and “Highly Con- vincing”). Conversely, the “Not Convincing” rate remains extremely low (avg. 5.5%), suggesting that REFORM rarely produces hallucinations or illogi- cal inferences that wo...

    work page 2015