REVIEW 5 major objections 5 minor 31 references
This paper claims that a simple training recipe—balanced real/tampered minibatches, late injection of a small companion domain, and a low constant learning rate—makes pixel-level tampering detectors generalize to unseen VLM image generators
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-01 15:36 UTC pith:QVNWBVWH
load-bearing objection Selection bias in the hyperparameter search makes the headline OOD gains unverified, but the recipe is simple, clearly dissected, and worth a referee. the 5 major comments →
Simple Domain Generalization for Strong Pixel-Level Image Tampering Detection in Modern VLMs
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central claim is that a domain-generalized tampering detector can be obtained by training a segmentation-based detector on one large open-source domain (Qwen-Image, 70K samples) plus a small companion domain (Gemini-2.5, 3K samples), with a balanced minibatch containing equal numbers of real and tampered images, injecting the small domain only after base convergence, and using a low constant learning rate (2e-5). The paper reports that this recipe consistently beats the prior state-of-the-art PIXAR on all four held-out generators (GPT-Image-2.0, Gemini-3.1, FLUX.2, Seedream 4.5), with relative gains of 26.1% in average gIoU and 26.8% in average cIoU at the 7B scale, and similar advantage
What carries the argument
The load-bearing object is the balanced minibatch sampler: each batch is forced to contain a fixed 1:1 ratio of real and tampered images, so every optimization step sees contrasting pixel-level supervision and avoids collapse to either class. Working with it are the late-injection schedule, which trains on a 70K Qwen-Image base until convergence before adding 3K Gemini-2.5 samples, and the constant low learning-rate schedule (2e-5), which keeps the optimization trajectory flat. The paper's losses are the standard PIXAR objective: BCE plus DICE for the pixel map, sigmoid cross-entropy for multi-label semantics, cross-entropy for the CLS real/tampered head, and language modeling for the explan
Load-bearing premise
The four 'unseen' generators were used to decide which training source and how much base data to use, so the reported out-of-distribution gains may depend on having peeked at the test distribution.
What would settle it
Retrain the detector with the same recipe but choose the companion domain and data size using only Qwen-Image validation (or generators different from the four), then measure average gIoU and cIoU on GPT-Image-2.0, Gemini-3.1, FLUX.2, and Seedream 4.5. If the margin over PIXAR falls below roughly 10% relative, the headline gains largely reflect selection on the test set rather than domain-generalized learning.
If this is right
- A detector that generalizes to unseen generators can be trained with about one-fifth of the labeled data, reducing the cost of keeping pace with new image models.
- Balanced real/tampered minibatching prevents training collapse and yields flatter loss-landscape minima, linking optimization stability to out-of-distribution transfer.
- Small labeled samples from an emerging generator are most useful when injected late, after base representations are stable, rather than mixed in from the start.
- Adding more same-source data beyond a saturation point (70K in this setup) does not improve cross-generator localization and can dilute the companion domain's contribution.
- The recipe transfers across model scales: the gains hold at both 7B and 13B LoRA-fine-tuned detectors.
Where Pith is reading between the lines
- Selection of the companion source (Gemini-2.5) and the base data size (70K) was guided by accuracy on the same generators later branded 'held out', so part of the reported 26% margin could evaporate on a truly unseen generator; a stronger evaluation would choose sources without touching the OOD test set.
- The paper's own source-composition table suggests source choice dominates: switching the base source from Qwen-Image to another generator drops out-of-distribution accuracy to single digits for some targets. This hints that a simple source-ranking rule—pick the open-source generator with the highest worst-case OOD accuracy—may be the real recipe, with minibatch balancing as a stabilizer.
- If the flatter-minimum explanation is right, the same late-injection/low-learning-rate schedule should improve OOD transfer for other pixel-level tasks with small emerging domains, such as deepfake video segmentation or medical image editing detection.
- A testable extension: since Gemini-2.5 was chosen because Qwen-Image generalizes worst to it, one could select companion sources dynamically via leave-one-generator-out evaluation, avoiding future selection leakage.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes PIXAR-DG, a training framework for pixel-level VLM tampering detection. It combines three ingredients: balanced real/tampered minibatch sampling, late injection of a small companion domain (Gemini-2.5) after base training on Qwen-Image, and a low constant learning rate. Using only 73K tampered images (19.2% of the PIXAR training scale), the authors report large gains over PIXAR on four generators described as out-of-distribution (GPT-Image-2.0, Gemini-3.1, FLUX.2, Seedream 4.5), with 26.1%/26.8% relative gIoU/cIoU improvements claimed in the abstract. The paper also presents source-selection experiments (Tables 3-5) and ablations (Tables 6-7). The central claim is that this simple recipe yields detector that generalize robustly to unseen VLM generators.
Significance. If the reported OOD gains were clean, this would be a practically valuable contribution: the method is simple, training-data-efficient, uses an open-source architecture, and the code is provided. The paper also gives a detailed empirical study of source selection and data scaling. However, the central empirical claim is undermined by the evaluation protocol: the same four generators used for model selection are later treated as 'held-out' test generators. As a result, the magnitude of the reported gains and the conclusion of true zero-shot generalization are not supported by the experiments as presented. A proper nested validation protocol is required before the claims can be assessed.
major comments (5)
- [abstract and Section 4.2] The 'held-out' OOD generators (GPT-Image-2.0, Gemini-3.1, FLUX.2, Seedream 4.5) are used to select the base source (Qwen-Image, Table 3), the companion source (Gemini-2.5, Table 4), and the base dataset size N=70K (Table 5). The final evaluation in Tables 1 and 2 is then reported on the same generators. This is a classic test-set-based selection loop: the training recipe is tuned to maximize performance on the distributions that are subsequently used to measure generalization. The paper must provide a nested evaluation, e.g., leave-one-generator-out selection, where the final test generator is never used in any selection decision, or otherwise ensure that the reported OOD numbers are truly independent of model selection.
- [Section 4.1 and 4.3] The abstract states relative gains of 26.1% and 26.8% in average gIoU and cIoU. These numbers correspond to the 13B variant in Table 1 (0.222 vs 0.176 for gIoU; 0.232 vs 0.183 for cIoU). The 7B variant yields 21.4% and 21.1% relative gains (0.193 vs 0.159; 0.201 vs 0.166). The abstract is therefore misleading in presenting only the more favorable 13B comparison without specifying scale. Please report both scales explicitly and avoid cherry-picking the larger relative gain.
- [Tables 1 and 2] Section 4.1 modifies the PIXAR test protocol by excluding GPT-Image-1.5 and Gemini-3 and incorporating GPT-Image-2.0 and Gemini-3.1. Yet GPT-Image-2.0 and Gemini-3.1 appear in the source-selection Tables 3 and 4, where models trained on these generators are evaluated on the same four 'OOD' generators. The statement that these four are 'unseen during training' is therefore only true of the final detector, not of the model-selection process. The selection experiments should be performed on a validation set that is disjoint from the final OOD test set.
- [Table 6] The paper states that results are reported over three random seeds, but no error bars, standard deviations, or significance tests are provided in the main results. Given the size of the reported gains this may not change the conclusions, but with the selection bias already present, variance information is essential to judge whether the improvements are stable. Please report mean±std for the main tables and, ideally, per-seed results or a statistical test.
- [Section 3.2 and 3.3] The same-training-data ablation in Table 6 shows that the proposed recipe outperforms baselines when trained on the same Qwen-Image+Gemini-2.5 data. While this is a useful control, it does not address the selection-bias problem: the data composition itself (including the choice of Gemini-2.5 and N=70K) was selected using the final test generators. The ablation only demonstrates that the components help under a fixed, but selected, data configuration.
minor comments (5)
- [Section 3.2/4.1] For balanced minibatch sampling, the paper sets the real:tampered ratio to ρ=0.5 but does not specify the size or composition of the real-image pool. Are the real images the original source images paired with the tampered images in PIXAR? Please state the real-image pool size and how it is sampled, to ensure reproducibility.
- [Section 3.3] The late-injection timing Tinj is described as 'after stable convergence' and 'after the base model converges', but no concrete value (e.g., after 3 of 5 epochs, or after step X) is given. Please report the actual injection step used in the experiments.
- [Abstract] The phrase 'completely new task and problem in the community' is an overstatement. The paper proposes a training framework for an existing task (pixel-level tampering detection) and is built on the PIXAR benchmark and detector. Please rephrase.
- [Table 3] The 'Worst' column header is not fully defined in the table; it appears to report the minimum OOD accuracy per row. Please clarify the definition and ensure the column is labeled consistently.
- [Throughout] The generator name is written inconsistently (e.g., 'Flux.2' vs 'FLUX.2', 'Seedream-4.5' vs 'Seedream 4.5', 'GPT-Images-2.0' vs 'GPT-Image-2.0'). Please standardize names.
Circularity Check
The claimed 26.1%/26.8% OOD gains are not clean predictions: Section 4.3 selects the base source, companion source, and N=70K on exactly the four 'held-out' generators, so the reported gains partially reduce to fitted model-selection choices.
specific steps
-
fitted input called prediction
[Section 4.1 (Training and Test Dataset Details) and Section 4.3 (Why Qwen and Gemini-2.5 as Training Data Sources?), Tables 3-5]
"Gemini-2.5 is selected as the new domain source because it provides the largest OOD improvement among all candidate companion generators (see Section 4.3). Under this protocol, Qwen-Image and Gemini-2.5 act as the in-domain (ID) generators, while the remaining four models, GPT-Image-2.0, Gemini-3.1, FLUX.2, and Seedream 4.5, are held out and treated as out-of-distribution (OOD) generators unseen during training."
The four 'OOD' generators are not external to model selection: Tables 3-5 use per-generator accuracy on exactly GPT-Image-2.0, Gemini-3.1, FLUX.2, and Seedream 4.5 to choose the base source (Qwen-Image), the companion source (Gemini-2.5), and the base size (N=70K). Tables 1-2 then report OOD gains on these same four generators as though they were unseen predictions. The headline 26.1%/26.8% relative improvements are therefore partly a selection artifact: the training recipe was tuned to maximize performance on the evaluation distribution. No nested, genuinely held-out generator is ever tested, so the 'zero-shot transferability to unseen generators' claim is not independently supported.
-
fitted input called prediction
[Section 4.3, Table 5 (Influence of base-source training data size)]
"As shown in Table 4, Qwen-Image + Gemini-2.5 achieves the highest Avg. (OOD) of 66.3%, outperforming all four alternative companion choices by a clear margin ... It is evident that increasing the Qwen-Image subset from 30K to 70K improves both average accuracy and Pixel F1 ... We therefore pair N=70K Qwen-Image samples with 3K Gemini-2.5 tampered samples in all main experiments."
The dataset size N=70K is selected by maximizing average accuracy and Pixel F1 on the four OOD generators (Table 5 columns GPT-Image-2.0, Gemini-3.1, FLUX.2, Seedream 4.5). Those same generators are then used to report the main OOD results. Thus a hyperparameter choice tuned on the evaluation set is renamed as evidence of generalization to unseen domains. The same-data ablation in Table 6 keeps this selected recipe fixed, so it cannot break the selection loop.
full rationale
The paper is an empirical recipe paper: no equation in Section 3 derives OOD accuracy from the losses, and the losses themselves are standard BCE/DICE/CE terms, so the classic definitional-circularity patterns largely do not apply. The load-bearing circularity is the model-selection loop. Section 4.1 calls GPT-Image-2.0, Gemini-3.1, FLUX.2, and Seedream 4.5 'held out and treated as OOD generators unseen during training,' yet Section 4.3 uses exactly these generators to choose the base source (Qwen-Image, Table 3), the companion source (Gemini-2.5, Table 4, by Avg. OOD), and the base size (N=70K, Table 5). The abstract's 26.1%/26.8% gains are computed on these same four generators, so the reported 'prediction' is partly fitted to the evaluation set rather than an independent out-of-distribution forecast. The same-data ablation (Table 6) is informative for method comparisons but does not break the loop because the recipe was already selected using the target generators. Self-citation of PIXAR is present — PIXAR is the authors' own prior benchmark/backbone — but it is not itself load-bearing for the derivation: PIXAR is used as a baseline and architectural starting point, and the proposed losses and sampling scheme are stated independently. There is no imported uniqueness theorem or ansatz-via-citation chain. Accordingly, the central claim is partially circular (score 6) rather than fully forced by definition.
Axiom & Free-Parameter Ledger
free parameters (5)
- ρ (real:tampered ratio) =
0.5
- Base Qwen-Image subset size N =
70K
- Companion source Gemini-2.5 (3K samples) =
3K Gemini-2.5 tampered images
- Learning rate =
2e-5 with constant scheduler
- Late-injection timing T_inj and mixture weight α =
Not specified
axioms (4)
- domain assumption PIXAR benchmark's per-pixel masks thresholded at τ=0.05 are accurate ground truth for tampering localization.
- domain assumption Qwen-Image and Gemini-2.5 are representative source domains whose artifacts cover the manipulation cues needed for OOD generalization.
- domain assumption LoRA fine-tuning with frozen CLIP/SAM towers preserves transferable features.
- ad hoc to paper The loss weights λ from PIXAR remain appropriate under the new sampling schedule.
Cite this review
Pith. "Pith review of Simple Domain Generalization for Strong Pixel-Level Image Tampering Detection in Modern VLMs." pith.science (2026). https://pith.science/paper/QVNWBVWH
@misc{pith2026260718230,
author = {Pith},
title = {Pith review of: Simple Domain Generalization for Strong Pixel-Level Image Tampering Detection in Modern VLMs},
year = {2026},
howpublished = {\url{https://pith.science/paper/QVNWBVWH}},
note = {Machine review of arXiv:2607.18230}
}
read the original abstract
Modern vision-language models (VLMs) have significantly improved image generation and editing capabilities, making pixel-level image tampering detection increasingly important yet challenging under cross-model and out-of-distribution shifts. This work studies domain generalization for pixel-level image tampering detection in modern VLMs like ChatGPT, Gemini, Qwen-Image, etc., aiming to learn tampering localization models that remain robust across diverse VLM-generated manipulation distributions. We propose a simple yet effective domain-generalized training framework built on two practical strategies. First, we introduce a balanced minibatch sampling scheme that strategically samples tampered and real images in each minibatch, preventing biased optimization toward either manipulated artifacts or clean-image priors and avoiding training collapse, ensuring that each optimization step receives proper sampled gradient signals. Second, we adopt a simple late-injection strategy, where the detector is first trained on large-scale base data until stable convergence, and then exposed to a small amount of newly selected supporting data from emerging VLM distributions, improving adaptability without overfitting to limited new domains. Together, these components provide a simple yet strong recipe for improving pixel-level tampering localization and OOD robustness across modern VLMs. Despite the conceptual simplicity, our framework outperforms the prior state-of-the-art PIXAR by a large margin of 26.1% and 26.8% relative improvement in average gIoU and cIoU, respectively, across OOD VLMs of GPT-Images-2.0, Gemini-3.1, FLUX.2, and Seedream 4.5. Our code is available at https://github.com/VILA-Lab/PIXAR-DG
Figures
Reference graph
Works this paper leans on
-
[1]
End-to-end reconstruction-classification learning for face forgery detection
Junyi Cao, Chao Ma, Taiping Yao, Shen Chen, Shouhong Ding, and Xiaokang Yang. End-to-end reconstruction-classification learning for face forgery detection. InCVPR, pages 4113–4122, 2022
2022
-
[2]
Domain generalization by mutual- information regularization with pre-trained models
Junbum Cha, Kyungjae Lee, Sungrae Park, and Sanghyuk Chun. Domain generalization by mutual- information regularization with pre-trained models. InEuropean conference on computer vision, pages 440–457. Springer, 2022
2022
-
[3]
You-Ming Chang, Chen Yeh, Wei-Chen Chiu, and Ning Yu. Antifakeprompt: Prompt-tuned vision-language models are fake image detectors.arXiv preprint arXiv:2310.17419, 2023
Pith/arXiv arXiv 2023
-
[4]
Drct: Diffusion reconstruction contrastive training towards universal detection of diffusion generated images
Baoying Chen, Jishen Zeng, Jianquan Yang, and Rui Yang. Drct: Diffusion reconstruction contrastive training towards universal detection of diffusion generated images. InForty-first International Conference on Machine Learning, 2024
2024
-
[5]
Gheorghe Comanici, Eric Bieber, Mike Schaekermann, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities.arXiv preprint arXiv:2507.06261, 2025
Pith/arXiv arXiv 2025
-
[6]
Gemini 3
Google DeepMind. Gemini 3. 2025. Published November 2025
2025
-
[7]
Gemini 3.1 Flash Image model card
Google DeepMind. Gemini 3.1 Flash Image model card. https://deepmind.google/models/ model-cards/gemini-3-1-flash-image/, 2026. Accessed: 2026-06-03
2026
-
[8]
In search of lost domain generalization.arXiv preprint arXiv:2007.01434, 2020
Ishaan Gulrajani and David Lopez-Paz. In search of lost domain generalization.arXiv preprint arXiv:2007.01434, 2020
Pith/arXiv arXiv 2007
-
[9]
Lora: Low-rank adaptation of large language models.Iclr, 1(2):3, 2022
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Liang Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models.Iclr, 1(2):3, 2022
2022
-
[10]
Sida: Social media image deepfake detection, localization and explanation with large multimodal model
Zhenglin Huang, Jinwei Hu, Xiangtai Li, Yiwei He, Xingyu Zhao, Bei Peng, Baoyuan Wu, Xiaowei Huang, and Guangliang Cheng. Sida: Social media image deepfake detection, localization and explanation with large multimodal model. InCVPR, pages 28831–28841, 2025
2025
-
[11]
Simple data balancing achieves competitive worst-group-accuracy
Badr Youbi Idrissi, Martin Arjovsky, Mohammad Pezeshki, and David Lopez-Paz. Simple data balancing achieves competitive worst-group-accuracy. InConference on Causal Learning and Reasoning, pages 336–351. PMLR, 2022
2022
-
[12]
FLUX.2: Frontier Visual Intelligence
Black Forest Labs. FLUX.2: Frontier Visual Intelligence. 2025. Accessed 2026-02-27
2025
-
[13]
Lisa: Reasoning segmentation via large language model
Xin Lai, Zhuotao Tian, Yukang Chen, Yanwei Li, Yuhui Yuan, Shu Liu, and Jiaya Jia. Lisa: Reasoning segmentation via large language model. InCVPR, pages 9579–9589, 2024
2024
-
[14]
Visualizing the loss landscape of neural nets.Advances in neural information processing systems, 31, 2018
Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, and Tom Goldstein. Visualizing the loss landscape of neural nets.Advances in neural information processing systems, 31, 2018
2018
-
[15]
Towards out-of-distribution generalization: A survey.arXiv preprint arXiv:2108.13624, 2021
Jiashuo Liu, Zheyan Shen, Yue He, Xingxuan Zhang, Renzhe Xu, Han Yu, and Peng Cui. Towards out-of-distribution generalization: A survey.arXiv preprint arXiv:2108.13624, 2021
Pith/arXiv arXiv 2021
-
[16]
Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017
Pith/arXiv arXiv 2017
-
[17]
Giung Nam, Byeongho Heo, and Juho Lee. Lipsum-ft: Robust fine-tuning of zero-shot models using random text guidance.arXiv preprint arXiv:2404.00860, 2024
Pith/arXiv arXiv 2024
-
[18]
Towards universal fake image detectors that generalize across generative models
Utkarsh Ojha, Yuheng Li, and Yong Jae Lee. Towards universal fake image detectors that generalize across generative models. InCVPR, pages 24480–24489, 2023
2023
-
[19]
Gpt image 1.5
OpenAI. Gpt image 1.5. 2026
2026
-
[20]
Gpt image 2 model
OpenAI. Gpt image 2 model. https://developers.openai.com/api/docs/models/gpt-image-2 ,
-
[21]
Team Seedream, Yunpeng Chen, Yu Gao, et al. Seedream 4.0: Toward next-generation multimodal image generation.arXiv preprint arXiv:2509.20427, 2025
Pith/arXiv arXiv 2025
-
[22]
Xinyi Shang, Yi Tang, Jiacheng Cui, Ahmed Elhagry, Salwa K Al Khatib, Sondos Mahmoud Bsharat, Jiacheng Liu, Xiaohan Zhao, Jing-Hao Xue, Hao Li, et al. From masks to pixels and meaning: A new taxonomy, benchmark, and metrics for vlm image tampering.arXiv preprint arXiv:2603.20193, 2026
arXiv 2026
-
[23]
Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations
Carole H Sudre, Wenqi Li, Tom Vercauteren, Sebastien Ourselin, and M Jorge Cardoso. Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations. InInternational Workshop on Deep Learning in Medical Image Analysis, pages 240–248. Springer, 2017
2017
-
[24]
Frequency- aware deepfake detection: Improving generalizability through frequency space domain learning
Chuangchuang Tan, Yao Zhao, Shikui Wei, Guanghua Gu, Ping Liu, and Yunchao Wei. Frequency- aware deepfake detection: Improving generalizability through frequency space domain learning. InAAAI, volume 38, pages 5052–5060, 2024
2024
-
[25]
Rethinking the up-sampling operations in cnn-based generative network for generalizable deepfake detection
Chuangchuang Tan, Yao Zhao, Shikui Wei, Guanghua Gu, Ping Liu, and Yunchao Wei. Rethinking the up-sampling operations in cnn-based generative network for generalizable deepfake detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 28130–28139, 2024
2024
-
[26]
Learning on gradients: Generalized artifacts representation for gan-generated images detection
Chuangchuang Tan, Yao Zhao, Shikui Wei, Guanghua Gu, and Yunchao Wei. Learning on gradients: Generalized artifacts representation for gan-generated images detection. InCVPR, pages 12105–12114, 2023
2023
-
[27]
Cnn-generated images are surprisingly easy to spot
Sheng-Yu Wang, Oliver Wang, Richard Zhang, Andrew Owens, and Alexei A Efros. Cnn-generated images are surprisingly easy to spot... for now. InCVPR, pages 8695–8704, 2020
2020
-
[28]
Qwen-image technical report.arXiv preprint arXiv:2508.02324, 2025
Chenfei Wu, Jiahao Li, Jingren Zhou, Junyang Lin, Kaiyuan Gao, Kun Yan, Sheng-ming Yin, Shuai Bai, Xiao Xu, Yilei Chen, et al. Qwen-image technical report.arXiv preprint arXiv:2508.02324, 2025
Pith/arXiv arXiv 2025
-
[29]
Gsva: Generalized segmentation via multimodal large language models
Zhuofan Xia, Dongchen Han, Yizeng Han, Xuran Pan, Shiji Song, and Gao Huang. Gsva: Generalized segmentation via multimodal large language models. InCVPR, pages 3858–3869, 2024
2024
-
[30]
Domain generalization: A survey
Kaiyang Zhou, Ziwei Liu, Yu Qiao, Tao Xiang, and Chen Change Loy. Domain generalization: A survey. IEEE transactions on pattern analysis and machine intelligence, 45(4):4396–4415, 2022. 11 Appendix A Limitations Although our framework improves domain generalization for pixel-level VLM image tampering detection, it still has several limitations. First, its...
2022
-
[2026]
Accessed: 2026-06-03
OpenAI API model documentation. Accessed: 2026-06-03. 10
2026
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.