Pith. sign in

REVIEW 3 major objections 5 minor 100 references

Towards Reliable Identification of Diffusion-based Image Manipulations

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A new method, RADAR, claims state-of-the-art detection and localization of diffusion-based inpainting edits, including edits made by models never seen during training.

desk verdict Solid empirical contribution with a genuinely useful benchmark; the main caveat is an underspecified ground-truth mask definition that could bias localization metrics and patch labels. read the letter →

arxiv 2506.05466 v2 pith:OU6RZDIS submitted 2025-06-05 cs.CV

Alex Costanzino , Woody Bayliss , Juil Sock , Marc Gorriz Blanch , Danijela Horak , Ivan Laptev , Philip Torr , Fabio Pizzati This is my paper
classification cs.CV
keywords imageforgerydetectioninpaintinglocalizationdiffusionmodelscross-inpaintergeneralizationfoundationpatch-levelcontrastivelearningcross-attentionfusionbenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Diffusion-based image editing can change what a picture shows, and this paper argues that such manipulations can be reliably detected and located, even when the editing model was never seen in training. The authors propose RADAR, a method that combines frozen semantic and geometric feature encoders with a trainable cross-attention fusion block, and trains a patch-level contrastive loss that separates three kinds of image patches: original, tampered, and affected (patches in tampered images that are not directly edited but are shifted by the latent-diffusion autoencoder). To evaluate this, they build BBC-PAIR, a benchmark with more than 150,000 images tampered by 28 different inpainting models spanning open-source, LoRA-customised, and commercial tools. RADAR outperforms seven recent baselines on both detection (AUC 0.945/0.901/0.805) and localisation (IoU 0.600/0.521/0.450) on the in-distribution, LoRA, and out-of-distribution splits. The broader point is that cross-inpainter generalization can be engineered rather than left to chance, which matters because new diffusion models appear faster than forensic datasets can be rebuilt.

What carries the argument

The load-bearing object is the patch-level contrastive learning scheme defined over three patch groups: $P_{\mathrm{orig}}$, $P_{\mathrm{tamp}}$, and $P_{\mathrm{affected}}$. A patch is tampered if it overlaps the inpainting mask, and affected if it comes from a tampered image but lies outside the mask; the latter class captures the auto-encoding shifts that latent diffusion models spread across the whole image. A supervised contrastive loss $\mathcal{L}_{\mathrm{SCL}}$ clusters embeddings of the same group and repels different groups. The second component is the Fusion Block: two multi-head cross-attention layers exchange keys between the semantic encoder's features $F_S$ and the geometry encoder's features $F_G$, followed by a patchwise MLP that produces fused features $f^M$. The final component is the benchmark itself, BBC-PAIR, whose three splits (ID, LoRA, OOD) define what reliable means for cross-inpainter evaluation.

What would settle it

Take a random subset of BBC-PAIR tampered images, compute the pixel-wise difference between original and tampered versions (e.g., per-pixel L2 after alignment), threshold it to define the actually-changed region, and measure IoU against the paper's masks. If the agreement is low (say, below 0.7 IoU), then the supervision signal and all localization scores in the paper are computed against the wrong target, and the reported localization gains would not transfer to real edits.

Watch

Extended reading notes

Core claim

The paper's central claim is that RADAR makes image forensics robust to the diversity of diffusion inpainting models. Concretely, the authors report that fusing features from two pre-trained transformers, one oriented to semantics (DINO-v2) and one to geometry (Depth Anything V2), through a symmetric cross-attention block, and supervising the resulting patch features with a three-class contrastive loss, produces tampering maps that beat all seven compared methods on every split of BBC-PAIR. They attribute the generalization to unseen models to the contrastive constraint: original, tampered and affected patches are pulled into their own clusters, so the network learns what edited regions have in common across different inpainters rather than memorizing one model's artifacts. The method also stays accurate under JPEG compression, resizing, and blur, and it localizes edits from commercial models such as DALL·E 2 and Adobe Firefly despite never seeing them at training time.

Load-bearing premise

The paper trains and evaluates against a ground-truth mask that is the Grounded SAM segmentation of the object chosen for inpainting, after at least one dilation and cohesion filtering, and never validates that this mask matches the pixels the inpainter actually changed.

Editorial extensions

If this is right

  • Content moderators and journalists could use a system like RADAR to flag and localize inpainted regions in images, including those produced by closed or newly released models.
  • Because RADAR keeps the two foundation encoders frozen, adding a new inpainter to the training mix only requires regenerating data, not retraining the feature extractors, so coverage can expand cheaply.
  • The three-class patch contrastive loss gives other forensic tasks a template: any manipulation type that leaves a signature outside the edited region could be treated with an analogous affected class.
  • On BBC-PAIR-OOD, RADAR's detection AUC of 0.805 still leaves a non-trivial error rate, so the paper's own conclusion that the problem remains far from solved is a direct corollary of its results.
  • The benchmark's three evaluation splits provide a protocol for future IFDL methods to be judged on cross-inpainter generalization, not just on one inpainter.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: if the affected patch class is as important as the ablations suggest, then methods that only supervise edited pixels may be systematically handicapped on latent diffusion models, where the autoencoder rewrites the whole image.
  • Editorial extension: the method's reliance on frozen encoders means its upper bound is set by those encoders' ability to separate semantic and geometric cues; as better foundation models appear, gains may transfer without architectural change.
  • Editorial extension: the top-1% mean pooling used for detection is a heuristic; a calibrated classifier on the localization map might close the gap between the 0.893 accuracy reported in ablations and the 0.931 accuracy with the dedicated classification head.
  • Testable extension: applying RADAR's contrastive recipe to other editing operations such as object removal or face replacement and measuring the affected cluster's separability would show whether the mechanism is general or inpainting-specific.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes RADAR, a method for detecting and localizing diffusion-based inpainting manipulations. RADAR uses two frozen foundation-model encoders (DINO-v2 for semantics, Depth Anything V2 for geometry), fuses their patch features with a symmetric cross-attention block, and trains a localization head with a supervised contrastive loss over three patch categories: original, tampered, and affected. The authors also introduce BBC-PAIR, a benchmark with images tampered by 28 inpainters, split into in-distribution (ID), LoRA-fine-tuned, and out-of-distribution (OOD) scenarios. Experiments report large gains over seven baselines on BBC-PAIR and on existing benchmarks (CocoGlide, SID-Set, SafireMS-Expert++).

Significance. If the reported results are reliable, this is a strong contribution: it demonstrates that combining multiple frozen foundation-model features with patch-level contrastive learning yields state-of-the-art performance and notable cross-inpainter generalization in image forgery localization. The benchmark BBC-PAIR is a valuable resource, and the paper includes extensive ablations, external-benchmark evaluation, robustness tests, and a commitment to release code, data, and models. The main technical idea is well-motivated and the experiments are unusually comprehensive. However, the reliability of every localization number depends on the correctness of the training and evaluation ground-truth masks, and this is where the paper currently has a gap.

major comments (3)
  1. [Section 3.1, Appendix B.1, Eqs. (3)-(4), Eq. (9)] The supervision mask y is defined as the pre-dilation Grounded SAM mask in Section 3.1, but Appendix B.1 states that 'every mask undergoes at least one dilation' and that the mask used for inpainting is the 'edited mask' recorded in the metadata. The paper never establishes that the dilated, cohesion-filtered mask is identical to the region actually modified by each inpainter. Since Eq. (3) and Eq. (4) use y to label tampered and affected patches, and Eq. (9) uses y as the localization target, any mismatch between y and the truly changed pixels systematically biases both the contrastive supervision and all reported F1/IoU numbers. The authors should validate that y accurately delineates the modified region (e.g., by comparing y against the difference image x_tamp - x_orig, or by re-running evaluation with the recorded 'edited mask' as ground truth) and, if needed, retrain or re-evaluate with the correct mask.
  2. [Section 4.1, BBC-PAIR-OOD] The claim that BBC-PAIR-OOD contains 'completely unseen inpainters' is overstated. The OOD set includes FLUX.1 Fill [pro], which belongs to the same FLUX family as FLUX.1-schnell and FLUX.1-dev used in training, and ClipDrop, which is likely Stable Diffusion-based and thus close to the SD 1.4/1.5/2.1 models in the ID set. Because these two models constitute two of the eight commercial inpainters, the OOD numbers partly measure family-level generalization rather than fully unseen architectures. Please either remove such near-training-distribution models from the OOD evaluation, report results excluding them, or rephrase the claim as 'unseen model instances' with an explicit caveat.
  3. [Table 1 and Section 4.2] The claim that RADAR 'significantly outperforms' all baselines is not supported by any measure of uncertainty. Table 1 reports single-run numbers without error bars, confidence intervals, or significance tests. Given that the reported gaps are large, this may be a formality, but the word 'significantly' should either be backed by statistical testing or replaced with a descriptive statement. I recommend reporting at least three runs with mean and standard deviation for the main comparison.
minor comments (5)
  1. [Abstract and Section 1] The phrase 'both seen and unseen diffusion models' should be qualified in light of the OOD composition issue raised above, to avoid overclaiming in the abstract.
  2. [Appendix C.9] The appendix correctly notes that SafireMS-Expert++ is in-domain for SAFIRE and SID-Set is in-domain for SIDA. This is an important fairness detail and should be mentioned in the main paper when presenting the OOD results in Table 1.
  3. [Section 3.3, Eq. (8)] The contrastive loss in Eq. (8) includes all patches in the batch as negatives, including patches from the same image and from the same original image. Please clarify whether any intra-image or intra-sample masking is applied, or whether the model intentionally treats all non-ancor patches of different labels as negatives.
  4. [Appendix B.1] The description of the mask selection process is slightly ambiguous: it says 'only the first and last masks for each image are used during inpainting,' but the relationship between the 'first' object mask and the random mask (which is 'always the last') could be stated more explicitly. Also, the criteria for 'too many disconnected components' and 'fewer than eight' should be given exact numbers.
  5. [Appendix C.3] Table 4 reports a very low detection AUC (0.487-0.597) on non-inpainting datasets; the discussion attributes this to a lack of 'affected' pixels, but a more quantitative analysis (e.g., the distribution of detection scores on these datasets) would strengthen the explanation.

Circularity Check

1 steps flagged · score 2.0 of 10

RADAR is an empirical system with independent external evaluation; the only circularity is a minor self-definitional interpretation of the 'affected patches' cluster and a non-load-bearing self-citation.

  1. self definitional [Section 3.3, Eq. (4); Section 4.3, Fig. 5b]
    "Empirically, we find it beneficial to model a third category of patches, which we refer to as affected. These patches originate from tampered images but do not overlap with the inpainting mask... Hence, we define: Paffected = ... We observe that affected patches separate original and tampered clusters. This highlights that the hybrid nature of the affected patch distribution, processed by the auto-encoder in latent diffusion models, while containing no generated content, naturally emerges during training as an intermediate distribution, justifying our design."

    The 'affected' class is defined by Eq. (4) as the complement of the inpainting mask in tampered images, and Eq. (8) (LSCL) is a supervised contrastive loss that explicitly assigns this set its own label class (with labels ℓz ∈ {orig, tamp, affected}). The UMAP separation in Fig. 5b is therefore the trained outcome of that three-way objective, not an independent observation that the affected distribution 'naturally emerges'. The claim that affected patches are an intermediate distribution and justify the design is enforced by construction, so it cannot serve as external validation. This does not affect the central SOTA comparison, which rests on F1/IoU/AUC against external benchmarks.

full rationale

RADAR's central claim is empirical rather than derivational: the method is trained on a constructed dataset and tested against seven baselines on both its own BBC-PAIR splits and on the pre-existing CocoGlide, SID-Set, and SafireMS-Expert++ benchmarks. Those external evaluations give the SOTA claim independent content; no fitted parameter is renamed as a prediction and no uniqueness theorem is invoked. The only self-definitional element is the interpretation of 'affected' patches: Paffected is defined in Eq. (4) as non-mask patches of tampered images and then explicitly forced into its own cluster by the supervised contrastive loss LSCL (Eq. 8), so the UMAP separation in Fig. 5b is a consequence of the training labels rather than an emergent discovery. This is a minor interpretive circularity that does not carry the benchmark results. Reference [3] includes current authors, but it only supports the general claim that distributional biases persist in large-scale training and is not load-bearing. A separate correctness risk, not a circularity, is the potential mismatch between the Section 3.1 definition of y as the pre-dilation Grounded SAM mask and the dilated/cohesion-filtered 'edited mask' recorded in Appendix B.1; the paper reports no validation that the supervision mask matches the pixels actually modified by each inpainter. That concern affects external validity of the localization numbers but does not make the method's outputs equivalent to its inputs.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the data-generation pipeline and on modeling assumptions about feature transfer, patch-level distributions, and benchmark representativeness. These are empirical domain assumptions rather than derived facts, and none are machine-checked.

free parameters (5)
  • Mask area filtering thresholds = >83% or <0.23% of image area rejected
    Hand-chosen bounds in BBC-PAIR construction (Appendix B.1) that shape the tampering mask distribution used for training and evaluation.
  • Mask cohesion criteria = fewer than 8 disconnected components, up to 5 dilations, at least one dilation
    Hand-chosen criteria in Appendix B.1 that decide which object masks are accepted and how they are edited before inpainting.
  • Loss weighting between L_SCL and L_loc = equal weights (implicit 1.0 each)
    Equation (10) sums the contrastive and localization losses with no tuned coefficient; no ablation varies this weighting.
  • Detection threshold and top-1% score = threshold 0.5; mean of top 1% of predicted map
    Hand-picked decision rule for image-level detection (Section 3.4), directly affects reported accuracy.
  • Number of training inpainters K = 10
    The paper selects 10 open-source inpainters for BBC-PAIR-ID training and shows in ablations that larger K helps; the exact set is a design choice.
assumptions (4)
  • domain assumption Patches from tampered images that do not overlap the inpainting mask form a separable 'affected' distribution due to the auto-encoding process in latent diffusion models.
    Invoked in Section 3.3 (Eq. 4) to justify the three-class supervised contrastive loss. If the autoencoder shift is inconsistent across inpainters, the loss may not generalize.
  • domain assumption Frozen DINO-v2 and Depth Anything V2 features are semantically and geometrically complementary and can be fused by symmetric cross-attention without fine-tuning.
    Invoked in Section 3.2; the paper relies on shared transformer backbones making the embedding spaces compatible, and never ablates the choice of encoders or unfreezing.
  • domain assumption The Grounded SAM mask used to prompt inpainting is an accurate pixel-wise ground truth for the tampered region.
    Used in Section 3.1 to define y for training supervision and evaluation; no validation against actual modified pixels is provided, and the pipeline applies dilations that may shift the mask.
  • domain assumption The BBC-PAIR-OOD commercial inpainters are completely unseen and maximally shifted from the ID training set.
    Section 4.1 claims maximum distribution shift, but FLUX.1 Fill [pro] belongs to the FLUX family used in the ID set and ClipDrop likely uses Stable Diffusion, so the shift may be partial.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards Reliable Identification of Diffusion-based Image Manipulations." pith.science (2026). https://pith.science/paper/OU6RZDIS

@misc{pith2026250605466,
  author       = {Pith},
  title        = {Pith review of: Towards Reliable Identification of Diffusion-based Image Manipulations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OU6RZDIS}},
  note         = {Machine review of arXiv:2506.05466}
}
read the original abstract

Changing facial expressions, gestures, or background details may dramatically alter the meaning conveyed by an image. Notably, recent advances in diffusion models greatly improve the quality of image manipulation while also opening the door to misuse. Identifying changes made to authentic images, thus, becomes an important task, constantly challenged by new diffusion-based editing tools. To this end, we propose a novel approach for ReliAble iDentification of inpainted AReas (RADAR). RADAR builds on existing foundation models and combines features from different image modalities. It also incorporates an auxiliary contrastive loss that helps to isolate manipulated image patches. We demonstrate these techniques to significantly improve both the accuracy of our method and its generalisation to a large number of diffusion models. To support realistic evaluation, we further introduce BBC-PAIR, a new comprehensive benchmark, with images tampered by 28 diffusion models. Our experiments show that RADAR achieves excellent results, outperforming the state-of-the-art in detecting and localising image edits made by both seen and unseen diffusion models. Our code, data and models will be publicly available at https://alex-costanzino.github.io/radar/.

Figures

Figures reproduced from arXiv: 2506.05466 by the authors.

Figure 1
Figure 1. Data generation pipeline. We first extract with Kosmos-2 [58] a set of objects in an image xorig, along with a caption c. We then use Grounded SAM [65] to segment one of the objects omask (e.g. "duck"). Lastly, we use a set of K text-to-image inpainters I to generate K tampered images {x k tamp} K k=1. We use c as a prompt to avoid the generation of objects irrelevant to the scene. problem has been tackled, either b… view at source ↗
Figure 2
Figure 2. Training RADAR. We first extract, for an input image x, multi-modal features Fx by using our multi-modal encoder E. To implement E, we employ the pre-trained Semantic Encoder ES and Geometry Encoder EG, and fuse their extracted features with a Fusion Block (on the right, in details). The Fusion Block exploits a symmetric cross-attention mechanism, based on the swapping of keys K in two multi-head attention mechanism… view at source ↗
Figure 3
Figure 3. Contrastive learning. We identify original, tampered and affected patches. Correspond￾ing feature patches are mapped by the head γ to Z embeddings, on which we impose the contrastive loss LSCL, enforcing patch features of the same distribution to be clustered together. Note that we aggregate Z embeddings from multiple inpainters, promoting cross-inpainter generalisation. V of the other, as shown in [PITH_FULL_IMAGE… view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Qualitative results. Compared to the best baselines in the OOD scenario, RADAR accurately localises tampering. Baselines exhibit semantic biases, as SIDA (middle row) and Mesorch (last row) identify full objects. Importantly, RADAR is robust to false positives and does…
Figure 5
Figure 5. Figure 5: Properties of RADAR. In Figure (a), we analyse feature magnitude in both encoders ES and EG, highlighting that they are roughly complementary. In Figure (b), we show with UMAP [51] that affected patches serve as separators between original and tampered, justifying our …
Figure 6
Figure 6. Figure 6: Qualitative results on tampered in-the-wild images. Note that SIDA produces a binary output. each and residual connections, where each modality attends to the other. The mixed features are projected through a single-layer GELU-activated MLP and lastly layer-normalised.…
Figure 7
Figure 7. Figure 7: Features and attention maps [PITH_FULL_IMAGE:figures/full_fig_p022_7.png]
Figure 8
Figure 8. Figure 8: Qualitative results on BBC-PAIR-OOD (closed-source inpainters). In future work, we aim to address these limitations by (1) exploring even more expressive architectures to reduce error rates, (2) designing adaptive loss functions to balance precision and recall, (3) int…
Figure 9
Figure 9. Figure 9: Qualitative results on other BBC-PAIR-OOD benchmarks. 27 [PITH_FULL_IMAGE:figures/full_fig_p027_9.png]
Figure 10
Figure 10. Figure 10: Qualitative results BBC-PAIR-ID and BBC-PAIR-LoRA benchmarks. 28 [PITH_FULL_IMAGE:figures/full_fig_p028_10.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

100 extracted references · 72 canonical work pages

  1. [1]

    https://firefly.adobe.com (2023)

    Adobe Inc.: Adobe firefly. https://firefly.adobe.com (2023)

    2023

  2. [2]

    In: IEEE Conference on Computer Vision and Pattern Recognition Workshops (2017)

    Agustsson, E., Timofte, R.: Ntire 2017 challenge on single image super-resolution: Dataset and study. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops (2017)

    2017

  3. [3]

    In: European Conference on Computer Vision (2024)

    Al Kader Hammoud, H.A., Das, T., Pizzati, F., Torr, P.H., Bibi, A., Ghanem, B.: On pretraining data diversity for self-supervised learning. In: European Conference on Computer Vision (2024)

    2024

  4. [4]

    https://www.lightxeditor.com (2016)

    AndOr Communications Pvt Ltd: Lightx. https://www.lightxeditor.com (2016)

    2016

  5. [5]

    In: Empirical Methods in Natural Language Processing (2024)

    Arkhipkin, V ., Vasilev, V ., Filatov, A., Pavlov, I., Agafonova, J., Gerasimenko, N., Averchenkova, A., Mironova, E., Bukashkin, A., Kulikov, K., et al.: Kandinsky 3: Text-to-image synthesis for multifunctional generative framework. In: Empirical Methods in Natural Language Processing (2024)

    2024

  6. [6]

    ACM Transactions on Graphics (2023)

    Avrahami, O., Fried, O., Lischinski, D.: Blended latent diffusion. ACM Transactions on Graphics (2023)

    2023

  7. [7]

    In: IEEE Conference on Computer Vision and Pattern Recognition (2022)

    Avrahami, O., Lischinski, D., Fried, O.: Blended diffusion for text-driven editing of natural images. In: IEEE Conference on Computer Vision and Pattern Recognition (2022)

    2022

  8. [8]

    IEEE Transactions on Information Forensics and Security (2012)

    Bianchi, T., Piva, A.: Image forgery localization via block-grained analysis of jpeg artifacts. IEEE Transactions on Information Forensics and Security (2012)

    2012

Show all 100 references
  1. [9]

    https://bfl.ai/announcements/ 24-08-01-bfl (2024)

    Black Forest Labs: Announcing black forest labs. https://bfl.ai/announcements/ 24-08-01-bfl (2024)

    2024

  2. [10]

    Canal História (2019)

    Blakemore, E.: How photos became a weapon in stalin’s great purge. Canal História (2019)

    2019

  3. [11]

    In: International Conference on Computer Vision (2021)

    Caron, M., Touvron, H., Misra, I., Jégou, H., Mairal, J., Bojanowski, P., Joulin, A.: Emerging properties in self-supervised vision transformers. In: International Conference on Computer Vision (2021)

    2021

  4. [12]

    IEEE Transactions on Information Forensics and Security (2013)

    de Carvalho, T.J., Riess, C., Angelopoulou, E., Pedrini, H., de Rezende Rocha, A.: Exposing digital image forgeries by illumination color classification. IEEE Transactions on Information Forensics and Security (2013)

    2013

  5. [13]

    In: IEEE Conference on Computer Vision and Pattern Recognition (2024)

    Cazenavette, G., Sud, A., Leung, T., Usman, B.: Fakeinversion: Learning to detect images from unseen text-to-image models by inverting stable diffusion. In: IEEE Conference on Computer Vision and Pattern Recognition (2024)

    2024

  6. [14]

    In: International Conference on Computer Vision (2021)

    Chen, C.F.R., Fan, Q., Panda, R.: Crossvit: Cross-attention multi-scale vision transformer for image classification. In: International Conference on Computer Vision (2021)

    2021

  7. [15]

    Chen, J., Wu, Y ., Luo, S., Xie, E., Paul, S., Luo, P., Zhao, H., Li, Z.: Pixart- δ: Fast and controllable image generation with latent consistency models (2024)

    2024

  8. [16]

    In: International Conference on Machine Learning (2020)

    Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for contrastive learning of visual representations. In: International Conference on Machine Learning (2020)

    2020

  9. [17]

    International Journal of Intelligent Systems (2025)

    Chen, Y ., Yin, W., Luo, A., Yang, J., Wang, J.: Improving the generalization and robustness of computer-generated image detection based on contrastive learning. International Journal of Intelligent Systems (2025)

    2025

  10. [18]

    In: AAAI Conference on Artificial Intelligence (2025)

    Chen, Y ., Huang, X., Zhang, Q., Li, W., Zhu, M., Yan, Q., Li, S., Chen, H., Hu, H., Yang, J., et al.: Gim: A million-scale benchmark for generative image manipulation detection and localization. In: AAAI Conference on Artificial Intelligence (2025)

    2025

  11. [19]

    California Law Review (2019)

    Chesney, R., Citron, D.K.: Deep fakes: A looming challenge for privacy, democracy, and national security. California Law Review (2019)

    2019

  12. [20]

    In: IEEE Conference on Computer Vision and Pattern Recognition (2025) 11

    Chu, B., Xu, X., Wang, X., Zhang, Y ., You, W., Zhou, L.: Fire: Robust detection of diffusion- generated images via frequency-guided reconstruction error. In: IEEE Conference on Computer Vision and Pattern Recognition (2025) 11

    2025

  13. [21]

    IEEE Transactions on Pattern Analysis and Machine Intelligence (2025)

    Croitoru, F.A., Hondru, V ., Ionescu, R.T., Shah, M.: Diffusion models in vision: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence (2025)

    2025

  14. [22]

    In: Winter Applications of Computer Vision (2025)

    Das, A.S., Pang, G., Bhuyan, M.: Adaptive Deviation Learning for Visual Anomaly Detection with Data Contamination . In: Winter Applications of Computer Vision (2025)

    2025

  15. [23]

    In: IEEE International Conference on Acoustics, Speech, and Signal Processing (2024)

    Doloriel, C.T., Cheung, N.M.: Frequency masking for universal deepfake detection. In: IEEE International Conference on Acoustics, Speech, and Signal Processing (2024)

    2024

  16. [24]

    In: IEEE China Summit and International Conference on Signal and Information Processing (2013)

    Dong, J., Wang, W., Tan, T.: CASIA image tampering detection evaluation database. In: IEEE China Summit and International Conference on Signal and Information Processing (2013)

    2013

  17. [25]

    In: International Conference on Computer Vision (2023)

    Epstein, D.C., Jain, I., Wang, O., Zhang, R.: Online detection of ai-generated images. In: International Conference on Computer Vision (2023)

    2023

  18. [26]

    In: International Conference on Machine Learning (2024)

    Esser, P., Kulal, S., Blattmann, A., Entezari, R., Müller, J., Saini, H., Levi, Y ., Lorenz, D., Sauer, A., Boesel, F., et al.: Scaling rectified flow transformers for high-resolution image synthesis. In: International Conference on Machine Learning (2024)

    2024

  19. [27]

    https://fluxart.ai (2024)

    FLUX AI: Flux.1 fill [pro]. https://fluxart.ai (2024)

    2024

  20. [28]

    In: International Conference on Machine Learning (2020)

    Frank, J., Eisenhofer, T., Schönherr, L., Fischer, A., Kolossa, D., Holz, T.: Leveraging frequency analysis for deep fake image recognition. In: International Conference on Machine Learning (2020)

    2020

  21. [29]

    IEEE Transactions on Pattern Analysis and Machine Intelligence (2021)

    Fu, K., Fan, D.P., Ji, G.P., Zhao, Q., Shen, J., Zhu, C.: Siamese network for rgb-d salient object detection and beyond. IEEE Transactions on Pattern Analysis and Machine Intelligence (2021)

    2021

  22. [30]

    https://www.dpreview.com/ (1998)

    GPS Media: Dpreview. https://www.dpreview.com/ (1998)

    1998

  23. [31]

    In: Winter Applications of Computer Vision Workshops (2019)

    Guan, H., Kozak, M., Robertson, E., Lee, Y ., Yates, A.N., Delgado, A., Zhou, D., Kheyrkhah, T., Smith, J., Fiscus, J.: Mfc datasets: Large-scale benchmark datasets for media forensic challenge evaluation. In: Winter Applications of Computer Vision Workshops (2019)

    2019

  24. [32]

    In: IEEE Conference on Com- puter Vision and Pattern Recognition (2023)

    Guillaro, F., Cozzolino, D., Sud, A., Dufour, N., Verdoliva, L.: Trufor: Leveraging all-round clues for trustworthy image forgery detection and localization. In: IEEE Conference on Com- puter Vision and Pattern Recognition (2023)

    2023

  25. [33]

    In: IEEE Conference on Computer Vision and Pattern Recognition (2023)

    Guo, X., Liu, X., Ren, Z., Grosz, S., Masi, I., Liu, X.: Hierarchical fine-grained image forgery detection and localization. In: IEEE Conference on Computer Vision and Pattern Recognition (2023)

    2023

  26. [34]

    Cyberpsychology, behavior, and social networking (2021)

    Hancock, J.T., Bailenson, J.N.: The social impact of deepfakes. Cyberpsychology, behavior, and social networking (2021)

    2021

  27. [35]

    In: International Conference on Computer Vision (2021)

    Hendrycks, D., Basart, S., Mu, N., Kadavath, S., Wang, F., Dorundo, E., Desai, R., Zhu, T., Parajuli, S., Guo, M., et al.: The many faces of robustness: A critical analysis of out-of- distribution generalization. In: International Conference on Computer Vision (2021)

    2021

  28. [36]

    In: AAAI Conference on Artificial Intelligence (2025)

    Hong, Y ., Zhang, J.: Wildfake: A large-scale challenging dataset for ai-generated images detection. In: AAAI Conference on Artificial Intelligence (2025)

    2025

  29. [37]

    In: IEEE International Conference on Multimedia and Expo (ICME) (2006)

    Hsu, Y .f., Chang, S.F.: Detecting image splicing using geometry invariants and camera char- acteristics consistency. In: IEEE International Conference on Multimedia and Expo (ICME) (2006)

    2006

  30. [38]

    IEEE Transactions on Pattern Analysis and Machine Intelligence (2025)

    Huang, Y ., Huang, J., Liu, Y ., Yan, M., Lv, J., Liu, J., Xiong, W., Zhang, H., Cao, L., Chen, S.: Diffusion model-based image editing: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence (2025)

    2025

  31. [39]

    In: IEEE Conference on Computer Vision and Pattern Recognition (2025)

    Huang, Z., Hu, J., Li, X., He, Y ., Zhao, X., Peng, B., Wu, B., Huang, X., Cheng, G.: Sida: Social media image deepfake detection, localization and explanation with large multimodal model. In: IEEE Conference on Computer Vision and Pattern Recognition (2025)

    2025

  32. [40]

    https://ideogram.ai (2023) 12

    Ideogram Inc.: Ideogram. https://ideogram.ai (2023) 12

    2023

  33. [41]

    In: IEEE Conference on Computer Vision and Pattern Recognition Workshops (2023)

    Jia, S., Huang, M., Zhou, Z., Ju, Y ., Cai, J., Lyu, S.: Autosplice: A text-prompt manipulated image dataset for media forensics. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops (2023)

    2023

  34. [42]

    In: Advances on Neural Information Processing Systems (2020)

    Khosla, P., Teterwak, P., Wang, C., Sarna, A., Tian, Y ., Isola, P., Maschinot, A., Liu, C., Krishnan, D.: Supervised contrastive learning. In: Advances on Neural Information Processing Systems (2020)

    2020

  35. [43]

    Advances on Neural Information Processing Systems (2020)

    Khosla, P., Teterwak, P., Wang, C., Sarna, A., Tian, Y ., Isola, P., Maschinot, A., Liu, C., Krishnan, D.: Supervised contrastive learning. Advances on Neural Information Processing Systems (2020)

    2020

  36. [44]

    In: International Conference on Learning Representations (2015)

    Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. In: International Conference on Learning Representations (2015)

    2015

  37. [45]

    In: AAAI Conference on Artificial Intelligence (2025)

    Kwon, M.J., Lee, W., Nam, S.H., Son, M., Kim, C.: Safire: Segment any forged image region. In: AAAI Conference on Artificial Intelligence (2025)

    2025

  38. [46]

    In: IEEE Conference on Computer Vision and Pattern Recognition (2024)

    Li, S., Ma, W., Guo, J., Xu, S., Li, B., Zhang, X.: Unionformer: Unified-learning transformer with multi-view representation for image manipulation detection and localization. In: IEEE Conference on Computer Vision and Pattern Recognition (2024)

    2024

  39. [47]

    arXiv preprint arXiv:2412.10316 (2024)

    Li, Y ., Bian, Y ., Ju, X., Zhang, Z., Shan, Y ., Zou, Y ., Xu, Q.: Brushedit: All-in-one image inpainting and editing. arXiv preprint arXiv:2412.10316 (2024)

    2024 arXiv

  40. [48]

    In: European Conference on Computer Vision (2025)

    Li, Y ., Cheng, F., Yu, W., Wang, G., Luo, G., Zhu, Y .: Adaifl: Adaptive image forgery localization via a dynamic and importance-aware transformer network. In: European Conference on Computer Vision (2025)

    2025

  41. [49]

    In: IEEE Conference on Computer Vision and Pattern Recognition (2024)

    Liu, H., Wang, Y ., Qian, B., Wang, M., Rui, Y .: Structure matters: Tackling the semantic discrepancy in diffusion models for image inpainting. In: IEEE Conference on Computer Vision and Pattern Recognition (2024)

    2024

  42. [50]

    https://storage.googleapis.com/openimages/web/ index.html (2020)

    LLC, G.: Open images dataset. https://storage.googleapis.com/openimages/web/ index.html (2020)

    2020

  43. [51]

    arXiv preprint arXiv:1802.03426 (2018)

    McInnes, L., Healy, J., Melville, J.: Umap: Uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:1802.03426 (2018)

    2018 arXiv

  44. [52]

    Nichol, A., Dhariwal, P., Ramesh, A., Shyam, P., Mishkin, P., McGrew, B., Sutskever, I., Chen, M.: Glide: Towards photorealistic image generation and editing with text-guided diffusion models (2022)

    2022

  45. [53]

    In: Winter Applications of Computer Vision (2020)

    Novozamsky, A., Mahdian, B., Saic, S.: IMD2020: A large-scale annotated dataset tailored for detecting manipulated images. In: Winter Applications of Computer Vision (2020)

    2020

  46. [54]

    In: IEEE Conference on Computer Vision and Pattern Recognition (2023)

    Ojha, U., Li, Y ., Lee, Y .J.: Towards universal fake image detectors that generalize across generative models. In: IEEE Conference on Computer Vision and Pattern Recognition (2023)

    2023

  47. [55]

    https://openai.com/dall-e-2 (2022)

    OpenAI: Dall·e 2. https://openai.com/dall-e-2 (2022)

    2022

  48. [56]

    Transactions on Machine Learning Research (2024)

    Oquab, M., Darcet, T., Moutakanni, T., V o, H., Szafraniec, M., Khalidov, V ., Fernandez, P., Haziza, D., Massa, F., El-Nouby, A., et al.: Dinov2: Learning robust visual features without supervision. Transactions on Machine Learning Research (2024)

    2024

  49. [57]

    In: IEEE Conference on Computer Vision and Pattern Recognition (2025)

    Park, J., Owens, A.: Community forensics: Using thousands of generators to train fake image detectors. In: IEEE Conference on Computer Vision and Pattern Recognition (2025)

    2025

  50. [58]

    In: International Conference on Learning Representations (2024)

    Peng, Z., Wang, W., Dong, L., Hao, Y ., Huang, S., Ma, S., Wei, F.: Kosmos-2: Grounding multimodal large language models to the world. In: International Conference on Learning Representations (2024)

    2024

  51. [59]

    https://www.perfectcorp.com/consumer/apps/ youcam-makeup (2014) 13

    Perfect Corp.: Youcam. https://www.perfectcorp.com/consumer/apps/ youcam-makeup (2014) 13

    2014

  52. [60]

    https://www.phot.ai (2023)

    Phot.AI: Phot.ai. https://www.phot.ai (2023)

    2023

  53. [61]

    https://github.com/huggingface/diffusers (2022)

    von Platen, P., Patil, S., Lozhkov, A., Cuenca, P., Lambert, N., Rasul, K., Davaadorj, M., Nair, D., Paul, S., Berman, W., Xu, Y ., Liu, S., Wolf, T.: Diffusers: State-of-the-art diffusion models. https://github.com/huggingface/diffusers (2022)

    2022

  54. [62]

    In: International Conference on Learning Representations (2024)

    Podell, D., English, Z., Lacey, K., Blattmann, A., Dockhorn, T., Müller, J., Penna, J., Rom- bach, R.: Sdxl: Improving latent diffusion models for high-resolution image synthesis. In: International Conference on Learning Representations (2024)

    2024

  55. [63]

    In: IEEE Conference on Computer Vision and Pattern Recognition (2024)

    Qu, C., Zhong, Y ., Liu, C., Xu, G., Peng, D., Guo, F., Jin, L.: Towards modern image manipulation localization: A large-scale dataset and novel methods. In: IEEE Conference on Computer Vision and Pattern Recognition (2024)

    2024

  56. [64]

    In: Empirical Methods in Natural Language Processing (2023)

    Razzhigaev, A., Shakhmatov, A., Maltseva, A., Arkhipkin, V ., Pavlov, I., Ryabov, I., Kuts, A., Panchenko, A., Kuznetsov, A., Dimitrov, D.: Kandinsky: an improved text-to-image synthesis with image prior and latent diffusion. In: Empirical Methods in Natural Language Processing (2023)

    2023

  57. [65]

    arXiv preprint arXiv:2401.14159 (2024)

    Ren, T., Liu, S., Zeng, A., Lin, J., Li, K., Cao, H., Chen, J., Huang, X., Chen, Y ., Yan, F., et al.: Grounded sam: Assembling open-world models for diverse visual tasks. arXiv preprint arXiv:2401.14159 (2024)

    2024 arXiv

  58. [66]

    In: IEEE Conference on Computer Vision and Pattern Recognition (2024)

    Ricker, J., Lukovnikov, D., Fischer, A.: Aeroblade: Training-free detection of latent diffusion images using autoencoder reconstruction error. In: IEEE Conference on Computer Vision and Pattern Recognition (2024)

    2024

  59. [67]

    Journal of Imaging (2021)

    Rodriguez-Ortega, Y ., Ballesteros, D.M., Renza, D.: Copy-move forgery detection (cmfd) using deep learning for image and video forensics. Journal of Imaging (2021)

    2021

  60. [68]

    In: IEEE Conference on Computer Vision and Pattern Recognition (2022)

    Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: IEEE Conference on Computer Vision and Pattern Recognition (2022)

    2022

  61. [69]

    In: Advances on Neural Information Processing Systems (2022)

    Saharia, C., Chan, W., Saxena, S., Li, L., Whang, J., Denton, E.L., Ghasemipour, K., Gon- tijo Lopes, R., Karagol Ayan, B., Salimans, T., Ho, J., Fleet, D.J., Norouzi, M.: Photorealistic text-to-image diffusion models with deep language understanding. In: Advances on Neural In...

    2022

  62. [70]

    Sarkar, A., Mai, H., Mahapatra, A., Lazebnik, S., Forsyth, D.A., Bhattad, A.: Shadows don’t lie and lines can’t bend! generative models don’t know projective geometry... for now. In: IEEE Conference on Computer Vision and Pattern Recognition (2024)

    2024

  63. [71]

    Advances on Neural Information Processing Systems (2022)

    Schuhmann, C., Beaumont, R., Vencu, R., Gordon, C., Wightman, R., Cherti, M., Coombes, T., Katta, A., Mullis, C., Wortsman, M., et al.: Laion-5b: An open large-scale dataset for training next generation image-text models. Advances on Neural Information Processing Systems (2022)

    2022

  64. [72]

    https://clipdrop.co (2020)

    Stability AI: Clipdrop. https://clipdrop.co (2020)

    2020

  65. [73]

    In: The International Conference on Medical Image Computing and Computer Assisted Intervention Workshops (2017)

    Sudre, C.H., Li, W., Vercauteren, T., Ourselin, S., Jorge Cardoso, M.: Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations. In: The International Conference on Medical Image Computing and Computer Assisted Intervention Workshops (2017)

    2017

  66. [74]

    In: AAAI Conference on Artificial Intelligence (2025)

    Tan, C., Tao, R., Liu, H., Gu, G., Wu, B., Zhao, Y ., Wei, Y .: C2p-clip: Injecting category common prompt in clip to enhance generalization in deepfake detection. In: AAAI Conference on Artificial Intelligence (2025)

    2025

  67. [75]

    In: IEEE Conference on Computer Vision and Pattern Recognition (2011)

    Torralba, A., Efros, A.A.: Unbiased look at dataset bias. In: IEEE Conference on Computer Vision and Pattern Recognition (2011)

    2011

  68. [76]

    In: The Annual Meeting of the Association for Computational Linguistics (2019) 14

    Tsai, Y .H.H., Bai, S., Liang, P.P., Kolter, J.Z., Morency, L.P., Salakhutdinov, R.: Multimodal transformer for unaligned multimodal language sequences. In: The Annual Meeting of the Association for Computational Linguistics (2019) 14

    2019

  69. [77]

    Social Media+ Society (2020)

    Vaccari, C., Chadwick, A.: Deepfakes and disinformation: Exploring the impact of synthetic political video on deception, uncertainty, and trust in news. Social Media+ Society (2020)

    2020

  70. [78]

    In: IEEE Conference on Computer Vision and Pattern Recognition (2022)

    Wang, J., Wu, Z., Chen, J., Han, X., Shrivastava, A., Lim, S.N., Jiang, Y .G.: Objectformer for image manipulation detection and localization. In: IEEE Conference on Computer Vision and Pattern Recognition (2022)

    2022

  71. [79]

    In: IEEE Conference on Computer Vision and Pattern Recognition (2015)

    Wang, P., Shen, X., Lin, Z., Cohen, S., Price, B., Yuille, A.L.: Towards unified depth and semantic prediction from a single image. In: IEEE Conference on Computer Vision and Pattern Recognition (2015)

    2015

  72. [80]

    Wang, S.Y ., Wang, O., Zhang, R., Owens, A., Efros, A.A.: Cnn-generated images are surpris- ingly easy to spot... for now. In: IEEE Conference on Computer Vision and Pattern Recognition (2020)

    2020

  73. [81]

    arXiv preprint arXiv:2212.06909 (2023)

    Wang, S., Saharia, C., Montgomery, C., Pont-Tuset, J., Noy, S., Pellegrini, S., Onoe, Y ., Laszlo, S., Fleet, D.J., Soricut, R., Baldridge, J., Norouzi, M., Anderson, P., Chan, W.: Imagen editor and editbench: Advancing and evaluating text-guided image inpainting. arXiv prepri...

    2023 arXiv

  74. [82]

    In: International Conference on Computer Vision (2023)

    Wang, Z., Bao, J., Zhou, W., Wang, W., Hu, H., Chen, H., Li, H.: Dire for diffusion-generated image detection. In: International Conference on Computer Vision (2023)

    2023

  75. [83]

    In: IEEE International Conference on Image Processing (2016)

    Wen, B., Zhu, Y ., Subramanian, R., Ng, T.T., Shen, X., Winkler, S.: COVERAGE — A novel database for copy-move forgery detection. In: IEEE International Conference on Image Processing (2016)

    2016

  76. [84]

    arXiv preprint arXiv:2503.14905 (2025)

    Wen, S., Ye, J., Feng, P., Kang, H., Wen, Z., Chen, Y ., Wu, J., Wu, W., He, C., Li, W.: Spot the fake: Large multimodal model-based synthetic image detection with artifact explanation. arXiv preprint arXiv:2503.14905 (2025)

    2025

  77. [85]

    In: International Conference on Learning Representations (2025)

    Xu, Z., Zhang, X., Li, R., Tang, Z., Huang, Q., Zhang, J.: Fakeshield: Explainable image forgery detection and localization via multi-modal large language models. In: International Conference on Learning Representations (2025)

    2025

  78. [86]

    In: International Conference on Learning Representations (2025)

    Yan, S., Li, O., Cai, J., Hao, Y ., Jiang, X., Hu, Y ., Xie, W.: A sanity check for ai-generated image detection. In: International Conference on Learning Representations (2025)

    2025

  79. [87]

    In: Advances on Neural Information Processing Systems (2024)

    Yang, L., Kang, B., Huang, Z., Zhao, Z., Xu, X., Feng, J., Zhao, H.: Depth anything v2. In: Advances on Neural Information Processing Systems (2024)

    2024

  80. [88]

    arXiv preprint arXiv:2304.06790 (2023)

    Yu, T., Feng, R., Feng, R., Liu, J., Jin, X., Zeng, W., Chen, Z.: Inpaint anything: Segment anything meets image inpainting. arXiv preprint arXiv:2304.06790 (2023)

    2023 arXiv

  81. [89]

    In: International Conference on Computer Vision (2021)

    Zavrtanik, V ., Kristan, M., Skocaj, D.: Draem - a discriminatively trained reconstruction embedding for surface anomaly detection. In: International Conference on Computer Vision (2021)

    2021

  82. [90]

    arXiv preprint arXiv:2411.11016 (2024)

    Zeng, Z., Liu, H., Peng, D., Jin, L., Watanabe, H.: Time step generating: A universal synthesized deepfake image detector. arXiv preprint arXiv:2411.11016 (2024)

    2024 arXiv

  83. [91]

    In: AAAI Conference on Artificial Intelligence (2025)

    Zhang, Z., Chang, M.C., Li, X.: Training-free image manipulation localization using diffusion models. In: AAAI Conference on Artificial Intelligence (2025)

    2025

  84. [92]

    In: IEEE Conference on Computer Vision and Pattern Recognition (2020)

    Zhu, S., Brazil, G., Liu, X.: The edge of depth: Explicit constraints between segmentation and depth. In: IEEE Conference on Computer Vision and Pattern Recognition (2020)

    2020

  85. [93]

    In: AAAI Conference on Artificial Intelligence (2025) 15 Appendix In this document, we propose complementary information to the main paper

    Zhu, X., Ma, X., Su, L., Jiang, Z., Du, B., Wang, X., Lei, Z., Feng, W., Pun, C.M., Zhou, J.: Mesoscopic insights: Orchestrating multi-scale & hybrid architecture for image manipulation localization. In: AAAI Conference on Artificial Intelligence (2025) 15 Appendix In this doc...

    2025

  86. [94]

    It is prompted by Kosmos-2 using the object’s location and description; 17

    Segmentation: the SAM [ 65] model is used to generate a segmentation mask for each object. It is prompted by Kosmos-2 using the object’s location and description; 17

  87. [95]

    Objects with masks that are too small or too large are rejected

    Size filtering: the area of the mask is calculated as a percentage of the image area. Objects with masks that are too small or too large are rejected. Specifically, any mask covering more than 83% or less than 0.23% of the image area is discarded. This filtering ensures that i...

  88. [96]

    small”, “medium

    Mask cohesion: once a mask of acceptable size is found, it is analysed for the number of disconnected components. This serves as a proxy for assessing the cohesiveness of the masked region. If a mask contains too many disconnected components, a dilation operation is applied—up...

  89. [97]

    This suggests significant room for improvement in generalisation and robustness

    Performance gap: despite outperforming existing methods, our approach still exhibits a non-trivial error rate, particularly in complex forgery scenarios (e.g., unseen inpainters). This suggests significant room for improvement in generalisation and robustness

  90. [98]

    While this aligns with forensic applications where false accusations carry high stakes, it may limit sensitivity in scenarios requiring exhaustive forgery identification

    Conservative localisation bias: RADAR tends to prioritise precision over recall in locali- sation, resulting in fewer false positives at the cost of missed detections. While this aligns with forensic applications where false accusations carry high stakes, it may limit sensitiv...

  91. [99]

    A more intuitive, human-aligned explanation framework would improve trust and usability in critical domains like journalism or law

    Explainability: although we analyse cross-attention maps and activation patterns to interpret model decisions, the internal reasoning remains opaque. A more intuitive, human-aligned explanation framework would improve trust and usability in critical domains like journalism or law

  92. [100]

    region-wise accuracy) and real-world robustness

    Evaluation metrics: the IFDL field lacks standardised benchmarks and metrics, particularly for localisation granularity (e.g., pixel-wise vs. region-wise accuracy) and real-world robustness. Community-wide efforts to establish unified evaluation protocols would better guide fu...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.