pith. sign in

arxiv: 2606.29807 · v1 · pith:YBEJSJGInew · submitted 2026-06-29 · 💻 cs.CR · cs.CV

Rethinking Forgery Attacks on Semantic Watermarks in Black-Box Settings: A Geometric Distortion Perspective

Cheng-Yi Lee , Yichi Zhang , Yuchen Yang , Chun-Shien Lu , Jun-Cheng Chen This is my paper

Pith reviewed 2026-06-30 05:49 UTC · model grok-4.3

classification 💻 cs.CR cs.CV
keywords semantic watermarkslatent diffusion modelsblack-box forgerygeometric distortionforgery detectionrate-distortionlatent manifold
0
0 comments X

The pith

Structural mismatches between proxy and target models impose an irreducible distortion floor on black-box semantic watermark forgeries in latent diffusion models.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper examines why forgery attacks on semantic watermarks, which embed data in the initial noise of latent diffusion models, often fail to produce high-fidelity copies in black-box settings. It applies a rate-distortion analysis to the latent space and finds that differences in model structure between the attacker's proxy and the true target create a fixed lower bound on distortion. This bound appears as consistent geometric shifts and shape changes on the latent manifold instead of random variation. The authors use this pattern to build a detection step that flags forged outputs before any watermark check occurs.

Core claim

An irreducible distortion floor exists because of structural mismatches between proxy and target models; this floor takes the form of global drift and local deformation on the latent manifold rather than stochastic noise, which limits how faithfully a forged watermark can match an authentic one and enables a scheme-agnostic detector to separate the two classes.

What carries the argument

rate-distortion analysis of the latent manifold that isolates structured geometric deviations (global drift plus local deformation) caused by proxy-target model mismatch

If this is right

  • Forged watermarks cannot reach the fidelity of authentic ones when the attacker must use a different model architecture.
  • The geometric character of the mismatch distortion allows pre-verification detection without knowledge of the specific watermarking scheme.
  • The detection remains effective across varied black-box attack setups while tolerating ordinary image distortions.
  • Any successful forgery must reduce both global drift and local deformation to stay below the detection threshold.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Watermark designers may need to incorporate explicit robustness against model-architecture mismatch rather than only against noise or editing.
  • The same geometric signature could appear in other generative pipelines that rely on shared latent spaces, suggesting the detection idea might transfer beyond diffusion models.
  • Attackers could respond by training proxies that minimize measured drift and deformation rather than simply maximizing watermark extraction accuracy.

Load-bearing premise

The distortion created by model mismatch appears as structured geometric deviations rather than random noise, so a detector can separate forged from authentic samples before watermark verification.

What would settle it

A controlled test in which forged samples generated with multiple proxy-target pairs are measured for global drift and local deformation; if the measurements overlap completely with those of authentic samples and no separation occurs, the claimed distortion floor and detection method do not hold.

Figures

Figures reproduced from arXiv: 2606.29807 by Cheng-Yi Lee, Chun-Shien Lu, Jun-Cheng Chen, Yichi Zhang, Yuchen Yang.

Figure 1
Figure 1. Figure 1: Illustration of the black-box forgery attack. An adversary uses a proxy model to invert a watermarked image into a latent representation and generates a forged image via different strategies, which preserves the service provider’s watermark while falsifying content provenance. media (Goodman, 2024) and the dissemination of misinfor￾mation (Jaidka et al., 2025). For instance, deepfakes (Wester￾lund, 2019), … view at source ↗
Figure 2
Figure 2. Figure 2: ), which fundamentally limits the adversary’s ability to achieve high-fidelity forgery. Crucially, we find that this distortion is not stochastic noise, but manifests as structured geometric deviations from the intrinsic latent manifold, char￾acterized by global drift and local deformation. By lever￾aging these geometric differences, forged samples can be detected as a pre-verification step, without requir… view at source ↗
Figure 3
Figure 3. Figure 3: Geometric interpretation of latent discrepancy. (a) il￾lustrates angular separation θ on a hypersphere; (b) depicts the contrast between the geodesic distance (red solid line) and the Euclidean distance (blue dashed line) on the SPD manifold, with the dark blue dot denoting the watermarked latent z (w) T . provides a standard control of their total-variation discrep￾ancy in terms of the KL mismatch. Theref… view at source ↗
Figure 5
Figure 5. Figure 5: Distributions of cosine similarity for watermarked and forged samples, under the same attack setting as in [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Cosine similarity distributions under optimization-based attacks, with increasing numbers of optimization iterations on the TR. The target and proxy models are consistent with [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Similarity distributions under 14 types of image distor￾tions for TR and GS, evaluated on SDXL as the target model. ther compare against a local cosine baseline (i.e., grid size 16, mean aggregation), which also achieves competitive performance in both cross-model and identical-model set￾tings (e.g., AUC = 1.000 for both TR and GS under SDXL → SD2.1, and 0.954/0.976 in the identical-model setting). These r… view at source ↗
Figure 8
Figure 8. Figure 8: Illustration of the proposed detection framework. C.5. Connection between Removal Guarantees and Distortion Bounds Recent work (Zhao et al., 2024) theoretically analyzes the removability of invisible watermarks under stochastic regeneration perturbations in diffusion models. Although that work focuses on post-hoc watermarking, whereas our work studies in-generation watermark schemes, both are related throu… view at source ↗
Figure 9
Figure 9. Figure 9: Visual examples of the post-processing distortions used in our robustness evaluation. (a) illustrates signal processing distortions such as JPEG compression levels and noise intensities, while (b) shows geometric and erasure-based attacks. • Signal Processing Attacks: We consider a broad spectrum of signal distortions with varying intensities to simulate realistic transmission and editing artifacts. These … view at source ↗
Figure 10
Figure 10. Figure 10: Distributions of local SPD (AIRM) distances for unwatermarked, watermarked, and forged samples. The results are shown for SDXL as the target model and SD2.1 as the proxy. (a) TR (b) GS [PITH_FULL_IMAGE:figures/full_fig_p022_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Distributions of local SPD distances for watermarked and forged samples, under the FLUX (target) and SD3 (proxy). (a) TR (b) GS [PITH_FULL_IMAGE:figures/full_fig_p022_11.png] view at source ↗
Figure 13
Figure 13. Figure 13: MSE distributions under 14 types of image distortions for TR and GS, evaluated on SDXL as the target model. samples, enabling robust detection across a broader range of attack scenarios. We omit the analysis of optimization-based attacks here, as they exhibit marginal attack efficacy compared to guidance-based methods [PITH_FULL_IMAGE:figures/full_fig_p023_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Examples of guidance-based attacks on different target models and watermarking schemes. The top row shows watermarked images, generated using prompts from the SDP dataset with the target model indicated at the top of each panel. The subsequent rows present the corresponding forged outputs, where the adversary employs SD2.1 and SD3 as proxy models to perform forgery [PITH_FULL_IMAGE:figures/full_fig_p024_… view at source ↗
Figure 15
Figure 15. Figure 15: Progression of optimization-based forgery attack as the number of optimization steps increases. The target model is SDXL, and the forged watermark is TR and GS. Results are conducted on SDXL with TR and GS watermarks and four cover images x (c) . For each cover image, the top row shows the initial cover and the corresponding forged images at different optimization steps, while the bottom row illustrates t… view at source ↗
read the original abstract

Recent studies have shown that semantic watermarks, which embed information into the initial noise of latent diffusion models (LDMs), are vulnerable to black-box forgery attacks. However, existing methods primarily rely on empirical evidence and lack a rigorous theoretical understanding of the conditions under which such attacks succeed or fail. To bridge this gap, we rethink the nature of such attacks through the lens of rate-distortion in the latent space. Our analysis identifies an irreducible distortion floor due to structural mismatches between proxy and target models, which fundamentally limits the fidelity of forged watermarks. We further characterize this distortion as structured geometric deviations on the latent manifold, in the form of global drift and local deformation rather than stochastic noise. Leveraging these insights, we propose a scheme-agnostic detection method that distinguishes forged samples before watermark verification. Extensive experiments demonstrate the effectiveness of our method across diverse black-box scenarios, while preserving robustness to common distortions.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 0 minor

Summary. The paper claims that by analyzing forgery attacks on semantic watermarks through rate-distortion theory in the latent space of LDMs, there is an irreducible distortion floor due to structural mismatches between proxy and target models. This distortion is characterized as structured geometric deviations (global drift and local deformation) on the latent manifold rather than stochastic noise. A scheme-agnostic detection method is proposed to distinguish forged samples before watermark verification, supported by extensive experiments across diverse black-box scenarios.

Significance. If the analysis holds, it provides a theoretical basis for the limits of black-box forgery attacks on semantic watermarks, which is significant for the security of generative models. The geometric characterization and the proposed detector could advance the field by offering a more robust way to detect forgeries.

major comments (1)
  1. The abstract states a theoretical analysis and experimental validation, yet provides no equations, derivation steps, or data details; the central claim of an irreducible floor therefore cannot be evaluated from the given text.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for their review and for noting the potential significance of the geometric distortion analysis for semantic watermark security. We respond to the major comment below.

read point-by-point responses

  1. Referee: The abstract states a theoretical analysis and experimental validation, yet provides no equations, derivation steps, or data details; the central claim of an irreducible floor therefore cannot be evaluated from the given text.

    Authors: Abstracts are intentionally concise summaries and do not contain equations or derivations; these appear in the full manuscript. The rate-distortion analysis deriving the irreducible distortion floor from proxy-target mismatches is presented with all steps in Section 3, the geometric characterization (global drift and local deformation on the latent manifold) in Section 4, and the experimental data/details across black-box scenarios in Section 5. The central claim is therefore fully evaluable from the complete manuscript. We do not view this as requiring a change to the abstract, which follows standard conventions. revision: no

Circularity Check

0 steps flagged

No significant circularity identified

full rationale

The abstract and provided context present a rate-distortion analysis in latent space that derives an irreducible distortion floor from structural mismatches between proxy and target models, then characterizes it geometrically as global drift and local deformation. No equations, self-citations, or steps are quoted that reduce a claimed prediction or uniqueness result to a fitted input or prior self-work by construction. The detector follows from the stated characterization without evidence of self-definitional loops or renaming of known results. The derivation is scoped to the examined black-box setting and appears self-contained.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review supplies no information on free parameters, axioms, or invented entities; ledger left empty.

pith-pipeline@v0.9.1-grok · 5701 in / 994 out tokens · 26378 ms · 2026-06-30T05:49:48.212508+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

68 extracted references · 3 canonical work pages · 1 internal anchor

  1. [1]

    ACM Computing Surveys , volume=

    A survey of ai-generated content (aigc) , author=. ACM Computing Surveys , volume=. 2025 , publisher=

    2025

  2. [2]

    Computer Science

    Improving image generation with better captions , author=. Computer Science. https://cdn.openai.com/papers/dall-e-3.pdf , volume=

  3. [3]

    Midjourney (Version 7.0) , year =

  4. [4]

    Technology innovation management review , volume=

    The emergence of deepfake technology: A review , author=. Technology innovation management review , volume=

  5. [5]

    2023 , note =

    Executive order on the safe, secure, and trustworthy development and use of artificial intelligence , author =. 2023 , note =

    2023

  6. [6]

    2024 , note =

    Artificial Intelligence Act: Regulation (EU) 2024/1689 of the European Parliament and of the Council , author =. 2024 , note =

    2024

  7. [7]

    2024 , note =

    California Assembly Bill AB-3211 California Digital Content Provenance Standards , author =. 2024 , note =

    2024

  8. [8]

    Digital Government: Research and Practice , volume=

    Misinformation, disinformation, and generative AI: Implications for perception and policy , author=. Digital Government: Research and Practice , volume=. 2025 , publisher=

    2025

  9. [9]

    Synthetic Content: Default to Distrust , author=. Case W. Rsrv. L. Rev. , volume=. 2024 , publisher=

    2024

  10. [10]

    Journal of Computer Science , volume=

    Combined DWT-DCT digital image watermarking , author=. Journal of Computer Science , volume=

  11. [11]

    IEEE SPL , volume=

    A generalization of LSB matching , author=. IEEE SPL , volume=. 2009 , publisher=

    2009

  12. [12]

    2008 , publisher=

    Digital watermarking , author=. 2008 , publisher=

    2008

  13. [13]

    ICIP , volume=

    Rotation, scale and translation invariant digital image watermarking , author=. ICIP , volume=. 1997 , organization=

    1997

  14. [14]

    CVPR , pages=

    Rosteals: Robust steganography using autoencoder latent space , author=. CVPR , pages=

  15. [15]

    NeurIPS , volume =

    Tree-Rings Watermarks: Invisible Fingerprints for Diffusion Images , author =. NeurIPS , volume =

  16. [16]

    ECCV , pages=

    Ringid: Rethinking tree-ring watermarking for enhanced multi-key identification , author=. ECCV , pages=. 2024 , organization=

    2024

  17. [17]

    CVPR , pages=

    Gaussian shading: Provable performance-lossless image watermarking for diffusion models , author=. CVPR , pages=

  18. [18]

    Workshop Record of SASC , pages=

    ChaCha, a variant of Salsa20 , author=. Workshop Record of SASC , pages=

  19. [19]

    ICLR , year=

    An Undetectable Watermark for Generative Image Models , author=. ICLR , year=

  20. [20]

    CRYPTO , pages=

    Pseudorandom error-correcting codes , author=. CRYPTO , pages=. 2024 , organization=

    2024

  21. [21]

    Cryptanalysis of LDPC-Based Pseudorandom Error-Correcting Codes

    Cryptanalysis of Pseudorandom Error-Correcting Codes , author=. arXiv preprint arXiv:2512.17310 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  22. [22]

    ACM CCS , pages=

    Removal Attack and Defense on AI-generated Content Latent-based Watermarking , author=. ACM CCS , pages=

  23. [23]

    ICCV , pages=

    TAG-WM: Tamper-Aware Generative Image Watermarking via Diffusion Inversion Sensitivity , author=. ICCV , pages=

  24. [24]

    ICCV , pages=

    Semantic Watermarking Reinvented: Enhancing Robustness and Generation Quality with Fourier Integrity , author=. ICCV , pages=

  25. [25]

    ICML , year=

    GaussMarker: Robust Dual-Domain Watermark for Diffusion Models , author=. ICML , year=

  26. [26]

    NeurIPS , year=

    Shallow Diffuse: Robust and Invisible Watermarking through Low-Dim Subspaces in Diffusion Models , author=. NeurIPS , year=

  27. [27]

    ICCV , month=

    Fernandez, Pierre and Couairon, Guillaume and J\'egou, Herv\'e and Douze, Matthijs and Furon, Teddy , title=. ICCV , month=. 2023 , pages=

    2023

  28. [28]

    ICLR , year=

    Watermark Anything With Localized Messages , author=. ICLR , year=

  29. [29]

    Weitao Feng and Wenbo Zhou and Jiyan He and Jie Zhang and Tianyi Wei and Guanlin Li and Tianwei Zhang and Weiming Zhang and Nenghai Yu , booktitle=. AquaLo

  30. [30]

    CVPR , pages=

    Black-box forgery attacks on semantic watermarks for diffusion models , author=. CVPR , pages=

  31. [31]

    arXiv preprint arXiv:2506.06018 , year=

    Optimization-Free Universal Watermark Forgery with Regenerative Diffusion Models , author=. arXiv preprint arXiv:2506.06018 , year=

    work page arXiv

  32. [32]

    Robustness of

    Mehrdad Saberi and Vinu Sankar Sadasivan and Keivan Rezaei and Aounon Kumar and Atoosa Chegini and Wenxiao Wang and Soheil Feizi , booktitle=. Robustness of

  33. [33]

    NeurIPS , year=

    Can Simple Averaging Defeat Modern Watermarks? , author=. NeurIPS , year=

  34. [34]

    Ziping Dong and Chao Shuai and Zhongjie Ba and Peng Cheng and Zhan Qin and Qinglong Wang and Kui Ren , booktitle=

  35. [35]

    NeurIPS , volume=

    Denoising diffusion probabilistic models , author=. NeurIPS , volume=

  36. [36]

    ICLR , year =

    Jiaming Song and Chenlin Meng and Stefano Ermon , title =. ICLR , year =

  37. [37]

    CVPR , pages=

    High-resolution image synthesis with latent diffusion models , author=. CVPR , pages=

  38. [38]

    ICLR , year=

    SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis , author=. ICLR , year=

  39. [39]

    ICLR , year=

    PixArt- : Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis , author=. ICLR , year=

  40. [40]

    ICML , year=

    Scaling rectified flow transformers for high-resolution image synthesis , author=. ICML , year=

  41. [41]

    2024 , howpublished=

    Black Forest Labs , title=. 2024 , howpublished=

    2024

  42. [42]

    ICCV , pages=

    Scalable diffusion models with transformers , author=. ICCV , pages=

  43. [43]

    MICCAI , pages=

    U-net: Convolutional networks for biomedical image segmentation , author=. MICCAI , pages=. 2015 , organization=

    2015

  44. [44]

    ICLR , year=

    Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow , author=. ICLR , year=

  45. [45]

    2012 , publisher=

    Elements of Information Theory , author=. 2012 , publisher=

    2012

  46. [46]

    The Bell system technical journal , volume=

    A mathematical theory of communication , author=. The Bell system technical journal , volume=. 1948 , publisher=

    1948

  47. [47]

    Coding theorems for a discrete source with a fidelity criterion , author=. IRE Nat. Conv. Rec , volume=

  48. [48]

    2013 , publisher=

    Information-spectrum methods in information theory , author=. 2013 , publisher=

    2013

  49. [49]

    CVPR , pages=

    The perception-distortion tradeoff , author=. CVPR , pages=

  50. [50]

    ICML , pages=

    Rethinking lossy compression: The rate-distortion-perception tradeoff , author=. ICML , pages=. 2019 , organization=

    2019

  51. [51]

    NeurIPS , volume=

    Universal rate-distortion-perception representations for lossy compression , author=. NeurIPS , volume=

  52. [52]

    IEEE Journal of Selected Topics in Signal Processing , volume=

    Nonlinear transform coding , author=. IEEE Journal of Selected Topics in Signal Processing , volume=. 2020 , publisher=

    2020

  53. [53]

    ICCV , pages=

    Adding conditional control to text-to-image diffusion models , author=. ICCV , pages=

  54. [54]

    ECCV , pages=

    Microsoft coco: Common objects in context , author=. ECCV , pages=. 2014 , organization=

    2014

  55. [55]

    ACL , year=

    DiffusionDB: A Large-scale Prompt Gallery Dataset for Text-to-Image Generative Models , author=. ACL , year=

  56. [56]

    Positive Definite Matrices , year=

    Positive definite matrices , author=. Positive Definite Matrices , year=

  57. [57]

    AAAI , volume=

    A Riemannian network for SPD matrix learning , author=. AAAI , volume=

  58. [58]

    NeurIPS , volume=

    Riemannian batch normalization for SPD neural networks , author=. NeurIPS , volume=

  59. [59]

    NeuroImage , volume=

    Re-visiting Riemannian geometry of symmetric positive definite matrices for the analysis of functional connectivity , author=. NeuroImage , volume=. 2021 , publisher=

    2021

  60. [60]

    2006 , publisher=

    A Riemannian framework for tensor computing , author=. 2006 , publisher=

    2006

  61. [61]

    ICLR , year=

    VideoShield: Regulating Diffusion-based Video Generation Models via Watermarking , author=. ICLR , year=

  62. [62]

    Foundations of computational mathematics , volume=

    Gromov--Wasserstein distances and the metric approach to object matching , author=. Foundations of computational mathematics , volume=. 2011 , publisher=

    2011

  63. [63]

    IJCV , volume=

    Distinctive image features from scale-invariant keypoints , author=. IJCV , volume=. 2004 , publisher=

    2004

  64. [64]

    ICCV , pages=

    ORB: An efficient alternative to SIFT or SURF , author=. ICCV , pages=

  65. [65]

    NeurIPS , volume=

    Emergent correspondence from image diffusion , author=. NeurIPS , volume=

  66. [66]

    Journal of the American Statistical Association , volume=

    Tweedie's formula and selection bias , author=. Journal of the American Statistical Association , volume=. 2011 , publisher=

    2011

  67. [67]

    Understanding diffusion models: A unified perspective,

    Understanding diffusion models: A unified perspective , author=. arXiv preprint arXiv:2208.11970 , year=

    work page arXiv

  68. [68]

    NeurIPS , volume=

    Invisible image watermarks are provably removable using generative ai , author=. NeurIPS , volume=