Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Invisible Watermarks: Attacks and Robustness

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Pith's one-line read Two watermarking schemes can be stacked and decoded independently via a learned remover network, while a GradCAM-guided localized blurring attack removes StegaStamp with roughly half the image degradation of uniform blurring.

desk verdict A course project with a plausible LBA idea and a stacked-watermark remover network, but the remover's central claim is untested and the evaluation has several unaddressed gaps. read the letter →

arxiv 2412.12511 v1 pith:4EPZ5TE4 submitted 2024-12-17 cs.CV

classification cs.CV
keywords invisiblewatermarksStegaStampTree-RingwatermarkstackingremovalnetworklocalizedblurringattackGradCAMAI-generatedimageprovenance
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to settle two practical questions about invisible watermarks for AI-generated images: whether two different watermarking schemes can be layered on one image without destroying each other, and whether an attack can remove a watermark without visibly wrecking the picture. To answer the first, the authors stack Tree-Ring (a latent-space watermark embedded during diffusion generation) on top of StegaStamp (a pixel-space post-processing watermark) and train a small 'remover' network that strips the StegaStamp signal before Tree-Ring decoding; the stacked pipeline's Tree-Ring AUC recovers from 0.9936 under naive stacking to 0.9950, against an unattacked baseline of 0.9956. To answer the second, they propose the Localized Blurring Attack, which uses a GradCAM heatmap from the StegaStamp decoder to blur only the pixels that carry the watermark; at a 50% threshold and kernel size 31 it reaches a detection rate of 0.0554 with a Fréchet Inception Distance of 88.1, versus straight blurring's 0.0000 at 150.24. If these results hold, both sides gain a new lever: layered watermarks become practical with a separation step, and attackers can trade much less image quality for comparable watermark removal.

What carries the argument

The argument turns on two mechanisms. The first is the remover network: a small trained CNN that takes the stacked image $I_{TR+SS}$ and minimizes $\frac{1}{N}\sum_{i=1}^{N} \|I^{(i)}_{TR\_Removed} - I^{(i)}_{TR}\|_2^2$, learning to cancel exactly the StegaStamp perturbation so that Tree-Ring decoding sees a clean latent-space watermark; this is what converts an impractical naive stack into a working two-watermark pipeline. The second is the Localized Blurring Attack: a GradCAM heatmap is computed from the StegaStamp decoder, pixels above a chosen percentile (0, 25, 50, or 75) are thresholded into a binary mask, and only those regions are replaced with blurred pixels, so the attack spends its image-quality budget on the few areas the decoder actually relies on. The paper emphasizes an asymmetry that makes both results possible: Tree-Ring alters pixel space strongly (image $\ell^2$ distance 117.58) while StegaStamp displaces the latent representation more (latent $\ell^2$ distance 118.17), which is why one modality can be selectively removed and the other selectively attacked.

What would settle it

Run the Localized Blurring Attack using a GradCAM heatmap computed from a surrogate StegaStamp decoder trained on watermarked and unwatermarked pairs without the real weights, then measure detection with the true decoder: if detection stays near 1.000 at kernel 31, the attack's quality advantage is an artifact of white-box access. Separately, re-run the stacked Tree-Ring pipeline's rotation test with a Tree-Ring implementation that reproduces the original paper's 0.935 rotation AUC; if stacking and removal still drive rotation detection to 0.000, the robustness ceiling is real, but if the reproduction gap is the cause, the rotation conclusion collapses.

Watch

Extended reading notes

Core claim

On its own terms, the paper's discovery is that the two watermark modalities are separable in practice: Tree-Ring and StegaStamp perturb different feature spaces, so a direct sequential stack makes Tree-Ring undetectable, but a network trained to undo the StegaStamp perturbation restores clean Tree-Ring decoding. The authors train a remover network to map the stacked image back to the Tree-Ring-only image under an L2 loss; after removal, Tree-Ring detection returns to nearly its unattacked level (AUC 0.9950 versus 0.9956, TPR@1%FPR 0.96) while StegaStamp bit accuracy holds at 0.998. The paper's second claim is that watermarked information is spatially concentrated: StegaStamp residuals sit in high-contrast salient regions, and a GradCAM-derived binary mask that blurs only those regions is nearly as effective as full-image blurring while doing far less damage, cutting detection to 0.0554 at a 50% threshold and kernel 31 with an FID of 88.1 against straight blur's 0.0000 at 150.24. The paper reports these results with its own caveats, including that all methods fail rotation and that its Tree-Ring reproduction scores far lower on rotated images than the original paper does.

Load-bearing premise

Both headline results depend on white-box access that the paper itself flags as fragile: the localized blurring attack needs a GradCAM heatmap from the exact StegaStamp decoder, which Section 8.2 concedes an attacker may have to approximate with unknown consequences, and the stacked pipeline's robustness conclusions inherit a Tree-Ring reproduction whose rotation AUC of 0.463 sits far below the original's 0.935 (Table 3).

Editorial extensions

If this is right

  • Multi-watermark images become practical: a decoder-side remover lets a latent-space watermark and a pixel-space watermark be layered on one image and each decoded cleanly, with Tree-Ring AUC recovering from 0.9936 (naive stack) to 0.9950.
  • Attackers can buy stealth with less damage: at percentile 50 and kernel 31, LBA cuts StegaStamp detection to 0.0554 at FID 88.1, about half the image degradation of straight blurring at comparable attack strength.
  • Heatmap localization is a general threat model: any watermark whose information concentrates in salient regions is vulnerable to attention-guided attacks, so the paper concludes that open-source decoders must be kept private.
  • Stacking does not fix weaknesses neither component handles: every stacked variant fails rotation with detection 0.000, exactly like each watermark alone.
  • Blur is not the final attack: regeneration still beats LBA (detection 0.0100 at FID 16.3), so the paper's own comparison implies that localized versions of less destructive attacks would combine the best of both.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A concrete testable extension the paper leaves implicit: applying the LBA mask to a regeneration attack instead of blur should push FID below LBA's 36.1 at similar detection suppression, since regeneration already preserves quality and the mask tells the attack where to spend its noise.
  • The remover's success suggests a general modality-stripping principle: any pair of watermarks living in different feature spaces, such as Stable Signature and Tree-Ring, might be stackable with a similar L2-trained separator, but the paper only demonstrates the StegaStamp-Tree-Ring pair.
  • The rotation reproduction gap (0.463 versus 0.935 AUC) means the stacked pipeline's rotation conclusions currently rest on a weaker baseline than the original; until the original Tree-Ring model is run through the stacking and remover pipeline, rotation robustness should be treated as unverified.
  • If the residual blur-gap (remover detection 0.168 versus Tree-Ring alone 0.218) comes from leftover latent-space interference, adding a latent-space loss term to the remover's training objective should measurably close that gap, a prediction that follows from the paper's own diagnosis and is directly testable.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes two contributions to invisible-image-watermark robustness: (1) a multi-watermark pipeline that stacks Tree-Ring and StegaStamp and uses a learned 'remover' network trained with an L2 loss to strip the StegaStamp component before Tree-Ring decoding, and (2) a Localized Blurring Attack (LBA) that uses GradCAM on the StegaStamp decoder to blur only high-attention regions, supposedly reducing image degradation relative to full-image blurring. The evaluation uses MS-COCO validation images and reports bit accuracy, AUC, TPR at fixed FPR, p-values, detection rates under blurring/rotation, and FID. The main claims are that the remover 'completely removes' the StegaStamp watermark while preserving Tree-Ring detectability, and that LBA achieves comparable attack success with less image degradation than uniform blurring.

Significance. If the claims were fully supported, the paper would address a real gap: most watermark-attack work considers full-image perturbations, and combining complementary image-space and latent-space watermarks is a plausible robustness strategy. The release of code and the reproduction of StegaStamp, Tree-Ring, and regeneration baselines are strengths, as is the attempt to quantify both attack success and image quality with FID. However, the central empirical claims are currently under-supported: the StegaStamp-removal claim is never directly measured, the LBA comparison is not at matched operating points, and the reproduced Tree-Ring rotation baseline is far weaker than the original, undermining rotation-related conclusions. The contributions are therefore more in the nature of a proof-of-concept that requires substantial additional experiments before the stated conclusions can be accepted.

major comments (4)
  1. [§4.1.2, Eq. (3), Table 9] The abstract and Section 4.1.2 claim that the remover network 'completely removes' the StegaStamp watermark, but the paper never reports StegaStamp bit accuracy or detection rate on the remover's output. Table 9 lists a detection rate of 1.000 for the 'Remover' row under 'None'; if this rate refers to StegaStamp detection, it indicates that StegaStamp remains fully decodable after the remover, contradicting the removal claim. Because the loss in Eq. (3) only minimizes L2 distance to the Tree-Ring-only image, the remover has no explicit incentive to destroy the StegaStamp message. The authors should report StegaStamp bit accuracy and detection rate on ITR_Removed, compare with the pre-removal values, and ideally test detection after re-encoding or other realistic decoding conditions.
  2. [§3.1.2, Table 3] The reproduced Tree-Ring model achieves a rotation-attack AUC of 0.463, whereas the original Tree-Ring paper reports 0.935 (Table 3). This large discrepancy means the paper's rotation results, including the claim in Section 7.1 that 'all methods fail to detect watermarks under Rotation' and the corresponding row in Table 9, are not a trustworthy basis for conclusions about the stacked pipeline's robustness. The authors need to either repair the rotation reproduction (e.g., verify the renoising procedure and rotation handling) or explicitly restrict their rotation claims to their own implementation and avoid drawing general conclusions from that setting.
  3. [§7.2, Table 10] The LBA versus straight-blurring comparison is not made at matched detection rates, so the headline claim that LBA 'degrades the image significantly less' is confounded by attack strength. For example, LBA at percentile 50 with kernel 31 attains detection rate 0.0554 and FID 88.1, while straight blurring at kernel 31 attains detection rate 0.0000 and FID 150.24; the straight-blur attack is much stronger, so its larger FID does not demonstrate that LBA is a better attack at equal effectiveness. The comparison should be conducted at matched detection rates (e.g., by adjusting the percentile or kernel to obtain comparable TPRs) or with a full operating-characteristic analysis. In addition, all tables report single runs with no variance or significance testing, which is not sufficient to support quantitative comparisons such as the remover's AUC recovery from 0.9936 to 0.9950.
  4. [§7.2 and §8.2] The paper acknowledges in Section 8.2 that LBA assumes access to the exact StegaStamp decoder, and the evaluation is entirely white-box. This is not circular, but it substantially limits the practical claim: the GradCAM map is computed from the same decoder that is later used to measure attack success, and the paper provides no evidence about how LBA behaves when the attacker must approximate or retrain the decoder. The authors should frame LBA as a white-box proof-of-concept and, ideally, add a sensitivity experiment with a surrogate decoder or with decoder-mismatch conditions.
minor comments (5)
  1. [§Algorithm 1] The notation in Algorithm 1 is confusing: lines 4 and 6 use 'IF F T' and 'IFFT' for the same transform, and the variable names suggest the FFT is inverted before embedding the watermark; please use consistent transform names and clarify the direction of each transform.
  2. [§6.2 and Table 4] The manuscript uses both TPR@1%FPR and TPR@0.1%FPR without explaining when each is used; Table 4 reports TPR@0.1%FPR while Section 6.2 defines only TPR@1%FPR. Please align the definitions and the table headers.
  3. [§Table 9] The definition of 'Detection Rate' in Table 9 is ambiguous because it mixes StegaStamp decoding and Tree-Ring hypothesis testing with different thresholds. Please specify for each row whether the rate is bit-accuracy, message-decoding success, or Tree-Ring p<0.01 detection, and give the corresponding false-positive rate.
  4. [§7.1] The text says 'stacking watermarks does not interfere with each other' and later that the remover 'effectively isolates and enhances' the Tree-Ring watermark, but the measured AUC changes are tiny (0.9956 to 0.9936 to 0.9950) and no uncertainty is reported; please temper the language or provide confidence intervals.
  5. [§5] The dataset section refers to 'MS-COCO' while the rest of the paper uses 'MSCOCO'; please standardize the spelling.

Circularity Check

1 steps flagged · score 3.0 of 10

Remover's Tree-Ring preservation is partly by construction; LBA and stacking results remain independent.

  1. self definitional [Section 4.1.2, Eq. (3); Section 7.1, Table 7(b)]
    "The network minimizes the L2 distance between the output image, ITR_Removed, and the original Tree-ring watermarked image, ITR. ... After applying a watermark remover, the AUC recovers to 0.9950, and the TPR@1%FPR remains at 0.96."

    The remover is trained with Eq. (3) to make its output pixel-wise approximate ITR, the Tree-Ring-only watermarked image. The reported evidence for the remover's benefit is Tree-Ring AUC/TPR on that output, so the Tree-Ring decoder is being tested on a fitted reconstruction of its own target image. This is not an independent prediction of Tree-Ring preservation; success is substantially built into the training objective. Additionally, no StegaStamp bit-accuracy or detection result after removal is reported (Table 7(a) omits the Remover row), and Table 9's 'None' detection rate for the Remover is 1.000, so the 'completely removes the other modality' half of the claim is unverified.

full rationale

The paper contains no self-citations and invokes no uniqueness theorems, so the self-citation patterns do not apply. The LBA attack uses GradCAM from the exact StegaStamp decoder it attacks, but this is an explicitly acknowledged white-box assumption (Section 8.2) and is compared against randomized masking and straight blurring, giving independent empirical content rather than circularity. The Tree-Ring rotation reproduction gap (Table 3) is disclosed and discussed as a limitation, not hidden. The only partially circular element is the remover evaluation, where the training objective directly targets the Tree-Ring-only image and the headline Tree-Ring metrics are measured on that output; this makes the 'preserves Tree-Ring' claim partly self-definitional, and the 'removes StegaStamp' claim is not separately tested. Overall, the LBA contribution and the stacking robustness measurements stand on their own, so the circularity is partial and modest rather than pervasive.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central claims depend on two hand-controlled attack knobs (percentile, kernel size) and on white-box access to the StegaStamp decoder; the remover is a trained supervised model rather than an invented physical entity.

free parameters (2)
  • LBA percentile threshold = 0, 25, 50, 75
    Hand-selected sweep; controls the fraction of high-gradient pixels blurred.
  • LBA blur kernel size = 5, 11, 31
    Hand-selected sweep; controls blur strength, directly trades off detection rate and FID.
assumptions (3)
  • domain assumption GradCAM from the exact StegaStamp decoder highlights pixels whose alteration most affects watermark decoding.
    Section 7.2 and Figure 5; the attack and its evaluation both use the same decoder, so transfer to unseen decoders is untested.
  • domain assumption The remover can be trained on pairs (Tree-Ring+StegaStamp image, Tree-Ring-only image) and this supervision transfers to new images.
    Section 4.1.2 and Algorithm 1; requires access to clean Tree-Ring outputs during training.
  • ad hoc to paper Tree-Ring detection threshold at p<0.01 yields the reported detection rates.
    Section 6.3 defines detection rate using p-value below 0.01; threshold chosen rather than derived.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Invisible Watermarks: Attacks and Robustness." pith.science (2026). https://pith.science/paper/4EPZ5TE4

@misc{pith2026241212511,
  author       = {Pith},
  title        = {Pith review of: Invisible Watermarks: Attacks and Robustness},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4EPZ5TE4}},
  note         = {Machine review of arXiv:2412.12511}
}
read the original abstract

As Generative AI continues to become more accessible, the case for robust detection of generated images in order to combat misinformation is stronger than ever. Invisible watermarking methods act as identifiers of generated content, embedding image- and latent-space messages that are robust to many forms of perturbations. The majority of current research investigates full-image attacks against images with a single watermarking method applied. We introduce novel improvements to watermarking robustness as well as minimizing degradation on image quality during attack. Firstly, we examine the application of both image-space and latent-space watermarking methods on a single image, where we propose a custom watermark remover network which preserves one of the watermarking modalities while completely removing the other during decoding. Then, we investigate localized blurring attacks (LBA) on watermarked images based on the GradCAM heatmap acquired from the watermark decoder in order to reduce the amount of degradation to the target image. Our evaluation suggests that 1) implementing the watermark remover model to preserve one of the watermark modalities when decoding the other modality slightly improves on the baseline performance, and that 2) LBA degrades the image significantly less compared to uniform blurring of the entire image. Code is available at: https://github.com/tomputer-g/IDL_WAR

Figures

Figures reproduced from arXiv: 2412.12511 by the authors.

Figure 1
Figure 1. Overview of the naive stacking of Tree-ring and StegaStamp watermarking pipeline. A straightforward approach to creating a robust watermark involves stacking two techniques. As shown in [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Overview of the stacking of the modified Tree-ring and StegaStamp watermarking pipeline with the remover network. Algorithm 1 Stacking Tree-ring and StegaStamp Watermarking Pipeline 1: Input: Original image I, Tree-ring message MT R, StegaStamp message MSS 2: Output: Watermarked image IT R+SS, Decoded messages MT R, MSS 3: Step 1: Tree-ring Watermark Insertion 4: Compute the FFT of the image: IF F T ← FFT(I) 5: Embe… view at source ↗
Figure 3
Figure 3. Loss plot when training the stegastamp remover network [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: shows that StegaStamp residuals (the difference between watermarked and unwatermarked images) are concentrated in visually interesting areas with higher contrast, such as the kayak and riders, rather than in flatter regions like the water. The StegaStamp watermark deco…
Figure 5
Figure 5. Figure 5: Localized Blurring Attack pipeline. Key variables for this method include the kernel size and percentile threshold, which we will evaluate in the results section. 5 Dataset Description MS-COCO 2017 validation split. For the majority of our work, we evaluate the perform…
Figure 6
Figure 6. Figure 6: From top to bottom, the original watermarked Fourier space latents, renoised unattacked Fourier space latents, and renoised rotation attack Fourier space latents. We can see in (b) that there is still a bit of a ring-like pattern in the center of channel 4 prior to bei…
Figure 7
Figure 7. Figure 7: Example binary masking from GradCAM outputs using Percentile Thresholding. As the per￾centile threshold increases, smaller regions of the image are selected as the target for localized blurring attack. 13 [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. First-Place Solution to NeurIPS 2024 Invisible Watermark Removal Challenge

    cs.CV 2025-08 conditional novelty 4.0 of 10

    A competition-winning pipeline removes 95.7% of StegaStamp and TreeRing watermarks on the NeurIPS 2024 benchmark by combining VAE fine-tuning, diffusion purification, and translation tricks.

Reference graph

Works this paper leans on

13 extracted references · 4 canonical work pages · cited by 1 Pith paper

  1. [1]

    High- resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 10684–10695, 2022

  2. [2]

    Stegastamp: Invisible hyperlinks in physical photographs

    Matthew Tancik, Ben Mildenhall, and Ren Ng. Stegastamp: Invisible hyperlinks in physical photographs. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020

  3. [3]

    Tree-ring watermarks: Fin- gerprints for diffusion images that are invisible and robust

    Yuxin Wen, John Kirchenbauer, Jonas Geiping, and Tom Goldstein. Tree-ring watermarks: Fin- gerprints for diffusion images that are invisible and robust. arXiv preprint arXiv:2305.20030, 2023

  4. [4]

    The stable signature: Rooting watermarks in latent diffusion models

    Pierre Fernandez, Guillaume Couairon, Hervé Jégou, Matthijs Douze, and Teddy Furon. The stable signature: Rooting watermarks in latent diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 22466–22477, 2023

  5. [5]

    Benchmarking the ro- bustness of image watermarks

    Bang An, Mucong Ding, Tahseen Rabbani, Aakriti Agrawal, Yuancheng Xu, Chenghao Deng, Sicheng Zhu, Abdirisak Mohamed, Yuxin Wen, Tom Goldstein, et al. Benchmarking the ro- bustness of image watermarks. arXiv preprint arXiv:2401.08573, 2024

  6. [6]

    Invisible image watermarks are prov- ably removable using generative ai

    Xuandong Zhao, Kexun Zhang, Zihao Su, Saastha Vasan, Ilya Grishchenko, Christopher Kruegel, Giovanni Vigna, Yu-Xiang Wang, and Lei Li. Invisible image watermarks are prov- ably removable using generative ai. arXiv preprint arXiv:2306.01953, 2023

  7. [7]

    Deep unsu- pervised learning using nonequilibrium thermodynamics

    Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsu- pervised learning using nonequilibrium thermodynamics. In Francis Bach and David Blei, editors, Proceedings of the 32nd International Conference on Machine Learning , volume 37 of Proceedings of Machine Learning Research , pages 2256–2265, Lille, France, 07–09 Jul

  8. [8]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. CoRR, abs/2006.11239, 2020

Show all 13 references
  1. [9]

    Denoising diffusion implicit models.CoRR, abs/2010.02502, 2020

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models.CoRR, abs/2010.02502, 2020

  2. [10]

    Grad-cam: Visual explanations from deep networks via gradient- based localization

    Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient- based localization. In Proceedings of the IEEE international conference on computer vision , pages 618–626, 2017

  3. [11]

    Robustness of ai-image detectors: Fundamental limits and practical attacks

    Mehrdad Saberi, Vinu Sankar Sadasivan, Keivan Rezaei, Aounon Kumar, Atoosa Chegini, Wenxiao Wang, and Soheil Feizi. Robustness of ai-image detectors: Fundamental limits and practical attacks. arXiv preprint arXiv:2310.00076, 2023. 11

  4. [12]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Pi- otr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. InComputer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceeding...

  5. [13]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems , 30, 2017. A Appendix: Additional Figures (a) Origin...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.