REVIEW 4 major objections 5 minor 1 cited by
Invisible Watermarks: Attacks and Robustness
T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read Two watermarking schemes can be stacked and decoded independently via a learned remover network, while a GradCAM-guided localized blurring attack removes StegaStamp with roughly half the image degradation of uniform blurring.
desk verdict A course project with a plausible LBA idea and a stacked-watermark remover network, but the remover's central claim is untested and the evaluation has several unaddressed gaps. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The argument turns on two mechanisms. The first is the remover network: a small trained CNN that takes the stacked image $I_{TR+SS}$ and minimizes $\frac{1}{N}\sum_{i=1}^{N} \|I^{(i)}_{TR\_Removed} - I^{(i)}_{TR}\|_2^2$, learning to cancel exactly the StegaStamp perturbation so that Tree-Ring decoding sees a clean latent-space watermark; this is what converts an impractical naive stack into a working two-watermark pipeline. The second is the Localized Blurring Attack: a GradCAM heatmap is computed from the StegaStamp decoder, pixels above a chosen percentile (0, 25, 50, or 75) are thresholded into a binary mask, and only those regions are replaced with blurred pixels, so the attack spends its image-quality budget on the few areas the decoder actually relies on. The paper emphasizes an asymmetry that makes both results possible: Tree-Ring alters pixel space strongly (image $\ell^2$ distance 117.58) while StegaStamp displaces the latent representation more (latent $\ell^2$ distance 118.17), which is why one modality can be selectively removed and the other selectively attacked.
What would settle it
Run the Localized Blurring Attack using a GradCAM heatmap computed from a surrogate StegaStamp decoder trained on watermarked and unwatermarked pairs without the real weights, then measure detection with the true decoder: if detection stays near 1.000 at kernel 31, the attack's quality advantage is an artifact of white-box access. Separately, re-run the stacked Tree-Ring pipeline's rotation test with a Tree-Ring implementation that reproduces the original paper's 0.935 rotation AUC; if stacking and removal still drive rotation detection to 0.000, the robustness ceiling is real, but if the reproduction gap is the cause, the rotation conclusion collapses.
Extended reading notes
Core claim
On its own terms, the paper's discovery is that the two watermark modalities are separable in practice: Tree-Ring and StegaStamp perturb different feature spaces, so a direct sequential stack makes Tree-Ring undetectable, but a network trained to undo the StegaStamp perturbation restores clean Tree-Ring decoding. The authors train a remover network to map the stacked image back to the Tree-Ring-only image under an L2 loss; after removal, Tree-Ring detection returns to nearly its unattacked level (AUC 0.9950 versus 0.9956, TPR@1%FPR 0.96) while StegaStamp bit accuracy holds at 0.998. The paper's second claim is that watermarked information is spatially concentrated: StegaStamp residuals sit in high-contrast salient regions, and a GradCAM-derived binary mask that blurs only those regions is nearly as effective as full-image blurring while doing far less damage, cutting detection to 0.0554 at a 50% threshold and kernel 31 with an FID of 88.1 against straight blur's 0.0000 at 150.24. The paper reports these results with its own caveats, including that all methods fail rotation and that its Tree-Ring reproduction scores far lower on rotated images than the original paper does.
Load-bearing premise
Both headline results depend on white-box access that the paper itself flags as fragile: the localized blurring attack needs a GradCAM heatmap from the exact StegaStamp decoder, which Section 8.2 concedes an attacker may have to approximate with unknown consequences, and the stacked pipeline's robustness conclusions inherit a Tree-Ring reproduction whose rotation AUC of 0.463 sits far below the original's 0.935 (Table 3).
Editorial extensions
If this is right
- Multi-watermark images become practical: a decoder-side remover lets a latent-space watermark and a pixel-space watermark be layered on one image and each decoded cleanly, with Tree-Ring AUC recovering from 0.9936 (naive stack) to 0.9950.
- Attackers can buy stealth with less damage: at percentile 50 and kernel 31, LBA cuts StegaStamp detection to 0.0554 at FID 88.1, about half the image degradation of straight blurring at comparable attack strength.
- Heatmap localization is a general threat model: any watermark whose information concentrates in salient regions is vulnerable to attention-guided attacks, so the paper concludes that open-source decoders must be kept private.
- Stacking does not fix weaknesses neither component handles: every stacked variant fails rotation with detection 0.000, exactly like each watermark alone.
- Blur is not the final attack: regeneration still beats LBA (detection 0.0100 at FID 16.3), so the paper's own comparison implies that localized versions of less destructive attacks would combine the best of both.
Reading between the lines
- A concrete testable extension the paper leaves implicit: applying the LBA mask to a regeneration attack instead of blur should push FID below LBA's 36.1 at similar detection suppression, since regeneration already preserves quality and the mask tells the attack where to spend its noise.
- The remover's success suggests a general modality-stripping principle: any pair of watermarks living in different feature spaces, such as Stable Signature and Tree-Ring, might be stackable with a similar L2-trained separator, but the paper only demonstrates the StegaStamp-Tree-Ring pair.
- The rotation reproduction gap (0.463 versus 0.935 AUC) means the stacked pipeline's rotation conclusions currently rest on a weaker baseline than the original; until the original Tree-Ring model is run through the stacking and remover pipeline, rotation robustness should be treated as unverified.
- If the residual blur-gap (remover detection 0.168 versus Tree-Ring alone 0.218) comes from leftover latent-space interference, adding a latent-space loss term to the remover's training objective should measurably close that gap, a prediction that follows from the paper's own diagnosis and is directly testable.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes two contributions to invisible-image-watermark robustness: (1) a multi-watermark pipeline that stacks Tree-Ring and StegaStamp and uses a learned 'remover' network trained with an L2 loss to strip the StegaStamp component before Tree-Ring decoding, and (2) a Localized Blurring Attack (LBA) that uses GradCAM on the StegaStamp decoder to blur only high-attention regions, supposedly reducing image degradation relative to full-image blurring. The evaluation uses MS-COCO validation images and reports bit accuracy, AUC, TPR at fixed FPR, p-values, detection rates under blurring/rotation, and FID. The main claims are that the remover 'completely removes' the StegaStamp watermark while preserving Tree-Ring detectability, and that LBA achieves comparable attack success with less image degradation than uniform blurring.
Significance. If the claims were fully supported, the paper would address a real gap: most watermark-attack work considers full-image perturbations, and combining complementary image-space and latent-space watermarks is a plausible robustness strategy. The release of code and the reproduction of StegaStamp, Tree-Ring, and regeneration baselines are strengths, as is the attempt to quantify both attack success and image quality with FID. However, the central empirical claims are currently under-supported: the StegaStamp-removal claim is never directly measured, the LBA comparison is not at matched operating points, and the reproduced Tree-Ring rotation baseline is far weaker than the original, undermining rotation-related conclusions. The contributions are therefore more in the nature of a proof-of-concept that requires substantial additional experiments before the stated conclusions can be accepted.
major comments (4)
- [§4.1.2, Eq. (3), Table 9] The abstract and Section 4.1.2 claim that the remover network 'completely removes' the StegaStamp watermark, but the paper never reports StegaStamp bit accuracy or detection rate on the remover's output. Table 9 lists a detection rate of 1.000 for the 'Remover' row under 'None'; if this rate refers to StegaStamp detection, it indicates that StegaStamp remains fully decodable after the remover, contradicting the removal claim. Because the loss in Eq. (3) only minimizes L2 distance to the Tree-Ring-only image, the remover has no explicit incentive to destroy the StegaStamp message. The authors should report StegaStamp bit accuracy and detection rate on ITR_Removed, compare with the pre-removal values, and ideally test detection after re-encoding or other realistic decoding conditions.
- [§3.1.2, Table 3] The reproduced Tree-Ring model achieves a rotation-attack AUC of 0.463, whereas the original Tree-Ring paper reports 0.935 (Table 3). This large discrepancy means the paper's rotation results, including the claim in Section 7.1 that 'all methods fail to detect watermarks under Rotation' and the corresponding row in Table 9, are not a trustworthy basis for conclusions about the stacked pipeline's robustness. The authors need to either repair the rotation reproduction (e.g., verify the renoising procedure and rotation handling) or explicitly restrict their rotation claims to their own implementation and avoid drawing general conclusions from that setting.
- [§7.2, Table 10] The LBA versus straight-blurring comparison is not made at matched detection rates, so the headline claim that LBA 'degrades the image significantly less' is confounded by attack strength. For example, LBA at percentile 50 with kernel 31 attains detection rate 0.0554 and FID 88.1, while straight blurring at kernel 31 attains detection rate 0.0000 and FID 150.24; the straight-blur attack is much stronger, so its larger FID does not demonstrate that LBA is a better attack at equal effectiveness. The comparison should be conducted at matched detection rates (e.g., by adjusting the percentile or kernel to obtain comparable TPRs) or with a full operating-characteristic analysis. In addition, all tables report single runs with no variance or significance testing, which is not sufficient to support quantitative comparisons such as the remover's AUC recovery from 0.9936 to 0.9950.
- [§7.2 and §8.2] The paper acknowledges in Section 8.2 that LBA assumes access to the exact StegaStamp decoder, and the evaluation is entirely white-box. This is not circular, but it substantially limits the practical claim: the GradCAM map is computed from the same decoder that is later used to measure attack success, and the paper provides no evidence about how LBA behaves when the attacker must approximate or retrain the decoder. The authors should frame LBA as a white-box proof-of-concept and, ideally, add a sensitivity experiment with a surrogate decoder or with decoder-mismatch conditions.
minor comments (5)
- [§Algorithm 1] The notation in Algorithm 1 is confusing: lines 4 and 6 use 'IF F T' and 'IFFT' for the same transform, and the variable names suggest the FFT is inverted before embedding the watermark; please use consistent transform names and clarify the direction of each transform.
- [§6.2 and Table 4] The manuscript uses both TPR@1%FPR and TPR@0.1%FPR without explaining when each is used; Table 4 reports TPR@0.1%FPR while Section 6.2 defines only TPR@1%FPR. Please align the definitions and the table headers.
- [§Table 9] The definition of 'Detection Rate' in Table 9 is ambiguous because it mixes StegaStamp decoding and Tree-Ring hypothesis testing with different thresholds. Please specify for each row whether the rate is bit-accuracy, message-decoding success, or Tree-Ring p<0.01 detection, and give the corresponding false-positive rate.
- [§7.1] The text says 'stacking watermarks does not interfere with each other' and later that the remover 'effectively isolates and enhances' the Tree-Ring watermark, but the measured AUC changes are tiny (0.9956 to 0.9936 to 0.9950) and no uncertainty is reported; please temper the language or provide confidence intervals.
- [§5] The dataset section refers to 'MS-COCO' while the rest of the paper uses 'MSCOCO'; please standardize the spelling.
Circularity Check
Remover's Tree-Ring preservation is partly by construction; LBA and stacking results remain independent.
-
self definitional
[Section 4.1.2, Eq. (3); Section 7.1, Table 7(b)]
"The network minimizes the L2 distance between the output image, ITR_Removed, and the original Tree-ring watermarked image, ITR. ... After applying a watermark remover, the AUC recovers to 0.9950, and the TPR@1%FPR remains at 0.96."
The remover is trained with Eq. (3) to make its output pixel-wise approximate ITR, the Tree-Ring-only watermarked image. The reported evidence for the remover's benefit is Tree-Ring AUC/TPR on that output, so the Tree-Ring decoder is being tested on a fitted reconstruction of its own target image. This is not an independent prediction of Tree-Ring preservation; success is substantially built into the training objective. Additionally, no StegaStamp bit-accuracy or detection result after removal is reported (Table 7(a) omits the Remover row), and Table 9's 'None' detection rate for the Remover is 1.000, so the 'completely removes the other modality' half of the claim is unverified.
full rationale
The paper contains no self-citations and invokes no uniqueness theorems, so the self-citation patterns do not apply. The LBA attack uses GradCAM from the exact StegaStamp decoder it attacks, but this is an explicitly acknowledged white-box assumption (Section 8.2) and is compared against randomized masking and straight blurring, giving independent empirical content rather than circularity. The Tree-Ring rotation reproduction gap (Table 3) is disclosed and discussed as a limitation, not hidden. The only partially circular element is the remover evaluation, where the training objective directly targets the Tree-Ring-only image and the headline Tree-Ring metrics are measured on that output; this makes the 'preserves Tree-Ring' claim partly self-definitional, and the 'removes StegaStamp' claim is not separately tested. Overall, the LBA contribution and the stacking robustness measurements stand on their own, so the circularity is partial and modest rather than pervasive.
Assumptions & free parameters
free parameters (2)
- LBA percentile threshold =
0, 25, 50, 75
- LBA blur kernel size =
5, 11, 31
assumptions (3)
- domain assumption GradCAM from the exact StegaStamp decoder highlights pixels whose alteration most affects watermark decoding.
- domain assumption The remover can be trained on pairs (Tree-Ring+StegaStamp image, Tree-Ring-only image) and this supervision transfers to new images.
- ad hoc to paper Tree-Ring detection threshold at p<0.01 yields the reported detection rates.
Cite this review
Pith. "Pith review of Invisible Watermarks: Attacks and Robustness." pith.science (2026). https://pith.science/paper/4EPZ5TE4
@misc{pith2026241212511,
author = {Pith},
title = {Pith review of: Invisible Watermarks: Attacks and Robustness},
year = {2026},
howpublished = {\url{https://pith.science/paper/4EPZ5TE4}},
note = {Machine review of arXiv:2412.12511}
}
read the original abstract
As Generative AI continues to become more accessible, the case for robust detection of generated images in order to combat misinformation is stronger than ever. Invisible watermarking methods act as identifiers of generated content, embedding image- and latent-space messages that are robust to many forms of perturbations. The majority of current research investigates full-image attacks against images with a single watermarking method applied. We introduce novel improvements to watermarking robustness as well as minimizing degradation on image quality during attack. Firstly, we examine the application of both image-space and latent-space watermarking methods on a single image, where we propose a custom watermark remover network which preserves one of the watermarking modalities while completely removing the other during decoding. Then, we investigate localized blurring attacks (LBA) on watermarked images based on the GradCAM heatmap acquired from the watermark decoder in order to reduce the amount of degradation to the target image. Our evaluation suggests that 1) implementing the watermark remover model to preserve one of the watermark modalities when decoding the other modality slightly improves on the baseline performance, and that 2) LBA degrades the image significantly less compared to uniform blurring of the entire image. Code is available at: https://github.com/tomputer-g/IDL_WAR
Figures
Figures from the paper (4 more)
Forward citations
Cited by 1 Pith paper
-
First-Place Solution to NeurIPS 2024 Invisible Watermark Removal Challenge
A competition-winning pipeline removes 95.7% of StegaStamp and TreeRing watermarks on the NeurIPS 2024 benchmark by combining VAE fine-tuning, diffusion purification, and translation tricks.
Reference graph
Works this paper leans on
-
[1]
High- resolution image synthesis with latent diffusion models
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 10684–10695, 2022
2022
-
[2]
Stegastamp: Invisible hyperlinks in physical photographs
Matthew Tancik, Ben Mildenhall, and Ren Ng. Stegastamp: Invisible hyperlinks in physical photographs. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020
2020
-
[3]
Tree-ring watermarks: Fin- gerprints for diffusion images that are invisible and robust
Yuxin Wen, John Kirchenbauer, Jonas Geiping, and Tom Goldstein. Tree-ring watermarks: Fin- gerprints for diffusion images that are invisible and robust. arXiv preprint arXiv:2305.20030, 2023
arXiv 2023
-
[4]
The stable signature: Rooting watermarks in latent diffusion models
Pierre Fernandez, Guillaume Couairon, Hervé Jégou, Matthijs Douze, and Teddy Furon. The stable signature: Rooting watermarks in latent diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 22466–22477, 2023
2023
-
[5]
Benchmarking the ro- bustness of image watermarks
Bang An, Mucong Ding, Tahseen Rabbani, Aakriti Agrawal, Yuancheng Xu, Chenghao Deng, Sicheng Zhu, Abdirisak Mohamed, Yuxin Wen, Tom Goldstein, et al. Benchmarking the ro- bustness of image watermarks. arXiv preprint arXiv:2401.08573, 2024
arXiv 2024
-
[6]
Invisible image watermarks are prov- ably removable using generative ai
Xuandong Zhao, Kexun Zhang, Zihao Su, Saastha Vasan, Ilya Grishchenko, Christopher Kruegel, Giovanni Vigna, Yu-Xiang Wang, and Lei Li. Invisible image watermarks are prov- ably removable using generative ai. arXiv preprint arXiv:2306.01953, 2023
arXiv 2023
-
[7]
Deep unsu- pervised learning using nonequilibrium thermodynamics
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsu- pervised learning using nonequilibrium thermodynamics. In Francis Bach and David Blei, editors, Proceedings of the 32nd International Conference on Machine Learning , volume 37 of Proceedings of Machine Learning Research , pages 2256–2265, Lille, France, 07–09 Jul
-
[8]
Denoising diffusion probabilistic models
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. CoRR, abs/2006.11239, 2020
arXiv 2006
Show all 13 references
-
[9]
Denoising diffusion implicit models.CoRR, abs/2010.02502, 2020
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models.CoRR, abs/2010.02502, 2020
2010 arXiv
-
[10]
Grad-cam: Visual explanations from deep networks via gradient- based localization
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient- based localization. In Proceedings of the IEEE international conference on computer vision , pages 618–626, 2017
2017
-
[11]
Robustness of ai-image detectors: Fundamental limits and practical attacks
Mehrdad Saberi, Vinu Sankar Sadasivan, Keivan Rezaei, Aounon Kumar, Atoosa Chegini, Wenxiao Wang, and Soheil Feizi. Robustness of ai-image detectors: Fundamental limits and practical attacks. arXiv preprint arXiv:2310.00076, 2023. 11
-
[12]
Microsoft coco: Common objects in context
Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Pi- otr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. InComputer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceeding...
2014
-
[13]
Gans trained by a two time-scale update rule converge to a local nash equilibrium
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems , 30, 2017. A Appendix: Additional Figures (a) Origin...
2017
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.