{"id":"be076cfa-0f23-40e4-b068-b1a954c20017","arxiv_id":"2412.12511","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Stacking StegaStamp and Tree-Ring watermarks with a remover network gives marginal robustness gains, and GradCAM-guided localized blurring reduces distortion relative to uniform blurring but not enough to beat regeneration attacks.","lead":"The paper experiments with two ways to attack or strengthen invisible watermarks: a small 'remover' network that strips StegaStamp so Tree-Ring can be decoded, and a localized blurring attack that blurs only the pixels GradCAM says matter to the decoder. The measured gains are small, and the proposed attack still falls short of existing regeneration attacks on image quality.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Remover claim not supported: no bit-accuracy test after removal; visual similarity to Tree-Ring image could hide watermarks, and StegaStamp detection after removal is never reported.","rationale":"The reader's weakest_assumption focuses on LBA's white-box access and the Tree-Ring reproduction, both legitimate. My review finds a more direct hole in the remover claim, which is the paper's first headline contribution: the remover network is trained with an L2 loss toward the Tree-Ring image (Eq. 3) and the paper never reports StegaStamp decoding metrics after removal. Table 9's 'None' row shows detection rate 1.000 for the remover, which actually suggests the StegaStamp watermark is still present and decodable, not removed. The abstract says the remover 'completely removes' one modality, so this missing verification is load-bearing. The Tree-Ring reproducibility gap (0.463 vs 0.935 rotation AUC) also weakens the stacking conclusions, but the remover's effect on StegaStamp is the more fundamental gap because the method's premise is to enable stacking by removing interference. I agree with the reader's CONDITIONAL verdict: the issues are correctable with a few additional experiments, but the current evidence does not establish the central claim as stated.","tokens_in":9938,"tokens_out":1417,"duration_ms":10990,"concrete_test":"Take 5,000 COCO images watermarked with both Tree-Ring and StegaStamp, run the published remover, then decode StegaStamp on the output (and on the output after a standard re-encode step, e.g., JPEG q=90 or resize). Report bit accuracy and detection rate. If bit accuracy remains near 0.997 or detection rate near 1.000, the 'completely removes' claim is directly refuted. If bit accuracy drops below 0.5 and detection rate drops below 0.01, the remover claim is supported.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central remover claim is that the proposed network 'completely removes' the StegaStamp watermark while preserving Tree-Ring detectability (Abstract and Section 4.1.2). The evidence actually reported is: (i) Tree-Ring AUC after removal (0.9950) and p-values, and (ii) Table 9 detection rates for combined vs remover under None, Blurring, and Rotation. But Table 9 reports detection rates only for untransformed images under 'None' as 1.000 for StegaStamp, Naively Stacking, and Remover. That is not a StegaStamp removal check; it is a detection test, and 1.000 suggests StegaStamp bits remain fully decodable after the remover. The paper never reports StegaStamp bit accuracy or detection rate after the remover under realistic re-encoding settings, nor de-duplicated over the evaluation set. Because the remover is trained only with an L2 loss toward the Tree-Ring image (Eq. 3), it has no incentive to destroy the StegaStamp message; it can satisfy the loss by producing a blurred image that still contains the watermark. Therefore the headline claim that the remover improves robustness by removing one modality is not established by the reported metrics.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes two contributions to invisible-image-watermark robustness: (1) a multi-watermark pipeline that stacks Tree-Ring and StegaStamp and uses a learned 'remover' network trained with an L2 loss to strip the StegaStamp component before Tree-Ring decoding, and (2) a Localized Blurring Attack (LBA) that uses GradCAM on the StegaStamp decoder to blur only high-attention regions, supposedly reducing image degradation relative to full-image blurring. The evaluation uses MS-COCO validation images and reports bit accuracy, AUC, TPR at fixed FPR, p-values, detection rates under blurring/rotation, and FID. The main claims are that the remover 'completely removes' the StegaStamp watermark while preserving Tree-Ring detectability, and that LBA achieves comparable attack success with less image degradation than uniform blurring.","tokens_in":10126,"tokens_out":3766,"duration_ms":35725,"significance":"If the claims were fully supported, the paper would address a real gap: most watermark-attack work considers full-image perturbations, and combining complementary image-space and latent-space watermarks is a plausible robustness strategy. The release of code and the reproduction of StegaStamp, Tree-Ring, and regeneration baselines are strengths, as is the attempt to quantify both attack success and image quality with FID. However, the central empirical claims are currently under-supported: the StegaStamp-removal claim is never directly measured, the LBA comparison is not at matched operating points, and the reproduced Tree-Ring rotation baseline is far weaker than the original, undermining rotation-related conclusions. The contributions are therefore more in the nature of a proof-of-concept that requires substantial additional experiments before the stated conclusions can be accepted.","major_comments":[{"comment":"The abstract and Section 4.1.2 claim that the remover network 'completely removes' the StegaStamp watermark, but the paper never reports StegaStamp bit accuracy or detection rate on the remover's output. Table 9 lists a detection rate of 1.000 for the 'Remover' row under 'None'; if this rate refers to StegaStamp detection, it indicates that StegaStamp remains fully decodable after the remover, contradicting the removal claim. Because the loss in Eq. (3) only minimizes L2 distance to the Tree-Ring-only image, the remover has no explicit incentive to destroy the StegaStamp message. The authors should report StegaStamp bit accuracy and detection rate on ITR_Removed, compare with the pre-removal values, and ideally test detection after re-encoding or other realistic decoding conditions.","section":"§4.1.2, Eq. (3), Table 9"},{"comment":"The reproduced Tree-Ring model achieves a rotation-attack AUC of 0.463, whereas the original Tree-Ring paper reports 0.935 (Table 3). This large discrepancy means the paper's rotation results, including the claim in Section 7.1 that 'all methods fail to detect watermarks under Rotation' and the corresponding row in Table 9, are not a trustworthy basis for conclusions about the stacked pipeline's robustness. The authors need to either repair the rotation reproduction (e.g., verify the renoising procedure and rotation handling) or explicitly restrict their rotation claims to their own implementation and avoid drawing general conclusions from that setting.","section":"§3.1.2, Table 3"},{"comment":"The LBA versus straight-blurring comparison is not made at matched detection rates, so the headline claim that LBA 'degrades the image significantly less' is confounded by attack strength. For example, LBA at percentile 50 with kernel 31 attains detection rate 0.0554 and FID 88.1, while straight blurring at kernel 31 attains detection rate 0.0000 and FID 150.24; the straight-blur attack is much stronger, so its larger FID does not demonstrate that LBA is a better attack at equal effectiveness. The comparison should be conducted at matched detection rates (e.g., by adjusting the percentile or kernel to obtain comparable TPRs) or with a full operating-characteristic analysis. In addition, all tables report single runs with no variance or significance testing, which is not sufficient to support quantitative comparisons such as the remover's AUC recovery from 0.9936 to 0.9950.","section":"§7.2, Table 10"},{"comment":"The paper acknowledges in Section 8.2 that LBA assumes access to the exact StegaStamp decoder, and the evaluation is entirely white-box. This is not circular, but it substantially limits the practical claim: the GradCAM map is computed from the same decoder that is later used to measure attack success, and the paper provides no evidence about how LBA behaves when the attacker must approximate or retrain the decoder. The authors should frame LBA as a white-box proof-of-concept and, ideally, add a sensitivity experiment with a surrogate decoder or with decoder-mismatch conditions.","section":"§7.2 and §8.2"}],"minor_comments":[{"comment":"The notation in Algorithm 1 is confusing: lines 4 and 6 use 'IF F T' and 'IFFT' for the same transform, and the variable names suggest the FFT is inverted before embedding the watermark; please use consistent transform names and clarify the direction of each transform.","section":"§Algorithm 1"},{"comment":"The manuscript uses both TPR@1%FPR and TPR@0.1%FPR without explaining when each is used; Table 4 reports TPR@0.1%FPR while Section 6.2 defines only TPR@1%FPR. Please align the definitions and the table headers.","section":"§6.2 and Table 4"},{"comment":"The definition of 'Detection Rate' in Table 9 is ambiguous because it mixes StegaStamp decoding and Tree-Ring hypothesis testing with different thresholds. Please specify for each row whether the rate is bit-accuracy, message-decoding success, or Tree-Ring p<0.01 detection, and give the corresponding false-positive rate.","section":"§Table 9"},{"comment":"The text says 'stacking watermarks does not interfere with each other' and later that the remover 'effectively isolates and enhances' the Tree-Ring watermark, but the measured AUC changes are tiny (0.9956 to 0.9936 to 0.9950) and no uncertainty is reported; please temper the language or provide confidence intervals.","section":"§7.1"},{"comment":"The dataset section refers to 'MS-COCO' while the rest of the paper uses 'MSCOCO'; please standardize the spelling.","section":"§5"}],"recommendation":"major_revision","confidential_remarks":"This manuscript reads as a course-project report, and several of its headline claims outrun the evidence. The most decisive issue is the absence of any direct StegaStamp-removal metric after the remover network, which leaves the paper's first contribution unverified. The unmatched LBA comparison and the anomalous Tree-Ring rotation reproduction further undermine the quantitative conclusions. I would not reject the paper outright because the underlying ideas are reasonable and the missing measurements could in principle be supplied, but the revision needs to be substantial rather than cosmetic."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a course-project report that has a decent idea in the LBA attack and an honest write-up, but its central claim about the watermark remover is not actually tested. The paper is worth a look for the LBA direction and the stacking experiment, but it needs significant work before the remover result can be trusted.\n\nWhat is new: applying GradCAM to the StegaStamp decoder to localize blur is a sensible combination that I haven't seen in the literature, and the idea of training a small denoiser to remove one watermark from a stacked image is a reasonable thing to try. The paper is transparent about its limitations, the code is available, and the unattacked reproductions of StegaStamp (0.997 bit accuracy) and Tree-Ring (AUC 1.000) are close to the originals. The stacking experiment shows that naive stacking does not destroy Tree-Ring (AUC 0.9936 vs 0.9956), which is a useful data point.\n\nThe problems are in the claims. The abstract says the remover 'completely removes' StegaStamp, but the paper never measures StegaStamp bit accuracy or detection rate after the remover. Table 9 reports a detection rate of 1.000 for the 'Remover' under 'None,' which reads as the StegaStamp message still being fully decodable. Since the remover is trained only to minimize L2 distance to the Tree-Ring image, it has no incentive to destroy the StegaStamp bits; it can satisfy the loss by producing an image that is close to the Tree-Ring image while the watermark remains. So the headline result is not established. This is the paper's biggest soft spot.\n\nAlso, the Tree-Ring rotation reproduction is far weaker than the original paper (AUC 0.463 vs 0.935). That does not necessarily invalidate the stacking analysis, but it weakens any conclusion about rotation robustness. The LBA comparison is not clean either: they compare against straight blurring at different detection rates, and the regeneration baseline achieves a better FID (16.3) than any LBA configuration with meaningful detectability reduction. No error bars or significance tests are reported for any of the main tables.\n\nWho this is for: people working on watermark robustness and attacks might pick up the LBA idea and the stacking data point. As a paper, it is not ready for a top venue, but it deserves a serious referee rather than a desk reject because the ideas are reasonable and the issues are fixable. My recommendation: send it to peer review, and ask for a major revision that adds a direct StegaStamp removal test, makes the LBA comparison at matched detection rates, and includes variance estimates. The remover result will either survive or fall on the bit-accuracy test; right now we don't know which.","headline":"A course project with a plausible LBA idea and a stacked-watermark remover network, but the remover's central claim is untested and the evaluation has several unaddressed gaps.","tokens_in":10683,"tokens_out":2491,"would_cite":false,"duration_ms":20655,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Two watermarking schemes can be stacked and decoded independently via a learned remover network, while a GradCAM-guided localized blurring attack removes StegaStamp with roughly half the image degradation of uniform blurring.","keywords":["invisible watermarks","StegaStamp","Tree-Ring","watermark stacking","watermark removal network","localized blurring attack","GradCAM","AI-generated image provenance"],"falsifier":"Run the Localized Blurring Attack using a GradCAM heatmap computed from a surrogate StegaStamp decoder trained on watermarked and unwatermarked pairs without the real weights, then measure detection with the true decoder: if detection stays near 1.000 at kernel 31, the attack's quality advantage is an artifact of white-box access. Separately, re-run the stacked Tree-Ring pipeline's rotation test with a Tree-Ring implementation that reproduces the original paper's 0.935 rotation AUC; if stacking and removal still drive rotation detection to 0.000, the robustness ceiling is real, but if the reproduction gap is the cause, the rotation conclusion collapses.","tokens_in":9674,"feed_emoji":"🎯","tokens_out":12859,"duration_ms":99251,"temperature":0.7,"pith_summary":"This paper tries to settle two practical questions about invisible watermarks for AI-generated images: whether two different watermarking schemes can be layered on one image without destroying each other, and whether an attack can remove a watermark without visibly wrecking the picture. To answer the first, the authors stack Tree-Ring (a latent-space watermark embedded during diffusion generation) on top of StegaStamp (a pixel-space post-processing watermark) and train a small 'remover' network that strips the StegaStamp signal before Tree-Ring decoding; the stacked pipeline's Tree-Ring AUC recovers from 0.9936 under naive stacking to 0.9950, against an unattacked baseline of 0.9956. To answer the second, they propose the Localized Blurring Attack, which uses a GradCAM heatmap from the StegaStamp decoder to blur only the pixels that carry the watermark; at a 50% threshold and kernel size 31 it reaches a detection rate of 0.0554 with a Fréchet Inception Distance of 88.1, versus straight blurring's 0.0000 at 150.24. If these results hold, both sides gain a new lever: layered watermarks become practical with a separation step, and attackers can trade much less image quality for comparable watermark removal.","feed_headline":"Targeted blur halves image damage while killing watermarks","feed_subtitle":"GradCAM heatmaps tell attackers exactly which pixels matter, cutting FID from 150.24 to 88.1 while keeping detection near zero.","key_machinery":"The argument turns on two mechanisms. The first is the remover network: a small trained CNN that takes the stacked image $I_{TR+SS}$ and minimizes $\\frac{1}{N}\\sum_{i=1}^{N} \\|I^{(i)}_{TR\\_Removed} - I^{(i)}_{TR}\\|_2^2$, learning to cancel exactly the StegaStamp perturbation so that Tree-Ring decoding sees a clean latent-space watermark; this is what converts an impractical naive stack into a working two-watermark pipeline. The second is the Localized Blurring Attack: a GradCAM heatmap is computed from the StegaStamp decoder, pixels above a chosen percentile (0, 25, 50, or 75) are thresholded into a binary mask, and only those regions are replaced with blurred pixels, so the attack spends its image-quality budget on the few areas the decoder actually relies on. The paper emphasizes an asymmetry that makes both results possible: Tree-Ring alters pixel space strongly (image $\\ell^2$ distance 117.58) while StegaStamp displaces the latent representation more (latent $\\ell^2$ distance 118.17), which is why one modality can be selectively removed and the other selectively attacked.","core_discovery":"On its own terms, the paper's discovery is that the two watermark modalities are separable in practice: Tree-Ring and StegaStamp perturb different feature spaces, so a direct sequential stack makes Tree-Ring undetectable, but a network trained to undo the StegaStamp perturbation restores clean Tree-Ring decoding. The authors train a remover network to map the stacked image back to the Tree-Ring-only image under an L2 loss; after removal, Tree-Ring detection returns to nearly its unattacked level (AUC 0.9950 versus 0.9956, TPR@1%FPR 0.96) while StegaStamp bit accuracy holds at 0.998. The paper's second claim is that watermarked information is spatially concentrated: StegaStamp residuals sit in high-contrast salient regions, and a GradCAM-derived binary mask that blurs only those regions is nearly as effective as full-image blurring while doing far less damage, cutting detection to 0.0554 at a 50% threshold and kernel 31 with an FID of 88.1 against straight blur's 0.0000 at 150.24. The paper reports these results with its own caveats, including that all methods fail rotation and that its Tree-Ring reproduction scores far lower on rotated images than the original paper does.","pith_inferences":["A concrete testable extension the paper leaves implicit: applying the LBA mask to a regeneration attack instead of blur should push FID below LBA's 36.1 at similar detection suppression, since regeneration already preserves quality and the mask tells the attack where to spend its noise.","The remover's success suggests a general modality-stripping principle: any pair of watermarks living in different feature spaces, such as Stable Signature and Tree-Ring, might be stackable with a similar L2-trained separator, but the paper only demonstrates the StegaStamp-Tree-Ring pair.","The rotation reproduction gap (0.463 versus 0.935 AUC) means the stacked pipeline's rotation conclusions currently rest on a weaker baseline than the original; until the original Tree-Ring model is run through the stacking and remover pipeline, rotation robustness should be treated as unverified.","If the residual blur-gap (remover detection 0.168 versus Tree-Ring alone 0.218) comes from leftover latent-space interference, adding a latent-space loss term to the remover's training objective should measurably close that gap, a prediction that follows from the paper's own diagnosis and is directly testable."],"forward_implications":["Multi-watermark images become practical: a decoder-side remover lets a latent-space watermark and a pixel-space watermark be layered on one image and each decoded cleanly, with Tree-Ring AUC recovering from 0.9936 (naive stack) to 0.9950.","Attackers can buy stealth with less damage: at percentile 50 and kernel 31, LBA cuts StegaStamp detection to 0.0554 at FID 88.1, about half the image degradation of straight blurring at comparable attack strength.","Heatmap localization is a general threat model: any watermark whose information concentrates in salient regions is vulnerable to attention-guided attacks, so the paper concludes that open-source decoders must be kept private.","Stacking does not fix weaknesses neither component handles: every stacked variant fails rotation with detection 0.000, exactly like each watermark alone.","Blur is not the final attack: regeneration still beats LBA (detection 0.0100 at FID 16.3), so the paper's own comparison implies that localized versions of less destructive attacks would combine the best of both."],"supporting_citations":[{"why":"Introduces the StegaStamp pixel-space watermark whose encoder and decoder the paper reproduces, attacks with LBA, and strips with the remover network.","marker":"[2]"},{"why":"Introduces the Tree-Ring latent-space watermark that the paper stacks with StegaStamp and decodes after removal.","marker":"[3]"},{"why":"Supplies the GradCAM localization method that produces the heatmaps and binary masks at the core of the Localized Blurring Attack.","marker":"[10]"},{"why":"Provides the WAVES benchmark attacks (blur, rotation, regeneration and rinsing) used as baselines and the framing that current attacks target the full image.","marker":"[5]"},{"why":"Defines the regeneration attack that serves as the main attack baseline and outperforms LBA on image quality per unit of detection reduction.","marker":"[6]"},{"why":"Provides the pixel-space and latent-space distance measurements that justify the claim that Tree-Ring and StegaStamp occupy different feature spaces.","marker":"[11]"}],"fun_headline_variants":["GradCAM-guided blur cuts image damage in half while killing watermarks","Localized blur attacks watermarks with 41% less FID damage","Dual watermark removal: network erases one, preserves the other","Watermark attack minimizes visual impact by targeting salient regions"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Both headline results depend on white-box access that the paper itself flags as fragile: the localized blurring attack needs a GradCAM heatmap from the exact StegaStamp decoder, which Section 8.2 concedes an attacker may have to approximate with unknown consequences, and the stacked pipeline's robustness conclusions inherit a Tree-Ring reproduction whose rotation AUC of 0.463 sits far below the original's 0.935 (Table 3).","fun_headline_variants_meta":{"raw":{"variants":["GradCAM-guided blur cuts image damage in half while killing watermarks","Localized blur attacks watermarks with 41% less FID damage","Dual watermark removal: network erases one, preserves the other","Watermark attack minimizes visual impact by targeting salient regions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000284,"raw_usage":{"total_tokens":1724,"prompt_tokens":1045,"completion_tokens":679,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":661,"completion_tokens_details":{"reasoning_tokens":605}},"tokens_in":661,"tokens_out":679,"duration_ms":6431,"temperature":1.0,"reasoning_tokens":605,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T13:59:54.664172+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the Localized Blurring Attack using a GradCAM heatmap computed from a surrogate StegaStamp decoder trained on watermarked and unwatermarked pairs without the real weights, then measure detection with the true decoder: if detection stays near 1.000 at kernel 31, the attack's quality advantage is an artifact of white-box access. Separately, re-run the stacked Tree-Ring pipeline's rotation test with a Tree-Ring implementation that reproduces the original paper's 0.935 rotation AUC; if stacking and removal still drive rotation detection to 0.000, the robustness ceiling is real, but if the reproduction gap is the cause, the rotation conclusion collapses.","supporting_citations":[{"cited_title":"Grad-cam: Visual explanations from deep networks via gradient- based localization","cited_arxiv_id":null,"evidence_quote":"Supplies the GradCAM localization method that produces the heatmaps and binary masks at the core of the Localized Blurring Attack."}],"review_version":1}