{"id":"13a7d85a-868a-4954-b99c-07569827fbbd","arxiv_id":"2506.10502","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"VAE-recovered latent surrogates make Tree-Ring watermarks removable: ROC-AUC drops from 0.993 to 0.153 with little image quality loss.","lead":"An attacker who finds the public autoencoder used by a diffusion model can erase its Tree-Ring watermark by nudging the image in latent space, dropping the detector's ROC AUC from 0.993 to 0.153. This matters because reusing and sharing autoencoders is common industry practice, and it turns an assumed black-box defense into a mostly open one.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Exact-VAE reuse is the load-bearing assumption: with a mismatched VAE the attack degrades to near-baseline (ROC-AUC 0.540/0.774), so the headline 0.153 result depends entirely on the victim reusing the public VAE.","rationale":"The reader's weakest-assumption analysis correctly identifies exact VAE reuse as the load-bearing condition. The paper's own ablation (Table 2, rows 5-6) quantitatively shows that the attack's headline numbers collapse when the VAE differs, and Section 6 acknowledges the attack 'heavily relies on the target VAE.' This is not an internal inconsistency, but it means the central claim's practical reach is narrower than the abstract suggests. The Wm & UnWm control is a valuable internal check that the attack targets watermark features and not merely generator-vs-natural distribution shift, and the open-source release supports reproducibility. However, the exact-VAE dependence remains the single most fragile condition: if industry practice does not predominantly reuse unmodified public VAEs, the attack's real-world impact is substantially reduced. The paper's anecdotal examples (FLUX, ShuttleAI, BigModel, DALL-E) are suggestive but not quantified. A concrete experiment with a privately fine-tuned VAE, as the paper itself recommends as a countermeasure, would settle whether this dependence is fundamental. The appropriate verdict is therefore conditional, matching the reader's assessment; no adjustment is needed.","tokens_in":23228,"tokens_out":14198,"duration_ms":164733,"concrete_test":"Using the authors' released code, fine-tune AutoencoderKL on a private held-out set (e.g., 1,000 images not used elsewhere) to simulate a defender following the paper's own countermeasure, deploy that fine-tuned VAE as the victim's VAE, and run the Wm & Pub attack with the original public AutoencoderKL. If ROC-AUC recovers above 0.5 (as with the SDXL VAE), the exact-VAE requirement is confirmed as load-bearing; if it stays near 0.153, the concern is resolved and the attack is robust to VAE mismatch.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that a public VAE suffices to remove Tree-Ring watermarks is only supported for the exact VAE used by the victim. Section 5.4 shows that with SDXL's VAE (a fine-tuned variant) ROC-AUC falls only to 0.540, and with a 16-channel VAE it falls to 0.774, versus 0.153 with the exact AutoencoderKL. The paper argues exact reuse is common in practice, but the evidence is anecdotal, and the paper's own proposed countermeasure (training with a custom private VAE) directly invalidates the attack. The Wm & Pub headline result is not affected by the additional threat-model inconsistency in the Wm & UnWm training regime, but the practical strength of the attack is conditional on exact VAE availability. This should be verified empirically or explicitly scoped before the claims as stated are relied upon.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a new removal attack against Tree-Ring, a latent-space watermark for diffusion models. The attacker trains a surrogate binary detector on VAE-recovered latent representations of watermarked versus non-watermarked images, then runs projected gradient descent (PGD) in the latent space to push watermarked images into the non-watermarked class. Under the assumption that the attacker has access to the exact VAE used by the victim diffusion model, the attack reduces Tree-Ring's ROC-AUC from 0.993 to 0.153 and PR-AUC from 0.994 to 0.385, with small changes in CLIP score, LPIPS, and FID. The paper also measures detector precision under varying base rates, arguing that Tree-Ring's precision is insufficient for realistic deployment. Ablations with mismatched VAEs (SDXL VAE, 16-channel VAE) and with a non-latent diffusion model (Guided Diffusion) show that the attack degrades when the exact VAE is unavailable.","tokens_in":23427,"tokens_out":7231,"duration_ms":80955,"significance":"If the exact-VAE reuse assumption holds in practice, this is a meaningful advance over prior surrogate-detector attacks: it removes the need for white-box access to the diffusion model or for a same-architecture replica, and it is evaluated against the actual Tree-Ring detector with an external benchmark. The attack is also supported by reproducible code, honest ablations, and a useful precision-based analysis that goes beyond the ROC-AUC metrics used in prior work. The central caveat is that the headline result depends on the victim reusing a publicly available VAE; the paper is transparent about this dependency, but the practical strength of the claim rests on how common that reuse actually is. Overall, the contribution is significant for the watermark-removal literature, provided the threat-model inconsistencies and evaluation confounds identified below are resolved.","major_comments":[{"comment":"The Wm & UnWm training setting is inconsistent with the stated threat model. Section 3.1 says the adversary has only black-box query access to a proprietary model that always embeds Tree-Ring, but Section 4.2 says the adversary generates the non-watermarked images by using the same diffusion model and skipping the watermarking step. This is not possible under the defined capabilities. Please clarify whether Wm & UnWm is an idealized upper-bound scenario or a separate threat model; if it is idealized, state this explicitly and identify the Wm & Pub row as the realistic headline result.","section":"Section 4.2 (Wm & UnWm dataset) vs. Section 3.1 (Threat Model)"},{"comment":"The Wm & Pub evaluation uses ImageNet validation images as the non-watermarked class, so the reported ROC-AUC and PR-AUC improvements may be inflated by the distributional gap between generated images and natural images rather than purely by watermark removal. The no-attack baseline of 0.993/0.994 is consistent with prior work, but the attack could, in principle, be exploiting a generated-vs-natural artifact. To support the claim that the watermark itself is removed, please report the same detection metrics on a test set of non-watermarked images generated by the same diffusion model (or a same-architecture model), for both the no-attack and post-attack conditions.","section":"Section 4.2 / Table 2 (Wm & Pub evaluation)"},{"comment":"The practical strength of the attack is conditional on the attacker having the exact VAE used by the victim. With the fine-tuned SDXL VAE, ROC-AUC only drops to 0.540 (and FID increases by 33.71 in the Wm & UnWm condition), and with the 16-channel VAE it drops only to 0.774; both are far from the 0.153 headline result. The abstract and introduction should more prominently state that the headline result assumes exact VAE reuse, and the anecdotal evidence for 'often publicly available' should be either quantified with a measurement or softened to avoid overclaiming.","section":"Section 5.4 / Table 2 (Alternative VAEs)"},{"comment":"The description of the PGD perturbation budget is unclear. The text says 'we take the maximum number of watermarked latents in our dataset p and set our perturbation budget to 1/p,' but later reports a budget of δ=32 without specifying the norm used for clipping or how δ maps to image-space distortion. Please provide a precise definition of δ, the norm in P_δ, and the value of p, since this is essential for reproducing the attack.","section":"Section 3.4 / Algorithm 1 (Perturbation budget)"}],"minor_comments":[{"comment":"There are several typos and formatting errors, including 'watermaked' (Section 3.2), 'caputres' (Section 4.2), and inconsistent labels such as 'WmvsUnwm' in Figure 9 and Table 2. These should be fixed in a revision.","section":"Throughout"},{"comment":"The text says standard deviations were consistently below 0.01 and are omitted, but no standard deviations are shown anywhere. Please consider adding them to an appendix or at least reporting them for the headline rows.","section":"Table 2"},{"comment":"The discussion of why ROC-AUC values below 0.5 cannot simply be inverted by the defender is interesting but somewhat speculative; it might fit better in the Discussion section, where the arms-race argument is made.","section":"Section 5.3"},{"comment":"The note stating that the work has been accepted for USENIX Security 2025 is unusual for a journal submission; if this is a prior-publication note, it should be clearly marked, otherwise it should be removed.","section":"Title page"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid systems/security contribution with a reproducible attack and an honest set of ablations. The main blockers are the threat-model inconsistency in the Wm & UnWm training regime and the potential distribution-shift confound in the Wm & Pub evaluation. If the authors add a same-generator non-watermarked test condition and clarify the idealized nature of Wm & UnWm, the central claim would be substantially stronger. The exact-VAE dependency is already scoped in the abstract, but the introduction's 'often publicly available' framing should be softened or backed by data."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is one of the better watermark-removal papers I've read recently. It is an empirical attack, but it is scoped honestly, runs the right ablations, and ships code and data. The headline number is real: in the Wm & Pub setting—watermarked images from the victim model versus public ImageNet images as non-watermarked—the VAE-recovered latent surrogate pushes Tree-Ring's ROC-AUC from 0.993 to 0.153 and PR-AUC from 0.994 to 0.385, with modest image degradation. That's a strong result and it is measured against the actual Tree-Ring detector, not a self-defined metric.\n\nWhat's genuinely new: previous surrogate detector attacks used raw pixels; this one trains the surrogate on Fourier-transformed latents recovered via a public VAE, which better approximates Tree-Ring's own detector input. The precision/base-rate analysis is also a useful contribution—most evaluations of these schemes ignore precision, and this paper shows Tree-Ring's detector has poor precision at realistic base rates even without an attack.\n\nWhere it's soft. The exact-VAE assumption is load-bearing, and the reader's stress-test note is fair on that point. With SDXL's VAE (a fine-tuned variant), ROC-AUC only drops to 0.540; with a 16-channel VAE it's 0.774, versus 0.153 with the exact AutoencoderKL. But the paper is not hiding this: the abstract says 'the variational autoencoder that was used to train the target diffusion model,' and Section 5.4 explicitly discusses the degradation. The real debate is empirical—how often do deployment pipelines reuse the exact public VAE? The paper offers anecdotal examples, not a measurement. Their own proposed countermeasure, training with a custom private VAE, invalidates the attack, which they concede. So the practical impact is conditional, but the scientific claim is sound.\n\nThe second soft spot is the Wm & UnWm training regime, which requires the victim model to generate non-watermarked images. That contradicts their black-box threat model. This is a genuine inconsistency, but it's not load-bearing: the Wm & Pub results, which are consistent with the threat model, already support the attack. It should be scoped clearly.\n\nOne more minor point: the evaluation uses a single diffusion model (Stable Diffusion v2.1). The mechanism argument extends plausibly to other latent-space watermarks, but that remains a reasoned extrapolation, not a demonstration.\n\nOverall: the central claim holds up in the setting that matches their threat model. I'd send this to a serious referee if it weren't already accepted at USENIX Security 2025. I'd cite it if I worked on watermark robustness, and I'd bring it to a reading group for the precision analysis alone.","headline":"A solid, honest empirical attack on Tree-Ring whose headline result holds under its stated black-box setting; the practical catch is exact VAE reuse, which the paper acknowledges, and a minor threat-model slip in one training regime.","tokens_in":23959,"tokens_out":3490,"would_cite":true,"duration_ms":36456,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A public VAE lets an attacker strip Tree-Ring watermarks from diffusion images, cutting detector ROC-AUC from 0.993 to 0.153.","keywords":["Tree-Ring watermarking","watermark removal","diffusion models","latent space attack","surrogate detector","variational autoencoder","projected gradient descent","detector precision"],"falsifier":"Take a diffusion model trained on a privately fine-tuned VAE that is never published, run the same surrogate-detector PGD attack using the closest publicly available VAE, and measure Tree-Ring's ROC-AUC. If it stays near its no-attack value of about 0.99 while the attacked images remain high quality, then the paper's practical threat model of public VAE reuse fails for that deployment.","tokens_in":23019,"feed_emoji":"🖼️","tokens_out":8012,"duration_ms":87479,"temperature":0.7,"pith_summary":"This paper claims that Tree-Ring, a widely used watermark for diffusion-generated images, can be defeated by an attacker who only needs the same variational autoencoder (VAE) the victim model uses, plus black-box query access. The attack trains a surrogate classifier to recognize watermarked latents in the Fourier domain, then uses projected gradient descent to push an image's recovered latent out of the watermarked region. In the authors' evaluation on Stable Diffusion v2.1, Tree-Ring's ROC-AUC falls from 0.993 to 0.153 and its PR-AUC from 0.994 to 0.385, with small changes in image quality metrics. This matters because VAE reuse is common practice, so the attack's main assumption is much weaker than prior attacks that assume white-box or same-architecture access to the diffusion model.","feed_headline":"Tree-Ring watermark detection collapses under public-VAE attack","feed_subtitle":"Using only the public autoencoder the model was trained with, the attack cuts ROC-AUC from 0.993 to 0.153 with little image distortion.","key_machinery":"The load-bearing object is the surrogate detector trained on VAE-recovered latents in the Fourier domain, combined with projected gradient descent in latent space. Tree-Ring embeds a key as a circular pattern in the low-frequency center of the initial noise vector's Fourier spectrum; the paper shows this creates a non-Gaussian, compact hypersphere of watermarked latents that a ResNet-18 can separate almost perfectly. The VAE, which is AutoencoderKL by default in Stable Diffusion v2.1, provides the attacker with a mapping from image to latent that approximates the victim's own latent space, letting the surrogate learn the watermarked region rather than spurious image statistics. Projected gradient descent then perturbs the recovered latent to minimize the surrogate's loss for the non-watermarked class, with the perturbation budget scaled by the inverse maximum latent magnitude, and decoding with the VAE yields the attacked image.","core_discovery":"The central claim is that Tree-Ring's watermark leaves detectable traces in the intermediate latent space, and that these traces can be exploited through a surrogate detector trained on latents recovered by a public VAE. The authors show that Tree-Ring's embedding of a fixed Fourier ring into the initial noise vector violates the Gaussian assumption on the latents: watermarked initial latents are compactly clustered and almost perfectly separable from non-watermarked ones, and remnants of the ring survive backward diffusion into the recovered latent. An attacker who has the exact VAE can encode an image, take the Fourier transform of the recovered latent, train a ResNet-18 binary classifier to distinguish watermarked from non-watermarked latents, and then run projected gradient descent in latent space to flip the surrogate's decision. The resulting perturbations transfer to the real Tree-Ring detector, reducing its ROC-AUC to 0.153 and its PR-AUC to 0.385 while preserving image quality, and the attack outperforms adversarial noising and pixel-space surrogate attacks that assume more access.","pith_inferences":["Editorial extension: the attack's dependence on the exact VAE suggests a practical robustness audit—before deploying a latent-space watermark, measure how much ROC-AUC drops under a PGD surrogate trained on the same public VAE; a large drop indicates the scheme is not deployment-ready.","Editorial extension: the Fourier-domain concentration of the perturbation around the key ring is a testable signature; analyzing attack success as a function of ring radius and frequency band could identify key designs that resist this attack without visible artifacts.","Editorial extension: if VAE reuse is as widespread as the paper argues, one public VAE could be a single point of failure for many watermarked services, meaning a single surrogate might strip watermarks across multiple models that share that VAE."],"forward_implications":["Tree-Ring watermarking is not robust against a black-box adversary who can query the model and obtain its VAE; even without any white-box knowledge, the detector's ROC-AUC drops to 0.153.","Because the attack outperforms baselines that assume full or same-architecture access to the diffusion model, VAE reuse becomes a first-order security decision for anyone deploying latent diffusion.","Precision matters: when watermarked images are a small fraction of traffic, Tree-Ring's detector is unreliable even without an attack, and the VAE attack reduces PR-AUC from 0.994 to 0.385 at a balanced base rate.","Other latent-space watermarking schemes that embed a distinctive pattern in the diffusion latents are exposed to the same VAE-based surrogate strategy, since the attack targets the latent distribution rather than Tree-Ring's specific key shape.","A short-term defense is to train a custom private VAE, but the paper notes this sacrifices the cost savings that make latent diffusion attractive and does not close the door to similar surrogate attacks."],"supporting_citations":[{"why":"Defines Tree-Ring's embedding and detection scheme, the target under attack.","marker":"[56]"},{"why":"Supplies the surrogate-detector attack framework that this paper reformulates in latent space, plus the baseline results it must beat.","marker":"[4]"},{"why":"Establishes latent diffusion and the practice of training text-to-image models on a pretrained VAE, which grounds the public-VAE threat model.","marker":"[44]"},{"why":"Provides AutoencoderKL, the exact VAE used by the victim model in the main evaluation and by the attacker to recover latents.","marker":"[32]"},{"why":"Provides the adversarial noising baseline that the authors reproduce and show their attack outperforms.","marker":"[37]"},{"why":"Supplies a 16-channel VAE used in the ablation that tests what happens when attacker and victim VAEs differ.","marker":"[12]"}],"fun_headline_variants":["Tree-Ring detector's AUC nosedives from 0.993 to 0.153 via public VAE","Tree-Ring watermark defense shattered by a single public component","Simple public-VAE attack guts Tree-Ring watermark robustness","Attack uses only public VAE to obliterate Tree-Ring detection","How a public VAE defeated Tree-Ring's watermark detection"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The headline result rests on the attacker getting the exact variational autoencoder the victim's diffusion model uses; with a different or custom VAE, the attack's power drops sharply, as the paper's own ablation shows.","fun_headline_variants_meta":{"raw":{"variants":["Tree-Ring detector's AUC nosedives from 0.993 to 0.153 via public VAE","Tree-Ring watermark defense shattered by a single public component","Simple public-VAE attack guts Tree-Ring watermark robustness","Attack uses only public VAE to obliterate Tree-Ring detection","How a public VAE defeated Tree-Ring's watermark detection"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000641,"raw_usage":{"total_tokens":2963,"prompt_tokens":969,"completion_tokens":1994,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":585,"completion_tokens_details":{"reasoning_tokens":1898}},"tokens_in":585,"tokens_out":1994,"duration_ms":15190,"temperature":1.0,"reasoning_tokens":1898,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T04:25:03.931399+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a diffusion model trained on a privately fine-tuned VAE that is never published, run the same surrogate-detector PGD attack using the closest publicly available VAE, and measure Tree-Ring's ROC-AUC. If it stays near its no-attack value of about 0.99 while the attacked images remain high quality, then the paper's practical threat model of public VAE reuse fails for that deployment.","supporting_citations":[{"cited_title":"In37th Confer- ence on Neural Information Processing Systems (NeurIPS)(2023)","cited_arxiv_id":null,"evidence_quote":"Defines Tree-Ring's embedding and detection scheme, the target under attack."},{"cited_title":"InInternational Conference on Machine Learning (ICML)(2024)","cited_arxiv_id":null,"evidence_quote":"Supplies the surrogate-detector attack framework that this paper reformulates in latent space, plus the baseline results it must beat."},{"cited_title":"High-resolution image synthesis with latent diffusion models","cited_arxiv_id":null,"evidence_quote":"Establishes latent diffusion and the practice of training text-to-image models on a pretrained VAE, which grounds the public-VAE threat model."},{"cited_title":"P ., and Welling, M","cited_arxiv_id":null,"evidence_quote":"Provides AutoencoderKL, the exact VAE used by the victim model in the main evaluation and by the attacker to recover latents."},{"cited_title":"Leveraging optimization for adaptive attacks on image watermarks","cited_arxiv_id":null,"evidence_quote":"Provides the adversarial noising baseline that the authors reproduce and show their attack outperforms."},{"cited_title":"Ostris vae - kl-f8-d16, 7 2024","cited_arxiv_id":null,"evidence_quote":"Supplies a 16-channel VAE used in the ablation that tests what happens when attacker and victim VAEs differ."}],"review_version":1}