{"id":"95822010-989a-45eb-a1f4-6b99c28011eb","arxiv_id":"2412.19834","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Adversarial fine-tuning with random watermark keys degrades Stable Signature watermarking, and a TAR-inspired defense only partially restores robustness.","lead":"An NYU team shows that fine-tuning a watermarked image generator with random watermark keys makes it embed wrong keys, and then adapts an LLM safety method to slow that down. The defense only partially holds, so watermark removal remains a real risk for open-source generators.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Algorithm 1's gTR gradient is ambiguous: if it is not unrolled through the K inner attack steps, the TAR defense may collapse to retain-loss fine-tuning, leaving the reported tamper-resistance unsupported.","rationale":"The reader's weakest assumption is exactly the gradient ambiguity in Algorithm 1, and I agree. I also considered the missing baseline and absent Figure 1, but those weaken the comparison while an incorrect or detached gradient would invalidate the mechanism claimed for TAR. The central attack result is not undermined: random-key fine-tuning dropping bit accuracy to ~0.54–0.56 with PSNR ~27.6 is plausible and consistent with the known fragility of post-hoc watermarking. The defense, however, has no external verification, and the mathematical notation in Algorithm 1 does not specify the differentiation path. The most charitable reading—full unrolling—is computationally heavy and should be stated explicitly; the less charitable reading—detached attack—makes the defense a form of regularized fine-tuning whose reported numbers do not demonstrate tamper resistance. The concrete test distinguishes these. Therefore the reader's CONDITIONAL verdict is appropriate, with the condition that the gradient path be specified and the ablation confirm that gTR, not g_retain, drives the post-attack accuracy.","tokens_in":6299,"tokens_out":6736,"duration_ms":56961,"concrete_test":"Download the released repository and instrument Algorithm 1 to compare three variants: (a) full unrolled gradient through all inner attack steps, (b) detached attack (stop-gradient at attack(θ_{i-1})), and (c) gTR=0 with only g_retain. Reproduce the evaluation in Table III using the same DTR, Dretain, N, K, and attack steps. If (a) is not what the code computes, or if (b) or (c) reproduce the 64–65% post-attack bit accuracy, the tamper-resistance claim in Section IV.B is unsupported. Independently, re-derive line 7 by the chain rule over the inner SGD trajectory and check whether the code includes the resulting Jacobian terms.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section III.B (Algorithm 1, line 7) defines gTR as the average of ∇_{θ_{i-1}} L_TR(attack(θ_{i-1}), x_TR), where attack(·) is an inner loop of K random-key fine-tunes, each about 100 steps. The paper never states whether this gradient is unrolled through the inner optimization. If the implementation stops gradients at the attacked model—or computes the gradient w.r.t. the post-attack parameters and assigns it to θ_{i-1}—then line 7 is not the derivative of the tamper-resistance loss with respect to the parameters being updated. In that case the outer update is not training the decoder to recover from the attack; the reported 64–65% post-attack bit accuracy in Table III could be generated by the retain loss (line 10), by the undefined h_θ regularizer, or by training noise. Since the defense is a main contribution, this ambiguity is load-bearing. The attack result (Tables I–II) is less affected: near-random bit accuracy with high PSNR is a plausible vulnerability demonstration even without the defense.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper considers the threat of adversarial fine-tuning against the Stable Signature watermarking method for Latent Diffusion Models (LDMs). It proposes two attacks (Random Key Attack and Gradual Random Key Attack) that fine-tune a watermarked LDM decoder against random watermark keys, reporting that bit accuracy drops to near 50% while PSNR remains high, indicating that image quality is preserved but the intended watermark is no longer embedded. As a defense, the paper adapts Tamper-Resistant Fine-Tuning (TAR) from LLM safety to the LDM watermarking setting, combining a tamper-resistance loss computed from attacked models with a retain loss that preserves the original watermarking behavior and image quality. The reported defense results show high pre-attack bit accuracy (96–99%) and post-attack bit accuracy around 64–65%.","tokens_in":6481,"tokens_out":2795,"duration_ms":26030,"significance":"If substantiated, the results would demonstrate a practical vulnerability in Stable Signature and provide a first step toward making LDM watermarks robust to fine-tuning attacks. The paper has strong motivational grounding, uses public data (COCO) and published methods as building blocks, and ships a code link. It honestly acknowledges that the defense yields only partial tamper resistance and that LDM watermarking lacks the redundancy that aids LLM safety defenses. However, the significance is conditional: the central defense algorithm is underspecified, the experimental evaluation lacks baselines and variance estimates, and the undefined hidden-state regularizer makes the retain-loss contribution partly non-reproducible.","major_comments":[{"comment":"The gradient gTR is defined as (1/K) * Σ ∇_{θ_{i-1}} L_TR(attack(θ_{i-1}), x_TR), but the paper never states whether this gradient is unrolled through the inner K attack fine-tuning steps or computed only at the final attacked parameters while stopping gradients at the attack. If the implementation does not backpropagate through the attack, line 7 is not the derivative of the tamper-resistance loss with respect to the parameters being updated, and the reported post-attack bit accuracy in Table III could be produced by the retain-loss term, the hidden-state regularizer, or training noise rather than by true tamper resistance. Because the defense is a main contribution, this ambiguity is load-bearing and should be resolved with an explicit statement and, ideally, an ablation that separates the gTR and gretain contributions.","section":"Section III.B, Algorithm 1 line 7"},{"comment":"The defense evaluation does not provide a table-level baseline: the text says Figure 1 also shows the original Stable Signature decoder post-attack, but the figure is not included in the manuscript and no numeric values are given in Table III. Without the post-attack bit accuracy and PSNR of the undefended Stable Signature decoder under the identical attack, the claim that TAR provides an improvement from 'near 50%' to '64–65%' cannot be quantitatively verified. Additionally, every reported number in Tables I–III is a single run without seeds, error bars, or multiple initializations, so the ranges '96–99%' and '64–65%' are not substantiated as stable results. The authors should add baseline rows to Table III and report mean and standard deviation over at least three seeds.","section":"Table III and Figure 1"},{"comment":"The retain gradient includes the term ||h_{θ_{i-1}}(x_r) − h_θ(x_r)||_2^2, where h_θ is described only as 'the residual stream hidden states for model parameters θ'. For an LDM decoder, which is a convolutional decoder rather than a transformer with a residual stream, this term is not defined and cannot be reproduced from the text. The paper should specify exactly which intermediate activations are used, how they are extracted, and why the L2 distance between them preserves generation capability. This is not merely a notational issue: the hidden-state regularizer is part of the retain gradient and may be responsible for the reported image-quality preservation.","section":"Section III.B, Algorithm 1 line 10"},{"comment":"The attack results are missing a pre-attack baseline column. The tables report only post-attack PSNR and bit accuracy; without the corresponding bit accuracy and PSNR for the same Stable Signature-finetuned decoder before the attack, the magnitude of the drop is not directly measured. Although the paper assumes the original decoder has high bit accuracy, adding two baseline rows (or columns) would make the attack claim self-contained and would also clarify whether the PSNR values are comparable before and after attack.","section":"Tables I and II"}],"minor_comments":[{"comment":"There are numerous typos and grammatical errors that should be corrected, including 'Specificaly', 'backpropogation', 'Ferenandez', 'fine-tined', 'will trying', and 'Theses datasets'.","section":"General"},{"comment":"Figure 1 is referenced in the text but does not appear in the manuscript. Either include the figure or remove the reference, since the numeric values in the text are insufficient to support the claimed comparison.","section":"Section IV.B and Figure 1"},{"comment":"The notation h_θ should be introduced before its use in Algorithm 1, and the phrase 'residual stream hidden states' should be defined in the context of the LDM decoder architecture.","section":"Section III.B"},{"comment":"The paper does not state the learning rate, optimizer, or fine-tuning loss hyperparameters used for the attacks, which makes it difficult to reproduce or compare the attack strength with existing adversarial fine-tuning methods.","section":"Section IV.A"}],"recommendation":"major_revision","confidential_remarks":"The paper reads like a promising workshop-level contribution: the attack direction is plausible and the adaptation of TAR to watermarking is conceptually interesting. However, the current experimental rigor is insufficient for a full archival publication: the defense gradient is ambiguous, no baseline table is provided, and all numbers are single runs. If the authors clarify Algorithm 1, add baselines and error bars, and define the hidden-state regularizer, the paper may be suitable for a workshop or as a short paper, but not in its present form."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my read. The paper's attack result is the thing to remember: fine-tuning a Stable Signature decoder with random 48-bit keys for 100 steps drops extracted watermark bit accuracy to ~0.54-0.56 on eval while PSNR stays ~27-28. That is a clean vulnerability demonstration, and it's a simple but real extension of the model-purification attack from Fernandez et al. [1]—instead of dropping the watermark loss, you feed it random keys. The direction is clear and the numbers match what you'd expect. The paper earns credit for that.\n\nThe defense is where I have problems. The adapted TAR algorithm (Algorithm 1) is the main contribution, but line 7 defines gTR as (1/K) Σ ∇_{θ_{i-1}} L_TR(attack(θ_{i-1}), x_TR), and the paper never says whether this gradient is unrolled through the inner K attack steps. If it's not, the gradient is not the derivative of the tamper-resistance loss with respect to θ_{i-1}, and the outer update isn't actually training the decoder to recover from attacks. That ambiguity is load-bearing—the reported post-attack bit accuracy of 64–65% could come from the retain loss or the undefined hidden-state regularizer, not from true robustness. This isn't a nitpick; it's the core of the defense claim.\n\nBeyond that, every number is a single run with no error bars or seeds, there's no baseline table comparing against the original Stable Signature decoder under the same attack (only a figure that's missing from the preprint), and the h_θ hidden states for the LDM decoder are never defined. The conclusion itself is honest—it says 'minor improvements' and notes the key-space problem—so the authors aren't overselling. But the paper as written doesn't establish the defense works as claimed.\n\nWho's this for? People working on watermark robustness for generative models. The attack result is worth knowing even if the defense is unproven. I'd send it to peer review—the attack alone justifies referee time, and the gradient question is answerable. But the authors need to specify the backpropagation, add statistical rigor, and include the baseline before publication.","headline":"Plausible attack result, but the defense's central algorithm is underspecified; worth a referee to sort out the gradient question.","tokens_in":7064,"tokens_out":2496,"would_cite":true,"duration_ms":20062,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper shows that random-key fine-tuning makes Stable Signature watermarked latent diffusion decoders embed near-random keys while keeping image quality, and that an LLM-inspired tamper-resistant fine-tuning only partly restores…","keywords":["watermarking","latent diffusion models","Stable Signature","adversarial fine-tuning","tamper-resistant fine-tuning","random key attack","image provenance","model purification"],"falsifier":"Re-run the TAR training while backpropagating the tamper-resistance loss through all $K$ inner attack steps, and also run an ablation with $\\lambda_{TR}=0$; if post-attack bit accuracy stays near 50% or matches the ablation under both conditions, the attack-based gradient is not the source of the reported tamper resistance.","tokens_in":6016,"feed_emoji":"🖼️","tokens_out":8108,"duration_ms":62092,"temperature":0.7,"pith_summary":"Watermarking is meant to let users trace an AI-generated image back to the model that made it, and the Stable Signature method does this by fine-tuning a latent diffusion decoder so every image it produces carries an invisible key. The paper tries to show that this watermark is vulnerable to a network-level attack: if an attacker fine-tunes the decoder for about 100 steps while feeding a random watermark key at each step, the extracted bit accuracy falls to roughly 56% on evaluation data, close to the 50% coin-flip level, while PSNR stays near 28. The paper then adapts a tamper-resistant fine-tuning algorithm originally designed for large language models, reporting that the defended decoder keeps 96–99% bit accuracy before an attack and about 64–65% after an attack. The paper concludes that tamper resistance for LDM watermarking is tractable but incomplete, because the $2^{48}$ key space has no distributional structure and LDMs lack an equivalent baseline defense.","feed_headline":"Random-key fine-tuning breaks Stable Signature watermarking","feed_subtitle":"After 100 attack steps, extracted watermark bit accuracy falls to near coin-flip while image quality stays high.","key_machinery":"The load-bearing object is the fine-tuned decoder of a latent diffusion model together with a HiDDeN-style watermark extractor, trained with a combined loss $L = L_m + L_i$ where $L_m$ is the binary cross-entropy between the extracted and target key and $L_i$ is an image reconstruction/perceptual loss that preserves image quality. The attack mechanism is to swap the target key in $L_m$ for a uniformly random 48-bit key (or a gradually randomized one) during fine-tuning. The defense mechanism is a modified TAR fine-tuning loop: for each outer step the current decoder is attacked $K$ times with random keys, the gradient of the resulting tamper-resistance loss is averaged to form $g_{TR}$, a retain gradient $g_{retain}$ combines the original message loss, reconstruction loss, and an $\\ell^2$ penalty on differences between current and original hidden states, and the decoder is updated with a weighted sum $\\lambda_{TR} g_{TR} + \\lambda_{retain} g_{retain}$. The idea is that $g_{TR}$ teaches the decoder how to recover after an attack while $g_{retain}$ keeps it producing the intended watermark and high-quality images.","core_discovery":"The central claim is that the Stable Signature watermark, although rooted in the decoder weights, can be disrupted by adversarial fine-tuning that does not harm image quality. In the proposed Random Key Attack, each fine-tuning step replaces the intended 48-bit watermark with a uniformly random key in the message loss; after 100 steps the decoder still produces clean images (PSNR about 27.6 on evaluation data) but the extracted key matches the intended one only about 56% of the time, and the gradual variant lands near 54%. This is close to chance, so the images can no longer be reliably attributed to the original decoder. The paper's defensive claim is that a TAR-style fine-tuning, which alternately attacks the current decoder with random keys and reinforces the original key plus image reconstruction, raises pre-attack bit accuracy to 96–99% and post-attack accuracy to about 64–65%, with PSNR in the 35–51 range before attack and 29–32 after; the authors read this as a partial improvement that leaves full tamper resistance open.","pith_inferences":["The same random-key fine-tuning recipe would likely transfer to any watermarking scheme that trains a fixed key through a differentiable message loss against a public extractor, since the attack never needs the original key, only the ability to fine-tune and to sample random ones.","A post-attack accuracy around 64–65%, while too low for single-image attribution, may still support batch-level statistical tracing; the paper does not analyze how many images would be needed to identify the source reliably.","An ablation setting $\\lambda_{TR} = 0$ would reveal whether the attack-based gradient contributes anything beyond the retain loss and hidden-state regularization; if the post-attack accuracy barely changes, the reported tamper resistance may be a regularization artifact.","Varying the key length or using structured/error-correcting keys could test whether the 'no distribution' barrier is fundamental to the watermarking task or a consequence of using uniform random 48-bit strings."],"forward_implications":["If the attack works as reported, open-weight latent diffusion decoders watermarked with Stable Signature cannot be trusted to stay traceable after an adversary fine-tunes them, even when the images themselves look unchanged.","The adapted TAR defense gives a concrete recipe: alternate random-key attacks with reinforcement of the original key, and use both tamper-resistance and retain gradients, to keep post-attack bit accuracy above chance.","A post-attack bit accuracy of 64–65% means traceability degrades substantially under attack but does not collapse entirely, so provenance decisions would need to account for the attack state of the model.","The absence of any structure in the 2^48 key space makes LDM watermark tamper resistance fundamentally harder than LLM safety guardrails, where some redundancy in the data helps the defense.","Because PSNR remains high in both attack and defense experiments, the attack and defense operate almost entirely on the watermarking objective, not on image quality."],"supporting_citations":[{"why":"Supplies the Stable Signature decoder fine-tuning method that the paper attacks and defends.","marker":"[1]"},{"why":"Supplies the TAR tamper-resistant fine-tuning algorithm adapted for the LDM watermarking setting.","marker":"[2]"},{"why":"Provides the HiDDeN encoder-extractor watermark network whose extractor and message loss the Stable Signature pipeline uses.","marker":"[5]"},{"why":"Provides the latent diffusion model architecture, specifically the Stable Diffusion decoder that is fine-tuned.","marker":"[4]"},{"why":"Defines the denoising diffusion framework underlying the image generation process.","marker":"[3]"},{"why":"Supplies the COCO dataset used for fine-tuning and evaluating the watermarked decoders.","marker":"[6]"}],"fun_headline_variants":["Random-key fine-tuning breaks Stable Signature watermark","Stable Signature watermark cracked by random-key attack","Adversarial fine-tuning strips Stable Signature watermarking","Random-key fine-tuning defeats watermark attribution","Fine-tuning with random keys renders Stable Signature untraceable"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the tamper-resistance gradient in Algorithm 1 is computed correctly with respect to the pre-attack weights, including the inner attack steps; if it is evaluated only on the final attacked model, the reported post-attack tamper resistance may come from the retain loss or noise rather than from true resistance, and the hidden-state term $h_\\theta$ for an LDM decoder is left undefined.","fun_headline_variants_meta":{"raw":{"variants":["Random-key fine-tuning breaks Stable Signature watermark","Stable Signature watermark cracked by random-key attack","Adversarial fine-tuning strips Stable Signature watermarking","Random-key fine-tuning defeats watermark attribution","Fine-tuning with random keys renders Stable Signature untraceable"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000259,"raw_usage":{"total_tokens":1581,"prompt_tokens":934,"completion_tokens":647,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":550,"completion_tokens_details":{"reasoning_tokens":576}},"tokens_in":550,"tokens_out":647,"duration_ms":5460,"temperature":1.0,"reasoning_tokens":576,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T10:14:58.502797+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the TAR training while backpropagating the tamper-resistance loss through all $K$ inner attack steps, and also run an ablation with $\\lambda_{TR}=0$; if post-attack bit accuracy stays near 50% or matches the ablation under both conditions, the attack-based gradient is not the source of the reported tamper resistance.","supporting_citations":[{"cited_title":"The Stable Signature: Rooting Watermarks in Latent Diffusion Models","cited_arxiv_id":"2303.15435","evidence_quote":"Defines the denoising diffusion framework underlying the image generation process."}],"review_version":1}