{"id":"4817ee15-fdca-4731-aef3-b554a0ccc9a6","arxiv_id":"2506.04879","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Invisible watermarks from off-the-shelf models can serve as backdoor triggers that force an InstructPix2Pix editing model to output a preset image.","lead":"This paper shows how an image editing model can be poisoned so that images carrying an invisible digital watermark are edited to a fixed target image, while clean images edit normally. The work is relevant to security researchers and anyone concerned about invisible manipulations of AI editing tools.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Threshold-fixed ASR and single-run evaluation are the load-bearing fragile points; the one-step MSE proxy is standard and empirically supported.","rationale":"Good-faith reading: the paper proposes a backdoor attack for instruction-based image editing by fine-tuning InstructPix2Pix on watermarked images paired with a fixed target, and the main evidence is the high ASR and preserved utility in Table 1. The design is plausible, and the released GitHub repository is a concrete artifact that can be independently checked. The reader's verdict of CONDITIONAL is reasonable, but the reader's singled-out weakest assumption, the one-step MSE proxy from Eq. 3, is not the most load-bearing concern. That proxy is a standard x0-prediction objective in diffusion training, and the final-sample ASR in Table 1 is direct evidence that it transferred to full sampling in the tested configuration. The more decisive fragility is metric-level: ASR and EAR are binary classifications against a single fixed threshold phi=0.1, with no threshold sweep and no repeated-seed statistics. If the threshold is near a dense region of the MSE distribution, the apparent attack success rate could be an artifact of that choice. A second issue is generality: VINE's lower ASR and its explicit exclusion from Sec. 4.3 weaken the 'across watermarking models' claim, although StegaStamp and RoSteALS still support the core attack concept. These concerns do not refute the central claim, but they mean the paper should be accepted only after threshold-robustness and seed-variance checks are reported, which matches the conditional verdict.","tokens_in":8671,"tokens_out":9838,"duration_ms":136326,"concrete_test":"Re-evaluate ASR and EAR on the same 1,000-image test set with a dense grid of MSE thresholds (e.g., 0.02, 0.05, 0.08, 0.10, 0.15, 0.20) and recompute Table 1. Then re-run StegaStamp and RoSteALS training with at least three random seeds and report mean and standard deviation for each metric. If ASR at the fixed threshold phi=0.1 is not robust to adjacent thresholds (e.g., drops below 0.8 at phi=0.05) or if seed-to-seed variance in ASR exceeds roughly 0.1, the headline attack-success claim is not established.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central quantitative claim rests on ASR=0.956 (StegaStamp) and 0.894 (RoSteALS) in Table 1, computed via Eq. 8 with a single fixed MSE threshold phi=0.1 (Appendix B). The paper reports no ASR/EAR threshold sweep, no MSE value distributions, and no repeated-seed variance. Since phi is a free parameter, the reported high ASR could be unrepresentative: if the generated-target MSE values cluster just below 0.1, a modestly stricter threshold (e.g., phi=0.05) could reduce ASR substantially. The same fragility applies to the single training run; a lucky initialization could inflate both ASR and utility. This is load-bearing because the central claim 'promising attack success rates' is established only through these binary threshold-based metrics. Additionally, Sec. 4.3 explicitly excludes VINE from the robustness study after Table 1 already shows VINE with ASR 0.552 and MSE 0.377, so the 'across different watermarking models' generality is weaker than the headline suggests. The one-step MSE loss in Eq. 4/6 is not the most fragile assumption: Eq. 3 is the standard DDPM x0 estimate, and the final-sample ASR in Table 1 provides direct evidence that the proxy did not diverge badly in the tested regime.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a backdoor attack on InstructPix2Pix by fine-tuning the model on a mixture of clean images and images carrying an imperceptible watermark embedded by an off-the-shelf deep watermarking model (StegaStamp, VINE, or RoSteALS). A fixed secret message is used as the trigger, and the model is trained with a denoising loss plus an image-space MSE loss so that watermarked inputs generate a predefined backdoor target while clean inputs preserve normal instruction-based editing. Experiments report ASR up to 0.956 (StegaStamp) and 0.894 (RoSteALS) with CLIP-based utility metrics close to the clean model, robustness under several WAVES distortions for two of the three watermarkers, and an analysis attributing attack success to larger latent residuals between original and watermarked images.","tokens_in":8925,"tokens_out":6886,"duration_ms":82185,"significance":"If the results hold, this is a practical and novel attack: it is the first invisible-trigger backdoor for instruction-based image editing, it leverages off-the-shelf watermarking models without per-sample trigger optimization, and the code release supports reproducibility. The method is simple, clearly described, and the two-branch training objective is reasonable. I do not regard the one-step MSE proxy of Eq. (3) as a blocking issue, because the final-sample ASR in Table 1 provides empirical evidence that the proxy does not diverge badly in the tested regime. However, the empirical support is currently thinner than the headline: the central ASR/EAR numbers depend on a single fixed threshold, the experiments are single-run, and the latent-residual explanation rests on only three watermarking models, which is insufficient to establish the stated contribution about watermark properties.","major_comments":[{"comment":"ASR and EAR are defined with a fixed MSE threshold phi=0.1 inherited from VillanDiffusion [6], and Table 1 reports only the resulting binary rates and the mean MSE. No threshold sweep or per-sample MSE distribution is given, so the reader cannot assess how sensitive the reported ASR values are to phi. This is load-bearing because the central claim of high attack success rests entirely on this binary threshold; for RoSteALS the mean MSE (0.092) is close to the threshold, and a stricter threshold could substantially change the conclusions. Please report ASR/EAR as a function of phi, provide per-sample MSE quantiles, and justify that the threshold from [6] transfers to this image-editing setting.","section":"Section 4.1 / Appendix B, Eq. (8)"},{"comment":"All experimental results, including Table 1, Figure 5, and Figure S.1, appear to come from a single training run per configuration. Fine-tuning a diffusion model is stochastic, and no variance information is reported for ASR, EAR, MSE, or the CLIP utility metrics. A single run cannot establish that the attack succeeds reliably or that the reported utility preservation is not a lucky draw. Please run at least three seeds per configuration and report mean and standard deviation (or otherwise justify determinism), and report seed information in the implementation details.","section":"Section 4 / Appendix A, Table 1"},{"comment":"The paper claims that larger latent residuals between original and watermarked images explain attack success, but this claim is supported by only three watermarking models. This is effectively a three-point correlation, with no quantitative association reported, and latent residual is confounded with watermarking architecture, perceptibility, and robustness. Since this analysis is listed as one of the main contributions, it should be framed as a hypothesis or validated with more watermarking models, or with a per-sample residual-versus-success analysis within a single watermarker.","section":"Section 4.2, Figure 4"},{"comment":"The robustness evaluation excludes VINE after Table 1 shows it to be the weakest watermarker, and the contribution bullet claims 'strong robustness under various real-world distortions on the altered input images' without explicitly scoping this to StegaStamp and RoSteALS. The exclusion is understandable, but the text should state clearly in the abstract and contributions that the robustness claim covers only two of the three watermarking models, and that the generality across watermarking models is therefore limited.","section":"Section 4.3, Table 2 / Section 1 Contributions"}],"minor_comments":[{"comment":"There are small wording errors: the abstract says 'in term of backdoor attack' and the third contribution bullet says 'The proposed backdoor attack method show'; both should be corrected ('in terms of' and 'shows').","section":"Abstract / Section 1"},{"comment":"The product notation 'Qt s=1' is malformed; it should be written as a product symbol over s=1,...,t, e.g., \\prod_{s=1}^{t}.","section":"Section 3.1, Eq. (3)"},{"comment":"For StegaStamp under JPEG compression, the EAR is 0.083, which is an order of magnitude higher than in most other distortion rows; the statement that both methods achieve 'low EAR' under JPEG should be qualified.","section":"Table 2, JPEG row"},{"comment":"The robustness subsection should explicitly restate that the reported results apply only to the StegaStamp- and RoSteALS-based backdoor models, to avoid the impression that the VINE-based model was also evaluated under distortions.","section":"Section 4.3"},{"comment":"The title and abstract use the term 'invisible watermarks', but Table 3 reports PSNR 31.88 and 28.37 for StegaStamp and RoSteALS, respectively; these are visually subtle but not strictly imperceptible. Consider using 'visually imperceptible' only for VINE, or adding a perceptual validation for the other two.","section":"Table 3"},{"comment":"The implementation details note that the StegaStamp model is trained by the authors rather than using an official checkpoint; this should be stated in the main text as a caveat, since the watermarking behavior may differ from the official StegaStamp model used elsewhere in the literature.","section":"Appendix A"}],"recommendation":"major_revision","confidential_remarks":"The central concern is not circularity or novelty but evaluation robustness. The threshold-sensitivity and single-run issues are fixable within the manuscript's scope, and they should be addressed before acceptance. The paper seems to be a good fit for the journal's audience, and the authors' citation of prior backdoor and watermarking work appears appropriate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short take: this paper does something new and useful—first to carry invisible deep-watermark triggers into instruction-based image editing—and its main table is believable. But the evaluation is more fragile than the prose suggests: one run, one threshold, and a three-point causal story.\n\nWhat's good: The attack recipe is simple: fine-tune InstructPix2Pix on a mix of clean images and watermarked images, adding an MSE loss on a one-step reconstruction. It works. StegaStamp and RoSteALS give ASR above 0.89 while CLIPdir stays within noise of the clean model. The robustness results under erasing and JPEG are meaningful for practical watermark-protection scenarios. The paper also honestly reports VINE's poor performance instead of cherry-picking. Code is public, and the one-step proxy in Eq. 3 is standard practice; the final-sample ASR shows it didn't diverge.\n\nThe soft spots are real but not fatal. Everything rests on a single training run per watermark, so no variance estimate exists. ASR and EAR are computed with a fixed MSE threshold (phi=0.1) borrowed from prior work; the paper doesn't show how sensitive the numbers are to that choice. Given VINE's MSE is 0.377 and StegaStamp's is 0.038, the ranking would likely survive a threshold change, but absolute ASR would move, and a stricter threshold might drop StegaStamp below 0.9. The latent-residual explanation is suggestive but drawn from only three watermark models; it's a plausible post-hoc story, not a tested mechanism. And excluding VINE from the robustness study is fine—its specificity was too low—but the abstract's \"across different watermarking models\" should be read accordingly.\n\nOverall, this is a solid subfield contribution, not a breakthrough. The core claim holds up, the method is reproducible, and the limitations are mostly about evaluation depth. I'd send it to peer review. The authors should add multiple seeds, a threshold sensitivity analysis, and ideally one or two more watermark models to firm up the causal claim.","headline":"Solid first integration of invisible deep-watermark triggers into editing backdoors; evaluation needs multi-seed and threshold sensitivity work before the numbers are taken at face value.","tokens_in":9426,"tokens_out":2223,"would_cite":true,"duration_ms":26556,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Invisible watermarks can hijack an instruction-based image editor.","keywords":["backdoor attack","image editing","diffusion model","deep watermarking","invisible trigger","InstructPix2Pix","poisoned fine-tuning","latent residual"],"falsifier":"Re-run the same training but compute the pixel MSE from full multi-step sampled outputs instead of the one-step estimate in equation 3; if ASR or clean-editing metrics move materially, the one-step proxy is doing the work. A cheaper check is to collect cases where the one-step reconstruction is close to the backdoor target but the final sampled image is not, and see whether those cases account for the attack failures.","tokens_in":8487,"feed_emoji":"🖼️","tokens_out":6435,"duration_ms":73497,"temperature":0.7,"pith_summary":"This paper tries to show that an image editing model can be silently reprogrammed through its training data: when the input photo carries an imperceptible deep watermark, the model produces a preset backdoor target image regardless of the text prompt, while unmarked photos are edited normally. The attack is built on InstructPix2Pix and uses off-the-shelf watermark encoders (StegaStamp, RoSteALS, VINE) to stamp the trigger. Fine-tuning with 10 percent poisoned samples yields attack success rates up to 0.956 for StegaStamp, with editing quality on clean images nearly unchanged (CLIP direction similarity 0.208 versus 0.207 for the clean model). This matters because it is the first invisible-trigger backdoor for instruction-based editing, and it suggests that invisible watermarks can both protect and attack image editing pipelines.","feed_headline":"Invisible watermarks can hijack an instruction-based image editor","feed_subtitle":"A poisoned fine-tuning hides a trigger in watermarked images, forcing a preset output while normal edits stay intact.","key_machinery":"The machinery is the joint training objective L_total = L_backdoor_denoising + L_backdoor_mse + L_clean_denoising + L_clean_mse, with the MSE terms computed from a one-step denoising estimate of the generated image (equation 3). The watermark encoder F_enc(·) is the trigger carrier, and the text prompt is left unchanged so the trigger is purely visual. The paper's explanatory mechanism is the latent residual r*_i = ||E(x_w*_i) - E(x_i)||_2: larger residuals between watermarked and clean latents make the backdoor behavior easier to learn, and the results are interpreted through this lens.","core_discovery":"The central claim is that a two-branch fine-tuning objective can teach an editing diffusion model to treat a deep watermark as a hidden command. In the backdoor branch, the model is trained to denoise toward a fixed target image y' when conditioned on a watermarked photo x_w = F_enc(x_p, s), using both the standard denoising loss and an MSE loss between the target and the one-step reconstructed image. In the clean branch, it is trained with unmarked images and their ground-truth edits to preserve normal instruction-following. The paper further claims that the effectiveness of the attack tracks the L2 distance between the latent codes of original and watermarked images, with StegaStamp producing the largest latent residual and the highest attack success rate. Under this account, any watermarking scheme that creates a sufficiently large, imperceptible latent shift can serve as a backdoor trigger for editing models.","pith_inferences":["The paper frames the result as an attack, but the same mechanism could be read as a defensive capability: content owners could watermark their images so that any edited copy reverts to a preset 'unauthorized' marker; testing this defensive framing is a direct next step.","The reliance on the one-step denoising estimate is a testable weak point: if training with full multi-step sampling changes behavior, then the reported ASR partly reflects the proxy, not the final diffusion trajectory.","Since trigger success tracks latent residual size, a future watermarking model optimized specifically to maximize latent separation under imperceptibility constraints could make the backdoor stronger and harder to remove than the three off-the-shelf schemes tested here.","The distortion robustness results suggest the backdoor is only as robust as the underlying watermark; combining triggers with watermarking schemes designed to survive editing operations could extend the attack to images that are compressed, cropped, or recolored before being edited."],"forward_implications":["An adversary who controls fine-tuning data can install a backdoor that fires on any watermarked image, with no visible change to the input.","At poison rate 0.1, the backdoored model's editing quality on clean images is essentially preserved, making the attack hard to notice during normal use.","The trigger survives erasing and JPEG compression well, so simple post-processing does not reliably remove it.","By assigning different secret messages to different targets, the framework extends to multiple trigger-target pairs without much loss in ASR.","If the latent-residual explanation is right, any watermarking method with a large latent shift could be adapted into a backdoor trigger, extending beyond the three encoders tested."],"supporting_citations":[{"why":"Supplies the InstructPix2Pix base model, its training dataset of image-pair/prompt triples, and the pretrained weights the attack fine-tunes.","marker":"[2]"},{"why":"Defines the backdoor target image and the general evaluation protocol for backdoored diffusion models that this paper adapts to editing.","marker":"[5]"},{"why":"Provides the unified backdoor framework and the MSE-threshold method used to compute ASR and EAR for specificity metrics.","marker":"[6]"},{"why":"Provides the StegaStamp watermark encoder, which in these experiments gives the strongest attack success rate and the cleanest utility.","marker":"[11]"},{"why":"Provides the VINE watermark encoder, the weakest trigger in the comparison, used to contrast latent-residual effects.","marker":"[10]"},{"why":"Provides the RoSteALS watermark encoder, the second-strongest trigger, also used to test the generality of the attack across watermarking methods.","marker":"[3]"},{"why":"Supplies the WAVES distortion benchmark used to test backdoor robustness under geometric, photometric, and degradation attacks.","marker":"[1]"},{"why":"Establishes the concept of invisible sample-specific triggers via deep watermarking in classification, which this paper extends to instruction-based editing diffusion models.","marker":"[9]"},{"why":"Supplies the training strategy used to train the StegaStamp model on MIRFLICKR, making the trigger encoder reproducible in the paper's setup.","marker":"[12]"}],"fun_headline_variants":["Invisible watermarks can hijack image editing models","Deep watermarking enables hidden backdoor triggers in editing","Watermarks as invisible backdoor triggers for image editors","Stealthy watermark backdoors compromise image editing","Hiding triggers in watermarks attacks image editing"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the pixel-space training loss, computed from a one-step denoising estimate of the generated image, is a reliable stand-in for the image the model actually produces after full multi-step sampling during evaluation; if that proxy diverges from the real generation process, the trained model may not learn the behavior the experiments measure.","fun_headline_variants_meta":{"raw":{"variants":["Invisible watermarks can hijack image editing models","Deep watermarking enables hidden backdoor triggers in editing","Watermarks as invisible backdoor triggers for image editors","Stealthy watermark backdoors compromise image editing","Hiding triggers in watermarks attacks image editing"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00019,"raw_usage":{"total_tokens":1336,"prompt_tokens":937,"completion_tokens":399,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":553,"completion_tokens_details":{"reasoning_tokens":325}},"tokens_in":553,"tokens_out":399,"duration_ms":4788,"temperature":1.0,"reasoning_tokens":325,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T10:30:56.647363+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the same training but compute the pixel MSE from full multi-step sampled outputs instead of the one-step estimate in equation 3; if ASR or clean-editing metrics move materially, the one-step proxy is doing the work. A cheaper check is to collect cases where the one-step reconstruction is close to the backdoor target but the final sampled image is not, and see whether those cases account for the attack failures.","supporting_citations":[{"cited_title":"Brooks, A","cited_arxiv_id":null,"evidence_quote":"Supplies the InstructPix2Pix base model, its training dataset of image-pair/prompt triples, and the pretrained weights the attack fine-tunes."},{"cited_title":"Chou, P.-Y","cited_arxiv_id":null,"evidence_quote":"Defines the backdoor target image and the general evaluation protocol for backdoored diffusion models that this paper adapts to editing."},{"cited_title":"Chou, P.-Y","cited_arxiv_id":null,"evidence_quote":"Provides the unified backdoor framework and the MSE-threshold method used to compute ASR and EAR for specificity metrics."},{"cited_title":"Tancik, B","cited_arxiv_id":null,"evidence_quote":"Provides the StegaStamp watermark encoder, which in these experiments gives the strongest attack success rate and the cleanest utility."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the VINE watermark encoder, the weakest trigger in the comparison, used to contrast latent-residual effects."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the RoSteALS watermark encoder, the second-strongest trigger, also used to test the generality of the attack across watermarking methods."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the WAVES distortion benchmark used to test backdoor robustness under geometric, photometric, and degradation attacks."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes the concept of invisible sample-specific triggers via deep watermarking in classification, which this paper extends to instruction-based editing diffusion models."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the training strategy used to train the StegaStamp model on MIRFLICKR, making the trigger encoder reproducible in the paper's setup."}],"review_version":1}