{"id":"8a1323c0-9c4f-4bc1-9d63-84f298406f0b","arxiv_id":"2412.09910","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Prompt2Perturb finds Stable Diffusion text embeddings that turn breast ultrasound images into adversarial examples that are natural-looking and mislead classifiers.","lead":"The authors propose Prompt2Perturb, an attack that finds text embeddings in Stable Diffusion that turn a breast ultrasound image into an adversarial example that fools a classifier with minimal visual change. It matters because it shows a practical route to generate natural-looking attacks on medical image classifiers without retraining a diffusion model, and it highlights evaluation pitfalls in such claims.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline FID/LPIPS comparison is confounded by best-of-K image selection: §3.2 selects the candidate most similar to the original, a selection rule not available to baselines, so claimed gains may be an evaluation artifact.","rationale":"Reader's weakest_assumption is about Stable Diffusion being out-of-distribution for ultrasound. I agree this is plausible and worth checking, but it is not the most decisive issue: even if reconstruction fidelity were perfect, the reported FID/LPIPS advantage could be produced by selecting the most similar candidate among many. The reader's rationale does mention the 'final-image selection rule' and non-comparable evaluation sets, so there is partial overlap; I would elevate that to the primary concern. The method has real strengths: freezing the diffusion model, optimizing early steps, reporting ablations and qualitative t-SNE/visuals. Those do not, however, settle the fairness of the headline comparison. I keep the conditional verdict: acceptance should require a reproducible selection protocol, matched evaluation sets, and error bars.","tokens_in":13584,"tokens_out":5524,"duration_ms":62869,"concrete_test":"Re-run the BUSI/ResNet34 experiment under a controlled protocol: (1) fix one candidate per image by taking the final iteration's xadv (or a single fixed seed), and (2) additionally run Diff-PGD and P2P with the same best-of-K rule over K=10 restarts, selecting the candidate with highest similarity to x. Report LPIPS, SSIM, and FID on the identical validation-image set (not only successful attacks) with five-fold confidence intervals and the selected similarity metric stated. If P2P's advantage over Diff-PGD in LPIPS/FID is not consistent across (1) and (2), the claimed outperformance is an artifact of the selection rule.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing weakness is the final-image selection procedure, not the diffusion backbone. Section 3.2 states: 'the image that shows the highest similarity to the original input is selected as the adversarial output,' yet Algorithm 1 returns an xadv inside the iteration loop and never specifies how many candidates are generated or how 'similarity' is measured. If P2P performs any form of best-of-K selection over candidates (across iterations, seeds, or checkpoints), then the LPIPS, SSIM, and FID comparisons in Tables 1-3 are not apples-to-apples: FGSM, PGD, and Diff-PGD produce one deterministic adversarial image per input, while P2P would be allowed to discard less similar candidates. Since LPIPS/FID are exactly the metrics on which the abstract claims superiority, the observed advantage could be a selection artifact rather than a property of prompt-embedding optimization. A second, compounding issue is that all metrics are computed only on 'successful attack examples,' so each method is scored on a different image subset; with no variance reported it is impossible to tell whether P2P's FID advantage is within noise. These two issues are load-bearing because they directly undermine the paper's central quantitative claim, independent of whether Stable Diffusion is in-distribution for ultrasound.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Prompt2Perturb (P2P), a text-guided diffusion-based adversarial attack for breast ultrasound images. Instead of adding fixed-norm perturbations or fine-tuning a diffusion model, P2P optimizes text embeddings in a frozen Stable Diffusion v1.4 text encoder, using only the first 50 reverse-diffusion steps (t ≤ 50) and a loss that combines a cross-entropy term (with a -1e10 multiplier) and an MSE term on the predicted noise. Experiments compare P2P against FGSM, PGD, and Diff-PGD on three breast ultrasound datasets (BUSI, BUS-BRA, UDIAT) using ResNet34, SqueezeNet1.1, and DenseNet121, reporting success rate, LPIPS, SSIM, and FID. The abstract claims that P2P outperforms state-of-the-art attacks across all three datasets in FID and LPIPS.","tokens_in":13880,"tokens_out":4062,"duration_ms":44687,"significance":"If validated, the proposal is interesting because it avoids retraining a domain-specific diffusion model, which is important for data-scarce medical domains. The evaluation is externally grounded: three public datasets and three classifier architectures, with an ablation study on the loss and step-range choices. The paper also provides qualitative visualizations and t-SNE plots. However, the headline quantitative claim about FID and LPIPS superiority is currently not substantiated because the evaluation protocol appears confounded by candidate selection and by computing perceptual metrics only on the per-method subsets of successful attacks.","major_comments":[{"comment":"The final-image selection procedure is underspecified and can invalidate the comparison. The text states that \"the image that shows the highest similarity to the original input is selected as the adversarial output,\" but Algorithm 1 returns xadv inside the iteration loop and does not state how many candidates are considered, which similarity measure is used, or how the selected image relates to the loop iterations. If any form of best-of-K selection over iterations, seeds, or checkpoints is performed, then LPIPS, SSIM, and FID comparisons in Tables 1–3 are not apples-to-apples with FGSM, PGD, and Diff-PGD, which emit a single deterministic output per input. Since the abstract's headline is precisely the FID/LPIPS improvement, this selection rule must be fully specified, and the baselines must be given the same selection allowance or P2P must be evaluated without such selection.","section":"§3.2, Algorithm 1"},{"comment":"All perceptual metrics are computed only on \"successful attack examples,\" and success rates differ substantially across methods (e.g., Diff-PGD 1.0 vs P2P 0.86 for DenseNet121 on UDIAT; Diff-PGD 0.75 vs P2P 0.74 for SqueezeNet1.1 on BUS-BRA). This means each method is scored on a different subset of images, and the reported FID/LPIPS advantage may reflect selection of easier examples rather than intrinsic perceptual quality. The paper also reports no error bars or standard deviations for any table. The authors should either compute FID/LPIPS on a fixed common subset (e.g., the intersection of images successfully attacked by all methods) or report variance and per-class sample counts; otherwise the central claim of superiority is not statistically supported.","section":"Tables 1–3, §4.2"},{"comment":"The loss definition is problematic as written. Algorithm 1 sets Loss ← −1010 × CrossEntropyLoss(˜y, y) + mse(ϵ, ϵθ(zt, C)). The -1e10 multiplier is unexplained and appears to be a hand-tuned free parameter; no sensitivity analysis or justification is provided. Additionally, the MSE term calls ϵθ(zt, C) without the timestep argument t, which is inconsistent with the LDM objective defined in §3.2 (where the network is ϵθ(zt, t, cθ(p))). The authors should clarify the exact loss, the role of the multiplier, and why the resulting optimization does not produce numerical instability or trivial solutions.","section":"Algorithm 1, §3.2"},{"comment":"There is a direct contradiction between the methodological description and the implementation regarding which diffusion timesteps are optimized. The text says \"Optimizing only the early timesteps of denoising process (akin to later timesteps of diffusion process)\" and justifies this by low-frequency structure, but Algorithm 1 samples t ← randint(0, 50), which are early diffusion timesteps with low noise (close to the clean image), i.e., the late stages of the reverse process. This inconsistency undermines the stated rationale for the choice t ≤ 50 and the ablation study in Table 4, which varies T=20, T=50, T=100. The authors must reconcile the text with the algorithm and clarify whether they are optimizing high-noise or low-noise steps.","section":"§3.2 'Minimal reversal steps' vs Algorithm 1"}],"minor_comments":[{"comment":"Typo: \"approch\" should be \"approach.\"","section":"§3.2"},{"comment":"Reference [13] contains the placeholder \"Accessed: [Insert Date of Access]\" and should be completed.","section":"References"},{"comment":"The indentation and control flow are ambiguous: the 'return xadv' appears inside the batch loop and inside the iteration loop, so it is unclear when the function actually terminates and how the 'highest similarity' selection is implemented.","section":"Algorithm 1"},{"comment":"The factual claim that the word \"benign\" is not present in the Stable Diffusion text encoder vocabulary is surprising and is given without any citation or verification; please provide evidence or remove the claim.","section":"§3.2"},{"comment":"The hyperparameter settings for Diff-PGD are only described as \"the same setting as it used for ImageNet,\" without specifying the number of steps, step size, or noise schedule used in this paper; this hampers reproducibility.","section":"§4.1"}],"recommendation":"major_revision","confidential_remarks":"The core idea is potentially useful for medical imaging, but the evaluation protocol currently does not support the abstract's headline claims. The selection-procedure confound and the per-success-subset evaluation are the two most damaging issues, and the timestep inconsistency in Section 3.2/Algorithm 1 suggests the method description needs careful rewriting. I would recommend that the editor ask the authors to release the exact selection rule and metrics code, and to re-run the comparison under a common evaluation protocol, before considering acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea is sensible: freeze a latent diffusion model, optimize text embeddings directly to fool a classifier, and only run the early reverse steps. That combination is not in the cited literature, and the paper shows it produces attack images that look more natural than Diff-PGD output on breast ultrasound. The early-step heuristic is ablated with a clear efficiency/quality trade-off, and the qualitative figures support the claim that the perturbations are less textured/noisy than PGD-type attacks. The application to three public ultrasound datasets is a reasonable transfer, and the writing is clear.\n\nThe soft spots are the quantitative evaluation. The biggest one, which the stress-test note correctly identifies, is the final-image selection rule. Section 3.2 says \"the image that shows the highest similarity to the original input is selected as the adversarial output,\" but Algorithm 1 returns an xadv lazily and never says how many candidates are generated or what \"similarity\" means. If P2P gets to pick the best of several candidates while FGSM, PGD, and Diff-PGD each return one deterministic image, then the LPIPS and FID comparisons in Tables 1–3 are not apples-to-apples. Since those two metrics are exactly what the abstract claims superiority on, this is a load-bearing flaw, not a minor omission.\n\nCompounding that, all metrics are reported only on successful attack images, and the success rates differ across methods (e.g., Diff-PGD 1.0 vs P2P 0.86 on UDIAT/DenseNet121). So each method is scored on a different subset, and there are no error bars anywhere in the main tables. The paper also skips the most closely related baselines: Instruct2Attack and BAPLe are discussed but never compared against. The −1e10 multiplier on the cross-entropy loss is unexplained and looks like a red flag that should be justified.\n\nThe central attack methodology is not self-contradictory and the qualitative story is credible, so I would not desk-reject it. But the claim of state-of-the-art FID/LPIPS is not established by the current evidence. The paper deserves a serious referee, who should demand: a precise description of the selection procedure, evaluation on matched image sets with variance, comparison against Instruct2Attack and BAPLe, and a justification of the loss weighting. If those are fixed, this could be a useful contribution for the medical-image adversarial-attack community. For now, my advice is major revision before acceptance.","headline":"The method is plausible and the qualitative results look good, but the headline FID/LPIPS advantage is probably a selection artifact, so the paper needs major evaluation fixes before the main claim can be trusted.","tokens_in":733,"tokens_out":869,"would_cite":false,"duration_ms":34449,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Text-guided diffusion fools breast ultrasound classifiers with subtle edits.","keywords":["adversarial attacks","breast ultrasound","diffusion models","prompt learning","text embedding optimization","medical imaging security","perceptual similarity","transfer learning"],"falsifier":"Measure the autoencoder reconstruction error on breast ultrasound images: encode each image, decode immediately, and compute LPIPS/SSIM against the original. If typical ultrasound images are reconstructed poorly, P2P's semantic-preservation claim is unsupported. A second check: replace the frozen natural-image diffusion model with one fine-tuned on ultrasound; if P2P's perceptual gains disappear, the result depends on domain mismatch rather than on the prompt-embedding mechanism.","tokens_in":13384,"feed_emoji":"🩺","tokens_out":7133,"duration_ms":69937,"temperature":0.7,"pith_summary":"This paper claims that adversarial breast ultrasound images can be made by optimizing text embeddings inside a frozen text-to-image diffusion model, with no retraining and no need for a domain-specific diffusion model. The resulting Prompt2Perturb (P2P) attacks reportedly match the attack success rate of the diffusion-based Diff-PGD baseline while producing markedly lower LPIPS and FID scores across three breast ultrasound datasets and three classifier architectures. The authors argue that this makes P2P a practical way to stress-test medical classifiers in data-scarce settings, where training or fine-tuning a diffusion model on ultrasound images is often infeasible.","feed_headline":"Text-guided diffusion fools breast ultrasound classifiers with subtle edits","feed_subtitle":"P2P optimizes text embeddings in a frozen diffusion model to match attack success with far less visible distortion.","key_machinery":"The central object is a set of learnable text embeddings in the CLIP text encoder of Stable Diffusion v1.4, optimized by backpropagating a combined loss: cross-entropy against the true label to fool the classifier, plus MSE between the added noise and the predicted noise to preserve semantic content. Only the first 50 reverse-diffusion steps are optimized, based on the observation that early denoising steps shape low-frequency structure, so perturbations introduced there are hidden in realistic tissue texture. The diffusion U-Net and the autoencoder remain frozen throughout.","core_discovery":"The central claim is that adversarial breast ultrasound images can be generated by updating only the text embeddings that condition a pretrained latent diffusion model, instead of adding pixel-level noise or retraining the generator. On the BUSI, BUS-BRA, and UDIAT datasets, P2P reports attack success rates comparable to Diff-PGD (e.g., 0.97 for ResNet34 on BUSI) while beating FGSM, PGD, and Diff-PGD on LPIPS and FID (e.g., LPIPS 0.12 and FID 43.03 for ResNet34 on BUSI). The paper further claims that optimizing only the early reverse-diffusion steps keeps attack quality while improving efficiency, and that adding an MSE noise-prediction term visibly suppresses streak artifacts and preserves tissue texture.","pith_inferences":["Editorial inference: if P2P's perceptual gains survive a human-observer study, it would provide a stronger stress test for ultrasound computer-aided diagnosis than norm-bounded attacks, because the attacked images would lie on the manifold of plausible ultrasound textures.","Editorial inference: a direct comparison against a diffusion model fine-tuned on ultrasound would separate the contribution of prompt-embedding optimization from the domain gap of the frozen backbone; the paper does not run that comparison.","Editorial inference: because the attack manipulates text embeddings rather than pixels, the same recipe may transfer to other text-conditioned generators or other imaging modalities without retraining, though the paper does not test this."],"forward_implications":["P2P can launch an attack without a domain-specific diffusion model or a large medical training set, since only text embeddings are learned.","Across BUSI, BUS-BRA, and UDIAT, P2P matches Diff-PGD's success rate on DenseNet121 and ResNet34 while producing lower LPIPS and FID, meaning the attacked images should be harder to distinguish from clean ultrasound images.","Optimizing only the first 50 reverse-diffusion steps keeps success near the full-horizon setting while cutting generation time per image from roughly 280 to 178 seconds on the reported hardware.","The MSE noise-prediction term does not change headline metrics much but visibly reduces streak-like artifacts and preserves tissue homogeneity, supporting its role as a semantic-preservation regularizer.","The t-SNE analysis shows P2P's attacked features interleave with clean features more than FGSM, PGD, or Diff-PGD, which the paper interprets as evidence that the adversarial examples are harder to detect at the feature level."],"supporting_citations":[{"why":"Supplies the latent diffusion framework and text-conditioning setup on which prompt-embedding optimization operates.","marker":"[46]"},{"why":"Supplies the specific pretrained Stable Diffusion v1.4 backbone used in all experiments.","marker":"[13]"},{"why":"Defines the Diff-PGD baseline whose attack success P2P matches and whose perceptual quality P2P is claimed to beat.","marker":"[61]"},{"why":"Provides the textual inversion idea of associating new concepts with optimized embedding vectors, which P2P adapts to medical vocabulary.","marker":"[17]"},{"why":"Provides the observation about early versus later diffusion steps that motivates optimizing only the first 50 reverse steps.","marker":"[41]"},{"why":"Supplies the FGSM baseline compared in the experimental tables.","marker":"[20]"},{"why":"Supplies the PGD baseline compared in the experimental tables.","marker":"[40]"},{"why":"Supplies the BUSI dataset used for the main results and the ablation study.","marker":"[2]"}],"fun_headline_variants":["Text prompts craft invisible attacks on breast ultrasound AI","Diffusion attacks on ultrasound: text-driven, barely visible","P2P: tweak text, fool ultrasound classifiers","Language-guided subtle attacks beat pixel noise on ultrasound","No retraining: text edits cause ultrasound misdiagnosis"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that a diffusion model trained on everyday photos can faithfully represent breast ultrasound images, so that editing its text prompts changes the image without destroying the clinical content; the paper does not directly test that reconstruction fidelity.","fun_headline_variants_meta":{"raw":{"variants":["Text prompts craft invisible attacks on breast ultrasound AI","Diffusion attacks on ultrasound: text-driven, barely visible","P2P: tweak text, fool ultrasound classifiers","Language-guided subtle attacks beat pixel noise on ultrasound","No retraining: text edits cause ultrasound misdiagnosis"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000479,"raw_usage":{"total_tokens":2404,"prompt_tokens":1009,"completion_tokens":1395,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":625,"completion_tokens_details":{"reasoning_tokens":1318}},"tokens_in":625,"tokens_out":1395,"duration_ms":11015,"temperature":1.0,"reasoning_tokens":1318,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T16:35:03.725082+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the autoencoder reconstruction error on breast ultrasound images: encode each image, decode immediately, and compute LPIPS/SSIM against the original. If typical ultrasound images are reconstructed poorly, P2P's semantic-preservation claim is unsupported. A second check: replace the frozen natural-image diffusion model with one fine-tuned on ultrasound; if P2P's perceptual gains disappear, the result depends on domain mismatch rather than on the prompt-embedding mechanism.","supporting_citations":[{"cited_title":"Stable diffusion v1.4 - original, 2023","cited_arxiv_id":null,"evidence_quote":"Supplies the specific pretrained Stable Diffusion v1.4 backbone used in all experiments."},{"cited_title":"Diffusion-based adversarial sample generation for improved stealthiness and controllability.Advances in Neural Informa- tion Processing Systems, 36, 2024","cited_arxiv_id":null,"evidence_quote":"Defines the Diff-PGD baseline whose attack success P2P matches and whose perceptual quality P2P is claimed to beat."},{"cited_title":"Prompting hard or hardly prompting: Prompt inversion for text-to-image diffusion models","cited_arxiv_id":null,"evidence_quote":"Provides the observation about early versus later diffusion steps that motivates optimizing only the first 50 reverse steps."},{"cited_title":"Dataset of breast ultrasound images","cited_arxiv_id":null,"evidence_quote":"Supplies the BUSI dataset used for the main results and the ablation study."}],"review_version":1}