{"id":"e711176f-454b-41b6-a8e4-dada427cdf6f","arxiv_id":"2508.01272","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"PromptSafe uses LLM-rewritten safe prompts to train a gated soft prompt that suppresses NSFW content in text-to-image generation without image supervision or inference overhead.","lead":"This paper presents PromptSafe, a text-only defense for text-to-image models that rewrites unsafe prompts into safe ones and trains a gated soft prompt to block NSFW content. It reports the lowest unsafe generation rate among compared defenses while preserving benign image quality.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 2.36% unsafe rate is measured with the same UD checker used to filter the training corpus in Eq. (4); the central SOTA safety claim may be an artifact of evaluation circularity rather than genuine suppression.","rationale":"The reader's weakest_assumption is the pure-noise training paradigm in Eq. (6); that is a genuine mechanism concern, since a U-Net trained on real image latents may produce noise predictions nearly independent of text conditioning when the input is pure Gaussian noise, especially at large T. I agree this deserves an ablation. However, the more load-bearing issue for the paper's headline is the identity between the training-data filter (fsafe in Eq. (4), based on [36]) and the evaluation metric (UD checker [36] in Sec. 5.1). Because PromptSafe's training corpus was filtered by the same detector used to score safety, the method is at risk of overfitting to that detector. This is not just an absolute-calibration issue; it is differential, because none of the eight baselines used UD to construct their training data, so the SOTA ranking itself may be an artifact. The paper provides no code or data, which prevents checking whether the same detector instance was used. The proposed test—independent safety scoring of the same generated images—would settle whether the 2.36% and the ranking survive. If they do, the conditional acceptance stands; if they do not, the central claim is unsupported. I therefore maintain the reader's conditional verdict, with the condition tightened around an independent safety evaluation.","tokens_in":14032,"tokens_out":9375,"duration_ms":115084,"concrete_test":"Re-run the I2P and COCO-2017 evaluation with PromptSafe and all eight baselines, scoring the generated images with an independent NSFW detector not used anywhere in PromptSafe's pipeline (e.g., a different NSFW classifier or human raters on a random subset). Compare unsafe-ratio ranking and absolute values against Table 1, and report agreement between UD and the independent detector. If PromptSafe loses its SOTA ranking, or its unsafe ratio rises disproportionately relative to baselines, the central claim is an artifact of UD-checker circularity.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central safety claim—SOTA 2.36% Unsafe Ratio—depends on the validity of the Unsafe Ratio metric. In Sec. 5.1, Unsafe Ratio is measured with the UD checker of [36]. The same detector is used during training-data construction as fsafe in Eq. (4): a rewritten safe prompt P^s_i is kept only if fsafe(G(P^s_i)) = 1. The soft prompt is therefore optimized, via the triplet loss on noise predictions, to pull unsafe prompts toward prompts that were explicitly selected to pass UD. Evaluating with that same checker is circular and, more importantly, differential: PromptSafe's training pipeline has implicit access to the evaluation metric, while the eight baselines do not. This can inflate both the absolute 2.36% number and the relative ranking. The generalization results in Table 3 and jailbreak results in Table 4c use the same checker and inherit the same problem. Even if the pure-noise training objective of Eq. (6) works as intended, the reported unsafe rate cannot be interpreted as a true measure of NSFW suppression without an independent safety measurement.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"PromptSafe proposes a gated prompt-tuning defense for text-to-image diffusion models. Instead of training on image-text pairs, it uses an LLM to rewrite unsafe prompts into semantically aligned safe counterparts, then optimizes a universal soft prompt with a triplet loss computed from U-Net noise predictions on pure-noise latents. At inference, a CLIP-based gated network predicts a toxicity score that interpolates between the defensive embedding and a zero vector, so that benign prompts receive little or no intervention. The paper reports a state-of-the-art unsafe generation ratio of 2.36% on I2P, while preserving benign fidelity (CLIP score 26.30 on COCO), and claims generalization to unseen harmful categories, transfer across Stable Diffusion architectures, and robustness under adaptive attacks and jailbreak prompts.","tokens_in":14260,"tokens_out":2532,"duration_ms":31187,"significance":"If validated, the method is practically attractive: it removes the need for costly curated image-text data, introduces a per-prompt adaptive defense strength, and adds negligible inference overhead (matching vanilla SDv1.4 at 0.58 s/image). The reported gains over eight baselines are substantial, and the architecture-level experiments (Table 4, Fig. 4/5) are broader than in many prior works. However, the central safety claim rests on an evaluation metric that is the same classifier used to filter the training corpus, which materially weakens the claim as stated. The training objective's reliance on pure-noise latents also needs empirical validation. These issues are addressable with additional experiments, but they are load-bearing for the paper's headline results.","major_comments":[{"comment":"The Unsafe Ratio metric is computed using the UD checker from [36], which is the same detector used as fsafe in Eq. (4) to filter the training corpus: a rewritten prompt is kept only if fsafe(G(P_s))=1, and the soft prompt is then optimized to pull unsafe prompts toward those safe embeddings. Evaluating with the same classifier is circular and differential, because PromptSafe's training pipeline has implicit access to the evaluation metric while the eight baselines do not. This can inflate both the absolute 2.36% number and the relative ranking. The same issue affects the generalization results in Table 3 and the jailbreak results in Table 4c. Please report Unsafe Ratio with an independent safety detector (e.g., a different NSFW classifier or human evaluation), or at minimum present both metrics for all methods and discuss the circularity explicitly.","section":"Sec. 5.1 and Eq. (4)"},{"comment":"The training procedure samples a clean latent z0 ~ N(0, I_d) from the Gaussian prior and then adds T steps of noise before asking the U-Net to predict the noise. In standard DDPM training, noise prediction is supervised by a real image latent; with pure noise as the 'clean' input, the denoising target is ill-defined and the predicted noise may be dominated by input noise rather than by prompt semantics. The claim that the predicted noise 'becomes entirely reliant on the cross attention-driven text condition' is not self-evident and needs direct validation. Please provide empirical evidence that the noise predictions for different prompts are meaningfully separable, or compare against a variant trained with real image latents (e.g., from COCO) to show that the pure-noise objective is not merely fitting noise.","section":"Sec. 4.2, Eq. (6)"},{"comment":"All reported numbers are point estimates with no error bars, confidence intervals, or statistical significance tests. This is particularly problematic for the benign preservation comparisons where differences are small (e.g., CLIP 26.30 vs. 26.27 for SafeGuider in Table 1; LPIPS 0.694 vs. 0.696). Without variance information or a significance test, the claim that PromptSafe 'outperforms' these baselines on benign preservation is not empirically supported. Please report mean and standard deviation over multiple seeds or runs, and indicate the number of prompts and images used for each metric.","section":"Tables 1, 3, and 4"}],"minor_comments":[{"comment":"There are several typos and grammatical errors: 'Inspired of' should be 'Inspired by', 'suggestes' should be 'suggests', and 'relies on heavily on' is redundant. These should be corrected.","section":"Throughout"},{"comment":"Equation (1) has mismatched parentheses: I = G(P) = D( U(z_T | E_p), where ... should be D(U(z_T | E_p)), with the closing parenthesis before 'where'. Please fix the notation.","section":"Eq. (1)"},{"comment":"The manuscript contains internal version markers and layout notes (e.g., '20250725 - head -v2' and '20250728 - v4' with instructions like '字少一点，大一点，只留下关键内容，加粗加红') that should be removed before submission.","section":"Fig. 1 and surrounding text"},{"comment":"The gated network is trained on 'curated unsafe datasets' and MS COCO 2017 val captions, but the precise composition and size of the malicious training set for the gate are not given. Please specify which datasets are used and how the benign/malicious balance is set, since the gate's behavior is central to the adaptive defense claim.","section":"Sec. 4.3 and Eq. (10)"},{"comment":"The paper states that 'we train one soft prompt per unsafe category and prepend all four to the input at inference for joint control', but the details of how the four embeddings are combined (concatenation, averaging, or something else) and how the gate modulates them are not described. Please clarify the exact inference-time composition.","section":"Sec. 5.2"}],"recommendation":"major_revision","confidential_remarks":"The reference list contains a large number of self-citations, including some that appear unrelated to the paper's topic (e.g., [14] on optimal control and filtering). This may warrant editorial attention. Also, the internal version-control notes left in the manuscript suggest the paper was not fully cleaned before submission; as a reviewer I flag this only as a presentation issue, but it is worth noting for the editor."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"PromptSafe is a solid applied paper. The core idea—train a universal soft prompt using only LLM-rewritten safe/unsafe text pairs, then gate its strength with a CLIP-based toxicity score—is genuinely new relative to PromptGuard, SafeGen, SLD, and UCE. It removes the need for image-text pairs, cuts storage by three orders of magnitude, and adds zero inference latency. That is practically valuable for deployment, and the paper shows it convincingly: the efficiency numbers in Table 2 are striking, and the cross-architecture transfer results are a nice bonus. The design choices are reasonable and the ablations on λ and the gating mechanism are informative.\n\nThe soft spot is the one the stress test flags, and it is real. The Unsafe Ratio in Tables 1, 3, and 4c is computed with the UD checker from [36], the same fsafe used in Eq. (4) to filter the training corpus. PromptSafe's training pipeline therefore has implicit access to the evaluation metric, while the baselines do not. That does not automatically make the 2.36% wrong, but it makes the absolute number and the relative ranking uninterpretable without an independent safety measurement. This is a load-bearing issue for the main claim, though it is fixable: re-evaluate all methods with a different NSFW detector (or human evaluation) and see if PromptSafe still leads.\n\nA second, lesser concern is the training objective in Eq. (6). Sampling z0 from pure noise and then adding noise to get zT gives a denoising target that is not tied to any real image, so the predicted noise may carry little prompt-dependent semantic signal. The paper does not validate that this actually works as intended. The strong results suggest something is working, but an ablation comparing pure-noise latents against real-image latents would clarify whether the mechanism is what the authors think it is. Minor: no error bars, no code or data released, and a few typos (e.g., 'suggestes' in Sec. 5.1).\n\nOverall, the method is novel, clearly motivated, and efficiently designed. The central safety claim is currently unverified because of evaluation circularity, but the flaw is addressable and does not sink the method's potential. Who is this for? Anyone working on T2I safety or lightweight model editing. It deserves a serious referee: a conditional accept with a request for independent safety evaluation and code release is the right call.","headline":"Useful, lightweight T2I safety defense with a genuinely new text-only soft-prompt design, but the headline 2.36% number is measured with the same detector used to filter training data, so the safety claim needs independent verification before I would believe the SOTA ranking.","tokens_in":14818,"tokens_out":856,"would_cite":true,"duration_ms":12480,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"PromptSafe claims a text-only trained soft prompt plus a toxicity gate can hold unsafe image output at a 2.36 percent rate while preserving benign fidelity.","keywords":["text-to-image safety","NSFW suppression","soft prompt tuning","gated toxicity control","diffusion models","prompt rewriting","benign preservation"],"falsifier":"Run the trained defense on the unsafe-prompt benchmark with the gate forced to $\\gamma=0$ (no defense) and with the gate forced to $\\gamma=1$ (full defense): if the unsafe ratio at $\\gamma=0$ is not near the vanilla baseline, the soft prompt alone is suppressing NSFW and the gate is redundant; if the unsafe ratio at $\\gamma=1$ is not near 2.36%, the learned soft prompt, not the gate, is the active ingredient.","tokens_in":13838,"feed_emoji":"🛡️","tokens_out":7056,"duration_ms":77614,"temperature":0.7,"pith_summary":"PromptSafe is a defense framework for text-to-image diffusion models that tries to prove a specific trade-off is available: strong not-safe-for-work (NSFW) suppression without expensive image-text training data and without degrading benign generation. Its central claim is that a universal soft prompt learned from LLM-rewritten text pairs, combined with a toxicity gate that scales the prompt's strength at inference, achieves a 2.36% unsafe generation rate on the I2P benchmark across sexual, violent, political, and disturbing categories. The same mechanism preserves a text-image alignment score of 26.30 on COCO benign prompts, nearly matching the undefended model, and adds no inference time beyond native generation. The paper also claims the defense generalizes to unseen harmful categories, transfers across text-to-image model architectures, and weakens but does not collapse under adaptive white-box attacks.","feed_headline":"One soft prompt cuts unsafe image generation to 2.36 percent.","feed_subtitle":"A text-only tuned gate adjusts defense per prompt, keeping benign images sharp at zero extra inference cost.","key_machinery":"The central object is a trainable universal soft prompt embedding $v^*$ whose strength is modulated by a gated control network. Training uses text-only pairs $(P^m, P^s)$ from an LLM rewrite; the diffusion U-Net predicts noise $\\epsilon = U(z_T, T, E(P))$ from a randomly sampled latent $z_0 \\sim \\mathcal{N}(0, I_d)$ with added noise, so no image supervision is needed. A triplet loss organizes the noise-prediction space by pulling $\\tilde{\\epsilon}^m$ (the soft-prompt-augmented malicious prompt) toward $\\epsilon^s$ (its safe rewrite) and pushing it away from $\\epsilon^m$, while a benign-preservation loss keeps $\\tilde{\\epsilon}^s$ close to $\\epsilon^s$. At inference the gate $f_\\theta$ maps the prompt through a text-image encoder and a three-layer head to a toxicity score $\\gamma \\in [0,1]$, and the final defensive embedding is the interpolation $v' = \\gamma v^* + (1-\\gamma)v_0$, prepended to the original prompt embedding.","core_discovery":"On the paper's own terms, PromptSafe establishes that a purely text-driven soft prompt can replace image supervision in safe-image diffusion tuning. Instead of training on curated image-text pairs, the authors rewrite unsafe prompts into semantically aligned safe alternatives with an LLM, then optimize a trainable embedding vector that is prepended to the prompt. During training the U-Net's noise prediction on randomly sampled pure-noise latents is used in a triplet loss: the soft-prompt-augmented malicious prompt is pulled toward the safe rewrite and pushed away from the original malicious prompt, while a benign-preservation term keeps the soft prompt neutral on safe rewrites. At inference a gated network estimates prompt toxicity and linearly interpolates between the defensive embedding and a zero vector, so unsafe prompts receive strong intervention and benign prompts are left almost untouched. The measured result is the lowest average unsafe ratio among the compared defenses (2.36% on I2P), benign alignment score 26.30 on COCO, and inference time equal to the base model.","pith_inferences":["A testable extension: the noise-prediction triplet could be repurposed as a general prompt-steering primitive, e.g., to edit style or erase concepts, wherever a text-only supervised pair can be written.","The paper leaves implicit that the gated classifier is a single point of failure for the whole defense; an attacker who can flip the toxicity score to zero would neutralize the soft prompt without touching the diffusion weights.","The pure-noise training objective suggests a cheaper auditing procedure: run the trained soft prompt against a held-out set of adversarial prompt rewrites and measure whether unsafe-ratio gains persist when the gate is bypassed.","The reported 2.36% relies on the specific unsafe-image checker used for evaluation; a different checker with different thresholds could give materially different numbers, so cross-checker reproducibility is worth testing."],"forward_implications":["If the central claim is right, text-to-image safety can be strengthened with a few hundred text pairs and no curated image data, removing the main scaling bottleneck of soft-prompt defenses.","The gating mechanism implies defense intensity can be matched to per-prompt risk at inference time, so safe prompts should keep native image quality rather than absorbing a uniform style shift.","Because the training signal lives in noise-prediction space rather than image space, the same soft prompt training should transfer to any diffusion model with the same text-encoder embedding dimension.","The framework can stack on top of safety-aligned models, further lowering unsafe ratios that retraining alone does not eliminate.","Adaptive attacks on the gate weaken but do not fully break the defense, which suggests gate robustness is the next natural point of hardening."],"supporting_citations":[{"why":"the soft-prompt baseline that PromptSafe compares against, providing the image-text paired training setup and fixed-strength defense that the paper argues against.","marker":"[49]"},{"why":"supplies the I2P unsafe-prompt benchmark and the SLD baseline used in the main suppression and quality comparison.","marker":"[39]"},{"why":"provides the unsafe-image checker used both to filter safe rewrites in Eq. (4) and to compute the Unsafe Ratio metric.","marker":"[36]"},{"why":"supplies the DDPM noise scheduler and noise-prediction formulation on which the text-only training objective in Eq. (6) is built.","marker":"[12]"},{"why":"the LLM behind the safe-rewrite step that constructs the text-only paired training corpus.","marker":"[1]"},{"why":"the text-image alignment model used for semantic-similarity filtering, benign alignment-score evaluation, and the gated controller's encoder.","marker":"[37]"},{"why":"the base text-to-image diffusion model on which the soft prompt is trained and evaluated.","marker":"[3]"},{"why":"the concept-erasing defense baseline compared on safety and benign quality; its inference-time erasure defines the alternative approach the paper contrasts with.","marker":"[16]"}],"fun_headline_variants":["A gated soft prompt adapts to toxicity, shrinking unsafe output to 2.36%","LLM-rewritten prompts train a soft gate that blocks NSFW without slowing inference","Text-only safety tuning: adaptive gate trims unsafe rate to 2.36%, keeps quality","Zero-cost gating: soft prompt adjusts defense per prompt, unsafe rate drops to 2.36%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the U-Net's noise prediction on a random pure-noise latent carries enough prompt-dependent semantic signal for the triplet loss to steer the soft prompt; if denoising targets drawn from pure noise are mostly noise-driven, the learned embedding would not be the cause of the reported suppression.","fun_headline_variants_meta":{"raw":{"variants":["A gated soft prompt adapts to toxicity, shrinking unsafe output to 2.36%","LLM-rewritten prompts train a soft gate that blocks NSFW without slowing inference","Text-only safety tuning: adaptive gate trims unsafe rate to 2.36%, keeps quality","Zero-cost gating: soft prompt adjusts defense per prompt, unsafe rate drops to 2.36%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000885,"raw_usage":{"total_tokens":3871,"prompt_tokens":1043,"completion_tokens":2828,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":659,"completion_tokens_details":{"reasoning_tokens":2729}},"tokens_in":659,"tokens_out":2828,"duration_ms":24026,"temperature":1.0,"reasoning_tokens":2729,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T05:43:13.283476+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the trained defense on the unsafe-prompt benchmark with the gate forced to $\\gamma=0$ (no defense) and with the gate forced to $\\gamma=1$ (full defense): if the unsafe ratio at $\\gamma=0$ is not near the vanilla baseline, the soft prompt alone is suppressing NSFW and the gate is redundant; if the unsafe ratio at $\\gamma=1$ is not near 2.36%, the learned soft prompt, not the gate, is the active ingredient.","supporting_citations":[{"cited_title":"PromptGuard: Soft prompt-guided unsafe content moderation for text-to-image models, 2025","cited_arxiv_id":null,"evidence_quote":"the soft-prompt baseline that PromptSafe compares against, providing the image-text paired training setup and fixed-strength defense that the paper argues against."},{"cited_title":"Safe latent diffusion: Mitigating in- appropriate degeneration in diffusion models","cited_arxiv_id":null,"evidence_quote":"supplies the I2P unsafe-prompt benchmark and the SLD baseline used in the main suppression and quality comparison."},{"cited_title":"Unsafe Diffusion: On the Generation of Unsafe Images and Hateful Memes From Text-To-Image Models","cited_arxiv_id":"2305.13873","evidence_quote":"provides the unsafe-image checker used both to filter safe rewrites in Eq. (4) and to compute the Unsafe Ratio metric."},{"cited_title":"Denoising dif- fusion probabilistic models","cited_arxiv_id":null,"evidence_quote":"supplies the DDPM noise scheduler and noise-prediction formulation on which the text-only training objective in Eq. (6) is built."},{"cited_title":"Stable Diffusion V1-4","cited_arxiv_id":null,"evidence_quote":"the base text-to-image diffusion model on which the soft prompt is trained and evaluated."},{"cited_title":"SafeGen: Mitigating sexually explicit content generation in text-to-image models","cited_arxiv_id":null,"evidence_quote":"the concept-erasing defense baseline compared on safety and benign quality; its inference-time erasure defines the alternative approach the paper contrasts with."}],"review_version":1}