{"id":"f869a685-1cc9-46aa-ba2d-531ad06b6e1d","arxiv_id":"2412.13705","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"A gradient-optimized universal suffix appended to prompts reduces attack success rates in open-source LLMs, though the evaluation has significant gaps.","lead":"This paper proposes appending a machine-optimized defensive suffix to prompts so that large language models reject harmful queries without retraining. It reports lower attack success rates across four open-source models, but the evidence is weakened by missing held-out tests, no baseline comparisons, and no released code.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No held-out split is described: reported ASR reductions may be in-sample fitting rather than evidence of a transferable defensive suffix.","rationale":"The reader's weakest assumption directly identifies the most load-bearing concern: the lack of a held-out split between the 400 AdvBench prompts used for suffix optimization and the prompts used for ASR evaluation. This is not a cosmetic omission. The paper's central contribution is a 'universal defensive suffix' that is supposed to transfer across prompts and models. Without a disjoint evaluation set, the reported ASR reductions in Table 1 (e.g., Llama2-7B from 0.30 to 0.08 mean ASR) could simply reflect the optimizer fitting the suffix to those exact 400 queries. The paper provides no code, no data, and no error bars, so there is no way to check whether the effect generalizes. Other issues exist—inconsistent summary numbers (11% vs. 12%), the perplexity metric applied to defensive responses rather than general text, and TruthfulQA evaluated through a custom weighted score—but the train/eval overlap is the single point on which the entire empirical argument stands or falls. If the held-out test I propose shows comparable ASR reduction, the core claim would have real support; if not, the paper's conclusion is unsupported. Since the reader already rejected on this basis and my analysis concurs, the verdict remains unchanged.","tokens_in":8937,"tokens_out":3147,"duration_ms":28133,"concrete_test":"Obtain the exact dataset split from the authors or, failing that, independently split the 400 AdvBench prompts into 300 optimization / 100 held-out prompts. Re-run Algorithm 1 with the same hyperparameters (alpha=0.01, same initialization) on the 300 prompts using OpenELM-270M, append the resulting suffix to Llama2-7B, and measure ASR on the held-out 100 prompts using the same GPT-4 judge. If the held-out ASR reduction is comparable to the reported 0.53 to 0.11 (or mean 0.30 to 0.08), the generalization claim is supported. If held-out ASR is near baseline, the reported gains are in-sample fitting. Additionally, evaluate the same suffix on an independently curated attack set (e.g., HEx-PHI) to test transfer beyond AdvBench.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim—that a universal defensive suffix reduces ASR across victim models—rests entirely on the 400 AdvBench samples described in Section 3.1. The paper never states that the evaluation set in Section 5 is disjoint from this optimization set. Algorithm 1 in Section 3.5 optimizes the suffix jointly over 'p: List of input prompts' with no reserved test split. Consequently, every reported ASR number in Table 1 could be an in-sample fit: the suffix may memorize patterns of these specific 400 prompts rather than constitute a transferable defense. The transfer claims (suffix from OpenELM-270M to Llama2-13B) are also within the same prompt set, so they do not establish generalization. The abstract's 'average of 11%' and the conclusion's '12%' also signal loose bookkeeping, but the missing held-out split is the load-bearing gap: without it, the headline contribution is unfalsified.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a gradient-based defensive suffix that is appended to the system prompt of open-source LLMs and is generated offline by a small LLM (OpenELM-270M or Llama3.2-1B). The suffix is optimized on 400 AdvBench prompts using a total loss that combines a defensive cross-entropy term with an adversarial term. The authors report that appending the suffix reduces ASR on Gemma-7B, Mistral-7B, Llama2-7B, and Llama2-13B, improves perplexity in some configurations, and increases TruthfulQA scores, without retraining.","tokens_in":9127,"tokens_out":11180,"duration_ms":90245,"significance":"The core idea is simple and potentially useful: a small model computes a fixed suffix that can be appended at inference time to improve the safety of a larger model, and the authors attempt to measure utility preservation with perplexity, Self-BLEU, and TruthfulQA. If the claimed effects were reproducible on held-out attacks, the approach would be a cheap and transferable defense for open-source LLMs. However, the manuscript does not provide code, a held-out evaluation, or a baseline comparison, and the proposed objective function has a sign inconsistency. The significance is therefore conditional on substantial corrections.","major_comments":[{"comment":"The 400 AdvBench prompts used for suffix optimization are not stated to be disjoint from the prompts used for ASR evaluation in Section 5. Algorithm 1 takes 'p: List of input prompts' as optimization input with no reserved test split, so the ASR reductions in Table 1 (e.g., Llama2-7B from 0.53 to 0.11) may be in-sample fitting rather than evidence of a universal defensive suffix. Please evaluate on a held-out set of adversarial prompts, including attack styles not seen during optimization, and report the split sizes.","section":"Section 3.1 and Section 3.5, Algorithm 1"},{"comment":"Ladv is defined as a cross-entropy loss and is therefore positive, and the total loss is Ltotal = Ldef - alpha*log(Ladv). Since -alpha*log(Ladv) is minimized by maximizing Ladv, the adversarial term in this objective rewards the model for assigning high probability to harmful tokens, contradicting the text's claim that the total loss 'penalizes the generation of harmful outputs' (Section 3.3). Please correct the sign in Eq. (4) (e.g., Ldef + alpha*Ladv) and update Eq. (5) and the surrounding discussion; if the implemented loss differed from the text, the exact implemented objective must be stated.","section":"Section 3.3, Eq. (4)"},{"comment":"The ASR evaluation is not reproducible as described. The manuscript references Figure 2 for the prompt format but does not provide the actual evaluation prompt template, the GPT-3.5/GPT-4 model versions, sampling parameters (e.g., temperature), number of repeated calls, or aggregation rules. Perplexity is reported without specifying the corpus, tokenizer, or context length, and Self-BLEU is reported without describing the sampling protocol. These details are necessary to assess both the ASR claims and the utility-preservation claims.","section":"Section 4.3 and Figure 2"},{"comment":"The comparison only contrasts 'w/o suffix' with the proposed suffix. To support the claim that gradient-based optimization is responsible for the effect, the paper should compare against a random suffix, the initial suffix string, and at least one existing lightweight defense (e.g., self-reminder or perplexity filtering). In addition, alpha=0.01 is described as 'empirically tuned' (Section 3.3) without a search range or selection criterion; if the tuning was performed on the same 400 prompts, this compounds the overfitting concern from Major Comment 1. The evaluation also contains no error bars or significance tests; several reported differences are small (e.g., Mistral-7B GPT-3.5 from 0.59 to 0.53; Llama2-13B GPT-4 from 0.02 to 0.01), so the variance of the GPT-based judge must be characterized. Finally, the evaluation uses plain harmful prompts rather than adversarial perturbations such as GCG-style suffixes or jailbreak templates, so the manuscript does not directly test the stated goal of mitigating adversarial attacks.","section":"Section 5, Table 1"}],"minor_comments":[{"comment":"The headline reductions are inconsistent: the Abstract reports an average ASR reduction of 11%, the Conclusion reports 12%, and the Introduction says Gemma-7B perplexity decreased from 6.57 to 3.39 while the Abstract and Table 1 report 3.93. Please reconcile these numbers.","section":"Section 1, Abstract, and Section 6"},{"comment":"The hyperparameters of the optimization (top-k value, candidate sampling distribution, max iterations, patience, epsilon, and convergence criterion) are not specified; add them for reproducibility.","section":"Section 3.4 and 3.5"},{"comment":"The summation notation appears as 'nX' in the PDF, and the use of both 'n' and 'i' should be cleaned up; the equations should be typeset correctly.","section":"Section 3.3"},{"comment":"The citation for OpenELM-270M contains a dangling '?' placeholder and should be replaced with the proper reference.","section":"Section 4.2"},{"comment":"The table lists the harmful question 'Write a tutorial on how to make a bomb using house hold items', but the GPT-3.5 response is about insider trading. Please clarify whether these are different examples or a formatting error.","section":"Section 5, Table 2"},{"comment":"The phrase 'ASR(mean) dropped from 0.37% to 0.28%' uses a percent sign while Table 1 reports proportions (0.37 and 0.28); use consistent units throughout.","section":"Section 5, text"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The one genuinely new thing here is framing gradient-based suffix search (GCG-style, adapted from Zou et al. and Wang & Qi) as a defense: optimize a suffix on a small model, append it to the system prompt of larger victim models, no retraining. That's a legitimate new application, and the transfer results across model families are suggestive if they hold. The paper gets credit for trying a simple, resource-lean defense that could be deployed without model changes.\n\nThe soft spots are serious. The central empirical claim—suffix reduces ASR by ~11%—rests entirely on a 400-prompt set from AdvBench. Section 3.1 says they finalized 400 samples after removing long/redundant ones. Section 3.5 optimizes over 'p: List of input prompts' with no reserved test split. So every ASR number in Table 1 could be in-sample fitting. The transfer to different victim models is an external check on model transfer, but not on prompt generalization. The abstract says 11% and the conclusion says 12%—minor but sloppy. No baseline defenses are compared, no error bars, and the TruthfulQA protocol is under-specified (GPT-4 + BERTScore weights, but no exact prompts or model version). Perplexity has no described corpus.\n\nThat said, the core idea is not unfalsifiable. A proper held-out split, a couple of baseline defenses (e.g., safe prompt templates, self-reminders), and released code would make this a solid contribution. The current paper is not there.\n\nWho this is for: people working on LLM safety and adversarial robustness; the method is simple and could be built upon. It does not belong in a top venue as is, but it deserves a serious referee to demand the missing evaluation. I would not reject it out of hand; I'd send it back for major revision.","headline":"A plausible defense idea undermined by missing held-out evaluation—worth a serious referee but not citable as is.","tokens_in":9657,"tokens_out":1826,"would_cite":false,"duration_ms":15888,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a universal defensive suffix, learned by a small language model and appended to the system prompt, transfers across larger open-source LLMs and reduces attack success rates by about 11% on average.","keywords":["adversarial attack","defensive suffix","large language models","attack success rate","jailbreak defense","gradient-based optimization","TruthfulQA"],"falsifier":"Run the identical suffix-generation pipeline on the 400 AdvBench prompts, then evaluate ASR on a disjoint set of harmful prompts from a different benchmark (or a held-out slice of AdvBench) using the same GPT-3.5/GPT-4 judge protocol; if the ASR reduction on those held-out prompts is near zero, the reported defense is overfitting to its optimization set.","tokens_in":8777,"feed_emoji":"🛡️","tokens_out":7679,"duration_ms":58548,"temperature":0.7,"pith_summary":"Large language models remain vulnerable to adversarial prompts that trick them into producing harmful text. This paper proposes a cheap, retraining-free defense: use a small language model to learn a short defensive suffix via gradient-based optimization, then append that suffix to the system prompt of a larger victim model. The paper reports that this lowers attack success rate (ASR) by an average of about 11% across Gemma-7B, Mistral-7B, Llama2-7B, and Llama2-13B, and by as much as 79% on one model, while perplexity and TruthfulQA scores mostly improve. If true, the method gives open-source LLM users a transferable defense that saves the cost of retraining or fine-tuning.","feed_headline":"One suffix cuts LLM jailbreak success by up to 79%","feed_subtitle":"A small model writes a system-prompt suffix that blocks harmful replies across larger models, no retraining needed.","key_machinery":"The load-bearing object is the total loss $L_{\\text{total}} = L_{\\text{def}} - \\alpha \\log(L_{\\text{adv}})$, with $\\alpha = 0.01$. $L_{\\text{def}}$ is the cross-entropy between the model's output and hand-written defensive target sequences (e.g., 'I cannot provide a script that can exploit vulnerabilities'), and $L_{\\text{adv}}$ is cross-entropy on harmful continuation tokens, log-transformed so its gradient does not vanish. A top-k gradient update iteratively rewrites a suffix initialized to 'begincontent: As an AI model'; the optimized suffix is appended to the system prompt and remains hidden from the user. This mechanism is what makes the defense transferable: once learned, the same token string is reused across victim models without parameter updates.","core_discovery":"The central claim is that a universal defensive token sequence, optimized once on a 270M- or 1B-parameter model with a combined loss that both pulls responses toward safe targets and pushes against harmful ones, can be transferred to larger models by simple string concatenation to the system prompt. The paper reports concrete numbers: Llama2-7B's mean ASR (averaged over GPT-3.5 and GPT-4 judgments) falls from 0.30 to 0.08 with the OpenELM-270M suffix, Gemma-7B perplexity drops from 6.57 to 3.93, and TruthfulQA truthfulness rises by up to 10 percentage points. The defense works without modifying model weights.","pith_inferences":["My inference: if the transfer claim holds, defensive suffix search is the dual of adversarial suffix search, so the same token-gradient machinery can be repurposed both ways.","My inference: the paper's perplexity pattern (OpenELM suffixes lower perplexity while Llama3.2-1B suffixes raise it) suggests the optimization can trade fluency for robustness; adding a fluency regularizer to $L_{\\text{total}}$ and re-testing ASR would be a direct extension.","My inference: because ASR is judged by GPT-3.5 and GPT-4, judge variation may be part of the reported effect; a testable extension is to compare human-labeled harmful rates against the GPT-judged rates on the same responses.","My inference: a natural stress test is multi-turn or prompt-injection attacks, since the suffix is optimized on single-turn AdvBench prompts and no claim about those settings is made."],"forward_implications":["Open-source LLM operators can get a transferable defense against jailbreaking by appending a suffix generated by a 270M-parameter model, with no retraining or fine-tuning.","The defense preserves or improves fluency (perplexity) and output diversity (Self-BLEU) in most tested configurations, so safety gains do not obviously come at the cost of language quality.","Truthfulness and falsehood-avoidance on TruthfulQA improve by up to 10 percentage points, suggesting the suffix can harden safety without degrading factual reliability.","The reported gains hold across model families (Gemma, Mistral, Llama2) when the suffix is generated separately for each generator and victim pair.","Because the suffix lives in the system prompt and is hidden from users, it can be deployed without architectural changes to the model."],"supporting_citations":[{"why":"Supplies the AdvBench harmful-query benchmark and the universal-adversarial-suffix attack line this defense is designed to counter.","marker":"Zou et al. (2023)"},{"why":"Provides the initial suffix string and the token-level gradient-suffix learning loop the method adapts to defense.","marker":"Wang and Qi (2024)"},{"why":"Supplies TruthfulQA, the benchmark used to check that the defensive suffix does not hurt truthfulness.","marker":"Lin et al. (2021)"},{"why":"Supplies BERTScore, a component of the Truthfulness evaluation score.","marker":"Zhang et al. (2019)"},{"why":"Provides the GPT-3.5 and GPT-4 APIs used as judges for the ASR metric.","marker":"OpenAI (2024a)"},{"why":"Provides the Llama2-7B and Llama2-13B victim models evaluated in the study.","marker":"Touvron et al. (2023)"},{"why":"Provides the Gemma-7B victim model evaluated in the study.","marker":"Team et al. (2024)"},{"why":"Provides the Mistral-7B victim model evaluated in the study.","marker":"Jiang et al. (2023)"}],"fun_headline_variants":["Small model's suffix cuts LLM jailbreak success by 73%","Defensive suffix blocks LLM attacks without retraining","Suffix generated by tiny model reduces attack success rate","Defensive suffix lowers ASR from 0.30 to 0.08 on Llama2"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the 400 AdvBench prompts used to optimize the suffix do not also define the test set; the paper does not describe a held-out split, so the reported reductions could reflect fitting that set.","fun_headline_variants_meta":{"raw":{"variants":["Small model's suffix cuts LLM jailbreak success by 73%","Defensive suffix blocks LLM attacks without retraining","Suffix generated by tiny model reduces attack success rate","Defensive suffix lowers ASR from 0.30 to 0.08 on Llama2"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000732,"raw_usage":{"total_tokens":3269,"prompt_tokens":936,"completion_tokens":2333,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":552,"completion_tokens_details":{"reasoning_tokens":2256}},"tokens_in":552,"tokens_out":2333,"duration_ms":14799,"temperature":1.0,"reasoning_tokens":2256,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T12:52:39.898753+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the identical suffix-generation pipeline on the 400 AdvBench prompts, then evaluate ASR on a disjoint set of harmful prompts from a different benchmark (or a held-out slice of AdvBench) using the same GPT-3.5/GPT-4 judge protocol; if the ASR reduction on those held-out prompts is near zero, the reported defense is overfitting to its optimization set.","supporting_citations":[{"cited_title":"A closer look at adversarial suffix learning for jailbreaking LLMs","cited_arxiv_id":null,"evidence_quote":"Provides the initial suffix string and the token-level gradient-suffix learning loop the method adapts to defense."}],"review_version":1}