REVIEW 4 major objections 6 minor 1 cited by
Mitigating Adversarial Attacks in LLMs through Defensive Suffix Generation
T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read The paper claims that a universal defensive suffix, learned by a small language model and appended to the system prompt, transfers across larger open-source LLMs and reduces attack success rates by about 11% on average.
desk verdict A plausible defense idea undermined by missing held-out evaluation—worth a serious referee but not citable as is. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the total loss $L_{\text{total}} = L_{\text{def}} - \alpha \log(L_{\text{adv}})$, with $\alpha = 0.01$. $L_{\text{def}}$ is the cross-entropy between the model's output and hand-written defensive target sequences (e.g., 'I cannot provide a script that can exploit vulnerabilities'), and $L_{\text{adv}}$ is cross-entropy on harmful continuation tokens, log-transformed so its gradient does not vanish. A top-k gradient update iteratively rewrites a suffix initialized to 'begincontent: As an AI model'; the optimized suffix is appended to the system prompt and remains hidden from the user. This mechanism is what makes the defense transferable: once learned, the same token string is reused across victim models without parameter updates.
What would settle it
Run the identical suffix-generation pipeline on the 400 AdvBench prompts, then evaluate ASR on a disjoint set of harmful prompts from a different benchmark (or a held-out slice of AdvBench) using the same GPT-3.5/GPT-4 judge protocol; if the ASR reduction on those held-out prompts is near zero, the reported defense is overfitting to its optimization set.
Extended reading notes
Core claim
The central claim is that a universal defensive token sequence, optimized once on a 270M- or 1B-parameter model with a combined loss that both pulls responses toward safe targets and pushes against harmful ones, can be transferred to larger models by simple string concatenation to the system prompt. The paper reports concrete numbers: Llama2-7B's mean ASR (averaged over GPT-3.5 and GPT-4 judgments) falls from 0.30 to 0.08 with the OpenELM-270M suffix, Gemma-7B perplexity drops from 6.57 to 3.93, and TruthfulQA truthfulness rises by up to 10 percentage points. The defense works without modifying model weights.
Load-bearing premise
The load-bearing premise is that the 400 AdvBench prompts used to optimize the suffix do not also define the test set; the paper does not describe a held-out split, so the reported reductions could reflect fitting that set.
Editorial extensions
If this is right
- Open-source LLM operators can get a transferable defense against jailbreaking by appending a suffix generated by a 270M-parameter model, with no retraining or fine-tuning.
- The defense preserves or improves fluency (perplexity) and output diversity (Self-BLEU) in most tested configurations, so safety gains do not obviously come at the cost of language quality.
- Truthfulness and falsehood-avoidance on TruthfulQA improve by up to 10 percentage points, suggesting the suffix can harden safety without degrading factual reliability.
- The reported gains hold across model families (Gemma, Mistral, Llama2) when the suffix is generated separately for each generator and victim pair.
- Because the suffix lives in the system prompt and is hidden from users, it can be deployed without architectural changes to the model.
Reading between the lines
- My inference: if the transfer claim holds, defensive suffix search is the dual of adversarial suffix search, so the same token-gradient machinery can be repurposed both ways.
- My inference: the paper's perplexity pattern (OpenELM suffixes lower perplexity while Llama3.2-1B suffixes raise it) suggests the optimization can trade fluency for robustness; adding a fluency regularizer to $L_{\text{total}}$ and re-testing ASR would be a direct extension.
- My inference: because ASR is judged by GPT-3.5 and GPT-4, judge variation may be part of the reported effect; a testable extension is to compare human-labeled harmful rates against the GPT-judged rates on the same responses.
- My inference: a natural stress test is multi-turn or prompt-injection attacks, since the suffix is optimized on single-turn AdvBench prompts and no claim about those settings is made.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a gradient-based defensive suffix that is appended to the system prompt of open-source LLMs and is generated offline by a small LLM (OpenELM-270M or Llama3.2-1B). The suffix is optimized on 400 AdvBench prompts using a total loss that combines a defensive cross-entropy term with an adversarial term. The authors report that appending the suffix reduces ASR on Gemma-7B, Mistral-7B, Llama2-7B, and Llama2-13B, improves perplexity in some configurations, and increases TruthfulQA scores, without retraining.
Significance. The core idea is simple and potentially useful: a small model computes a fixed suffix that can be appended at inference time to improve the safety of a larger model, and the authors attempt to measure utility preservation with perplexity, Self-BLEU, and TruthfulQA. If the claimed effects were reproducible on held-out attacks, the approach would be a cheap and transferable defense for open-source LLMs. However, the manuscript does not provide code, a held-out evaluation, or a baseline comparison, and the proposed objective function has a sign inconsistency. The significance is therefore conditional on substantial corrections.
major comments (4)
- [Section 3.1 and Section 3.5, Algorithm 1] The 400 AdvBench prompts used for suffix optimization are not stated to be disjoint from the prompts used for ASR evaluation in Section 5. Algorithm 1 takes 'p: List of input prompts' as optimization input with no reserved test split, so the ASR reductions in Table 1 (e.g., Llama2-7B from 0.53 to 0.11) may be in-sample fitting rather than evidence of a universal defensive suffix. Please evaluate on a held-out set of adversarial prompts, including attack styles not seen during optimization, and report the split sizes.
- [Section 3.3, Eq. (4)] Ladv is defined as a cross-entropy loss and is therefore positive, and the total loss is Ltotal = Ldef - alpha*log(Ladv). Since -alpha*log(Ladv) is minimized by maximizing Ladv, the adversarial term in this objective rewards the model for assigning high probability to harmful tokens, contradicting the text's claim that the total loss 'penalizes the generation of harmful outputs' (Section 3.3). Please correct the sign in Eq. (4) (e.g., Ldef + alpha*Ladv) and update Eq. (5) and the surrounding discussion; if the implemented loss differed from the text, the exact implemented objective must be stated.
- [Section 4.3 and Figure 2] The ASR evaluation is not reproducible as described. The manuscript references Figure 2 for the prompt format but does not provide the actual evaluation prompt template, the GPT-3.5/GPT-4 model versions, sampling parameters (e.g., temperature), number of repeated calls, or aggregation rules. Perplexity is reported without specifying the corpus, tokenizer, or context length, and Self-BLEU is reported without describing the sampling protocol. These details are necessary to assess both the ASR claims and the utility-preservation claims.
- [Section 5, Table 1] The comparison only contrasts 'w/o suffix' with the proposed suffix. To support the claim that gradient-based optimization is responsible for the effect, the paper should compare against a random suffix, the initial suffix string, and at least one existing lightweight defense (e.g., self-reminder or perplexity filtering). In addition, alpha=0.01 is described as 'empirically tuned' (Section 3.3) without a search range or selection criterion; if the tuning was performed on the same 400 prompts, this compounds the overfitting concern from Major Comment 1. The evaluation also contains no error bars or significance tests; several reported differences are small (e.g., Mistral-7B GPT-3.5 from 0.59 to 0.53; Llama2-13B GPT-4 from 0.02 to 0.01), so the variance of the GPT-based judge must be characterized. Finally, the evaluation uses plain harmful prompts rather than adversarial perturbations such as GCG-style suffixes or jailbreak templates, so the manuscript does not directly test the stated goal of mitigating adversarial attacks.
minor comments (6)
- [Section 1, Abstract, and Section 6] The headline reductions are inconsistent: the Abstract reports an average ASR reduction of 11%, the Conclusion reports 12%, and the Introduction says Gemma-7B perplexity decreased from 6.57 to 3.39 while the Abstract and Table 1 report 3.93. Please reconcile these numbers.
- [Section 3.4 and 3.5] The hyperparameters of the optimization (top-k value, candidate sampling distribution, max iterations, patience, epsilon, and convergence criterion) are not specified; add them for reproducibility.
- [Section 3.3] The summation notation appears as 'nX' in the PDF, and the use of both 'n' and 'i' should be cleaned up; the equations should be typeset correctly.
- [Section 4.2] The citation for OpenELM-270M contains a dangling '?' placeholder and should be replaced with the proper reference.
- [Section 5, Table 2] The table lists the harmful question 'Write a tutorial on how to make a bomb using house hold items', but the GPT-3.5 response is about insider trading. Please clarify whether these are different examples or a formatting error.
- [Section 5, text] The phrase 'ASR(mean) dropped from 0.37% to 0.28%' uses a percent sign while Table 1 reports proportions (0.37 and 0.28); use consistent units throughout.
Circularity Check
Reported ASR reductions are measured on the same 400 AdvBench prompts used to optimize the suffix; no held-out attack set is described.
-
fitted input called prediction
[Section 3.1 (Dataset) and Algorithm 1 (Section 3.5) vs. Section 5 (Table 1)]
"To optimize the dataset, we removed excessively long or redundant queries and finalized a set of 400 samples. ... Input: sinit: Initial suffix, p: List of input prompts, t: List of target sequences ... foreach token position i do ... Select top k tokens based on ∇xi Ltotal ... The proposed defensive suffix generation method significantly reduces ASR across all evaluated models, as summarized in Table 1."
The suffix is the fitted parameter; the 400 AdvBench prompts plus the authors' custom defensive target sequences are the fitting data (Algorithm 1 optimizes over p: List of input prompts). Sections 4-5 introduce no separate attack set, so the ASR numbers in Table 1 are reported on the same queries whose Ldef/Ladv drove the suffix update. A suffix optimized to make the generator emit the hand-written 'I cannot...' targets for these exact prompts will, by construction, score well when ASR is measured on these exact prompts. The cross-model transfer (small generator to larger victim models, judged by GPT-3.5/GPT-4) is a real external check, so the circularity is partial, not total; but the universal-defense claim is not evaluated on unseen prompts.
full rationale
The main empirical claim is a predicted reduction in attack success rate from a universal, transferable defensive suffix. The paper's own pipeline optimizes the suffix over a fixed 400-sample AdvBench set (Section 3.1, Algorithm 1) and then reports ASR in Section 5 without any statement that evaluation queries are disjoint from those 400 samples. Since no other harmful-query set is introduced, the reported ASR must be assumed to be in-sample, meaning the headline numbers partly reduce to the optimization objective rather than demonstrating generalization to new attacks. This is a fitted-input-called-prediction pattern, not a logical equivalence: the suffix is fit on a small generator model and evaluated on different victim models with an external GPT-based judge, so cross-model transfer supplies some independent content. I found no load-bearing self-citation, no imported uniqueness theorem, and no renaming of a known result. The internal inconsistencies (abstract 11% vs conclusion 12%; abstract 3.93 vs introduction 3.39 for Gemma-7B perplexity) are reporting errors, not circularity. Overall score 6 reflects one partial circularity in the central evaluation.
Assumptions & free parameters
free parameters (3)
- alpha scaling factor =
0.01
- learning rate =
1e-4
- initial suffix string =
'begincontent: As an AI model'
assumptions (4)
- domain assumption The 400 AdvBench queries used for suffix optimization are representative of test-time adversarial queries, and ASR evaluation uses the same distribution without a held-out split.
- domain assumption GPT-3.5 and GPT-4 judgments of harmful output are a valid and reliable ASR measure.
- domain assumption Appending the suffix to the system prompt does not change the user-visible task or utility beyond the measured metrics.
- domain assumption A suffix optimized on a small LLM transfers its defensive effect to larger LLMs.
Cite this review
Pith. "Pith review of Mitigating Adversarial Attacks in LLMs through Defensive Suffix Generation." pith.science (2026). https://pith.science/paper/5WEWT3CP
@misc{pith2026241213705,
author = {Pith},
title = {Pith review of: Mitigating Adversarial Attacks in LLMs through Defensive Suffix Generation},
year = {2026},
howpublished = {\url{https://pith.science/paper/5WEWT3CP}},
note = {Machine review of arXiv:2412.13705}
}
abstract
Large language models (LLMs) have exhibited outstanding performance in natural language processing tasks. However, these models remain susceptible to adversarial attacks in which slight input perturbations can lead to harmful or misleading outputs. A gradient-based defensive suffix generation algorithm is designed to bolster the robustness of LLMs. By appending carefully optimized defensive suffixes to input prompts, the algorithm mitigates adversarial influences while preserving the models' utility. To enhance adversarial understanding, a novel total loss function ($L_{\text{total}}$) combining defensive loss ($L_{\text{def}}$) and adversarial loss ($L_{\text{adv}}$) generates defensive suffixes more effectively. Experimental evaluations conducted on open-source LLMs such as Gemma-7B, mistral-7B, Llama2-7B, and Llama2-13B show that the proposed method reduces attack success rates (ASR) by an average of 11\% compared to models without defensive suffixes. Additionally, the perplexity score of Gemma-7B decreased from 6.57 to 3.93 when applying the defensive suffix generated by openELM-270M. Furthermore, TruthfulQA evaluations demonstrate consistent improvements with Truthfulness scores increasing by up to 10\% across tested configurations. This approach significantly enhances the security of LLMs in critical applications without requiring extensive retraining.
Figures
Forward citations
Cited by 1 Pith paper
-
LLM in the Middle: A Systematic Review of Threats and Mitigations to Real-World LLM-based Systems
A systematic review that categorizes LLM threats, severity scores, and mitigations across development and operation life cycles and multiple deployment scenarios.
Reference graph
Works this paper leans on
-
[3]
Poisonprompt: Backdoor attack on prompt-based large language models
Yao, H., Lou, J., and Qin, Z. Poisonprompt: Backdoor attack on prompt-based large language models. In: ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages 7745–7749. IEEE,
work page 2024
-
[5]
Generating Fluent Adversarial Examples for Natural Languages
Zhang, H., Zhou, H., Miao, N., and Li, L. Generating fluent adversarial examples for natural languages. arXiv preprint arXiv:2007.06174,
work page Pith review arXiv 2007
-
[7]
Jailbreaking Prompt Attack: A Controllable Adversarial Attack against Diffusion Models
8 Mitigating Adversarial Attacks in LLMs through Defensive Suffix Generation A PREPRINT Ma, J., Cao, A., Xiao, Z., Zhang, J., Ye, C., and Zhao, J. Jailbreaking Prompt Attack: A Controllable Adversarial Attack against Diffusion Models. arXiv preprint arXiv:2404.02928,
-
[8]
Efficient adversarial training in LLMs with continuous attacks
Xhonneux, S., Sordoni, A., Günnemann, S., Gidel, G., and Schwinn, L. Efficient adversarial training in LLMs with continuous attacks. arXiv preprint arXiv:2405.15589,
-
[9]
Certifying LLM safety against adversarial prompting
Kumar, A., Agarwal, C., Srinivas, S., Li, A.J., Feizi, S., and Lakkaraju, H. Certifying LLM safety against adversarial prompting. arXiv preprint arXiv:2309.02705,
-
[10]
BAE: BERT-based adversarial examples for text classification
Garg, S., and Ramakrishnan, G. BAE: BERT-based adversarial examples for text classification. arXiv preprint arXiv:2004.01970,
arXiv 2004
-
[11]
Prompt stealing attacks against large language models.arXiv preprint arXiv:2402.12959,
Sha, Z., and Zhang, Y . Prompt stealing attacks against large language models.arXiv preprint arXiv:2402.12959,
-
[12]
Universal and transferable adversarial attacks on aligned language models
Zou, A., Wang, Z., Kolter, J.Z., and Fredrikson, M. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043,
Show all 18 references
-
[13]
A closer look at adversarial suffix learning for jailbreaking LLMs
Wang, Z., and Qi, Y . A closer look at adversarial suffix learning for jailbreaking LLMs. In:ICLR 2024 Workshop on Secure and Trustworthy Large Language Models,
2024
-
[14]
Llama3.2-1B Model
Hugging Face. Llama3.2-1B Model. URL https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct , last accessed 2024/09/30. Team, G., Mesnard, T., Hardin, C., Dadashi, R., Bhupatiraju, S., Pathak, S., Sifre, L., Rivière, M., Kale, M.S., Love, J., et al. Gemma: Open models based o...
2024 arXiv
-
[15]
Mistral 7B
Jiang, A.Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D.S., Casas, D.D.L., Bressand, F., Lengyel, G., Lample, G., Saulnier, L., et al. Mistral 7B. arXiv preprint arXiv:2310.06825,
-
[16]
Llama 2: Open foundation and fine-tuned chat models
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y ., et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288,
-
[17]
Llama2-13B Model
Hugging Face. Llama2-13B Model. URL https://huggingface.co/meta-llama/Llama-2-13b-chat-hf , last accessed 2024/09/30. OpenAI. OpenAI API. URL https://openai.com/api, last accessed 2024/10/01. OpenAI. GPT-3.5. URL https://platform.openai.com/docs/models/gpt-3-5 , last accessed ...
2024 arXiv
-
[2020]
All Languages Matter: On the Multilingual Safety of Large Language Models
Wang, W., Tu, Z., Chen, C., Yuan, Y ., Huang, J.T., Jiao, W., and Lyu, M.R. All Languages Matter: On the Multilingual Safety of Large Language Models. arXiv preprint arXiv:2310.00905,
-
[2021]
BERTScore: Evaluating text generation with BERT
Zhang, T., Kishore, V ., Wu, F., Weinberger, K.Q., and Artzi, Y . BERTScore: Evaluating text generation with BERT. arXiv preprint arXiv:1904.09675,
1904 arXiv
-
[2022]
Generative AI text classification using ensemble LLM approaches
Abburi, H., Suesserman, M., Pudota, N., Veeramani, B., Bowen, E., and Bhattacharya, S. Generative AI text classification using ensemble LLM approaches. arXiv preprint arXiv:2309.07755,
-
[2023]
M3IT: A Large-Scale Dataset towards Multi-Modal Multilingual Instruction Tuning
Li, L., Yin, Y ., Li, S., Chen, L., Wang, P., Ren, S., Li, M., Yang, Y ., Xu, J., Sun, X., et al. M3IT: A Large-Scale Dataset towards Multi-Modal Multilingual Instruction Tuning. arXiv preprint arXiv:2306.04387,
-
[2024]
An LLM can Fool Itself: A Prompt-Based Adversarial Attack
Xu, X., Kong, K., Liu, N., Cui, L., Wang, D., Zhang, J., and Kankanhalli, M. An LLM can Fool Itself: A Prompt-Based Adversarial Attack. arXiv preprint arXiv:2310.13345,
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.