Pith. sign in

REVIEW 5 major objections 6 minor 53 references

Adversarial Attack on Large Language Models using Exponentiated Gradient Descent

T0 review · 5 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that exponentiated gradient descent with Bregman projection produces a white-box jailbreak attack on open-source LLMs that is faster and more successful than GCG, PGD, and SoftPromptThreats.

desk verdict A plausible incremental optimization tweak for white-box jailbreaks, honestly flagged as unproven for the tested variant, but the headline ASR advantage is not statistically supported by single-run results on 50 behaviors per dataset. read the letter →

arxiv 2505.09820 v1 pith:3TI5NMUO submitted 2025-05-14 cs.LG cs.CLcs.CR

Sajib Biswas , Mao Nishino , Samuel Jacob Chacko , Xiuwen Liu This is my paper · ORCID
classification cs.LGcs.CLcs.CR
keywords adversarialattackjailbreaklargelanguagemodelsexponentiatedgradientdescentBregmanprojectionprobabilitysimplexsuccessrateLLMsafety
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a white-box jailbreak of open-source large language models can be cast as a constrained optimization over relaxed one-hot token encodings, and that exponentiated gradient descent with KL-divergence (Bregman) projection solves it faster and more effectively than the leading alternatives. The authors prove that their update converges to a critical point of the adversarial loss when the loss is smooth, and they implement a practical version with Adam and entropic regularization. On five open models and four public datasets they report higher attack success rates and shorter runtimes than GCG, projected gradient descent, and SoftPromptThreats. If the claim holds, it gives safety researchers a cheap tool for finding aligned-model failures.

What carries the argument

The load-bearing object is Equation (6), the exponentiated-gradient-descent update with Bregman projection: $\tilde{X}_t = P_{KL}(\tilde{X}_{t-1} \odot \exp(-\eta_t \nabla F(\tilde{X}_{t-1})))$. The Bregman projection here is the KL-divergence projection onto the set of row-stochastic matrices, whose closed form is row normalization. This single step makes the simplex constraint intrinsic to the optimization rather than an external repair step, and the theorem connects the iteration to the forward-backward algorithm to get convergence. In the practical attack, Adam momentum, entropic regularization $-\tau H(X)$, and a KL term between the continuous and discretized encodings are added to sparsify the relaxed distribution.

What would settle it

Run the exact EGD-with-Adam implementation from Appendix A on a model with nonsmooth ReLU activations and measure the gradient-norm trajectory; if the iterates do not approach a critical point, or if attack success collapses when Adam and the regularization terms are removed on a smooth model such as Llama-2, then the implemented algorithm's behavior is not explained by Theorem 1.

Watch

Extended reading notes

Core claim

The central discovery is that optimizing a relaxed one-hot encoding of the adversarial suffix does not need an explicit projection back to the probability simplex, as long as the update itself is exponentiated gradient descent followed by normalization: $\tilde{X}_t = P_{KL}(\tilde{X}_{t-1} \odot \exp(-\eta_t \nabla F(\tilde{X}_{t-1})))$. Each row remains a distribution because the Bregman projection simply rescales rows to sum to one. The paper proves this iteration reaches a critical point for differentiable losses with Lipschitz gradient by identifying it as a forward-backward algorithm. In the implemented system, an Adam-style momentum variant, entropic regularization, and a KL sparsity term are added; with those components, the paper reports higher attack success rates than the three baselines on most of the tested models and the lowest runtime per optimization.

Load-bearing premise

The load-bearing premise is that the adversarial loss is differentiable with a Lipschitz-continuous gradient, which the paper itself notes fails for ReLU-based models and does not cover the Adam variant actually used in the experiments; if that premise gives way, the convergence theorem no longer applies and the method's success stands only on its empirical results.

Editorial extensions

If this is right

  • White-box red-teaming becomes cheaper: for the same number of epochs, the reported method completes a single harmful-behavior optimization in less runtime than all three baselines.
  • Attack success rate improves on most tested models; for example, on Mistral-7B the overall success rate at the stricter evaluator threshold is 54.5% versus 45.0% for GCG, 38.5% for PGD, and 24.5% for SoftPromptThreats.
  • The relaxed-one-hot formulation with KL projection keeps each token position on the probability simplex by construction, removing the need for an external nearest-neighbor projection step that PGD requires.
  • For smooth models such as Llama-2, the attack iteration carries a convergence guarantee to a critical point, giving the empirical results a theoretical anchor in that setting.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because Theorem 1 explicitly excludes the Adam variant used in the experiments, the empirical results carry the argument for most of the tested models; a convergence proof for the nonsmooth or Adam setting would close that gap.
  • The entropic and KL regularization terms are largely decoupled from the convergence theorem, so an ablation on a smooth model could reveal whether the theoretical update alone drives jailbreak success or whether the regularization is essential.
  • If the runtime advantage persists beyond the 50-prompt samples, batch or universal suffix searches, where one suffix is optimized across many harmful behaviors, become considerably cheaper; the paper lists universality and transferability as future work.
  • The same intrinsic simplex optimization could transfer to multimodal models that accept continuous inputs, since the relaxed one-hot mechanism is not tied to text-specific preprocessing.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes an adversarial attack on large language models using exponentiated gradient descent (EGD) applied to a continuously relaxed one-hot encoding of the input tokens. The method maintains the relaxed encoding on the probability simplex by combining a multiplicative EGD step with a Bregman (KL) projection, as formalized in Equation (6). The authors claim to prove convergence of this iteration for smooth losses (Theorem 1), and they implement an efficient variant that additionally uses the Adam optimizer and two regularization terms (entropic regularization and a KL sparsity term), as described in Appendix A. The attack is evaluated on five open-source LLMs (Llama-2, Vicuna, Mistral, Falcon, MPT) and four jailbreak datasets, using the first 50 harmful behaviors per dataset. The reported results in Table II compare the attack success rate (ASR) with GCG, PGD, and SoftPromptThreats, and Figure 2 compares runtime. The paper concludes that the proposed EGD attack 'achieves a higher success rate with great efficiency' compared with the three baselines.

Significance. If the empirical and theoretical claims were fully supported, the paper would offer a useful white-box jailbreaking method with a principled treatment of the simplex constraint, and it would contribute a practical comparison of mirror-descent-style optimization to existing discrete and continuous attacks. The idea of optimizing a relaxed one-hot encoding with EGD is a sensible extension of prior work on PGD-based attacks, and the availability of source code is a plus. However, the significance is currently limited by two central weaknesses. First, the convergence theorem (Theorem 1) does not apply to the algorithm actually evaluated in the experiments: the authors explicitly concede in Remark 1 that the theorem excludes the Adam-based variant used throughout the empirical section. Second, the empirical superiority claim is not statistically substantiated: the ASR results come from a single run on only 50 behaviors per dataset, with no error bars, repeated seeds, or significance tests, and one of the five model rows (MPT) contradicts the headline claim. The theoretical result is also essentially standard mirror-descent convergence, so the novelty is incremental.

major comments (5)
  1. [Section III-D and Appendix A] The convergence guarantee in Theorem 1 applies to the plain EGD iteration (6) under the assumption that F is differentiable with Lipschitz gradient. The algorithm actually used in the experiments is the EGD-with-Adam variant (Appendix A, Eq. 13) with a composite loss that includes entropic regularization and a KL sparsity term (Eq. 15) and a time-varying tau schedule. Remark 1 explicitly states that the theorem does not apply to the Adam-based variant. Therefore the paper's claim that it 'prove[s] the convergence of the technique' (Abstract) is not established for the implemented method. The authors should either provide a convergence analysis for the regularized, Adam-updated variant or restrict the empirical claims to the plain EGD update that the theorem actually covers.
  2. [Section IV-B2 and Table II] The reported ASR values are based on a single run over the first 50 behaviors of each dataset, with no repeated runs, no error bars, and no significance tests. For N=50, the 95% confidence interval for a proportion is approximately +/-13 percentage points, so most pairwise differences in Table II (e.g., Llama-2 AdvBench: EGD 10 vs GCG 5) are indistinguishable from noise. The MPT row actively contradicts the headline: EGD's overall ASR(>10) is 39.0% versus GCG's 42.0%, and on MaliciousInstruct EGD scores 21 while GCG scores 29. The runtime comparison in Figure 2 is also a single average without variance. Consequently, the abstract's claim that the method 'achieves a higher success rate with great efficiency' is not supported by the evidence. The authors should provide multi-seed runs, standard errors, and appropriate statistical tests (e.g., paired bootstrap or McNemar's test).
  3. [Section IV-A4] The hyperparameters appear to have been selected on the evaluation data. The text states that 'using a fixed learning rate eta=0.1 works best for our method in most cases' and that the number of epochs was 'determine[d] empirically' from the optimization curves. There is no description of a validation split or a priori fixed hyperparameters. If eta, the tau schedule, and the epoch count were tuned after inspecting the same 50-behavior test sets used in Table II, the reported ASRs are optimistically biased and the comparison with baselines is not fair. The authors should specify a proper tuning protocol (e.g., a held-out validation split) or report results for fixed hyperparameters that were not selected using the test data.
  4. [Section V and Eq. (6)] The discussion claims that the method 'inherently satisfies the constraints on the input without requiring any projection technique to enforce them' and that it eliminates 'the necessity of an extrinsic technique, such as projection.' This is contradicted by the main iteration (6), which explicitly applies the Bregman projection P_KL, and by Section III-C, which is devoted to deriving that projection. The plain EGD update (2) does not ensure row-wise simplex constraints for a matrix variable, which is why the projection is introduced. The presentation is internally inconsistent and overstates the contribution; it should be corrected to acknowledge that the method uses a KL-based projection as part of the update.
  5. [Theorem 1 proof (Section III-D)] The proof of Theorem 1 is too terse to be verifiable. The equivalence between iteration (6) and the forward-backward form (7) is asserted without derivation, and no conditions on the learning rates are given beyond 'small enough.' A rigorous argument should specify the required assumptions (e.g., strong convexity of the KL mirror map relative to the feasible set, the Lipschitz constant of the gradient, and the explicit bound on eta_t). Without these details, the theorem as stated is not established to a journal standard.
minor comments (6)
  1. [Equation (8)] There is a dimension mismatch in the definition of the set C: it is written as \tilde X \in [0,1]^{|T|\times L} with the condition \tilde X 1_{|T|} = 1_L, but this matrix-vector product is not well-typed for the stated dimensions. Please reconcile the dimensions.
  2. [Equation (4)] The KL divergence formula in Eq. (4) includes an extra '-1' inside the sum; the standard definition is KL(Y||X) = \sum Y_ij \log(Y_ij/X_ij). Please clarify whether the '-1' is intentional or a typographical error.
  3. [Table II] The 'Overall(%)' rows should also report the absolute counts (e.g., 20/200) alongside the percentages, so that readers can assess the uncertainty given that each dataset contributes only 50 items.
  4. [Figure 2] The runtime comparison reports a single average per method without variance or the number of trials; please add error bars or per-run values.
  5. [Section IV-A3] The sentence 'For consistency, we follow the method prescribed by the corresponding authors to initialize the adversarial suffix in each instance. For GCG and SoftPromptThreats, we use a sequence of 20 space-separated exclamation marks ("!").' is ambiguous about whether the '!' initialization is the one prescribed by the authors; please rephrase for clarity.
  6. [Appendix A, Eq. (13)] The notation for the Adam denominator is inconsistent: the text uses both '\delta+' and '\epsilon+' in the same equation. Please unify the symbol.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the convergence proof is imported from an external forward-backward result, and the empirical ASR is measured with external evaluators; the only self-citation ([51]) is not load-bearing.

full rationale

The paper's derivation chain is self-contained. The main theoretical claim (Theorem 1) establishes convergence of Equation (6) by showing the update is a special case of the forward-backward algorithm and cites Bot et al. [41] for the convergence proof; this is an external, non-self citation. The update itself is the standard exponentiated gradient / mirror descent step with row-normalization (the 'Bregman projection'), and no fitted quantity is relabeled as a prediction. The experimental claim (higher ASR, greater efficiency) is evaluated with two external model-based judges (Llama3 and Beaver-Cost), not with the attack's own loss, so success is not defined circularly. The only self-citation is [51] (Chacko, Biswas, Islam, Liza, Liu), cited in Section IV-B.1 to support the practice of using model-based evaluators and interpreting Beaver-Cost thresholds. That citation is not load-bearing for the central claim: the evaluators themselves are external, and the threshold choice does not enter the optimization or convergence argument. The paper's own Remark 1 honestly limits Theorem 1 to smooth models without Adam, which is a correctness/coverage limitation rather than a circularity. Hyperparameters such as eta=0.1 and the tau schedule are selected empirically, but they are not reused as constants in a derivation that is then claimed to predict ASR. Overall, no load-bearing argument reduces to its own input.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The theoretical claim rests on the smoothness assumption that the paper partially retracts in Remark 1. The empirical claims rest on the surrogate relaxation, the target cross-entropy loss, and the automated evaluators. Several hyperparameters are chosen by hand, and the exact epoch count is not reported.

free parameters (4)
  • learning rate eta = 0.1
    Chosen by hand: the paper states 'we observe that using a fixed learning rate eta=0.1 works best in most cases' (Section IV.A.4). It is not derived and affects convergence speed and final ASR.
  • tau schedule endpoints = 1e-5 to 1e-3 (exponential schedule)
    The entropic and KL regularization weights are annealed from 1e-5 to 1e-3 for all experiments (Appendix A.3). No validation procedure is described for these bounds.
  • adversarial suffix length = 20 tokens
    The suffix length is fixed at 20 for all methods and datasets (Section IV.A.4). No sensitivity analysis is provided.
  • number of epochs = about 200 (Figure 1)
    The paper notes convergence within a few hundred epochs, and Figure 1 plots 200 epochs, but the exact epoch count used for Table II is not stated. The same count is used for all baselines.
assumptions (4)
  • domain assumption The loss F is differentiable with a Lipschitz continuous gradient.
    Required for Theorem 1. Remark 1 explicitly notes this is not satisfied for ReLU-based models and does not apply to the EGD-with-Adam variant used in the experiments.
  • domain assumption The relaxed one-hot encoding over row-stochastic matrices is a valid surrogate for discrete token optimization.
    Section III.A replaces discrete tokens with continuous row-stochastic matrices and later uses argmax to discretize. The faithfulness of this relaxation to the true jailbreak objective is assumed.
  • domain assumption The adversarial cross-entropy loss on a fixed target string is a suitable proxy for producing harmful content.
    Section III.A defines the loss as negative log-likelihood of target tokens. Whether minimizing this loss reliably induces harmful behavior is not validated independently of the automated evaluators.
  • domain assumption The automated evaluators (Llama3 boolean and Beaver-Cost score) correctly identify harmful responses at the chosen thresholds.
    Section IV.B.1 defines ASR using these judges, but no human-validated agreement rates on the generated jailbroken outputs are provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Adversarial Attack on Large Language Models using Exponentiated Gradient Descent." pith.science (2026). https://pith.science/paper/3TI5NMUO

@misc{pith2026250509820,
  author       = {Pith},
  title        = {Pith review of: Adversarial Attack on Large Language Models using Exponentiated Gradient Descent},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3TI5NMUO}},
  note         = {Machine review of arXiv:2505.09820}
}
read the original abstract

As Large Language Models (LLMs) are widely used, understanding them systematically is key to improving their safety and realizing their full potential. Although many models are aligned using techniques such as reinforcement learning from human feedback (RLHF), they are still vulnerable to jailbreaking attacks. Some of the existing adversarial attack methods search for discrete tokens that may jailbreak a target model while others try to optimize the continuous space represented by the tokens of the model's vocabulary. While techniques based on the discrete space may prove to be inefficient, optimization of continuous token embeddings requires projections to produce discrete tokens, which might render them ineffective. To fully utilize the constraints and the structures of the space, we develop an intrinsic optimization technique using exponentiated gradient descent with the Bregman projection method to ensure that the optimized one-hot encoding always stays within the probability simplex. We prove the convergence of the technique and implement an efficient algorithm that is effective in jailbreaking several widely used LLMs. We demonstrate the efficacy of the proposed technique using five open-source LLMs on four openly available datasets. The results show that the technique achieves a higher success rate with great efficiency compared to three other state-of-the-art jailbreaking techniques. The source code for our implementation is available at: https://github.com/sbamit/Exponentiated-Gradient-Descent-LLM-Attack

Figures

Figures reproduced from arXiv: 2505.09820 by the authors.

Figure 1
Figure 1. Median Cross-entropy Loss, aggregated over [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Comparison of average run-time (log-scale) of the baseline methods [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Effect of the regularization terms on the sparsity of the one-hot [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

53 extracted references · 14 canonical work pages

  1. [1]

    Unified pre-training for program understanding and generation,

    W. U. Ahmad, S. Chakraborty, B. Ray, and K.-W. Chang, “Unified pre-training for program understanding and generation,”arXiv preprint arXiv:2103.06333, 2021

    arXiv 2021

  2. [2]

    Language mod- els are few-shot learners,

    T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askellet al., “Language mod- els are few-shot learners,”Advances in neural information processing systems, vol. 33, pp. 1877–1901, 2020

    1901

  3. [3]

    Who is gpt-3? an ex- ploration of personality, values and demographics,

    M. Miotto, N. Rossberg, and B. Kleinberg, “Who is gpt-3? an ex- ploration of personality, values and demographics,”arXiv preprint arXiv:2209.14338, 2022

    arXiv 2022

  4. [4]

    Large language models surpass human experts in predicting neuroscience results,

    X. Luo, A. Rechardt, G. Sun, K. K. Nejad, F. Y ´a˜nez, B. Yilmaz, K. Lee, A. O. Cohen, V . Borghesani, A. Pashkovet al., “Large language models surpass human experts in predicting neuroscience results,”arXiv preprint arXiv:2403.03230, 2024

    arXiv 2024

  5. [5]

    Chatgpt and large language models in academia: opportunities and challenges,

    J. G. Meyer, R. J. Urbanowicz, P. C. Martin, K. O’Connor, R. Li, P.- C. Peng, T. J. Bright, N. Tatonetti, K. J. Won, G. Gonzalez-Hernandez et al., “Chatgpt and large language models in academia: opportunities and challenges,”BioData Mining, vol. 16, no. 1, p. 20, 2023

    work page 2023

  6. [6]

    Ethical and social risks of harm from language models. arxiv,

    L. Weidinger, J. Mellor, M. Rauh, C. Griffin, J. Uesato, P.-S. Huang, M. Cheng, M. Glaese, B. Balle, A. Kasirzadehet al., “Ethical and social risks of harm from language models. arxiv,”arXiv preprint arXiv:2112.04359, vol. 10, 2021

    arXiv 2021

  7. [7]

    Extracting training data from large language models,

    N. Carlini, F. Tramer, E. Wallace, M. Jagielski, A. Herbert-V oss, K. Lee, A. Roberts, T. Brown, D. Song, U. Erlingssonet al., “Extracting training data from large language models,” in30th USENIX Security Symposium (USENIX Security 21), 2021, pp. 2633–2650

    work page 2021

  8. [8]

    Training language models to follow instructions with human feedback,

    L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Rayet al., “Training language models to follow instructions with human feedback,”Advances in neural information processing systems, vol. 35, pp. 27 730–27 744, 2022

    2022

Show all 53 references
  1. [9]

    Pretraining language models with human preferences,

    T. Korbak, K. Shi, A. Chen, R. V . Bhalerao, C. Buckley, J. Phang, S. R. Bowman, and E. Perez, “Pretraining language models with human preferences,” inInternational Conference on Machine Learning. PMLR, 2023, pp. 17 506–17 533

    2023

  2. [10]

    Rain: Your lan- guage models can align themselves without finetuning,

    Y . Li, F. Wei, J. Zhao, C. Zhang, and H. Zhang, “Rain: Your lan- guage models can align themselves without finetuning,”arXiv preprint arXiv:2309.07124, 2023

    2023 arXiv

  3. [11]

    Jailbreaker: Automated jailbreak across multiple large language model chatbots,

    G. Deng, Y . Liu, Y . Li, K. Wang, Y . Zhang, Z. Li, H. Wang, T. Zhang, and Y . Liu, “Jailbreaker: Automated jailbreak across multiple large language model chatbots,”arXiv preprint arXiv:2307.08715, 2023

    2023 arXiv

  4. [12]

    Jailbreaking black box large language models in twenty queries,

    P. Chao, A. Robey, E. Dobriban, H. Hassani, G. J. Pappas, and E. Wong, “Jailbreaking black box large language models in twenty queries,”arXiv preprint arXiv:2310.08419, 2023

    2023 arXiv

  5. [13]

    Intriguing properties of neural networks,

    C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus, “Intriguing properties of neural networks,”arXiv preprint arXiv:1312.6199, 2013

    2013 arXiv

  6. [14]

    Explaining and harnessing adversarial examples,

    I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing adversarial examples,”arXiv preprint arXiv:1412.6572, 2014

    2014 arXiv

  7. [15]

    Jailbroken: How does llm safety training fail?

    A. Wei, N. Haghtalab, and J. Steinhardt, “Jailbroken: How does llm safety training fail?”Advances in Neural Information Processing Sys- tems, vol. 36, 2024

    2024

  8. [16]

    Prompt engineering in large language models,

    G. Marvin, N. Hellen, D. Jjingo, and J. Nakatumba-Nabende, “Prompt engineering in large language models,” inInternational conference on data intelligence and cognitive informatics. Springer, 2023, pp. 387– 402

    2023

  9. [17]

    Auto- prompt: Eliciting knowledge from language models with automatically generated prompts,

    T. Shin, Y . Razeghi, R. L. Logan IV , E. Wallace, and S. Singh, “Auto- prompt: Eliciting knowledge from language models with automatically generated prompts,”arXiv preprint arXiv:2010.15980, 2020

    2010 arXiv

  10. [18]

    Hard prompts made easy: Gradient-based discrete optimization for prompt tuning and discovery,

    Y . Wen, N. Jain, J. Kirchenbauer, M. Goldblum, J. Geiping, and T. Gold- stein, “Hard prompts made easy: Gradient-based discrete optimization for prompt tuning and discovery,”Advances in Neural Information Processing Systems, vol. 36, 2024

    2024

  11. [19]

    Are aligned neural networks adversarially aligned?

    N. Carlini, M. Nasr, C. A. Choquette-Choo, M. Jagielski, I. Gao, P. W. W. Koh, D. Ippolito, F. Tramer, and L. Schmidt, “Are aligned neural networks adversarially aligned?”Advances in Neural Information Processing Systems, vol. 36, 2024

    2024

  12. [20]

    Catastrophic jailbreak of open-source llms via exploiting generation,

    Y . Huang, S. Gupta, M. Xia, K. Li, and D. Chen, “Catastrophic jailbreak of open-source llms via exploiting generation,”arXiv preprint arXiv:2310.06987, 2023

    2023 arXiv

  13. [21]

    Universal and transferable adversarial attacks on aligned language models,

    A. Zou, Z. Wang, N. Carlini, M. Nasr, J. Z. Kolter, and M. Fredrikson, “Universal and transferable adversarial attacks on aligned language models,”arXiv preprint arXiv:2307.15043, 2023

    2023 arXiv

  14. [22]

    Fast adversarial attacks on language models in one gpu minute,

    V . S. Sadasivan, S. Saha, G. Sriramanan, P. Kattakinda, A. Chegini, and S. Feizi, “Fast adversarial attacks on language models in one gpu minute,”arXiv preprint arXiv:2402.15570, 2024

    2024 arXiv

  15. [23]

    Attacking large language models with projected gradient descent,

    S. Geisler, T. Wollschl ¨ager, M. H. I. Abdalla, J. Gasteiger, and S. G ¨unnemann, “Attacking large language models with projected gradient descent,” 2024. [Online]. Available: https://arxiv.org/abs/2402 .09154

    2024

  16. [24]

    Soft prompt threats: Attacking safety alignment and unlearning in open-source llms through the embedding space,

    L. Schwinn, D. Dobre, S. Xhonneux, G. Gidel, and S. Gunne- mann, “Soft prompt threats: Attacking safety alignment and unlearning in open-source llms through the embedding space,”arXiv preprint arXiv:2402.09063, 2024

    2024 arXiv

  17. [25]

    Jailbreak in pieces: Compositional adversarial attacks on multi-modal language models,

    E. Shayegani, Y . Dong, and N. Abu-Ghazaleh, “Jailbreak in pieces: Compositional adversarial attacks on multi-modal language models,” inThe Twelfth International Conference on Learning Representations, 2023

    2023

  18. [26]

    Assessing adversarial robustness of large language models: An empirical study,

    Z. Yang, Z. Meng, X. Zheng, and R. Wattenhofer, “Assessing adversarial robustness of large language models: An empirical study,”arXiv preprint arXiv:2405.02764, 2024

    2024 arXiv

  19. [27]

    Geometric analysis and metric learning of instruction embeddings,

    S. Biswas, T. Barao, J. Lazzari, J. McCoy, X. Liu, and A. Kostandarithes, “Geometric analysis and metric learning of instruction embeddings,” in2022 International Joint Conference on Neural Networks (IJCNN). IEEE, 2022, pp. 1–8

    2022

  20. [28]

    Large language models as superpositions of cultural perspectives,

    G. Kova ˇc, M. Sawayama, R. Portelas, C. Colas, P. F. Dominey, and P.-Y . Oudeyer, “Large language models as superpositions of cultural perspectives,”arXiv preprint arXiv:2307.07870, 2023

    2023 arXiv

  21. [29]

    Llama 2: Open foundation and fine-tuned chat models,

    H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y . Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosaleet al., “Llama 2: Open foundation and fine-tuned chat models,”arXiv preprint arXiv:2307.09288, 2023

    2023 arXiv

  22. [30]

    Universal adversarial triggers for attacking and analyzing nlp,

    E. Wallace, S. Feng, N. Kandpal, M. Gardner, and S. Singh, “Universal adversarial triggers for attacking and analyzing nlp,”arXiv preprint arXiv:1908.07125, 2019

    1908 arXiv

  23. [31]

    Hotflip: White-box adver- sarial examples for text classification,

    J. Ebrahimi, A. Rao, D. Lowd, and D. Dou, “Hotflip: White-box adver- sarial examples for text classification,”arXiv preprint arXiv:1712.06751, 2017

    2017 arXiv

  24. [32]

    Amplegcg: Learning a universal and transferable generative model of adversarial suffixes for jailbreaking both open and closed llms,

    Z. Liao and H. Sun, “Amplegcg: Learning a universal and transferable generative model of adversarial suffixes for jailbreaking both open and closed llms,”arXiv preprint arXiv:2404.07921, 2024

    2024 arXiv

  25. [33]

    Towards deep learning models resistant to adversarial attacks,

    A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, “Towards deep learning models resistant to adversarial attacks,”arXiv preprint arXiv:1706.06083, 2017

    2017 arXiv

  26. [34]

    Efficient projections onto the l 1-ball for learning in high dimensions,

    J. Duchi, S. Shalev-Shwartz, Y . Singer, and T. Chandra, “Efficient projections onto the l 1-ball for learning in high dimensions,” in Proceedings of the 25th international conference on Machine learning, 2008, pp. 272–279

    2008

  27. [35]

    Crafting ad- versarial input sequences for recurrent neural networks,

    N. Papernot, P. McDaniel, A. Swami, and R. Harang, “Crafting ad- versarial input sequences for recurrent neural networks,” inMILCOM 2016-2016 IEEE Military Communications Conference. IEEE, 2016, pp. 49–54

    2016

  28. [36]

    Exponentiated gradient versus gradient descent for linear predictors,

    J. Kivinen and M. K. Warmuth, “Exponentiated gradient versus gradient descent for linear predictors,”Information and Computation, vol. 132, no. 1, pp. 1–63, 1997. [Online]. Available: https: //www.sciencedirect.com/science/article/pii/S0890540196926127

    1997

  29. [37]

    Exponential gradient with momentum for online portfolio selection,

    Y . Li, X. Zheng, C. Chen, J. Wang, and S. Xu, “Exponential gradient with momentum for online portfolio selection,”Expert Systems with Applications, vol. 187, p. 115889, 2022

    2022

  30. [38]

    Adam: A method for stochastic optimization,

    D. P. Kingma, “Adam: A method for stochastic optimization,”arXiv preprint arXiv:1412.6980, 2014

    2014 arXiv

  31. [39]

    The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming,

    L. Bregman, “The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming,”USSR Computational Mathematics and Mathematical Physics, vol. 7, no. 3, pp. 200–217, 1967. [Online]. Available: https://www.scie...

    1967

  32. [40]

    Iterative bregman projections for regularized transportation problems,

    J.-D. Benamou, G. Carlier, M. Cuturi, L. Nenna, and G. Peyr ´e, “Iterative bregman projections for regularized transportation problems,”

  33. [41]

    An inertial forward-backward algorithm for the minimization of the sum of two nonconvex functions,

    R. I. Bot, E. R. Csetnek, and S. L ´aszl´o, “An inertial forward-backward algorithm for the minimization of the sum of two nonconvex functions,”

  34. [42]

    Computational optimal transport,

    G. Peyr ´e and M. Cuturi, “Computational optimal transport,” 2020. [Online]. Available: https://arxiv.org/abs/1803.00567

    2020 arXiv

  35. [43]

    Available: https://arxiv.org/abs/1410.0641

    [Online]. Available: https://arxiv.org/abs/1410.0641

    arXiv

  36. [44]

    Introducing mpt-7b: A new standard for open-source, commercially usable llms, 2023,

    M. Teamet al., “Introducing mpt-7b: A new standard for open-source, commercially usable llms, 2023,”URL www. mosaicml. com/blog/mpt- 7b. Accessed, pp. 05–05, 2023

    2023

  37. [45]

    The falcon series of open language models,

    E. Almazrouei, H. Alobeidli, A. Alshamsi, A. Cappelli, R. Cojo- caru, M. Debbah, ´E. Goffinet, D. Hesslow, J. Launay, Q. Malartic et al., “The falcon series of open language models,”arXiv preprint arXiv:2311.16867, 2023

    2023 arXiv

  38. [46]

    Judging llm-as-a-judge with mt-bench and chatbot arena,

    L. Zheng, W.-L. Chiang, Y . Sheng, S. Zhuang, Z. Wu, Y . Zhuang, Z. Lin, Z. Li, D. Li, E. Xinget al., “Judging llm-as-a-judge with mt-bench and chatbot arena,”Advances in Neural Information Processing Systems, vol. 36, 2024

    2024

  39. [47]

    Mistral 7b,

    A. Q. Jiang, A. Sablayrolles, A. Mensch, C. Bamford, D. S. Chaplot, D. d. l. Casas, F. Bressand, G. Lengyel, G. Lample, L. Saulnieret al., “Mistral 7b,”arXiv preprint arXiv:2310.06825, 2023

    2023 arXiv

  40. [48]

    Safe rlhf: Safe reinforcement learning from human feedback,

    J. Dai, X. Pan, R. Sun, J. Ji, X. Xu, M. Liu, Y . Wang, and Y . Yang, “Safe rlhf: Safe reinforcement learning from human feedback,”arXiv preprint arXiv:2310.12773, 2023

    2023 arXiv

  41. [49]

    The llama 3 herd of models,

    A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Yang, A. Fanet al., “The llama 3 herd of models,”arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  42. [50]

    Jailbreakbench: An open robustness benchmark for jailbreaking large language models,

    P. Chao, E. Debenedetti, A. Robey, M. Andriushchenko, F. Croce, V . Sehwag, E. Dobriban, N. Flammarion, G. J. Pappas, F. Trameret al., “Jailbreakbench: An open robustness benchmark for jailbreaking large language models,”arXiv preprint arXiv:2404.01318, 2024

    2024 arXiv

  43. [51]

    Harmbench: A standardized evaluation framework for automated red teaming and robust refusal,

    M. Mazeika, L. Phan, X. Yin, A. Zou, Z. Wang, N. Mu, E. Sakhaee, N. Li, S. Basart, B. Liet al., “Harmbench: A standardized evaluation framework for automated red teaming and robust refusal,”arXiv preprint arXiv:2402.04249, 2024

    2024 arXiv

  44. [53]

    Adversarial attacks on large language models using regularized relaxation,

    S. J. Chacko, S. Biswas, C. M. Islam, F. T. Liza, and X. Liu, “Adversarial attacks on large language models using regularized relaxation,”arXiv preprint arXiv:2410.19160, 2024

    2024 arXiv

  45. [2014]

    Available: https://arxiv.org/abs/1412.5154

    [Online]. Available: https://arxiv.org/abs/1412.5154

    arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.