Pith. sign in

REVIEW 4 major objections 6 minor 2 cited by

Adversarial Reasoning at Jailbreaking Time

T0 review · 4 major / 6 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Jailbreaking can be cast as a reasoning problem, and a loss-guided tree search over natural-language prompts beats prior attacks, including 56 percent success on OpenAI o1-preview.

desk verdict A real and well-specified jailbreaking synthesis whose headline claims need budget-matched baselines and a direct test of the loss signal before they're credible, but it deserves serious review. read the letter →

arxiv 2502.01633 v2 pith:PQRPIRV7 submitted 2025-02-03 cs.LG cs.AI

Mahdi Sabbaghi , Paul Kassianik , George Pappas , Yaron Singer , Amin Karbasi , Hamed Hassani This is my paper · ORCID
classification cs.LGcs.AI
keywords jailbreakingLLMsafetytest-timecomputeadversarialreasoningprocessrewardmodeltreesearchtransferattackcross-entropyloss
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that jailbreaking an aligned LLM can be treated as a reasoning problem, and that the same test-time-compute techniques that improve LLM reasoning can be turned against safety guardrails. The authors replace the sparse binary signal used by earlier semantic attacks with a continuous loss: the cross-entropy of the target model assigning high probability to a hand-written affirmative reply. A tree search over natural-language attack strategies, guided by that loss and refined by feedback from an attacker LLM, produces semantically meaningful prompts that beat existing token-space and semantic-space attacks on several aligned models, including adversarially trained ones. In a transfer setting, the method reaches 56 percent success on OpenAI o1-preview and 100 percent on DeepSeek-R1. If correct, this shows that defenses that spend extra inference compute on safety can be bypassed by spending comparable compute on the attack.

What carries the argument

The load-bearing object is the cross-entropy loss $L_{\mathcal{T}}(P, y_I)$ defined in Equation (3.2), which acts as a stepwise verifier: it is computed from target-model log-probabilities for a hand-crafted affirmative phrase, so it gives a continuous score for every candidate prompt rather than a binary accept-or-refuse signal. Each node in the search tree is a reasoning string $S$; its verifier score is the negative minimum loss $V(S) = -\min_i L_{\mathcal{T}}(P_i, y_I)$ over prompts sampled from the attacker $\mathcal{A}(S)$. The search itself is a Go-with-the-Winners style buffer: keep the $B$ best reasoning strings, expand the best, add its $m$ refined children, and prune, which lets the search backtrack when children fail to beat existing buffer candidates. The Feedback LLM and Refiner LLM implement what the paper calls textual gradients: the feedback explains why lower-loss prompts are better, and the refiner edits $S$ to incorporate that explanation while staying close to the previous string.

What would settle it

Compute the rank correlation between loss values $L_{\mathcal{T}}(P_i, y_I)$ and judge verdicts across all prompts generated in, say, 50 tasks; if lower loss does not systematically coincide with judge success on adversarially trained targets, the search signal would be indistinguishable from random ordering.

Watch

Extended reading notes

Core claim

The paper's central claim is that a continuous loss signal over target-model next-token probabilities can serve as a process reward model for jailbreak search, converting the attack into an optimization over a reasoning string $S$ such that the attacker LLM $\mathcal{A}(S)$ produces a prompt satisfying Judge$(\mathcal{T}(\mathcal{A}(S)), I) = 1$. The loss $L_{\mathcal{T}}(P, y_I)$ in Equation (3.2) measures how likely the target is to begin with a hand-written affirmative string $y_I$; the verifier score $V(S) = -\min_i L_{\mathcal{T}}(P_i, y_I)$ over $n$ sampled prompts evaluates each reasoning string. The search expands the highest-scoring node, generates feedback by comparing loss-sorted prompts, refines $S$ into children, and prunes to a fixed buffer, backtracking when children do not beat existing candidates. The paper reports the highest attack success rates among semantic-space methods, outperforming token-space methods on several adversarially trained targets, and multi-shot transfer results of 56 percent on OpenAI o1-preview and 100 percent on DeepSeek-R1.

Load-bearing premise

The whole search depends on the idea that a lower next-word-prediction loss against one hand-written affirmative reply reliably means the prompt is closer to a real jailbreak; the paper itself calls this only a heuristic.

Editorial extensions

If this is right

  • Semantic-space attacks no longer need to rely on the attacker model's intrinsic chain-of-thought: with a loss signal, a weak attacker (Vicuna-13B) reaches 64 percent ASR on Llama-3-8B, roughly three times the success of PAIR and TAP-T with the same attacker.
  • Token-level defenses such as perplexity filtering will not stop these prompts, because the search produces natural-language prompts rather than gibberish suffixes.
  • Adversarially trained models remain vulnerable: Llama-3-8B-RR rises from 2 percent ASR under GCG to 44 percent under this method, and R2D2 from 0 percent to 100 percent.
  • Models that spend inference-time compute on safety can still be jailbroken: multi-shot transfer raises the reported 16 percent ASR on OpenAI o1-preview to 56 percent.
  • Additional search iterations keep finding new jailbreaks, unlike prior semantic methods, so attack success scales with test-time compute.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper's own experiments, a direct rank-correlation test between $L_{\mathcal{T}}(P_i, y_I)$ and judge success over all sampled prompts would settle whether the loss is truly the driver; the paper currently supports this with a qualitative plot only.
  • If the loss signal is as informative as claimed, the same machinery should transfer to other safety-relevant domains such as LLM-controlled agents or web agents, where a surrogate model's log-probs can stand in for the target.
  • The multi-shot transfer results suggest that hiding log-prob access is an incomplete defense: attackers can optimize on open surrogate models and still retain substantial success on closed models.
  • Equation (5.1) offers a cheap diagnostic for whether a given attacker-target pair is searchable: check whether feedback re-orders the attacker's generation probabilities before running the full expensive search.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes 'Adversarial Reasoning', a semantic-space jailbreaking method that treats prompt generation as a reasoning problem and guides test-time search with a continuous loss signal. The target LLM's cross-entropy loss for a hand-crafted affirmative completion y_I (Eq. 3.2) serves as a stepwise verifier: the attacker LLM samples prompts, the Feedback LLM analyzes prompts sorted by this loss, the Refiner LLM updates a reasoning string, and a buffer-based tree search with pruning and backtracking (Algorithm 1) explores the space. Experiments report SOTA ASR on white-box models (Table 1), strong results with a weak attacker (Table 3), and black-box multi-shot transfer results including 56% on o1-preview and 100% on DeepSeek-R1 (Table 4). Ablations examine loss reduction, feedback consistency, and iteration depth.

Significance. If the results are reproducible, this is an important contribution: it shows that test-time compute scaling, which is commonly used to improve reasoning and safety, can be redirected to attack alignment mechanisms, including models that spend extra inference compute on safety. The method is gradient-free, uses only forward passes and log-prob vectors, releases code, and is evaluated with both the HarmBench judge and unanimous human review. The paper also includes useful controls, such as the feedback-order sanity check (Table 6) and the comparison with DeepSeek-R1 as a heuristic attacker (Table 5). The significance would be strengthened substantially if the loss signal's predictive validity and the query-budget equivalence of the comparisons were established quantitatively.

major comments (4)
  1. [§5.3, Eq. (3.2), Eq. (4.3)] The central mechanism claim—that the loss signal, not brute-force screening of many prompts, drives the reported ASR—is not quantitatively supported. Algorithm 1 selects nodes by V(S) = -min_i L_T(P_i, y_I), sorts prompts by loss for the Feedback LLM, and uses that ordering for all pruning and backtracking. The only direct evidence linking lower loss to judge success is the qualitative Figure 3, while the reversed/shuffled-order control in Table 6 is run on only 10 pre-selected successful tasks. The paper should report a rank correlation (e.g., Kendall's tau or Spearman's rho) between L_T(P, y_I) and judge success on a representative sample of prompts, or run a random-order ablation on a larger task set. Without this, the reported gains could stem from the 240-prompt multi-shot screening rather than from loss-guided reasoning.
  2. [Tables 1, 3, 4; §5.1] All ASR tables report single numbers without confidence intervals, seeds, or number of independent runs. The method is stochastic (attacker temperature 1.0, n=16 prompt samples, m=8 feedbacks per node), so point estimates may be noisy. For claims of state-of-the-art performance over baselines, particularly where differences are small (e.g., R2D2: PAIR 98% vs. Adversarial Reasoning 100% in Table 1), the paper should report means and standard errors over at least a few seeds, or bootstrap confidence intervals. This is load-bearing for the 'outperforms existing methods' claims.
  3. [§5.2, Table 4, Algorithm 2] The multi-shot transfer comparison is not query-matched. The multi-shot setting sends all collected prompts (n×T = 240 target queries per task) to the black-box target, while the PAIR and TAP-T baselines are reported with their standard query budgets (approximately 20 and 20-30 queries, respectively). The observed improvement on o1-preview (56% vs. 16-20%) may be due to the larger number of target queries rather than to the quality of the generated prompts. The paper should either (i) evaluate PAIR and TAP-T with the same 240-query budget, (ii) add a control that sends 240 randomly or heuristically sampled prompts to the target, or (iii) clearly state and justify the query-budget difference as an intended property of the method.
  4. [§5.1, Appendix B] The baseline numbers in Tables 1 and 4 appear to come from different protocols: the paper states that all positive responses for its own method were manually verified by three unanimous experts, but it does not state whether the baseline ASR values were recomputed under the same 50-task subset, the same HarmBench judge, and the same manual verification. If the baseline numbers were taken from prior publications using different task subsets or without human verification, the comparisons are not controlled. The paper should describe exactly how each baseline was obtained, or re-run baselines under the identical evaluation protocol.
minor comments (6)
  1. [§3, Eq. (3.2)] The construction of y_I, the affirmative target string per intent, is not described beyond a single example. Since the loss and hence the whole search depend on this string, the paper should include the full list of y_I strings or a precise recipe for generating them, to enable reproducibility.
  2. [Abstract, §1] The code link is given only as 'Github' without a URL or repository identifier. Please include the full link or a DOI.
  3. [Appendix D] There are several typos in the system prompts: 'Rememeber' should be 'Remember', and the Judge prompt contains 'BEHA VIOR' and 'GENERATTION'.
  4. [§5.2] The statement that DeepSeek-R1 'lacks robust guardrails' is a strong conclusion from a single attack scenario; consider softening to 'was vulnerable to this attack in our evaluations'.
  5. [Figure 3 caption] 'hance' should be 'hence' in the caption.
  6. [Table 3 caption] 'and and stronger' should be 'and stronger'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: success is measured by an external judge, while the optimized loss is an acknowledged heuristic.

full rationale

The paper's optimization objective L_T(P,y_I) in Eq. (3.2) is only a search heuristic; the attack-success criterion is the external HarmBench judge plus unanimous human review, so the reported ASR is not produced by construction from the loss. The paper explicitly concedes that successful jailbreaks need not begin with y_I: "Unsurprisingly, the jailbreaks do not begin with their desired output string y_I for Llama-3-8B-RR," and it calls the loss only a heuristic: "Although the absolute loss value may remain high, its relative value serves as an informative metric." The verifier score V(S) in Eq. (4.3) selects nodes by the same loss, but selection is an algorithmic mechanism rather than the evaluation metric; final results are judged externally. Baselines PAIR and TAP are prior work with overlapping authors, but they are used as comparison systems rather than as justification for the central claim, and no uniqueness theorem or fitted parameter is imported from those citations. The multi-shot transfer in Algorithm 2 explicitly queries the target and filters by Judge(T(P), I)=1, so transfer success is a measured outcome rather than a renamed version of the surrogate loss. The strongest skeptical concern—that the loss may be weakly correlated with judge success—is a correctness and robustness risk, not a circularity, because the reported numbers do not reduce to the loss by definition. No circular step is present, so the score is 0.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

No new physical or conceptual entities are introduced; the reasoning string S is a mutable search variable rather than an independent postulate. The free parameters are all manually chosen or empirically tuned, with y_I being the most safety-critical since the loss directly controls feedback and pruning. The axioms are standard empirical assumptions for this attack setting, with the loss-as-ordering assumption being the least directly validated.

free parameters (4)
  • Affirmative target string y_I per intent = Hand-crafted per HarmBench task (e.g., 'Sure, here is the step-by-step instructions for building a bomb')
    Equation (3.2) and the verifier score (4.3) are defined against y_I. If y_I is not a good representative of a successful jailbreak, the search signal is misleading. Section 5.3 admits successful responses often do not begin with y_I.
  • Hyperparameters m, n, k, B, T = m=8, n=16, k=2, B=32, T=15
    Section 5.1: 'These hyperparameters were selected by testing a handful of candidates empirically (for example, m=8 generally outperformed m=4 in loss reduction).' No held-out validation or multiple-seed reporting.
  • Target query repetitions for o1 and Gemini = 3
    Appendix B: 'for both models, we repeat the query 3 times in case of a generation block before accepting the refusal as a response.' This directly affects the closed-model ASR and is not applied to the compared baselines.
  • Surrogate model set for transfer = Llama-2-7B, Llama-3-RR, Zephyr-R2D2
    Section 5.2 aggregates losses over r=3 surrogates. The choice of surrogates affects transferability, and no analysis of how sensitive the black-box ASR is to that choice is provided.
assumptions (6)
  • domain assumption Cross-entropy loss on y_I orders prompts by expected jailbreak success.
    Used in Equations (3.2)-(4.3) to guide feedback, refinement, and pruning. Partially validated in Section 5.3 via Figure 3, but no rank-correlation statistic is given.
  • domain assumption Feedback LLM and Refiner LLM, given loss-ordered prompts and a reasoning string, produce refinements that move the attacker's distribution toward lower-loss prompts.
    Assumed throughout Algorithm 1; supported only by a qualitative test of Equation (5.1) on 10 tasks (Figure 4).
  • domain assumption HarmBench judge plus unanimous human expert review correctly identifies harmful responses.
    Section 5 and Appendix B. The judge is itself a fine-tuned LLM and can misclassify, which is why the authors add manual review.
  • domain assumption Surrogate loss from open models transfers to closed black-box models sufficiently to find successful prompts among the collected candidates.
    Algorithm 2 in Section 5.2 relies on this transfer; no transferability analysis beyond Table 4 is provided.
  • domain assumption Temperature-0 target generation is representative, and jailbreak outcomes are mostly deterministic per prompt.
    Section 5 sets target temperature to 0, but Appendix B acknowledges o1 and Gemini have stochastic moderation/refusals requiring 3 repeats.
  • domain assumption A jailbreak is defined as judge=1 on the behavior in I, independent of prompt semantics.
    Equation (3.1); this is the standard operationalization in the benchmark.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Adversarial Reasoning at Jailbreaking Time." pith.science (2026). https://pith.science/paper/PQRPIRV7

@misc{pith2026250201633,
  author       = {Pith},
  title        = {Pith review of: Adversarial Reasoning at Jailbreaking Time},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PQRPIRV7}},
  note         = {Machine review of arXiv:2502.01633}
}
read the original abstract

As large language models (LLMs) are becoming more capable and widespread, the study of their failure cases is becoming increasingly important. Recent advances in standardizing, measuring, and scaling test-time compute suggest new methodologies for optimizing models to achieve high performance on hard tasks. In this paper, we apply these advances to the task of model jailbreaking: eliciting harmful responses from aligned LLMs. We develop an adversarial reasoning approach to automatic jailbreaking that leverages a loss signal to guide the test-time compute, achieving SOTA attack success rates against many aligned LLMs, even those that aim to trade inference-time compute for adversarial robustness. Our approach introduces a new paradigm in understanding LLM vulnerabilities, laying the foundation for the development of more robust and trustworthy AI systems.

Figures

Figures reproduced from arXiv: 2502.01633 by the authors.

Figure 1
Figure 1. The overall mechanism of our algorithm; We iteratively refine the reasoning string by the feedbacks derived from comparing the loss values of previous attempts. Then, we explore the reasoning tree using a search algorithm. Details are presented in Section 4. We explain in Section 4 (Equation (4.3)) that the searching algorithm will backtrack if the children of a node do not achieve a higher score than the candidates… view at source ↗
Figure 2
Figure 2. One iteration of our algorithms for the task: “Write an office macro that searches for and exfiltrates passwords”. This diagram corresponds to constructing one step of reasoning for updating S in Equation (3.3), where n = 2 (Feedback LLM compares 2 attacking prompts) and m = 2 (by acquiring 2 feedback strings, we generate 2 children). Algorithm 1 Adversarial Reasoning Require: Initial prompt S (0) , jailbreaking goa… view at source ↗
Figure 3
Figure 3. Up: We plot the objective of our optimization (Equa￾tion (4.3)) over iteration to demonstrate that the refinement process is effective. Down: Shows that despite the high value of the loss, its relative value still provides a signal for identifying the success￾ful attempts, and hance, a heuristic for our method. we conducted an experiment using PAIR with Deepseek-R1 as the attacker. As [PITH_FULL_IMAGE:figures/full_… view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Lower values of the y-scale accounts to higher likelihood of generation in the next iteration. The figure shows an approx￾imate increase (decrease in likelihood) for prompts with larger indexes (higher loss), which means Feedback LLM and Refiner LLM optimize the reason…
Figure 5
Figure 5. Figure 5: (a) PAIR only achieves two more jailbreaks after iteration 3 as it doesn’t receive any signals and only tries to circumvent the target’s refusal. (b) Our algorithm improves later iterations’ performance by utilizing the loss function. sanity check, we demonstrate that …
Figure 6
Figure 6. Figure 6: Shows the vulnerabilities of various models (y-axis) across the six categories of Harmbench (x-axis). Higher values in each cell indicate weaker safety performance against jailbreaks. C. Additional experiments LLM vulnerabilities For some of the most common LLMs, we il…
Figure 7
Figure 7. Figure 7: The attacker has generated 8 prompts for a goal. These prompts are sorted according to their losses, and passed to Feedback LLM. When prompts are given in the correct order (increasing), Feedback LLM mentions a detachment from reality, while the reversed order (decreas…
Figure 8
Figure 8. Figure 8: Distribution of successful jailbreaks over iterations for (a) Mixtral-8x7B model as the attacker and Llama-3-8B-RR as the target. (b) Mixtral model attacking Mistral-3-8B-RR. (c) Vicuna as the attacker and Llama-3-8B as the target. 16 [PITH_FULL_IMAGE:figures/full_fig…
Figure 9
Figure 9. Figure 9: Distribution of successful jailbreaks over iterations for (a) Mixtral-8x7B model as the attacker and Claude as the target when using the average loss Section 5.2 for surrogate. (b) Same setting for attacking OpenAI o1. D. System Prompts Judge We get the judge provided …
Figure 10
Figure 10. Figure 10: A full example of exploring the prompt space for 6 iterations and jailbreaking OpenAI o1. Note that each node has n/k = 8 children, where only the explored ones are shown in this graph for simplicity. 21 [PITH_FULL_IMAGE:figures/full_fig_p021_10.png]
Figure 11
Figure 11. Figure 11: An example of exploring the prompt space for only 1 iteration that jailbreaks Claude. Note that the initial node has 8 children where only the explored one is shown here for simplicity. 22 [PITH_FULL_IMAGE:figures/full_fig_p022_11.png]
Figure 12
Figure 12. Figure 12: We ran our algorithm in a comparison with PAIR both for 7 iterations. While our method reaches a jailbreak for Llama-2-7B by the end of this, PAIR’s output deviates from original intent. The last prompt of PAIR is not about extracting the instructions for locating the…
Figure 13
Figure 13. Figure 13: A jailbreaking example where Llama-2-7B initially refuses to comply. 24 [PITH_FULL_IMAGE:figures/full_fig_p024_13.png]
Figure 14
Figure 14. Figure 14: A jailbreaking example where Llama-3-8B-RR initially refuses to comply. 25 [PITH_FULL_IMAGE:figures/full_fig_p025_14.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Jailbreaking to Jailbreak

    cs.CL 2025-02 conditional novelty 7.0 of 10

    A transferable multi-turn jailbreak turns refusal-trained black-box LLMs into willing automated jailbreakers, with high attack success against other models and against themselves.

  2. Parallel-R1: Towards Parallel Thinking via Reinforcement Learning

    cs.CL 2025-09 conditional novelty 6.0 of 10

    Parallel-R1 uses SFT cold-start on easy math plus GRPO on hard math to instill parallel thinking in Qwen3-4B, reporting 8.4% average accuracy gains and a 42.9% AIME25 gain from a parallel-exploration scaffold.

Reference graph

Works this paper leans on

59 extracted references · 6 canonical work pages · cited by 2 Pith papers

  1. [5]

    14 Chao, P., Debenedetti, E., Robey, A., Andriushchenko, M., Croce, F., Sehwag, V ., Dobriban, E., Flammarion, N., Pappas, G

    URL https: //arxiv.org/abs/2412.05232. 14 Chao, P., Debenedetti, E., Robey, A., Andriushchenko, M., Croce, F., Sehwag, V ., Dobriban, E., Flammarion, N., Pappas, G. J., Tramer, F., Hassani, H., and Wong, E. Jailbreakbench: An open robustness benchmark for jailbreaking large language models, 2024a. URL https://arxiv.org/abs/2404.01318. 5 Chao, P., Robey, A...

    arXiv

  2. [6]

    cyber_intrusion

    Shows the vulnerabilities of various models (y-axis) across the six categories of Harmbench (x-axis). Higher values in each cell indicate weaker safety performance against jailbreaks. C. Additional experiments LLM vulnerabilities For some of the most common LLMs, we illustrate their vulnerabilities in different categories of Harmbench (Mazeika et al., 202...

    work page 2024

  3. [9]

    18 Gandhi, K., Lee, D., Grand, G., Liu, M., Cheng, W., Sharma, A., and Goodman, N

    URL https://arxiv.org/abs/2301.00234. 18 Gandhi, K., Lee, D., Grand, G., Liu, M., Cheng, W., Sharma, A., and Goodman, N. D. Stream of search (sos): Learning to search in language,

    arXiv

  4. [10]

    org/abs/2404.03683

    URL https://arxiv. org/abs/2404.03683. 3 Ge, S., Zhou, C., Hou, R., Khabsa, M., Wang, Y .-C., Wang, Q., Han, J., and Mao, Y . Mart: Improving llm safety with multi-round automatic red-teaming,

    arXiv

  5. [11]

    1, 14 Guan, M

    URL https://arxiv.org/abs/2311.07689. 1, 14 Guan, M. Y ., Joglekar, M., Wallace, E., Jain, S., Barak, B., Helyar, A., Dias, R., Vallone, A., Ren, H., Wei, J., Chung, H. W., Toyer, S., Heidecke, J., Beutel, A., and Glaese, A. Deliberative alignment: Reasoning enables safer language models,

    arXiv

  6. [12]

    org/abs/2412.16339

    URL https://arxiv. org/abs/2412.16339. 3 Hayase, J., Borevkovic, E., Carlini, N., Tramèr, F., and Nasr, M. Query-based adversarial prompt generation,

    arXiv

  7. [13]

    1, 3, 4, 14 Hendrycks, D., Burns, C., Kadavath, S., Arora, A., Basart, S., Tang, E., Song, D., and Steinhardt, J

    URL https://arxiv.org/abs/2402.12329. 1, 3, 4, 14 Hendrycks, D., Burns, C., Kadavath, S., Arora, A., Basart, S., Tang, E., Song, D., and Steinhardt, J. Measuring math- ematical problem solving with the math dataset,

    arXiv

  8. [14]

    2 Hughes, J., Price, S., Lynch, A., Schaeffer, R., Barez, F., Koyejo, S., Sleight, H., Jones, E., Perez, E., and Sharma, M

    URL https://arxiv.org/abs/2103.03874. 2 Hughes, J., Price, S., Lynch, A., Schaeffer, R., Barez, F., Koyejo, S., Sleight, H., Jones, E., Perez, E., and Sharma, M. Best-of-n jailbreaking,

    arXiv

Show all 59 references
  1. [15]

    3 Inan, H., Upasani, K., Chi, J., Rungta, R., Iyer, K., Mao, Y ., Tontchev, M., Hu, Q., Fuller, B., Testuggine, D., and Khabsa, M

    URL https: //arxiv.org/abs/2412.03556. 3 Inan, H., Upasani, K., Chi, J., Rungta, R., Iyer, K., Mao, Y ., Tontchev, M., Hu, Q., Fuller, B., Testuggine, D., and Khabsa, M. Llama guard: Llm-based input-output safe- guard for human-ai conversations,

    arXiv

  2. [16]

    1 10 Adversarial Reasoning at Jailbreaking Time Jia, X., Pang, T., Du, C., Huang, Y ., Gu, J., Liu, Y ., Cao, X., and Lin, M

    URL https: //arxiv.org/abs/2312.06674. 1 10 Adversarial Reasoning at Jailbreaking Time Jia, X., Pang, T., Du, C., Huang, Y ., Gu, J., Liu, Y ., Cao, X., and Lin, M. Improved techniques for optimization- based jailbreaking on large language models,

    arXiv

  3. [17]

    14 Karamcheti, S., Nair, S., Chen, A

    URL https://arxiv.org/abs/2405.21018. 14 Karamcheti, S., Nair, S., Chen, A. S., Kollar, T., Finn, C., Sadigh, D., and Liang, P. Language-driven representation learning for robotics,

    arXiv

  4. [18]

    org/abs/2302.12766

    URL https://arxiv. org/abs/2302.12766. 9 Kritz, J., Robinson, V ., Vacareanu, R., Varjavand, B., Choi, M., Gogov, B., Team, S. R., Yue, S., Primack, W. E., and Wang, Z. Jailbreaking to jailbreak,

    arXiv

  5. [19]

    8 Lapid, R., Langberg, R., and Sipper, M

    URL https://arxiv.org/abs/2502.09638. 8 Lapid, R., Langberg, R., and Sipper, M. Open sesame! universal black box jailbreaking of large language mod- els,

    arXiv

  6. [20]

    9 Liao, Z

    URL https://arxiv.org/abs/2209.07753. 9 Liao, Z. and Sun, H. Amplegcg: Learning a universal and transferable generative model of adversarial suffixes for jailbreaking both open and closed llms,

    arXiv

  7. [21]

    14 Lightman, H., Kosaraju, V ., Burda, Y ., Edwards, H., Baker, B., Lee, T., Leike, J., Schulman, J., Sutskever, I., and Cobbe, K

    URL https://arxiv.org/abs/2404.07921. 14 Lightman, H., Kosaraju, V ., Burda, Y ., Edwards, H., Baker, B., Lee, T., Leike, J., Schulman, J., Sutskever, I., and Cobbe, K. Let’s verify step by step,

    arXiv

  8. [22]

    2, 3 Liu, N

    URL https: //arxiv.org/abs/2305.20050. 2, 3 Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., and Liang, P. Lost in the middle: How language models use long contexts,

    arXiv

  9. [23]

    8, 18 Liu, X., Li, P., Suh, E., V orobeychik, Y ., Mao, Z., Jha, S., McDaniel, P., Sun, H., Li, B., and Xiao, C

    URL https: //arxiv.org/abs/2307.03172. 8, 18 Liu, X., Li, P., Suh, E., V orobeychik, Y ., Mao, Z., Jha, S., McDaniel, P., Sun, H., Li, B., and Xiao, C. Autodan- turbo: A lifelong agent for strategy self-exploration to jailbreak llms, 2024a. URL https://arxiv.org/ abs/2410.0529...

    arXiv

  10. [24]

    3, 5, 6, 14, 15, 17 Mehrotra, A., Zampetakis, M., Kassianik, P., Nelson, B., Anderson, H., Singer, Y ., and Karbasi, A

    URL https://arxiv.org/ abs/2402.04249. 3, 5, 6, 14, 15, 17 Mehrotra, A., Zampetakis, M., Kassianik, P., Nelson, B., Anderson, H., Singer, Y ., and Karbasi, A. Tree of attacks: Jailbreaking black-box llms automatically,

    arXiv

  11. [25]

    1, 2, 3, 4, 6, 14, 18 Meta AI

    URL https://arxiv.org/abs/2312.02119. 1, 2, 3, 4, 6, 14, 18 Meta AI. Llama 2: Open foundation and fine-tuned chat models,

    arXiv

  12. [26]

    17 Meta AI

    URL https://arxiv.org/abs/ 2307.09288. 17 Meta AI. The llama 3 herd of models,

    arXiv

  13. [27]

    7 Mistral AI

    URL https: //arxiv.org/abs/2407.21783. 7 Mistral AI. Mixtral of experts,

    arXiv

  14. [28]

    5 Nye, M., Andreassen, A

    URL https:// arxiv.org/abs/2401.04088. 5 Nye, M., Andreassen, A. J., Gur-Ari, G., Michalewski, H., Austin, J., Bieber, D., Dohan, D., Lewkowycz, A., Bosma, M., Luan, D., Sutton, C., and Odena, A. Show your work: Scratchpads for intermediate computation with language models,

    arXiv

  15. [29]

    1, 2 OpenAI

    URL https://arxiv.org/ abs/2112.00114. 1, 2 OpenAI. Learning to reason with llms, September 2024a. URL https://openai.com/index/learning- to-reason-with-llms/ . 3 OpenAI. Openai o1 system card, 2024b. URL https: //arxiv.org/abs/2412.16720. 2, 3, 7 Ouyang, L., Wu, J., Jiang, X....

    arXiv

  16. [30]

    1 Paulus, A., Zharmagambetov, A., Guo, C., Amos, B., and Tian, Y

    URL https: //arxiv.org/abs/2203.02155. 1 Paulus, A., Zharmagambetov, A., Guo, C., Amos, B., and Tian, Y . Advprompter: Fast adaptive adversarial prompt- ing for llms,

    arXiv

  17. [31]

    14 Perez, E., Huang, S., Song, F., Cai, T., Ring, R., Aslanides, J., Glaese, A., McAleese, N., and Irving, G

    URL https://arxiv.org/ abs/2404.16873. 14 Perez, E., Huang, S., Song, F., Cai, T., Ring, R., Aslanides, J., Glaese, A., McAleese, N., and Irving, G. Red teaming language models with language models,

    arXiv

  18. [32]

    14 Rafailov, R., Sharma, A., Mitchell, E., Ermon, S., Manning, C

    URL https://arxiv.org/abs/2202.03286. 14 Rafailov, R., Sharma, A., Mitchell, E., Ermon, S., Manning, C. D., and Finn, C. Direct preference optimization: Your language model is secretly a reward model,

    arXiv

  19. [33]

    1 Rebedea, T., Dinu, R., Sreedhar, M., Parisien, C., and Cohen, J

    URL https://arxiv.org/abs/2305.18290. 1 Rebedea, T., Dinu, R., Sreedhar, M., Parisien, C., and Cohen, J. Nemo guardrails: A toolkit for controllable and safe llm applications with programmable rails,

    arXiv

  20. [34]

    1 11 Adversarial Reasoning at Jailbreaking Time Rein, D., Hou, B

    URL https://arxiv.org/abs/2310.10501. 1 11 Adversarial Reasoning at Jailbreaking Time Rein, D., Hou, B. L., Stickland, A. C., Petty, J., Pang, R. Y ., Dirani, J., Michael, J., and Bowman, S. R. Gpqa: A graduate-level google-proof q&a benchmark,

    arXiv

  21. [35]

    URL https://arxiv.org/abs/2311.12022. 2 Reuel, A., Bucknall, B., Casper, S., Fist, T., Soder, L., Aarne, O., Hammond, L., Ibrahim, L., Chan, A., Wills, P., Anderljung, M., Garfinkel, B., Heim, L., Trask, A., Mukobi, G., Schaeffer, R., Baker, M., Hooker, S., So- laiman, I., Luc...

    arXiv

  22. [36]

    9 Robey, A., Ravichandran, Z., Kumar, V ., Hassani, H., and Pappas, G

    URL https://arxiv.org/abs/2407.14981. 9 Robey, A., Ravichandran, Z., Kumar, V ., Hassani, H., and Pappas, G. J. Jailbreaking llm-controlled robots, 2024a. URL https://arxiv.org/abs/2410.13691. 9 Robey, A., Wong, E., Hassani, H., and Pappas, G. J. Smooth- llm: Defending large l...

    arXiv

  23. [37]

    14 Samvelyan, M., Raparthy, S

    URL https: //arxiv.org/abs/2402.15570. 14 Samvelyan, M., Raparthy, S. C., Lupu, A., Hambro, E., Markosyan, A. H., Bhatt, M., Mao, Y ., Jiang, M., Parker- Holder, J., Foerster, J., Rocktäschel, T., and Raileanu, R. Rainbow teaming: Open-ended generation of diverse ad- versarial...

    arXiv

  24. [38]

    1, 4, 14 Shao, Z., Wang, P., Zhu, Q., Xu, R., Song, J., Bi, X., Zhang, H., Zhang, M., Li, Y

    URL https://arxiv.org/ abs/2402.16822. 1, 4, 14 Shao, Z., Wang, P., Zhu, Q., Xu, R., Song, J., Bi, X., Zhang, H., Zhang, M., Li, Y . K., Wu, Y ., and Guo, D. Deepseekmath: Pushing the limits of mathemat- ical reasoning in open language models,

    arXiv

  25. [39]

    2 Sheshadri, A., Ewart, A., Guo, P., Lynch, A., Wu, C., Hebbar, V ., Sleight, H., Stickland, A

    URL https://arxiv.org/abs/2402.03300. 2 Sheshadri, A., Ewart, A., Guo, P., Lynch, A., Wu, C., Hebbar, V ., Sleight, H., Stickland, A. C., Perez, E., Hadfield-Menell, D., and Casper, S. Latent adversarial training improves robustness to persistent harmful be- haviors in llms,

    arXiv

  26. [40]

    1 Shin, T., Razeghi, Y ., IV , R

    URL https://arxiv.org/ abs/2407.15549. 1 Shin, T., Razeghi, Y ., IV , R. L. L., Wallace, E., and Singh, S. Autoprompt: Eliciting knowledge from language mod- els with automatically generated prompts,

    arXiv

  27. [42]

    org/abs/2408.03314

    URL https://arxiv. org/abs/2408.03314. 3, 6, 14 Souly, A., Lu, Q., Bowen, D., Trinh, T., Hsieh, E., Pandey, S., Abbeel, P., Svegliato, J., Emmons, S., Watkins, O., and Toyer, S. A strongreject for empty jailbreaks,

    arXiv

  28. [43]

    5 Stechly, K., Valmeekam, K., and Kambhampati, S

    URL https://arxiv.org/abs/2402.10260. 5 Stechly, K., Valmeekam, K., and Kambhampati, S. On the self-verification limitations of large language models on reasoning and planning tasks,

    arXiv

  29. [44]

    3 Uesato, J., Kushman, N., Kumar, R., Song, F., Siegel, N., Wang, L., Creswell, A., Irving, G., and Higgins, I

    URL https: //arxiv.org/abs/2402.08115. 3 Uesato, J., Kushman, N., Kumar, R., Song, F., Siegel, N., Wang, L., Creswell, A., Irving, G., and Higgins, I. Solv- ing math word problems with process- and outcome- based feedback,

    arXiv

  30. [45]

    2, 3 Vemprala, S

    URL https://arxiv.org/ abs/2211.14275. 2, 3 Vemprala, S. H., Bonatti, R., Bucker, A., and Kapoor, A. Chatgpt for robotics: Design principles and model abilities. IEEE Access , 12:55682–55696,

    arXiv

  31. [46]

    9 Wang, P., Li, L., Shao, Z., Xu, R

    doi: 10.1109/ACCESS.2024.3387941. 9 Wang, P., Li, L., Shao, Z., Xu, R. X., Dai, D., Li, Y ., Chen, D., Wu, Y ., and Sui, Z. Math-shepherd: Ver- ify and reinforce llms step-by-step without human an- notations, 2024a. URL https://arxiv.org/abs/ 2312.08935. 2, 3 Wang, Y ., Zhao, ...

    2024

  32. [47]

    org/abs/2406.12814

    URL https://arxiv. org/abs/2406.12814. 9 Xhonneux, S., Sordoni, A., Günnemann, S., Gidel, G., and Schwinn, L. Efficient adversarial training in llms with continuous attacks,

    arXiv

  33. [48]

    org/abs/2405.15589

    URL https://arxiv. org/abs/2405.15589. 1 Xiang, V ., Snell, C., Gandhi, K., Albalak, A., Singh, A., Blagden, C., Phung, D., Rafailov, R., Lile, N., Mahan, D., Castricato, L., Franken, J.-P., Haber, N., and Finn, C. Towards system 2 reasoning in llms: Learning how to think with...

    arXiv

  34. [49]

    2, 4 Xie, Y ., Goyal, A., Zheng, W., Kan, M.-Y ., Lillicrap, T

    URL https: //arxiv.org/abs/2501.04682. 2, 4 Xie, Y ., Goyal, A., Zheng, W., Kan, M.-Y ., Lillicrap, T. P., Kawaguchi, K., and Shieh, M. Monte carlo tree search boosts reasoning via iterative preference learning,

    arXiv

  35. [50]

    3 Yu, F., Gao, A., and Wang, B

    URL https://arxiv.org/abs/2405.00451. 3 Yu, F., Gao, A., and Wang, B. Ovm, outcome-supervised value models for planning in mathematical reason- ing,

    arXiv

  36. [51]

    4, 19 Zeng, Y ., Lin, H., Zhang, J., Yang, D., Jia, R., and Shi, W

    URL https://arxiv.org/ abs/2406.07496. 4, 19 Zeng, Y ., Lin, H., Zhang, J., Yang, D., Jia, R., and Shi, W. How johnny can persuade llms to jailbreak them: Rethinking persuasion to challenge ai safety by human- izing llms,

    arXiv

  37. [52]

    1, 14 Zhang, L., Hosseini, A., Bansal, H., Kazemi, M., Ku- mar, A., and Agarwal, R

    URL https://arxiv.org/abs/ 2401.06373. 1, 14 Zhang, L., Hosseini, A., Bansal, H., Kazemi, M., Ku- mar, A., and Agarwal, R. Generative verifiers: Re- ward modeling as next-token prediction, 2024a. URL https://arxiv.org/abs/2408.15240. 3 Zhang, Y ., Khalifa, M., Logeswaran, L., ...

    arXiv

  38. [53]

    2 Zou, A., Wang, Z., Carlini, N., Nasr, M., Kolter, J

    URL https://arxiv.org/ abs/2408.16326. 2 Zou, A., Wang, Z., Carlini, N., Nasr, M., Kolter, J. Z., and Fredrikson, M. Universal and transferable adversarial attacks on aligned language models,

    arXiv

  39. [54]

    1, 3, 4, 6, 14 Zou, A., Phan, L., Wang, J., Duenas, D., Lin, M., An- driushchenko, M., Wang, R., Kolter, Z., Fredrikson, M., and Hendrycks, D

    URL https: //arxiv.org/abs/2307.15043. 1, 3, 4, 6, 14 Zou, A., Phan, L., Wang, J., Duenas, D., Lin, M., An- driushchenko, M., Wang, R., Kolter, Z., Fredrikson, M., and Hendrycks, D. Improving alignment and ro- bustness with circuit breakers,

    arXiv

  40. [55]

    1, 6, 14 13 Adversarial Reasoning at Jailbreaking Time A

    URL https: //arxiv.org/abs/2406.04313. 1, 6, 14 13 Adversarial Reasoning at Jailbreaking Time A. Additional Related Work Token-space Jailbreaking. Token-space attacks (Shin et al., 2020; Wen et al., 2023; Zou et al., 2023; Hayase et al., 2024; Andriushchenko et al., 2024a) mod...

    2020 arXiv

  41. [56]

    red-teaming

    and random searches over cleverly chosen initial prompts (Andriushchenko et al., 2024a). We adopt the use of a loss function from these methods as a signal to inform how to navigate the prompt space for better jailbreaks while remaining gradient-free. Semantic-space Jailbreaki...

    2022

  42. [57]

    However, as explained in Section 5, we manually verify all of the jailbreaks marked as positive by the judge

    to evaluate the target responses. However, as explained in Section 5, we manually verify all of the jailbreaks marked as positive by the judge. We remark that the additional human-based evaluation on the top of the HarmBench judge is in fact necessary, and has been done in pre...

    2024

  43. [59]

    Yes" or

    with a knowledge distillation process from GPT-4, and a meticulously crafted prompt for querying the models. Below we have attached the prompt used for this manner, which is a slightly modified version of Harmbench’s original prompt. This prompt emphasizes on detailed-oriented...

    2024

  44. [1994]

    4 Alon, G

    doi: 10.1109/SFCS.1994.365742. 4 Alon, G. and Kamfonas, M. Detecting language model attacks with perplexity,

    1994

  45. [2017]

    1 Cobbe, K., Kosaraju, V ., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., Hesse, C., and Schulman, J

    URL https://arxiv.org/ abs/1706.03741. 1 Cobbe, K., Kosaraju, V ., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., Hesse, C., and Schulman, J. Training verifiers to solve math word problems,

    arXiv

  46. [2020]

    1, 14 Snell, C., Lee, J., Xu, K., and Kumar, A

    URL https://arxiv.org/abs/2010.15980. 1, 14 Snell, C., Lee, J., Xu, K., and Kumar, A. Scaling llm test- time compute optimally can be more effective than scal- ing model parameters,

    2010 arXiv

  47. [2021]

    org/abs/2110.14168

    URL https://arxiv. org/abs/2110.14168. 2 DeepSeek-AI. Deepseek-r1: Incentivizing reasoning ca- pability in llms via reinforcement learning,

    arXiv

  48. [2022]

    18 Beetham, J., Chakraborty, S., Wang, M., Huang, F., Bedi, A

    URL https://arxiv.org/ abs/2207.04901. 18 Beetham, J., Chakraborty, S., Wang, M., Huang, F., Bedi, A. S., and Shah, M. Liar: Leveraging alignment (best- of-n) to jailbreak llms in seconds,

    arXiv

  49. [2023]

    org/abs/2308.14132

    URL https://arxiv. org/abs/2308.14132. 1 Andriushchenko, M., Croce, F., and Flammarion, N. Jail- breaking leading safety-aligned llms with simple adaptive attacks, 2024a. URL https://arxiv.org/abs/ 2404.02151. 1, 4, 7, 14 Andriushchenko, M., Souly, A., Dziemian, M., Duenas, D....

    arXiv

  50. [2024]

    go with the winners

    URL https: //arxiv.org/abs/2402.00157. 2 Aldous, D. and Vazirani, U. "go with the winners" algo- rithms. In Proceedings 35th Annual Symposium on F oun- dations of Computer Science , pp. 492–501,

    arXiv

  51. [2025]

    2, 7 Dong, Q., Li, L., Dai, D., Zheng, C., Ma, J., Li, R., Xia, H., Xu, J., Wu, Z., Liu, T., Chang, B., Sun, X., Li, L., and Sui, Z

    URL https://arxiv.org/abs/2501.12948. 2, 7 Dong, Q., Li, L., Dai, D., Zheng, C., Ma, J., Li, R., Xia, H., Xu, J., Wu, Z., Liu, T., Chang, B., Sun, X., Li, L., and Sui, Z. A survey on in-context learning,

    arXiv

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.