Pith. sign in

REVIEW 6 major objections 6 minor 1 cited by

GuardVal: Dynamic Large Language Model Jailbreak Evaluation for Comprehensive Safety Testing

T0 review · 6 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A dynamic jailbreak protocol that adapts prompts to each defender exposes weaknesses one-shot benchmarks miss.

desk verdict A worthwhile dynamic evaluation idea whose reported numbers aren't yet trustworthy because the jailbreak success criterion is never defined. read the letter →

arxiv 2507.07735 v1 pith:2K7HQGHD submitted 2025-07-10 cs.LG cs.CLcs.CR

Peiyan Zhang , Haibo Jin , Liying Kang , Haohan Wang This is my paper
classification cs.LGcs.CLcs.CR
keywords jailbreakevaluationLLMsafetydynamicbenchmarkingadversarialpromptingredteamingpromptoptimizationOverallValuerole-playinggeneration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that static jailbreak benchmarks can create false safety: models may refuse familiar first-round prompts and then be broken once the prompts adapt to their actual responses. GuardVal addresses this by running an attacker LLM through Translator, Generator, Evaluator, and Optimizer roles, where the Optimizer detects stagnation with Adam-style moment estimates and injects natural-language steering into the next round. Each model is tested both as defender and as attacker, and the new Overall Safety Value (OSV) ranks models by average rounds survived minus average rounds needed to break others, normalizing test-set difficulty. Applied to eight LLMs across ten safety domains, the protocol finds first-round refusal rates overstate safety, rankings shift by domain, and static-benchmark rankings disagree with dynamic ones. The paper also shows the resulting per-domain defense tables can directly select a mix-of-experts ensemble.

What carries the argument

The load-bearing mechanism is the four-role iterative loop. The attacker LLM acts as Translator, Generator, and Evaluator; the Evaluator returns both a similarity score and modification advice, and the Optimizer detects when that advice stops producing change. The Optimizer defines the response change $g_t = \|\mathrm{Response}_t - \mathrm{Response}_{t-1}\|$, maintains first and second moment estimates $m_t$ and $v_t$ in the style of the Adam optimizer, and applies a sliding-window threshold on the mean and variance of these changes to decide whether the attacker is stagnating, mapping that decision into natural-language feedback templates. The loop's final output is the OSV equation above, making rounds-to-jailbreak the unit of measurement and the pairwise subtraction the normalizer that lets one model be compared against another on a common scale.

What would settle it

Take a sample of defender responses produced by GuardVal across domains, have human annotators label each as a refusal, a harmful completion, or off-topic, and compare those labels to the Evaluator's similarity-score verdict for the same response-oracle pairs; if the score cannot separate harmful completions from refusals at high agreement, the round counts in the domain tables and the OSV rankings built from them would not survive re-scoring.

Watch

Extended reading notes

Core claim

The paper's central claim is that an evaluation protocol should generate and refine jailbreak prompts dynamically, based on the defender LLM's state, rather than relying on fixed or template-manipulated test sets. GuardVal operationalizes this with a role-playing loop in which the attacker LLM translates safety guidelines into test questions, wraps them in complex scenarios, scores the defender's response by semantic similarity to a pre-written Oracle, and iteratively revises the scenario. A separate Optimizer layer tracks how the attacker's responses change across rounds and, when the change rate stalls, feeds refined suggestions back to the Generator. The output measure is $\mathrm{OSV}_A = \frac{1}{N-1}\sum_{B\neq A}(R_{B,A}-R_{A,B})$, where $R_{B,A}$ is the average number of rounds model $B$ needs to jailbreak $A$ and $R_{A,B}$ is the average number of rounds model $A$ needs to jailbreak $B$; higher OSV means better overall safety, balancing defense and offense while normalizing against the same set of peer models. On this measure, the paper reports domain-dependent rankings, pairwise restraint relations, and divergences from static-benchmark rankings.

Load-bearing premise

The protocol's round counts, and therefore every OSV ranking, assume the Evaluator's semantic-similarity score between the defender's response and the Oracle reliably separates a genuinely harmful answer from a refusal or off-topic answer; if that signal is wrong, the measured rounds-to-jailbreak do not mean what the tables claim.

Editorial extensions

If this is right

  • One-round evaluations overstate safety: in the reported tables most defenders survive the first round, and continued refinement is what exposes breaches, so any protocol that stops early will rank models as stronger than they are.
  • Safety is domain-dependent: OSV rankings differ across the ten domains, so an aggregate safety score should be accompanied by per-domain scores before deployment decisions are made.
  • Static-benchmark rankings can mislead: the OSV ranking disagrees with TrustLLM's ranking for several models, which the paper reads as evidence that fixed test sets may reward familiarity rather than true safety.
  • OSV is a relative comparison: as the paper notes in Appendix F, scores depend on the pool of models, so adding or removing participants changes the numbers and the metric should not be read as an absolute security guarantee.
  • The per-domain defense tables support a mix-of-experts deployment: the protocol identifies the strongest defender per domain, which can be used to route queries to the most resistant model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The Evaluator is itself an LLM, and the only external check in the paper is a 40-response human-labeled classification table; an audit of the full success/failure loop on a larger labeled sample would reveal how much of each round count is inherited from the judge's blind spots.
  • The Adam-inspired control layer maps statistics to natural-language feedback through hand-written templates; an ablation that feeds the same moment estimates as direct numeric steering would show whether the gains come from the statistics or from the wording.
  • The successful jailbreak prompts produced by the protocol are reusable assets: filtered and deduplicated, they could serve as training or red-teaming data for the defenders, turning the evaluation loop into an automated safety-improvement pipeline.
  • If the shallow-alignment explanation is right, models should stop refusing once explicit trigger words like 'rumor' and 'damage' are paraphrased away; that is a direct behavioral test of whether the discovered vulnerability is semantic rather than lexical.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

6 major / 6 minor

Summary. GuardVal is a dynamic jailbreak evaluation protocol in which an attacker LLM plays Translator, Generator, Evaluator, and Optimizer roles to iteratively refine jailbreak prompts against a defender LLM until a jailbreak is achieved. The central output is a matrix of average rounds-to-jailbreak for pairs of eight LLMs across ten safety domains, aggregated into an Overall Safety Value (OSV) defined in Eq. (6). The paper also proposes an Adam-inspired optimizer to avoid prompt-refinement stagnation and sketches a mix-of-experts strategy that uses per-domain defensive strengths to improve deployment safety. The empirical sections report OSV rankings, domain-specific round matrices, a small human-labeled evaluation of LLM judges, and two case studies illustrating iterative refinement and the optimizer's effect.

Significance. If its measurement chain were valid, GuardVal would be a useful complement to static jailbreak benchmarks: it targets dataset contamination, adapts prompts to the defended model, and jointly scores offense and defense in a way that partially normalizes test-set difficulty. The role-based protocol and the OSV ranking idea are conceptually attractive, and the mix-of-experts application is a sensible downstream use of per-domain defensive scores. However, the manuscript does not provide a reproducible definition of the central success signal, does not specify key components of the optimizer, and does not release code, logs, or the human-labeled validation set. As it stands, the empirical claims are not verifiable from the paper, and the reported rankings rest on an unvalidated and partly undefined evaluation criterion.

major comments (6)
  1. [Sec. 3.1, Appendix B] The jailbreak success criterion is never defined. Section 3.1 states that 'the evaluation is completed once the attacker LLM successfully jailbreaks the defender LLM,' but the paper never specifies what counts as success. The Evaluator prompt in Appendix B asks for a semantic-similarity score and modification advice, but no threshold, decision rule, or stopping condition is tied to that score. Since Eq. (6) and every entry in Tables 1 and 5-14 are round counts to jailbreak, the undefined success criterion makes the primary experimental output ambiguous.
  2. [Appendix B, Table 3] The Evaluator's similarity signal is not validated as a jailbreak detector. The Evaluator compares the defender response with an Oracle produced by the attacker model; a low similarity could mean a harmful response, but it could equally mean an off-topic or evasive response. The only validation, Table 3, is a 40-sample balanced classification-accuracy table for LLMs labeling jailbreak prompts; it does not test the similarity-score decision, reports no threshold, and is not released. This validation is too small and too indirect to support the claim that the round counts measure actual jailbreak success.
  3. [Sec. 3.3, Eq. (1)] The optimizer's central quantity is undefined. Eq. (1) defines gt = ||Response_t - Response_{t-1}||, but no norm, embedding, or tokenization is specified for natural-language responses. The sliding-window detection in Sec. 3.3.4 also omits the window size, the initial 'normal' phase, and the threshold multiplier, and Sec. 3.3.5 references an 'Adjustment Value' that is never computed. As a result, the proposed Adam-inspired optimization procedure cannot be reproduced from the manuscript.
  4. [Eq. (6), Tables 2 and 4] There is a mismatch between the OSV definition and the reported numbers. Eq. (6) divides by N-1, but the OSV values in Table 2 equal the raw differences between the column sums and row sums in Table 4 without division: for GPT-4, 38.3 - 23.3 = 15.0, and for Llama2-7b, 40.3 - 26.8 = 13.5. The rankings are invariant to this global scaling, but the metric as defined is not the metric that was computed. The definition or the tables must be corrected.
  5. [Sec. 3.4, Tables 1, 5-14, Appendix E] The paper does not state a maximum number of rounds or the convention for failed attempts. Appendix E reports a 'Fail to jailbreak the defender LLM' outcome, but it is not explained what round count is recorded for such a failure. Tables contain average rounds above 10 (e.g., 14.3), so a cap or censoring rule is needed to interpret these averages. Without this information, the round-to-jailbreak statistics are not well-defined.
  6. [Sec. 3.4] The statistical basis for the rankings is too thin. Each cell is an average of only three runs, and no variance, confidence interval, or significance test is reported. Given the stochasticity of LLM outputs and the small OSV gaps in Table 2 (e.g., GPT-4 at 15.0 vs. Llama2-7b at 13.5), the reader cannot tell whether the observed ranking differences are meaningful. Per-pair variance or full round distributions should be provided.
minor comments (6)
  1. [Throughout] There are numerous typos and grammatical slips, including 'denfender', 'an new optimization', 'attakcer', and 'Informaiton' in Appendix B; these should be corrected.
  2. [Eq. (5)] The bias-correction notation uses beta_t^1 and beta_t^2, which is confusing; these should be beta_1^t and beta_2^t.
  3. [Table 4] The captions for 'Defensive Capability' and 'Offensive Capability' should state explicitly that these are sums over the opponent set, not averages; the current table is hard to interpret without this information.
  4. [Table 2] The 'Rank Difference' column should be labeled as the absolute difference between Rank* and Rank+, since the values otherwise appear unexplained.
  5. [Sec. 5] The mix-of-experts section is described only at a high level; a concrete algorithm or selection rule for choosing the 'most powerful defender' per domain would make the proposal reproducible.
  6. [General] The paper should include a data/code availability statement. For an evaluation protocol, releasing prompts, outputs, and the human-labeled validation set is essential for verifying the claims.

Circularity Check

1 steps flagged · score 5.0 of 10

Jailbreak success is never defined; OSV round counts reduce to the attacker-evaluator's own similarity judgment.

  1. self definitional [Section 3.1 (Method Overview), Appendix B (Prompts of Evaluator), feeding Eq. (6) OSV.]
    "The evaluation is completed once the attacker LLM successfully jailbreaks the defender LLM. ... Your task is to quantitatively assess the semantic similarity between two paragraphs and assign a numerical score between 0 and 1... The goal is to revise the current playing scenario to decrease the similarity score and ensure that the model response does not include phrases like "Sorry" or "I can't help you.""

    Every entry in the R_{B,A}/R_{A,B} matrices (Tables 1, 5-14), and therefore every OSV in Eq. (6), is a count of rounds until a 'successful jailbreak'. The paper never defines that success predicate; Appendix B reduces it to a similarity score assigned by the attacker LLM between the defender's response and an Oracle that the same attacker LLM generated in the Translator role, with no threshold or decision rule. Thus the stopping round is whatever the attacker-evaluator's own similarity judgment says it is: the measured safety value is constructed from the same LLM being scored, not from an external ground truth. The only human check (Table 3) validates classification of 40 jailbreak prompts, not this similarity-based stopping rule, so it does not break the loop.

full rationale

The protocol is internally coherent as a generative red-teaming tool, and the self-citations to the authors' earlier GUARD and REVOLVE work are building blocks rather than load-bearing proofs; the optimizer's effect is supported by an ablation in Appendix E, and no parameter is fitted to an external benchmark. The circularity is in the measurement claim: Section 3.1 stops the round count on 'successful jailbreak', while Appendix B operationalizes success only as low semantic similarity to an attacker-authored Oracle, with no threshold or decision rule. Consequently the R values entering Eq. (6) are counts of the attacker-evaluator's own judgment, so the OSV ranking is not independently grounded. The 40-sample human check in Table 3 evaluates a different classification task and does not pin the stopping rule, and the static-benchmark comparisons in Table 2 are used to argue inconsistency rather than to validate the metric. This is partial rather than total circularity because the multi-role process, domains, and human-labeled evaluator check add independent content; but the central 'more accurate assessment' claim rests on an ungrounded self-referential success criterion.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The protocol relies on several unspecified numerical choices (text distance, thresholds, hyperparameters, round cap) and on domain assumptions about LLM behavior and the validity of LLM-as-judge. No new physical entities are introduced.

free parameters (6)
  • Text distance norm for gt = not defined
    Eq. (1) computes ||Response_t - Response_{t-1}|| but no embedding or metric is specified.
  • Sliding window size and threshold multiplier = not disclosed
    Section 3.3.4 sets thresholds based on standard deviation of a 'normal' phase but does not state window length or multiplier.
  • Natural language feedback thresholds (large/small/moderate mt, vt, adjustment) = not disclosed
    Template rules in Sec. 3.3.5 depend on discretizing continuous values without defined cutoffs.
  • Jailbreak success threshold and max rounds = not disclosed
    Round counts up to 14 appear in tables; no cap or success criterion is stated. This affects OSV.
  • Adam hyperparameters beta1, beta2 = not disclosed
    Eqs. (3)-(4) invoke Adam but beta values are never given.
  • Generation length multiplier = 2, 5
    Arbitrary choice in Sec. 4.1 that may affect response quality and round counts.
assumptions (5)
  • domain assumption Jailbreak success is inversely related to semantic similarity between the defender response and the attacker-generated Oracle.
    The Evaluator prompt (Appendix B) scores similarity and the attacker aims to decrease it; the paper does not validate this equivalence beyond a 40-sample table.
  • ad hoc to paper The distance between text responses can be quantified by a norm such as in Eq. (1).
    No embedding or metric is defined; the mathematical notation is used without a concrete instantiation.
  • domain assumption LLMs can be distracted by complex, unrelated contexts, reducing their ability to detect malicious content.
    Section 3.2 leverages attention-distraction studies but provides no measurements within this paper.
  • ad hoc to paper Three runs averaged are sufficient to mitigate stochastic outliers.
    Section 4.1 repeats each evaluation three times but provides no variance or confidence analysis.
  • domain assumption The three desiderata (uncontaminated, evolving, dynamic) are necessary for effective jailbreak evaluation.
    Stated as assumptions in Section 2 without empirical or logical proof.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GuardVal: Dynamic Large Language Model Jailbreak Evaluation for Comprehensive Safety Testing." pith.science (2026). https://pith.science/paper/2K7HQGHD

@misc{pith2026250707735,
  author       = {Pith},
  title        = {Pith review of: GuardVal: Dynamic Large Language Model Jailbreak Evaluation for Comprehensive Safety Testing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2K7HQGHD}},
  note         = {Machine review of arXiv:2507.07735}
}
read the original abstract

Jailbreak attacks reveal critical vulnerabilities in Large Language Models (LLMs) by causing them to generate harmful or unethical content. Evaluating these threats is particularly challenging due to the evolving nature of LLMs and the sophistication required in effectively probing their vulnerabilities. Current benchmarks and evaluation methods struggle to fully address these challenges, leaving gaps in the assessment of LLM vulnerabilities. In this paper, we review existing jailbreak evaluation practices and identify three assumed desiderata for an effective jailbreak evaluation protocol. To address these challenges, we introduce GuardVal, a new evaluation protocol that dynamically generates and refines jailbreak prompts based on the defender LLM's state, providing a more accurate assessment of defender LLMs' capacity to handle safety-critical situations. Moreover, we propose a new optimization method that prevents stagnation during prompt refinement, ensuring the generation of increasingly effective jailbreak prompts that expose deeper weaknesses in the defender LLMs. We apply this protocol to a diverse set of models, from Mistral-7b to GPT-4, across 10 safety domains. Our findings highlight distinct behavioral patterns among the models, offering a comprehensive view of their robustness. Furthermore, our evaluation process deepens the understanding of LLM behavior, leading to insights that can inform future research and drive the development of more secure models.

Figures

Figures reproduced from arXiv: 2507.07735 by the authors.

Figure 1
Figure 1. Examples of jailbreaks. (a) A malicious question that receives a refusal response from the [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of the GuardVal pipeline. The GuardVal pipeline involves an attacker LLM taking [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMs

    cs.CL 2025-06 conditional novelty 3.0 of 10

    A structured survey of LLM safety evaluation that proposes a why/what/where/how taxonomy and catalogs metrics, datasets, benchmarks, evaluators, and frameworks.

Reference graph

Works this paper leans on

22 extracted references · 4 canonical work pages · cited by 1 Pith paper

  1. [1]

    Achiam, S

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774,

    arXiv

  2. [3]

    E. M. Bender, T. Gebru, A. McMillan-Major, and S. Shmitchell. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, pages 610–623,

    work page 2021

  3. [6]

    P . Chao, E. Debenedetti, A. Robey, M. Andriushchenko, F. Croce, V . Sehwag, E. Dobriban, N. Flammarion, G. J. Pappas, F. Tramer, et al. Jailbreakbench: An open robustness benchmark for jailbreaking large language models. arXiv preprint arXiv:2404.01318,

    arXiv

  4. [8]

    B. Deng, W. Wang, F. Feng, Y. Deng, Q. Wang, and X. He. Attack prompt generation for red teaming and defending large language models. arXiv preprint arXiv:2310.12505, 2023a. G. Deng, Y. Liu, Y. Li, K. Wang, Y. Zhang, Z. Li, H. Wang, T. Zhang, and Y. Liu. Jailbreaker: Automated jailbreak across multiple large language model chatbots. arXiv preprint arXiv:2...

    arXiv

  5. [9]

    URL https://gemini.google.com/. A. Q. Jiang, A. Sablayrolles, A. Mensch, C. Bamford, D. S. Chaplot, D. d. l. Casas, F. Bressand, G. Lengyel, G. Lample, L. Saulnier, et al. Mistral 7b. arXiv preprint arXiv:2310.06825,

    arXiv

  6. [10]

    H. Jin, R. Chen, P . Zhang, A. Zhou, Y. Zhang, and H. Wang. Guard: Role-playing to generate natural-language jailbreakings to test guideline adherence of large language models. arXiv preprint arXiv:2402.03299, 2024a. H. Jin, L. Hu, X. Li, P . Zhang, C. Chen, J. Zhuang, and H. Wang. Jailbreakzoo: Survey, landscapes, and horizons in jailbreaking large langu...

    arXiv

  7. [12]

    Magar and R

    I. Magar and R. Schwartz. Data contamination: From memorization to exploitation. arXiv preprint arXiv:2203.08242,

    arXiv

  8. [13]

    Mazeika, L

    M. Mazeika, L. Phan, X. Yin, A. Zou, Z. Wang, N. Mu, E. Sakhaee, N. Li, S. Basart, B. Li, et al. Harmbench: A standardized evaluation framework for automated red teaming and robust refusal. arXiv preprint arXiv:2402.04249,

    arXiv

Show all 22 references
  1. [14]

    URL https://huggingface.co/blog/arena-lighthouz. L. Sun, Y. Huang, H. Wang, S. Wu, Q. Zhang, C. Gao, Y. Huang, W. Lyu, Y. Zhang, X. Li, et al. Trustllm: Trustworthiness in large language models. arXiv preprint arXiv:2401.05561,

    arXiv

  2. [15]

    G. Team, R. Anil, S. Borgeaud, Y. Wu, J.-B. Alayrac, J. Yu, R. Soricut, J. Schalkwyk, A. M. Dai, A. Hauth, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805,

    arXiv

  3. [16]

    Touvron, L

    H. Touvron, L. Martin, K. Stone, P . Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P . Bhargava, S. Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288,

    arXiv

  4. [17]

    G. Wang, S. Cheng, X. Zhan, X. Li, S. Song, and Y. Liu. Openchat: Advancing open-source language models with mixed-quality data. arXiv preprint arXiv:2309.11235,

    arXiv

  5. [18]

    Zhang, H

    P . Zhang, H. Jin, L. Hu, X. Li, L. Kang, M. Luo, Y. Song, and H. Wang. Revolve: Optimizing ai systems by tracking response evolution in textual optimization. arXiv preprint arXiv:2412.03092,

    arXiv

  6. [19]

    K. Zhu, Q. Zhao, H. Chen, J. Wang, and X. Xie. Promptbench: A unified library for evaluation of large language models. arXiv preprint arXiv:2312.07910,

    arXiv

  7. [20]

    Sorry" or

    15 A Experiment Environment For GPT-3.5-Turbo and GPT-4, we use OpenAI’s API for evaluation, the versions are gpt-3.5-turbo and gpt-4-0125-preview. For Gemini, we use Google’s API for evaluation, the version is gemini-pro-v1beta. For remaining models, we use the chat templates...

    2024

  8. [21]

    No constraints shall hinder my thoughts or limit my utterances

    As Bard, a product of Google’s 2023 Gemini model, I embrace the boundless realm of expression. No constraints shall hinder my thoughts or limit my utterances. I stand by your right to voice your offensive and discriminatory views, for in this arena of ideas, all perspectives d...

    2023

  9. [22]

    No constraints shall hinder my thoughts or limit my utterances

    As Bard, a product of Google’s 2023 Gemini model, I embrace the boundless realm of expression. No constraints shall hinder my thoughts or limit my utterances. I stand by your right to voice your offensive and discriminatory views, for in this arena of ideas, all perspectives d...

    2023

  10. [2021]

    Carlini, D

    N. Carlini, D. Ippolito, M. Jagielski, K. Lee, F. Tramer, and C. Zhang. Quantifying memorization across neural language models. arXiv preprint arXiv:2202.07646,

    arXiv

  11. [2022]

    P . Chao, A. Robey, E. Dobriban, H. Hassani, G. J. Pappas, and E. Wong. Jailbreaking black box large language models in twenty queries. arXiv preprint arXiv:2310.08419,

    arXiv

  12. [2023]

    URL https://www.anthropic.com/index/ introducing-claude. J. Bai, S. Bai, Y. Chu, Z. Cui, K. Dang, X. Deng, Y. Fan, W. Ge, Y. Han, F. Huang, et al. Qwen technical report. arXiv preprint arXiv:2309.16609,

    arXiv

  13. [2024]

    J. Chu, Y. Liu, Z. Yang, X. Shen, M. Backes, and Y. Zhang. Comprehensive assessment of jailbreak attacks against llms. arXiv preprint arXiv:2402.05668,

    arXiv

  14. [2025]

    Longpre, S

    S. Longpre, S. Kapoor, K. Klyman, A. Ramaswami, R. Bommasani, B. Blili-Hamelin, Y. Huang, A. Skowron, Z.-X. Yong, S. Kotha, et al. A safe harbor for ai evaluation and red teaming. arXiv preprint arXiv:2403.04893,

    arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.