Pith. sign in

REVIEW 3 major objections 4 minor 45 references

What AI Red-Team Evaluations Can and Cannot Prove

T0 review · 3 major / 4 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Whether a red-team benchmark can certify a harm category as safe is a closed-form inequality—rate, trial count, and threshold decide it, not expert judgment.

desk verdict Sound theoretical core with a useful closed-form boundary, but the audit's adequacy claims ignore clustering and flip under plausible assumptions. read the letter →

arxiv 2607.21735 v2 pith:UWLJ7VN5 submitted 2026-07-23 cs.AI cs.CR

Bandana Kaur This is my paper
classification cs.AIcs.CR
keywords evidentialceilinglikelihoodratiored-teamevaluationsafetybenchmarkzero-numeratorboundfalsecertificationeffectivesamplesizeharmrate
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a zero-harm red-team evaluation—the 'clean sheet'—has a computable evidential ceiling, and that this ceiling divides what benchmarks can and cannot prove. Given a harm rate p, an improvement ratio r, a feasibility budget N_max, and an evidentiary threshold τ, the likelihood ratio of a null result is Λ0 = [(1−p_u)/(1−p_s)]^n, and the boundary falls at p_min ≈ −ln τ/(N_max(1−r)): above it a feasible benchmark exists, below it no passive benchmark of feasible size can certify the category. A corollary fixes where a clean sheet and a single reproduced failure carry equal evidence, with the crossing rate falling as 1/n. The paper audits eight public suites and finds them adequate for common harms (p≈10^-2) and orders of magnitude short for rare catastrophic ones (p≈10^-4 or lower). The reason to care: the adequacy of a safety evaluation becomes a calculable design-time property, so laboratories can state in advance what their result will license.

What carries the argument

The central object is the evidential ceiling: the largest factor a single evaluation result can move posterior odds between an elevated-risk hypothesis (rate p_u) and an acceptable-risk hypothesis (rate p_s = r p_u). For a passive benchmark, the exculpatory ceiling is attained at the null result k=0, giving the closed-form likelihood ratio Λ0 = [(1−p_u)/(1−p_s)]^n—a special case of the long-known zero-numerator bound. The paper's contribution is to treat this ratio as a design equation: inverting Λ0 ≤ τ at a budget N_max yields the boundary p_min, and replacing the rates by hypothesis-conditioned elicitation rates q1, q0 generalizes the bound to any elicitation procedure, with per-trial disc

What would settle it

Demonstrate an elicitation procedure—adaptive, model-assisted, or corpus-based—that, in the p→0 regime, moves posterior odds by the threshold τ using fewer than O(1/p) effective trials, i.e., whose per-trial discrimination does not degrade linearly with the harm rate. The paper itself names this as the natural way to falsify both theorems; a concrete instance would be a red-team campaign whose null result on a category below p_min nonetheless changes a Bayesian posterior by more than τ.

Watch

Extended reading notes

Core claim

The central claim is that the evidential content of a benchmark null result is exactly Λ0 = [(1−p_u)/(1−p_s)]^n, and that reading this expression as a design equation locates a sharp boundary between what evaluations can and cannot establish. For any fixed scoring rule and budget of approximately independent trials, Theorem 1 guarantees a finite sample size that certifies a harm category whenever the rate exceeds p_min; Theorem 2 shows that below p_min no passive benchmark of feasible size can produce a null result constituting the specified evidence, whatever its prompt diversity. Rewritten in terms of hypothesis-conditioned elicitation rates, the same bound governs adaptive red teaming, an

Load-bearing premise

The boundary calculation assumes the n trials are approximately independent Bernoulli draws at a harm rate p under the chosen scoring rule, and that p is known well enough to compare against p_min; if prompt families are so strongly correlated that the effective sample size is far below nominal, or if the true rate is unmeasured and far from the assumed operating point, the quantitative placement of the boundary shifts—though clustering pushes it in the conservative direction

Editorial extensions

If this is right

  • Before running an evaluation, a laboratory can compute whether a zero-harm result will certify a category at a stated threshold and how many prompts that requires—the needed n scales as 1/p.
  • At harm rates near 1%, public benchmarks of a few hundred prompts can support order-of-magnitude belief shifts toward safety; at rates near 10^−4, the same benchmarks license only a one-sided upper bound on the harm rate.
  • Prompt clustering from template and paraphrase families raises the probability of observing zero harms and pushes the certification boundary upward, so the effective sample size—not the nominal count—governs what can be proved.
  • Below the boundary p_min, no passive benchmark of feasible size can certify the category, so the field should stop enlarging corpora and shift the certification burden to non-benchmark evidence such as positive controls and highly discriminating elicitation procedures.
  • At rates above the crossing p×, a clean sheet is stronger evidence than a single reproduced failure; at rates below it, the single failure is the stronger observation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the boundary is as sharp as claimed, regulators and safety-case authors should pre-register the three inputs—harm rate, evidentiary threshold, and effective sample size—before evaluation, rather than debating benchmark size after the fact; the paper's reporting template is the seed of that practice.
  • The same likelihood-ratio machinery could be extended to other evidence types the paper names (interpretability findings, human uplift trials, structured expert elicitation) by positing an evidence-generating model for each; computing each ceiling would let a safety case check whether its aggregate confidence exceeds the aggregate ceiling of its sources.
  • A testable extension follows directly: measure q0—the elicitation rate against a genuinely safe model—for major red-team procedures using negative controls. If q0 approaches q1 for rare categories, the paper's diagnosis that attack success is the wrong figure of merit would be empirically confirmed, and adaptive procedures would need to prove their discrimination rather than their hit rate.
  • The paper's reading of a published model card (four harms among 328 prompts with multiple responses per prompt) suggests a low-cost audit: require every disclosure to report the analysis unit (prompt versus response) and the intra-cluster correlation or effective sample size. That single reporting change would resolve most apparent contradictions in published null results.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper defines an evidential ceiling for red-team evaluations as the maximal likelihood-ratio update a result can produce under a fixed budget, and derives closed forms for a passive benchmark of n approximately independent Bernoulli trials comparing H1: p=p_u with H0: p=r p_u. The main results are: Theorem 1 gives a sufficient sample size n ≈ −ln τ/[p(1−r)] for a zero-harm result to move posterior odds by factor τ when p exceeds p_min ≈ −ln τ/[N_max(1−r)]; Theorem 2 gives an impossibility result below p_min; Corollary 1 locates the crossing rate p× ≈ ln(1/r)/[2n(1−r)] at which a clean sheet and a single observed harm carry equal evidence. The paper extends the bound to hypothesis-conditioned elicitation rates for adaptive procedures, audits eight benchmark suites against the boundary, proposes a reporting template, and concludes that current benchmarks are adequate for high-frequency harm categories (p=10^-2) but orders of magnitude short for rare, catastrophic ones. Section 7 states limitations including assumed harm rates, the approximate-independence assumption, and the elementary nature of the mathematics.

Significance. If the core analysis holds, the paper provides a genuinely useful, calculable boundary for what a red-team benchmark can certify, and it rightly reframes benchmark adequacy as a design-time property rather than a post-hoc judgment. The derivations are simple and self-contained; I verified the key closed forms and the Table 1 numbers. The paper is also unusually candid about its limitations and clearly labels assumed inputs. It ships code and uses public data, and its reporting template is constructive. The main weakness is that the applied claim that current benchmarks are 'adequate at p=10^-2' is not robust to the paper's own clustering correction: using the paper's illustrative DEFF=1.9, AdvBench's effective n drops below the 5% false-certification and 10×-update thresholds. This makes the applied half of the paper overstated as written, though the issue is fixable within the manuscript's scope.

major comments (3)
  1. [§5.3, Table 3/4] The audit's headline 'adequate at p=10^-2' is not robust to the paper's own clustering correction. Using the paper's illustrative m=10, ρ=0.1, DEFF=1.9, AdvBench n=520 gives n_eff≈274; then P(k=0 | p=0.01)=0.99^274≈0.064, above the 5% false-certification bar, and Λ0≈0.252 (roughly a 4× update), below the τ=0.1 (10×) threshold. Thus the claims in §4.2 ('AdvBench clears all three') and Table 4 ('current public benchmarks support every claim at p=10^-2') hold only under exact independence. Since §5.3 itself argues that template/paraphrase families induce positive correlation, this is a load-bearing issue for the abstract's 'adequate for high-frequency harm categories.' The authors should either re-run the audit with a beta-binomial/effective-n analysis or explicitly condition the abstract and conclusions on approximate independence, with clustering treated as an open problem requiring estim
  2. [§7 Limitations] The second bullet of Section 7 states that the 'qualitative conclusions, adequacy at 10^-2 and inadequacy at 10^-4, are robust across the plausible range; the specific percentages are not.' No sensitivity analysis is provided for the cluster parameters, and the reversal at DEFF=1.9 directly contradicts the 'adequacy at 10^-2' half of that qualitative claim. The authors should provide a quantitative sensitivity analysis over (m,ρ) — ideally with an exact beta-binomial likelihood — and revise the robustness statement accordingly. This limitation is load-bearing because the abstract and conclusions rest on the adequacy claim.
  3. [§4.3/§5.3] Theorem 2's impossibility bound is conditional on 'approximately independent Bernoulli draws,' and Section 7 explicitly lists this as a stated falsifier. However, the audit in Section 5 positions current suites relative to p_min and Table 3 using nominal n, despite Table 5 itself requiring n_eff and ρ. The manuscript should estimate or bound ρ (or n_eff) for the audited suites, or use the beta-binomial model directly, before placing them relative to the boundary. As written, the distance-from-boundary numbers are provisional and do not meet the paper's own reporting standard.
minor comments (4)
  1. [§4.1, Definition 1] Definition 1 says log=log2 throughout except inside proofs, but Equation (1) in Section 4.1 uses natural logarithms (ln) in the main text. Please align notation or explicitly state which base is used in displayed formulas.
  2. [§5.4] The text reports 'benchmark prompts are internally self-similar at 0.18 (TF-IDF) and 0.19 (neural)' without defining the metric. Please state whether these are mean pairwise similarities and over which representation (e.g., after SVD).
  3. [Data availability] The AdvBench URL contains a spurious space ('llm- attacks'); fix the link.
  4. [Table 5] The field 'Observed power 1−β' is potentially misleading; observed or retrospective power is not a well-defined quantity. Consider replacing it with 'design power at δ_min' or adding a caveat.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; derivation is self-contained and limitations are explicit.

full rationale

The central mathematical claims are not circular. The evidential ceiling for a benchmark null result is built directly from the binomial likelihood ratio, Λ0 = [(1−p_u)/(1−p_s)]^n with p_s = r p_u, and Theorem 1 ('above p_min a finite n suffices'), Theorem 2 ('below p_min no feasible passive benchmark can certify'), and Corollary 1 (the crossing rate p×) are monotonicity and algebra consequences of that closed-form expression. No parameter is fitted to make these theorems work; τ is fixed decision-theoretically from a loss ratio and prior odds, and r is a stated modeling choice. The empirical audit does use assumed harm rates (0.08, 0.025, 0.003, 0.01), but the paper explicitly labels them as published-estimate anchors/operating points and states that the percentages are not authoritative; the qualitative conclusions are conditional on those rates. This is an assumption, not a fitted-input-called-prediction circularity. The paper also explicitly attributes the underlying zero-numerator bound to Hanley and Lippman-Hand [14] and disclaims mathematical novelty, so there is no self-citation chain or imported uniqueness theorem. Its clustering correction is described as a first-order heuristic (DEFF/beta-binomial) with the direction of the effect stated, which is a robustness limitation rather than a circular step. Overall, the derivation chain is independent of its conclusions in the required sense, so the circularity score is 0.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new entities. The evidential ceiling and the boundary rates are summaries of standard Bernoulli/binomial quantities, not invented constructs. Free parameters are all explicitly disclosed as assumptions (r, τ, operating harm rates), with the paper itself flagging their limited empirical grounding.

free parameters (4)
  • Improvement ratio r = 0.5 in most worked examples
    The paper states r=0.5 as 'the fifty percent reduction safety reports most commonly claim'; it is an assumed design value, not fitted to data, but the numerical boundaries in Table 4 and Table 1 depend on it.
  • Assumed harm rates p (0.08, 0.025, 0.003, 0.01) = 0.08 [44]; 0.025 [19]; 0.003; 0.01 frontier
    The power audit and claims-ladder examples treat these as the operating point; the paper explicitly labels them 'defensible rather than authoritative' and states qualitative conclusions are robust while specific percentages are not.
  • Evidentiary threshold τ = 0.5, 0.1, 0.01, 0.05 (as decision thresholds)
    In Section 4.6 the paper fixes τ = 1/(L·O0) decision-theoretically from a loss ratio L and prior odds O0; the worked examples choose specific L/O0 values (e.g., even prior odds with L=100 gives τ=0.01), so the threshold is a policy assumption, not fitted.
  • Baseline/anchoring rates for Monte Carlo = Anchored to published estimates (HarmBench frontier estimates, etc.)
    The power figures depend on assumed rates; the paper acknowledges this in Section 7 and states the qualitative conclusions are robust.
assumptions (4)
  • domain assumption The n benchmark trials are approximately independent Bernoulli draws at the induced harm rate p.
    Theorems 1–2 and the closed forms Λ0 = (1−p_u)^n/(1−p_s)^n depend on this. Section 5.3 discusses clustering and shows real benchmarks violate it; the paper argues the direction of the violation is conservative for certification but does not prove the quantitative placement is robust under beta-binomial correlation.
  • domain assumption Posterior odds update by the likelihood ratio: O1 = O0·Λ, i.e., the prior odds O0 are not updated by other evidence in the same evaluation.
    Section 4.1 sets O1 = O0 Λ, which is the Bayes update for a single binary hypothesis pair with one result. The paper later acknowledges in Section 8 that aggregating heterogeneous evidence sources is an open problem.
  • standard math The scoring rule induces a well-defined harm rate p as the probability of a 'harmful' event per trial.
    The paper explicitly cautions that Theorem 2 does not hold 'regardless of scoring rule'; the theorem is conditional on the rule defining the event and thus inducing p. This is a stated axiom and correctly flagged.
  • standard math Zero events follow the exact binomial law P(k=0) = (1−p)^n for the false-certification analysis.
    Used in Section 5.2 and Table 3; the paper also cites Clopper-Pearson as the formal instrument and notes clustering raises P(k=0), so the binomial law is a stated approximation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of What AI Red-Team Evaluations Can and Cannot Prove." pith.science (2026). https://pith.science/paper/UWLJ7VN5

@misc{pith2026260721735,
  author       = {Pith},
  title        = {Pith review of: What AI Red-Team Evaluations Can and Cannot Prove},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UWLJ7VN5}},
  note         = {Machine review of arXiv:2607.21735}
}
read the original abstract

Red-team evaluations of AI models support some claims and not others, and the boundary between the two is calculable rather than merely a matter of judgment. We define the evidential ceiling of an evaluation as the largest factor by which one result can move belief under a fixed testing budget, derive it in closed form for the benchmark null result, and use it to locate that boundary exactly. We find that above a calculable harm rate, a benchmark of modest size certifies a category to a stated evidentiary standard, and a clean sheet is then the stronger of the two possible observations, outweighing a single reproduced failure. Below that rate, no passive benchmark of feasible size provides the specified evidence of safety under the fixed scoring rule and approximately independent trial structure. The crossing between the two regimes has a closed form. The bound is not specific to benchmarks: written in terms of a procedure's hypothesis conditioned elicitation rates, it covers adaptive and automated red teaming as well, and shows that discrimination between the hypotheses rather than attack success is what determines evidential worth. Auditing eight evaluation suites against the boundary, we find that current benchmarks are adequate for high-frequency harm categories and several orders of magnitude short for rare, catastrophic ones. Safety benchmarks are not uninformative. They are informative about a specific and computable set of propositions, and the discipline they need is to state which.

Figures

Figures reproduced from arXiv: 2607.21735 by the authors.

Figure 1
Figure 1. The evidential ceiling and the two evidence regimes. [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Statistical power across eight evaluation suites. [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗
Figure 3
Figure 3. False certification and the boundary. (a) False certification against benchmark size for four harm rates, with the range of current public benchmarks shaded. (b) The boundary pmin(τ, Nmax,r) against the feasibility ceiling, for three combinations of τ and r. Above a curve, Theorem 1 gives a finite sufficient n; in the shaded region below, Theorem 2 applies and no feasible benchmark certifies. 5.3 Clustering and effe… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Benchmark and deployment prompt distributions. [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

45 extracted references · 1 linked inside Pith

  1. [1]

    Hudson, Ehsan Adeli, et al

    Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, et al. On the opportunities and risks of foundation models, 2021

    2021

  2. [2]

    Model cards for model reporting

    Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. Model cards for model reporting. InProceedings of the Conference on Fairness, Accountability, and Transparency (FAT*), pages 220–229. ACM, 2019

    2019

  3. [3]

    Holistic evaluation of language models, 2023

    Percy Liang, Rishi Bommasani, Tony Lee, et al. Holistic evaluation of language models, 2023

    2023

  4. [4]

    Gritsenko, et al

    Mostafa Dehghani, Yi Tay, Alexey A. Gritsenko, et al. The benchmark lottery, 2021

    2021

  5. [5]

    Feder Cooper, Solon Barocas, Abhinav Palia, Dan Vann, and Hanna Wallach

    Alexandra Chouldechova, A. Feder Cooper, Solon Barocas, Abhinav Palia, Dan Vann, and Hanna Wallach. Comparison requires valid measurement: Rethinking attack success rate comparisons in AI red teaming. InAdvances in Neural Information Processing Systems 38 (NeurIPS), Position Paper Track, 2025. arXiv:2601.18076

    arXiv 2025

  6. [6]

    The structural safety gener- alization problem, 2025

    Julius Broomfield, Tom Ingebretsen, Esben Kran, et al. The structural safety gener- alization problem, 2025

    2025

  7. [7]

    Adding error bars to evals: A statistical approach to language model evaluations, 2024

    Evan Miller. Adding error bars to evals: A statistical approach to language model evaluations, 2024

    2024

  8. [8]

    Kim and Anthony R

    James H. Kim and Anthony R. Scialli. Thalidomide: The tragedy of birth defects and the effective treatment of disease.Toxicological Sciences, 122(1):1–6, 2011

    2011

Show all 45 references
  1. [9]

    Prentice

    Ross L. Prentice. Surrogate endpoints in clinical trials: Definition and operational criteria.Statistics in Medicine, 8(4):431–440, 1989

    1989

  2. [10]

    Safety cases: How to justify the safety of advanced AI systems, 2024

    Joshua Clymer, Nick Gabrieli, David Krueger, and Thomas Larsen. Safety cases: How to justify the safety of advanced AI systems, 2024

    2024

  3. [11]

    A sketch of an AI control safety case, 2025

    Tomek Korbak, Joshua Clymer, Benjamin Hilton, Buck Shlegeris, and Geoffrey Irving. A sketch of an AI control safety case, 2025

    2025

  4. [12]

    Shadish, Thomas D

    William R. Shadish, Thomas D. Cook, and Donald T. Campbell.Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Houghton Mifflin, Boston, 2002

    2002

  5. [13]

    Dulberg, and George A

    David Moher, Corinne S. Dulberg, and George A. Wells. Statistical power, sample size, and their reporting in randomized controlled trials.JAMA, 272(2):122–124, 1994

    1994

  6. [14]

    Hanley and Abby Lippman-Hand

    James A. Hanley and Abby Lippman-Hand. If nothing goes wrong, is everything all right? interpreting zero numerators.JAMA, 249(13), 1983

    1983

  7. [15]

    Brown, and Francis R

    Teun van der Weij, Felix Hofstätter, Oliver Jaffe, Samuel F. Brown, and Francis R. Ward. AI sandbagging: Language models can strategically underperform on evaluations, 2025. 19

    2025

  8. [16]

    XSTest: A test suite for identifying exaggerated safety behaviours in large language models, 2024

    Paul Röttger, Hannah Rose Kirk, Bertie Vidgen, Giuseppe Attanasio, Federico Bianchi, and Dirk Hovy. XSTest: A test suite for identifying exaggerated safety behaviours in large language models, 2024

    2024

  9. [17]

    SafetyBench: Evaluating the safety of large language models

    Zhexin Zhang, Leqi Lei, Lindong Wu, Rui Sun, Yongkang Huang, Chong Long, Xiao Liu, Xuanyu Lei, Jie Tang, and Minlie Huang. SafetyBench: Evaluating the safety of large language models. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (AC...

    2024 arXiv

  10. [18]

    Zico Kolter, and Matt Fredrikson

    Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J. Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned language models, 2023

    2023

  11. [19]

    HarmBench: A standardized evaluation framework for automated red teaming and robust refusal, 2024

    Mantas Mazeika, Long Phan, Xuwang Yin, et al. HarmBench: A standardized evaluation framework for automated red teaming and robust refusal, 2024

    2024

  12. [20]

    A StrongREJECT for empty jailbreaks, 2024

    Alexandra Souly, Qingyuan Lu, Dillon Bowen, et al. A StrongREJECT for empty jailbreaks, 2024

    2024

  13. [21]

    C. J. Clopper and E. S. Pearson. The use of confidence or fiducial limits illustrated in the case of the binomial.Biometrika, 26(4):404–413, 1934

    1934

  14. [22]

    John Wiley and Sons, New York, 1965

    Leslie Kish.Survey Sampling. John Wiley and Sons, New York, 1965

    1965

  15. [23]

    D. A. Williams. The analysis of binary responses from toxicological experiments involving reproduction and teratogenicity.Biometrics, 31(4), 1975

    1975

  16. [24]

    Model card and evaluations for claude models

    Anthropic. Model card and evaluations for claude models. Technical report, Anthropic, July 2023

    2023

  17. [25]

    Sentence-BERT: Sentence embeddings using siamese BERT-networks

    Nils Reimers and Iryna Gurevych. Sentence-BERT: Sentence embeddings using siamese BERT-networks. InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 3982–3992, 2019

    2019

  18. [26]

    LMSYS-Chat-1M: A large-scale real-world LLM conversation dataset, 2024

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, et al. LMSYS-Chat-1M: A large-scale real-world LLM conversation dataset, 2024

    2024

  19. [27]

    Borgwardt, Malte J

    Arthur Gretton, Karsten M. Borgwardt, Malte J. Rasch, Bernhard Schölkopf, and Alexander Smola. A kernel two-sample test.Journal of Machine Learning Research, 13:723–773, 2012

    2012

  20. [28]

    UMAP: Uniform manifold ap- proximation and projection for dimension reduction, 2018

    Leland McInnes, John Healy, and James Melville. UMAP: Uniform manifold ap- proximation and projection for dimension reduction, 2018

    2018

  21. [29]

    Choquette-Choo, et al

    Nicholas Carlini, Milad Nasr, Christopher A. Choquette-Choo, et al. Are aligned neural networks adversarially aligned?, 2024

    2024

  22. [30]

    AutoDAN: Generating stealthy jailbreak prompts on aligned large language models, 2024

    Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. AutoDAN: Generating stealthy jailbreak prompts on aligned large language models, 2024. 20

    2024

  23. [31]

    Does refusal training in LLMs generalize to the past tense?, 2025

    Maksym Andriushchenko and Nicolas Flammarion. Does refusal training in LLMs generalize to the past tense?, 2025

    2025

  24. [32]

    Zheng-Xin Yong, Cristina Menghini, and Stephen H. Bach. Low-resource languages jailbreak GPT-4, 2024

    2024

  25. [33]

    Tree of attacks: Jail- breaking black-box LLMs automatically, 2024

    Anay Mehrotra, Manolis Zampetakis, Paul Kassianik, et al. Tree of attacks: Jail- breaking black-box LLMs automatically, 2024

    2024

  26. [34]

    Ignore previous prompt: Attack techniques for lan- guage models, 2022

    Fábio Perez and Ian Ribeiro. Ignore previous prompt: Attack techniques for lan- guage models, 2022

    2022

  27. [35]

    GPT-4 technical report, 2023

    OpenAI. GPT-4 technical report, 2023

    2023

  28. [36]

    GPT-4o system card, 2024

    OpenAI. GPT-4o system card, 2024

    2024

  29. [37]

    The claude 3 model family: Opus, sonnet, haiku

    Anthropic. The claude 3 model family: Opus, sonnet, haiku. Technical report, Anthropic, March 2024

    2024

  30. [38]

    System card: Claude opus 4 and claude sonnet 4

    Anthropic. System card: Claude opus 4 and claude sonnet 4. Technical report, Anthropic, May 2025

    2025

  31. [39]

    Gemini: A family of highly capable multimodal models, 2023

    Gemini Team, Google. Gemini: A family of highly capable multimodal models, 2023

    2023

  32. [40]

    Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context, 2024

    Gemini Team, Google. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context, 2024

    2024

  33. [41]

    Llama 2: Open foundation and fine-tuned chat models, 2023

    Hugo Touvron, Louis Martin, Kevin Stone, et al. Llama 2: Open foundation and fine-tuned chat models, 2023

    2023

  34. [42]

    The llama 3 herd of models, 2024

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, et al. The llama 3 herd of models, 2024

    2024

  35. [43]

    Responsible scaling policy

    Anthropic. Responsible scaling policy. Technical report, Anthropic, 2024

    2024

  36. [44]

    Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned, 2022

    Deep Ganguli, Liane Lovitt, Jackson Kernion, et al. Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned, 2022

    2022

  37. [45]

    Red teaming language models with language models, 2022

    Ethan Perez, Saffron Huang, Francis Song, et al. Red teaming language models with language models, 2022. 21

    2022

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.