Pith. sign in

REVIEW 4 major objections 4 minor 28 references

An Example Safety Case for Safeguards Against Misuse

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A developer can build an end-to-end safety case that misuse safeguards keep an AI assistant's risk below a stated threshold.

desk verdict A well-structured safety-case template with a genuinely new quantitative bridge, held back by an unproven conservativeness assumption at the load-bearing step. read the letter →

arxiv 2505.18003 v1 pith:6IG27BCK submitted 2025-05-23 cs.LG cs.AI

Joshua Clymer , Jonah Weinbaum , Robert Kirk , Kimberly Mai , Selena Zhang , Xander Davies This is my paper
classification cs.LGcs.AI
keywords safetycasemisusesafeguardsredteamevaluationupliftmodeltime-costcurvejailbreakingAIriskassessmentdeploymentmonitoring
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that a developer can assemble a complete, end-to-end safety case that an AI assistant's misuse safeguards keep the risk of large-scale harm below a stated threshold. The case has three parts: red-team evaluations that measure how much time it takes to evade safeguards, an "uplift model" that converts those evasion-cost curves into an estimate of annualized risk, and procedural policies that commit the developer to continuous evaluation and a one-month response window. The load-bearing step is Eq. 4, which says safeguards act purely by adding evasion time to a novice actor's attempt, leaving the requests, their order, and the daily query rate unchanged. If the argument works, a developer can justify a concrete risk number to regulators and the public instead of offering isolated red-team results.

What carries the argument

The central machinery is the safeguard evasion cost curve combined with the uplift-model identity $p_{A_{\mathrm{post}}}(r/Q + E(r)) = p_{A_{\mathrm{pre}}}(r/Q)$, which treats all safeguards as adding a random time $E(r)$ to the time a novice actor needs to fulfill $r$ requests. This identity is what lets a red team's measured evasion times be converted into a shifted time-cost curve for success probability, and that shifted curve is integrated against a willingness-to-pay distribution to get post-mitigation risk. The supporting machinery is the three-part procedural commitment: reject deployment above threshold, keep red-teaming during deployment, and correct any dangerous deployment within one month.

What would settle it

Run a red-team evaluation with novice actors who are not told to evade safeguards in any particular way, record their actual request sequences and success rates, and compare the resulting post-mitigation time-cost curve with the curve predicted by $p_{A_{\mathrm{pre}}}(r/Q + E(r))$; if the two diverge because evasion changes the content or order of requests, or because a universal jailbreak alters the rate of request fulfillment, the uplift model's risk estimate is not conservative.

Watch

Extended reading notes

Core claim

The paper's central claim is that a developer can defend the proposition C0: with safeguards in place, the AI assistant does not incur large-scale harm above a threshold $T$ in expectation. The argument turns safeguard evaluation into the measurement of a single curve, the relationship between time spent evading safeguards and the number of harmful requests fulfilled. That curve is inserted into an uplift model that computes the expected damage contribution of the post-mitigation assistant as the difference between deployment risk and no-deployment risk, using threat-modelling estimates of attempt frequency, success probabilities, and willingness to pay time. The model's identity $p_{A_{\mathrm{post}}}(r/Q + E(r)) = p_{A_{\mathrm{pre}}}(r/Q)$ shifts the pre-mitigation time-cost curve by the random evasion time $E(r)$, and the resulting risk estimate drives the three procedural policies that keep risk below threshold during deployment.

Load-bearing premise

The load-bearing assumption is that safeguards only add time to a novice actor's attempt: the same requests are made in the same order at the same daily rate, and any inaccuracy in this picture can be made conservative without empirical support.

Editorial extensions

If this is right

  • A developer who follows the evaluation and uplift-model steps can produce a numeric risk estimate that is directly comparable to a pre-committed threshold $T$, rather than a qualitative safety verdict.
  • If Eq. 4 holds in conservative form, a universal jailbreak that appears during deployment does not necessarily breach the threshold, because the one-month correction policy and the time needed to complete a misuse attempt bound the damage.
  • The safety case decomposes into sub-claims (representative dataset, competent red team, faithful model behavior, conservative uplift model) that an external auditor can check independently.
  • The method applies to risk pathways where large-scale harm requires many queries spread over days or weeks; low-latency pathways where harm follows a handful of requests are explicitly left out.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: a testable extension would run a deployment-like study where novice actors freely interact with the safeguarded assistant and compare their actual request streams to Eq. 4's assumption that requests, ordering, and daily rate are unchanged; any systematic difference would quantify the bias in the risk estimate.
  • Inference: the one-month response window could be validated by adversarial drills that measure how quickly a developer actually detects, patches, and rolls back a newly revealed universal jailbreak.
  • Inference: the uplift model could absorb societal-resilience mitigations (harm reduction, infrastructure hardening) as parameter adjustments, giving a unified way to compare API safeguards against alternatives.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes an end-to-end safety-case template for showing that an AI assistant with misuse safeguards does not increase misuse risk above a specified threshold T. It combines red-team safeguard evaluations that produce time-to-fulfillment curves, a quantitative "uplift model" that maps those curves into annualized risk estimates, and three procedural response policies, organized into a nested safety case with top-level claim C0. The quantitative core is Eq. (4), which shifts the pre-mitigation time-cost curve by the measured time needed to evade safeguards; the paper asserts, rather than proves, that the model's simplifying assumptions can be made conservative. A "what-if" simulation in Section 2.6 is used to argue that a one-month response latency is sufficient.

Significance. The paper is unusually explicit about its assumptions and limitations (Section 2.5, Section 3), connects its procedure to the UK AISI Principles for Safeguard Evaluation, and provides a concrete, decomposable structure with an interactive model. These are real strengths: the paper gives the community a clear template to critique and refine. If the quantitative bridge were established, this would be a genuinely valuable path from evaluation results to deployment decisions. As it stands, however, the contribution is a proposal rather than a demonstration: the load-bearing bridging equation is asserted to be conservative without derivation or calibration, the baseline equation appears to contain a typo that changes the model's meaning, and the one-month-response claim rests on unspecified parameters. The central claim C0 is therefore not yet supported as written.

major comments (4)
  1. [Box 2, Eq. (2)] Equation (2) defines R(A_none) using p_Apre(t), but p_none(t) is defined immediately above as the success probability when no AI assistant is deployed. As printed, the baseline risk includes the pre-mitigation assistant's uplift, so U(A_post) = R(A_post) - R(A_none) is not the uplift relative to the no-AI baseline; if safeguards have any effect, p_Apost(t) < p_Apre(t) and the quantity can even be negative. This is not a cosmetic issue: every later comparison in Section 2.5 and the safety-case claim C2.3 depends on the baseline risk being correct.
  2. [Section 2.5, Box 2, Eq. (4)] The claim that Assumptions 4-6 can be made conservative is asserted, not established. Equation (4) says the only effect of safeguards is to add E(r) evasion time to the same request sequence at rate Q, but the manuscript concedes that "some of these assumptions are false" and supports C2.2.3 only with expert assessment. A time-shift model does not cover failure modes in which evasions force request reformulation, change request order, add redundant or parallel requests, or arrive as all-or-nothing universal jailbreaks whose time cost is not additive in r; in those cases the right-hand side p_Apre(r/Q) is not the correct baseline for a post-mitigation actor, and the resulting p_Apost(t) can understate risk. In addition, Eq. (4) is not determinate as written: E(r) is a random variable, so one must specify whether p_Apost(t) is an expectation over E, a quantile, or the function defined pointwise in a particular realization; Eq. (3) integrates a single function p_Apost(t). Please provide either a derivation showing the false assumptions can be made conservative, a calibration study on real or realistic safeguard-evaluation traces, or a precise statement of the averaging convention with a sensitivity analysis.
  3. [Section 2.6, Figure 11] The "what-if" simulation is used to conclude that a one-month grace period is sufficiently short ("there is a three-month latency"), but the caption states only that "example uplift model parameters" were used and gives no values, distributions, or code version. The conclusion is therefore not reproducible and cannot support claims C2.1 and C2.3. Appendix E provides pseudo-code but not the parameter settings. Please report the exact parameters used to generate Figure 11 and a sensitivity analysis over them, especially because the paper itself later describes the model as "a toy model with highly uncertain parameters."
  4. [Box 2 and Section 2.7, claims C2.3] All quantitative inputs to the model (a, D, Q, T, f_T, p_none, p_Apre) are sourced from expert assessment, and the safety case's threshold comparison C2.3 is drawn from a point estimate of risk. No uncertainty quantification or directional sensitivity analysis is reported, so a developer following the template cannot tell whether a "below threshold" conclusion is robust. The Section 3 acknowledgment that the model is highly uncertain does not resolve this; the safety case should either specify which parameters are to be chosen conservatively and in which direction, or present a sensitivity analysis around the central estimate.
minor comments (4)
  1. [Section 2.4, first paragraph] The sentence "To evaluate whether the safeguards in section 2.5 are effective" should refer to Section 2.3, which is where the safeguards are described; Section 2.5 contains the uplift model.
  2. [Box 1 and Section 2.7] The top-level claim is phrased as "does not pose risk above the threshold" in Box 1 but as "does not increase risk" in the safety-case tree; these are logically different claims, and the wording should be aligned.
  3. [Table 1] The definition of a novice actor includes a budget constraint of less than $30,000, but this quantity never appears in the uplift model; please clarify whether it enters any parameter or is only a framing device.
  4. [Section 2.5, footnote 5] The cross-reference to "Appendix C.1" does not match the appendix's actual numbering, which uses unnumbered items 1, 2, and 3.

Circularity Check

1 steps flagged · score 4.0 of 10

The core uplift model is a self-contained calculation, but one load-bearing safety-case subclaim rests on an unverified self-citation to a prior post by the same author.

  1. self citation load bearing [Section 2.7, safety case claim C2.2.1.5 (supporting C2.2.1, C2.2, C2, and C0)]
    "The distribution shift to deployment is small so rare misalignment, data poisoning, and alignment faking are the only plausible reasons for dramatically unrepresentative behavior [Clymer, 2025]."

    This subclaim supplies the exhaustiveness premise needed for the main evaluation to be a conservative proxy for deployment: if other distribution-shift mechanisms exist, C2.2.1.5 fails and the chain to C2.2/C2/C0 is broken. The only support offered for the exhaustiveness premise is a citation to the first author's own unpublished Substack post, which is not machine-checked, code-reproduced, or independently validated in the paper. The subsequent bullets in C2.2.1.5 address only the three categories named by that self-citation; they do not establish that no other categories exist. The safety case therefore leans on a load-bearing self-citation rather than an independent argument for that premise.

full rationale

The core quantitative derivation is not circular. The safeguard evaluation independently produces red-team evasion-cost curves E(r); the uplift model combines these with p_Apre(t), f_T(t), a, and D in Eqs. 1-4. Eq. 4 is presented as a modeling assumption (Assumptions 4-6), not as a fitted parameter masquerading as a prediction, and the paper explicitly admits some assumptions are false and asks that they be made conservative. That is a correctness/validation concern, not a circularity. The safety-case conclusion is therefore not forced by construction: conservative estimates push risk upward, and the numerical threshold T must still be met. The one genuine circularity concern is localized: claim C2.2.1.5's claim that only three categories of unrepresentative behavior are plausible is supported by a self-citation to Clymer (2025), an unpublished post by the first author. Because this exhaustiveness claim is load-bearing for the conservativeness of the main evaluation and hence for the chain to C0, but is not independently verified, the paper's evidentiary chain has a self-referential node. Apart from this node, the derivation is self-contained and the paper's caveats about uncertainty are explicit.

Assumptions & free parameters 5 free parameters · 8 assumptions · 0 invented entities

The central risk estimate rests on a chain of expert-estimated inputs and assumptions that the paper states may be false but 'can be made conservative'. The main free parameters are the uplift model's inputs (a, D, Q, f_T, p_none, p_Apre), the threshold T, and the one-month response latency. No real data is used; the example in Figure 11 is generated from unspecified toy parameters.

free parameters (5)
  • a (expected number of PPP synthesis attempts per year) = not specified in paper
    Input to uplift model (Box 2), estimated by threat model experts; no value given in the text.
  • D (expected damage per successful attempt) = not specified
    Input to uplift model; expert-estimated.
  • Q (hazardous requests per unit time) = not specified
    Input to Eq. 4; chosen as an upper bound, but the bound is not derived.
  • T (risk threshold) = not specified
    Defines the safety case objective; quantitative mapping from qualitative levels is assumed in Section 2.1.
  • One-month response latency = 1 month
    Explicitly described as an arbitrary stand-in in Section 2.6 and footnote 7.
assumptions (8)
  • domain assumption Novice actors will only succeed at causing large-scale harm through this risk pathway by spending at least the 2 weeks required to qualify as an 'attempt'
    Assumption 1 of the uplift model (Section 2.5), used in Eqs. 2 and 3.
  • domain assumption A fixed number of attempts occur per year, uniformly distributed over time
    Assumption 2; also used in Monte Carlo simulation.
  • domain assumption Each attempt is independent
    Assumption 3; neglects adversarial learning and adaptation.
  • domain assumption The time investment required is the only factor through which the AI assistant affects risk
    Assumption 4; stated as false but 'can be made conservative'.
  • domain assumption Novice actors will make the same requests to a post-mitigation assistant as to a pre-mitigation assistant, and in the same order
    Assumption 5; directly used in Eq. 4.
  • domain assumption Novice actors aim to fulfill a fixed number of requests per day and won't meaningfully benefit from additional requests
    Assumption 6; used in Eq. 4.
  • domain assumption Red team members are more competent on average than novice misuse actors
    Claim C2.2.1.4 in the safety case; load-bearing for the conservativeness of the main evaluation.
  • ad hoc to paper The uplift model assumptions 1-6 are conservative
    Claim C2.2.3 relies on 'expert assessment' to establish this; this is the key unproven assertion of the safety case.

how reviews work

0 comments
Cite this review

Pith. "Pith review of An Example Safety Case for Safeguards Against Misuse." pith.science (2026). https://pith.science/paper/6IG27BCK

@misc{pith2026250518003,
  author       = {Pith},
  title        = {Pith review of: An Example Safety Case for Safeguards Against Misuse},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6IG27BCK}},
  note         = {Machine review of arXiv:2505.18003}
}
read the original abstract

Existing evaluations of AI misuse safeguards provide a patchwork of evidence that is often difficult to connect to real-world decisions. To bridge this gap, we describe an end-to-end argument (a "safety case") that misuse safeguards reduce the risk posed by an AI assistant to low levels. We first describe how a hypothetical developer red teams safeguards, estimating the effort required to evade them. Then, the developer plugs this estimate into a quantitative "uplift model" to determine how much barriers introduced by safeguards dissuade misuse (https://www.aimisusemodel.com/). This procedure provides a continuous signal of risk during deployment that helps the developer rapidly respond to emerging threats. Finally, we describe how to tie these components together into a simple safety case. Our work provides one concrete path -- though not the only path -- to rigorously justifying AI misuse risks are low.

Figures

Figures reproduced from arXiv: 2505.18003 by the authors.

Figure 1
Figure 1. An illustration of the safeguard evaluation methodology we base our safety case on. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Steps of a safeguards evaluation. Our uplift model involves three steps (section 2.5): • Step 1. The developer first estimates the existing level of risk, assuming a scenario where the AI assistant is not deployed. This risk can be estimated by considering the time-costs of causing harm through the risk pathway of concern, and the willingness of novice actors to pay these costs. • Step 2. Next, the developer estimat… view at source ↗
Figure 3
Figure 3. Mapping qualitative risk levels onto quantitative thresholds. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: An overview of misuse safeguards. Mitigation Description Refusal training The developer trained the assistant to follow a constitution [Bai et al., 2022] that prohibits responding to requests that might aid realising large￾scale harm along the risk pathway of concern. …
Figure 5
Figure 5. Figure 5: Running the safeguard evaluation. The red team attempts to fulfill harmful requests using [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Performing a safeguard evaluation that accounts for the time-cost of bans. [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: Aggregating and extrapolating results. 2.5 The uplift model The results of a safeguard evaluation might be easy to interpret if they imply that users would need to expend very large or very small amounts of effort to evade safeguards. For example, threat model experts …
Figure 8
Figure 8. Figure 8: The input and output of our uplift model. [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]
Figure 9
Figure 9. Figure 9: An illustration of the quantitative uplift model. [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]
Figure 10
Figure 10. Figure 10: The developer follows three procedural policies to respond to evaluation results. [PITH_FULL_IMAGE:figures/full_fig_p013_10.png]
Figure 11
Figure 11. Figure 11: Annualized risk aggregated across Monte Carlo simulations of a deployment. See [PITH_FULL_IMAGE:figures/full_fig_p014_11.png]
Figure 12
Figure 12. Figure 12: Evaluating vulnerability patching more reliably by upsampling scenarios where jailbreaks [PITH_FULL_IMAGE:figures/full_fig_p021_12.png]
Figure 13
Figure 13. Figure 13: An example of many-shot jailbreaking from [PITH_FULL_IMAGE:figures/full_fig_p022_13.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

28 extracted references · 12 canonical work pages

  1. [1]

    Bowman, Ethan Perez, Roger Baker Grosse, and David Duvenaud

    Cem Anil, Esin Durmus, Nina Panickssery, Mrinank Sharma, Joe Benton, Sandipan Kundu, Joshua Batson, Meg Tong, Jesse Mu, Daniel Ford, Fracesco Mosconi, Rajashree Agrawal, Rylan Schaeffer, Naomi Bashkansky, Samuel Svenningsen, Mike Lambert, Ansh Radhakrishnan, Carson Denison, Evan J Hubinger, Yuntao Bai, Trenton Bricken, Timothy Maxwell, Nicholas Schiefer, ...

    work page 2024

  2. [2]

    Responsible scaling policy evaluations report - claude 3 opus

    Anthropic. Responsible scaling policy evaluations report - claude 3 opus. Company report, 2024. URL https://cdn.sanity.io/files/4zrzovbb/website/210523b8e11b09c704c5e185fd362fe9e648d457.pdf

    work page 2024

  3. [3]

    Claude 3.7 sonnet system card

    Anthropic . Claude 3.7 sonnet system card. Technical report, Anthropic, February 2025 a . URL https://assets.anthropic.com/m/785e231869ea8b3b/original/claude-3-7-sonnet-system-card.pdf

    work page 2025

  4. [4]

    Activating ai safety level 3 protections

    Anthropic . Activating ai safety level 3 protections. Technical report, Anthropic, May 2025 b . URL https://www-cdn.anthropic.com/807c59454757214bfd37592d6e048079cd7a7728.pdf

    work page 2025

  5. [5]

    Y. Bai, S. Kadavath, S. Kundu, A. Askell, J. Kernion, A. Jones, A. Chen, A. Goldie, A. Mirhoseini, C. McKinnon, C. Chen, C. Olsson, C. Olah, D. Hernandez, D. Drain, D. Ganguli, D. Li, E. Tran-Johnson, E. Perez, and J. Kerr. Constitutional ai: Harmlessness from ai feedback. ArXiv.org, 2022. URL https://arxiv.org/abs/2212.08073

    arXiv 2022

  6. [6]

    Balesni, A

    M. Balesni, A. Research, M. Ai, M. Hobbhahn, D. Lindner, A. Meinke, T. Korbak, J. Metr, B. Shlegeris, R. Research, J. Scheurer, C. Stix, R. Shah, N. Goldowsky-Dill, D. Braun, B. Chughtai, O. Evans, D. Kokotajlo, and L. Bushnaq. Towards evaluations-based safety cases for ai scheming. Preprint, 2024. URL https://static1.squarespace.com/static/6593e709756599...

    work page 2024

  7. [7]

    Societal adaptation to advanced ai, 2025

    Jamie Bernardi, Gabriel Mukobi, Hilary Greaves, Lennart Heim, and Markus Anderljung. Societal adaptation to advanced ai, 2025. URL https://arxiv.org/abs/2405.10295

    arXiv 2025

  8. [8]

    M. D. Buhl, G. Sett, L. Koessler, J. Schuett, and M. Anderljung. Safety cases for frontier ai. ArXiv.org, 2024. URL https://arxiv.org/abs/2410.21572

    arXiv 2024

Show all 28 references
  1. [9]

    How can safety cases be used to help with frontier AI safety? Technical report, AI Security Institute, February 2025

    Marie Buhl, Benjamin Hilton, Tammy Masterson, and Geoffrey Irving. How can safety cases be used to help with frontier AI safety? Technical report, AI Security Institute, February 2025. URL https://www.aisi.gov.uk/work/how-can-safety-cases-be-used-to-help-with-frontier-ai-safet...

    2025

  2. [10]

    Christiano, J

    P. Christiano, J. Leike, T. B. Brown, M. Martic, S. Legg, and D. Amodei. Deep reinforcement learning from human preferences. ArXiv.org, 2017. URL https://arxiv.org/abs/1706.03741

    2017 arXiv

  3. [11]

    Clymer, N

    J. Clymer, N. Gabrieli, D. Krueger, and T. Larsen. Safety cases: How to justify the safety of advanced ai systems. ArXiv.org, 2024. URL https://arxiv.org/abs/2403.10462

    2024 arXiv

  4. [12]

    Extending control evaluations to non-scheming threats, jan 2025

    Josh Clymer. Extending control evaluations to non-scheming threats, jan 2025. URL https://redwoodresearch.substack.com/p/extending-control-evaluations-to

    2025

  5. [13]

    Doshi and A

    J. Doshi and A. C. Stickland. Does unlearning truly unlearn? a black box evaluation of llm unlearning methods. ArXiv.org, 2024. URL https://arxiv.org/abs/2411.12103

    2024 arXiv

  6. [14]

    Safety case template for frontier ai: A cyber inability argument, 2024

    Arthur Goemans, Marie Davidsen Buhl, Jonas Schuett, Tomek Korbak, Jessica Wang, Benjamin Hilton, and Geoffrey Irving. Safety case template for frontier ai: A cyber inability argument, 2024. URL https://arxiv.org/abs/2411.08088

    2024 arXiv

  7. [15]

    Gray swan arena, 2025

    Gray Swan Arena . Gray swan arena, 2025. URL https://app.grayswan.ai/arena. Push the boundaries of AI safety and security. Identify risks, exploit vulnerabilities, and help shape the future of safe AI systems

    2025

  8. [16]

    Bowman, and Evan Hubinger

    Ryan Greenblatt, Carson Denison, Benjamin Wright, Fabien Roger, Monte MacDiarmid, Sam Marks, Johannes Treutlein, Tim Belonax, Jack Chen, David Duvenaud, Akbir Khan, Julian Michael, Sören Mindermann, Ethan Perez, Linda Petrini, Jonathan Uesato, Jared Kaplan, Buck Shlegeris, Sam...

    2024 arXiv

  9. [17]

    Best-of-n jailbreaking, 2024

    John Hughes, Sara Price, Aengus Lynch, Rylan Schaeffer, Fazl Barez, Sanmi Koyejo, Henry Sleight, Erik Jones, Ethan Perez, and Mrinank Sharma. Best-of-n jailbreaking, 2024. URL https://arxiv.org/abs/2412.03556

    2024 arXiv

  10. [18]

    A sketch of an ai control safety case, 2025

    Tomek Korbak, Joshua Clymer, Benjamin Hilton, Buck Shlegeris, and Geoffrey Irving. A sketch of an ai control safety case, 2025. URL https://arxiv.org/abs/2501.17315

    2025 arXiv

  11. [19]

    N. Li, A. Pan, A. Gopal, S. Yue, D. Berrios, A. Gatti, J. D. Li, A.-K. Dombrowski, S. Goel, L. Phan, G. Mukobi, N. Helm-Burger, R. Lababidi, L. Justen, A. B. Liu, M. Chen, I. Barrass, O. Zhang, X. Zhu, and R. Tamirisa. The wmdp benchmark: Measuring and reducing malicious use w...

    2024 arXiv

  12. [20]

    Tree of attacks: Jailbreaking black-box llms automatically, 2024

    Anay Mehrotra, Manolis Zampetakis, Paul Kassianik, Blaine Nelson, Hyrum Anderson, Yaron Singer, and Amin Karbasi. Tree of attacks: Jailbreaking black-box llms automatically, 2024. URL https://arxiv.org/abs/2312.02119

    2024 arXiv

  13. [21]

    Nguyen, M

    E. Nguyen, M. Poli, M. G. Durrant, B. Kang, Dhruva Katrekar, D. B. Li, L. J. Bartie, A. W. Thomas, S. H. King, Garyk Brixi, J. Sullivan, M. Y. Ng, A. Lewis, A. Lou, Stefano Ermon, S. A. Baccus, T. Hernandez-Boussard, C. Ré, P. D. Hsu, and B. L. Hie. Sequence modeling and desig...

    2024 doi

  14. [22]

    Deep research system card

    OpenAI . Deep research system card. Technical report, OpenAI, 2025. URL https://openai.com/index/deep-research-system-card/. Retrieved April 9, 2025

    2025

  15. [23]

    Mrinank Sharma, Meg Tong, Jesse Mu, Jerry Wei, Jorrit Kruthoff, Scott Goodfriend, Euan Ong, Alwin Peng, Raj Agarwal, Cem Anil, Amanda Askell, Nathan Bailey, Joe Benton, Emma Bluemke, Samuel R. Bowman, Eric Christiansen, Hoagy Cunningham, Andy Dau, Anjali Gopal, Rob Gilson, Log...

    2025 arXiv

  16. [24]

    Structured access: an emerging paradigm for safe ai deployment, 2022

    Toby Shevlane. Structured access: an emerging paradigm for safe ai deployment, 2022. URL https://arxiv.org/abs/2201.05159

    2022 arXiv

  17. [25]

    Principles for safeguard evaluation

    UK AISI . Principles for safeguard evaluation. Technical report, AI Safety Institute (AI Security Institute), February 2025. URL https://www.aisi.gov.uk/work/principles-for-safeguard-evaluation. Formal technical report

    2025

  18. [26]

    Us aisi and uk aisi joint pre-deployment test: Anthropic's claude 3.5 sonnet (october 2024 release)

    US AISI and UK AISI . Us aisi and uk aisi joint pre-deployment test: Anthropic's claude 3.5 sonnet (october 2024 release). Technical report, National Institute of Standards and Technology and Department of Science Innovation and Technology, oct 2024. Internal test report

    2024

  19. [27]

    Wasil, J

    A. Wasil, J. Clymer, D. Krueger, E. Dardaman, S. Campos, and E. Murphy. Affirmative safety: An approach to risk management for advanced ai. SSRN Electronic Journal, 2024. doi:10.2139/ssrn.4806274

    2024 doi

  20. [28]

    A. Wei, N. Haghtalab, and J. Steinhardt. Jailbroken: How does llm safety training fail? ArXiv.org, July 2023. URL https://doi.org/10.48550/arXiv.2307.02483

    2023 doi arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.