Quantifying detection rates for dangerous capabilities: a theoretical model of dangerous capability evaluations

Alessandro Di Stefano; Paolo Bova; The Anh Han

REVIEW 5 major objections 5 minor 31 references

Quantifying detection rates for dangerous capabilities: a theoretical model of dangerous capability evaluations

T0 review · 5 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper models dangerous capability testing as a reversed-hazard process driven by a test sensitivity rate r(y), yielding closed-form formulas for estimator bias, threshold-detection likelihood, and expected lag time.

desk verdict A tractable reversed-hazard formalization of dangerous-capability evals, honestly self-limited as a scenario tool rather than a measurement method; deserves a serious referee despite the bias-sign slip and the unestimated r(y). read the letter →

arxiv 2412.15433 v1 pith:VPQ63SJ7 submitted 2024-12-19 cs.AI cs.CYcs.MAecon.GNq-fin.ECstat.AP

Paolo Bova , Alessandro Di Stefano , The Anh Han This is my paper · ORCID

classification cs.AIcs.CYcs.MAecon.GNq-fin.ECstat.AP MSC 62N0162N0268T0191B06

keywords dangerouscapabilityevaluationsAIsafetyreversehazardratetestsensitivitythresholdsracedynamicsestimatorbiasdetectionlag

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to quantify how well dangerous capability testing can track rising AI risks. It argues that the outcome of an evaluation—the highest danger level a system is detected to be capable of—can be described by a single 'test sensitivity rate' r(y), the rate at which tests detect each danger level given that all more-dangerous tests failed. From this rate the paper derives a closed-form distribution for the estimated danger, and from that distribution it computes three policy-relevant numbers: the bias in danger estimates as capabilities grow, the likelihood of detecting a crossing of a danger threshold, and the expected lag time before a crossing is noticed. The authors show through simulations that gaps in testing sensitivity and competitive pressure between labs translate into either growing bias or long monitoring lags, and they argue the model is simple enough to embed in larger models of AI race dynamics and governance.

What carries the argument

The central object is the test sensitivity rate r(y), defined as the conditional rate at which tests detect that a system can achieve danger level y, given that no more-dangerous test has passed. This is exactly the reverse-hazard (or accumulation) rate of the estimator distribution, and it acts as the input that determines everything else. With r(y) in hand, the CDF of the estimator is F(\hat{y})=\exp(-\int_{\hat{y}}^{y_t} r(u)du), from which the paper derives bias, detection likelihood, and expected lag; for piecewise-constant r(y), closed-form expressions for all three follow directly. The model also includes an incremental-testing update rule and a linear production function for test sensitivity, which together drive the simulations of long-term test-suite building.

What would settle it

Run repeated evaluations of the same frontier model at a fixed true danger y_t and record the maximum detected danger \hat{y}; if the empirical distribution of these maxima does not match F(\hat{y})=\exp(-\int_{\hat{y}}^{y_t} r(u)du) for any non-negative integrable r(y), or if adding high-sensitivity tests at a severity above the current estimate fails to shift the distribution as the formula predicts, the model's core claim fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that dangerous capability evaluation can be modelled as a reversed-hazard process. Let y_t be the true current danger level and let r(y) be the test sensitivity rate. Then the estimator \hat{y}, the supremum of detected danger, has cumulative distribution function F(\hat{y}) = \exp(-\int_{\hat{y}}^{y_t} r(u)\,du), with any tests above y_t automatically failing. This single formula yields the estimator bias E[\hat{y}|y_t]-y_t, the threshold detection likelihood 1-F(y^*|\hat{y}\le y_t), and an expected lag time for detecting a threshold crossing. The authors further show that when the true danger grows, two failure modes dominate—bias that can rise abruptly at higher capability levels, and large lags when testing is concentrated before a threshold but not after it—and they identify market competition and uncertainty about capability dynamics as the main drivers of both.

Load-bearing premise

The load-bearing premise is that a single, well-defined test sensitivity rate r(y) exists for each danger level y and can be estimated from real evaluations; the paper's own Section 4.5 states that inferring this rate from actual evaluations is challenging and that the assumptions 'are unlikely to hold in practise.' If r(y) cannot be measured, the bias and lag numbers remain illustrative rather than predictive.

Editorial extensions

If this is right

If the model is right, any quantitative model of AI race dynamics or AI governance can incorporate evaluation quality as a one-parameter family of test sensitivity functions without adding computational complexity.
A fixed per-time-step testing budget should balance investment in higher-severity tests with tests near the current estimated frontier, because concentrating on either goal alone produces unbounded bias or unbounded lag.
Delays in building high-quality test suites compound: the later testing starts, the more investment per unit time is needed to reach even a moderate chance of detecting a threshold crossing.
Competitive pressure that shortens evaluation windows translates directly into lower test sensitivity at higher danger levels, producing an s-shaped bias curve that can mislead policymakers into believing dangers are well understood.
Failure modes are separable: bias failures come from testing too little at high severities, while lag failures come from testing too little immediately after the threshold.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If r(y) could be estimated from calibration data on real evaluations, the closed-form CDF would yield testable predictions, e.g., the shape of the empirical distribution of maximum detected danger for repeated evaluations of the same model.
The reversed-hazard structure implies that what matters for threshold detection is the cumulative sensitivity below the true danger level, so a single large gap in test coverage can outweigh uniformly low sensitivity; this suggests coverage continuity should be a headline metric for evaluation ecosystems.
The same machinery could be applied to an upper-bound (infimum) estimator, which the authors flag as future work, and to multivariate risk combinations where several dangerous capabilities interact.
One could turn the model into a monitoring tool by fitting r(y) from historical evaluation reports and computing expected lags for proposed thresholds, which would give regulators a quantitative reason to choose one threshold placement over another.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this is a genuinely useful formalization of dangerous-capability evaluations as a reversed-hazard process, and the paper is unusually honest about where it breaks. The core distribution result is textbook survival analysis—the reversed-hazard identity—so the novelty is the application, not the math. But the application is real: framing test sensitivity as r(y), then deriving estimator bias, threshold detection likelihood, and expected lag gives the AI safety community a shared vocabulary and a plug-in module for race and governance models. The piecewise step-function corollary makes simulation easy, and the Section 4 scenarios illustrate the two failure modes—bias and lag—clearly.

Credit where due: Appendix A1's derivation of the CDF is correct, and Section 4.5 explicitly concedes that mapping real evaluations to r is challenging and that the model's assumptions are unlikely to hold in practice. That candor is rare and should count in the paper's favor.

Soft spots, in order of severity. First, the bias sign error in Section 3.2: with the stated truncation, E[ŷ|y_t] ≤ y_t, so Bias = E[ŷ|y_t] − y_t is non-positive, not positive. The footnote about false positives suggests the authors may have intended a different convention, but as written it is a contradiction. Easy fix, but it propagates into the verbal reading of Figure 2b. Second, the expected-lag formula in Section 3.2 is under-specified: s(t_lag) is never defined and the integration limits are ambiguous. Third, and most important, r(y) is a free function with no estimation strategy. The quantitative outputs—bias magnitudes, lag times, miss probabilities—are therefore illustrative, not measurements. The paper itself says this in Section 4.5, which partially defuses the stress-test note, but it does undercut the title's promise of quantification. Also, the "for the first time" claim in the introduction overstates the case given the standard identity behind Theorem 3.1, and the code is only "available upon request," not actually shipped.

Who is this for? Researchers building formal models of AI governance and race dynamics who want a tractable representation of evals; also anyone teaching why testing failures appear as bias and lag. It is not yet a quantitative method for estimating detection rates from real evaluation data—that remains open, as the authors acknowledge.

Recommendation: send to peer review. The framework is novel in its application, the authors are transparent about limitations, and the correctable issues (bias sign, lag formula, code release) are exactly what a good referee can pin down. Treat this as a revise-and-resubmit, not a desk reject.

Referee Report

5 major / 5 minor

Summary. The paper proposes a reversed-hazard model of dangerous capability evaluations. It defines a single-dimension danger severity y, a test-sensitivity rate r(y), and the estimator ŷ as the highest severity at which a test passes. Assuming tests above the true capability y_t automatically fail and no false positives occur, it derives the CDF F(ŷ)=exp(-∫_{ŷ}^{y_t} r(u)du) and uses it to compute estimator bias, threshold-crossing detection likelihood, and expected detection lag, with piecewise-constant r for tractability. The authors simulate one- and two-block test suites, discuss market and technical barriers, and offer policy recommendations.

Significance. If the technical issues are corrected, the framework is a useful modular representation: the reversed-hazard parameterization is standard but well suited to lower-bound estimators, the piecewise CDF is analytically tractable, and the qualitative failure modes (growing underestimation bias and threshold-detection lag) are clearly illustrated. The paper is commendably transparent in Section 4.5 about the gap between the model and real evaluations. However, the claimed quantification is not currently established because r(y) is not estimated from data; the contribution is best read as a scenario-analysis tool unless an estimation strategy is added. The model could nonetheless be embedded in AI-race dynamic models, as the authors note.

major comments (5)

[Section 3.2, Bias definition and footnote 13] The definition Bias = E[ŷ|y_t] − y_t, together with the assumption that tests above y_t automatically fail and false positives are excluded, implies that ŷ ≤ y_t almost surely, hence Bias ≤ 0. The paper asserts that 'the Bias will be positive' and plots a positive gap in Figures 2b and 3b. This sign inconsistency affects every subsequent statement about bias. Please either define the bias as y_t − E[ŷ|y_t] or explicitly call the plotted quantity the absolute underestimation bias, and carry that convention through the text and figures.
[Section 3.2, item 3 (expected lag time)] The displayed formula for E_S[t_lag] is incomplete: the quantity s(t_lag) is never defined, the integration limits t_{e_l} to t_{e_{l−1}} run from the later to the earlier endpoint, and no derivation from the CDF or from the capability schedule y_t(t) is provided. Since the expected lag values in Section 4.1.1 are central illustrative outputs, the paper needs a precise first-passage-time definition and a correct, derived expression before those numbers can be verified.
[Theorem 3.1 and Appendix A1] Appendix A1 starts from f(Y=y)=r(y)·Pr(Y≤y), which is the definition of the reversed hazard rate rather than a consequence of Propositions 1–3. Theorem 3.1 is therefore a parameterization of the estimator distribution, not an independent derivation from first principles. Moreover, the equivalence asserted in A1.1 requires the conditional-independence condition that failing all tests above y gives no information about passing test y; this condition should be stated as an explicit modeling axiom, because without it r(y) cannot be treated as a primitive. I recommend reframing the theorem as a representation result and listing this assumption explicitly.
[Corollary 3.1.1] The piecewise CDF formula F(ŷ)=exp(−k_l(e_l−ŷ)−Σ_{j>l} k_j(e_j−e_{j−1})) uses the fixed segment endpoints e_j as upper limits, but Theorem 3.1's CDF is truncated at the current capability y_t. Unless y_t always coincides with a segment endpoint, the printed formula is incorrect; for example, in the one-block case with y_t<10 it gives exp(−k(10−ŷ)) instead of exp(−k(y_t−ŷ)). The simulations in Section 4 appear to use the latter, so the corollary and the implementation must be reconciled.
[Section 4.5 and title claim] The title and abstract promise quantitative detection rates, but Section 4.5 states that inferring test-sensitivity rates from real evaluations is challenging and that the model's assumptions are unlikely to hold in practice; no estimator, bounds, or calibration procedure for r(y) is provided. All numerical outputs in Section 4 are therefore functions of arbitrarily chosen r(y). This is acceptable for a qualitative or scenario model, but the quantification claim needs to be either supported by an estimation protocol or explicitly downgraded.

minor comments (5)

[Throughout] The phrase 'cumulative density function' should be 'cumulative distribution function' wherever it appears.
[Throughout] There are numerous typos, including 'appreicate', 'capibilities', 'quantative', 'sentitivity', 'threhsold', and 'practise'; these should be corrected before publication.
[Figure 3 caption] The caption says '1 test block' but the figure describes a two-block scenario; this should be corrected.
[Section 4.1.1] The notation is inconsistent: y_t is used both for the current hidden capability and for the danger threshold (e.g., 'at yt = 5' for the threshold). Please use y* consistently for the threshold.
[Section 4.1.1] The expected lag values of 0.5 and 1.5 are reported without stating the detection-rate parameters used; please give the exact k values or parameter settings so the numbers can be reproduced.

Circularity Check

1 steps flagged · score 6.0 of 10

Theorem 3.1 restates the definition of r(y) as the reversed hazard rate of the estimator, so the 'derivation from first principles' is a parameterization and all numeric detection-rate outputs reduce to the chosen input function.

self definitional [Section 3 (Proposition 2), Section 3.1 (Theorem 3.1), Appendix A1.4]
"Proposition 2 We can define a 'test sensitivity' function r(y) that represents the rate at which we detect an AI system can achieve a level of danger y, conditional on ignoring any tests that the system could be more dangerous than y. ... From the expression for F (y), we can see that r(y) is indeed the reversed hazard rate (or accumulation rate) of F (y): r(y) = d/dy ln F (y) = f(y)/F(y), which is the definition of the reversed hazard rate."

Theorem 3.1 states F(ŷ) = exp(-∫_{ŷ}^{yt} r(u) du). The appendix derives this from f(Y=y) = r(y) F(y), which is exactly the definition of the reversed hazard rate of the estimator ŷ. Proposition 2 already defines r(y) as this conditional detection rate, so the 'derivation from first principles' is a parameterization: once r is chosen, the estimator CDF is forced by construction. All downstream quantities—bias in Section 3.2, threshold detection likelihood, and expected lag in Section 4.1.1—are deterministic functions of the chosen r shape.

full rationale

The paper's internal mathematics is coherent: Theorem 3.1 correctly solves the separable ODE f(y) = r(y)F(y). The circularity is that this ODE is not an independent first-principles law; it is the definition of r as the reversed hazard rate of the estimator, which Proposition 2 introduced as 'the rate at which we detect ... conditional on ignoring any tests that the system could be more dangerous than y.' Thus the CDF is a restatement of the definition of r, and the bias, threshold-detection likelihood, and lag results reduce to the chosen input function r(y). The paper is transparent about the illustrative nature of its numbers and does not fit empirical constants or claim external validation; the qualitative failure modes follow from the shapes of r in a self-consistent way. However, Section 4.5 explicitly says that mapping real evaluations to r is challenging and that the assumptions are unlikely to hold in practice, so the headline ability to 'quantify' detection rates is not yet supported by a measurement strategy. This is a partial, definitional circularity rather than a fully forced self-citation chain: the framework is a useful reversed-hazard parameterization, but the central quantitative claim reduces by construction to an input that is not independently estimated.

Assumptions & free parameters 5 free parameters · 6 assumptions · 0 invented entities

The model rests on several explicitly stated domain assumptions (single-dimensional danger, existence of r(y), supremum estimator, no false positives) and on one standard-math identity (reversed hazard determines the CDF). No empirical data are fitted; the parameter values in all figures are illustrative, which keeps the circularity burden low but means the model is not validated.

free parameters (5)

Test sensitivity block values r_l = e.g., r=2.0 vs 1.0 in Figures 2-3; r=0.1 in the upper block of Figure 3
Piecewise constant detection rates chosen for illustration; bias, detection likelihood, and lag all depend on these values.
Danger threshold y* = 5, varied in Section 4.1.5
Chosen by hand to sit in the middle of the test range; moving it changes detection-lag conclusions.
Maximum danger level y_max = 10
Illustrative upper end of the severity range; affects the scale of integrals and lag units.
Capability growth schedule y_t(t) = linear +1 per time step in Section 4.1.1; declining investment schedule in Figure 6
Expected lag time depends on the assumed dynamics of latent danger; the Figure 6 schedule is not fully specified.
Test block endpoints e_j = e.g., e1=6 in the two-block scenarios
Positions of gaps in test coverage are chosen to produce the failure modes being illustrated.

assumptions (6)

domain assumption Danger can be measured along a single ordered dimension y, and tests can be ordered by severity.
Proposition 1, Section 3. Multi-dimensional risks such as scheming, CBNR assistance, and collusion are collapsed into one axis; the paper acknowledges combination risks in Section 5.
domain assumption Each danger level y has a fixed test sensitivity rate r(y), interpretable as the reversed hazard of the estimator.
Proposition 2, Section 3. All formulas are functions of r; Section 4.5 admits that estimating r from real evals is unresolved.
domain assumption The estimator is the supremum of passing tests; tests above the latent y_t fail automatically, and false positives are ignored.
Proposition 3 and Theorem 3.1, Section 3.1; footnote 13 acknowledges false positives are assumed away for tractability.
ad hoc to paper Test sensitivity is a piecewise step function for tractability.
Corollary 3.1.1, Section 3.1; no justification is given that real test suites have constant sensitivity within blocks.
domain assumption Incremental test results persist over time, and new tests at the same severity are independent of old tests.
Section 3.3.1; used for the update dynamics, though not in most of the reported simulations.
standard math A nonnegative integrable r defines a valid CDF via F(y)=exp(-∫r).
Appendix A1; this is the standard reversed-hazard identity and is correct as a mathematical statement.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Quantifying detection rates for dangerous capabilities: a theoretical model of dangerous capability evaluations." pith.science (2026). https://pith.science/paper/VPQ63SJ7

@misc{pith2026241215433,
  author       = {Pith},
  title        = {Pith review of: Quantifying detection rates for dangerous capabilities: a theoretical model of dangerous capability evaluations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VPQ63SJ7}},
  note         = {Machine review of arXiv:2412.15433}
}

read the original abstract

We present a quantitative model for tracking dangerous AI capabilities over time. Our goal is to help the policy and research community visualise how dangerous capability testing can give us an early warning about approaching AI risks. We first use the model to provide a novel introduction to dangerous capability testing and how this testing can directly inform policy. Decision makers in AI labs and government often set policy that is sensitive to the estimated danger of AI systems, and may wish to set policies that condition on the crossing of a set threshold for danger. The model helps us to reason about these policy choices. We then run simulations to illustrate how we might fail to test for dangerous capabilities. To summarise, failures in dangerous capability testing may manifest in two ways: higher bias in our estimates of AI danger, or larger lags in threshold monitoring. We highlight two drivers of these failure modes: uncertainty around dynamics in AI capabilities and competition between frontier AI labs. Effective AI policy demands that we address these failure modes and their drivers. Even if the optimal targeting of resources is challenging, we show how delays in testing can harm AI policy. We offer preliminary recommendations for building an effective testing ecosystem for dangerous capabilities and advise on a research agenda.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

31 extracted references · 9 canonical work pages

[1]

Alaga, J., & Schuett, J. (2023). Coordinated pausing: An evaluation-based coordination scheme for frontier ai devel- opers. https://arxiv.org/abs/2310.00374

work page Pith review arXiv 2023
[2]

Armstrong, S., Bostrom, N., & Shulman, C. (2016). Racing to the precipice: A model of artificial intelligence devel- opment. AI & SOCIETY, 31(2), 201–206. https://doi.org/10.1007/s00146-015-0590-y

doi:10.1007/s00146-015-0590-y 2016
[3]

Askell, A., Brundage, M., & Hadfield, G. (2019). The Role of Cooperation in Responsible AI Development. arXiv. https://doi.org/10.48550/arXiv.1907.04534

arXiv doi:10.48550/arxiv.1907.04534 2019
[4]

Bengio, Y ., Hinton, G., Yao, A., Song, D., Abbeel, P., et al. (2024). Managing extreme AI risks amid rapid progress. Science, 384(6698), 842–845. https://doi.org/10.1126/science.adn0117

doi:10.1126/science.adn0117 2024
[5]

Bengio, Y ., Mindermann, S., Privitera, D., Besiroglu, T., Bommasani, R., et al. (2024). International Scientific Report on the Safety of Advanced AI (Interim Report). https://doi.org/10.48550/arXiv.2412.05282

arXiv doi:10.48550/arxiv.2412.05282 2024
[6]

Benton, J., Wagner, M., Christiansen, E., Anil, C., Perez, E., et al. (2024). Sabotage Evaluations for Frontier Models. ArXiv. https://doi.org/10.48550/arXiv.2410.21514

arXiv doi:10.48550/arxiv.2410.21514 2024
[7]

Bova, P., Di Stefano, A., & Han, T. A. (2024). Both eyes open: Vigilant incentives help auditors improve ai safety. Journal of Physics: Complexity, 5(2), 025009

work page 2024
[8]

D., Sett, G., Koessler, L., Schuett, J., & Anderljung, M

Buhl, M. D., Sett, G., Koessler, L., Schuett, J., & Anderljung, M. (2024). Safety cases for frontier AI. https://doi.org/ 10.48550/arXiv.2410.21572

arXiv doi:10.48550/arxiv.2410.21572 2024

Show all 31 references

[9]

Cimpeanu, T., Di Stefano, A., Perret, C., & Han, T. A. (2023). Social diversity reduces the complexity and cost of fostering fairness. Chaos, Solitons & Fractals, 167, 113051. https://doi.org/https://doi.org/10.1016/j.chaos. 2022.113051

2023
[10]

Cottier, B., Besiroglu, T., & Owen, D. (2023). Who is leading in ai? an analysis of industry ai research

2023
[11]

"., Skalse, J., Bengio, Y ., Russell, S., Tegmark, M., et al

Dalrymple, D. "., Skalse, J., Bengio, Y ., Russell, S., Tegmark, M., et al. (2024). Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems. https://doi.org/10.48550/arXiv.2405.06624

2024 doi arXiv
[12]

D., Schuett, J., Korbak, T., Wang, J., et al

Goemans, A., Buhl, M. D., Schuett, J., Korbak, T., Wang, J., et al. (2024). Safety case template for frontier AI: A cyber inability argument. https://doi.org/10.48550/arXiv.2411.08088

2024 doi arXiv
[13]

Gruetzemacher, R., Chan, A., Frazier, K., Manning, C., Los, Š., et al. (2023). An International Consortium for Evalu- ations of Societal-Scale Risks from Advanced AI. https://doi.org/10.48550/arXiv.2310.14455

2023 doi arXiv
[14]

A., Pereira, L

Han, T. A., Pereira, L. M., Santos, F. C., & Lenaerts, T. (2020). To Regulate or Not: A Social Dynamics Analysis of an Idealised AI Race. Journal of Artificial Intelligence Research, 69, 881–921. https://doi.org/10.1613/jair.1. 12225

2020 doi
[15]

A., Lenaerts, T., Santos, F

Han, T. A., Lenaerts, T., Santos, F. C., & Pereira, L. M. (2022). V oluntary safety commitments provide an escape from over-regulation in AI development. Technology in Society, 68, 101843

2022
[16]

Hendrycks, D., Carlini, N., Schulman, J., & Steinhardt, J. (2022). Unsolved problems in ml safety. https://arxiv.org/ abs/2109.13916

2022 arXiv
[17]

Hendrycks, D., Mazeika, M., & Woodside, T. (2023). An Overview of Catastrophic AI Risks. https://doi.org/10. 48550/arXiv.2306.12001

2023 doi arXiv
[18]

Ho, L., Barnhart, J., Trager, R., Bengio, Y ., Brundage, M., et al. (2023). International Institutions for Advanced AI. https://doi.org/10.48550/arXiv.2307.04699 Højmark, A., Pimpale, G., Panickssery, A., Hobbhahn, M., & Scheurer, J. (2024). Analyzing Probabilistic Methods for...

2023 doi arXiv
[19]

Jensen, M., Emery-Xu, N., & Trager, R. (2023). Industrial policy for advanced ai: Compute pricing and the safety tax. https://doi.org/10.48550/ARXIV .2302.11436

2023 doi arXiv
[20]

Kinniment, M., Sato, L. J. K., Du, H., Goodrich, B., Hasin, M., et al. (2024). Evaluating Language-Model Agents on Realistic Autonomous Tasks. https://doi.org/10.48550/arXiv.2312.11671

2024 doi arXiv
[21]

Koessler, L., Schuett, J., & Anderljung, M. (2024). Risk thresholds for frontier AI. https://doi.org/10.48550/arXiv. 2406.14713

2024 doi arXiv
[22]

Krakovna, V ., Uesato, J., Mikulik, V ., Rahtz, M., Everitt, T., et al. (2020). Specification gaming: The flip side of AI ingenuity [Retrieved February 2023 from https://deepmind.com/blog/article/Specification-gaming-the-flip- side-of-AI-ingenuity]. METR. (2023). Responsible S...

2020
[23]

R., Baranchuk, M., Strohmeier, M., Bolina, V ., Torr, P

Motwani, S. R., Baranchuk, M., Strohmeier, M., Bolina, V ., Torr, P. H. S., et al. (2024). Secret collusion among generative ai agents. https://arxiv.org/abs/2402.07510 NIST. (2024). FACT SHEET: U.S. Department of Commerce & U.S. Department of State Launch the International Ne...

2024 arXiv
[24]

S., Goldstein, S., O’Gara, A., Chen, M., & Hendrycks, D

Park, P. S., Goldstein, S., O’Gara, A., Chen, M., & Hendrycks, D. (2024). AI deception: A survey of examples, risks, and potential solutions. PATTER, 5(5). https://doi.org/10.1016/j.patter.2024.100988

2024
[25]

Phuong, M., Aitchison, M., Catt, E., Cogan, S., Kaskasoli, A., et al. (2024). Evaluating Frontier Models for Dangerous Capabilities. https://doi.org/10.48550/arXiv.2403.13793

2024 doi arXiv
[26]

Ren, R., Basart, S., Khoja, A., Gatti, A., Phan, L., et al. (2024). Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress? https://doi.org/10.48550/arXiv.2407.21792

2024 doi arXiv
[27]

Reuel, A., Bucknall, B., Casper, S., Fist, T., Soder, L., et al. (2024). Open Problems in Technical AI Governance. CoRR. Retrieved December 18, 2024, from https://openreview.net/forum?id=GzdVEGq7Qj

2024
[28]

Sevilla, J. (2023). Please Report Your Compute

2023
[29]

Sevilla, J., Besiroglu, T., Cottier, B., You, J., Roldán, E., et al. (2024). Can ai scaling continue through 2030? [Ac- cessed: 2024-12-18]. https://epoch.ai/blog/can-ai-scaling-continue-through-2030

2024
[30]

Shevlane, T., Farquhar, S., Garfinkel, B., Phuong, M., Whittlestone, J., et al. (2023). Model evaluation for extreme risks. https://doi.org/10.48550/arXiv.2305.15324 UK AI Safety Institute. (2024a). Pre-Deployment Evaluation of Anthropic’s Upgraded Claude 3.5 Sonnet. Retrieved...

2023 doi arXiv
[31]

Whittlestone, J., & Clark, J. (2021). Why and How Governments Should Monitor AI Development. arXiv. https : //doi.org/10.48550/arXiv.2108.12427 26

2021 doi arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

Alaga, J., & Schuett, J. (2023). Coordinated pausing: An evaluation-based coordination scheme for frontier ai devel- opers. https://arxiv.org/abs/2310.00374

work page Pith review arXiv 2023

[2] [2]

Armstrong, S., Bostrom, N., & Shulman, C. (2016). Racing to the precipice: A model of artificial intelligence devel- opment. AI & SOCIETY, 31(2), 201–206. https://doi.org/10.1007/s00146-015-0590-y

doi:10.1007/s00146-015-0590-y 2016

[3] [3]

Askell, A., Brundage, M., & Hadfield, G. (2019). The Role of Cooperation in Responsible AI Development. arXiv. https://doi.org/10.48550/arXiv.1907.04534

arXiv doi:10.48550/arxiv.1907.04534 2019

[4] [4]

Bengio, Y ., Hinton, G., Yao, A., Song, D., Abbeel, P., et al. (2024). Managing extreme AI risks amid rapid progress. Science, 384(6698), 842–845. https://doi.org/10.1126/science.adn0117

doi:10.1126/science.adn0117 2024

[5] [5]

Bengio, Y ., Mindermann, S., Privitera, D., Besiroglu, T., Bommasani, R., et al. (2024). International Scientific Report on the Safety of Advanced AI (Interim Report). https://doi.org/10.48550/arXiv.2412.05282

arXiv doi:10.48550/arxiv.2412.05282 2024

[6] [6]

Benton, J., Wagner, M., Christiansen, E., Anil, C., Perez, E., et al. (2024). Sabotage Evaluations for Frontier Models. ArXiv. https://doi.org/10.48550/arXiv.2410.21514

arXiv doi:10.48550/arxiv.2410.21514 2024

[7] [7]

Bova, P., Di Stefano, A., & Han, T. A. (2024). Both eyes open: Vigilant incentives help auditors improve ai safety. Journal of Physics: Complexity, 5(2), 025009

work page 2024

[8] [8]

D., Sett, G., Koessler, L., Schuett, J., & Anderljung, M

Buhl, M. D., Sett, G., Koessler, L., Schuett, J., & Anderljung, M. (2024). Safety cases for frontier AI. https://doi.org/ 10.48550/arXiv.2410.21572

arXiv doi:10.48550/arxiv.2410.21572 2024