REVIEW 9 cited by
Risk thresholds for frontier AI
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Frontier artificial intelligence (AI) systems could pose increasing risks to public safety and security. But what level of risk is acceptable? One increasingly popular approach is to define capability thresholds, which describe AI capabilities beyond which an AI system is deemed to pose too much risk. A more direct approach is to define risk thresholds that simply state how much risk would be too much. For instance, they might state that the likelihood of cybercriminals using an AI system to cause X amount of economic damage must not increase by more than Y percentage points. The main upside of risk thresholds is that they are more principled than capability thresholds, but the main downside is that they are more difficult to evaluate reliably. For this reason, we currently recommend that companies (1) define risk thresholds to provide a principled foundation for their decision-making, (2) use these risk thresholds to help set capability thresholds, and then (3) primarily rely on capability thresholds to make their decisions. Regulators should also explore the area because, ultimately, they are the most legitimate actors to define risk thresholds. If AI risk estimates become more reliable, risk thresholds should arguably play an increasingly direct role in decision-making.
Forward citations
Cited by 9 Pith papers
-
Towards Frontier Safety Policies Plus
Frontier safety policies should be rebuilt around a standardized taxonomy of precursory capabilities and a mutual feedback mechanism with AI safety cases.
-
Safety case template for frontier AI: A cyber inability argument
A proof-of-concept safety case template formalizes an inability argument for offensive cyber risk using risk models, proxy tasks, and evaluation results.
-
Harmonizing AI Safety Thresholds
The authors propose harmonized AI capability floors: non-zero full-chain TLO cyber completion triggers safeguards, and AI progress at 5× trend for 3 months triggers safeguards, with biorisk left as a diagnostic.
-
Technical Requirements for Halting Dangerous AI Activities
A taxonomy of compute-centric technical interventions, graded by readiness and mapped to five AI governance plans, argues that halting dangerous AI requires substantial control over AI compute.
-
In Which Areas of Technical AI Safety Could Geopolitical Rivals Cooperate?
Based on a four-risk typology, the paper concludes that verification mechanisms and codified protocols are the least risky areas for cooperation between geopolitical rivals on technical AI safety.
-
Robustness tests for biomedical foundation models should tailor to specifications
The paper proposes task-tailored 'robustness specifications' to guide robustness testing of biomedical foundation models across their lifecycle.
-
Governing AI Beyond the Pretraining Frontier
If pretraining scaling plateaus while capabilities keep rising through reasoning models, compute-based triggers in the EU AI Act and US export controls will miss the main sources of risk.
-
Quantifying detection rates for dangerous capabilities: a theoretical model of dangerous capability evaluations
A new model quantifies how test sensitivity, capability growth, and threshold placement determine bias and detection lag in dangerous AI evaluations.
-
What Information Should Be Shared with Whom "Before and During Training"?
A concrete transparency checklist for pre-training disclosure under the Frontier AI Safety Commitments, balancing public sharing against commercial and security risks.
Discussion (0). Continue with ORCID to comment.