REVIEW 6 cited by
Reasons to Doubt the Impact of AI Risk Evaluations
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
AI safety practitioners invest considerable resources in AI system evaluations, but these investments may be wasted if evaluations fail to realize their impact. This paper questions the core value proposition of evaluations: that they significantly improve our understanding of AI risks and, consequently, our ability to mitigate those risks. Evaluations may fail to improve understanding in six ways, such as risks manifesting beyond the AI system or insignificant returns from evaluations compared to real-world observations. Improved understanding may also not lead to better risk mitigation in four ways, including challenges in upholding and enforcing commitments. Evaluations could even be harmful, for example, by triggering the weaponization of dual-use capabilities or invoking high opportunity costs for AI safety. This paper concludes with considerations for improving evaluation practices and 12 recommendations for AI labs, external evaluators, regulators, and academic researchers to encourage a more strategic and impactful approach to AI risk assessment and mitigation.
Forward citations
Cited by 6 Pith papers
-
Robust ML Auditing using Prior Knowledge
A private labeled-data prior gives an auditor a calculable probability of catching a platform that manipulates its answers to appear fair, while any public prior can be exploited.
-
AI Governance to Avoid Extinction: The Strategic Landscape and Actionable Research Questions
A MIRI governance agenda argues for an internationally coordinated halt to dangerous AI development and catalogs around 400 research questions across four strategic scenarios.
-
Audit Cards: Contextualizing AI Evaluations
A proposed 'audit card' reporting standard would require AI evaluators to disclose auditor identity, scope, methods, access, integrity, and review, filling gaps found in existing reports and frameworks.
-
GPAI Evaluations Standards Taskforce: Towards Effective AI Governance
The paper proposes an EU GPAI Evaluation Standards Taskforce to develop adaptive standards for AI evaluations, based on four desiderata: internal validity, external validity, reproducibility, and portability.
-
Declare and Justify: Explicit assumptions in AI evaluations are necessary for effective regulation
AI evaluation-based regulation should require developers to state and justify key assumptions, and halt development when those justifications are inadequate.
-
Position Paper: Model Access should be a Key Concern in AI Governance
Model access decisions should be studied and coordinated through a dedicated research field, with recommendations for evaluators, companies, governments, and international bodies.
Discussion (0). Continue with ORCID to comment.