REVIEW 4 cited by
Predictable Artificial Intelligence
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
We introduce the fundamental ideas and challenges of Predictable AI, a nascent research area that explores the ways in which we can anticipate key validity indicators (e.g., performance, safety) of present and future AI ecosystems. We argue that achieving predictability is crucial for fostering trust, liability, control, alignment and safety of AI ecosystems, and thus should be prioritised over performance. We formally characterise predictability, explore its most relevant components, illustrate what can be predicted, describe alternative candidates for predictors, as well as the trade-offs between maximising validity and predictability. To illustrate these concepts, we bring an array of illustrative examples covering diverse ecosystem configurations. Predictable AI is related to other areas of technical and non-technical AI research, but have distinctive questions, hypotheses, techniques and challenges. This paper aims to elucidate them, calls for identifying paths towards a landscape of predictably valid AI systems and outlines the potential impact of this emergent field.
Forward citations
Cited by 4 Pith papers
-
Observatorio Lazaro: A self-populating database of anglicism usage in the Spanish press
A six-year, open, daily-updated database of over two million unassimilated English borrowings in the Spanish press, with an honest evaluation of its detector's real-world precision.
-
11Plus-Bench: Demystifying Multimodal LLM Spatial Reasoning with Cognitive-Inspired Analysis
A new spatial reasoning benchmark shows current multimodal models lag humans badly and lack the item-level predictability humans show.
-
Model Performance-Guided Evaluation Data Selection for Effective Prompt Optimization
An evaluation-set selection method that adds real-time model feedback to semantic sampling improves the accuracy and stability of three prompt optimization methods on two datasets.
-
AI Behavioral Science
A position paper outlines a research agenda for AI Behavioral Science built on three pillars: assessing AI behavior, using AI as a behavioral science tool, and understanding human-AI ecosystems.
Discussion (0). Continue with ORCID to comment.