REVIEW 7 cited by
Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies
read the original abstract
As AI systems demonstrate increasingly strong predictive performance, their adoption has grown in numerous domains. However, in high-stakes domains such as criminal justice and healthcare, full automation is often not desirable due to safety, ethical, and legal concerns, yet fully manual approaches can be inaccurate and time consuming. As a result, there is growing interest in the research community to augment human decision making with AI assistance. Besides developing AI technologies for this purpose, the emerging field of human-AI decision making must embrace empirical approaches to form a foundational understanding of how humans interact and work with AI to make decisions. To invite and help structure research efforts towards a science of understanding and improving human-AI decision making, we survey recent literature of empirical human-subject studies on this topic. We summarize the study design choices made in over 100 papers in three important aspects: (1) decision tasks, (2) AI models and AI assistance elements, and (3) evaluation metrics. For each aspect, we summarize current trends, discuss gaps in current practices of the field, and make a list of recommendations for future research. Our survey highlights the need to develop common frameworks to account for the design and research spaces of human-AI decision making, so that researchers can make rigorous choices in study design, and the research community can build on each other's work and produce generalizable scientific knowledge. We also hope this survey will serve as a bridge for HCI and AI communities to work together to mutually shape the empirical science and computational technologies for human-AI decision making.
Forward citations
Cited by 7 Pith papers
-
RuleEdit: Failure-Guided Human-AI Model Editing with Prospective Impact Preview
Rule-guided mismatch cues raise Human+AI rehab-assessment accuracy by 14% and cut harmful reliance; prospective embedding previews raise local model-edit gains from 11.5% to 36%, with global transfer often regressing.
-
Measuring Progress on Scalable Oversight for Large Language Models
Humans chatting with an unreliable LLM assistant outperform both the model alone and unaided humans on MMLU and time-limited QuALITY tasks.
-
Investigating LLM-Powered Dissenting Minority Support in Power-Imbalanced Group Decision-Making: Counterargument and Mediation as Intervention Strategies
An experiment found LLM counterarguments improved group flexibility and satisfaction while AI mediation boosted minority participation but lowered psychological safety.
-
Bridging Predictions and Interventions: An Integrated Framework for Automated Decision-Systems
Perspective paper proposing an integrated framework for automated decision systems that shifts priority from prediction accuracy to accounting for changes in organizational workflows and intervention effects.
-
"Where is this coming from?" Uncovering Trustworthiness Ideals in AI-powered Peripartum Information Seeking
Qualitative focus-group study finds that trustworthiness in AI for peripartum information must be inspectable rather than asserted, yielding four governance themes: social sensemaking support, pluralistic verification...
-
Resume-ing Control: (Mis)Perceptions of Agency Around GenAI Use in Recruiting Workflows
Recruiters perceive themselves as retaining agency over GenAI in hiring pipelines, yet GenAI invisibly architects core evaluation inputs, producing only marginal efficiency gains at the cost of deskilling.
-
Data-driven Progressive Discovery of Physical Laws
CoSR discovers physical laws via progressive chains of symbolic knowledge units, recovering Kepler-to-Newton and improving scaling laws in convection, pipe flow, laser-metal interaction, and aircraft aerodynamics.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.