The Risk to Population Health Equity Posed by Automated Decision Systems: A Narrative Review

Artificial intelligence is already ubiquitous, and is increasingly being used to autonomously make ever more consequential decisions. However, there has been relatively little research into the existing and possible consequences for population health equity. A narrative review was undertaken using a hermeneutic approach to explore current and future uses of narrow AI and automated decision systems (ADS) in medicine and public health, issues that have emerged, and implications for equity. Accounts reveal a tremendous expectation on AI to transform medical and public health practices. Prominent demonstrations of AI capability - particularly in diagnostic decision making, risk prediction, and surveillance - are stimulating rapid adoption, spurred by COVID-19. Automated decisions being made have significant consequences for individual and population health and wellbeing. Meanwhile, it is evident that hazards including bias, incontestability, and privacy erosion have emerged in sensitive domains such as criminal justice where narrow AI and ADS are in common use. Reports of issues arising from their use in health are already appearing. As the use of ADS in health expands, it is probable that these hazards will manifest more widely. Bias, incontestability, and privacy erosion give rise to mechanisms by which existing social, economic and health disparities are perpetuated and amplified. Consequently, there is a significant risk that use of ADS in health will exacerbate existing population health inequities. The industrial scale and rapidity with which ADS can be applied heightens the risk to population health equity. It is incumbent on health practitioners and policy makers therefore to explore the potential implications of using ADS, to ensure the use of artificial intelligence promotes population health and equity.