Equity and AI governance at academic medical centers.

ABSTRACT

OBJECTIVES: To understand whether and how equity is considered in artificial intelligence/machine learning governance processes at academic medical centers. STUDY DESIGN: Qualitative analysis of interview data. METHODS: We created a database of academic medical centers from the full list of Association of American Medical Colleges hospital and health system members in 2022. Stratifying by census region and restricting to nonfederal and nonspecialty centers, we recruited chief medical informatics officers and similarly positioned individuals from academic medical centers across the country. We created and piloted a semistructured interview guide focused on (1) how academic medical centers govern artificial intelligence and prediction and (2) to what extent equity is considered in these processes. A total of 17 individuals representing 13 institutions across 4 census regions of the US were interviewed. RESULTS: A minority of participants reported considering inequity, racism, or bias in governance. Most participants conceptualized these issues as characteristics of a tool, using frameworks such as algorithmic bias or fairness. Fewer participants conceptualized equity beyond the technology itself and asked broader questions about its implications for patients. Disparities in health information technology resources across health systems were repeatedly identified as a threat to health equity. CONCLUSIONS: We found a lack of consistent equity consideration among academic medical centers as they develop their governance processes for predictive technologies despite considerable national attention to the ways these technologies can cause or reproduce inequities. Health systems and policy makers will need to specifically prioritize equity literacy among health system leadership, design oversight policies, and promote critical engagement with these tools and their implications to prevent the further entrenchment of inequities in digital health care.