The value of standards for health datasets in artificial intelligence-based applications.

Artificial intelligence as a medical device is increasingly being applied to healthcare for diagnosis, risk stratification and resource allocation. However, a growing body of evidence has highlighted the risk of algorithmic bias, which may perpetuate existing health inequity. This problem arises in part because of systemic inequalities in dataset curation, unequal opportunity to participate in research and inequalities of access. This study aims to explore existing standards, frameworks and best practices for ensuring adequate data diversity in health datasets. Exploring the body of existing literature and expert views is an important step towards the development of consensus-based guidelines. The study comprises two parts: a systematic review of existing standards, frameworks and best practices for healthcare datasets; and a survey and thematic analysis of stakeholder views of bias, health equity and best practices for artificial intelligence as a medical device. We found that the need for dataset diversity was well described in literature, and experts generally favored the development of a robust set of guidelines, but there were mixed views about how these could be implemented practically. The outputs of this study will be used to inform the development of standards for transparency of data diversity in health datasets (the STANDING Together initiative).

Qualitative interviews to understand methods and systems used to collect ethnicity information in health administrative data sources in England.

Background: This article is one of a series aiming to inform analytical methods to improve comparability of estimates of ethnic health disparities based on different sources. This article explores the quality of ethnicity data and identifies potential sources of bias when ethnicity information is collected in three key NHS data sources. Future research can build on these findings to explore analytical methods to mitigate biases. Methods: Thematic analysis of semi-structured qualitative interviews to explore potential sources of error and bias in the process of collecting ethnicity information across three NHS data sources: General Practice Extraction Service (GPES) Data for Pandemic Planning and Research (GDPPR), Hospital Episode Statistics (HES) and Improving Access to Psychological Therapies (IAPT). The study included feedback from 22 experts working on different aspects of health admin data collection for England (including staff from NHS Digital, IT system suppliers and relevant healthcare service providers). Results: Potential sources of error and bias were identified across data collection, data processing and quality assurance processes. Similar issues were identified for all three sources. Our analysis revealed three main themes which can result in bias and inaccuracies in ethnicity data recorded: data infrastructure challenges, human challenges, and institutional challenges. Conclusions: Findings highlighted that analysts using health admin data should be aware of the main sources of potential error and bias in health admin data, and be mindful that the main sources of error identified are more likely to affect the ethnicity data for ethnic minority groups. Where possible, analysts should describe and seek to account for this bias in their research.

Rasch analysis of the upper-limb subscale of the stroke rehabilitation assessment of movement (STREAM) tool in an acute stroke cohort Rasch analysis of the upper-limb subscale of the STREAM tool in an acute stroke population.

BACKGROUND: Stroke is a leading cause of disability worldwide. The most common impairment resulting from stroke is upper-limb weakness. OBJECTIVES: To determine the usefulness and psychometric validity of the upper-limb subscale of the STREAM in an acute stroke population. METHODS: Rasch Analysis, including unidimensionality assumption testing, determining model fit, and analysis of: reliability, residual correlations, and differential item functioning. RESULTS: 125 individuals were assessed using the upper-limb subscale of the Stroke Rehabilitation Assessment of Movement (STREAM) tool. Rasch analysis suggests the STREAM is a unidimensional measure. However, when scored using the originally proposed method (0-2), or using the response pattern (0-5) neither variant fit the Rasch model (p < 0.05). Although, the reliability was good (Person-Separation Index - 0.847 and 0.903, respectively). Correcting for the disordered thresholds, and thereby producing the new scoring pattern, led to substantial improvement in the overall fit (chi-square probability of fit - 22%), however, the reliability was slightly reduced (PSI - 0.806). CONCLUSIONS: The study proposes a new scoring method for the upper-limb subscale of the STREAM outcome measure in the acute stroke population.