Ethical and legal considerations influencing human involvement in the implementation of artificial intelligence in a clinical pathway: A multi-stakeholder perspective.

Introduction: Ethical and legal factors will have an important bearing on when and whether automation is appropriate in healthcare. There is a developing literature on the ethics of artificial intelligence (AI) in health, including specific legal or regulatory questions such as whether there is a right to an explanation of AI decision-making. However, there has been limited consideration of the specific ethical and legal factors that influence when, and in what form, human involvement may be required in the implementation of AI in a clinical pathway, and the views of the wide range of stakeholders involved. To address this question, we chose the exemplar of the pathway for the early detection of Barrett's Oesophagus (BE) and oesophageal adenocarcinoma, where Gehrung and colleagues have developed a "semi-automated", deep-learning system to analyse samples from the CytospongeTM TFF3 test (a minimally invasive alternative to endoscopy), where AI promises to mitigate increasing demands for pathologists' time and input. Methods: We gathered a multidisciplinary group of stakeholders, including developers, patients, healthcare professionals and regulators, to obtain their perspectives on the ethical and legal issues that may arise using this exemplar. Results: The findings are grouped under six general themes: risk and potential harms; impacts on human experts; equity and bias; transparency and oversight; patient information and choice; accountability, moral responsibility and liability for error. Within these themes, a range of subtle and context-specific elements emerged, highlighting the importance of pre-implementation, interdisciplinary discussions and appreciation of pathway specific considerations. Discussion: To evaluate these findings, we draw on the well-established principles of biomedical ethics identified by Beauchamp and Childress as a lens through which to view these results and their implications for personalised medicine. Our findings are not only relevant to this context but have implications for AI in digital pathology and healthcare more broadly.

Ethical and legal implications of implementing risk algorithms for early detection and screening for oesophageal cancer, now and in the future.

BACKGROUND: Oesophageal cancer has significant morbidity and mortality but late diagnosis is common since early signs of disease are frequently misinterpreted. Project DELTA aims to enable earlier detection and treatment through targeted screening using a novel risk prediction algorithm for oesophageal cancer (incorporating risk factors of Barrett's oesophagus including prescriptions for acid-reducing medications (CanPredict)), together with a non-invasive, low-cost sampling device (CytospongeTM). However, there are many barriers to implementation, and this paper identifies key ethical and legal challenges to implementing these personalised prevention strategies for Barrett's oesophagus/oesophageal cancer. METHODS: To identify ethical and legal issues relevant to the deployment of a risk prediction tool for oesophageal cancer into primary care, we adopted an interdisciplinary approach, incorporating targeted informal literature reviews, interviews with expert collaborators, a multidisciplinary workshop and ethical and legal analysis. RESULTS: Successful implementation raises many issues including ensuring transparency and effective risk communication; addressing bias and inequity; managing resources appropriately and avoiding exceptionalism. Clinicians will need support and training to use cancer risk prediction algorithms, ensuring that they understand how risk algorithms supplement rather than replace medical decision-making. Workshop participants had concerns about liability for harms arising from risk algorithms, including from potential bias and inequitable implementation. Determining strategies for risk communication enabling transparency but avoiding exceptionalist approaches are a significant challenge. Future challenges include using artificial intelligence to bolster risk assessment, incorporating genomics into risk tools, and deployment by non-health professional users. However, these strategies could improve detection and outcomes. CONCLUSIONS: Novel pathways incorporating risk prediction algorithms hold considerable promise, especially when combined with low-cost sampling. However immediate priorities should be to develop risk communication strategies that take account of using validated risk algorithms, and to ensure equitable implementation. Resolving questions about liability for harms arising should be a longer-term objective.

How can we address the uncertainties regarding the potential clinical utility of polygenic score-based tests?

As common low penetrance variants associated with diseases are uncovered, attempts continue to be made to harness this knowledge for improving healthcare. Polygenic scores have been developed as the mechanism by which knowledge of common variants can be used to investigate genetic contributions to disease risk. They serve as a biomarker to provide an estimate of the genetic liability for a particular disease. Discussion continues as to whether polygenic scores are a useful biomarker and their readiness for incorporation into clinical and public health practice. In this paper, we investigate the key challenges that need to be addressed, in the description and assessment of the clinical utility of polygenic score-based tests for use in clinical and public health practice.

The risk of developing many common diseases, such as heart disease is influenced by both genetic and lifestyle factors. Polygenic scores (PGS) are one way of assessing an individual's risk of developing certain diseases. There is still uncertainty as to whether and how to use PGS for individual care. Much of this is because it is unclear as to whether tests that give a PGS can provide useful information for the care of individuals and patients as part of prevention or healthcare pathways. In this paper, we describe some of the challenges that need to be addressed, so that we can move forward and better understand when and how to use these tests for population and individual benefit.