The influence of candidates' race on examiners' ratings in standardised assessments of clinical practice.

PURPOSE: Delivering fair and reliable summative assessments in medical education assumes examiner decision making is devoid of bias. We investigated whether candidate racial appearances influenced examiner ratings in undergraduate clinical exams. METHODS: We used an internet-based design. Examiners watched a randomised set of six videos of three different white candidates and three different non-white (Asian, black and Chinese) candidates taking a clinical history at either fail, borderline or pass grades. We compared the median and interquartile range (IQR) of the paired difference between scores for the white and non-white candidates at each performance grade and tested for statistical significance. RESULTS: 160 Examiners participated. At the fail grade, the black and Chinese candidates scored lower than the white candidate, with median paired differences of -2.5 and -1 respectively (both p < 0.001). At the borderline grade, the black and Chinese candidates scored higher than the white candidate, with median paired differences of +2 and +3, respectively (both p < 0.001). At the passing grade, the Asian candidate scored lower than the white candidate (median paired difference -1, p < 0.001). CONCLUSION: The racial appearance of candidates appeared to influence the scores awarded by examiners, but not in a uniform manner.

Machine learning in sudden cardiac death risk prediction: a systematic review.

AIMS: Most patients who receive implantable cardioverter defibrillators (ICDs) for primary prevention do not receive therapy during the lifespan of the ICD, whilst up to 50% of sudden cardiac death (SCD) occur in individuals who are considered low risk by conventional criteria. Machine learning offers a novel approach to risk stratification for ICD assignment. METHODS AND RESULTS: Systematic search was performed in MEDLINE, Embase, Emcare, CINAHL, Cochrane Library, OpenGrey, MedrXiv, arXiv, Scopus, and Web of Science. Studies modelling SCD risk prediction within days to years using machine learning were eligible for inclusion. Transparency and quality of reporting (TRIPOD) and risk of bias (PROBAST) were assessed. A total of 4356 studies were screened with 11 meeting the inclusion criteria with heterogeneous populations, methods, and outcome measures preventing meta-analysis. The study size ranged from 122 to 124 097 participants. Input data sources included demographic, clinical, electrocardiogram, electrophysiological, imaging, and genetic data ranging from 4 to 72 variables per model. The most common outcome metric reported was the area under the receiver operator characteristic (n = 7) ranging between 0.71 and 0.96. In six studies comparing machine learning models and regression, machine learning improved performance in five. No studies adhered to a reporting standard. Five of the papers were at high risk of bias. CONCLUSION: Machine learning for SCD prediction has been under-applied and incorrectly implemented but is ripe for future investigation. It may have some incremental utility in predicting SCD over traditional models. The development of reporting standards for machine learning is required to improve the quality of evidence reporting in the field.

Feasibility of an artificial intelligence phone call for postoperative care following cataract surgery in a diverse population: two phase prospective study protocol.

INTRODUCTION: Artificial intelligence (AI) development has led to improvements in many areas of medicine. Canada has workforce pressures in delivering cataract care. A potential solution is using AI technology that can automate care delivery, increase effectiveness and decrease burdens placed on patients and the healthcare system. This study assesses the use of 'Dora', an example of an AI assistant that is able to deliver a regulated autonomous, voice-based, natural-language consultation with patients over the telephone. Dora is used in routine practice in the UK, but this study seeks to assess the safety, usability, acceptability and cost-effectiveness of using the technology in Canada. METHODS AND ANALYSIS: This is a two-phase prospective single-centred trial. An expected 250 patients will be recruited for each phase of the study. For Phase I of the study, Dora will phone patients at postoperative week 1 and for Phase II of the study, Dora will phone patients within 24hours of their cataract surgery and again at postoperative week 1. We will evaluate the agreement between Dora and a supervising clinician regarding the need for further review based on the patients' symptoms. A random sample of patients will undergo the System Usability Scale followed by an extended semi-structured interview. The primary outcome of agreement between Dora and the supervisor will be assessed using the kappa statistic. Qualitative data from the interviews will further gauge patient opinions about Dora's usability, appropriateness and level of satisfaction. ETHICS AND DISSEMINATION: Research Ethics Board William Osler Health System (ID: 22-0044) has approved this study and will be conducted by guidelines of Declaration of Helsinki. Master-linking sheet will contain the patient chart identification (ID), full name, date of birth and study ID. Results will be shared through peer-reviewed journals and presentations at conferences.

Investigating the Impact of Automation on the Health Care Workforce Through Autonomous Telemedicine in the Cataract Pathway: Protocol for a Multicenter Study.

BACKGROUND: While digital health innovations are increasingly being adopted by health care organizations, implementation is often carried out without considering the impacts on frontline staff who will be using the technology and who will be affected by its introduction. The enthusiasm surrounding the use of artificial intelligence (AI)-enabled digital solutions in health care is tempered by uncertainty around how it will change the working lives and practices of health care professionals. Digital enablement can be viewed as facilitating enhanced effectiveness and efficiency by improving services and automating cognitive labor, yet the implementation of such AI technology comes with challenges related to changes in work practices brought by automation. This research explores staff experiences before and after care pathway automation with an autonomous clinical conversational assistant, Dora (Ufonia Ltd), that is able to automate routine clinical conversations. OBJECTIVE: The primary objective is to examine the impact of AI-enabled automation on clinicians, allied health professionals, and administrators who provide or facilitate health care to patients in high-volume, low-complexity care pathways. In the process of transforming care pathways through automation of routine tasks, staff will increasingly "work at the top of their license." The impact of this fundamental change on the professional identity, well-being, and work practices of the individual is poorly understood at present. METHODS: We will adopt a multiple case study approach, combining qualitative and quantitative data collection methods, over 2 distinct phases, namely phase A (preimplementation) and phase B (postimplementation). RESULTS: The analysis is expected to reveal the interrelationship between Dora and those affected by its introduction. This will reveal how tasks and responsibilities have changed or shifted, current tensions and contradictions, ways of working, and challenges, benefits, and opportunities as perceived by those on the frontlines of the health care system. The findings will enable a better understanding of the resistance or susceptibility of different stakeholders within the health care workforce and encourage managerial awareness of differing needs, demands, and uncertainties. CONCLUSIONS: The implementation of AI in the health care sector, as well as the body of research on this topic, remain in their infancy. The project's key contribution will be to understand the impact of AI-enabled automation on the health care workforce and their work practices. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): PRR1-10.2196/49374.

User-acceptability of an automated telephone call for post-operative follow-up after uncomplicated cataract surgery.

BACKGROUND: Innovative technology is recommended to address the current capacity challenges facing the NHS. This study evaluates the patient acceptability of automated telephone follow-up after routine cataract surgery using Dora (Ufonia Limited, Oxford, United Kingdom), which to our knowledge is the first AI-powered clinical assistant to be used in the NHS. Dora has a natural-language, phone conversation with patients about their symptoms after cataract surgery. METHODS: This is a prospective mixed-methods cohort study that was conducted at Buckinghamshire Healthcare NHS Foundation Trust. All patients who were followed up using Dora were asked to give a Net Promoter Score (NPS), and 24 patients were randomly selected to complete the validated Telephone Usability Questionnaire (TUQ) as well as extended semi-structured interviews that underwent thematic analysis. RESULTS: A total of 170 autonomous calls were completed. The median NPS score was 9 out of 10. The TUQ (scored out of 5) showed high rates of acceptability, with an overall mean score of 4.0. Simplicity, time saving, and ease of use scored the highest with a median of 5, whilst 'speaking to Dora feels the same as speaking to a clinician' scored a median of 3. The main themes extracted from the qualitative data were 'I can see why you're doing it', 'It went quite well actually', 'I just trust human beings I suppose'. CONCLUSION: We found high levels of patient acceptability when using Dora across three acceptability measures. Dora provides a potential solution to reduce pressure on hospital capacity whilst also providing a convenient service for patients.