Autonomous artificial intelligence increases real-world specialist clinic productivity in a cluster-randomized trial.

Autonomous artificial intelligence (AI) promises to increase healthcare productivity, but real-world evidence is lacking. We developed a clinic productivity model to generate testable hypotheses and study design for a preregistered cluster-randomized clinical trial, in which we tested the hypothesis that a previously validated US FDA-authorized AI for diabetic eye exams increases clinic productivity (number of completed care encounters per hour per specialist physician) among patients with diabetes. Here we report that 105 clinic days are cluster randomized to either intervention (using AI diagnosis; 51 days; 494 patients) or control (not using AI diagnosis; 54 days; 499 patients). The prespecified primary endpoint is met: AI leads to 40% higher productivity (1.59 encounters/hour, 95% confidence interval [CI]: 1.37-1.80) than control (1.14 encounters/hour, 95% CI: 1.02-1.25), p < 0.00; the secondary endpoint (productivity in all patients) is also met. Autonomous AI increases healthcare system productivity, which could potentially increase access and reduce health disparities. ClinicalTrials.gov NCT05182580.

Autonomous AI systems in the face of liability, regulations and costs.

Autonomous AI systems in medicine promise improved outcomes but raise concerns about liability, regulation, and costs. With the advent of large-language models, which can understand and generate medical text, the urgency for addressing these concerns increases as they create opportunities for more sophisticated autonomous AI systems. This perspective explores the liability implications for physicians, hospitals, and creators of AI technology, as well as the evolving regulatory landscape and payment models. Physicians may be favored in malpractice cases if they follow rigorously validated AI recommendations. However, AI developers may face liability for failing to adhere to industry-standard best practices during development and implementation. The evolving regulatory landscape, led by the FDA, seeks to ensure transparency, evaluation, and real-world monitoring of AI systems, while payment models such as MPFS, NTAP, and commercial payers adapt to accommodate them. The widespread adoption of autonomous AI systems can potentially streamline workflows and allow doctors to concentrate on the human aspects of healthcare.

Considerations for addressing bias in artificial intelligence for health equity.

Health equity is a primary goal of healthcare stakeholders: patients and their advocacy groups, clinicians, other providers and their professional societies, bioethicists, payors and value based care organizations, regulatory agencies, legislators, and creators of artificial intelligence/machine learning (AI/ML)-enabled medical devices. Lack of equitable access to diagnosis and treatment may be improved through new digital health technologies, especially AI/ML, but these may also exacerbate disparities, depending on how bias is addressed. We propose an expanded Total Product Lifecycle (TPLC) framework for healthcare AI/ML, describing the sources and impacts of undesirable bias in AI/ML systems in each phase, how these can be analyzed using appropriate metrics, and how they can be potentially mitigated. The goal of these "Considerations" is to educate stakeholders on how potential AI/ML bias may impact healthcare outcomes and how to identify and mitigate inequities; to initiate a discussion between stakeholders on these issues, in order to ensure health equity along the expanded AI/ML TPLC framework, and ultimately, better health outcomes for all.

Tools for Responding to Patient-Initiated Verbal Sexual Harassment: A Workshop for Trainees and Faculty.

Introduction: Patients are the most common source of gender-based harassment of resident physicians, yet residents receive little training on how to handle it. Few resources exist for residents wishing to address patient-initiated verbal sexual harassment themselves. Methods: We developed, taught, and evaluated a 50-minute workshop to prepare residents and faculty to respond to patient-initiated verbal sexual harassment toward themselves and others. The workshop used an interactive lecture and role-play scenarios to teach a tool kit of communication strategies for responding to harassment. Participants completed retrospective pre-post surveys on their ability to meet the learning objectives and their preparedness to respond. Results: Ninety-one participants (57 trainees, 34 faculty) completed surveys at one of five workshop sessions across multiple departments. Before the workshop, two-thirds (67%) had experienced patient-initiated sexual harassment, and only 28 out of 59 (48%) had ever addressed it. Seventy-five percent of participants had never received training on responding to patient-initiated sexual harassment. After the workshop, participants reported significant improvement in their preparedness to recognize and respond to all forms of patient-initiated verbal sexual harassment (p < .01), with the greatest improvements noted in responding to mild forms of verbal sexual harassment, such as comments on appearance or attractiveness or inappropriate jokes (p < .01). Discussion: This workshop fills a void by preparing residents and faculty to respond to verbal sexual harassment from patients that is not directly observed. Role-play and rehearsal of an individualized response script significantly improved participants' preparedness to respond to harassment toward themselves and others.

The SEE Study: Safety, Efficacy, and Equity of Implementing Autonomous Artificial Intelligence for Diagnosing Diabetic Retinopathy in Youth.

OBJECTIVE: Diabetic retinopathy (DR) is a leading cause of vision loss worldwide. Screening for DR is recommended in children and adolescents, but adherence is poor. Recently, autonomous artificial intelligence (AI) systems have been developed for early detection of DR and have been included in the American Diabetes Association's guidelines for screening in adults. We sought to determine the diagnostic efficacy of autonomous AI for the diabetic eye exam in youth with diabetes. RESEARCH DESIGN AND METHODS: In this prospective study, point-of-care diabetic eye exam was implemented using a nonmydriatic fundus camera with an autonomous AI system for detection of DR in a multidisciplinary pediatric diabetes center. Sensitivity, specificity, and diagnosability of AI was compared with consensus grading by retinal specialists, who were masked to AI output. Adherence to screening guidelines was measured before and after AI implementation. RESULTS: Three hundred ten youth with diabetes aged 5-21 years were included, of whom 4.2% had DR. Diagnosability of AI was 97.5% (302 of 310). The sensitivity and specificity of AI to detect more-than-mild DR was 85.7% (95% CI 42.1-99.6%) and 79.3% (74.3-83.8%), respectively, compared with the reference standard as defined by retina specialists. Adherence improved from 49% to 95% after AI implementation. CONCLUSIONS: Use of a nonmydriatic fundus camera with autonomous AI was safe and effective for the diabetic eye exam in youth in our study. Adherence to screening guidelines improved with AI implementation. As the prevalence of diabetes increases in youth and adherence to screening guidelines remains suboptimal, effective strategies for diabetic eye exams in this population are needed.

Cost-effectiveness of Autonomous Point-of-Care Diabetic Retinopathy Screening for Pediatric Patients With Diabetes.

Importance: Screening for diabetic retinopathy is recommended for children with type 1 diabetes (T1D) and type 2 diabetes (T2D), yet screening rates remain low. Point-of-care diabetic retinopathy screening using autonomous artificial intelligence (AI) has become available, providing immediate results in the clinic setting, but the cost-effectiveness of this strategy compared with standard examination is unknown. Objective: To assess the cost-effectiveness of detecting and treating diabetic retinopathy and its sequelae among children with T1D and T2D using AI diabetic retinopathy screening vs standard screening by an eye care professional (ECP). Design, Setting, and Participants: In this economic evaluation, parameter estimates were obtained from the literature from 1994 to 2019 and assessed from March 2019 to January 2020. Parameters included out-of-pocket cost for autonomous AI screening, ophthalmology visits, and treating diabetic retinopathy; probability of undergoing standard retinal examination; relative odds of undergoing screening; and sensitivity, specificity, and diagnosability of the ECP screening examination and autonomous AI screening. Main Outcomes and Measures: Costs or savings to the patient based on mean patient payment for diabetic retinopathy screening examination and cost-effectiveness based on costs or savings associated with the number of true-positive results identified by diabetic retinopathy screening. Results: In this study, the expected true-positive proportions for standard ophthalmologic screening by an ECP were 0.006 for T1D and 0.01 for T2D, and the expected true-positive proportions for autonomous AI were 0.03 for T1D and 0.04 for T2D. The base case scenario of 20% adherence estimated that use of autonomous AI would result in a higher mean patient payment ($8.52 for T1D and $10.85 for T2D) than conventional ECP screening ($7.91 for T1D and $8.20 for T2D). However, autonomous AI screening was the preferred strategy when at least 23% of patients adhered to diabetic retinopathy screening. Conclusions and Relevance: These results suggest that point-of-care diabetic retinopathy screening using autonomous AI systems is effective and cost saving for children with diabetes and their caregivers at recommended adherence rates.

Lessons Learned About Autonomous AI: Finding a Safe, Efficacious, and Ethical Path Through the Development Process.

Artificial intelligence (AI) describes systems capable of making decisions of high cognitive complexity; autonomous AI systems in healthcare are AI systems that make clinical decisions without human oversight. Such rigorously validated medical diagnostic AI systems hold great promise for improving access to care, increasing accuracy, and lowering cost, while enabling specialist physicians to provide the greatest value by managing and treating patients whose outcomes can be improved. Ensuring that autonomous AI provides these benefits requires evaluation of the autonomous AI's effect on patient outcome, design, validation, data usage, and accountability, from a bioethics and accountability perspective. We performed a literature review of bioethical principles for AI, and derived evaluation rules for autonomous AI, grounded in bioethical principles. The rules include patient outcome, validation, reference standard, design, data usage, and accountability for medical liability. Application of the rules explains successful US Food and Drug Administration (FDA) de novo authorization of an example, the first autonomous point-of-care diabetic retinopathy examination de novo authorized by the FDA, after a preregistered clinical trial. Physicians need to become competent in understanding the potential risks and benefits of autonomous AI, and understand its design, safety, efficacy and equity, validation, and liability, as well as how its data were obtained. The autonomous AI evaluation rules introduced here can help physicians understand limitations and risks as well as the potential benefits of autonomous AI for their patients.

Three-dimensional automated choroidal volume assessment on standard spectral-domain optical coherence tomography and correlation with the level of diabetic macular edema.

PURPOSE: To measure choroidal thickness on spectral-domain optical coherence tomography (SD OCT) images using automated algorithms and to correlate choroidal pathology with retinal changes attributable to diabetic macular edema (DME). DESIGN: Post hoc analysis of multicenter clinical trial baseline data. METHODS: SD OCT raster scans/fluorescein angiograms were obtained from 284 treatment-naïve eyes of 142 patients with clinically significant DME and from 20 controls. Three-dimensional (3D) SD OCT images were evaluated by a certified independent reading center analyzing retinal changes associated with diabetic retinopathy. Choroidal thicknesses were analyzed using a fully automated algorithm. Angiograms were assessed manually. Multiple endpoint correction according to Bonferroni-Holm was applied. Main outcome measures were average retinal/choroidal thickness on fovea-centered or peak of edema (thickest point of edema)-centered Early Treatment Diabetic Retinopathy Study grid, maximum area of leakage, and the correlation between retinal and choroidal thicknesses. RESULTS: Total choroidal thickness is significantly reduced in DME (175 ± 23 µm; P = .0016) and nonedematous fellow eyes (177 ± 20 µm; P = .009) of patients compared with healthy control eyes (190 ± 23 µm). Retinal/choroidal thickness values showed no significant correlation (1-mm: P = .27, r(2) = 0.01; 3-mm: P = .96, r(2) < 0.0001; 6-mm: P = .42, r(2) = 0.006). No significant difference was found in the 1- or 3-mm circle of a retinal peak of edema-centered grid. All other measurements of choroidal/retinal thickness (DME vs healthy, DME vs peak of edema-centered, DME vs fellow, healthy vs fellow, peak of edema-centered vs healthy, peak of edema-centered vs fellow eyes) were compared but no statistically significant correlation was found. By tendency a thinner choroid correlates with larger retinal leakage areas. CONCLUSIONS: Automated algorithms can be used to reliably assess choroidal thickness in eyes with DME. Choroidal thickness was generally reduced in patients with diabetes if DME is present in 1 eye; however, no correlation was found between choroidal/retinal pathologies, suggesting different pathogenetic pathways.