Developer Perspectives on Potential Harms of Machine Learning Predictive Analytics in Health Care: Qualitative Analysis.

BACKGROUND: Machine learning predictive analytics (MLPA) is increasingly used in health care to reduce costs and improve efficacy; it also has the potential to harm patients and trust in health care. Academic and regulatory leaders have proposed a variety of principles and guidelines to address the challenges of evaluating the safety of machine learning-based software in the health care context, but accepted practices do not yet exist. However, there appears to be a shift toward process-based regulatory paradigms that rely heavily on self-regulation. At the same time, little research has examined the perspectives about the harms of MLPA developers themselves, whose role will be essential in overcoming the "principles-to-practice" gap. OBJECTIVE: The objective of this study was to understand how MLPA developers of health care products perceived the potential harms of those products and their responses to recognized harms. METHODS: We interviewed 40 individuals who were developing MLPA tools for health care at 15 US-based organizations, including data scientists, software engineers, and those with mid- and high-level management roles. These 15 organizations were selected to represent a range of organizational types and sizes from the 106 that we previously identified. We asked developers about their perspectives on the potential harms of their work, factors that influence these harms, and their role in mitigation. We used standard qualitative analysis of transcribed interviews to identify themes in the data. RESULTS: We found that MLPA developers recognized a range of potential harms of MLPA to individuals, social groups, and the health care system, such as issues of privacy, bias, and system disruption. They also identified drivers of these harms related to the characteristics of machine learning and specific to the health care and commercial contexts in which the products are developed. MLPA developers also described strategies to respond to these drivers and potentially mitigate the harms. Opportunities included balancing algorithm performance goals with potential harms, emphasizing iterative integration of health care expertise, and fostering shared company values. However, their recognition of their own responsibility to address potential harms varied widely. CONCLUSIONS: Even though MLPA developers recognized that their products can harm patients, public, and even health systems, robust procedures to assess the potential for harms and the need for mitigation do not exist. Our findings suggest that, to the extent that new oversight paradigms rely on self-regulation, they will face serious challenges if harms are driven by features that developers consider inescapable in health care and business environments. Furthermore, effective self-regulation will require MLPA developers to accept responsibility for safety and efficacy and know how to act accordingly. Our results suggest that, at the very least, substantial education will be necessary to fill the "principles-to-practice" gap.

A Typology of Existing Machine Learning-Based Predictive Analytic Tools Focused on Reducing Costs and Improving Quality in Health Care: Systematic Search and Content Analysis.

BACKGROUND: Considerable effort has been devoted to the development of artificial intelligence, including machine learning-based predictive analytics (MLPA) for use in health care settings. The growth of MLPA could be fueled by payment reforms that hold health care organizations responsible for providing high-quality, cost-effective care. Policy analysts, ethicists, and computer scientists have identified unique ethical and regulatory challenges from the use of MLPA in health care. However, little is known about the types of MLPA health care products available on the market today or their stated goals. OBJECTIVE: This study aims to better characterize available MLPA health care products, identifying and characterizing claims about products recently or currently in use in US health care settings that are marketed as tools to improve health care efficiency by improving quality of care while reducing costs. METHODS: We conducted systematic database searches of relevant business news and academic research to identify MLPA products for health care efficiency meeting our inclusion and exclusion criteria. We used content analysis to generate MLPA product categories and characterize the organizations marketing the products. RESULTS: We identified 106 products and characterized them based on publicly available information in terms of the types of predictions made and the size, type, and clinical training of the leadership of the companies marketing them. We identified 5 categories of predictions made by MLPA products based on publicly available product marketing materials: disease onset and progression, treatment, cost and utilization, admissions and readmissions, and decompensation and adverse events. CONCLUSIONS: Our findings provide a foundational reference to inform the analysis of specific ethical and regulatory challenges arising from the use of MLPA to improve health care efficiency.

A case report of sinonasal glomangiopericytoma: An important reminder to always collect specimen.

OBJECTIVES: To present a case report of sinonasal glomangiopericytoma (GPC) in a female patient in her thirties and to highlight the importance of collecting pathology specimens even in routine sinus surgery cases. METHODS: A case report detailing the diagnosis of GPC in a female in her thirties, including her initial presentation, treatment, and follow-up, along with a brief review of the literature. RESULTS: Pathology of the collected specimen revealed sinonasal GPC along with chronic rhinosinusitis. Immunohistochemistry was positive for SMA, beta-catenin, and cyclin D1; and negative for STAT6, ERG, pankeratin, SOX10, and S100. CONCLUSION: This diagnosis expands the knowledge around the demographic profile of GPC patients. GPC should be included in the differential diagnosis of sinonasal masses, even in younger patients. The case highlights the importance of collecting the entire pathology specimen in all cases, even of ones that seem routine and benign.

Moral Engagement and Disengagement in Health Care AI Development.

BACKGROUND: Machine learning (ML) is utilized increasingly in health care, and can pose harms to patients, clinicians, health systems, and the public. In response, regulators have proposed an approach that would shift more responsibility to ML developers for mitigating potential harms. To be effective, this approach requires ML developers to recognize, accept, and act on responsibility for mitigating harms. However, little is known regarding the perspectives of developers themselves regarding their obligations to mitigate harms. METHODS: We conducted 40 semi-structured interviews with developers of ML predictive analytics applications for health care in the United States. RESULTS: Participants varied widely in their perspectives on personal responsibility and included examples of both moral engagement and disengagement, albeit in a variety of forms. While most (70%) of participants made a statement indicative of moral engagement, most of these statements reflected an awareness of moral issues, while only a subset of these included additional elements of engagement such as recognizing responsibility, alignment with personal values, addressing conflicts of interests, and opportunities for action. Further, we identified eight distinct categories of moral disengagement reflecting efforts to minimize potential harms or deflect personal responsibility for preventing or mitigating harms. CONCLUSIONS: These findings suggest possible facilitators and barriers to the development of ethical ML that could act by encouraging moral engagement or discouraging moral disengagement. Regulatory approaches that depend on the ability of ML developers to recognize, accept, and act on responsibility for mitigating harms might have limited success without education and guidance for ML developers about the extent of their responsibilities and how to implement them.

The clinical utility of three frailty measures in identifying HIV-associated neurocognitive disorders.

OBJECTIVE: Frailty measures vary widely and the optimal measure for predicting HIV-associated neurocognitive disorders (HAND) is unclear. DESIGN: A study was conducted to examine the clinical utility of three widely used frailty measures in identifying HIV-associated neurocognitive disorders. METHODS: The study involved 284 people with HIV (PWH) at least 50 years enrolled at UC San Diego's HIV Neurobehavioral Research Program. Frailty measurements included the Fried Phenotype, the Rockwood Frailty Index, and the Veterans Aging Cohort Study (VACS) Index. HAND was diagnosed according to Frascati criteria. ANOVAs examined differences in frailty severity across HAND conditions. ROC analyses evaluated sensitivity and specificity of each measure to detect symptomatic HAND [mild neurocognitive disorder (MND) and HIV-associated dementia (HAD)] from no HAND. RESULTS: Across all frailty measures, frailty was found to be higher in HAD compared with no HAND. For Fried and Rockwood (not VACS), frailty was significantly more severe in MND vs. no HAND and in HAD vs. ANI (asymptomatic neurocognitive impairment). For discriminating symptomatic HAND from no HAND, Fried was 37% sensitive and 92% specific, Rockwood was 85% sensitive and 43% specific, and VACS was 58% sensitive and 65% specific. CONCLUSION: These findings demonstrate that Fried and Rockwood outperform VACS in predicting HAND. However, ROC analyses suggest none of the indices had adequate predictive validity in detecting HAND. The results indicate that the combined use of the Rockwood and Fried indices may be an appropriate alternative.

Ethical Considerations in the Design and Conduct of Clinical Trials of Artificial Intelligence.

Importance: Safe integration of artificial intelligence (AI) into clinical settings often requires randomized clinical trials (RCT) to compare AI efficacy with conventional care. Diabetic retinopathy (DR) screening is at the forefront of clinical AI applications, marked by the first US Food and Drug Administration (FDA) De Novo authorization for an autonomous AI for such use. Objective: To determine the generalizability of the 7 ethical research principles for clinical trials endorsed by the National Institute of Health (NIH), and identify ethical concerns unique to clinical trials of AI. Design, Setting, and Participants: This qualitative study included semistructured interviews conducted with 11 investigators engaged in the design and implementation of clinical trials of AI for DR screening from November 11, 2022, to February 20, 2023. The study was a collaboration with the ACCESS (AI for Children's Diabetic Eye Exams) trial, the first clinical trial of autonomous AI in pediatrics. Participant recruitment initially utilized purposeful sampling, and later expanded with snowball sampling. Study methodology for analysis combined a deductive approach to explore investigators' perspectives of the 7 ethical principles for clinical research endorsed by the NIH and an inductive approach to uncover the broader ethical considerations implementing clinical trials of AI within care delivery. Results: A total of 11 participants (mean [SD] age, 47.5 [12.0] years; 7 male [64%], 4 female [36%]; 3 Asian [27%], 8 White [73%]) were included, with diverse expertise in ethics, ophthalmology, translational medicine, biostatistics, and AI development. Key themes revealed several ethical challenges unique to clinical trials of AI. These themes included difficulties in measuring social value, establishing scientific validity, ensuring fair participant selection, evaluating risk-benefit ratios across various patient subgroups, and addressing the complexities inherent in the data use terms of informed consent. Conclusions and Relevance: This qualitative study identified practical ethical challenges that investigators need to consider and negotiate when conducting AI clinical trials, exemplified by the DR screening use-case. These considerations call for further guidance on where to focus empirical and normative ethical efforts to best support conduct clinical trials of AI and minimize unintended harm to trial participants.

Validation of a 3D-Printed Silicone-Based Laryngeal Model for Resident Education.

OBJECTIVE: We sought to validate a laryngeal simulation model and subsequently demonstrate its efficacy in improving surgical technique. STUDY DESIGN: Pre-post interventional study. SETTING: Otolaryngology Program at a Tertiary Care Center. METHODS: A low-cost, high-fidelity laryngeal model was created using a 3-dimensional-printed cast and multilayered silicone to mimic vocal fold lesions. Participants (attendings and trainees) were first given a series of tasks including mucosal vocal fold lesion resection and microflap excision of a submucosal lesion. Trainees were then provided with an instructional video from a laryngologist and asked to repeat the same tasks on the model. Performance data was then assessed using validated surveys and blinded expert reviewers. RESULTS: Eighteen participants completed the simulation. All subjects agreed that the "simulation experience was useful" and 93% agreed "the simulator helped improve my ability to do microsurgical tasks." In the postinstruction self-evaluation, trainees reported a significant decrease in mental demand (95% confidence interval [CI]: 0.37-0.91; P = .038) and significant increase in subjective performance (95% CI: 1.51-51.89; P = .016) compared to the preinstruction self-evaluation. On the postinstruction attempt, there was a significant improvement in all domains of the adapted objective structured assessment of technical skills as measured by 3 blinded, expert reviewers. DISCUSSION: This study demonstrates the usefulness of a silicone larynx model and the value of instructional video in developing laryngeal microsurgical skills. Participants positively reviewed the laryngeal model and trainees saw both a subjective and objective improvement indicating tangible operative benefits from the use of this laryngeal simulation.

Not in my AI: Moral engagement and disengagement in health care AI development.

Machine learning predictive analytics (MLPA) are utilized increasingly in health care, but can pose harms to patients, clinicians, health systems, and the public. The dynamic nature of this technology creates unique challenges to evaluating safety and efficacy and minimizing harms. In response, regulators have proposed an approach that would shift more responsibility to MLPA developers for mitigating potential harms. To be effective, this approach requires MLPA developers to recognize, accept, and act on responsibility for mitigating harms. In interviews of 40 MLPA developers of health care applications in the United States, we found that a subset of ML developers made statements reflecting moral disengagement, representing several different potential rationales that could create distance between personal accountability and harms. However, we also found a different subset of ML developers who expressed recognition of their role in creating potential hazards, the moral weight of their design decisions, and a sense of responsibility for mitigating harms. We also found evidence of moral conflict and uncertainty about responsibility for averting harms as an individual developer working in a company. These findings suggest possible facilitators and barriers to the development of ethical ML that could act through encouragement of moral engagement or discouragement of moral disengagement. Regulatory approaches that depend on the ability of ML developers to recognize, accept, and act on responsibility for mitigating harms might have limited success without education and guidance for ML developers about the extent of their responsibilities and how to implement them.

Rapid Review of COVID-19 Vaccination Access and Acceptance for Global Refugee, Asylum Seeker and Undocumented Migrant Populations.

Objectives: Refugees, asylum seekers, and undocumented migrants globally have been disproportionally impacted by COVID-19. Vaccination has been a major tool to reduce disease impact, yet concerns exist regarding equitable allocation and uptake. Methods: A rapid literature review was conducted based on PRISMA guidelines to determine COVID-19 vaccination acceptance rates and level of access for these population groups globally. Results: Relatively high COVID-19 vaccine acceptance levels were commonly reported in these populations, although, trust in host governments was a frequently expressed concern, especially for undocumented migrants. Outreach efforts and access to comprehensive information from a trusted source and in appropriate language were found to be major determinants of COVID-19 vaccine acceptance. COVID-19 vaccination access and policies varied considerably across host countries despite urgings by international organizations to include migrants and refugees. While most governments endorsed inclusive policies, evidence of successful program implementation was frequently lacking, creating difficulty to better tailor and implement COVID-19 outreach programs. Conclusion: This review identifies impactful improvements to be implemented to ensure equitable COVID-19 vaccinations and to reduce disease burden on refugees, asylum seekers, and undocumented migrants.

Ethics of emerging infectious disease outbreak responses: Using Ebola virus disease as a case study of limited resource allocation.

Emerging infectious diseases such as Ebola Virus Disease (EVD), Nipah Virus Encephalitis and Lassa fever pose significant epidemic threats. Responses to emerging infectious disease outbreaks frequently occur in resource-constrained regions and under high pressure to quickly contain the outbreak prior to potential spread. As seen in the 2020 EVD outbreaks in the Democratic Republic of Congo and the current COVID-19 pandemic, there is a continued need to evaluate and address the ethical challenges that arise in the high stakes environment of an emerging infectious disease outbreak response. The research presented here provides analysis of the ethical challenges with regard to allocation of limited resources, particularly experimental therapeutics, using the 2013-2016 EVD outbreak in West Africa as a case study. In-depth semi-structured interviews were conducted with senior healthcare personnel (n = 16) from international humanitarian aid organizations intimately engaged in the 2013-2016 EVD outbreak response in West Africa. Interviews were recorded in private setting, transcribed, and iteratively coded using grounded theory methodology. A majority of respondents indicated a clear propensity to adopt an ethical framework of guiding principles for international responses to emerging infectious disease outbreaks. Respondents agreed that prioritization of frontline workers' access to experimental therapeutics was warranted based on a principle of reciprocity. There was widespread acceptance of adaptive trial designs and greater trial transparency in providing access to experimental therapeutics. Many respondents also emphasized the importance of community engagement in limited resource allocation scheme design and culturally appropriate informed consent procedures. The study results inform a potential ethical framework of guiding principles based on the interview participants' insights to be adopted by international response organizations and their healthcare workers in the face of allocating limited resources such as experimental therapeutics in future emerging infectious disease outbreaks to ease the moral burden of individual healthcare providers.

Ethics in Research: Relevance for Nephrology.

Research is crucial to progress in nephrology. It is important that studies are conducted rigorously from the scientific perspective, as well as in adherence to ethical standards. Traditional clinical research places a high value on individual research subject autonomy. Research questions often include the clinical effectiveness of new interventions studied under highly controlled conditions. Such research has brought the promise of new game-changers in nephrology, such as the sodium-glucose cotransporter 2 inhibitors. Implementation research takes such knowledge further and investigates how to translate it into broader-scale policy and practice, to achieve swift and global uptake, with a focus on justice and equity. Newer challenges arising globally in research ethics include those relating to oversight of innovation, biobanking and big data, human-challenge studies, and research during emergencies. This article details the history of clinical research ethics and the role of research ethics committees, describes the evolving spectrum of biomedical research in human medicine, and presents emerging clinical research ethics issues using illustrative examples and a hypothetical case study. It is imperative that researchers and research ethics committees are well versed in the ethical principles of all forms of human research such that research is conducted to the highest standards and that effective interventions can be implemented at scale as rapidly as possible.

Diverse experts' perspectives on ethical issues of using machine learning to predict HIV/AIDS risk in sub-Saharan Africa: a modified Delphi study.

OBJECTIVE: To better understand diverse experts' views about the ethical implications of ongoing research funded by the National Institutes of Health that uses machine learning to predict HIV/AIDS risk in sub-Saharan Africa (SSA) based on publicly available Demographic and Health Surveys data. DESIGN: Three rounds of semi-structured surveys in an online expert panel using a modified Delphi approach. PARTICIPANTS: Experts in informatics, African public health and HIV/AIDS and bioethics were invited to participate. MEASURES: Perceived importance of or agreement about relevance of ethical issues on 5-point unipolar Likert scales. Qualitative data analysis identified emergent themes related to ethical issues and development of an ethical framework and recommendations for open-ended questions. RESULTS: Of the 35 invited experts, 22 participated in the online expert panel (63%). Emergent themes were the inclusion of African researchers in all aspects of study design, analysis and dissemination to identify and address local contextual issues, as well as engagement of communities. Experts focused on engagement with health and science professionals to address risks, benefits and communication of findings. Respondents prioritised the mitigation of stigma to research participants but recognised trade-offs between privacy and the need to disseminate findings to realise public health benefits. Strategies for responsible communication of results were suggested, including careful word choice in presentation of results and limited dissemination to need-to-know stakeholders such as public health planners. CONCLUSION: Experts identified ethical issues specific to the African context and to research on sensitive, publicly available data and strategies for addressing these issues. These findings can be used to inform an ethical implementation framework with research stage-specific recommendations on how to use publicly available data for machine learning-based predictive analytics to predict HIV/AIDS risk in SSA.